GPT3微调分步指南

　　从2022年底开始，OpenAI推出ChatGPT，被我们很多人认为是AI的iPhone时刻。然而，OpenAI的聊天机器人并不是第一个生成式人工智能文本机器学习模型，它是在两年前推出的GPT-3之后推出的。
　　OpenAI为我们提供了一个现成的GPT-3训练模型。此外，可以在较小的数据集上对特定任务进行微调。例如，假设您要创建特定于您公司的电子邮件响应生成器。首先，您必须收集大量关于您的特定业务领域的数据，如客户电子邮件查询和响应。然后，您可以使用此数据来微调GPT-3，以了解公司的特定语言模式和短语。通过微调GPT-3，可以创建高度自定义和专业化的电子邮件响应生成器，专门针对特定业务领域中使用的语言模式和单词进行定制。
　　在这篇博文中，我将向您展示如何微调GPT-3。我们将使用python代码来执行此操作，而不需要事先了解GPT-3。前提预备
　　不像在HuggingFace（写这篇博客的时候）上提供的GPT-2模型，我们不能直接访问GPT-3模型。因此，首先需要从OpenAI获取API密钥并安装Python包 奥佩奈 ，可通过以下方式快速完成 尖点。
　　对于OpenAI的API密钥：请访问 https：//platform.openai.com/account/api-keys ，创建帐户，单击＂创建新密钥＂ 以及复制一把钥匙。
　　密钥是一个以＂sk-＂开头的长字符串。一定要保密！获得密钥后，访问密钥的简单方法是在终端中执行以下操作：（我个人，为了简单起见，我把它放在我的 .bachrc）：export OPENAI_API_KEY=sk-t59pgejhtrff5(...)
　　使用GPT-3型号是有成本的。我们需要学分。在撰写本文时，当您创建一个新帐户时，您可以获得免费的积分来试用该工具。我不知道这会不会继续下去...
　　现在我们有了key和Python包，是时候考虑需要微调的数据了。首先，我们需要一个用于微调的示例文件，其中每个示例都是一个 提示符 ，后跟适当的 完成。描述生成工具
　　我们将为这个演示构建一个工具来创建虚构的超级英雄的描述。最后，这个工具会接收到超级英雄的年龄、性别和力量，它会自动生成我们的超级英雄的描述。
　　在下面的例子中，在微调模型之后，我们只需要说＂40，woman，Healing ->＂，然后我们将自动从模型中获得一个描述。
　　这就是一切！！创建用于微调的数据集
　　在某些情况下，您可能有一个要用于微调的数据集。但是，由于我没有一个，让我们看看如何直接从GPT-3创建一个包含超级英雄描述的合成数据集。下面的代码将给予我一个CSV文件，其中包含 提示 和相应的 完成示例。import os import openai import pandas as pd  openai.api_key = os.getenv(＂OPENAI_API_KEY＂)  l_age = [＂18＂, ＂20＂, ＂30＂, ＂40＂, ＂50＂, ＂60＂, ＂90＂] l_gender = [＂man＂, ＂woman＂] l_power = [＂invisibility＂, ＂read in the thoughts＂, ＂turning lead into gold＂, ＂immortality＂, ＂telepathy＂, ＂teleport＂, ＂flight＂]   f_prompt = ＂Imagine a complete and detailed description of a {age}-year-old {gender} fictional character who has the superpower of {power}. Write out the entire description in a maximum of 100 words in great detail:＂ f_sub_prompt = ＂{age}, {gender}, {power}＂  df = pd.DataFrame() for age in l_age:  for gender in l_gender:   for power in l_power:    for i in range(3): ## 3 times each     prompt = f_prompt.format(age=age, gender=gender, power=power)     sub_prompt = f_sub_prompt.format(age=age, gender=gender, power=power)     print(sub_prompt)      response = openai.Completion.create(      model=＂text-davinci-003＂,      prompt=prompt,      temperature=1,      max_tokens=500,      top_p=1,      frequency_penalty=0,      presence_penalty=0     )          finish_reason = response[＂choices＂][0][＂finish_reason＂]     response_txt = response[＂choices＂][0][＂text＂]          new_row = {       ＂age＂:age,        ＂gender＂:gender,        ＂power＂:power,        ＂prompt＂:prompt,        ＂sub_prompt＂:sub_prompt,        ＂response_txt＂:response_txt,        ＂finish_reason＂:finish_reason}     new_row = pd.DataFrame([new_row])     df = pd.concat([df, new_row], axis=0, ignore_index=True)  df.to_csv(＂out_openai_completion.csv＂)
　　让我们看看这段代码是如何工作的。
　　变量 f_prompt 包含以下句子，其中缺少 {age}、 {gender} 和 {power} 。
　　想象一个{age}岁的{gender}虚构人物拥有{power}超能力的完整而详细的描述。用最多100字的篇幅详细写出整个描述：
　　在代码的前三个 for 循环中，我们迭代{age}、{gender}和{power}的不同值。在循环的每一步，我们用不同的值替换3个缺失的变量。然后使用 openai.Completion.create 函数要求GPT生成对提示的响应。
　　此函数最重要的参数包括model ：用于生成响应的模型。OpenAI提供了四种标准的GPT-3模型（ada， babbage， curie，或 davinci），它们的大小...和使用价格各不相同。这是 达芬奇 -最大的模型。瞬发：要使用GPT-3完成的提示。温度：温度是一个介于0和1之间的数字，控制输出的随机性。我们将温度设置为最大值，以使模型在创建响应时尽可能具有创造性。最大令牌数：定义响应的最大长度。
　　在此脚本的结尾，我们有一个Pandas表存储在文件 out_openai_complement. csv中。& nbsp; 此表中我们感兴趣的两个主要列是 sub_prompt 和 response_txt。该 子提示符 例如 ‘18岁，伙计，隐形人。 它包含被替换的三个值，以逗号分隔。该 响应_文本 包含GPT模型的输出。微调GPT模型
　　下面的代码检索以前创建的文件 out_openai_completion.csv ，并使用 openai 微调GPT-3模型。import pandas as pd import openai import subprocess  df = pd.read_csv(＂out_openai_completion.csv＂)  prepared_data = df.loc[:,[＂sub_prompt＂,＂response_txt＂]] prepared_data.rename(columns={＂sub_prompt＂:＂prompt＂, ＂response_txt＂:＂completion＂}, inplace=True) prepared_data.to_csv(＂prepared_data.csv＂,index=False)   ## prepared_data.csv --> prepared_data_prepared.json subprocess.run(＂openai tools fine_tunes.prepare_data --file prepared_data.csv --quiet＂.split())  ## Start fine-tuning subprocess.run(＂openai api fine_tunes.create --training_file prepared_data_prepared.jsonl --model davinci --suffix ＂SuperHero＂＂.split())
　　让我们花点时间来理解这段代码吧！
　　一、档案的内容 输出_打开_完成. csv 被载入数据帧 df。作为提醒，在我们的任务中，如果用户输入＂40，女性，治疗＂，我们希望有一个具有治疗能力的40岁女性角色的描述。要执行微调，必须向GPT提供用户可能键入的内容和相应的期望响应的示例。在数据框中 df ，列 子提示符 以及 响应_文本 包含具有相应期望响应的输入示例。在上面的代码中，我们首先提取这两个冒号，然后将它们重命名为 瞬发 以及 完成分别为。生成的数据框存储在新文件中 准备好的数据. csv 仅包含这两列。
　　此文件 prepared_data.csv 如下所示：prompt,completion ＂18, man, invisibility＂,＂He looks to be in his late teens, with dark eyes and unruly black hair which reach his shoulders. His build is athletic and strong, but not muscular. He often wears clothing that makes him blend in with his surroundings, in order to stay as inconspicuous as possible.He has the ability of invisibility, which he can use to become an invisible observer or to make himself or objects around him disappear from view. He has honed his skills to make himself undetectable, able to even make sounds vanish, allowing him to go unnoticed. His invisibility power is his greatest asset, and he uses it to protect those he cares about.＂ ＂18, man, invisibility＂,＂18 year-old Jackson is a powerful young man with the incredible power of invisibility. He stands at 6＂2 and has a strong athletic frame with wavy brown hair, startling blue eyes and a warm and gentle smile. His strength and agility make him especially good at using his ability. He can become invisible in an instant, moving through walls and other obstacles without being detected. What＂s more, he can move objects with his mind and use his power to protect those he loves. His power is a blessing and a curse, as it can be abused to do harm. Jackson is a brave and noble person who uses his power to do good and make the world a better place.＂ ＂18, man, invisibility＂,＂Brandon is an 18-year-old of average build, standing roughly 5＂10 with an inquisitive look. He has naturally wavy chestnut brown hair and bright blue eyes. His demeanor is usually quite serious, but he also has an easy and gentle smile. He has a natural gift of invisibility, which he uses to protect himself and his family from harm. He＂s an inquisitive and creative young man who uses his gift to explore the world, observe people, and uncover the truth. His consistent practice of mindfulness helps him remain unseen, even when his emotions are visible. His intelligence and persistent passion for truth drives him to help those in need.＂ ＂18, man, read in the thoughts＂,＂This 18-year-old man has a muscular stature and long, ash blonde hair. His bright blue eyes are full of knowledge and power, hinting at the strange ability he has - he can read minds. Now and then, strange sparks of electricity streak from his eyes when he concentrates hard enough. He is often calm and collected, but when provoked has the ability to blend his learning of your thoughts with his immense strength. His wit, intelligence and empathy are weapons that are often underestimated - yet, very lethal.＂ ＂18, man, read in the thoughts＂,＂He＂s an 18-year-old man with short, cropped dark hair and bright green eyes. His usual outfit consists of an olive green hoodie, blue jeans and purple trainers. He＂s relaxed, confident and always reads the atmosphere well. One of his great skills is read in the thoughts of others. By doing so, he is able to understand people more easily and relate to their concerns out of the blue. He always uses his superpower with the utmost dignity and respect, never using it to manipulate. He uses it to seek the truth, identify people＂s emotions and gain insight into their perspectives. He＂s the sort of person that can make anyone feel comfortable and is never afraid of a challenge. His superpower is merely an extension of his already charismatic disposition.＂ (...)
　　Python的 subprocess.run（） 函数将命令作为子进程运行。它经常被用来执行外部程序，就好像它们在终端中运行一样。
　　我们使用 subprocess.run （） 来执行＂openai工具微调.准备数据＂。此函数接受输入文件 prepared_data.csv，检查数据是否正确，并生成一个名为 prepared_data_prepared.jsonl的JSONL文件。JSONL文件是一种将每个JSON对象存储在单独行中的格式。JSONL文件包含一系列JSON对象，每个对象由换行符分隔。
　　请注意，我们添加了选项＂- - quiet＂来自动接受＂openai tools fine_tunes.prepare_data＂提供的所有建议。例如 ，它建议在所有提示符的末尾添加一个＂-->＂，并在每个响应的末尾添加一个标记 END 。
　　该JSONL文件的第一行如下所示：{＂prompt＂:＂18, man, invisibility ->＂,＂completion＂:＂   He looks to be in his late teens, with dark eyes and unruly black hair which reach his shoulders. His build is athletic and strong, but not muscular. He often wears clothing that makes him blend in with his surroundings, in order to stay as inconspicuous as possible.  He has the ability of invisibility, which he can use to become an invisible observer or to make himself or objects around him disappear from view. He has honed his skills to make himself undetectable, able to even make sounds vanish, allowing him to go unnoticed. His invisibility power is his greatest asset, and he uses it to protect those he cares about. END＂} {＂prompt＂:＂18, man, invisibility ->＂,＂completion＂:＂   18 year-old Jackson is a powerful young man with the incredible power of invisibility. He stands at 6＂2 and has a strong athletic frame with wavy brown hair, startling blue eyes and a warm and gentle smile. His strength and agility make him especially good at using his ability. He can become invisible in an instant, moving through walls and other obstacles without being detected. What＂s more, he can move objects with his mind and use his power to protect those he loves. His power is a blessing and a curse, as it can be abused to do harm. Jackson is a brave and noble person who uses his power to do good and make the world a better place. END＂} (...)
　　GPT-3模型的微调是在第二阶段真正实现的 subprocess.run （）， 其中 openai api微调.创建 被执行。在这个函数中，我们首先给出之前创建的JSONL文件的名称。然后，您需要选择要微调的型号。OpenAI提供四种主要模型，具有不同的性能水平，适用于各种任务。& nbsp; 达温奇 是最强大的模型，而Ada是最快的。nbsp; 达温奇 也是最贵的型号。
　　由于我的模型的目的是创建对超级英雄的描述，我们给我的新模型加了后缀＂超级英雄＂。
　　就这样几分钟后，您将拥有一个微调好的模型，可以随时使用。现在是时候测试你的新模型了
　　有不同的方式来使用模型来完成。主要通过OpenAI提供的Playground或通过Python等编程语言。
　　最简单的方法可能是使用 操场。请访问 https：//platform.openai.com/playground。点击＂型号＂并搜索后缀为＂＂的型号超级英雄＂.在＂停止序列＂中添加标记＂END＂。
　　现在是时候让我们的模型做出新的预测了。我们会要求描述一个18岁的男性角色，他真的拥有不必要的力量。我们会要求描述一个拥有力量的角色......＂可以吃很多＂......看看会发生什么......
　　不算太糟
　　你想用Python来做吗？简单!点击屏幕右上角的＂查看代码＂。
　　在我们的例子中，在＂查看代码＂中，我们有：import os import openai  openai.api_key = os.getenv(＂OPENAI_API_KEY＂)  response = openai.Completion.create(   model=＂davinci:ft-klhm:superhero-2023-02-01-14-56-48＂,   prompt=＂18, Man, can eat a lot -> ＂,   temperature=0.7,   max_tokens=256,   top_p=1,   frequency_penalty=0,   presence_penalty=0,   stop=[＂END＂] )
　　只需要复制粘贴就可以了。结论
　　在这个博客中，我们已经看到了如何生成合成数据来完善我们的模型，以及如何进行微调。我们使用了一个创建超级英雄的用例，但是同样的方法可以用于您可能有的任何用例。最重要的是要有足够的高质量的例子，有提示和期望的反应。