CO-STAR 的一个实际应用 这里通过一个实际案例说明 CO-STAR 为什么有用。假设你是一位社交媒体管理者,你需要帮助草拟一篇 Facebook 帖文,其內容是宣传你公司的新产品。 如果不使用 CO-STAR,那么你可能会使用这样的 prompt: Write a facebook post to advertise my company’s new product. My company’s name is Alpha and the product is called Beta, a new ultra-fast hairdryer.
GPT-4 的输出如下: Introducing the Revolutionary Beta Hairdryer by Alpha!- Say goodbye to long, tedious hair drying sessions! Alpha is proud to unveil the Beta, our latest innovation in haircare technology.- Ultra-Fast Drying: The Beta is designed to cut your drying time in half, thanks to its advanced, powerful motor. Get ready faster and enjoy more time doing what you love!Shop Now: [Link to product page]
这个结果也许够用了,但却只是泛泛而谈,缺乏具体的细节和有针对性的吸引力,而这些是吸引目标受众所必需的。 那如果使用 CO-STAR 模板呢?CO-STAR 会提醒你将任务涉及的其它方面也整合进 prompt 之中,具体来说就是上面的简单 prompt 中缺失的 STYLE、TONE、AUDIENCE 方面: # CONTEXT #I want to advertise my company’s new product. My company’s name is Alpha and the product is called Beta, which is a new ultra-fast hairdryer.# OBJECTIVE #Create a Facebook post for me, which aims to get people to click on the product link to purchase it.# STYLE #Follow the writing style of successful companies that advertise similar products, such as Dyson.# TONE #Persuasive# AUDIENCE #My company’s audience profile on Facebook is typically the older generation. Tailor your post to target what this audience typically looks out for in hair products.# RESPONSE #The Facebook post, kept concise yet impactful.
通过使用 CO-STAR,GPT-4 的响应会变得明显更有针对性、效果更好: Embrace Effortless Elegance with Alpha Beta — The Ultra-Fast Hairdryer for the Wise Generation.Rediscover the joy of simple, effective hair care with Alpha Beta. Our latest innovation is more than just a hairdryer; it’s a promise of swift, gentle, and reliable hair styling for those who appreciate the finer things in life.- Easy and Intuitive Use: Say goodbye to complicated gadgets. Alpha Beta iscrafted for comfort and simplicity, perfect for those who value straightforward,hassle-free technology.- Time-Saving Technology: We understand your time is precious. That’s why Alpha Beta cuts down drying time significantly, giving you more moments to enjoy life’s pleasures.Make every day a good hair day with Alpha Beta. Experience the blend of sophistication and simplicity today.[Your Product Link Here]
CO-STAR 框架能引导用户以一种结构化的方式向 LLM 提供有关你的任务所有关键信息,从而确保得到完全符合你的需求的定制化的更优的响应。 2. [?] 使用分隔符为 prompt 设置分节 图片由 DALL・E 3 生成 分隔符是一种特殊 token,可帮助 LLM 分辨 prompt 的哪些部分应被视为单个含义单元。这很重要,因为输入 LLM 的整个 prompt 是单个的 token 长序列。分隔符能将 prompt 中不同部分隔离开,从而为这个 token 序列提供结构,让其中各个部分能被区别对待。需要说明的是,如果任务很简单,那么分隔符对 LLM 的响应质量的影响不大。但是,任务越复杂,使用分隔符分节对 LLM 响应的影响就越大。 用特殊字符当分隔符 分隔符可以使用任何通常不会同时出现的特殊字符序列,举些例子:###、===、>>> 特殊字符的数量和类型并不重要,只要它们足够独特即可,这样才能让 LLM 将它们理解成内容分隔符,而不是普通的标点符号。 下面通过一个例子说明如何在 prompt 中使用分隔符: Classify the sentiment of each conversation in <<‘Positive’ or ‘Negative’. Give the sentiment classifications without any other preamble text. ###EXAMPLE CONVERSATIONS[Agent]: Good morning, how can I assist you today?[Customer]: This product is terrible, nothing like what was advertised![Customer]: I’m extremely disappointed and expect a full refund.[Agent]: Good morning, how can I help you today? [Customer]: Hi, I just wanted to say that I’m really impressed with yourproduct. It exceeded my expectations!EXAMPLE OUTPUTSNegativePositive###<<< [Agent]: Hello! Welcome to our support. How can I help you today?[Customer]: Hi there! I just wanted to let you know I received my order, andit’s fantastic![Agent]: That’s great to hear! We’re thrilled you’re happy with your purchase.Is there anything else I can assist you with? [Customer]: No, that’s it. Just wanted to give some positive feedback. Thanksfor your excellent service![Agent]: Hello, thank you for reaching out. How can I assist you today?[Customer]: I’m very disappointed with my recent purchase. It’s not what I expected at all.[Agent]: I’m sorry to hear that. Could you please provide more details so I can help?[Customer]: The product is of poor quality and it arrived late. I’m reallyunhappy with this experience.>>>
上面例子中使用的分隔符是 ###,同时每一节都带有完全大写的标题以示区分,如 EXAMPLE CONVERSATIONS 和 EXAMPLE OUTPUTS。前置说明部分陈述了要分类的对话是在 << GPT-4 对此 prompt 给出的输出如下,其给出的情感分类结果不带任何附加文本,这符合我们的要求: PositiveNegative
用 XML 标签当分隔符 另一种方法是使用 XML 标签作为分隔符。XML 标签是使用尖括号括起来的成对标签,包括开始和结束标签。比如。这很有效,因为 LLM 在训练时就看过了大量用 XML 标注的网络内容,已经学会了理解其格式。 下面用 XML 标签作为分隔符重写上面的 prompt: Classify the sentiment of the following conversations into one of two classes, using the examples given. Give the sentiment classifications without any otherpreamble text. PositiveNegative [Agent]: Good morning, how can I assist you today?[Customer]: This product is terrible, nothing like what was advertised![Customer]: I’m extremely disappointed and expect a full refund.[Agent]: Good morning, how can I help you today? [Customer]: Hi, I just wanted to say that I’m really impressed with yourproduct. It exceeded my expectations! NegativePositive [Agent]: Hello! Welcome to our support. How can I help you today?[Customer]: Hi there! I just wanted to let you know I received my order, andit’s fantastic![Agent]: That’s great to hear! We’re thrilled you’re happy with your purchase.Is there anything else I can assist you with? [Customer]: No, that’s it. Just wanted to give some positive feedback. Thanksfor your excellent service![Agent]: Hello, thank you for reaching out. How can I assist you today?[Customer]: I’m very disappointed with my recent purchase. It’s not what Iexpected at all. [Agent]: I’m sorry to hear that. Could you please provide more details so Ican help?[Customer]: The product is of poor quality and it arrived late. I’m reallyunhappy with this experience.
为了达到更好的效果,在 XML 标签中使用的名词应该与指令中用于描述它们的名词一样。在上面的 prompt 中,我们给出的指令为: Classify the sentiment of the following conversations into one of two classes, using the examples given. Give the sentiment classifications without any other preamble text. 其中使用的名词有 conversations、classes 和 examples。也因此,后面的分隔 XML 标签就对应为 同样的,使用这样的分隔符能以清晰的结构化方式对 prompt 进行分节,从而确保 GPT-4 输出的内容就刚好是你想要的结果: PositiveNegative
举个例子,System Prompt 可能是这样的: You will answer questions using this text: [insert text].You will respond with a JSON object in this format: {“Question”: “Answer”}.If the text does not contain sufficient information to answer the question, do not make up information and give the answer as “NA”. You are only allowed to answer questions related to [insert scope]. Never answer any questions related to demographic information such as age, gender, and religion.
其中每部分的类别如下: 对上述 System Prompt 的分解 那么「普通」prompt 又该包含哪些内容呢? 现在你可能会想:看起来 System Prompt 中已经给出了大量信息。那么我们又该在「普通」prompt(也称为用户 prompt)中放什么内容? System Prompt 会大致描述任务概况。在上面的 System Prompt 示例中,任务被定义为仅使用特定的文本进行问答,并指示 LLM 以 {"Question": "Answer"} 的格式进行响应。 You will answer questions using this text: [insert text].You will respond with a JSON object in this format: {“Question”: “Answer”}.
在这个案例中,聊天中的每个用户 prompt 都只是你希望得到文本解答的问题。举个例子,用户 prompt 可能是这样「What is the text about?」。而 LLM 的响应会是这样:{"What is the text about?": "The text is about..."}。 但我们可以进一步泛化这个示例任务。在实践中,你更可能会有多个希望得到解答的问题,而不只是一个。在这个案例中,我们可以将上述 System Prompt 的第一行从 You will answer questions using this text: [insert text].
改成 You will answer questions using the provided text.
对于这些类型的基于模式的任务,实际上相比于使用代码,仅使用 LLM 可能还能在更短的时间内得到更好的结果。下面通过一个示例来完整演示一番。 仅使用 LLM 来分析 Kaggle 数据集 该示例会使用一个常用的真实世界 Kaggle 数据集,该数据集是为客户个性分析任务收集整理的,其中的任务目标是对客户群进行细分,以更好地了解客户。 为了方便后面验证 LLM 的分析结果,这里仅取用一个子集,其中包含 50 行和最相关的列。之后,用于分析的数据集如下所示,其中每一行都代表一个客户,列则描述了客户信息: 数据集的前三行 假设你在该公司的宣传团队工作。你的任务使用这个客户信息数据集来指导营销工作。这个任务分为两步:第一步,使用数据集生成有意义的细分客户群。第二步,针对每个细分群生成最好的营销策略。现在,这个问题就成了模式发现(第一步)的实际业务问题,这也正是 LLM 擅长的能力。 下面针对这个任务草拟一个 prompt,这里用到了 4 种提示工程技术(后面还有更多!): 1. 将复杂任务分解为简单步骤2. 索引每一步的中间输出3. 设置 LLM 的响应的格式4. 将指令与数据集分离开 System Prompt:I want you to act as a data scientist to analyze datasets. Do not make up information that is not in the dataset. For each analysis I ask for, provide me with the exact and definitive answer and do not provide me with code or instructions to do the analysis on other platforms.Prompt:# CONTEXT #I sell wine. I have a dataset of information on my customers: [year of birth, marital status, income, number of children, days since last purchase, amount spent].############# # OBJECTIVE #I want you use the dataset to cluster my customers into groups and then give me ideas on how to target my marketing efforts towards each group. Use this step-by-step process and do not use code: 1. CLUSTERS: Use the columns of the dataset to cluster the rows of the dataset, such that customers within the same cluster have similar column values while customers in different clusters have distinctly different column values. Ensure that each row only belongs to 1 cluster.For each cluster found,2. CLUSTER_INFORMATION: Describe the cluster in terms of the dataset columns.3. CLUSTER_NAME: Interpret [CLUSTER_INFORMATION] to obtain a short name for the customer group in this cluster.4. MARKETING_IDEAS: Generate ideas to market my product to this customer group.5. RATIONALE: Explain why [MARKETING_IDEAS] is relevant and effective for this customer group.############# # STYLE #Business analytics report############## TONE #Professional, technical############## AUDIENCE #My business partners. Convince them that your marketing strategy is well thought-out and fully backed by data.############## RESPONSE: MARKDOWN REPORT #— Customer Group: [CLUSTER_NAME]— Profile: [CLUSTER_INFORMATION]— Marketing Ideas: [MARKETING_IDEAS]— Rationale: [RATIONALE] Give a table of the list of row numbers belonging to each cluster, in order to back up your analysis. Use these table headers: [[CLUSTER_NAME], List of Rows].############# # START ANALYSIS #If you understand, ask me for my dataset.
高品位爱好者的完整数据 这同样与 LLM 识别出的人群画像非常符合! 这个例子彰显了 LLM 发现模式的能力,其能从多维度的数据集中解读和提炼出有意义的见解,这能确保其分析深深植根于数据集的事实真相。 如果使用 ChatGPT 的高级数据分析插件呢? 为了完整比较,我使用同样的 prompt 尝试了同样的任务,但是让 ChatGPT 使用代码执行分析,这会激活其高级数据分析插件。这里的思路是让插件直接在数据集上运行 k - 均值聚类等聚类算法的代码,从而得到每个客户群,之后再合成每个聚类的人群画像,以提供营销策略。 但是,多次尝试都得到了以下报错信息,并没有输出,尽管这个数据集只有 50 行: 第一次尝试的报错信息,没有输出 第二次尝试的报错信息,没有输出 目前而言,使用高级数据分析插件只能执行更简单的数据集任务,比如计算描述性统计信息或创建图表,但需要算法的更高级任务有时可能会遭遇报错,无法得到输出结果,原因可能是计算限制等问题。 那么 LLM 适合在什么时候用来分析数据集? 答案是取决于分析的类型。 对于需要精准数学计算或基于规则的复杂处理的任务,常规的编程方法依然更优。 对于基于模式识别的任务,使用常规的编程和算法方法可能很困难且非常耗时。而 LLM 擅长这些任务,而且甚至还能提供额外的输出,比如用于支撑其分析结果的附加说明,以 markdown 格式编写完整的分析报告。 最终,决定是否使用 LLM,取决于当前任务的性质以及 LLM 的模式识别能力与传统编程技术提供的精确性和针对性之间的权衡。 现在回到提示工程! 本章节最后,我们回到用于生成数据集分析的 prompt,分解一下其中使用的关键性提示工程技术: Prompt:# CONTEXT #I sell wine. I have a dataset of information on my customers: [year of birth, marital status, income, number of children, days since last purchase, amount spent].############# # OBJECTIVE #I want you use the dataset to cluster my customers into groups and then give me ideas on how to target my marketing efforts towards each group. Use this step-by-step process and do not use code: 1. CLUSTERS: Use the columns of the dataset to cluster the rows of the dataset, such that customers within the same cluster have similar column values while customers in different clusters have distinctly different column values. Ensure that each row only belongs to 1 cluster. For each cluster found,2. CLUSTER_INFORMATION: Describe the cluster in terms of the dataset columns.3. CLUSTER_NAME: Interpret [CLUSTER_INFORMATION] to obtain a short name for the customer group in this cluster. 4. MARKETING_IDEAS: Generate ideas to market my product to this customer group.5. RATIONALE: Explain why [MARKETING_IDEAS] is relevant and effective for this customer group.############# # STYLE #Business analytics report############## TONE #Professional, technical############# # AUDIENCE #My business partners. Convince them that your marketing strategy is well thought-out and fully backed by data.############## RESPONSE: MARKDOWN REPORT #— Customer Group: [CLUSTER_NAME]— Profile: [CLUSTER_INFORMATION]— Marketing Ideas: [MARKETING_IDEAS]— Rationale: [RATIONALE] Give a table of the list of row numbers belonging to each cluster, in order to back up your analysis. Use these table headers: [[CLUSTER_NAME], List of Rows].############## START ANALYSIS #If you understand, ask me for my dataset.
技术 1:将复杂任务分解为简单步骤 LLM 擅于执行简单任务,并不很擅长复杂任务。因此,对于这样的复杂任务,一种很好的做法是将其分解成简单的逐步指示,以便 LLM 遵从。这里的思路是为 LLM 提供你希望采取的步骤。 在这个案例中,给出步骤的方式为: Use this step-by-step process and do not use code:1. CLUSTERS: Use the columns of the dataset to cluster the rows of the dataset, such that customers within the same cluster have similar column values while customers in different clusters have distinctly different column values. Ensure that each row only belongs to 1 cluster.For each cluster found,2. CLUSTER_INFORMATION: Describe the cluster in terms of the dataset columns.3. CLUSTER_NAME: Interpret [CLUSTER_INFORMATION] to obtain a short name for the customer group in this cluster.4. MARKETING_IDEAS: Generate ideas to market my product to this customer group.5. RATIONALE: Explain why [MARKETING_IDEAS] is relevant and effective for this customer group.
这里并没有直接简单地给 LLM 提供一个整体的任务描述,比如「将客户聚类成不同的客户群,然后针对每个客户群给出营销见解。」 通过使用逐步指示,LLM 更有可能给出正确结果。 技术 2:索引每一步的中间输出 在为 LLM 提供逐步过程时,我们给出了每一步的中间输出结果,其中用的大写变量名指代,即 CLUSTERS、CLUSTER_INFORMATION、CLUSTER_NAME、MARKETING_IDEAS 和 RATIONALE。 使用大写可以将这些变量名与指令主体区分开。然后,可以通过加方括号的形式 [变量名] 索引这些中间输出。 技术 3:设置 LLM 的响应的格式 这里我们要求输出 markdown 报告格式,这能美化 LLM 的响应结果。在这里,中间输出的变量名再次派上用场,可以更方便地指定报告的结构。 # RESPONSE: MARKDOWN REPORT #— Customer Group: [CLUSTER_NAME]— Profile: [CLUSTER_INFORMATION]— Marketing Ideas: [MARKETING_IDEAS]— Rationale: [RATIONALE]Give a table of the list of row numbers belonging to each cluster, in order to back up your analysis. Use these table headers: [[CLUSTER_NAME], List of Rows].
事实上,你之后也可以让 ChatGPT 提供可下载的报告文件,让其直接完成你的最终报告。 将 GPT-4 的响应保存为一个文件 技术 4:将任务指令与数据集分离开 可以看到,我们从未在第一个 prompt 中向 LLM 提供数据集。相反,该 prompt 只给出了数据集分析的任务指令,最后再加上了以下内容: # START ANALYSIS #If you understand, ask me for my dataset.