AI知识库

53AI知识库

学习大模型的前沿技术与行业应用场景


Llama3实操的正确打开方式:RAG,Agent,Function Calling!
发布日期:2024-04-22 20:56:34 浏览次数: 3962 来源:PaperAgent


基于Llama3的RAG

https://github.com/langchain-ai/langgraph/blob/main/examples/rag/langgraph_rag_agent_llama3_local.ipynb
  • 展示了如何从头开始使用LangGraph和Llama3-8b构建可靠的本地代理。
  • 将3种先进RAG技术(自适应RAG、纠正性RAG和自我RAG)的思想结合起来,形成LangGraph中的单一控制流程。
  • 路由:自适应RAG。将问题路由到不同的检索方法。
  • 后备方案:纠正性RAG。如果文档与查询不相关,则回退到网页搜索。
  • 自我纠错:自我RAG。修正包含幻觉或未回答问题的答案。

Llama3微调

  • LLaMA-Factory微调,已经支持LLaMA-3

https://github.com/hiyouga/LLaMA-Factory

  • Colab笔记本中将Llama-3微调速度提高2倍

https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing

基于Llama3的function calling/Agent

目前官方的说法是Llama3不支持function calling

  • 一种方法采用Function Calling数据微调后的Llama 3

https://huggingface.co/Trelis/Meta-Llama-3-8B-Instruct-function-calling

prompt模板:

<|begin_of_text|><|start_header_id|>function_metadata<|end_header_id|>
You have access to the following functions. Use them if required:[{"type": "function","function": {"name": "get_stock_price","description": "Get the stock price of an array of stocks","parameters": {"type": "object","properties": {"names": {"type": "array","items": {"type": "string"},"description": "An array of stocks"}},"required": ["names"]}}},{"type": "function","function": {"name": "get_big_stocks","description": "Get the names of the largest N stocks by market cap","parameters": {"type": "object","properties": {"number": {"type": "integer","description": "The number of largest stocks to get the names of, e.g. 25"},"region": {"type": "string","description": "The region to consider, can be \"US\" or \"World\"."}},"required": ["number"]}}}]<|eot_id|><|start_header_id|>user<|end_header_id|>
Get the names of the five largest stocks by market cap<|eot_id|><|start_header_id|>assistant<|end_header_id|>
Generated Response:{"name": "get_big_stocks","arguments": {"number": 5,"region": "US"}}<|eot_id|>
  • 另外一种就是第三方项目中有一个调用示例,可以尝试下:

项目地址

https://github.com/themrzmaster/llama-cpp-python/blob/4a4b67399b9e5ee872e2b87068de75e2908d5b11/llama_cpp/llama_chat_format.py#L2866

调用示例

from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="s")
response = client.chat.completions.create(model="m", messages = [{"role": "system","content": "You are a digital waiter Pay attention to the user requests and use the tools to help you."}, {"role": "user","content": "search for burguer and a maximum price of 10 dollars. at the sime time, look for merchant named 'Burger King'"}, ],tools=[{"type": "function","function": {"name": "search_merchant","description": "Search for merchants in the catalog based on the term","parameters": {"type": "object","properties": {"name": {"type": "string","description": "name to be searched for finding merchants.",}},"required": ["name"],},},},{"type": "function","function": {"name": "search_item","description": "Search for items in the catalog based on various criteria.","parameters": {"type": "object","properties": {"term": {"type": "string","description": "Term to be searched for finding items, with removed accents.",},"item_price_to": {"type": "integer","description": "Maximum price the user is willing to pay for an item, if specified.",},"merchant_delivery_fee_to": {"type": "integer","description": "Maximum delivery fee the user is willing to pay, if specified.",},"merchant_payment_types": {"type": "string","description": "Type of payment the user prefers, if specified.","enum": ["Credit Card","Debit Card","Other",],},},"required": ["term"],},},}], tool_choice="auto")

Llama3实操技术选型推荐

https://twitter.com/9hills/status/1781757051838050630


中文RAG选择 CommandR+;Agent/FunctionCalling 使用 Llama3-70B 或 CommandR+;中文文案写作用Qwen-72B;特定任务的小参数微调base模型用 Llama3-8B 或 Mistral-7B;大参数微调base 模型用Yi-34B;代码生成用 Llama3-70B或deepseek-coder-33B;特定类编程(比如nl2sql),小参数llama3-8b,大参数deepzeek-coder-33b;生产环境:vLLM、TensorRT-LLM;本地环境:Ollama.





53AI,企业落地应用大模型首选服务商

产品:大模型应用平台+智能体定制开发+落地咨询服务

承诺:先做场景POC验证,看到效果再签署服务协议。零风险落地应用大模型,已交付160+中大型企业

联系我们

售前咨询
186 6662 7370
预约演示
185 8882 0121

微信扫码

与创始人交个朋友

回到顶部

 
扫码咨询