微信扫码
与创始人交个朋友
我要投稿
https://github.com/langchain-ai/langgraph/blob/main/examples/rag/langgraph_rag_agent_llama3_local.ipynb
LLaMA-Factory微调,已经支持LLaMA-3
https://github.com/hiyouga/LLaMA-Factory
Colab笔记本中将Llama-3微调速度提高2倍
https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing
目前官方的说法是Llama3不支持function calling
一种方法采用Function Calling数据微调后的Llama 3
https://huggingface.co/Trelis/Meta-Llama-3-8B-Instruct-function-calling
prompt模板:
<|begin_of_text|><|start_header_id|>function_metadata<|end_header_id|>
You have access to the following functions. Use them if required:
[
{
"type": "function",
"function": {
"name": "get_stock_price",
"description": "Get the stock price of an array of stocks",
"parameters": {
"type": "object",
"properties": {
"names": {
"type": "array",
"items": {
"type": "string"
},
"description": "An array of stocks"
}
},
"required": [
"names"
]
}
}
},
{
"type": "function",
"function": {
"name": "get_big_stocks",
"description": "Get the names of the largest N stocks by market cap",
"parameters": {
"type": "object",
"properties": {
"number": {
"type": "integer",
"description": "The number of largest stocks to get the names of, e.g. 25"
},
"region": {
"type": "string",
"description": "The region to consider, can be \"US\" or \"World\"."
}
},
"required": [
"number"
]
}
}
}
]<|eot_id|><|start_header_id|>user<|end_header_id|>
Get the names of the five largest stocks by market cap<|eot_id|><|start_header_id|>assistant<|end_header_id|>
Generated Response:
{
"name": "get_big_stocks",
"arguments": {
"number": 5,
"region": "US"
}
}<|eot_id|>
另外一种就是第三方项目中有一个调用示例,可以尝试下:
项目地址
https://github.com/themrzmaster/llama-cpp-python/blob/4a4b67399b9e5ee872e2b87068de75e2908d5b11/llama_cpp/llama_chat_format.py#L2866
调用示例
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="s")
response = client.chat.completions.create(
model="m",
messages = [
{
"role": "system",
"content": "You are a digital waiter Pay attention to the user requests and use the tools to help you."
},
{
"role": "user",
"content": "search for burguer and a maximum price of 10 dollars. at the sime time, look for merchant named 'Burger King'"
},
],
tools=[
{
"type": "function",
"function": {
"name": "search_merchant",
"description": "Search for merchants in the catalog based on the term",
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "name to be searched for finding merchants.",
}
},
"required": ["name"],
},
},
},
{
"type": "function",
"function": {
"name": "search_item",
"description": "Search for items in the catalog based on various criteria.",
"parameters": {
"type": "object",
"properties": {
"term": {
"type": "string",
"description": "Term to be searched for finding items, with removed accents.",
},
"item_price_to": {
"type": "integer",
"description": "Maximum price the user is willing to pay for an item, if specified.",
},
"merchant_delivery_fee_to": {
"type": "integer",
"description": "Maximum delivery fee the user is willing to pay, if specified.",
},
"merchant_payment_types": {
"type": "string",
"description": "Type of payment the user prefers, if specified.",
"enum": [
"Credit Card",
"Debit Card",
"Other",
],
},
},
"required": ["term"],
},
},
}],
tool_choice="auto"
)
https://twitter.com/9hills/status/1781757051838050630
中文RAG选择 CommandR+;Agent/FunctionCalling 使用 Llama3-70B 或 CommandR+;中文文案写作用Qwen-72B;特定任务的小参数微调base模型用 Llama3-8B 或 Mistral-7B;大参数微调base 模型用Yi-34B;代码生成用 Llama3-70B或deepseek-coder-33B;特定类编程(比如nl2sql),小参数llama3-8b,大参数deepzeek-coder-33b;生产环境:vLLM、TensorRT-LLM;本地环境:Ollama.
53AI,企业落地应用大模型首选服务商
产品:大模型应用平台+智能体定制开发+落地咨询服务
承诺:先做场景POC验证,看到效果再签署服务协议。零风险落地应用大模型,已交付160+中大型企业
2024-04-26
2024-05-14
2024-03-30
2024-04-12
2024-05-10
2024-05-28
2024-07-18
2024-05-22
2024-04-25
2024-04-26
2024-11-22
2024-11-22
2024-11-21
2024-11-20
2024-11-19
2024-11-18
2024-11-18
2024-11-16