微信扫码
与创始人交个朋友
我要投稿
01。
概述
02。
训练效率与性能
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "cerebras/Llama3-DocChat-1.0-8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
system = "This is a chat between a user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions based on the context. The assistant should also indicate when the answer cannot be found in the context."
instruction = "Please give a full and complete answer for the question."
document = """
# Cerebras Wafer-Scale Cluster
Exa-scale performance, single device simplicity
## AI Supercomputers
Condor Galaxy (CG), the supercomputer built by G42 and Cerebras, is the simplest and fastest way to build AI models in the cloud. With over 16 ExaFLOPs of AI compute, Condor Galaxy trains the most demanding models in hours rather than days. The terabyte scale MemoryX system natively accommodates 100 billion+ parameter models, making large scale training simple and efficient.
| Cluster | ExaFLOPs | Systems | Memory |
| -------- | -------- | -------- | ------ |
| CG1 | 4 | 64 CS-2s | 82 TB |
| CG2 | 4 | 64 CS-2s | 82 TB |
| CG3 | 8 | 64 CS-3s | 108 TB |
"""
question = "How many total CS systems does Condor Galaxy 1, 2, and 3 have combined, and how many flops does this correspond to?"
user_turn = f"""<context>
{document}
</context>
{instruction} {question}"""
messages = [
{"role": "system", "content": system},
{"role": "user", "content": user_turn}
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
outputs = model.generate(
input_ids,
max_new_tokens=256,
eos_token_id=terminators,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
03。
开源承诺
04。
基准比较
05。
面临的挑战与未来展望
53AI,企业落地应用大模型首选服务商
产品:大模型应用平台+智能体定制开发+落地咨询服务
承诺:先做场景POC验证,看到效果再签署服务协议。零风险落地应用大模型,已交付160+中大型企业
2024-11-08
Ollama 更新!手把手教你用Ollama轻松搭建Llama 3.2 Vision + 视觉RAG系统(本地安装)
2024-11-08
全参微调与LoRA的区别,及7种LoRA变种方法解析
2024-11-08
开发一款大模型需要经过哪些步骤?开发一款大模型的完整流程
2024-11-08
推算LLM训练的GPU内存需求
2024-11-07
GPT-4o加钱能变快!新功能7秒完成原先23秒的任务
2024-11-05
【导读】SELA:基于树搜索增强的LLM Agent用于自动化机器学习
2024-11-05
全参微调与LoRA的区别,及7种LoRA变种方法解析
2024-11-04
再也不用为写文档摘要发愁了!AI总结助手帮你实现
2024-07-11
2024-07-11
2024-07-09
2024-09-18
2024-06-11
2024-07-23
2024-07-20
2024-07-12
2024-07-23
2024-07-26