微信扫码
与创始人交个朋友
我要投稿
通过前两篇文章,相信你已经了解到GraphRag.Net目前只支持OpenAI规范的接口,但许多小伙伴在社区中提议,希望能增加对本地模型(例如:ollama等)的支持。所以这次,我们将探讨如何在GraphRag.Net中使用自定义模型和本地模型。
GraphRag.Net采用了Semantic Kernel作为基础,让我们能够非常简洁地抽象出会话与向量接口。因此,用户可以非常方便地实现自己定制的解决方案。接下来,我们会通过一个具体的例子,展示如何将本地模型和国产模型集成到GraphRag.Net中。
首先,我们来看看如何进行默认配置:
// OpenAI配置
builder.Configuration.GetSection("OpenAI").Get<OpenAIOption>();
// 文档切片配置
builder.Configuration.GetSection("TextChunker").Get<TextChunkerOption>();
// 配置数据库连接
builder.Configuration.GetSection("GraphDBConnection").Get<GraphDBConnectionOption>();
// 注意,需要先注入配置文件,然后再注入GraphRag.Net
builder.Services.AddGraphRagNet();
这里,我们将在默认配置中注入OpenAI的配置、文本切片的配置和数据库连接的配置。然后,依次注入这些配置文件和GraphRag.Net的服务。
如果需要自定义模型或本地模型,可能需要实现一些额外的服务接口,下面是自定义配置的示例:
var kernelBuild = Kernel.CreateBuilder();
kernelBuild.Services.AddKeyedSingleton<ITextGenerationService>("mock-text", new MockTextCompletion());
kernelBuild.Services.AddKeyedSingleton<IChatCompletionService>("mock-chat", new MockChatCompletion());
kernelBuild.Services.AddSingleton<ITextEmbeddingGenerationService>(new MockTextEmbeddingGeneratorService());
kernelBuild.Services.AddKeyedSingleton("mock-embedding", new MockTextEmbeddingGeneratorService());
builder.Services.AddGraphRagNet(kernelBuild.Build());
在这个自定义配置示例中,我们引入了三个自定义服务接口:ITextGenerationService
、IChatCompletionService
和ITextEmbeddingGenerationService
。
接下来,我们需要为每个服务接口提供具体的实现。以下是三个接口的具体实现:
IChatCompletionService
public class MockChatCompletion : IChatCompletionService
{
private readonly Dictionary<string, object?> _attributes = new();
private string _chatId;
private static readonly JsonSerializerOptions _jsonSerializerOptions = new()
{
NumberHandling = JsonNumberHandling.AllowReadingFromString,
Encoder = JavaScriptEncoder.Create(UnicodeRanges.All)
};
public IReadOnlyDictionary<string, object?> Attributes => _attributes;
public MockChatCompletion()
{
}
public async Task<IReadOnlyList<ChatMessageContent>> GetChatMessageContentsAsync(ChatHistory chatHistory, PromptExecutionSettings? executionSettings = null, Kernel? kernel = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
StringBuilder sb = new();
string result = $"这是一条Mock数据,便于聊天测试,你的消息是:{chatHistory.LastOrDefault().ToString()}";
return [new(AuthorRole.Assistant, result.ToString())];
}
public async IAsyncEnumerable<StreamingChatMessageContent> GetStreamingChatMessageContentsAsync(ChatHistory chatHistory, PromptExecutionSettings? executionSettings = null, Kernel? kernel = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
StringBuilder sb = new();
string result = $"这是一条Mock数据,便于聊天测试,你的消息是:{chatHistory.LastOrDefault().ToString()}";
foreach (var c in result)
{
yield return new StreamingChatMessageContent(AuthorRole.Assistant, c.ToString());
}
}
}
ITextGenerationService
public class MockTextCompletion : ITextGenerationService, IAIService
{
private readonly Dictionary<string, object?> _attributes = new();
private string _chatId;
private static readonly JsonSerializerOptions _jsonSerializerOptions = new()
{
NumberHandling = JsonNumberHandling.AllowReadingFromString,
Encoder = JavaScriptEncoder.Create(UnicodeRanges.All)
};
public IReadOnlyDictionary<string, object?> Attributes => _attributes;
public MockTextCompletion()
{
}
public async Task<IReadOnlyList<TextContent>> GetTextContentsAsync(string prompt, PromptExecutionSettings? executionSettings = null, Kernel? kernel = null, CancellationToken cancellationToken = default)
{
StringBuilder sb = new();
string result = $"这是一条Mock数据,便于聊天测试,你的消息是:{prompt}";
return [new(result.ToString())];
}
public async IAsyncEnumerable<StreamingTextContent> GetStreamingTextContentsAsync(string prompt, PromptExecutionSettings? executionSettings = null, Kernel? kernel = null, CancellationToken cancellationToken = default)
{
StringBuilder sb = new();
string result = $"这是一条Mock数据,便于聊天测试,你的消息是:{prompt}";
foreach (var c in result)
{
var streamingTextContent = new StreamingTextContent(c.ToString(), modelId: "mock");
yield return streamingTextContent;
}
}
}
ITextEmbeddingGenerationService
public sealed class MockTextEmbeddingGeneratorService : ITextEmbeddingGenerationService
{
private Dictionary<string, object?> AttributesInternal { get; } = [];
public IReadOnlyDictionary<string, object?> Attributes => this.AttributesInternal;
public MockTextEmbeddingGeneratorService()
{
}
public async Task<IList<ReadOnlyMemory<float>>> GenerateEmbeddingsAsync(
IList<string> data,
Kernel? kernel = null,
CancellationToken cancellationToken = default)
{
IList<ReadOnlyMemory<float>> results = new List<ReadOnlyMemory<float>>();
float[] array1 = { 1.0f, 2.0f, 3.0f };
float[] array2 = { 4.0f, 5.0f, 6.0f };
float[] array3 = { 7.0f, 8.0f, 9.0f };
// 将数组包装为ReadOnlyMemory<float>并添加到列表中
results.Add(new ReadOnlyMemory<float>(array1));
results.Add(new ReadOnlyMemory<float>(array2));
results.Add(new ReadOnlyMemory<float>(array3));
return results;
}
public void Dispose()
{
}
}
看到这里,你可能已经发现,集成自定义模型和本地模型非常简单。只需按照上述步骤,实现相应的接口并注入配置,你就可以在GraphRag.Net中使用这些自定义的功能。
通过本文的介绍,我们了解了如何在GraphRag.Net中集成国产模型和本地模型。希望大家能够根据这些示例,开发出更多适合自己需求的功能。更多精彩内容,欢迎关注我的公众号,并发送进群加入我们的GraphRag.Net交流群,与社区小伙伴们一起交流学习!
感谢阅读,我们下期再见!
53AI,企业落地应用大模型首选服务商
产品:大模型应用平台+智能体定制开发+落地咨询服务
承诺:先做场景POC验证,看到效果再签署服务协议。零风险落地应用大模型,已交付160+中大型企业
2024-11-23
FastRAG半结构化RAG实现思路及OpenAI O1-long COT蒸馏路线思考
2024-11-23
检索增强生成(RAG):解密AI如何融合记忆与搜索
2024-11-23
如何提高RAG系统准确率?12大常见痛点及巧妙解!
2024-11-23
RAG 2.0性能提升:优化索引与召回机制的策略与实践
2024-11-22
RAG技术在实际应用中的挑战与解决方案
2024-11-22
从普通RAG到RAPTOR,10个最新的RAG框架
2024-11-22
如何使用 RAG 提高 LLM 成绩
2024-11-21
提升RAG性能的全攻略:优化检索增强生成系统的策略大揭秘 | 深度好文
2024-07-18
2024-05-05
2024-07-09
2024-05-19
2024-07-09
2024-06-20
2024-07-07
2024-07-07
2024-07-08
2024-07-09
2024-11-06
2024-11-06
2024-11-05
2024-11-04
2024-10-27
2024-10-25
2024-10-21
2024-10-21