AI知识库

53AI知识库

学习大模型的前沿技术与行业应用场景


Omost!让AI大模型代写AI绘画提示词
发布日期:2024-06-16 07:59:12 浏览次数: 2427 来源:星空财研


对于AI绘画者来说,写提示词是非常头疼的一件事。

SDXL能够比较好的理解自然语言,但在构图细节等方面,依然还有欠缺。

刚刚发布的SD3对自然语言的理解更加精准,一定程度上解决了提示词的难题,但目前不认识中文。

前一段时间,著名的ControlNet、Fooocus等开源插件的作者张吕敏发布了Omost插件,这个插件神奇之处在于,它集成了LLM大模型,你只需要输入简单的句子,会通过大模型补充完善,增加细节描述词汇,然后生成非常专业的提示词。

这个思路太妙了。

去年的时候,我曾经介绍过用百度翻译等翻译插件来翻译中文提示词的工具,但大模型会把你的句子重新整理、组合,生成AI绘画大模型能理解的格式。

这个插件有独立安装板,也有ComfyUI节点,使用很简单,但无法科学上网的情况下,安装非常繁琐。

1、节点安装

节点地址:

https://github.com/huchenlei/ComfyUI_omost

安装后先不要重启启动器,先下载并安装模型。

2、安装模型

如果在墙内,模型无法联网自动下载。

如果贸然重启启动器,然后使用Omost会报错。

我制作了安装包,请下载后解压缩,安装在这个路径(以秋叶启动器为例):

ComfyUI\.cache\huggingface\hub

其中,最复杂的是模型安装的位置,这是llama的8b模型:

这个插件的作者比较懒,没有很好的处理好模型存放路径,导致很多人走到这里就报错。

安装完毕后,重启启动器。

3、节点说明

加载工作流,工作流网盘下载:

https://pan.quark.cn/s/b365f8a8e144

Omost主要有三个节点:

Omost LLM Loader,默认有三个模型,由于另外两个模型比较大,本文提供下载的是omost-llama-3-8b-4bits模型,选择这个即可。

Omost LLM chat,这是和大模型对话的节点,输入想要的提示词,支持汉语。然后大模型会把它转化成复杂、专业的提示词。

比如我输入的是“正在跑马拉松的女孩”。

大模型转化为:

[{'rect': (0, 90, 0, 90), 'prefixes': ['A girl running a marathon.'], 'suffixes': ['The image captures a determined young girl in the midst of a marathon run.', 'She is dressed in athletic gear, including a moisture-wicking tank top, shorts, and running shoes.', 'Her face shows a mix of focus and determination as she pushes through the challenging course.', 'The background features a scenic route with spectators cheering her on, adding to the sense of competition and community.', 'The sky is clear with a few scattered clouds, and the sun casts a warm glow over the entire scene.', 'The girl’s posture is upright, and her arms are pumping rhythmically as she strides confidently.', 'The image conveys a strong message of perseverance and the willpower to overcome obstacles.', 'girl, marathon, running, athletic, determined, focus, competition, community, spectators, scenic route, clear sky, clouds, sun, warm glow, upright posture, pumping arms, perseverance, willpower, obstacles.'], 'color': (135, 206, 235), 'mask': tensor([[1., 1., 1., ..., 1., 1., 1.],
[1., 1., 1., ..., 1., 1., 1.],
[1., 1., 1., ..., 1., 1., 1.],
...,
[1., 1., 1., ..., 1., 1., 1.],
[1., 1., 1., ..., 1., 1., 1.],
[1., 1., 1., ..., 1., 1., 1.]])}, {'color': (255, 215, 0), 'rect': [45, 85, 35, 90], 'prefixes': ['A girl running a marathon.', 'Marathon finish line.'], 'suffixes': ['The marathon finish line is visible on the bottom-right side of the image, slightly to the upper-left.', 'The finish line is marked with a prominent banner and flags, indicating the end of the marathon course.', 'The banner is decorated with motivational messages and the words “Finish” in bold letters.', 'The flags are the same vibrant red as the course markers.', 'The presence of the finish line adds a sense of accomplishment and finality to the marathon run.', 'marathon finish line, prominent banner, flags, end of course, motivational messages, bold letters, vibrant red, accomplishment, finality.', 'Triumphant and celebratory, marking the end of the marathon journey.', 'Event photography capturing the essence of the marathon finish line.', 'High-resolution image with sharp details and vibrant colors.'], 'mask': tensor([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]])}, {'color': (255, 0, 0), 'rect': [0, 50, 0, 50], 'prefixes': ['A girl running a marathon.', 'Marathon flags.'], 'suffixes': ['Marathon flags are positioned on the top-left side of the image.', 'These flags are vibrant red and are part of the course markings.', 'They are slightly to the lower-right, indicating the direction of the course.', 'The flags are well-defined, with clear text and symbols indicating the distance markers.', 'The presence of the flags adds to the authenticity and challenge of the marathon run.', 'marathon flags, vibrant red, course markings, direction, well-defined, text, symbols, distance markers, authenticity, challenge.', 'Authentic and challenging, emphasizing the marathon route.', 'Detailed photography focusing on the marathon flags and their significance.', 'Clear and well-defined flags with vibrant colors and sharp details.'], 'mask': tensor([[1.0000, 1.0000, 1.0000, ..., 0.0000, 0.0000, 0.0000],
[1.0000, 1.0000, 1.0000, ..., 0.0000, 0.0000, 0.0000],
[1.0000, 1.0000, 1.0000, ..., 0.0000, 0.0000, 0.0000],
...,
[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 0.0000]])}, {'color': (255, 255, 224), 'rect': [30, 80, 35, 90], 'prefixes': ['A girl running a marathon.', 'Scenic route background.'], 'suffixes': ['The background on the right side of the image features a scenic route.', 'The road is lined with trees and has a clear pathway marked by marathon flags.', 'The sky is clear with a few scattered clouds, and the sun casts a warm glow over the entire scene.', 'The distant horizon is visible, with rolling hills and a few buildings.', 'The scenic route adds a sense of challenge and beauty to the marathon course.', 'background, scenic route, road, trees, clear pathway, marathon flags, clear sky, clouds, sun, warm glow, horizon, rolling hills, buildings, challenge, beauty.', 'Challenging yet beautiful, enhancing the overall marathon experience.', 'Landscape photography showcasing the scenic route of the marathon.', 'High-quality background with detailed scenery and vibrant lighting.'], 'mask': tensor([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]])}, {'color': (250, 250, 210), 'rect': [5, 65, 0, 35], 'prefixes': ['A girl running a marathon.', 'Spectators cheering.'], 'suffixes': ['A group of spectators is visible on the left side of the image, slightly higher up.', 'They are holding signs and banners, cheering the girl on.', 'The spectators are dressed in casual attire, and their faces are lit up with excitement and support.', 'Some are taking photos, while others are simply enjoying the event.', 'The presence of the spectators adds a sense of community and shared experience to the image.', 'spectators, cheering, signs, banners, casual attire, excitement, support, photos, community, shared experience.', 'Supportive and enthusiastic, creating a lively environment.', 'Candid photography capturing the genuine emotions of the spectators.', 'Detailed depiction of spectators with clear expressions and actions.'], 'mask': tensor([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]])}, {'color': (240, 128, 128), 'rect': [5, 85, 20, 70], 'prefixes': ['A girl running a marathon.', 'The girl running.'], 'suffixes': ['The girl is positioned centrally in the image, her athletic attire clearly visible.', 'She is wearing a tank top designed for moisture-wicking, shorts, and high-quality running shoes.', 'Her face is a picture of determination, with sweat glistening on her forehead.', 'Her eyes are fixed forward, showing her concentration and resolve.', 'Her posture is upright, and her arms are pumping rhythmically as she strides confidently.', 'Her hair is tied back in a ponytail, keeping it out of her face as she runs.', 'girl, running, athletic attire, tank top, shorts, running shoes, determination, concentration, upright posture, pumping arms, ponytail.', 'Determined and focused, with a strong sense of perseverance.', 'Realistic photography capturing the essence of the marathon run.', 'High-resolution image with clear details and vibrant colors.'], 'mask': tensor([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]])}]

不难发现,大模型不仅把中文翻译成了英文,还添加了非常详实的细节:什么样的头发,穿什么衣服,什么颜色,背景的旗帜,路边观众... ...

Omost layout cond,该节点把转化后的提示词送给AI绘画模型,生成图片。

4、生成图片

给足了提示词信息后,AI绘画大模型输出图片,细节非常丰富。

当然了,这个大模型提示词也并非完美,因为额外加载一个大模型,跑图的速度非常缓慢。

网盘下载:

https://pan.quark.cn/s/39180f253900


53AI,企业落地应用大模型首选服务商

产品:大模型应用平台+智能体定制开发+落地咨询服务

承诺:先做场景POC验证,看到效果再签署服务协议。零风险落地应用大模型,已交付160+中大型企业

联系我们

售前咨询
186 6662 7370
预约演示
185 8882 0121

微信扫码

与创始人交个朋友

回到顶部

 
扫码咨询