基于 instantX-research/diffusers_sd3_control: 🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX. (github.com)
和
ZHO-ZHO-ZHO/ComfyUI-SD3-Medium-CN-Diffusers: ComfyUI SD3-Medium ControlNet（Diffusers） (github.com)
改的，这两个repo都有些问题，中间看提交记录，有好些人在参与改动，为了能跑的了，我分别fork了这两个分支， 在这里：
prodogape/ComfyUI-SD3-Medium-CN-Diffusers: ComfyUI SD3-Medium ControlNet（Diffusers） (github.com)
prodogape/diffusers_sd3_control: refactor name from "diffusers" to "diffusers_sd3_control" (github.com)
1. 先说怎么装
打开第一个橘色背景的repo：
prodogape/ComfyUI-SD3-Medium-CN-Diffusers: ComfyUI SD3-Medium ControlNet（Diffusers） (github.com)
按照Install这里的步骤安装
时间有限只写了手动安装的步骤，分4步：
（1）git 克隆 comfyui的节点
（2）下载hf的两个仓库的模型（一个controlnet一个sd3）
（3）修改sd3模型文件夹下的文件model_index.json，因为要安装的特殊版本的diffusers我改名为diffusers_sd3_control了
（4）安装diffusers_sd3_control
2. 工作流及效果
这个版本的实现应该还是有些问题的，可能跟control的百分比和control了不太对的部分有关
3. 上图的工作流（非API版本工作流）
{
"last_node_id": 82,
"last_link_id": 139,
"nodes": [
{
"id": 77,
"type": "CannyEdgePreprocessor",
"pos": [
-1756,
-33
],
"size": {
"0": 315,
"1": 106
},
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 133
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
134,
139
],
"shape": 3,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "CannyEdgePreprocessor"
},
"widgets_values": [
100,
200,
512
]
},
{
"id": 82,
"type": "PreviewImage",
"pos": [
-1753,
110
],
"size": {
"0": 299.701171875,
"1": 326.380859375
},
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 139
}
],
"properties": {
"Node name for S&R": "PreviewImage"
}
},
{
"id": 80,
"type": "PreviewImage",
"pos": [
-970,
-40
],
"size": {
"0": 438.1802978515625,
"1": 471.4967041015625
},
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 137
}
],
"properties": {
"Node name for S&R": "PreviewImage"
}
},
{
"id": 74,
"type": "LoadImage",
"pos": [
-2113,
-24
],
"size": {
"0": 349.269775390625,
"1": 459.3564147949219
},
"flags": {},
"order": 0,
"mode": 0,
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
133
],
"shape": 3,
"slot_index": 0
},
{
"name": "MASK",
"type": "MASK",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"IMG_20240617_070212_131.jpeg",
"image"
]
},
{
"id": 76,
"type": "SD3MCN_BaseModelLoader_Zho",
"pos": [
-1424,
-42
],
"size": {
"0": 428.3705749511719,
"1": 95.87933349609375
},
"flags": {},
"order": 1,
"mode": 0,
"outputs": [
{
"name": "pipe",
"type": "SD3MMODEL",
"links": [
132
],
"shape": 3,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "SD3MCN_BaseModelLoader_Zho"
},
"widgets_values": [
"stabilityai/stable-diffusion-3-medium-diffusers",
"InstantX/SD3-Controlnet-Canny"
]
},
{
"id": 78,
"type": "SD3MCN_Generation_Zho",
"pos": [
-1436,
94
],
"size": {
"0": 444.2547607421875,
"1": 344.82293701171875
},
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "pipe",
"type": "SD3MMODEL",
"link": 132
},
{
"name": "image",
"type": "IMAGE",
"link": 134
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
137
],
"shape": 3,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "SD3MCN_Generation_Zho"
},
"widgets_values": [
"close-up of a young girl, masterpiece, best quality, photo, realistic, very aesthetic",
"worst quality, low quality, toon",
512,
512,
40,
9,
1,
542410872807786,
"randomize"
]
}
],
"links": [
[
132,
76,
0,
78,
0,
"SD3MMODEL"
],
[
133,
74,
0,
77,
0,
"IMAGE"
],
[
134,
77,
0,
78,
1,
"IMAGE"
],
[
137,
78,
0,
80,
0,
"IMAGE"
],
[
139,
77,
0,
82,
0,
"IMAGE"
]
],
"groups": [],
"config": {},
"extra": {
"workspace_info": {
"id": "-gBBQ2xIYibjLVAhj5iML"
},
"ds": {
"scale": 1.3513057093104035,
"offset": {
"0": 2165.8774539604974,
"1": 287.4375476041587
}
}
},
"version": 0.4
}