1、stable diffusion git下载代码
https://github.com/CompVis/stable-diffusion
安装环境ldm
2、下载模型
从网站中 CompVis (CompVis) 下载compvis的模型，有如下，下载模型如果有问题，运行中会提示错误。需要重新下载
3、运行
到1的主目录下，运行：其中init-img是自己选的图片，prompt是生成的提示词
python scripts/img2img.py --init-img demoimg/man3.png --n_samples 4 --outdir outputs/img2img/0 --ckpt models/ldm/stable-diffusion-v1-4/sd-v1-4.ckpt --prompt “a men in black, tied hair and waist, tall and fit, deep eyes and gnarled beard”
4、bug1:
img2img.py 53: w, h = image.size
print(f"loaded input image of size ({w}, {h}) from {path}") # 158*162
最终resize到128*160，为32的倍数，然而有bug，size 问题，提示如下。
解决：w, h = map(lambda x: x - x % 64, (w, h))
改成64的倍数即可——这里是因为unet里有3次downsample，即unet的输入需要是8的倍数。若32则仅保证4倍
Traceback (most recent call last):
File "/cfs/cfs_dataV/workdir/Research_stage2/stable-diffusion/scripts/img2img.py", line 308, in <module>
main()
File "/cfs/cfs_dataV/workdir/Research_stage2/stable-diffusion/scripts/img2img.py", line 271, in main
samples = sampler.decode(z_enc, c, t_enc, unconditional_guidance_scale=opt.scale,
File "/data/miniconda3/envs/env-novelai/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/cfs/cfs_dataV/workdir/Research_stage2/stable-diffusion/ldm/models/diffusion/ddim.py", line 238, in decode
x_dec, _ = self.p_sample_ddim(x_dec, cond, ts, index=index, use_original_steps=use_original_steps,
File "/data/miniconda3/envs/env-novelai/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/cfs/cfs_dataV/workdir/Research_stage2/stable-diffusion/ldm/models/diffusion/ddim.py", line 177, in p_sample_ddim
e_t_uncond, e_t = self.model.apply_model(x_in, t_in, c_in).chunk(2)
File "/cfs/cfs_dataV/workdir/Research_stage2/stable-diffusion/ldm/models/diffusion/ddpm.py", line 987, in apply_model
x_recon = self.model(x_noisy, t, **cond)
File "/data/miniconda3/envs/env-novelai/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/cfs/cfs_dataV/workdir/Research_stage2/stable-diffusion/ldm/models/diffusion/ddpm.py", line 1410, in forward
out = self.diffusion_model(x, t, context=cc)
File "/data/miniconda3/envs/env-novelai/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/cfs/cfs_dataV/workdir/Research_stage2/stable-diffusion/ldm/modules/diffusionmodules/openaimodel.py", line 751, in forward
h = th.cat([h, hs.pop()], dim=1)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 3 but got size 5 for tensor number 1 in the list.