Stability AI (creators of Stable Diffusion) just released SDXL v1.0, and shortly after released another model called SDXL v1.0 w/ VAE. This confused a lot of users.
We'll help clear the confusion.
First off, VAE is used with your Stable Diffusion model to decode the image created by your model (transform back to pixel space) and can affect the resulting image's color vibrancy and crispness/smoothness of lines. Because VAE's are trained on input images, they are usually paired with a model (so for example you can't use SDXL 1.0 vae with a SD 1.5 model).
In terms of the VAE for SDXL v1.0, the VAE is baked in.
What this means for you
If using Automatic1111: You're all set, just set the model and the vae should run automatically (if you want to, you can customize by changing your vae in the settings screen)
If using ComfyUI: You can pipe the vae directly from your model file (This is the meaning of baked in for SDXL v1.0)
Make sure you download the correct model file (correct models specified below). SDXL v1.0 uses two step process for generations, so make sure to download the correct file for both the base model and the refiner.
For the curious, the SDXL v1.0 Base without VAE is for users who train models using the base model and include their own custom VAE's.
Which Model to Use
For most users, wanting to try out SDXL v1.0, download the models with the VAE included. (Hint: it's the ones with the green checkmark).
More advanced users may customize with their own vae's and so they'll choose the base model without a vae
SDXL v1.0 Base
- ✅ SDXL v1.0 Base w/ vae: https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0_0.9vae.safetensors
- ❌ SDXL v1.0 Base: https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors
SDXL v1.0 Refiner
- ✅ SDXL v1.0 Refiner w/ vae: https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/blob/main/sd_xl_refiner_1.0_0.9vae.safetensors
- ❌ SDXL v1.0 Refiner: https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/blob/main/sd_xl_refiner_1.0.safetensors
Troubleshooting
- For users who downloaded the VAE separately, the VAE that was released with SDXL v1.0 will not work with SD 1.5 (Results will look half finished - abstract)
References
Here it is confirmed by CivitAI:
and MysteryGuitarM a Stability Staff Member:
What's next
Try generating your own images with SDXL v1.0 using our Comprehensive ComfyUI Guide and try some of these Fantastic Prompt Ideas. (Prompt templates included)
About @workingforlatte
Hey there, I'm @workingforlatte, an engineer and solopreneur on a mission to make AI accessible and affordable while indulging in my love for lattes. One day, I hope to treat myself to a luxurious mcol latte, and who knows, maybe even upgrade to an hcol latte (a lofty dream, I know)!
With a deep passion for technology and AI, I bring my engineering expertise to the forefront as I explore the ever-evolving AI landscape. I'm thrilled to be part of this AI blog, sharing my unique insights and unraveling the mysteries of artificial intelligence.
Join me on this exciting journey as we navigate the intricate world of AI together. From tackling complex technical challenges to discovering the endless possibilities of AI, I'm here to inspire and empower fellow tech enthusiasts like you.
Together, let's embark on this adventure of uncovering the wonders of AI and embracing the incredible ways it will shape our future. Get ready to dive into the AI realm alongside me, and let's make AI knowledge accessible and enjoyable for everyone. Cheers to a future filled with AI breakthroughs and, of course, plenty of delicious lattes!