Stable Diffusion 3 medium #SD3
September 6, 2024 in Technology
Stable Diffusion 3 medium #SD3
- It’s a good model with a blend of speed & performance
- It was iteratively trained by Robin’s team & rest of Stability AI team to blend wide use but also be good out of the box
- It’s clear some of the safety alignment stuff got wonky at the last stage, we’ve seen this with DALL-E, Google models etc
- In particular it doesn’t like folk laying on grass. The safety stuff is needed due to regulatory obligations & more but is an art versus a science. Stability AI models also get way more use than any others so obligation is heavier – you may not care if models are used in bad ways but I can tell you it gave me sleepless nights.
- Unlike DALL-E or Imagen etc the model weights are available and while being great for the vast majority of stuff can be adjusted to fix the issues as well as become even better.
- Model perturbation, ELLA, MoE’ing, prompt augmentation, SPIN’ing & others are likely to have good results
- This will also emphasise how SD3 will fit nicely in pipelines, just like the ultra API is a pipeline like Midjourney, dall-e, ideogram and other image “models”
- The new license changes seem a bit confusing but from responses seem fine for creators as they basically cover inference services. Do give feedback.
- It’s nice there are optimised versions for various hardware. Tuning will take some time to get right as it’s a bit difference, but I think we will see loads more leg work and impact with loras and ip adapters etc due to the quality of the base model, Vae upgrade etc
