大纲

In this notebook, we're going to cover two main approaches for adapting existing diffusion models:

With fine-tuning, we'll re-train existing models on new data to change the type of output they produce
With guidance, we'll take an existing model and steer the generation process at inference time for additional control

By the end of this notebook, you will know how to:

Create a sampling loop and generate samples faster using a new scheduler
Fine-tune an existing diffusion model on new data, including:
- Using gradient accumulation to get around some of the issues with small batches
- Logging samples to Weights and Biases during training to monitor progress (via the accompanying example script)
- Saving the resulting pipeline and uploading it to the hub
Guide the sampling process with additional loss functions to add control over existing models, including:
- Exploring different guidance approaches with a simple color-based loss
- Using CLIP to guide generation using a text prompt
- Sharing a custom sampling loop using Gradio and 🤗 Spaces

步骤