Title image

This sequence of notes introduces the mathematics of diffusion models, starting from the basics and without assuming advanced mathematics. The exercises will build up knowledge, intuition, and mathematical modelling skills to enable building and debugging diffusion models from the ground up, and carrying out diffusion research.

I’ll occasionally post new pages (topics such as distillation and guidance are in the works): follow me on twitter at dwsaxton to receive these updates.

Feedback is welcome.

Brief foreword

There are a few different “formulations” of diffusion models:

  1. Denoising Diffusion Probabilistic Models (DDPMs) [1, 2]Denoising diffusion probabilistic models
    Ho, Jonathan and Jain, Ajay and Abbeel, Pieter
    Advances in neural information processing systems, 2020

    Deep unsupervised learning using nonequilibrium thermodynamics
    Sohl-Dickstein, Jascha and Weiss, Eric and Maheswaranathan, Niru and Ganguli, Surya
    International conference on machine learning, 2015    .
  2. Score-Based Generative Models [3, 4]Generative Modeling by Estimating Gradients of the Data Distribution
    Song, Yang
    https://yang-song.net/blog/2021/score/

    Improved techniques for training score-based generative models
    Song, Yang and Ermon, Stefano
    Advances in neural information processing systems, 2020    .
  3. (Stochastic) Differential Equations [5]Score-based generative modeling through stochastic differential equations
    Song, Yang and Sohl-Dickstein, Jascha and Kingma, Diederik P and Kumar, Abhishek and Ermon, Stefano and Poole, Ben
    arXiv preprint arXiv:2011.13456, 2020    . This is a generalization of 1 and 2.

We mostly follow the third one, and do not rely on stochastic calculus until later. For those curious, survey papers such as [6, 7]Diffusion models: A comprehensive survey of methods and applications
Yang, Ling and Zhang, Zhilong and Song, Yang and Hong, Shenda and Xu, Runsheng and Zhao, Yue and Zhang, Wentao and Cui, Bin and Yang, Ming-Hsuan
ACM Computing Surveys, 2023

Diffusion models in vision: A survey
Croitoru, Florinel-Alin and Hondru, Vlad and Ionescu, Radu Tudor and Shah, Mubarak
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023 give an additional overview of the landscape.

If you would like to cite this tutorial series, you can use this BibTeX snippet:

@misc{saxton2024diffusion,
  author = {Saxton, David},
  title = {Diffusion tutorial},
  url = {http://saxton.ai/diffusion},
  year = {2024}
}