MIT Computer Science Class 6.S184: Generative AI with Stochastic Differential Equations
Diffusion and flow-based models have become the state of the art for generative AI across a wide range of data modalities, including images, videos, shapes, molecules, music, and more! This course aims to build up the mathematical framework underlying these models from first principles. At the end of the class, students will have built a toy image diffusion model from scratch, and along the way, will have gained hands-on experience with the mathematical toolbox of stochastic differential equations that is useful in many other fields. This course is ideal for students who want to develop a principled understanding of the theory and practice of generative AI.
Course Notes
The course notes serve as the backbone of the course and provide a self-contained explanation of all material in the class. In contrast, lecture slides will generally not be self-contained and are intended to provide accompanying visualizations during the lecture. You may view the notes by clicking on the colored link below.
The full video course and slides/course materials can be found here:
Here is a list of topics that will be covered during this school
Abstract
Robotics has been a momentous field of research and development over the past sixty years, since the computer chip revolution in the 1960s. Realising the promise of the field entails research into motion planning and control of robotic systems in many scenarios: from fixedbase robots (e.g. industrial arms) to floating-base wheeled orlegged robots. For such schemes to be able to deal with more complex robots, they must plan ahead and satisfy physical and operational constraints, and do so, hopefully, in a reactive manner. For many years, they were rather simple, limited to simple robots doing simple things. Greater complexity and capability was unlocked with more sophisticated planners and controllers, that had access to better models of their subservient systems (geometries, inertia, the existence of contact). An adequate framework to reconcile the domain's requirements is that of optimal control. It allows for predictive models of robot behaviour over given time horizons! , and constraint satisfaction, while optimising for a given performance metric. As most optimal control problems cannot be solved in closed-form, we resort to numerical methods. This numerical optimal control has a proven track record for online motion generation and control on legged robots with real-time requirements (albeit mostly while using simplified models of the robot and environment). However, it typically leads to large-scale mathematical optimisation problems with thousands of variables โ a computationally expensive endeavour. Thus, its use in robotics has relied on two axes of progress: faster chips, and efficient, structure-exploiting algorithms (with carefully engineered implementations). This thesis focuses on the latter axis: development of more performant, real-time capable solvers for numerical optimal control, with the objective of โon-the-flyโ complex motion generation the for predictive control of sophisticated robots.
Winter School on Numerical Methods for Optimal Control of Nonsmooth Systems
Monday, February 03, 2025, 8:00 - Wednesday, February 05, 2025, 16:00
Mines Paris-PSL, 60 Boulevard Saint-Michel, 75006 Paris
This course is organized by Franรงois Pacaud and Armin Nurkanovic. Lectures will be given by Armin Nurkanovic. This three-day course aims to provide both theoretical background and hands-on practical knowledge in formulating optimal control problems subject to hybrid systems with switches and state jumps. Participants will learn to:
Lecture slides