Sidebar

Blog


Blog
Timeline


https://horea.caramizaru.xyz


โ† Go Back


Search by Tags




Timeline



MIT Computer Science Class 6.S184: Generative AI with Stochastic Differential Equations


Diffusion and flow-based models have become the state of the art for generative AI across a wide range of data modalities, including images, videos, shapes, molecules, music, and more! This course aims to build up the mathematical framework underlying these models from first principles. At the end of the class, students will have built a toy image diffusion model from scratch, and along the way, will have gained hands-on experience with the mathematical toolbox of stochastic differential equations that is useful in many other fields. This course is ideal for students who want to develop a principled understanding of the theory and practice of generative AI.


Course Notes

The course notes serve as the backbone of the course and provide a self-contained explanation of all material in the class. In contrast, lecture slides will generally not be self-contained and are intended to provide accompanying visualizations during the lecture. You may view the notes by clicking on the colored link below.

The full video course and slides/course materials can be found here:

Course web page




Here is a list of topics that will be covered during this school

  • Movement primitives
  • Pinocchio: kinematics and derivatives in C++ and python
  • Optimal control solvers
  • Contact planner
  • Walking robot navigation and state estimation
  • Motion planning for legged locomotion (tentative title, probably focus on quadrupedal locomotion)
  • Motion planning workflow for the atalante exoskeleton
  • TALOS control system architecture and whole body controller
  • Deep RL
  • QP-based task-space inverse
  • Dynamics: theory and implementation
  • Robust optimization with application to task-space inverse dynamics


Abstract

Robotics has been a momentous field of research and development over the past sixty years, since the computer chip revolution in the 1960s. Realising the promise of the field entails research into motion planning and control of robotic systems in many scenarios: from fixedbase robots (e.g. industrial arms) to floating-base wheeled orlegged robots. For such schemes to be able to deal with more complex robots, they must plan ahead and satisfy physical and operational constraints, and do so, hopefully, in a reactive manner. For many years, they were rather simple, limited to simple robots doing simple things. Greater complexity and capability was unlocked with more sophisticated planners and controllers, that had access to better models of their subservient systems (geometries, inertia, the existence of contact). An adequate framework to reconcile the domain's requirements is that of optimal control. It allows for predictive models of robot behaviour over given time horizons! , and constraint satisfaction, while optimising for a given performance metric. As most optimal control problems cannot be solved in closed-form, we resort to numerical methods. This numerical optimal control has a proven track record for online motion generation and control on legged robots with real-time requirements (albeit mostly while using simplified models of the robot and environment). However, it typically leads to large-scale mathematical optimisation problems with thousands of variables โ€“ a computationally expensive endeavour. Thus, its use in robotics has relied on two axes of progress: faster chips, and efficient, structure-exploiting algorithms (with carefully engineered implementations). This thesis focuses on the latter axis: development of more performant, real-time capable solvers for numerical optimal control, with the objective of โ€œon-the-flyโ€ complex motion generation the for predictive control of sophisticated robots.

2025/06/09 10:53 · Horea Caramizaru · 0 Comments · 0 Linkbacks


Winter School on Numerical Methods for Optimal Control of Nonsmooth Systems

Monday, February 03, 2025, 8:00 - Wednesday, February 05, 2025, 16:00

Mines Paris-PSL, 60 Boulevard Saint-Michel, 75006 Paris

This course is organized by Franรงois Pacaud and Armin Nurkanovic. Lectures will be given by Armin Nurkanovic. This three-day course aims to provide both theoretical background and hands-on practical knowledge in formulating optimal control problems subject to hybrid systems with switches and state jumps. Participants will learn to:

  • Formulate and solve optimal control problems for hybrid systems
  • Master specialized numerical methods for hybrid system optimization
  • Apply these concepts to real-world engineering problems

Lecture slides

  • Theory and algorithms for nonlinear programming - (slides, recording)
  • Direct methods for smooth nonlinear optimal control - (slides, recording)
  • Introduction to nonsmooth differential equations and hybrid systems - (slides, recording)
  • Direct methods for nonsmooth nonlinear optimal control - (slides, recording)
  • MPECs: nonsmooth modelling and optimality conditions - (slides, recording)
  • Numerical methods for mathematical programs with complementarity constraints - (slides, recording)


2025/06/02 00:00 · Horea Caramizaru · 0 Comments · 0 Linkbacks


2025/05/30 21:20 · Horea Caramizaru · 0 Comments · 0 Linkbacks


2025/05/16 11:51 · Horea Caramizaru · 0 Comments · 0 Linkbacks


2025/05/15 11:42 · Horea Caramizaru · 0 Comments · 0 Linkbacks


2025/05/13 11:27 · Horea Caramizaru · 0 Comments · 0 Linkbacks

Older entries >>

feed/start.txt ยท Last modified: 2025/04/24 20:30 by Horea Caramizaru