𝐶𝑎𝑟𝑎𝑚𝑖𝑧𝑎𝑟𝑢'𝑠 𝐵𝑙𝑜𝑔 feed:2023:11:02

Differentiable Simulation Methods for Robotic Agent Design

anonymous@undisclosed.example.com (Anonymous) — Sun, 24 Mar 2024 17:38:52 +0000

Differentiable Simulation Methods for Robotic Agent Design

https://people.iiis.tsinghua.edu.cn/~taodu/

Thesis: "Differentiable Simulation Methods for Robotic Agent Design" by Tao Du

Abstract:

Designing robots with extreme performance in a given task has long been an exciting research problem drawing attention from researchers in robotics, graphics, and artificial intelligence. As a robot is a combination of its hardware and software, an optimal robot requires both an excellent implementation of its hardware (e.g., morphological, topological, and geometrical designs) and an outstanding design of its software (e.g., perception, planning, and control algorithms). While we have seen promising breakthroughs for automating a robot's software design with the surge of deep learning in the past decade, exploration of optimal hardware design is much less automated and is still mainly driven by human experts, a process that is both labor-intensive and error-prone. Furthermore, experts typically optimize a robot's hardware and software separately, which may miss optimal designs that can only be revealed by optimizing its hardware and software simultaneously. This thesis argues that it is time to rethink robot design as a holistic process where a robot's body and brain should be co-optimized jointly and automatically. In this thesis, we present a computational robot design pipeline with differentiable simulation as a key player. We first introduce the concept of computational robot design on a real-world copter whose geometry and controller are co-optimized with a differentiable simulator, resulting in a custom copter that outperforms designs suggested by human experts by a substantial margin. Next, we push the boundary of differentiable simulation by developing advanced differentiable simulators for soft-body and fluid dynamics. Contrary to traditional belief, we show that deriving gradients for such intricate, high-dimensional physics systems can be both science and art. Finally, we discuss challenges in transferring computational designs discovered in simulation to real-world hardware platforms. We present a solution to this simulation-to-reality transfer problem using our differentiable simulator on an example of modeling and controlling a real-world soft underwater robot. We conclude this thesis by discussing open research directions in differentiable simulation and envisioning a fully automated computational design pipeline for real-world robots in the future.

Do Differentiable Simulators Give Better Policy Gradients?

anonymous@undisclosed.example.com (Anonymous) — Sun, 12 Nov 2023 19:04:33 +0000

Do Differentiable Simulators Give Better Policy Gradients?

"Do Differentiable Simulators Give Better Policy Gradients?" by H.J. Terry Suh, Max Simchowitz, Kaiqing Zhang, Russ Tedrake

Abstract:

Differentiable simulators promise faster computa- tion time for reinforcement learning by replacing zeroth-order gradient estimates of a stochastic objective with an estimate based on first-order gradients. However, it is yet unclear what fac- tors decide the performance of the two estimators on complex landscapes that involve long-horizon planning and control on physical systems, despite the crucial relevance of this question for the util- ity of differentiable simulators. We show that characteristics of certain physical systems, such as stiffness or discontinuities, may compromise the efficacy of the first-order estimator, and ana- lyze this phenomenon through the lens of bias and variance. We additionally propose an α-order gra- dient estimator, with α ∈ [0, 1], which correctly utilizes exact gradients to combine the efficiency of first-order estimates with the robustness of zero- order methods. We demonstrate the pitfalls of traditional estimators and the advantages of the α-order estimator on some numerical examples.

robotics, differentiable-simulators, reinforcement-learning

gradSim: Differentiable simulation for system identification and visuomotor control

anonymous@undisclosed.example.com (Anonymous) — Sun, 12 Nov 2023 19:04:33 +0000

gradSim: Differentiable simulation for system identification and visuomotor control

"∇Sim: DIFFERENTIABLE SIMULATION FOR SYSTEM IDENTIFICATION AND VISUOMOTOR CONTROL" by Krishna Murthy Jatavallabhula et. all

Code

Abstract:

We consider the problem of estimating an object’s physical properties such as mass, friction, and elasticity directly from video sequences. Such a system identification problem is fundamentally ill-posed due to the loss of information during image formation. Current solutions require precise 3D labels which are labor-intensive to gather, and infeasible to create for many systems such as deformable solids or cloth. We present gradSim, a framework that overcomes the dependence on 3D supervision by leveraging differentiable multiphysics simulation and differentiable rendering to jointly model the evolution of scene dynamics and image formation. This novel combination enables backpropagation from pixels in a video sequence through to the underlying physical attributes that generated them. Moreover, our unified computation graph — spanning from the dynamics and through the rendering process — enables learning in challenging visuomotor control tasks, without relying on state-based (3D) supervision, while obtaining performance competitive to or better than techniques that rely on precise 3D labels.

differentiable-simulation, system-identification, gradSim, multiphysics, differentiable-rendering

Introduction to Algorithmic Differentiation (AD I)

anonymous@undisclosed.example.com (Anonymous) — Sat, 27 Apr 2024 11:52:28 +0000

Introduction to Algorithmic Differentiation (AD I)

Course AD

The Art of Differentiating Computer Programs

anonymous@undisclosed.example.com (Anonymous) — Sat, 27 Apr 2024 11:52:51 +0000

The Art of Differentiating Computer Programs

ad, algorithmic-differentiation, automatic-differentiation, book