Sidebar

Blog


Blog


https://horea.caramizaru.xyz


โ† Go Back


Search by Tags



You are not allowed to perform this action


gradSim: Differentiable simulation for system identification and visuomotor control


"โˆ‡Sim: DIFFERENTIABLE SIMULATION FOR SYSTEM IDENTIFICATION AND VISUOMOTOR CONTROL" by Krishna Murthy Jatavallabhula et. all

Code

Abstract:

We consider the problem of estimating an objectโ€™s physical properties such as mass, friction, and elasticity directly from video sequences. Such a system identification problem is fundamentally ill-posed due to the loss of information during image formation. Current solutions require precise 3D labels which are labor-intensive to gather, and infeasible to create for many systems such as deformable solids or cloth. We present gradSim, a framework that overcomes the dependence on 3D supervision by leveraging differentiable multiphysics simulation and differentiable rendering to jointly model the evolution of scene dynamics and image formation. This novel combination enables backpropagation from pixels in a video sequence through to the underlying physical attributes that generated them. Moreover, our unified computation graphโ€‰โ€”โ€‰spanning from the dynamics and through the rendering processโ€‰โ€”โ€‰enables learning in challenging visuomotor control tasks, without relying on state-based (3D) supervision, while obtaining performance competitive to or better than techniques that rely on precise 3D labels.

Discussion

Enter your comment. Wiki syntax is allowed:
 
feed/2023/11/02/gradsim_differentiable_simulation_for_system_identification_and_visuomotor_control.txt ยท Last modified: 2023/11/12 19:04 by Horea Caramizaru