Physics-based simulations have accelerated progress in robot learning for driving, manipulation, and locomotion. Yet, a fast, accurate, and robust surgical simulation environment remains a challenge. In this paper, we present ORBIT-Surgical, a physics-based surgical robot simulation framework with photorealistic rendering in NVIDIA Omniverse. We provide 14 benchmark surgical tasks for the da Vinci Research Kit (dVRK) and Smart Tissue Autonomous Robot (STAR) which represent common subtasks in surgical training. ORBIT-Surgical leverages GPU parallelization to train reinforcement learning and imitation learning algorithms to facilitate study of robot learning to augment human surgical skills. ORBIT-Surgical also facilitates realistic synthetic data generation for active perception tasks. We demonstrate ORBIT-Surgical sim-to-real transfer of learned policies onto a physical dVRK robot.
ORBIT-Surgical supports various RL frameworks such as RSL-RL and RL-Games to train RL policies for surgical tasks involving rigid and soft objects.
RL Policy for Shunt Insertion
Long-horizon tasks are computationally expensive and can be hard to be solved by plain Imitation Learning (IL) algorithms. Instead, multi-stage IL policy can be learned efficiently, where long-horizon tasks are divided into subtasks, and an IL policies are trained to perform each subtask separataely. All subtasks are then chained for end-to-end policy execution. Here, we divide a "Needle Pick and Transfer" task into three stages, i.e. Pick, Handover, and Reach, and trained subpolicies to perform each subtask using behavior cloning. Figure below shows a successful end-to-end policy execution in simulation.
DualArm - Needle Pick and Transfer task with multi-stage imitation learning. a) starting state of both PSMs, b) right PSM moves down to pick up the needle, c) needle handoff occurs between the PSMs, d) PSMs move away from each other, e) needle is fully transferred to other arm.
We list, categorize and conduct a sensitivity analysis on the simulation parameters in the NVIDIA Isaac Sim. We focus on the representative environment, 'Threaded Needle Pass Ring', where the task is to pass a suture needle with a soft thread through a ring. This environment is selected because it involves extensive interactions between objects, including interactions between deformable objects and rigid objects. To determine the impact of individual parameters, we held all others constant, varied one specific parameter at a time, and noted results.
Q: Is ORBIT-Surgical fully free and open-source?
A: The underlying robotics simulation application, NVIDIA Isaac Sim is free for individual use. ORBIT-Surgical will be released as a free and open-source package upon publication.
Q: What are the differences between ORBIT-Surgical and Intuitive Surgical's proprietary SimNow?
A: SimNow is a commercial platform for surgeons to practice the skills and techniques needed to perform surgical procedures. It provides a simulated environment that can give the surgeon a realistic experience; ORBIT-Surgical is intended as a benchmark to spur research on supervised autonomy for robot-assisted surgery. We aim to provide simulated environments with a large number of parallel environments for efficient data collection and fast surgical robot learning, as well as accurate physics for sim-to-real transfer of learned policies. In addition, the differences also lie in the way how the simulators can be utilized. Specifically:
Q: How much time is required to train the RL policies?
A: Both tasks involving "Suture Needle Lift" task and "Shunt Insertion" task can be trained under 2 hours on a single NVIDIA RTX 3090 GPU.
|
For any questions, please feel free to contact Masoud Moghani and Animesh Garg.
