Safe Sentinel · 2026
UMI → Sim
Demonstrations captured with handheld UMI-style grippers (no robot in the loop), then retargeted onto a two-arm system and replayed in simulation against the recorded footage, frame for frame.
Why handheld capture
Teleoperating a bimanual arm to collect data is slow, expensive, and bounded by how many robots you own. A handheld gripper carrying the same camera and the same jaw geometry as the end effector collects demonstrations anywhere, at roughly the speed of doing the task.
The trade is a retargeting problem: the human wrist moves through poses the arm cannot reach, and the recorded trajectory has to be reconciled with the robot's kinematics and joint limits before it means anything.
09Divergence is the error signal
Every episode replays in simulation next to the original video. Left panel: the retargeted arms executing the trajectory. Right panel: the human demonstration that produced it.
Where the panels diverge, something is wrong: bad retargeting, unreachable poses, gripper-width mismatch. All of it visible without running anything on hardware.
Gripper hardware
Parallel-jaw demonstrations run on Franka-class arms with a custom mount carrying the wrist camera in the same relative pose as the handheld rig. Identical camera extrinsics between capture and deployment is what makes the visual policy transfer at all.
Gallery
Yonder Robotics