KIWI

Kinematic Interface for the Wild

Off the shelf. Into the wild. Bimanual robot data collection with two consumer 360° cameras.

Benjamin C. Yang^*, Weiying Wang^*, Shenggao Li, Keming Yan, Sasha Wilkinson, Zelin Wang, Yip Fun Yeung, Lingfeng Sun^†

^* Equal contribution ^† Project lead

Paper Code (coming soon)

Watch KIWI 2:42

Capture

Reconstruct

The result

Two hands. One shared world.

Reconstructed from a single demo recorded with two Insta360 X5 cameras. No dedicated mapping pass. Even the ceiling is sharp: the cameras see in 360°.

Static scene, for now. Dynamic real-to-sim is becoming practical as foundation models improve. Contributions welcome; updates to come.

0.1%

of query frames fail to localize across six bimanual recordings.

Front lens alone: 24.8%

4.5mm

median localization error against evaluation fiducials.

2× Insta360 X5

Four lenses and one demo reconstruct the whole scene.

Millisecondsync accuracy

Audio-based synchronization keeps timing error across all cameras within a few milliseconds.

The hardware

One camera. Interchangeable tools.

A shared quick-release connects the camera module to chopstick grippers, parallel jaws, or a wrist cuff, with corresponding robot-side mounts.

A closer look at the hardware.

Compliant chopstick fingertips and a shared mounting interface for Franka, YAM, and OpenArm.

Built around the human grasp.

Passive, finger-driven jaws preserve mechanical feedback. An angled grip gives the camera clearance, while bearing-supported pivots guide the opening motion.

Handheld

Parallel jaw Soon

Camera module strapped to the wrist of a bare hand

Wrist mount Soon

Robot mounted

Parallel-jaw gripper and camera module on a robot flange

Parallel jaw Soon

Camera module on the wrist of a robot hand

Robot hand Soon

How it works

Carry less. Capture more.

01 / CAPTURE

Record the task.

Record both hands with dual-lens wrist cameras, with an optional head camera for an egocentric view.

02 / RECONSTRUCT

Recover the motion.

Align both hands in a shared scene using visual maps, synchronized audio, and camera IMU measurements.

03 / EXPORT

Prepare for learning.

Export synchronized video, tool-tip poses, and gripper opening alongside a reconstructed 3D Gaussian scene.

Offline pipeline: front and rear videos build aligned visual maps, per-arm visual and inertial estimation recovers motion, and shared-frame outputs include a 3D Gaussian scene and demonstration dataset. — Build the scene maps once, then reuse them to reconstruct each episode. Swipe the diagram to explore.

Citation

Cite KIWI.

@misc{kiwi2026,
  title         = {Kinematic Interface for the Wild: Modular Bimanual Loco-Manipulation Capture from 360$^{\circ}$ Cameras Alone},
  author        = {Yang, Benjamin C. and Wang, Weiying and Li, Shenggao and Yan, Keming and
                   Wilkinson, Sasha and Wang, Zelin and Yeung, Yip Fun and Sun, Lingfeng},
  year          = {2026},
  eprint        = {2609.22809},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2609.22809}
}

Contact: lingfengsun1996@gmail.com