Datasets / Physical AI

UMI Dual-Camera Annotated Manipulation Dataset

500 hours · bimanual demonstrations

Train robot manipulation policies from bimanual UMI demonstrations that pair left and right gripper views with pose, gripper width, task structure, and outcomes.

Dataset specification

  • Dual wrist-mounted GoPro video with embedded accelerometer and gyroscope
  • RGB observations, 6-DoF end-effector pose, and gripper width
  • Aligned observations and telemetry at 10–20 Hz
  • Bimanual clock and frame matching within 1/60 second
  • Zarr episodes with task, object, success, and subtask labels

Intended uses

  • Robot manipulation learning from gripper demonstrations
  • Bimanual coordination and action prediction
  • Task-conditioned policy training

Scope notes

  • The selected tasks, demonstration subset, and delivery scope are confirmed during review.
  • Target robot action conventions and training requirements are agreed for the intended use.

Collection scope

Demonstrations

  • Bimanual tasks performed by consented operators using instrumented grippers
  • Left-hand, right-hand, and coordinated bimanual execution

Capture

  • Dual wrist-mounted GoPro video
  • Embedded accelerometer and gyroscope telemetry

Catalog volume

  • 500 hours
  • Task mix and the selected delivery subset confirmed during review

Delivery

  • Zarr episodes with RGB, 6-DoF end-effector pose, and gripper width aligned at 10–20 Hz
  • Episode boundaries, subtask actions, and success labels

Annotation & metadata fields

  • Task and object labels
  • Hand and manipulation mode
  • Left-hand, right-hand, and bimanual execution
  • Success and episode boundaries
  • Subtask action labels

Capture methodology

  • Consented operators perform manipulation tasks with instrumented grippers
  • Left and right camera views are paired with gripper telemetry
  • Clock and frame matching within 1/60 second
  • RGB observations, end-effector pose, and gripper width aligned at 10–20 Hz

Provenance & rights chain

  • Recorded by consented participants or operators
  • Licensing and the selected delivery scope are reviewed with Datoric

Quality, duplicates & PII

How submissions are reviewed and cleaned before they are accepted into the dataset.

  • SLAM and calibration review
  • Dropped-episode checks
  • Trajectory integrity and bimanual alignment checks

Formats & delivery

  • Dual GoPro MP4 video with embedded inertial telemetry
  • Zarr episodes with aligned observations, poses, and gripper width
  • Episode and subtask annotations

Rights & license scope

Licensed by Datoric for robotics training, with final scope governed by the agreement for the selected delivery.

Frequently asked

What is included in a UMI episode?
Left and right gripper views are paired with RGB observations, 6-DoF end-effector pose, gripper width, task and object labels, success, and episode boundaries.
How are the two hands aligned?
The catalog specifies bimanual clock and frame matching within 1/60 second, with observations and telemetry aligned at 10–20 Hz in Zarr.
Can we review a task-specific subset?
Request a review package and share your task mix, target embodiment, and training schema so the proposed subset can be scoped for your pipeline.

Related dataset specifications