Vision, motion, touch, gaze, location and sound are captured together and lined up to the millisecond.
Vision
Egocentric, 360°, stereo depth and LiDAR
Body
Full-body motion and pressure-sensing insoles
Hands
Kinematics, tactile feedback and grip force
Intent
Gaze and face pose, which show where someone is about to go
Location
GNSS and LiDAR SLAM, accurate even in crowds
Sound
Spatial microphone array and speech channel
Egocentric video
First-person video from real sidewalk routes in New York City, in snow, rain, crowds and at night.
Synced to one master clock
Dashboard
Each recording lands in our dashboard, where every stream plays back in sync, both during and after recording.
Processed
Processing
Every session goes from raw sensor streams to clean, labeled data. The models that process it are ones we train ourselves, on our own real-world recordings.
Cleaned in ROS 2
Published as ROS 2 topics and bags (rosbag2 / MCAP), sync-verified, denoised and calibrated.
Our own models
Trained in-house on our recordings, then run across every stream.
Traversability
Our algorithm maps which ground a robot can safely move across.
Annotated
Boxes, segments, events and intent labels layered onto every stream.
Packaged as episodes
Dataset
Robot teams license our recordings as annotated datasets, delivered in ROS 2.
