Use Cases

RoboX data supports a range of robotics and AI applications. Each use case draws from one or more of the five active campaigns, using first-person video and sensor data recorded by contributors in real environments.

1. Activity and Scene Campaigns: EgoNav, EgoDaily

Autonomous systems need to understand how humans move through and interact with real environments: cluttered spaces, changing conditions, dynamic surroundings. Traditional data comes from fixed sensors or controlled lab setups. Neither captures how humans actually perceive and respond to the physical world in real time.

RoboX egocentric data captures:

• Movement patterns through complex, unstructured spaces

• Obstacle awareness around pedestrians and unexpected barriers

• Environmental adaptation across varying conditions

• Attention patterns that reveal what signals people use to make decisions in real time

2. Perception Campaigns: All

Robots need to perceive and understand their environment: identify objects, assess surfaces, recognize situations, and predict what happens next. Most perception training data comes from static images or fixed-position video, which doesn't match the dynamic first-person view a robot experiences during operation.

RoboX egocentric data captures:

• Objects as they appear during approach with changing scale, angle, and occlusion

• Surface characteristics from a walking perspective

• Dynamic scene changes like people moving and lighting shifts

• Contextual relationships showing how spaces are organized

3. Manipulation Campaigns: EgoGrasp

Teaching robots to grasp, use, and interact with objects requires data that shows the full context of how humans handle things: the approach, the grip, the adjustment, the release. Lab-based demonstrations provide some of this, but in controlled settings that don't reflect the variety of real-world objects and environments.

RoboX egocentric data captures:

• Close-range hand-object interaction across 200+ object categories

• 8+ domains including kitchen, medical, office, household, and tools

• Natural, unscripted manipulation in real environments

• The variety models need to generalize beyond lab conditions

4. Humanoid Robotics Campaigns: EgoDaily, EgoGrasp

Humanoid robots aim to operate in human environments and perform human-like tasks. This requires understanding not just what humans do, but how: the subtle movement patterns, balance adjustments, and behavioral rhythms of everyday activity. Lab demonstrations and teleoperation provide some training data, but neither reflects natural human behavior.

RoboX egocentric data captures:

• Movement patterns: walking, turning, stopping, navigating obstacles

• Manipulation context: how objects are approached, grasped, and used

• Behavioral rhythms: pacing, pauses, attention shifts

• Environmental interaction with doors, furniture, tools, and everyday objects

5. Spatial Mapping Campaigns: EgoScene

Robotic systems need to interpret 3D space, detect surfaces, and build accurate representations of physical environments. Spatial data from diverse real-world settings delivers what controlled lab environments cannot.

RoboX egocentric data captures:

• Room-scale 3D geometry from diverse indoor environments

• Real-world lighting conditions and variations

• Surface characteristics and materials

• Spatial relationships across different room types and layouts

6. Indoor Perception Campaigns: EgoNav

Robots operating indoors need to understand how humans move through enclosed spaces, respond to obstacles, and orient themselves without GPS. Egocentric data captured across diverse indoor environments provides what simulated floorplans cannot.

RoboX egocentric data captures:

• First-person movement through indoor environments

• How people approach doorways, corridors, and tight spaces

• Obstacle awareness in cluttered or dynamic interiors

• Orientation and wayfinding patterns in unfamiliar buildings

7. Research & Academia Campaigns: All

RoboX is designed to support academic research across computer vision, robot learning, navigation, human activity recognition, spatial computing, and sensor fusion. As the contributor network grows, researchers will be able to access diverse, well-documented egocentric datasets collected across varied environments and conditions.