Action Agent

The AI agent for 
robot video data. 

Tell the Action Agent what your episodes need. It plans the steps, picks the models, asks before it runs, and hands back labeled and multiplied data for training robots.

Open the Action Agent
Action Agent

Source
Use it in the Studio or fromClaudeCursorWindsurfAny MCP client
Find every clip where the gripper drops somethingLabel hands, objects and depth on every frameMake 50 cluttered versions of each episodeRelight these kitchen picks into 12 scenes, day to nightRetarget my demos to a humanoid hand
Segment every part in the assembly episodesReplay this episode in MuJoCoTrack the cup through every occlusionSwap the table for a warehouse benchScore my policy on last week’s runs

You describe it. The agent plans it. You approve it. It runs.

From a sentence to a finished run

  1. 01

    Describe

    Attach your episodes or pick a sample, then say what you need in plain words: label, segment, relight, retarget, find.

  2. 02

    Approve

    The agent picks the action and model and shows you exactly what will run. Nothing starts until you approve it.

  3. 03

    Watch it run

    Steps and progress stream in live. Results land next to your source, ready to inspect, compare and export.

One capture

Ask for more data.

One agent for every kind of robot footage

Hand pose and depth on every kitchen frame

“Label hands and depth on every frame of my kitchen picks”

⇆
Hand poseDepth map

Segment every part in assembly episodes

“Segment every part and both hands in the assembly episodes”

⇆
Hand poseSegmentation

Turn one household demo into new scenes

“Make cluttered versions of each household episode”

⇆
Your captureNew scene

Depth and labels for first-person game footage

“Add depth to all of my gameplay runs”

⇆
Your captureDepth map

Find and label what matters in aerial footage

“Segment the vehicles in every drone flight”

⇆
Your captureSegmentation

Every action, one conversation

The agent chooses from the same actions you can run by hand in the Studio, and picks a model for each one. Ask for one, or chain several over the same capture.

Questions

What is the Action Agent?

An AI agent for robot and Physical AI video data. You describe the job in plain words, and it plans the steps, chooses the action and model, and runs it on your episodes once you approve.

What can it do with my videos?

Anything you can run in the World Actions Studio: segmentation, depth maps, hand, body and face landmarks, gesture detection, cleanup, search, analysis, diffusion for new scenes, and image to simulator.

Does it run anything without asking?

No. It can read your datasets and check models on its own, but every run waits for you to approve it, and you can change the model before you do.

Is it free?

Runs are on us for now. Attach your own episodes or start with a sample dataset at no cost.

Can I use it from Claude or Cursor?

Yes. Add the World Actions MCP server to Claude, Cursor, Windsurf or any MCP client and ask it to run actions on your datasets from there.

Can I export the results?

Yes. Results stay next to your source in the Studio, and you can export datasets to LeRobot, RLDS, MCAP, ROS 2 bags or HDF5.

Hand it your next dataset.

Start with a sample or your own episodes. Runs are on us for now.