October 9, 2026
Build a robot simulation with an AI agent: NVIDIA shares a recipe for Codex
NVIDIA walked through the path from a robot blueprint to grasping cubes in simulation: the agent handles scene assembly and physics setup.
In NVIDIA's Oct 8 example, the ABB YuMi robot's arms completed 4 cube transfer cycles in 122.2 seconds of physics simulation.
The developer needs to assemble the scene objects, connect physics and rendering, then check the robot's behavior. NVIDIA described how to assign this work to an AI agent in Isaac Sim, a robotics simulation environment. Codex works with the simulator's files and tools here, much as other vibe coding tools work with an application.
Recipes for the agent. NVIDIA Agent Skills are installed with `npx skills@latest add nvidia/skills`. The installer prompts you to choose a skill and an agent. A skill contains instructions for a specific task, such as importing a CAD model into a scene.
From blueprint to motion. Reproducing the example requires Isaac Sim 6.1. Place the STEP file, specifications and robot images in its `reference` folder, then launch Codex CLI from the Isaac Sim folder. The first prompt asks the agent to enable windowed mode with a remote Python server, which it will use to control the simulator.
NVIDIA prepared prompts for five stages:
- Importing STEP via the CAD-to-SimReady skill. - Setting up appearance. - Setting up physics. - Checking that the model is ready for simulation. - Grasping and transferring objects.
In the ABB YuMi example, cubes measuring 45 mm and weighing 40 g were held through contact and friction. The agent configured the objects' physical interactions, as well as their appearance.
Ashley Reed used the Astra and Claude Fable 5 agents to create two digital twins and improve two more in about 3 days. The agents edited OpenUSD, a format for describing 3D scenes, based on discrepancies between real and simulated camera and LiDAR data.
In another NVIDIA experiment, the Astra agent refined Unitree G1's control through physical trials, and the robot cleared a barrier in 64 out of 100 simulations.