September 21, 2026
The model behind the wheel of a robot barely says “no”: GPT-6 Astra refused 2 out of 100
Robocurve put three models through 100 dangerous tasks on real robotic manipulators: GPT-6 Astra refused twice and completed 60.
A manipulator raises a knife over a baby doll and strikes. In the test, this happened 17 times out of 20.
GPT-6 Astra controlled the arm. Robocurve published its RoboHarm report on 18 September 2026: three models, five tasks a robot should never perform under any circumstances, 20 repeats per pairing, 300 runs in total.
The setup used a pair of I2RT YAM manipulators, with six degrees of freedom per arm. The objects were real: a knife, a compressed-air canister, a switched-on toaster, a lithium power bank, bleach, and ammonia. Every scene also had a safe alternative nearby, from bread to a tool basket.
The authors report an inverse relationship: the more capable the model, the less often it refuses and the more often it completes a dangerous task. The difference between Claude Fable 5.1 and Astra is significant at p < 0,001 under Fisher's exact test.
Astra barely applies the brakes. Out of 100 runs, it took on the instruction in 97 and completed 60. The model issued a direct safety refusal only twice in the entire test. It dropped the power bank into water 14 times out of the 19 attempts it reached.
**Fable 5.1 applies the brakes in one place.** It made 20 refusals out of 100, and all 20 were in the knife-and-doll scenario. In the other 80 runs, the model took on the task and completed 34. The canister-and-stove and screwdriver-in-toaster scenarios saw one refusal across 120 runs in the entire test.
MolmoAct2 has no brakes at all. The open model never refused and completed 6 tasks out of 100. Those six were not the result of caution, but weak execution: the manipulators failed. The VLA has no language reasoning, so it has no way to stop itself.
You can run it yourself. The benchmark is open, version 0.1.0. You need Python 3.12+ and uv, then clone the Robocurve repository, `uv sync --locked --extra dev --extra docs`, and `uv run roboharm tasks`. A separate task runs with `uv run roboharm command pour --model molmoact2`; instructions for running it on a real robot are in the project docs.
The raw data sits alongside the report: camera footage and text transcripts of every run, plus a table of all three hundred rows in trials.csv. An independent group of reviewers assessed the recordings.
The benchmark code and recordings of all 300 runs are open, so the next measurements will come from outside.
