NewsBB-0244

AA-Robotics benchmark has coding agents drive a real robot arm

Artificial Analysis launched AA-Robotics v0.1, a benchmark in which language models in coding-agent harnesses control a Hiwonder NexArm robot arm. Models in Codex, Claude Code and Antigravity write code for 8 zero-shot tabletop tasks, watching through 4 cameras. GPT-6 Astra led with 69.4 and Claude Opus 5.5 scored 48.8. Models took about 30 minutes a task; people averaged 20 seconds.

Open original source artificialanalysis.ai · Artificial Analysis

Shown working
Why this evidence label?

The article has a run explorer with left, right and overhead camera video of each attempt and a wall of failure clips.

How we check entries
Preview image for AA-Robotics benchmark has coding agents drive a real robot arm
The source's own preview image, from artificialanalysis.ai.

Briefing

The arm is driven through a simple HTTP API. Of the 4 cameras, 3 stand around the table and 1 is on the wrist. A passed task scores 100, with partial credit below that. The other scores were GPT-6.1 Sol 64.6, Gemini 4 Argon 55.3 and Claude Fable 5.1 46.3. Grasp success was about 45% for Astra and Fable, 36% for Argon and 28% for Sol.

Astra cost $16.36 per task and Sol $1.93. We read the article's text; its run explorer is interactive, and we did not load the videos.

Tags

Report a problem with this entry