Briefing
Most of the study ran in simulation: control toys, a Unitree G1 humanoid and Go2, and a Franka arm on kitchen tasks. Models mostly failed when setting joint torques directly and did better writing controllers or supervising pretrained policies. A compass aid helped every locomotion setup tested.
On the real Go2, with a crosshair overlay, Opus 4.6 decided a trash can was safely to one side when it was dead ahead. The dog caught a leg and dragged the can a couple of meters before the team stopped it. In a cluttered corridor, a model given a depth heatmap turned toward the obstacles. No model completed a loop of the office hallway on vision alone.
The authors conclude that tools and access belong in any evaluation of a model. The post has no video of the real runs.
Written by the Boards & Bots curator
“A VLM's real-world influence can change by orders of magnitude depending on the information it has access to.”