NewsBB-0191

EEBench grades agents on circuit design in atopile; Claude Opus 5 leads at 61.6%

The atopile team published EEBench, a benchmark that grades agents on 13 circuit-design tasks written in atopile. The harness builds each design and runs SPICE checks at worst-case tolerance corners. The post says Claude Opus 5 led V1 at 61.6%. One failed design used a 22 uF capacitor that gave about 11.4 uF under bias. V1 is simulation only.

Open original source eebench.org · EEBench (atopile team)

Claimed
Why this evidence label?

Simulation benchmark; V1 does not cover layout, manufacturing or bring-up. Run by the makers of atopile.

How we check entries
Preview image for EEBench grades agents on circuit design in atopile; Claude Opus 5 leads at 61.6%
The source's own preview image, from eebench.org.

Briefing

For each design the harness builds the atopile project, generates the circuit graph and BOM, and runs SPICE and design checks. Cost counts toward the score only if the circuit works. In the 1 September results, Grok 4.6 followed at 57.1%, then Claude Fable 5.1 at 56.4%, Claude Fable 5 at 54.3% and Claude Opus 4.8 Max at 51.4%.

The capacitor failure was in a hold-up design that needed 545 uF. At 4.7 V bias the nominal 22 uF part gave about 11.4 uF, and the rail fell below 3 V after 0.85 ms instead of holding for 20 ms. The atopile team built and funded the benchmark, and all 13 tasks are written in its language.

Tags

Agent

Report a problem with this entry