Briefing
The five tasks are process-control jobs such as tank fill and drain, thermal chamber hysteresis and a pressure vessel interlock. The firmware runs in QEMU and exchanges sensor and actuator signals over UART with deterministic Python plant models. In the primary self-verification condition, gpt-5.4 passed 14 of 15 runs and qwen3.5-27B passed 9 of 15. The other configurations were gpt-5.4-mini and four smaller Qwen3.5 models.
The authors write that simulation does not replace validation on devices, and they name hardware-in-the-loop studies as future work. The suite leaves out communication stacks, storage, power management and interrupt-heavy firmware. Claude Opus was not evaluated.
Written by the Boards & Bots curator