Briefing
Python code above the firmware handles keyboard and gamepad jogging, colored-cube detection, and open-vocabulary grounding with Grounding DINO and SAM 2.1. The planner addresses objects by id, such as cube_2 or slot_4, instead of coordinates. When a pick or place is impossible, it reports the physical limit that blocked it rather than picking a nearby object. The mt4_mcp server runs over stdio or Streamable HTTP.
Setup assumes Windows for the jog client, a CH340 serial link and a fixed USB camera. A script restores the stock Grbl-derived firmware from a backup. One video shows a cube-stacking script, the other a Qwen3-VL session acting on typed English instructions. Neither shows the MCP path.
Written by the Boards & Bots curator
From the GitHub API
Repository facts
| Repository | enghoff/ |
|---|---|
| Stars | 0 |
| Language | Python |
| Last push | 20 Aug 2026 2 mo ago |