ESP32-S3 — first trained three-scheme model on silicon
physical_hil
Pinned Apache-2.0 MNIST MLP, 784→256→256→10, 269,322
parameters. Thirteen fixed host candidates share one 800-example calibration
split and untouched balanced 1,000-example test. Four cells run on the exact
ESP32-S3-DevKitC-1 N16R8, Xtensa LX7 @ 240 MHz, ESP-IDF v6.0.2,
release-verified generic-swar, three independent cold boots.
Every cell reproduces host logits bit for bit on the declared smoke inputs.
Float top-1 is 96.60%. The full candidate spread is 0.4 percentage points,
below the registered 0.5-point materiality floor, and every Wilson 95%
interval overlaps. Quality is not the separator. PCQ4 is 24.4% faster than
SPQ4; every SPQ4 grouping exceeds the registered 200 KB persistent-model
limit. KARQ P2 follows the float reference more closely but produces no
material task gain.
Why this reverses the synthetic result
The private-control win is a failed differentiation gate.
PCQ4 is alone on every measured silicon front for this model; it is not a
product candidate. KARQ and SPQ4 must beat that control in a workload that
exercises their declared adaptive-scale or structured-sparsity advantages.
Separately, the combined report does not yet evaluate the full policy once per
candidate: the ESP-IDF stack value is minimum-free space rather than bytes
used, total deployed size omits disjoint linked-code accounting, and
deterministic recompilation did not run. Until those are repaired, no
candidate has a complete promotion verdict.
Scope, stated plainly: this exact board, ISA, SDK, toolchain,
clock, portable kernel, and one easy dense classifier. It does not validate a
real CNN/audio model, another ESP32-S3 variant, the unimplemented PIE
esp32-s3-simd kernel, any Arm or RISC-V board, loaded-system
latency, or energy. No cross-ISA claim exists.
Raspberry Pi Pico 2 / RP2350 Arm — SPQ4 dense on silicon
physical_hil
Cortex-M33 @ 150 MHz. Firmware built through the Pico SDK, flashed, and captured
by our hardware-in-the-loop pipeline. This is an SPQ4 record and
says nothing about KARQ.
Reproduce the physical result: with a Pico 2 on your desk and the
exact Pico SDK 2.3.0 environment, one command re-runs the whole validation —
python3 studio/validate.py --board pico2-rp2350-arm --synthetic dense —
and prints the same hashes and serial transcript recorded in the evidence report
that ships with the result. Without the board and a matching environment the
pipeline reports unavailable; it will never quietly substitute a
desktop number.
Host harness output
Reference 2-layer demo model (64×128 → 32×64), SPQ4.
These are host measurements (x86 / Apple Silicon,
-O3). Host latency is wall-clock, noise-sensitive, and not a
target metric. The Cortex-M4/M7 __SMUAD backend is compiled and
bit-exact-parity-tested against the portable reference, but
no optimized kernel has ever produced an on-silicon cycle count
— both silicon runs above deliberately use the portable kernel.
Reproduce It Yourself
Three commands. Nothing beyond Python 3.12+ and a C++11 compiler.
$ make test
== make test: ALL CHECKS PASSED ==
$ python3 compiler/main.py --synthetic-demo --out benchmark/model_assets.h
SUCCESS: bare-metal assets written
$ g++ -std=c++11 -O3 -Iruntime/include -Ibenchmark \
benchmark/benchmark.cpp -o benchmark_run && ./benchmark_run
Async vs. Blocking exact bit parity : PASS (bit-identical)
Relative L1 error : 0.2785 %
Flash size reduction : 80.47% vs FP32
Dynamic heap allocation : 0.00 KB
How an engagement runs
- Audit — model, activation ranges, memory map, timing budget.
- Design — SPQ4 or KARQ selected and configured against your accuracy target and silicon.
- Integrate — kernels, RTOS scheduling, build-system wiring.
- Verify — parity, WCET, and footprint evidence delivered with hashes.