Last-mile TinyML compiler engineering

Your model.
Our compiler.
Their microcontroller.

The specialist team you call when generic edge-AI tooling misses your flash, SRAM, latency, or firmware-integration budget. Two INT4 quantization schemes, one header-only zero-allocation C++ runtime, and an evidence pipeline that refuses to round up.

Request a paid architecture audit Try the live compiler
Dynamic heap
0.00 KB

Every buffer static. No arena, no fragmentation.

Flash vs FP32
−80%

Measured end-to-end on the demo model, metadata included.

Boards on silicon
2

Pico 2 RP2350 Arm and ESP32-S3-DevKitC-1 N16R8.

Cross-ISA claim
None

KARQ has run on one ISA. We say so everywhere.

What you are hiring

Two schemes. One honest compiler.

Both are signed INT4, both run through the same importer, planner, ABI-v5 codegen, and runtime. They are not interchangeable, and we do not pretend the newer one always wins.

Default

SPQ4

Scale-Protected Quantization, 4-bit

  • Per-block MSE-optimal clip search. Outliers saturate at the INT4 rail instead of destroying the scale resolution of every neighbouring weight.
  • Fixed-point requantization (acc·m0) >> nb with round-to-nearest — integer MACs, masks and shifts only.
  • Free structural sparsity. All-zero blocks compile to m0 == 0 and are skipped at zero storage cost, with a compile-time-constant skip pattern (WCET-safe).
  • The regression oracle. Numerically frozen by golden fixtures; every KARQ change must leave it bit-identical.

Production default and rollback path. Silicon-validated on Raspberry Pi Pico 2.

Selectable · non-default

KARQ

Kernel-Cost-Aware Reconstruction Quantization

  • Exponent palettes. Aligned blocks pick from a small per-row palette of power-of-two relative scales (P=1/2/4); block dot products accumulate in palette buckets so scaling hoists out of the inner loop.
  • Cost-aware configuration search. Candidates are really compiled, filtered by a versioned accuracy-preservation gate, then minimised against a measured target cost profile — or an explicitly labelled conservative fallback. Never a guessed cycle weight.
  • Three disjoint data roles. Optimization/calibration, configuration selection, and final acceptance are hash-traceable and never overlap.
  • Division-free, float-free, heap-free production execution, proven at source and object level.

Silicon-validated on ESP32-S3-DevKitC-1 N16R8 for dense and CNN, portable kernel, exact ESP-IDF v6.0.2 — that environment and no other.

When SPQ4 is still the right answer

Keep SPQ4 when a KARQ candidate fails the accuracy gate, when the active environment has only a conservative cost fallback, when selector/palette/epoch metadata dominates a short-row workload, when the requested target has no scoped KARQ evidence, or whenever the complete measured result — generated and linked bytes, peak SRAM, stack, latency — is Pareto-dominated by SPQ4. On the one board that has run both, SPQ4 is about 2.1× faster than KARQ on the dense demo. We publish that.

The full engineering comparison — what each format stores byte for byte, the two inner loops side by side, why the measured latency, flash, and SRAM results landed where they did, and which trade-offs are inherent versus unfixed implementation gaps — is docs/SCHEME_SELECTION.md in the repository.

Live showcase

The SPQ4 compiler playground

Upload weights — or synthesize them — and watch the real pipeline run: scale-protected INT4 quantization, SRAM tile solving, dead-block sparsity analysis, and C++ codegen. Raw .bin/.raw float32 is quantized for real in your browser. .onnx/.tflite uploads are illustrated with a synthetic stand-in — the browser does not parse graphs; real graph parsing runs only in the server-side toolchain.

🧪 Evaluation demo — output is illustrative, not calibrated against your dataset and not validated on silicon.

1 Model ingestion

Upload raw float32 weights, or configure a synthetic layer.

or synthesize
One fixed-point scale (m0, nb) per block
Drives the 2D weight-tile solver
30%
Zero blocks compile to m0 = 0 and are skipped free

2 Quantization map

Live block Dead block Clipped outlier

Hover any weight to inspect its INT4 code, block scale, and packed byte.

Nibble packing — 2 × INT4 → 1 byte

w0
–
····
+
w1
–
····
→
Packed byte
–
········
–Compiled flash
–vs FP32
–Dead blocks skipped
–SRAM tile
–MACs / inference
–Reconstruction RMSE

3 Generated C++ assets

model_assets.h — flash-aligned static arrays. An evaluation artifact from this browser demo: not calibrated against your dataset and not validated on your silicon. Real headers come from the full toolchain under an engagement's parity and HIL gates.


// Run the SPQ4 compiler to generate C++ assets...

This is a browser demo of the SPQ4 pipeline, not a production compiler.

Raw float32 weights are quantized live; ONNX/TFLite uploads synthesize a representative dense layer client-side. Real graph parsing runs in the server-side toolchain over a supported operator subset — see the support matrix. In an engagement we run the full compiler against your actual model, calibrate on your dataset, and produce validated target evidence.

Scope a paid audit

Production truth

Support matrix

Exactly what the toolchain compiles, runs, and can prove today — and what it does not. These cards are generated from the same truth source as docs/support_matrix.md in the repository; CI fails if this page drifts from it. Gaps are additive registry extension points, not a different product.

Supported production path, tested Partial MVP / placeholder / host-only Roadmap modeled extension point, not in-tree Not yet explicitly rejected or not claimed

Operators

Dense / Gemm / MatMul (constant weights, bias, transB)Supported
ReLU / Clip(clamp) fused as dense/conv activationsSupported
Identity / Flatten / Reshape (shape-safe)Supported
Conv2D — 1×1 pointwise & 3×3, stride 1/2, same/validCNN MVP
Depthwise Conv2D 3×3 (channel multiplier 1)CNN MVP
MaxPool / AvgPool 2×2 s2 · GlobalAveragePoolCNN MVP
Softmax (argmax / top-k on int32 logits)Placeholder
Hidden-layer sigmoid / tanh (fixed-domain LUT)Supported
Attention / transformer, GELU, final-layer sigmoid/tanhRejected
CNN MVP is correctness-first with host-sim ↔ C++ bit-exact parity (incl. padding/stride variants); NHWC only, static shapes.

Quantization Backends & Import Formats

spq4 — signed-int4-per-block-fixed-scale; operators: dense, conv2d, depthwise_conv2ddefault · portable-runtime · promoted physical evidence scoped by environment
karq — signed-int4-exponent-palette; operators: dense, conv2d, depthwise_conv2dselectable · portable-runtime · promoted physical evidence scoped by environment
Raw float32 weights .bin / .rawSupported
ONNX — Gemm / MatMul / Relu / Clip / Identity / Flatten / Reshape (strict subset)Supported
cnn-json explicit NHWC specSupported
ONNX Conv → NHWC conversion (static CNN subset)Supported
TFLite ingestion (stdlib flatbuffer reader, documented subset)Supported
PyTorch / Keras (via export to ONNX)Indirect
Scheme rows are generated from the live selectable backend registry, including maturity and environment-scoped evidence status. SPQ4 remains the default. Unsupported ops or target/scheme combinations fail before codegen — no synthetic or cross-scheme fallback in production mode.

MCU Families & Boards · ISA Features

rp2350-arm-m33 · board pico2-rp2350-arm (Raspberry Pi Pico 2)Silicon-validated: SPQ4 dense, generic-swar
rp2350-riscv-hazard3 · board pico2-rp2350-riscv (Pico 2, RISC-V path)Profile + port
generic-cortex-m7 (ARMv7E-M · arm-dsp, simd32, FPU)Compiles
generic-cortex-m55 (ARMv8.1-M · arm-dsp + Helium/MVE)Compiles
esp32-s3 (Xtensa LX7 · board esp32-s3-devkitc1-n16r8)Silicon-validated: KARQ dense + CNN, generic-swar, ESP-IDF v6.0.2
portenta-h7-m7 (STM32H747 M7 only · board arduino-portenta-h7-m7)M7 board pack template; no committed silicon evidence
generic-rv32imac (RISC-V RV32IMAC)Profile
Board nucleo-h743zi (concrete BSP template)Unverified
ISA caps: arm-dsp · arm-mve · riscv-p · riscv-v · xtensa-simd · custom:*Modeled
Targets are pure-data profiles (ISA, FPU, memory banks, DMA, linker sections, toolchain flags). A new MCU is profile data, not a code change. "Silicon-validated" = flashed and measured on a real board by the in-repo HIL pipeline (functional parity + measured sources; portable kernel). "Compiles" = cross-compile verified in this repo. Portenta H7 M7 board pack is present as a template; no committed silicon evidence yet. ESP32-S3-DevKitC-1 N16R8 is PHYSICAL-HIL VALIDATED for KARQ dense and CNN on the portable generic-swar kernel under exact ESP-IDF v6.0.2. Pico 2 Arm and ESP32-S3-DevKitC-1 N16R8 are the committed physical-HIL boards. The ESP32-S3 claim is exact-board/environment scoped and is not cross-ISA evidence.

Kernel Backends · Runtime Ports · Evidence

Kernel generic-swar (portable INT4 MAC, always-safe fallback)In-tree
Kernel arm-dsp-smuad (Cortex-M4/M7 __SMUAD)In-tree
Kernels Helium/MVE · RISC-V P/V · Xtensa SIMDRoadmap
Ports: host · baremetal-generic · cortex-m scaffold · RP2350 Arm/RISC-V · STM32H7 M7 · ESP32-S3 · custom MQ_PORT_HEADERSupported
Evidence — host benchmark & bit-exact parityIn repo
Evidence — QEMU semihosting (labeled qemu)Section-L KARQ dense/CNN passed · QEMU 11.0.2
Evidence — physical HIL on real silicon (RP2350 SPQ4; ESP32-S3 N16R8 KARQ dense/CNN)In repo ✓
Evidence — optimized-DSP-kernel cycles on siliconNext boards
Evidence — cross-ISA KARQ validationNot claimed — one ISA only (plan item N19 open)
The HIL pipeline (runners incl. Pico SDK and ESP-IDF; probe adapters: picotool/uf2/openocd/pyocd/jlink/stm32cubeprog/esptool; board profiles) labels evidence physical_hil / qemu / host_simulation. Section-L KARQ dense and CNN system-machine runs passed under QEMU 11.0.2 with exact parity; this proves instruction semantics, not physical timing. Promoted physical evidence is scoped to SPQ4 on Raspberry Pi Pico 2 and KARQ dense/CNN on esp32-s3-devkitc1-n16r8 under exact ESP-IDF v6.0.2 with generic-swar. Neither promotes another board, ISA, kernel, SDK, toolchain, or workload. Missing bench prerequisites remain honest unavailable states. Intrinsics stay isolated behind ISA-gated backend files; generic builds pull in no vendor headers.

Current Release Environment

python — python-runtime>=3.12; tested 3.12/3.13/3.14
setuptools — python-package83; evidence 83.0.0
onnx — python-package1.22; evidence 1.22.0
PySide6 UI extra — python-package6.11; evidence 6.11.1
Arm GNU Toolchain (standalone) — standalone15.3.Rel1; evidence arm-none-eabi-gcc (Arm GNU Toolchain 15.3.Rel1 (Build arm-15.149)) 15.3.1 20260627
pico-sdk — vendor-managed2.3.0
ArduinoCore-mbed (Portenta) — vendor-managed4.6.0
esp-idf — vendor-managedv6.0.2
qemu — standalone11.0.2; evidence QEMU emulator version 11.0.2
actions/checkout — ci-servicev7.0.1 @ 3d3c42e5aac5
actions/setup-python — ci-servicev7.0.0 @ 5fda3b95a4ea
actions/upload-artifact — ci-servicev7.0.1 @ 043fb46d1a93
GitHub Actions runner — ci-serviceubuntu-24.04
Generated from dev/current_release_baseline.json. Standalone tools track their approved maintained release; vendor-managed SDK/BSP rows own their subordinate compiler, debugger, flasher, and build-tool identities. Compatibility environments never replace the release-evidence identity.

Measured, not marketed

Evidence

Every number below comes from a versioned evidence report with artifact hashes, toolchain and SDK versions, and a source label per measurement. Reports reach the archive only through a promotion gate that refuses anything non-passing, unhashed, or broader than what was actually run.

host_simulation

Functional parity on the host CPU. Never a target number.

qemu

Target-ISA instruction semantics under QEMU 11.0.2. Never silicon timing.

physical_hil

Flashed and measured on a real board. Never upgraded from anything else.

ESP32-S3-DevKitC-1 N16R8 — KARQ dense and CNN on silicon

physical_hil

Xtensa LX7 @ 240 MHz, exact ESP-IDF v6.0.2 with its IDF-owned Xtensa toolchain, MQ_PORT_ESP32S3, portable generic-swar kernel, FreeRTOS task pinned to core 1, internal memory placement, DMA disabled. 100-sample distributions; cycles cross-checked by an independent esp-timer.

WorkloadCycles p50Latency p50Free-stack watermarkLinked flashLinked RAM
KARQ dense453,0391,889 µs23,300 B196,635 B467,081 B
KARQ CNN1,883,7357,850 µs21,420 B192,075 B453,249 B

Both pass host↔silicon output parity; dense also passes on-target blocking↔cooperative parity. The CNN runtime has no cooperative executor, so its second slot is a blocking determinism re-run and is labelled as such.

KARQ vs SPQ4, same board, same session

SchemeCycles p50Total µs (4 vectors)Linked flashLinked RAM
SPQ4212,1473,861195,859 B470,473 B
KARQ453,0397,870196,635 B467,081 B
Read that honestly: on this board, this model, and the portable kernel, SPQ4 is roughly 2.1× faster than KARQ and marginally smaller in flash. KARQ's demonstrated advantage is planned peak SRAM on a different workload (388 B vs 896 B on the gesture model). That is exactly why SPQ4 remains the default and the rollback path, and why we do not sell KARQ as a free upgrade.
Scope, stated plainly: this exact board, ISA, SDK, toolchain, clock, and kernel backend. It does not validate another ESP32-S3 variant, the unimplemented esp32-s3-simd kernel, any Arm or RISC-V board, optimized-kernel cycles, or energy. KARQ has run on one ISA, so no cross-ISA claim exists anywhere in this project.

Raspberry Pi Pico 2 / RP2350 Arm — SPQ4 dense on silicon

physical_hil

Cortex-M33 @ 150 MHz. Firmware built through the Pico SDK, flashed, and captured by the in-repo HIL pipeline. This is an SPQ4 record and satisfies no KARQ gate.

Check / measurementResultSource
Host ↔ silicon output paritybit-identicalhost reference vs captured serial transcript
Blocking ↔ cooperative parity on siliconbit-identicalon-target AsyncExecutor vs blocking run
Inference latency (on-chip)968.8 µsDWT-derived port timer (mq-port-timer)
Cycle count (on-chip)145,208 cyclesdwt-cyccnt, enable-verified; cross-checks the timer within 0.1%
Stack high-water904 Bstack-paint-scan between linker symbols
Reset causepower-on-resetpowman-chip-reset register
Reproduce the physical result: under the exact Pico SDK 2.3.0 environment, connect a Pico 2 and run python3 studio/validate.py --board pico2-rp2350-arm --synthetic dense. Compare the generated hashes and transcript against evidence/rp2350-pico2-arm/2026-07-06_physical_hil_v0.7.0.json. Without the board and a matching environment the honest result is unavailable — never a substituted host number.

Host harness output

Reference 2-layer demo model (64×128 → 32×64), SPQ4.

MetricFP32 referenceMicroQuant SPQ4Delta
Weight flash footprint40.00 KB7.81 KB−80.5%
Dynamic heap allocationallocator-dependent0.00 KBstatic memory model
Inference latency (host)3.07 µs1.74 µs1.77× faster
Fixed-point accuracybaseline0.28% relative L1round-to-nearest requant
Async vs blocking parity—bit-identicalPASS
These are host measurements (x86 / Apple Silicon, -O3). Host latency is wall-clock, noise-sensitive, and not a target metric. The Cortex-M4/M7 __SMUAD backend is compiled and bit-exact-parity-tested against the portable reference, but no optimized kernel has ever produced an on-silicon cycle count — both silicon runs above deliberately use the portable kernel.

Reproduce It Yourself

Three commands. Nothing beyond Python 3.12+ and a C++11 compiler.

terminal
$ make test == make test: ALL CHECKS PASSED == $ python3 compiler/main.py --synthetic-demo --out benchmark/model_assets.h SUCCESS: bare-metal assets written $ g++ -std=c++11 -O3 -Iruntime/include -Ibenchmark \ benchmark/benchmark.cpp -o benchmark_run && ./benchmark_run Async vs. Blocking exact bit parity : PASS (bit-identical) Relative L1 error : 0.2785 % Flash size reduction : 80.47% vs FP32 Dynamic heap allocation : 0.00 KB

The engagement

Paid architecture audit

A fixed-scope first step. The deliverable is a firmware-grade feasibility answer for your model on your silicon — valuable even when the conclusion is "keep this layer at int8" or "MicroQuant is not your fit."

What you get

  • Model & operator audit against the support matrix: what compiles today, what needs co-engineering, what should stay int8 or float.
  • Memory & latency risk register: flash / SRAM / TCM pressure, accumulator-overflow bounds, cooperative-scheduling and watchdog risks.
  • Scheme study on your weights: SPQ4 and KARQ compared under identical data roles, with per-layer error and saturation evidence from the host harness.
  • Target feasibility matrix: kernel backend and ISA-feature fit for your MCU family, with the compiler flags and port work required.
  • Scoped pilot SOW: acceptance gates, a verification plan (host → QEMU → physical HIL), and the footprint/latency envelope to validate.
Typical timeline: 1–2 weeks from receiving your model and target profile. The deliverable is a written report and a review call — not production firmware.

What you bring · what we guarantee

You bring

  • Model artifact (ONNX / raw weights / architecture description), under NDA if needed.
  • Target MCU or board, memory map, RTOS or bare-metal environment, and toolchain.
  • Flash, SRAM, latency — and power, if relevant — budgets.
  • A representative calibration and test set if accuracy must be validated.

We guarantee

  • A reproducible measurement method with explicit pass/fail gates and hashed artifacts.
  • Honest operator-support boundaries and a written "keep it at int8" or "not a fit" answer when that is the truth.
  • No hidden heap in any delivered runtime path.

We do not guarantee

  • A specific INT4 accuracy before the audit measures your model.
  • On-silicon cycle numbers without a physical HIL run on your board.
  • Support for operators or frameworks outside the matrix within the audit window.

Legal clarity

One license. No copyleft. No surprises.

MicroQuant is commercial-only proprietary software under a single clean EULA. Nothing in this stack can contaminate your codebase.

Commercial production license

For shipping products

  • Object-form distribution of the runtime and generated assets inside your licensed product lines — fully closed-source, no disclosure obligations.
  • No open-source strings attached: there is no AGPL/GPL track anywhere in this stack; a single proprietary EULA governs everything.
  • Explicit zero-liability terms: quantization precision, runtime behaviour, timing, and execution outcomes are validated by you for your application.
  • Engagement-scoped deliverables: custom kernels and integration work are licensed per Statement of Work with clear IP boundaries.
Read the commercial EULA
Evaluation & recruitment sandbox

For reviewers & hiring teams

  • Free technical review: hiring managers, assessors, and prospective clients may clone, read, build, and benchmark this repository locally at no charge.
  • Zero contamination: sandbox review creates no license obligations for your employer's proprietary code — no copyleft, no reciprocity, no disclosure.
  • Zero liability, both directions: evaluation runs as-is at your own risk, and we assert no claims for good-faith evaluation activity.
  • Clean scope: anything production, commercial, or customer-facing requires the commercial license. The boundary is explicit.
Read the sandbox waiver (§4)

Qualify yourself first

When MicroQuant fits — and when it doesn't

We would rather lose a bad-fit lead than oversell. If you are on the right side of this line, the audit pays for itself. If you are not, we will say so on the first call.

Strong fit
  • A dense/MLP, small 1D or 2D CNN, keyword-spotting, or anomaly model that barely misses a flash, SRAM, latency, or power target.
  • Firmware that cannot tolerate heap or needs deterministic, cooperative execution under an RTOS.
  • A Cortex-M, RISC-V, or Xtensa target where generic vendor tooling left performance or memory on the table.
  • A team that controls the firmware build and can share a model plus representative data under NDA.
  • You want evidence — parity, footprint, and per-engagement cycle reports — not a black box.
Not the right tool (yet)
  • You want a self-serve "upload any ONNX → production C++" download. That does not exist here, by design.
  • You need full computer vision, transformer/LLM, or general TFLite deployment today — outside the support matrix.
  • You need safety-certified (automotive or medical) evidence without funding a safety-grade scope.
  • You cannot share model details or representative data, so accuracy cannot be validated.
  • Your constraint is comfortably met by CMSIS-NN, TFLM, Edge Impulse, or ST tooling. Use those — we are the last-mile specialist for when they are not enough.

Start here

Request a paid architecture audit

Tell us about your model and your silicon. A compiler engineer — not a salesperson — reviews every request and replies with a scope, or an honest "not a fit." The more technical detail you give, the faster we can triage.

What the first call covers

Feasibility, honestly

We look at your model class, accuracy budget, and memory map and tell you what INT4 will and will not do for it — before any contract.

Bottleneck diagnosis

Flash pressure, SRAM contention, cycle budgets, watchdog constraints — we map where your current inference path actually hurts.

A concrete integration plan

You leave with a scoped proposal: target kernels, expected footprint and latency envelopes, verification deliverables, and timeline.

Audit intake

Model
Target hardware
Budgets & timeline
A compiler engineer reviews every submission. Under NDA on request. No marketing list. We store only what you enter here to assess and respond to your request (lawful basis: legitimate interest). No model or dataset is uploaded. Retained per our retention policy; email us any time to have your submission erased.
✓

Audit request received

A TinyML compiler engineer will review your model and target details and reply to name@company.com with a scope and next steps — or an honest no-fit — typically within two business days.