Last-mile TinyML compiler engineering

Your model.
Our compiler.
Their microcontroller.

The specialist team you call when generic edge-AI tooling misses your flash, SRAM, latency, or firmware-integration budget. Two product INT4 schemes, one private control, a header-only zero-allocation C++ runtime, and an evidence pipeline that reports when the control wins.

Dynamic heap
0.00 KB

Every buffer static. No arena, no fragmentation.

First trained model
PCQ4

24.4% faster than SPQ4 on the pinned ESP32-S3 MLP.

Boards on silicon
2

Pico 2 RP2350 Arm and ESP32-S3-DevKitC-1 N16R8.

Promotion verdict
Open

One model, portable kernel, final policy merge still incomplete.

What you are hiring

Two product schemes. One control that keeps them honest.

PCQ4, SPQ4, and KARQ are signed INT4 and share the importer, planner, ABI-v5 codegen, and portable runtime. PCQ4 is private evaluation infrastructure; SPQ4 and KARQ keep their product positions only where measured model/target constraints justify the extra machinery.

Private control

PCQ4

Per-channel Quantization, 4-bit

  • One scale per output row. No per-block scale metadata, selectors, or palettes.
  • One requantization per row. It prices the runtime cost of SPQ4's local scales and KARQ's palette machinery.
  • Current trained-model leader. 4.58 M cycles p50 and 141,702 B persistent model on the pinned ESP32-S3 MLP.
  • Not customer-selectable. It is the required simplicity/latency baseline, not a public support promise.

If a product scheme cannot beat this control on a declared constraint, complexity has not earned its place.

Released default · rollback

SPQ4

Scale-Protected Quantization, 4-bit

  • Per-block MSE-optimal clip search. Outliers saturate at the INT4 rail instead of destroying the scale resolution of every neighbouring weight.
  • Fixed-point requantization (acc·m0) >> nb with round-to-nearest — integer MACs, masks and shifts only.
  • Free structural sparsity. All-zero blocks compile to m0 == 0 and are skipped at zero storage cost, with a compile-time-constant skip pattern (WCET-safe).
  • The regression oracle. Numerically frozen by golden fixtures and retained as the shipped rollback.

Strongest where local block ranges or structured dead blocks create value. It was slowest and exceeded the model-byte limit on the first trained ESP32-S3 model.

Selectable · non-default

KARQ

Kernel-Cost-Aware Reconstruction Quantization

  • Exponent palettes. Aligned blocks pick from a small per-row palette of power-of-two relative scales (P=1/2/4); block dot products accumulate in palette buckets so scaling hoists out of the inner loop.
  • Cost-aware configuration search. Candidates are compiled and filtered before selection. Automatic latency selection is currently withheld for four of five operators until the release-cost profile is recaptured.
  • Three disjoint data roles. Optimization/calibration, configuration selection, and final acceptance are hash-traceable and never overlap.
  • Division-free, float-free, heap-free production execution, proven at source and object level.

P1 is a compact per-row fallback. P>1 is the actual palette proposition; it reduced float disagreement but created no task/Pareto value on the first trained model.

Current decision: preserve the rollback, do not promote a winner yet

The synthetic dense fixture made SPQ4 fastest because it supplied 47 dead blocks to skip. The trained MLP removed that free sparsity: PCQ4 became 24.4% faster than SPQ4, and every SPQ4 grouping exceeded the registered 200 KB persistent-model limit. KARQ P1 was compact and second-fastest; P2 changed the intended numeric behavior but not task accuracy.

SPQ4 remains the v0.8 default/rollback because changing a shipped policy needs more than one easy dense model, and the ESP32-S3 PIE kernel is not implemented. No candidate yet has a complete merged promotion-policy verdict: stack-use semantics, disjoint linked-size accounting, and deterministic recompilation are open. We publish that boundary with the numbers.

Live showcase

The SPQ4 compiler playground

Upload weights — or synthesize them — and watch the real pipeline run: scale-protected INT4 quantization, SRAM tile solving, dead-block sparsity analysis, and C++ codegen. Raw .bin/.raw float32 is quantized for real in your browser. .onnx/.tflite uploads are illustrated with a synthetic stand-in — the browser does not parse graphs; real graph parsing runs only in the server-side toolchain.

🧪 Evaluation demo — output is illustrative, not calibrated against your dataset and not validated on silicon.

1 Model ingestion

Upload raw float32 weights, or configure a synthetic layer.

or synthesize
One fixed-point scale (m0, nb) per block
Drives the 2D weight-tile solver
30%
Zero blocks compile to m0 = 0 and are skipped free

2 Quantization map

Live block Dead block Clipped outlier

Hover any weight to inspect its INT4 code, block scale, and packed byte.

Nibble packing — 2 × INT4 → 1 byte

w0
····
+
w1
····
Packed byte
········
Compiled flash
vs FP32
Dead blocks skipped
SRAM tile
MACs / inference
Reconstruction RMSE

3 Generated C++ assets

model_assets.h — flash-aligned static arrays. An evaluation artifact from this browser demo: not calibrated against your dataset and not validated on your silicon. Real headers come from the full toolchain under an engagement's parity and HIL gates.


// Run the SPQ4 compiler to generate C++ assets...
                        

This is a browser demo of the SPQ4 pipeline, not a production compiler.

Raw float32 weights are quantized live; ONNX/TFLite uploads synthesize a representative dense layer client-side. Real graph parsing runs in the server-side toolchain over a supported operator subset — see the support matrix. In an engagement we run the full compiler against your actual model, calibrate on your dataset, and produce validated target evidence.

Scope a paid audit

Production truth

Support matrix

Exactly what the toolchain compiles, runs, and can prove today, what is actively in development, and what we will not claim. If your model or your part is not on this page, that is the conversation to have on the first call — not a surprise three weeks into an engagement.

Supported production path, tested Limited works within a stated subset In development planned, not available today Not supported out of scope, deliberately

Operators

Dense / Gemm / MatMul (constant weights, bias, transB)Supported
ReLU / Clip(clamp) fused as dense/conv activationsSupported
Identity / Flatten / Reshape (shape-safe)Supported
Conv2D — 1×1 pointwise & 3×3, stride 1/2, same/validEarly access
Depthwise Conv2D 3×3 (channel multiplier 1)Early access
MaxPool / AvgPool 2×2 s2 · GlobalAveragePoolEarly access
Softmax (argmax / top-k on int32 logits)Limited
Hidden-layer sigmoid / tanh (fixed-domain LUT)Supported
Attention / transformer, GELU, final-layer sigmoid/tanhRejected
The CNN path is correctness-first: every supported layer is bit-exact between the host reference and the generated C++, including padding and stride variants. NHWC layouts and static shapes only.

Quantization Backends & Import Formats

spq4 — signed-int4-per-block-fixed-scale; operators: dense, conv2d, depthwise_conv2ddefault · portable-runtime · promoted physical evidence scoped by environment
karq — signed-int4-exponent-palette; operators: dense, conv2d, depthwise_conv2dselectable · portable-runtime · promoted physical evidence scoped by environment
pcq4 — private per-channel INT4 measurement control; not customer-selectableEvaluation baseline
Raw float32 weights .bin / .rawSupported
ONNX — Gemm / MatMul / Relu / Clip / Identity / Flatten / Reshape (strict subset)Supported
cnn-json explicit NHWC specSupported
ONNX Conv → NHWC conversion (static CNN subset)Supported
TFLite ingestion (stdlib flatbuffer reader, documented subset)Supported
PyTorch / Keras (via export to ONNX)Indirect
SPQ4 is the released default and rollback; KARQ is selectable per project. PCQ4 is a private per-channel INT4 control used to test whether either product scheme earns its extra format/runtime complexity. The first trained ESP32-S3 model favours PCQ4, so no public scheme is presented as universally best. An unsupported operator, or an unsupported target and scheme combination, fails before code generation — you get a clear error, never a silent substitution.

MCU Families & Boards · ISA Features

rp2350-arm-m33 · board pico2-rp2350-arm (Raspberry Pi Pico 2)Silicon-validated: SPQ4 dense, generic-swar
rp2350-riscv-hazard3 · board pico2-rp2350-riscv (Pico 2, RISC-V path)Builds; not yet on silicon
generic-cortex-m7 (ARMv7E-M · arm-dsp, simd32, FPU)Compiles
generic-cortex-m55 (ARMv8.1-M · arm-dsp + Helium/MVE)Compiles
esp32-s3 (Xtensa LX7 · board esp32-s3-devkitc1-n16r8)Silicon-validated: KARQ dense/CNN; trained PCQ4/SPQ4/KARQ evaluation, generic-swar, ESP-IDF v6.0.2
portenta-h7-m7 (STM32H747 M7 only · board arduino-portenta-h7-m7)Board support ready; not yet on silicon
generic-rv32imac (RISC-V RV32IMAC)Builds; not yet on silicon
Board nucleo-h743zi (concrete BSP template)In development
ISA caps: arm-dsp · arm-mve · riscv-p · riscv-v · xtensa-simd · custom:*Declarable; kernels vary
A target is a data profile — ISA, FPU, memory banks, DMA, linker sections, toolchain flags — so adding an MCU is configuration work, not a rewrite. "Silicon-validated" means flashed and measured on that physical board, with functional parity and a named source for every number. "Compiles" means cross-compilation is verified but nothing has been measured on that part. Pico 2 Arm and ESP32-S3-DevKitC-1 N16R8 are the committed physical-HIL boards. ESP32-S3-DevKitC-1 N16R8 is PHYSICAL-HIL VALIDATED for KARQ dense and CNN on the portable generic-swar kernel under exact ESP-IDF v6.0.2. A separate fixed-candidate trained-model evaluation compares PCQ4, SPQ4 and KARQ on that exact environment; it does not promote a default. Portenta H7 M7 board pack is present as a template; no committed silicon evidence yet. Each result is scoped to that exact board and environment and is not evidence for another ISA.

Kernel Backends · Runtime Ports · Evidence

Kernel generic-swar (portable INT4 MAC, always-safe fallback)Available
Kernel arm-dsp-smuad (Cortex-M4/M7 __SMUAD)Available
Kernels Helium/MVE · RISC-V P/V · Xtensa SIMDIn development
Ports: host · baremetal-generic · cortex-m scaffold · RP2350 Arm/RISC-V · STM32H7 M7 · ESP32-S3 · custom MQ_PORT_HEADERSupported
Evidence — host benchmark & bit-exact parityIncluded
Evidence — QEMU semihosting (labeled qemu)KARQ dense + CNN passed · QEMU 11.0.2
Evidence — physical HIL on real silicon (RP2350 SPQ4; ESP32-S3 N16R8 KARQ dense/CNN + trained three-scheme evaluation)Delivered ✓
Evidence — complete merged promotion-policy verdict for the trained ESP32-S3 modelIn development: stack/size semantics + deterministic compile
Evidence — optimized-DSP-kernel cycles on siliconIn development
Evidence — cross-ISA KARQ validationNot claimed — validated on one ISA to date
Every measurement is labelled by how it was obtained: physical_hil on real silicon, qemu under emulation, host_simulation on a development machine. A label is never upgraded. KARQ dense and CNN pass under QEMU 11.0.2 with exact parity, which proves instruction semantics rather than timing on a part. Silicon evidence covers SPQ4 on Raspberry Pi Pico 2; on ESP32-S3 it covers promoted KARQ dense/CNN and a model-scoped trained PCQ4/SPQ4/KARQ comparison with the portable kernel. The trained Pareto result is not yet a full-policy verdict: stack-use semantics, disjoint linked-size accounting and deterministic recompilation remain open. None of this extends to another board, ISA, kernel, SDK, model family or energy claim.

Validated Toolchains & SDKs

Python — compiler host>=3.12; tested 3.12/3.13/3.14
ONNX — model import1.22; evidence 1.22.0
Arm GNU Toolchain — firmware toolchain15.3.Rel1; evidence arm-none-eabi-gcc (Arm GNU Toolchain 15.3.Rel1 (Build arm-15.149)) 15.3.1 20260627
Raspberry Pi Pico SDK — vendor SDK2.3.0
ArduinoCore-mbed — vendor SDK — Portenta4.6.0
Espressif ESP-IDF — vendor SDK — ESP32-S3v6.0.2
QEMU — emulation11.0.2; evidence QEMU emulator version 11.0.2
The exact toolchain and SDK versions this release is built and validated against. Vendor SDK rows also fix their subordinate compiler, debugger, and flasher identities, so a delivered result can be reproduced rather than approximated. Other versions are frequently fine; these are the ones we can state as tested. Tell us which toolchain you are on and we will confirm before any engagement starts.

Measured, not marketed

Evidence

Every number below comes from a versioned evidence report with artifact hashes, toolchain and SDK versions, and a source label per measurement. Public support reports reach evidence/ only through a promotion gate; model-scoped research records stay explicitly labelled under the evaluation archive until every promotion condition is met.

host_simulation

Functional parity on the host CPU. Never a target number.

qemu

Target-ISA instruction semantics under QEMU 11.0.2. Never silicon timing.

physical_hil

Flashed and measured on a real board. Never upgraded from anything else.

ESP32-S3 — first trained three-scheme model on silicon

physical_hil

Pinned Apache-2.0 MNIST MLP, 784→256→256→10, 269,322 parameters. Thirteen fixed host candidates share one 800-example calibration split and untouched balanced 1,000-example test. Four cells run on the exact ESP32-S3-DevKitC-1 N16R8, Xtensa LX7 @ 240 MHz, ESP-IDF v6.0.2, release-verified generic-swar, three independent cold boots. Every cell reproduces host logits bit for bit on the declared smoke inputs.

CellTop-1Persistent modelCycles p50 / p99Latency p50Linked flash
PCQ496.30%141,702 B4,579,909 / 4,580,04719.084 ms306,947 B
KARQ g32 P196.70%145,414 B5,097,361 / 5,098,82821.240 ms313,563 B
KARQ g32 P296.50%153,222 B5,354,452 / 5,355,81122.311 ms321,371 B
SPQ4 g3296.40%215,750 B5,694,911 / 5,695,97023.730 ms380,727 B

Float top-1 is 96.60%. The full candidate spread is 0.4 percentage points, below the registered 0.5-point materiality floor, and every Wilson 95% interval overlaps. Quality is not the separator. PCQ4 is 24.4% faster than SPQ4; every SPQ4 grouping exceeds the registered 200 KB persistent-model limit. KARQ P2 follows the float reference more closely but produces no material task gain.

Why this reverses the synthetic result

Release-build fixturePCQ4SPQ4KARQ P1What dominates
Synthetic dense139,887131,444215,164SPQ4 skips 47 fixture-created dead blocks
Synthetic CNN639,6671,044,368948,522Per-row vs per-block requantization
Trained MLP4,579,9095,694,9115,097,361No free block sparsity; PCQ4's simpler hot path wins
The private-control win is a failed differentiation gate. PCQ4 is alone on every measured silicon front for this model; it is not a product candidate. KARQ and SPQ4 must beat that control in a workload that exercises their declared adaptive-scale or structured-sparsity advantages. Separately, the combined report does not yet evaluate the full policy once per candidate: the ESP-IDF stack value is minimum-free space rather than bytes used, total deployed size omits disjoint linked-code accounting, and deterministic recompilation did not run. Until those are repaired, no candidate has a complete promotion verdict.
Scope, stated plainly: this exact board, ISA, SDK, toolchain, clock, portable kernel, and one easy dense classifier. It does not validate a real CNN/audio model, another ESP32-S3 variant, the unimplemented PIE esp32-s3-simd kernel, any Arm or RISC-V board, loaded-system latency, or energy. No cross-ISA claim exists.

Raspberry Pi Pico 2 / RP2350 Arm — SPQ4 dense on silicon

physical_hil

Cortex-M33 @ 150 MHz. Firmware built through the Pico SDK, flashed, and captured by our hardware-in-the-loop pipeline. This is an SPQ4 record and says nothing about KARQ.

Check / measurementResultSource
Host ↔ silicon output paritybit-identicalhost reference vs captured serial transcript
Blocking ↔ cooperative parity on siliconbit-identicalon-target AsyncExecutor vs blocking run
Inference latency (on-chip)968.8 µsDWT-derived port timer (mq-port-timer)
Cycle count (on-chip)145,208 cyclesdwt-cyccnt, enable-verified; cross-checks the timer within 0.1%
Stack high-water904 Bstack-paint-scan between linker symbols
Reset causepower-on-resetpowman-chip-reset register
Reproduce the physical result: with a Pico 2 on your desk and the exact Pico SDK 2.3.0 environment, one command re-runs the whole validation — python3 studio/validate.py --board pico2-rp2350-arm --synthetic dense — and prints the same hashes and serial transcript recorded in the evidence report that ships with the result. Without the board and a matching environment the pipeline reports unavailable; it will never quietly substitute a desktop number.

Host harness output

Reference 2-layer demo model (64×128 → 32×64), SPQ4.

MetricFP32 referenceMicroQuant SPQ4Delta
Weight flash footprint40.00 KB7.81 KB−80.5%
Dynamic heap allocationallocator-dependent0.00 KBstatic memory model
Inference latency (host)3.07 µs1.74 µs1.77× faster
Fixed-point accuracybaseline0.28% relative L1round-to-nearest requant
Async vs blocking paritybit-identicalPASS
These are host measurements (x86 / Apple Silicon, -O3). Host latency is wall-clock, noise-sensitive, and not a target metric. The Cortex-M4/M7 __SMUAD backend is compiled and bit-exact-parity-tested against the portable reference, but no optimized kernel has ever produced an on-silicon cycle count — both silicon runs above deliberately use the portable kernel.

Reproduce It Yourself

Three commands. Nothing beyond Python 3.12+ and a C++11 compiler.

terminal
$ make test == make test: ALL CHECKS PASSED == $ python3 compiler/main.py --synthetic-demo --out benchmark/model_assets.h SUCCESS: bare-metal assets written $ g++ -std=c++11 -O3 -Iruntime/include -Ibenchmark \ benchmark/benchmark.cpp -o benchmark_run && ./benchmark_run Async vs. Blocking exact bit parity : PASS (bit-identical) Relative L1 error : 0.2785 % Flash size reduction : 80.47% vs FP32 Dynamic heap allocation : 0.00 KB

The engagement

Paid architecture audit

A fixed-scope first step. The deliverable is a firmware-grade feasibility answer for your model on your silicon — valuable even when the conclusion is "keep this layer at int8" or "MicroQuant is not your fit."

What you get

  • Model & operator audit against the support matrix: what compiles today, what needs co-engineering, what should stay int8 or float.
  • Memory & latency risk register: flash / SRAM / TCM pressure, accumulator-overflow bounds, cooperative-scheduling and watchdog risks.
  • Scheme study on your weights: PCQ4 control, SPQ4 configurations, KARQ P1/P2/P4, and justified per-layer candidates compared under identical data roles with task, error, saturation, complete-size, and target-cost evidence.
  • Target feasibility matrix: kernel backend and ISA-feature fit for your MCU family, with the compiler flags and port work required.
  • Scoped pilot SOW: acceptance gates, a verification plan (host → QEMU → physical HIL), and the footprint/latency envelope to validate.
Typical timeline: 1–2 weeks from receiving your model and target profile. The deliverable is a written report and a review call — not production firmware.

What you bring · what we guarantee

You bring

  • Model artifact (ONNX / raw weights / architecture description), under NDA if needed.
  • Target MCU or board, memory map, RTOS or bare-metal environment, and toolchain.
  • Flash, SRAM, latency — and power, if relevant — budgets.
  • A representative calibration and test set if accuracy must be validated.

We guarantee

  • A reproducible measurement method with explicit pass/fail gates and hashed artifacts.
  • Honest operator-support boundaries and a written "keep it at int8" or "not a fit" answer when that is the truth.
  • No hidden heap in any delivered runtime path.

We do not guarantee

  • A specific INT4 accuracy before the audit measures your model.
  • On-silicon cycle numbers without a physical HIL run on your board.
  • Support for operators or frameworks outside the matrix within the audit window.

Legal clarity

One license. No copyleft. No surprises.

MicroQuant is commercial-only proprietary software under a single clean EULA. Nothing in this stack can contaminate your codebase.

Commercial production license

For shipping products

  • Object-form distribution of the runtime and generated assets inside your licensed product lines — fully closed-source, no disclosure obligations.
  • No open-source strings attached: there is no AGPL/GPL track anywhere in this stack; a single proprietary EULA governs everything.
  • Explicit zero-liability terms: quantization precision, runtime behaviour, timing, and execution outcomes are validated by you for your application.
  • Engagement-scoped deliverables: custom kernels and integration work are licensed per Statement of Work with clear IP boundaries.
Read the commercial EULA
Evaluation & recruitment sandbox

For reviewers & hiring teams

  • Free technical review: hiring managers, assessors, and prospective clients may clone, read, build, and benchmark this repository locally at no charge.
  • Zero contamination: sandbox review creates no license obligations for your employer's proprietary code — no copyleft, no reciprocity, no disclosure.
  • Zero liability, both directions: evaluation runs as-is at your own risk, and we assert no claims for good-faith evaluation activity.
  • Clean scope: anything production, commercial, or customer-facing requires the commercial license. The boundary is explicit.
Read the sandbox waiver (§4)

Qualify yourself first

When MicroQuant fits — and when it doesn't

We would rather lose a bad-fit lead than oversell. If you are on the right side of this line, the audit pays for itself. If you are not, we will say so on the first call.

Strong fit
  • A dense/MLP, small 1D or 2D CNN, keyword-spotting, or anomaly model that barely misses a flash, SRAM, latency, or power target.
  • Firmware that cannot tolerate heap or needs deterministic, cooperative execution under an RTOS.
  • A Cortex-M, RISC-V, or Xtensa target where generic vendor tooling left performance or memory on the table.
  • A team that controls the firmware build and can share a model plus representative data under NDA.
  • You want evidence — parity, footprint, and per-engagement cycle reports — not a black box.
Not the right tool (yet)
  • You want a self-serve "upload any ONNX → production C++" download. That does not exist here, by design.
  • You need full computer vision, transformer/LLM, or general TFLite deployment today — outside the support matrix.
  • You need safety-certified (automotive or medical) evidence without funding a safety-grade scope.
  • You cannot share model details or representative data, so accuracy cannot be validated.
  • Your constraint is comfortably met by CMSIS-NN, TFLM, Edge Impulse, or ST tooling. Use those — we are the last-mile specialist for when they are not enough.

Start here

Request a paid architecture audit

Tell us about your model and your silicon. A compiler engineer — not a salesperson — reviews every request and replies with a scope, or an honest "not a fit." The more technical detail you give, the faster we can triage.

What the first call covers

Feasibility, honestly

We look at your model class, accuracy budget, and memory map and tell you what INT4 will and will not do for it — before any contract.

Bottleneck diagnosis

Flash pressure, SRAM contention, cycle budgets, watchdog constraints — we map where your current inference path actually hurts.

A concrete integration plan

You leave with a scoped proposal: target kernels, expected footprint and latency envelopes, verification deliverables, and timeline.

Audit intake

Model
Target hardware
Budgets & timeline
A compiler engineer reviews every submission. Under NDA on request. No marketing list. We store only what you enter here to assess and respond to your request (lawful basis: legitimate interest). No model or dataset is uploaded. Retained per our retention policy; email us any time to have your submission erased.

Audit request received

A TinyML compiler engineer will review your model and target details and reply to name@company.com with a scope and next steps — or an honest no-fit — typically within two business days.