Production INT4 for constrained silicon

Your model.
Our compiler.
Their microcontroller.

Turn trained neural networks into compact, deterministic firmware for Cortex-M, Xtensa, RISC-V, and custom silicon. MicroQuant combines adaptive INT4 formats, a zero-heap C++ runtime, capability-selected vector kernels, and reproducible physical-silicon validation.

Request a paid architecture audit Try the live compiler
Dynamic heap
0.00 KB

Every buffer static. No arena, no fragmentation.

ESP32-S3 inference
18.52 ms

Complete 269K-parameter MLP with KARQ P1 and PIE at 240 MHz.

Model RAM
43.0 KB

KARQ P1 under PIE, 42.5% below the per-channel INT4 baseline.

Physical proof
24 sessions

Eight complete cells, three independent cold boots each, all bit-exact.

What you are hiring

Two product schemes. One control that keeps them honest.

PCQ4, SPQ4, and KARQ are signed INT4 and share the importer, planner, ABI-v5 codegen, and portable runtime. PCQ4 is private evaluation infrastructure; SPQ4 and KARQ keep their product positions only where measured model/target constraints justify the extra machinery.

Fully implemented control

PCQ4

Per-channel Quantization, 4-bit

  • One scale per output row. No per-block scale metadata, selectors, or palettes.
  • One requantization per row. It prices the runtime cost of SPQ4's local scales and KARQ's palette machinery.
  • Independent comparison floor. 141,186 B model flash and 4.718 M cycles p50 on the portable ESP32-S3 path.
  • Benchmark reference. It quantifies what KARQ and SPQ4 gain beyond standard per-channel INT4.

If a product scheme cannot beat this control on a declared constraint, complexity has not earned its place.

Released default · rollback

SPQ4

Scale-Protected Quantization, 4-bit

  • Per-block MSE-optimal clip search. Outliers saturate at the INT4 rail instead of destroying the scale resolution of every neighbouring weight.
  • Fixed-point requantization (acc·m0) >> nb with round-to-nearest — integer MACs, masks and shifts only.
  • Free structural sparsity. All-zero blocks compile to m0 == 0 and are skipped at zero storage cost, with a compile-time-constant skip pattern (WCET-safe).
  • The regression oracle. Numerically frozen by golden fixtures and retained as the shipped rollback.

The robust choice for locally heterogeneous weights and models that can turn structured zero blocks into real skipped work.

ESP32-S3 performance leader

KARQ

Kernel-Cost-Aware Reconstruction Quantization

  • Exponent palettes. Aligned blocks pick from a small per-row palette of power-of-two relative scales (P=1/2/4); block dot products accumulate in palette buckets so scaling hoists out of the inner loop.
  • Cost-aware configuration search. Candidates are compiled and filtered against task quality, model bytes, memory, and target cost before selection.
  • Three disjoint data roles. Optimization/calibration, configuration selection, and final acceptance are hash-traceable and never overlap.
  • Division-free, float-free, heap-free production execution, proven at source and object level.

KARQ P1 is the current public leader: 15.8% faster, 32.7% less model flash, and 40.6% less model RAM than SPQ4 on the trained portable benchmark.

KARQ now leads the complete trained-model matrix

On the production portable ESP32-S3 capture, KARQ P1 completes the 269K-parameter MLP in 4.663 M cycles / 19.43 ms. The PIE vector path reaches 4.445 M cycles / 18.52 ms while reducing model RAM to 43.0 KB. Every result is reproduced across three independent cold boots.

SPQ4 remains available for the workloads it was designed to win: heterogeneous local ranges and structured sparsity. MicroQuant selects the format and backend against the customer's actual task, memory map, and silicon rather than forcing one global quantization recipe.

Live showcase

The SPQ4 compiler playground

Upload weights — or synthesize them — and watch the real pipeline run: scale-protected INT4 quantization, SRAM tile solving, dead-block sparsity analysis, and C++ codegen. Raw .bin/.raw float32 is quantized for real in your browser. .onnx/.tflite uploads are illustrated with a synthetic stand-in — the browser does not parse graphs; real graph parsing runs only in the server-side toolchain.

🧪 Evaluation demo — output is illustrative, not calibrated against your dataset and not validated on silicon.

1 Model ingestion

Upload raw float32 weights, or configure a synthetic layer.

or synthesize
One fixed-point scale (m0, nb) per block
Drives the 2D weight-tile solver
30%
Zero blocks compile to m0 = 0 and are skipped free

2 Quantization map

Live block Dead block Clipped outlier

Hover any weight to inspect its INT4 code, block scale, and packed byte.

Nibble packing — 2 × INT4 → 1 byte

w0
–
····
+
w1
–
····
→
Packed byte
–
········
–Compiled flash
–vs FP32
–Dead blocks skipped
–SRAM tile
–MACs / inference
–Reconstruction RMSE

3 Generated C++ assets

model_assets.h — flash-aligned static arrays. An evaluation artifact from this browser demo: not calibrated against your dataset and not validated on your silicon. Real headers come from the full toolchain under an engagement's parity and HIL gates.


// Run the SPQ4 compiler to generate C++ assets...

This is a browser demo of the SPQ4 pipeline, not a production compiler.

Raw float32 weights are quantized live; ONNX/TFLite uploads synthesize a representative dense layer client-side. Real graph parsing runs in the server-side toolchain over a supported operator subset — see the support matrix. In an engagement we run the full compiler against your actual model, calibrate on your dataset, and produce validated target evidence.

Scope a paid audit

Deployment coverage

What MicroQuant can ship

A production compiler/runtime surface for dense networks and compact CNNs, with strict graph validation, configurable target profiles, and physical proof on RP2350 Arm and ESP32-S3.

Supported production path, tested Limited works within a stated subset Custom available through a scoped engagement Outside scope requires a different solution

Operators

Dense / Gemm / MatMul (constant weights, bias, transB)Supported
ReLU / Clip(clamp) fused as dense/conv activationsSupported
Identity / Flatten / Reshape (shape-safe)Supported
Conv2D — 1×1 pointwise & 3×3, stride 1/2, same/validSupported
Depthwise Conv2D 3×3 (channel multiplier 1)Supported
MaxPool / AvgPool 2×2 s2 · GlobalAveragePoolSupported
Softmax (argmax / top-k on int32 logits)Limited
Hidden-layer sigmoid / tanh (fixed-domain LUT)Supported
Attention / transformer, GELU, final-layer sigmoid/tanhRejected
The CNN path is correctness-first: every supported layer is bit-exact between the host reference and the generated C++, including padding and stride variants. NHWC layouts and static shapes only.

Quantization Backends & Import Formats

spq4 — signed-int4-per-block-fixed-scale; operators: dense, conv2d, depthwise_conv2ddefault · portable-runtime · promoted physical evidence scoped by environment
karq — signed-int4-exponent-palette; operators: dense, conv2d, depthwise_conv2dselectable · portable-runtime · promoted physical evidence scoped by environment
pcq4 — per-channel INT4 comparison baselineFully implemented · evaluation baseline
Raw float32 weights .bin / .rawSupported
ONNX — Gemm / MatMul / Relu / Clip / Identity / Flatten / Reshape (strict subset)Supported
cnn-json explicit NHWC specSupported
ONNX Conv → NHWC conversion (static CNN subset)Supported
TFLite ingestion (stdlib flatbuffer reader, documented subset)Supported
PyTorch / Keras (via export to ONNX)Indirect
SPQ4 is the local-scale and structured-sparsity option; KARQ is the performance-and-memory option selected per model. The trained ESP32-S3 MLP gives KARQ P1 the leading public latency/RAM result under both portable and PIE execution. Unsupported graph combinations are identified before asset generation.

MCU Families & Boards · ISA Features

rp2350-arm-m33 · board pico2-rp2350-arm (Raspberry Pi Pico 2)Silicon-validated: SPQ4 dense, generic-swar
rp2350-riscv-hazard3 · board pico2-rp2350-riscv (Pico 2, RISC-V path)Cross-compile verified
generic-cortex-m7 (ARMv7E-M · arm-dsp, simd32, FPU)Compiles
generic-cortex-m55 (ARMv8.1-M · arm-dsp + Helium/MVE)Compiles
esp32-s3 (Xtensa LX7 · board esp32-s3-devkitc1-n16r8)Silicon-validated: KARQ dense/CNN + trained eight-cell generic-SWAR/PIE matrix, ESP-IDF v6.0.2
portenta-h7-m7 (STM32H747 M7 only · board arduino-portenta-h7-m7)Integration-ready board pack
generic-rv32imac (RISC-V RV32IMAC)Cross-compile verified
Board nucleo-h743zi (concrete BSP template)Available as a custom board engagement
ISA caps: arm-dsp · arm-mve · riscv-p · riscv-v · xtensa-simd · custom:*Declarable; kernels vary
Targets are data profiles—ISA, memory banks, DMA, linker placement, toolchain, and runtime contracts—so a new MCU is configured behind stable compiler/runtime boundaries. Physical proof is committed for Pico 2 Arm and ESP32-S3 N16R8. The ESP32-S3 runs promoted KARQ dense/CNN plus the complete trained PCQ4/SPQ4/KARQ MLP matrix under generic SWAR and PIE at ESP-IDF v6.0.2.

Kernel Backends · Runtime Ports · Evidence

Kernel generic-swar (portable INT4 MAC, always-safe fallback)Available
Kernel arm-dsp-smuad (Cortex-M4/M7 __SMUAD)Available
Kernel esp32-s3-pie (Xtensa PIE packed-S4 bank)Complete-model physical HIL · four configurations · bit-exact
Kernels Helium/MVE · RISC-V P/VAvailable through target-specific kernel work
Ports: host · baremetal-generic · cortex-m scaffold · RP2350 Arm/RISC-V · STM32H7 M7 · ESP32-S3 · custom MQ_PORT_HEADERSupported
Evidence — host benchmark & bit-exact parityIncluded
Evidence — QEMU semihosting (labeled qemu)KARQ dense + CNN passed · QEMU 11.0.2
Evidence — physical HIL on real silicon (RP2350 SPQ4; ESP32-S3 KARQ dense/CNN + eight-cell trained-model matrix)Delivered ✓
Evidence — model-attributed flash/RAM and complete portable policyDelivered · KARQ P1/P2 accepted
Evidence — optimized ESP32-S3 PIE complete-model cyclesDelivered · 4 cells × 3 cold boots
Evidence — cross-ISA KARQ validationNot claimed — validated on one ISA to date
Physical HIL means flashed and measured on the named board. ESP32-S3 evidence covers promoted KARQ dense/CNN plus eight complete trained-model cells: generic SWAR and PIE, PCQ4/SPQ4/KARQ, model-attributed flash/RAM, corrected stack, deterministic builds, and bit-exact fixtures. The portable policy accepts KARQ P1/P2 and the reference control; PIE awaits only the matching host backend identity for a merged verdict.

Validated Toolchains & SDKs

Python — compiler host>=3.12; tested 3.12/3.13/3.14
ONNX — model import1.22; evidence 1.22.0
Arm GNU Toolchain — firmware toolchain15.3.Rel1; evidence arm-none-eabi-gcc (Arm GNU Toolchain 15.3.Rel1 (Build arm-15.149)) 15.3.1 20260627
Raspberry Pi Pico SDK — vendor SDK2.3.0
ArduinoCore-mbed — vendor SDK — Portenta4.6.0
Espressif ESP-IDF — vendor SDK — ESP32-S3v6.0.2
QEMU — emulation11.0.2; evidence QEMU emulator version 11.0.2
The exact toolchain and SDK versions this release is built and validated against. Vendor SDK rows also fix their subordinate compiler, debugger, and flasher identities, so a delivered result can be reproduced rather than approximated. Other versions are frequently fine; these are the ones we can state as tested. Tell us which toolchain you are on and we will confirm before any engagement starts.

Silicon proof

A complete model. Two kernel paths. One clear leader.

The same trained 269K-parameter neural network runs through four INT4 configurations and two ESP32-S3 execution backends, with model-attributed memory, measured stack, and repeatable physical timing.

host_simulation

Functional parity on the host CPU. Never a target number.

qemu

Target-ISA instruction semantics under QEMU 11.0.2. Never silicon timing.

physical_hil

Flashed and measured on a real board. Never upgraded from anything else.

ESP32-S3 — complete trained-model benchmark

24 physical sessions

Pinned Apache-2.0 MNIST MLP, 784→256→256→10, 269,322 parameters. Four retained configurations share one calibration split and untouched 1,000-example test. Eight complete cells run on the exact ESP32-S3-DevKitC-1 N16R8, Xtensa LX7 @ 240 MHz, ESP-IDF v6.0.2: four portable and four PIE. Every cell reproduces all 10 declared outputs across three independent compile/flash/cold-boot sessions.

ConfigurationBackendTop-1p50 cyclesp50Model flashModel RAM
KARQ P1PIE96.7%4,445,09418.52 ms144,800 B43,968 B
KARQ P1Portable96.7%4,662,76919.43 ms144,800 B47,616 B
PCQ4 baselinePortable96.3%4,717,59919.66 ms141,186 B80,064 B
PCQ4 baselinePIE96.3%4,802,32320.01 ms141,186 B76,448 B
KARQ P2PIE96.5%4,992,66920.80 ms152,520 B43,968 B
KARQ P2Portable96.5%5,172,08521.55 ms152,520 B47,616 B
SPQ4PIE96.4%5,334,34022.23 ms215,288 B76,480 B
SPQ4Portable96.4%5,538,36723.08 ms215,288 B80,128 B

KARQ P1 is the current public performance-and-memory configuration. Against SPQ4 on the portable path it is 15.8% faster, uses 32.7% less model flash, and 40.6% less model RAM. Against the per-channel baseline under PIE it is 7.4% faster with 42.5% less model RAM for a 2.6% flash premium. Top-1 values are properties of the corresponding quantized assets on the untouched 1,000-example task set; each physical cell separately proves its backend with 10/10 declared outputs across three cold boots.

Why both public formats matter

FormatBest fitCommercial value
KARQ P1Dense and memory-constrained modelsLeading measured latency/RAM balance
KARQ palettesRows with reusable exponent regimesAdaptive precision without per-block full-scale metadata
SPQ4Local range variation and structured sparsityArbitrary block scales plus compile-time dead-block elimination
Customer-pilot ready. Complete model execution, deterministic compilation, model-attributed flash/RAM, measured stack, portable fallback, and ESP32-S3 vector execution are all present in one reproducible evidence chain.
Results are scoped to the named ESP32-S3 board, model, clock, SDK, and build. Every customer engagement repeats the same proof on the customer's graph, memory placement, firmware conditions, and acceptance budget.

Raspberry Pi Pico 2 / RP2350 Arm — SPQ4 dense on silicon

physical_hil

Cortex-M33 @ 150 MHz. Firmware built through the Pico SDK, flashed, and captured by our hardware-in-the-loop pipeline. This is an SPQ4 record and says nothing about KARQ.

Check / measurementResultSource
Host ↔ silicon output paritybit-identicalhost reference vs captured serial transcript
Blocking ↔ cooperative parity on siliconbit-identicalon-target AsyncExecutor vs blocking run
Inference latency (on-chip)968.8 µsDWT-derived port timer (mq-port-timer)
Cycle count (on-chip)145,208 cyclesdwt-cyccnt, enable-verified; cross-checks the timer within 0.1%
Stack high-water904 Bstack-paint-scan between linker symbols
Reset causepower-on-resetpowman-chip-reset register
Reproduce the physical result: with a Pico 2 on your desk and the exact Pico SDK 2.3.0 environment, one command re-runs the whole validation — python3 studio/validate.py --board pico2-rp2350-arm --synthetic dense — and prints the same hashes and serial transcript recorded in the evidence report that ships with the result. Physical timing is produced only when the named board and matching environment are present.

Host harness output

Reference 2-layer demo model (64×128 → 32×64), SPQ4.

MetricFP32 referenceMicroQuant SPQ4Delta
Weight flash footprint40.00 KB7.81 KB−80.5%
Dynamic heap allocationallocator-dependent0.00 KBstatic memory model
Inference latency (host)3.07 µs1.74 µs1.77× faster
Fixed-point accuracybaseline0.28% relative L1round-to-nearest requant
Async vs blocking parity—bit-identicalPASS
These are host measurements (x86 / Apple Silicon, -O3). Host latency is wall-clock, noise-sensitive, and not a target metric. The Cortex-M4/M7 __SMUAD backend is compiled and bit-exact-parity-tested against the portable reference. The ESP32-S3 PIE backend has complete-model physical cycles for all four retained configurations in the matrix above.

Reproduce It Yourself

Three commands. Nothing beyond Python 3.12+ and a C++11 compiler.

terminal
$ make test == make test: ALL CHECKS PASSED == $ python3 compiler/main.py --synthetic-demo --out benchmark/model_assets.h SUCCESS: bare-metal assets written $ g++ -std=c++11 -O3 -Iruntime/include -Ibenchmark \ benchmark/benchmark.cpp -o benchmark_run && ./benchmark_run Async vs. Blocking exact bit parity : PASS (bit-identical) Relative L1 error : 0.2785 % Flash size reduction : 80.47% vs FP32 Dynamic heap allocation : 0.00 KB

The engagement

Paid architecture audit

A fixed-scope first step that turns your model, memory map, and latency budget into a measured deployment strategy for your silicon.

What you get

  • Model & operator audit against the support matrix: what compiles today, what needs co-engineering, what should stay int8 or float.
  • Memory & latency risk register: flash / SRAM / TCM pressure, accumulator-overflow bounds, cooperative-scheduling and watchdog risks.
  • Scheme study on your weights: PCQ4 control, SPQ4 configurations, KARQ P1/P2/P4, and justified per-layer candidates compared under identical data roles with task, error, saturation, complete-size, and target-cost evidence.
  • Target feasibility matrix: kernel backend and ISA-feature fit for your MCU family, with the compiler flags and port work required.
  • Scoped pilot SOW: acceptance gates, a verification plan (host → QEMU → physical HIL), and the footprint/latency envelope to validate.
Typical timeline: 1–2 weeks from receiving your model and target profile. You receive a decision-ready report, review session, and scoped path to a production pilot.

What you bring · what we guarantee

You bring

  • Model artifact (ONNX / raw weights / architecture description), under NDA if needed.
  • Target MCU or board, memory map, RTOS or bare-metal environment, and toolchain.
  • Flash, SRAM, latency — and power, if relevant — budgets.
  • A representative calibration and test set if accuracy must be validated.

We guarantee

  • A reproducible measurement method with explicit pass/fail gates and hashed artifacts.
  • A layer-by-layer precision recommendation with clear engineering rationale.
  • No hidden heap in any delivered runtime path.

Every result is project-specific

  • Accuracy is established against your representative task data.
  • Cycle numbers are captured on your physical target.
  • Custom operators and target work are scoped explicitly.

Legal clarity

One license. No copyleft. No surprises.

MicroQuant is commercial-only proprietary software under a single clean EULA. Nothing in this stack can contaminate your codebase.

Commercial production license

For shipping products

  • Object-form distribution of the runtime and generated assets inside your licensed product lines — fully closed-source, no disclosure obligations.
  • No open-source strings attached: there is no AGPL/GPL track anywhere in this stack; a single proprietary EULA governs everything.
  • Explicit zero-liability terms: quantization precision, runtime behaviour, timing, and execution outcomes are validated by you for your application.
  • Engagement-scoped deliverables: custom kernels and integration work are licensed per Statement of Work with clear IP boundaries.
Read the commercial EULA
Evaluation & recruitment sandbox

For reviewers & hiring teams

  • Free technical review: hiring managers, assessors, and prospective clients may clone, read, build, and benchmark this repository locally at no charge.
  • Zero contamination: sandbox review creates no license obligations for your employer's proprietary code — no copyleft, no reciprocity, no disclosure.
  • Zero liability, both directions: evaluation runs as-is at your own risk, and we assert no claims for good-faith evaluation activity.
  • Clean scope: anything production, commercial, or customer-facing requires the commercial license. The boundary is explicit.
Read the sandbox waiver (§4)

Where MicroQuant creates leverage

Built for demanding embedded-ML products

The strongest projects have a real model, a real microcontroller, and a constraint that conventional deployment tooling cannot quite meet.

Strong fit
  • A dense/MLP, small 1D or 2D CNN, keyword-spotting, or anomaly model that barely misses a flash, SRAM, latency, or power target.
  • Firmware that cannot tolerate heap or needs deterministic, cooperative execution under an RTOS.
  • A Cortex-M, RISC-V, or Xtensa target where generic vendor tooling left performance or memory on the table.
  • A team that controls the firmware build and can share a model plus representative data under NDA.
  • You want evidence — parity, footprint, and per-engagement cycle reports — not a black box.
Custom advantage
  • You need a model-specific balance of flash, SRAM, latency, and task accuracy.
  • Your product benefits from per-layer format selection instead of one global quantization rule.
  • You need target-specific kernels for Cortex-M DSP, Xtensa PIE, RISC-V, or a custom MAC.
  • You want generated assets, integration wrappers, and physical evidence delivered as one traceable package.
  • You value an engineering partner who can move from model analysis to firmware integration and silicon proof.

Start here

Request a paid architecture audit

Tell us about your model and your silicon. A compiler engineer reviews every request and returns a concrete scope for the architecture audit and pilot. The more technical detail you provide, the faster we can start.

What the first call covers

Feasibility, honestly

We look at your model class, accuracy budget, and memory map and tell you what INT4 will and will not do for it — before any contract.

Bottleneck diagnosis

Flash pressure, SRAM contention, cycle budgets, watchdog constraints — we map where your current inference path actually hurts.

A concrete integration plan

You leave with a scoped proposal: target kernels, expected footprint and latency envelopes, verification deliverables, and timeline.

Audit intake

Model
Target hardware
Budgets & timeline
A compiler engineer reviews every submission. Under NDA on request. No marketing list. We store only what you enter here to assess and respond to your request (lawful basis: legitimate interest). No model or dataset is uploaded. Retained per our retention policy; email us any time to have your submission erased.
✓

Audit request received

A TinyML compiler engineer will review your model and target details and reply to name@company.com with a scope and next steps — or an honest no-fit — typically within two business days.