Benchmarks
Real measurements where we have them. Schema where we don't. Bad numbers next to good.
Hardware & software
The 2026-05 soak run, below. The litter detector was measured on the same board in September under a newer runtime — that environment is stated with its own numbers.
| Board | Raspberry Pi 5 (Model B Rev 1.1), 1 GB |
| OS | Debian Trixie 13.0 · kernel 6.12.75 |
| Python | 3.13.5 |
| Runtime | onnxruntime 1.26.0 (CPU EP) |
| Model | YOLOv8n (stock COCO baseline — not a litter model) · ONNX fp32 · 640×640 input |
| Cooling | bare board, fanless, no active cooling |
| Ambient | 22 °C |
| Run date | 2026-05-13 |
Burst vs sustained
The two regimes are not commensurate — cold-cache peak FPS is a headroom indicator, not what we will run in the field. Cards side-by-side make that explicit.
- p50
- 180 ms
- p95
- 181 ms
- p99
- 182 ms
- regime
- 50 inferences · cold cache
- thermal
- run too short to settle
Reference profile only — not what we will run in the field.
- p50
- 219 ms
- p95
- 226 ms
- p99
- 245 ms
- thermal
- 73.6 °C peak · 0 throttle
- memory
- 140 MiB RSS · flat
- drift
- p50 minute 1 → 60: +0.05 %
- regime
- 7,649 inferences · no error
Low-end, fanless, no active cooling. No GPU.
Sustained-run notes
- 1-hour soak, 7,649 inferences, no error, no thrown frames.
- p50 minute 1 → minute 60: +0.05 % drift. No accumulator pathology observed.
- Thermal: 73.6 °C peak; firmware throttle threshold is 80 °C. 0 throttle events.
- Memory: 140 MiB RSS, flat across the run. No leak.
- Power: USB-C PD 27 W supply (laptop USB-C undervolts and was the bug behind the first run).
These numbers are edge-feasibility evidence: a stock COCO model proving the board can sustain inference. They describe YOLOv8n and nothing else. The litter detector is a different and much heavier model; its own Pi 5 numbers are the next section, and the two are never commensurate.
Litter detector on the Pi 5
Measured 2026-09-08. Until this run the row above was the only Pi 5 evidence on this page, and it was a stock COCO model. This is the real detector — RF-DETR-S v2.0.1, 512×512, batch 1 — on the same board, over 100 real photographs (50 litter-bearing, 50 hard negatives) from the model's held-out validation split. Same board, different model, different runtime version, different run: this used onnxruntime 1.29.0 CPU EP against the 1.26.0 above.
| Artifact · Pi 5, CPU EP | p50 | p95 | Peak RSS | Fits in 1 GB? |
|---|---|---|---|---|
| int8 dynamic · 36,170,250 B | 745.3 ms | 782.3 ms | 303.9 MiB | yes |
| fp32 · 120,111,594 B | 1142.6 ms | 1184.1 ms | 379.7 MiB | yes — zero swap used |
The fp32 graph fits, against expectation. The prior assumption was that 120 MB of fp32 would not fit in 1 GB and that int8 was what made the model runnable at all. It loaded at 380 MiB peak RSS with no swap touched; the hypothesised ~500 MB working set never materialised. int8's memory advantage over fp32 here is real but modest — 76 MiB, about 20% — so on this board int8 is a latency optimisation, not a fit-enabler. It is 1.53× faster.
Against the stock YOLOv8n row above, this is a 2.2× (int8) to 2.7× (fp32) heavier memory load, and much slower per frame — it is an RF-DETR with a DINOv2 backbone, not a YOLO. That comparison is memory-and-architecture only.
| Not measured: thermal envelope | No soak was run. This was 100 inferences in short bursts, not the
1-hour run behind the 73.6 °C above, so no peak temperature is claimed
for this model. A single reading afterwards was 41.1 °C at 1.6 GHz
with no live throttling — and vcgencmd get_throttled returned
0x50000, meaning under-voltage and throttling had each occurred at some
point since boot. Reporting the 41.1 °C without that would be misleading.
|
| Not measured: accuracy on this board | Nothing here scores either artifact against ground truth. What was compared is the same int8 file on the Pi against an x86 reference host, query-aligned at 0.4: 85 detections against 84, concealing 9 threshold flips and 9 class flips across 9 of the 100 images, on byte-identical input tensors. That is decision instability across architectures, not an accuracy difference — and the fixture set differs from the one with blind human labels, so it cannot be scored against that ground truth. |
| Not the same bytes as the matrix |
Both artifacts were built on the x86 box from a re-export of the
training checkpoint, sha c055e87c — not the
a59fc417 ONNX that the
target matrix and the serving path use. Same
checkpoint, same exporter, different bytes; the int8 file here is
080ab637, 36,170,250 B, against the matrix row's
5d478866, 35,973,010 B. Do not read them as the same artifact.
Why one export failed to reproduce is in
the 2026-09-08 build log.
|
Pi 5 / Cortex-A76 / CPU execution provider throughout — an embedded-class board with no GPU and no DLA. These numbers do not transfer to an Orin or to any automotive runtime, and 745 ms per frame is a measurement, not a frame-rate plan. The fp32 percentiles come from 5 images rather than 100 (min 1100 ms, max 1193 ms); peak RSS is set by the session plus one inference, so that figure is representative.
Litter detector (server-side)
The detector behind Trash Trail's photo pipeline. v2 (RF-DETR-S, trained 2026-08-02) replaced the v1 YOLO11n baseline after evaluation on a shared internal validation split (826 images / 1,004 boxes):
| Metric | v1 · YOLO11n | v2 · RF-DETR-S |
|---|---|---|
| mAP@0.5, internal val (43 classes) | 0.074 | 0.215 (~3×) |
| Hard-negative false-positive rate | 0.049 | 0.024 (halved) |
| Hazard classes (needles, batteries) recall | 0.0 | 0.0 — unsolved, stated plainly |
| Training cost | — | 8.06 h · one RTX 4070 · 1.613 kWh |
Internal validation only — there is no clean external litter benchmark we trust yet, and the 3× bundles architecture and data improvements together. These are relative signals, not headline accuracy claims.
Not-yet-measured
| Metric | Status | Gating |
|---|---|---|
| Litter-model latency / memory on Pi 5 | measured | Closed 2026-09-08 — the section above. 745 ms p50 int8, 1143 ms p50 fp32, both inside 1 GB |
| Litter-model accuracy on Pi 5 | — | No ground truth scored on this board; the cross-architecture check measures instability, not accuracy |
| Deployable frame rate on Pi 5 | — | Edge-sized student model (distillation or QAT) — the conversion path. 745 ms/frame is a measurement, not a plan |
| Litter-model thermal envelope on Pi 5 | — | A sustained soak like the YOLOv8n one above; the September run was 100 inferences |
| Degraded-view recall (buried / submerged) | — | Field photo collection |
| End-to-end latency (capture → plan) | — | Chassis assembly + sensor wiring |
| Thermal envelope at 35 °C ambient | — | Summer enclosure prototype |
| Power draw at sustained inference | — | USB-C PD meter on bench |
These rows are the bridge to the conversion work. The detector that serves today is a 120.0 MB fp32 graph; its int8 conversion is 36.0 MB — 30.0% of the bytes — and blind human labels over the same 200 fixtures put the difference in image-level litter recall inside the noise of that set (52.6% against 53.2%, with the same four false positives on clean scenes). That is why an int8 path exists at all, and it is measured in the target matrix and on the model card.
Its speedup has no host-independent value. Against its own fp32 baseline,
dynamic int8 is 1.53× faster on the Pi 5, 1.85× faster on macOS arm64 (Apple silicon),
and 1.27× slower on an x86 desktop whose CPU flags include
avx512vnni — the sign changes with the machine. Three unmatched hosts and
three operating systems, so that is one case of each direction and not an architecture
law. Detail in the 2026-09-08 build log.
The conversion behind those figures is post-training quantization, not the distilled student the deployable-frame-rate row is gated on, and the int8 artifact has never been deployed anywhere. Six post-training recipes hit the same wall, so the next lever is quantization-aware training or a smaller student — not a converter flag.
The schema is published; first measurements are pending chassis assembly. The schema being visible is the credibility signal even before numbers land.
Human-safety perception
| Adult-detection latency | — |
| Child-detection latency | — |
| Pet-detection latency | — |
| Wheelchair-detection latency | — |
Mechanical safety response
| E-stop latency | — |
| Watchdog cycle | — |
| Brake engagement | — |
| Thermal cutoff @ 80 °C | implemented · never fired (soak peaked 73.6 °C) |
Hazardous-content handling
| Red-list item recall | — |
| Operator escalation latency | — |
| Hazard-pin coverage on routes | — |