YOLO Distillation for Real-Time Inference on a Raspberry Pi Zero

Joint project with Theo Coulson, Northwestern University

Distilling a fine-tuned YOLO detector into a custom architecture small enough to run in real time on a Raspberry Pi Zero 2W, so a drone can spot and land on a landing platform on its own — at about a tenth of the teacher’s parameter count. Read the full report for the complete writeup.

The Platform

Built on a swarm-robotics drone platform from Mike Rubenstein’s lab at Northwestern, with a Raspberry Pi camera module added to the underside. The model itself wasn’t integrated with the flight controller due to a hardware serial issue, so it was evaluated by holding the drone over the landing pad by hand.

Architecture

A standard YOLO has a backbone, a multi-scale neck (FPN), and a head — the neck exists to handle objects of wildly different sizes. Since we only ever detect one object of roughly fixed size, we drop the neck entirely and predict from a single scale, replacing YOLO11’s C3k2 blocks with cheaper depthwise-separable ones:

• DSConv — splits a regular convolution into a per-channel spatial pass and a 1×1 channel-mixing pass, for a fraction of the compute.
• CSP block — splits the input in half, runs one half through two DSConvs, skips the other, concatenates and mixes — a cheap stand-in for C3k2 with a clean gradient path.
• Backbone — strided ConvBnAct + CSP blocks downsampling 224 → 14, same principle as YOLO11 with lighter blocks.
• Spatial pooling to 4×4 instead of 1×1, so the head keeps a coarse sense of where the pad is, not just whether it’s present.
• Head — a 2-layer FC net predicting confidence + box corners.

283,541 parameters vs. YOLO11n’s 2,624,080 — about a 9× reduction.

Figure — custom student model architecture.

Training

Teacher: YOLO11l, fine-tuned 50 epochs on a Roboflow-labeled dataset. Student loss combines detection loss (BCE on confidence + weighted MSE on box corners) with a feature-distillation loss — matching teacher and student feature maps location-by-location, via a learned adapter, so the teacher’s sense of where the pad is transfers along with whether it’s there. Distillation took about 45 minutes on an RTX 500 Ada.

Results

On held-out test set images, the student localizes the pad well across a range of scales and distances:

Benchmarked on a laptop, 50 runs on a test image:

Roughly 10× less memory and 100× faster inference than the teacher, on the same hardware. On the Pi Zero 2W itself, inference averaged ~180ms — plenty fast for spotting a static landing pad. Testing was done by holding the drone over the platform by hand:

Examples of inference from the Pi:

Left to right: a successful detection, a rejected borderline case, and a correct non-detection.

On-device performance was noticeably shakier than on the laptop test set — a mismatch between training data (phone camera, portrait, night) and the Pi’s camera (no IR filter, daylight, different aspect ratio) accounts for most of the gap.