Skip to content
← All projects
View the code on GitHub

EDGE AI · ACM/IEEE SEC 2026

Facial expression recognition, compressed for a 15 W Jetson

CAFE finds the faces in a camera frame and reads the expression on each one, on a 15 W NVIDIA Jetson Orin Nano. In a paper I co-authored for ACM/IEEE SEC 2026, we made the full pipeline 3.33× faster and 2.12× smaller.

View the code on GitHub
A scatter plot of all 88 detector and classifier configurations measured on a Jetson Orin Nano, model size against end-to-end latency, with an arrow from the dense P0 FP32 baseline (228 ms, 17.0 MB) to the P10 FP16 pipeline (69 ms, 8.0 MB).
  • 3.33× faster
  • 2.12× smaller
  • 88 configurations

THE GAP

Most expression models are tested on tidy, pre-cropped faces, not real rooms.

Ambient intelligence means computing that sits quietly in a room and responds to the people in it. Reading facial expressions is a big part of that. But most research scores its models on single faces someone already cropped, while a real camera sees a wide scene full of small, partly hidden faces under shifting light.

The model also has to run all the time on a small board. YOLOv12n is a good fit for edge vision, but even a nano model goes over budget when it never stops, and the usual fix, structured pruning, fails on this architecture.

01

Pre-cropped benchmarks

Scores assume someone already found and cropped every face, so a missed face never counts as an error.

02

Shared memory

A Jetson Orin Nano has 7.4 GB of shared RAM for the OS, frame buffers and every model.

03

Pruning breaks

YOLOv12n's area-attention and C3K2 routing blocks make standard pruning tools fail while tracing the model.

WHAT IT DOES

Two small YOLOv12n models in one pipeline, sized for a 15 W board.

CAFE runs two YOLOv12n models back to back: a detector that finds faces in the whole frame, then a classifier that reads the expression on each face. Keeping them separate means each one can be compressed or swapped on its own. It is meant for camera systems that have to work on the device itself, inside tight memory and latency budgets. The paper is joint work with Mohammad Mahruf Mahdi (first author), Syed Rafay Hasan, Terry N. Guo and Syed Ali Asad Rizvi, and I maintain the public artifact.

Model surgery

We replace each opaque C3K2 block with a traceable twin, so a standard dependency-graph pruner can map the model.

Checked before pruning

Each rebuilt block matches its original bit for bit, and the full model matches to floating-point tolerance.

88 pipelines measured

Eight detectors times eleven classifiers, including five outside baselines, measured on a desktop GPU and a Jetson.

A deployment score

EDUS ranks each pipeline by memory and latency as well as accuracy, with weights set from stated principles.

HOW IT WORKS

How we built and tested it

  1. 1

    Surgery

    Swap each C3K2 block for a traceable twin and confirm the outputs still match.

  2. 2

    Prune

    Remove 10% or 15% of channels with structured pruning.

  3. 3

    Fine-tune

    Retrain at full deployment resolution, straight from the pruned model in memory.

  4. 4

    Compile

    Build FP16 and INT8 TensorRT engines on each target device.

  5. 5

    Measure

    Run all 88 pipelines end to end on 20 real multi-face scenes.

  6. 6

    Rank

    Score every pipeline with EDUS against memory and latency budgets.

SYSTEM DESIGN

Two decoupled stages, compiled on the device that runs them.

The detector runs once per frame and the classifier once per detected face, so classifier cost grows with the crowd. Both run as TensorRT engines built on the Jetson Orin Nano itself, because engines are not portable between devices.

RESULTS

What compression bought, and what it cost

3.33×

faster end to end on a 15 W Jetson Orin Nano: 228.4 ms to 68.6 ms, about 15 FPS

2.12×

smaller combined model size: 17.0 MB to 8.0 MB

96.7%

of baseline top-1 accuracy kept by the compressed classifier on FER2013

−7.3 pts

end-to-end accuracy on the 20-scene stress set, the price of that speed

A controlled ablation on a desktop GPU showed where the speed came from: TensorRT with FP16 gave the classifier 3.418×, and pruning gave 0.993×.

WHAT IT TAUGHT ME

What it taught me, and where it stops

The lesson I keep is about attribution. Our headline compares a dense FP32 model with a pruned FP16 engine, which changes three things at once. Pulled apart, compilation delivered the speed, pruning delivered the smaller footprint, and the surgery is what made pruning possible at all. File size was a poor guide to memory, too: a 6.8 MB classifier used 391 MB once it ran on the Jetson. The limits are plain. Twenty scenes and 132 faces support rankings, not population-scale accuracy, and every edge number comes from one Jetson Orin Nano.

BUILT WITH

  • Python
  • PyTorch
  • Ultralytics YOLOv12n
  • TensorRT
  • ONNX Runtime
  • NVIDIA Jetson Orin Nano

Like what you see?

Thanks for reading. There's more where this came from.

See all projectsView the code on GitHub