Neuromorphic driver distraction detection demonstrates embedded SNN deployment on event cameras

Spiking-DD: Neuromorphic Event Camera based Driver Distraction Detection with Spiking Neural Network (arxiv 2407.20633) presents a complete SNN inference pipeline for a real application: detecting driver distraction from an event camera mounted in a vehicle cabin. The paper is notable for demonstrating actual embedded deployment rather than theoretical efficiency analysis.

Why driver distraction is a useful proxy

Driver distraction detection and airborne target detection are not the same problem. But the hardware requirements overlap substantially: both need low-latency classification from event-camera input on an embedded processor with tight power constraints, and both need to operate reliably in uncontrolled lighting conditions.

Driver distraction detection is an easier classification problem (the possible states are limited and the sensor has a fixed mounting geometry), but it is a real application with real hardware constraints. Papers that demonstrate actual embedded deployment on this problem are more credible benchmarks for hardware capability than papers that simulate deployment.

The SNN implementation

Spiking-DD implements a multi-layer spiking neural network in which the first layers extract spatial features from the per-pixel event stream and later layers integrate features over time to classify behavior. The network architecture is similar to a standard CNN but with binary activations (spikes) rather than continuous activations.

The network runs on a Loihi 2 neuromorphic chip. The paper measures actual power consumption during inference (not estimated from model size): 220 mW during peak inference, dropping to below 50 mW during idle periods when the scene is static and few events arrive.

The 50 mW idle power is particularly significant: it demonstrates that an SNN on neuromorphic hardware genuinely exploits input sparsity at runtime, not just in theory. A conventional CNN on a GPU would draw near-peak power even during idle periods because it processes frames regardless of content.

Latency measurements

End-to-end latency from event to classification output is measured at under 2 ms. This includes event streaming, network inference, and classification output. The 2 ms latency is substantially below what frame-based systems achieve (minimum one frame period, typically 16–33 ms) and is compatible with real-time use in safety-critical applications.

For terminal guidance, a 2 ms classification latency is acceptable — a seeker operating with a 10 ms guidance loop has time to run inference and update the guidance command within a single loop iteration.

Transferability lessons

The Spiking-DD paper is one of the few examples of an SNN evaluated on real hardware with real measured power and latency. The lessons it offers for other application domains are:

1. Loihi 2 is capable of running real event-camera inference tasks at useful accuracy
2. The idle power scaling with input event rate is genuine and measurable
3. 220 mW during inference is the bottom of the practical range; more complex networks will draw more

These measured numbers provide a basis for estimating what is achievable in a seeker package running more complex detection and tracking networks on similar hardware.

---

Spiking-DD: Neuromorphic Event Camera based Driver Distraction Detection with Spiking Neural Network. arxiv 2407.20633