Spiking neural networks for fast-moving object detection close the gap with conventional CNNs

The most natural computational partner for an event camera is a spiking neural network. Both the sensor and the network operate on sparse, asynchronous discrete events rather than dense synchronous frames. The event camera fires pixels when local contrast changes; the SNN fires neurons when accumulated input crosses a threshold. In principle, the two should compose into a processing pipeline that is more efficient than an ANN processing a frame-based representation of the same data — because the SNN only performs computation where and when the sensor provides data.

Arxiv 2403.10677 makes this argument concrete for fast-moving object detection. The paper trains SNN architectures on event camera datasets of fast objects and benchmarks them against equivalent ANN approaches in terms of detection accuracy, power consumption, and latency.

Architecture choices

The SNN architectures in the paper use leaky integrate-and-fire (LIF) neurons as the basic unit. The key design choices are the membrane time constants — how quickly a neuron’s accumulated charge decays between spikes — and the connectivity structure. For detection tasks, the authors find that deeper networks with short time constants outperform shallower networks on fast targets, because the short time constants allow the network to resolve rapid changes in the event stream without the activation smearing that occurs when the membrane integrates over too long a window.

The result that stands out: on targets moving at angular rates above 200 degrees per second, the SNN approach matches or exceeds the detection rate of an equivalent ANN approach while consuming 4-8x less energy. At lower angular rates the gap closes. This is consistent with the intuition that SNNs are most advantageous when the data is sparsest — at high motion rates, most of the events come from the target and the background event rate is relatively low, playing directly to the SNN’s sparse-compute advantage.

Energy consumption matters at the system level

An interceptor has a tight power budget. Every milliwatt spent on computation is a milliwatt not available for navigation, propulsion, or the fuze. An 8x reduction in inference energy relative to an ANN approach is not a minor optimization — it potentially changes what is achievable within a fixed power envelope.

The paper reports measurements on Intel’s Loihi 2 neuromorphic processor, which is the most mature commercial neuromorphic hardware available. Loihi 2’s SNN inference is genuinely more energy-efficient than a GPU or even a typical embedded processor for sparse SNN workloads. The question for airborne deployment is whether Loihi 2’s power, weight, and reliability characteristics are compatible with a seeker environment. At the moment, neuromorphic processors are research hardware — they’ve been through a few iterations of the chip but not through qualification processes for defense-relevant environments.

The alternative is implementing the SNN on an FPGA, where the engineer controls the memory architecture and can optimize for the specific sparsity pattern of the target detection workload. FPGA implementations of LIF networks have been demonstrated at lower absolute efficiency than Loihi 2 but with much better environmental qualification paths. That’s the more deployable near-term option.

Training challenges

The main practical challenge with SNNs is training. The spike mechanism is not differentiable, which breaks standard backpropagation. The paper uses surrogate gradient methods — replacing the spike function with a smooth approximation during the backward pass — which works but requires careful tuning of the surrogate function shape and the learning rate. The authors note that their SNN models required significantly more training time than equivalent ANNs, which is a real engineering cost.

Quantized ANNs running on fixed-point embedded hardware may ultimately be more practical than SNNs for the terminal guidance problem in the near term, but the energy efficiency results from this paper suggest that as neuromorphic hardware matures, the SNN path deserves continued investment. The physics of the sensor and the physics of the computation are well-matched.