ICML 2025 paper combines spiking networks with vision transformers for event-based detection

Hybrid Spiking Vision Transformer for Object Detection with Event Cameras (ICML 2025, arxiv 2505.07715) proposes a hybrid architecture that pairs the representational power of vision transformers with the energy efficiency of spiking computation. The result is a detection backbone that closes the accuracy gap between spiking and conventional networks while retaining much of the energy advantage.

The tension between accuracy and efficiency

Spiking neural networks have a well-known tradeoff: their inherent sparsity makes them energy-efficient, but they consistently underperform conventional networks on perception tasks. The dominant explanation is that spiking networks have difficulty representing the full range of intermediate activations that conventional networks rely on — the binary spike/no-spike constraint discards information.

Vision transformers, by contrast, are currently the dominant architecture for detection and recognition. They achieve state-of-the-art accuracy on standard benchmarks but require dense computation across all attention heads for every input patch, which translates to high energy consumption.

The hybrid approach in this paper assigns spiking computation to early feature extraction — where sparse, local edge-like features are well-suited to spike representations — and conventional attention to the later stages where global context integration drives accuracy. This division of labor captures the best of both architectures.

Results on event-camera benchmarks

The paper evaluates on three standard event-camera detection benchmarks: Gen1, N-Caltech101, and the 1 Mpx dataset. On all three, the hybrid architecture outperforms spiking-only baselines by a substantial margin and approaches the accuracy of conventional transformer baselines while consuming 40–60% less energy per inference.

The key finding is that the energy cost is dominated by the spiking early stages, which handle the majority of the computation, and the transformer stages remain small relative to a full transformer network. The architecture achieves accuracy from accuracy stages and efficiency from the efficiency stages independently.

Detection of small, fast targets

For the C-UAS application, the most relevant result is performance on the 1 Mpx dataset, which includes small, fast-moving aerial targets recorded with a high-resolution event camera. The hybrid architecture achieves the highest mean average precision of any spiking-inclusive architecture on this benchmark.

Small target detection is structurally harder than detection of large objects: fewer events per target, more ambiguity between target signatures and background clutter, shorter target track history. The architectures that do well here are the ones that matter for the seeker application.

Deployment implications

The paper trains and evaluates on GPU hardware but analyzes the theoretical energy consumption on neuromorphic hardware. The theoretical analysis suggests that on an Intel Loihi 2 or comparable neuromorphic processor, the hybrid architecture would consume roughly 200 mW during inference — within the power budget of a missile-class payload.

The practical demonstration on neuromorphic hardware is left as future work, which is the standard gap in the literature. But the theoretical analysis is grounded in actual chip measurements rather than optimistic projections, which makes it a credible starting point for hardware planning.

---

Hybrid Spiking Vision Transformer for Object Detection with Event Cameras. ICML 2025. arxiv 2505.07715