Temporal-guided spiking networks advance event-based action recognition

Temporal-Guided Spiking Neural Networks for Event-Based Human Action Recognition (arxiv 2503.17132) proposes a training and architecture modification that improves the temporal representation of spiking networks on event-camera data. The primary application is human action recognition, but the temporal processing technique is relevant to any classification task where target behavior over time matters.

The temporal representation problem in SNNs

Spiking neural networks process time differently than conventional networks. In a conventional network, the temporal dimension of a video or event sequence is handled through RNN/LSTM units or temporal convolutions that explicitly model time steps. In an SNN, time is encoded implicitly in the spike timing — when a neuron fires relative to its inputs encodes information that conventional activations cannot represent.

The challenge is that current SNN training methods often fail to learn effective temporal representations. The surrogate gradient methods used to train SNNs approximate the true gradient through the spike function, and this approximation tends to under-utilize the temporal coding capacity of spiking neurons, producing networks that work by rate coding (how often neurons fire) rather than temporal coding (when they fire relative to each other).

Temporal coding is more efficient — it carries information at lower average spike rates — but harder to learn.

The temporal guidance approach

The paper introduces a teacher-student mechanism during training: a conventional transformer network with full access to temporal context (the “teacher”) guides the SNN’s learning toward temporal representations that carry more information per spike. The SNN (the “student”) is trained to match the temporal patterns of the teacher network, not just the output labels.

This guidance steers the SNN toward temporal coding without requiring the SNN to solve the full temporal representation problem from scratch. The result is networks that carry more information per spike and perform comparably to teacher networks on several benchmarks.

Implications beyond action recognition

The application in this paper is recognizing human activities from event cameras: walking, running, clapping, waving. The event camera’s ability to capture subtle high-speed motion — finger flicks, rapid weight shifts — makes it better than frame cameras for fine-grained action distinction.

For the C-UAS problem, the analogous task is target behavior classification: hovering vs. transit flight vs. approach maneuver, or distinguishing rotor signatures that indicate a specific drone class. These are temporal classification tasks over event streams, and the temporal-guidance SNN approach would be directly applicable.

The difference is that drone behavior events unfold at lower angular rates than human gestures at comparable distances, but over longer time horizons. Whether the temporal coding benefits transfer across this difference in dynamics is an interesting question for follow-on work.

Efficiency comparison

The temporal-guided SNN achieves accuracy within 2% of the teacher transformer network while consuming roughly 5x less energy per inference. This efficiency margin, combined with the sparse event input, makes the approach attractive for embedded sensing where both power and compute are tightly constrained.

---

Temporal-Guided Spiking Neural Networks for Event-Based Human Action Recognition. arxiv 2503.17132