SMamba applies adaptive sparsification to state-space models for event-camera processing

SMamba: Adaptive Sparsification in State-Space Models for Event-Camera Processing (arxiv 2501.11971) extends the Mamba state-space model architecture — which has shown strong performance on sequential data — with an adaptive mechanism that exploits the natural sparsity of event camera outputs.

Why state-space models for events

The event camera produces a stream of timestamped measurements: (x, y, t, polarity) for each brightness change event. This is a variable-rate, asynchronous sequence — the rate varies from near-zero in static scenes to millions of events per second in fast motion.

Recurrent models (LSTMs, GRUs) process sequences step by step and have limited ability to capture long-range dependencies. Transformers capture long-range dependencies but scale quadratically in sequence length, which is prohibitive for the long, high-rate sequences that event cameras produce.

State-space models, including Mamba, offer a middle ground: linear-time scaling with sequence length and good long-range dependency modeling through the state representation. They are becoming the preferred architecture for long-sequence problems across multiple domains, and event-camera processing is a natural application.

Adaptive sparsification

The key innovation in SMamba is an adaptive mechanism that skips state updates when incoming events carry little new information. In periods of low scene activity, consecutive events in a small spatial neighborhood are often redundant — they all report the same local motion and carry overlapping information. Processing each event individually wastes computation.

SMamba learns to identify which events are informative and which are redundant, updating the model state only for the informative ones. This reduces computation during low-activity periods while maintaining full resolution during high-activity periods (fast target motion, rapidly changing scene).

The adaptive mechanism is trained end-to-end: the sparsification decisions are differentiable (using a soft gating function) and the model learns to make them jointly with learning the task itself.

Benchmark results

On object recognition and optical flow benchmarks, SMamba achieves accuracy comparable to dense state-space baselines at 60–70% of the computation cost. The efficiency advantage is larger on sequences with variable activity levels — where the adaptive mechanism can skip more computation during quiet periods.

For a seeker application, the operating scenario has exactly the variable activity profile that benefits most from adaptive sparsification: long periods of background-only observation interspersed with brief, high-rate events when a target appears and is tracked.

Integration with spike-based hardware

One attractive property of SMamba’s adaptive sparsification is that it produces naturally compatible output for spiking hardware: the skipped events become non-firing time steps in the state update, which is the same operation that spiking neurons perform. This suggests a pathway toward implementing SMamba on neuromorphic hardware with minimal modification to the core computation.

The paper does not demonstrate neuromorphic hardware implementation, but the architectural compatibility is noted and is a direction that subsequent work is likely to pursue.

---

SMamba: Adaptive Sparsification in State-Space Models for Event-Camera Processing. arxiv 2501.11971