Mixture-of-experts detector and new benchmark for event-camera object detection

A 2024 paper introduces both a new large-scale event-camera detection benchmark and a mixture-of-experts detector that adapts its architecture to local event density — a dual contribution that advances both evaluation and methods.

Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset (arxiv 2412.06647) combines two contributions: a new benchmark dataset specifically designed to challenge event-camera detectors across a range of object sizes and motion conditions, and a mixture-of-experts (MoE) detector architecture that uses a heat conduction analogy to route information across event representation layers.

The benchmark contribution

The paper introduces a benchmark that extends beyond the widely-used Gen1 and 1 Mpx datasets by including objects across a wider range of scales — from distant small objects that subtend fewer than 10 pixels to nearby large objects that subtend several hundred pixels — and across a wider range of motion speeds.

Standard detection benchmarks evaluate mean average precision (mAP) averaged across the full scale range. This average can obscure failure modes on small, fast objects if performance on large, slow objects is high enough. The new benchmark separately reports accuracy by scale category and speed category, making it possible to evaluate how a detector specifically handles small, fast objects — the case that matters for C-UAS.

The benchmark results reveal a consistent pattern: current event-camera detectors achieve high mAP at the large-object, slow-motion end of the scale-speed distribution but fail significantly at the small-object, fast-motion end. This confirms the existence of an accuracy gap at the conditions most relevant to terminal seeker sensing.

The MoE architecture

The detection architecture uses a mixture-of-experts layer that routes different parts of the event feature map to different specialized processing pathways based on the local event density. High-density regions (many events per pixel per millisecond) are routed to pathways designed for high-event processing; low-density regions (sparse events from distant or slow objects) are routed to pathways that accumulate over longer time windows.

The heat conduction analogy comes from the way information propagates through the event feature map: like heat diffusing from sources in a medium, event information spreads spatially from high-density event clusters to neighboring regions where complementary cues can be found.

Relevance to multi-target scenarios

The MoE architecture’s ability to handle heterogeneous event densities within a single scene is valuable for multi-target scenarios: in a scene with one fast-moving target and several slow-moving targets simultaneously present, the detector can apply appropriate processing to each without forcing a single time-window decision that is optimal for none.

Ground-based C-UAS sensors monitoring a perimeter may encounter multiple simultaneous targets — a salvo attack or simply a busy airspace. A detector that handles mixed event densities gracefully is more practically useful than one optimized only for the single-target case.

Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset. arxiv 2412.06647