Survey of event-based object detection for autonomous driving shows transfer potential to aerial sensing

Deep Event-based Object Detection in Autonomous Driving: A Survey (arxiv 2405.03995) covers the event-camera detection literature as applied to the autonomous driving problem — a use case that shares several features with aerial sensing: fast relative motion, challenging lighting, and real-time detection requirements on embedded hardware.

Why autonomous driving is a useful reference domain

Autonomous driving and C-UAS seeker sensing are different applications, but the sensor physics and engineering requirements overlap. Both involve an event camera on a moving platform observing targets that are also moving. Both require fast, reliable detection that fails gracefully under sensor degradation. Both must run on embedded hardware in a physically constrained package.

The autonomous driving literature has more research resources and a richer dataset ecosystem than the defense sensing domain. Tracking which methods work best in the autonomous driving context and why transfers insight to the seeker design problem without requiring parallel research investment.

Key findings from the survey

The survey identifies several findings that are broadly applicable:

Detection networks trained on event data outperform retrained frame networks by 15–25% on fast-moving targets. Methods that use event-specific network architectures (those that preserve temporal event structure rather than converting to frames) consistently outperform methods that convert events to pseudo-frames and run standard CNN detectors. The conversion discards temporal structure that is useful for fast-target detection.

Recurrent architectures handle temporal context better than single-window approaches. Detectors with recurrent components (RNNs, recurrent transformers) that maintain state across time windows outperform stateless single-window detectors on targets with variable speeds, because the recurrent state provides context about prior target motion.

Label quality matters more for events than for frames. Event-camera training data annotation is harder than frame annotation because events do not produce a clearly legible visual representation. Noisy or imprecise labels degrade event-based detectors more than frame-based detectors because the temporal precision of the data makes it more sensitive to label timing errors.

The sim-to-real gap

The survey discusses the sim-to-real gap for event cameras specifically. Synthetic event generation from frame video (using simulators like ESIM or V2E) produces event data that matches real sensor output in some conditions but diverges in others — particularly in the noise characteristics (hot pixels, threshold variation) that real sensors exhibit.

Methods trained exclusively on synthetic data show measurable accuracy degradation on real sensor data. This sim-to-real gap is relevant to any effort to build large training datasets using simulation rather than field collection.

---

Deep Event-based Object Detection in Autonomous Driving: A Survey. arxiv 2405.03995