ESVO2 achieves direct visual-inertial odometry with stereo event cameras

ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras (arxiv 2410.09374) presents the second generation of the Event Stereo Visual Odometry system, incorporating IMU fusion and improving robustness under the aggressive motion conditions that make event cameras attractive in the first place.

What makes ESVO2 different from its predecessor

The original ESVO (Event Stereo Visual Odometry) demonstrated stereo event-based depth estimation and odometry but performed poorly at high angular rates because its frame-to-map alignment step — which compares incoming events to a maintained 3D map — became unreliable when many events arrived simultaneously from rapid motion.

ESVO2 addresses this with two changes: IMU integration provides a motion prior that constrains the alignment step during high-rate periods, and the depth estimation pipeline runs asynchronously at the event rate rather than at a fixed cadence that can fall behind the event rate.

Direct depth estimation

The “direct” in the title refers to direct stereo event matching — rather than computing intermediate depth maps by matching event features, the pipeline directly estimates depth by finding the disparity that minimizes the difference between time-surface representations from the two camera heads.

This approach is more computationally intensive than indirect feature matching but more robust: it does not require reliable feature extraction from the event stream, which can fail when the stream is noisy or the features are poorly localized.

Outdoor evaluation

Most event-camera VIO evaluations use indoor handheld datasets (MVSEC, TUM-VIE). ESVO2 includes outdoor evaluation in natural lighting with wind-induced vegetation motion that generates background events. The outdoor results are significantly worse than indoor results for all methods evaluated, and ESVO2 shows smaller degradation than baselines on the outdoor sequences.

The wind vegetation problem is directly relevant to low-altitude C-UAS sensing: sensors looking into a tree line or past vegetation will see substantial background events from plant motion in wind. Methods that were not evaluated in this condition may show surprising degradation when deployed.

Stereo geometry at seeker-relevant ranges

The paper evaluates at ranges of 5–50 meters with a 12 cm stereo baseline. Depth accuracy at 50 meters with this baseline is approximately 0.5 meters (1% of range), which is acceptable for mid-flight navigation but is marginal for terminal seeker accuracy requirements.

Increasing the baseline to 30 cm would improve depth accuracy to roughly 0.2 meters at 50 meters, which is more useful for terminal guidance. The integration challenge is packaging a 30 cm stereo head into a seeker that must fit within an interceptor’s diameter constraint.

This geometry tradeoff — stereo baseline vs. package size vs. depth accuracy at target range — is a fundamental design parameter for any event-camera-based seeker concept, and ESVO2’s outdoor results provide useful data points for trading it.

---

ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras. arxiv 2410.09374