Asynchronous visual-inertial odometry for event cameras sets a new performance bar in ISPRS 2024

Most event-camera processing pipelines introduce a temporal binning step: accumulate events over a fixed window, convert to a pseudo-image, and process with a conventional frame-based algorithm. This is understandable — it lets you reuse frame-based architectures — but it discards the property that makes event cameras useful. If you bin events into 10 ms windows, you’ve given yourself 10 ms of temporal resolution regardless of what the sensor is capable of.

An ISPRS 2024 paper on asynchronous visual-inertial odometry takes the opposite approach: process events as they arrive, with no binning. Each event is incorporated into the state estimate at its exact timestamp. The IMU integration runs continuously between events. The result is a VIO system that operates at the native time resolution of the event camera rather than at an artificially imposed frame rate.

Why asynchronous matters for VIO

In standard VIO, camera observations and IMU measurements alternate: you process a frame, then pre-integrate the IMU until the next frame, then process the next frame. The camera update is a batch correction to an accumulated IMU error. If the camera frame rate is low — 30 or 60 Hz — the IMU integrates for 16 to 33 ms between corrections, during which it drifts.

An asynchronous event VIO eliminates this structure. Events arrive at high frequency — typically tens of thousands per second during motion — and each event can trigger a lightweight state update. The state is being corrected continuously rather than in discrete jumps. For high-angular-rate maneuvers, this matters: 500 degrees per second angular rate over a 16 ms frame period is 8 degrees of rotation, which is a large correction to apply all at once and may exceed the linearization accuracy of the update step.

The ISPRS paper demonstrates that asynchronous event VIO maintains lower drift during aggressive maneuvers than frame-synchronized event VIO, and dramatically lower drift than frame-camera VIO on the same sequences. The improvement is most pronounced at angular rates above 200 degrees per second.

The computational challenge

Asynchronous processing sounds computationally expensive, but the key insight is that each individual event generates a very small amount of information. The per-event update can be made lightweight — a single-event filter update rather than a full bundle adjustment — at the cost of some accuracy relative to a batch approach. The ISPRS paper uses a factor graph backend with incremental updates, which allows new events to be incorporated efficiently without reprocessing the entire history.

Even so, the system in the paper runs on a desktop-class processor and is not real-time on an embedded target. Bringing asynchronous event VIO into an embedded envelope requires either hardware acceleration of the factor graph operations or a simplified state representation that trades accuracy for compute.

The FPGA path is attractive here: the factor graph structure is regular enough that a custom fixed-function accelerator could handle the per-event update in a few microseconds, which would allow the system to process events at their natural arrival rate on hardware compatible with an airborne platform.

Navigation in contested environments

For terminal guidance specifically, the value of low-drift asynchronous VIO is in contested GPS environments. Commercial small UAS are already carrying GPS jammers, and the next generation of threats will include electronic countermeasures specifically targeting interceptor guidance. A seeker that degrades gracefully in a GPS-denied environment — using event-camera VIO to maintain its own navigation solution during terminal approach — is more robust than one that depends on GPS all the way to intercept.

The ISPRS 2024 result establishes that the navigation accuracy needed for guidance is achievable from an event camera and IMU combination, even in high-angular-rate conditions. The remaining work is packaging that capability into an airborne-qualified hardware envelope and verifying the accuracy against the guidance requirements for a specific interceptor class.