DEIO: deep event inertial odometry integrates events and IMU for robust state estimation

DEIO: Deep Event Inertial Odometry (arxiv 2411.03928) presents a learned visual-inertial odometry system that fuses event-camera measurements with IMU data using a neural architecture rather than classical filter design. The result is robust pose estimation during the high-angular-rate maneuvers that most visual-inertial systems fail on.

The VIO problem for event cameras

Visual-inertial odometry (VIO) is the combination of camera-based visual measurements with inertial measurement unit (IMU) data to estimate a moving platform’s position and attitude. It is the core state estimation component in mobile robots, drones, and any platform that needs to navigate without GPS.

Classical VIO systems extract sparse visual features from camera frames and track them between frames to estimate inter-frame motion. The IMU provides high-rate measurements between frames and constrains the feature motion with physical dynamics. The combination is more robust than either alone.

Event cameras improve the visual component of VIO significantly: they do not blur during fast motion, and they continue to produce useful measurements at angular rates that render frame-camera feature tracking unreliable. But integrating event data into the classical VIO framework is non-trivial because event data is asynchronous and does not fit the frame-synchronous feature-tracking model.

The learned fusion approach

DEIO replaces the classical feature tracking pipeline with a learned component: a neural network takes raw event data and IMU measurements as inputs and produces motion increments and uncertainty estimates as outputs. The network is trained on sequences with ground truth motion, learning to extract the relevant motion information from the joint event-IMU stream without explicit feature matching.

The advantage over classical approaches is robustness: the learned component can handle event rate variations, sensor noise, and motion patterns that would require manual re-tuning in a classical filter.

Results in high-dynamic scenarios

The paper evaluates on sequences with aggressive motion — angular rates above 500 deg/s and linear accelerations above 5g — that are too fast for frame-camera VIO to handle. On these sequences, DEIO maintains translational drift below 2% of total distance traveled while classical VIO methods diverge.

The 5g acceleration result is particularly relevant: interceptor seekers experience launch accelerations significantly above this, followed by sustained lateral acceleration during homing maneuvers. Seeker state estimation must work reliably through the full acceleration profile.

Limitations and open questions

DEIO is trained and evaluated on ground and aerial robotic scenarios with targets at close range (10–50 meters) and moderate closing speeds. The terminal guidance scenario involves much higher closing rates and shorter engagement windows.

The neural approach in DEIO in principle generalizes to any motion profile given appropriate training data, but collecting training data at terminal seeker dynamics requires either expensive captive-carry flight tests or high-fidelity simulation that can generate realistic synthetic event data. Neither is currently straightforward.

The gap between DEIO’s demonstrated performance and seeker requirements is real — but the architectural approach (learned event-inertial fusion) is likely the right one, and the remaining work is primarily data collection and regime extension.

---

DEIO: Deep Event Inertial Odometry. arxiv 2411.03928