Synthetic event camera data for spacecraft rendezvous shows domain transfer that generalizes to other terminal problems

One of the persistent bottlenecks in event-camera development is data. Training detection and tracking networks requires labeled data — images with ground truth target positions, orientations, and velocities. For conventional cameras this is manageable: you can collect data at a range, label it manually, and augment with rendering from 3D models. For event cameras the situation is harder. Event cameras are less common, the data format is unfamiliar to labelers, and the relationship between target motion and the event pattern depends on the scene geometry and the sensor’s bias settings in ways that don’t transfer trivially between setups.

Synthetic data generation is the obvious solution. If you can simulate the physics of event generation accurately enough, you can produce labeled datasets for any target class and any flight condition by simulating the scenario in a renderer. The question is whether networks trained on synthetic data generalize to real sensor data — the sim-to-real transfer problem.

A 2025 paper from ScienceDirect addresses this for spacecraft rendezvous operations. The context is spacecraft proximity operations — a spacecraft approaching a station or another vehicle — where the target is a rigid object with known geometry undergoing relative motion at known velocities. The paper develops a synthetic event data generation pipeline, trains detection and pose estimation networks on synthetic data, and evaluates their performance on real event camera data collected at a ground test facility.

Why spacecraft rendezvous and C-UAS are similar problems

The surface similarity is low: a space station and a commercial drone look completely different. But the underlying sensing problem has similar structure. In both cases, the target is a rigid object at known (approximate) size, approaching on a closing trajectory, and you need to estimate its position and attitude accurately enough to support a guidance or docking maneuver. The background is complex — stars and earth limb for the spacecraft case, sky clutter and ground features for the UAS case — and the relative motion is fast enough that frame cameras produce motion blur at the relevant closing velocities.

The sim-to-real transfer result from the spacecraft paper is encouraging. The authors find that networks trained on synthetic event data generalize to real data with relatively small accuracy penalties, provided the simulation includes sufficient noise modeling — specifically, refractory noise and leaked events that appear in real sensors but are absent from naive physics-based event simulation. The lesson for C-UAS is direct: if you have good 3D models of target drones and an accurate event simulation pipeline, you can generate training data for any target class at any flight condition without collecting real flight data.

Simulation pipeline

The paper’s simulation pipeline converts a rendered 3D scene into events using the ESIM simulator and adds realistic noise based on characterization of the specific sensor used in the experiment. The sensor characterization step — measuring the noise parameters of the real hardware — is essential; without it the sim-to-real gap is larger and the transfer quality degrades.

For a C-UAS application, this pipeline would use 3D models of the target UAS classes of interest (Group 1–3), simulate them at realistic engagement geometries and velocities, and generate the training dataset. The pipeline is open-source and the simulation is fast enough to generate large datasets on modest hardware.

The data deficit problem

Events happen at closing velocities between an interceptor and a target UAS that are difficult to replicate in a ground test. A closing velocity of 50 meters per second at 100 meters range produces an angular crossing rate that requires large, expensive range infrastructure to test. Synthetic data generation lets you create arbitrarily large datasets at these conditions without range time. If the sim-to-real transfer holds — and the spacecraft paper suggests it does with proper noise modeling — this is a significant acceleration of the development timeline.

The remaining uncertainty is whether the sim-to-real transfer that works for spacecraft (near-uniform background, controlled illumination) also works for C-UAS (complex sky clutter, variable illumination, multipath from ground). That validation is something that can only be done empirically, which brings us back to needing real flight data for calibration even if synthetic data handles most of the training corpus.