We present the first annotated benchmark datasets for evaluating free-flyer visual-inertial localization and mapping algorithms in a zero-g spacecraft interior. The datasets were collected by the Astrobee free-flying robots that operate inside the International Space Station (ISS). Space intra-vehicular free-flyers face unique localization challenges: their IMU does not provide a gravity vector, their attitude is fully arbitrary, and they operate in a dynamic, cluttered environment. We extensively evaluate state-of-the-art visual navigation algorithms on these challenging Astrobee datasets, showing superior performance of classical geometry-based methods over recent data-driven approaches. The datasets include monocular images and IMU measurements, with multiple sequences performing a variety of maneuvers and covering four ISS modules. The sensor data is spatio-temporally aligned, and extrinsic/intrinsic calibrations, ground-truth 6-DoF camera poses, and detailed 3D CAD models are included to support evaluation.

Our datasets consist of various intra-vehicular activities of Astrobee free-flying robots currently operating on the ISS between May 13, 2019 and July 14, 2022. The datasets are collected inside the largest single ISS module, the Japanese experiment module (JEM) nicknamed Kibo, and contain ground-truth 6-DoF camera poses.

We also offer detailed 3D CAD models for evaluation. You can download the 3D CAD models for the JEM, USLab, and Node 2 modules. For more CAD models of various modules, please refer to the original source link

We provide each Astrobee sequence in the TUM RGB-D format in the following format:

• description.yaml: a text file containing an overall description of the sequence (the name of the robot used, the original rosbag file, and the date of recording, etc.)
• gray/: a folder containing all undistorted gray images (PNG format, 1 channel, 8-bit per channel)
• gray_raw/: a folder containing all raw (distorted) gray images with FOV lens distortion
• gray.txt: a text file with a consecutive list of all gray images (format: timestamp filename)
• undistorted_calib.txt: a text file containing camera intrinsic parameters for undistorted gray images (format: fx fy cx cy)
• distorted_calib.txt: a text file containing camera intrinsic parameters for distorted gray images with FOV lens distortion coefficient (format: fx fy cx cy w)
• imu.txt: a text file containing the timestamped gyro and accelerometer measurements expressed in IMU body frame (format: timestamp wx wy wz ax ay az)
• groundtruth.txt: a text file containing the ground-truth 6-DoF camera poses for all gray images stored as the timestamped translation vector and unit quaternion expressed in ISS world coordinate frame (format: timestamp tx ty tz qx qy qz qw)

The Astrobee free-flying IVA robots used for dataset collection. This is the positions of the sensors and the rigid body transformations that link them.

• NavCam provides monocular image sequences at 5 Hz.
• IMU sensor is logged at 100Hz and expressed in IMU body cam.

@article{kang2024astrobee,
         title = {Astrobee ISS Free-Flyer Datasets for Space Intra-Vehicular Robot Navigation Research},
         author = {Kang, Suyoung and Soussan, Ryan and Lee, Daekyeong and Coltin, Brian and Vargas, Andres Mora and Moreira, Marina and Hamilton, Kathryn and Garcia, Ruben and Bualat, Maria and Smith, Trey and others},
         journal = {IEEE Robotics and Automation Letters},
         year = {2024},
         publisher={IEEE}
        } 

We thank the ISS program and astronauts for their support of Astrobee operations. This work was funded in part by the Game Changing Development Program in the NASA Space Technology Mission Directorate. This work was authored by employees of KBR Wyle Services, LLC under Contract No. 80ARC020D0010 with the National Aeronautics and Space Administration. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to reproduce, prepare derivative works, distribute copies to the public, and perform publicly and display publicly, or allow others to do so, for United States Government purposes. All other rights are reserved by the copyright owner. Pyojin Kim was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2021R1F1A1061397).