Data for "Image-based Backbone Reconstruction for Non-Slender Soft Robots"

# Data for "Image-based Backbone Reconstruction for Non-Slender Soft Robots"

This dataset provides the data for the forthcoming paper 
"Image-based Backbone Reconstruction for Non-Slender Soft Robots".
The backbone reconstruction method used is based on the method described in Hoffmann et al. [1]. The modifications to this method to support the non-slender soft robot in this dataset are described in the forthcoming paper mentioned above.
This dataset holds raw images of pressurized and elongated soft robots and the corresponding reconstructed backbones.

## Dataset

The dataset is split into two subsets with similar structure. The first subset is contained in `dataset_01`. The second dataset is contained in `dataset_02`.

Each subset consists of five folders and one schedule file.
The schedule file `schedule.csv` contains the index of the schedule entry, the angle $\alpha$ in degree, the pressure of each chamber $p_1$ to $p_3$ in bar and if the pressurization is active.
Furthermore, the five folders of the subset can be described as follows

- `raw`: Contains the raw cropped images. The filenames are formatted as `CROPPED_C{CAMERA_INDEX}_E{SCHEDULE_ENTRY}.png` with the camera index `CAMERA_INDEX` and the schedule entry `SCHEDULE_ENTRY`.

- `constant_curvature_slender`, `constant_curvature_volumetric`, `cubic_curvature_slender` and `cubic_curvature_volumetric`. These folders contain the actual reconstructed backbones based on the raw data from the `raw` folder. A different reconstruction approach was used in each of these folders
    - `constant_curvature_slender` - A constant curvature backbone kinematic based on the slender model,
    - `constant_curvature_volumetric` - A constant curvature backbone kinematic based on the volumetric model,
    - `cubic_curvature_slender` - A cubic curvature backbone kinematic based on the slender model,
    - `cubic_curvature_volumetric` -  A cubic curvature backbone kinematic based on the volumetric model.

    Each of these folders contain a `data` and `figures` folder. The data folder consists of `PARAMETER_E{SCHEDULE_ENTRY}.json` files listing the optimization parameters for each schedule entry `SCHEDULE_ENTRY` in the JSON format.
    The `figures` folder contains annotated images of the reconstructed backbone on the cropped raw images. The filenames are structured `ANNOTATED_E{SCHEDULE_ENTRY}_C{CAMERA_INDEX}_EPOCH{EPOCH}.png` with the schedule entry `SCHEDULE_ENTRY`, the camera index `CAMERA_INDEX` and the epoch `EPOCH` of the optimization algorithm.

The optimization parameters include the base position `base_position` of the reconstructed backbone in world coordinates, the coefficients for the curvature polynomials `ux` and `uy`, and the constant coefficient for the elongation polynomial `la`.

## Calibration Data

The calibration data is located in the `calibration` folder and consists of multiple `.npy` files in the numpy format. The corresponding camera index for the calibrated camera is abbreviated with `CAMERA_INDEX` in the following:

- `C{CAMERA_INDEX}.npy` - Stores the reprojection error, camera matrix, distortion coefficients, rotation, and translation vectors as returned by the `cv2.calibrateCamera` [2] method. 
- `C{CAMERA_INDEX}_camera_matrix.npy` - Stores the camera_matrix as returned by the `cv2.calibrateCamera` [2] method. 
- `C{CAMERA_INDEX}_distortion_coefficients.npy` - Stores the distortion coefficients as returned by the `cv2.calibrateCamera` [2] method. 
- `C{CAMERA_INDEX}_projection_matrix.npy` - Stores the projection matrix from world space to pixel space based on the stereo camera calibration.
- `STEREO.npy` - Stores the reprojection error, R, T, E, F as returned by the `cv2.stereoCalibrate` [2] method as an object datatype. 

## Acknowledgement

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 501861263 – SPP2353

## References

[1] M. K. Hoffmann, J. Mühlenhoff, Z. Ding, T. Sattel and K. Flaßkamp. An iterative closest point algorithm for marker-free 3D shape registration of continuum robots. arXiv.
https://arxiv.org/abs/2405.15336

[2] OpenCV. Camera Calibration and 3D Reconstruction. OpenCV Documentation. https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html, accessed May 27, 2024.

Introducing paper: Image-based backbone reconstruction for non-slender soft robots