# Design and Validation of a U-Net-Based Algorithm for Star Sensor Image Segmentation
## 2 Materials and Methods
### Dataset Creation and Input Data Preprocessing  
The dataset created and used for the U-Net training phase is provided at [31]. It is composed of 5600 squared monochromatic jpeg images and 5600 associated jpeg masks. All of them have a size of 512 \(\times\) 512 px and a bit depth of 8. Masks' pixels can have one of two possible values: 0 for background and 255 for foreground pixels. The samples for the training and validation phases are the first 5300 ones. They are generated from a batch of 600 images to which 180\({}^{\circ}\)clockwise rotation, 90\({}^{\circ}\) clockwise  
Figure 1: U-Net model with features dimensions and number of features [29]. rotation, added noise, increased stray light, blur effects, increased luminosity, contrast and further modifications have been applied. This original batch is made of 300 samples coming from several night sky acquisition campaigns and 300 samples randomly acquired using Stellarium [32] software in night sky conditions considering different FOV and attitudes (Figure 2). Acquisition campaigns were performed using several kinds of cameras and objectives: ZWO ASI 120 MM-S camera with default optics, reflex Nikon D3100 equipped with a Nikkor 18-105 mm and a ProLine PL16803 camera coupled with an Officina Stellare RiFast 400 telescope. Last. 300 dataset images were obtained with a high fidelity star tracker image simulator (HFSTIS). It is used to simulate realistic night sky images, simulating all the physical, functional and geometrical characteristics of the star sensor, along with all the instrumental and environmental noises [33; 34; 35; 36; 37]. Simulator images were used only in the test phase to provide independent samples the network has never been trained on to monitor the generalization capability of the trained network. The dataset contains images of night sky where stars are the main actors but where planets, light pollution caused stray light noise, SEUs, clouds, airplanes' streaks, satellites, comets, nebulae and galaxies appear as well. Masks were obtained using specific thresholds for different groups of images from different campaigns, sensors and software. In particular, mask creation was carried out using Adobe Photoshop CC 2019 with the scope of obtaining just the most salient objects in the FOV and filtering all the noise and undesired objects cited above. An example of a dataset image and mask is provided in Figure 3, where it is clearly visible that just the brightest points appear inside the mask. Input images and mask preprocessing is performed in order to organize these data in two float 4D tensors for NN training and testing. Every image is converted into floating point arrays and normalized using its maximum value to carry out an image adaptive normalization process; every most significant image pixel will have an associated value close to 1. This will help the network detect the most significant pixels and make the learning process easier for the net by constraining the interval of signal values between 0 and 1. Moreover, this normalization process improves the algorithm capability of working with images that have different range of energy levels. The same process applies for mask data vectorization and normalization, with the only difference being that the normalizing factor is constant and equal to 255 for every mask. Figure 2: Dataset samples: real night sky image (on the **left**) and Stellarium-generated night sky (on the **right**). It must be pointed out that the dataset does not contain only point-like stars but also even images where they appear as streaks due to the angular speed of the sensor. Considered angular speeds are within the \([0^{\circ}/s,1^{\circ}/s]\). This allows the proposed AI-based algorithm to work properly even in situations where the star sensor is rotating with respect to the fixed stars or where a sidereal pointing of the camera occurs together with RSO passages. Concerning the noise, 5300 images for training and validation are generated from a batch of 300 images.