# Self-Supervised Learning across the Spectrum
## 6 Experiments and Results
### Implementation Details  
**Preprocessing:** We preprocess data using mean-std standardization using values from the fine-tuned dataset. We preprocess both the pretrain and fine-tuned datasets. For optical images, we use only the RGB channels in our experimentation rather than the 10-12 multispectral channels to be consistent with the prior self-supervised baseline approaches that were designed for 3-channel images [33]. **Training:** Across experiments, we set \(\lambda\) (the joint hyperparameter weighting \(\mathcal{L}_{c}\) and \(\mathcal{L}_{r}\)) to be \(10^{-2}\) when pretraining. Additionally, we split our original datasets into the train, validation, and test splits. For PASTIS-R, we use folds 1, 2, and 3 for training, fold 4 for validation, and fold 5 for testing as specified by the authors in [43]. For Africa Crop Type Mapping, we use the original partitions specified by the curators of the dataset for training, validation, and testing [58]. We run each segmentation model on the validation set after every epoch during training; at test time, we evaluate using the model checkpoint that attains the highest validation IoU. We train all models on a 2 \(\times\) NVIDIA A100 GPUs. Pretraining takes approximately 3 h and fine-tuning takes around 1.5 h. We use the Adam optimizer [64] with a learning rate of \(10^{-3}\). In our evaluation of the Africa Crop Type dataset, due to high class imbalance and irrelevance of predicting the background class, we ignore the background class when computing the mIoU score. \begin{table}
\begin{tabular}{l r} \hline
**Class Name** & Number of Defects \\ \hline Meadow & 31,292 \\ \hline Soft winter wheat & 8206 \\ \hline Corn & 13,123 \\ \hline Winter barley & 2766 \\ \hline Winter rapeseed & 1769 \\ \hline Spring barley & 908 \\ \hline Sunflower & 1355 \\ \hline Grapevine & 10,640 \\ \hline Beet & 871 \\ \hline Winter triticale & 1208 \\ \hline Winter durum wheat & 1704 \\ \hline Fruits, vegetables, flowers & 2619 \\ \hline Potatoes & 551 \\ \hline Leguminous fodder & 3174 \\ \hline Soybeans & 1212 \\ \hline Orchard & 2998 \\ \hline Mixed cereal & 848 \\ \hline Sorghum & 707 \\ \hline _Void label_ & 35,924 \\ \hline \end{tabular}
\end{table} 
Table 1: Table showing class names and the number of parcels in the PASTIS-R dataset [43]. \begin{table}
\begin{tabular}{l r} \hline
**Class Name** & Number of Defects \\ \hline Groundnut & 59 \\ \hline Rice & 75 \\ \hline Maize & 84 \\ \hline Sorghum & 619 \\ \hline \end{tabular}
\end{table} 
Table 2: Table showing class names and the number of parcels in the Africa Crop Type Mapping South Sudan dataset [14]. **Model Architecture:** We follow the implementation of 3D U-Net provided by _[PERSON]_[9]. The **encoder** takes as input either an optical or radar image. In total, the encoder consists of a total of five 3D convolution layers. In each 3D convolution layer, stride and padding of 1 are used. The input dimension of the first convolution block is modified to either three or ten based on whether the input image is radar or optical. Three-dimensional batchnorm followed by a leaky ReLU activation function is used after all convolution layers. When training on the PASTIS-R crop segmentation dataset, 3D max-pooling layers are used after the 2 nd and 4 th convolution layers. The max-pooling layers use a kernel size of 2, stride of 2, and padding of 0. The multimodal fusion model uses this backbone as well. The **decoder** consists of a total of four 3D convolution layers. The first two convolution layers are followed by 3D transposed convolution layers. The transposed convolution layers use a stride size of 2 and padding of 0. Each of these layers is also followed by a 3D batchnorm and leaky ReLU activation function. The output dimensions of the final convolution layer are modified to match the number of classes for the corresponding segmentation task. In the case of using the architecture for the reconstruction task, the final number of output features is equivalent to the number of input features to the encoder.