## 7 Experiments and Results  
### Task 1: Priority Classification  
The first challenge we want to address is the classification of observations across different levels of priority. This is a 3-class classification problem with issue severities ranging from _light_ to _medium_ to _low_. The \"priority\" represents a general assessment score for the end users, which can encode multiple aspects of a safety issue. The criteria for marking an observation priority as _high_ include the danger level of the observation, the probability of an accident occurring, possible legal or economic ramifications to the business or this safety issue persisting for large periods of time, across different safety inspections. Our goal was to use multimodal cues for automatic priority assessment of observations. Table 7 summarizes the priority classification results on the Safety4 All test set. We evaluated the unimodal and bimodal classification accuracy of all combinations for the fusion methods, the visual and text encoders described in Section 6, as well as the score for the random baseline. The random baseline was calculated as the majority class score, i.e., the accuracy score that we would receive, if we had a system that always predicted the majority class. We can see the scores for the unimodal models in the first four rows and the scores for the multimodal models in the eight rows that follow. The random baseline score is shown in the last row of Table 7. First, we observe that the majority class score is 49.17%, which indicates some class imbalance in the 3-class classification problem. All models surpass the random baseline by a large margin (\(\approx\)10%). Second, for the unimodal configurations, we observe that the text and visual modalities contain balanced information for the task of priority classification. The weakest unimodal models are Greek BERT and Resnet18, while the strongest are GR-Electra and ViT. Third, we see that all multimodal configurations surpass the unimodal model performance, regardless of the fusion approach. This indicates that both the text and visual modalities contain valuable information regarding the observation's priority. Regarding the fusion method, we observe that the \(CLSS\) approach outperforms the naive concatenation consistently, resulting in a 0.5-1.5% absolute improvement in the accuracy scores. One interesting observation is that, while the text-only GR-Electra model surpasses the Greek BERT by 1.3%, the best results are achieved using the combination of Greek BERT, ViT and CLSS. The \"\(\Delta\uparrow\) unimodal\" column indicates the improvement in the multimodal configuration over the best encoder involved in that configuration. For example, for a \"Resnet18+GR-Electra\" multimodal configuration we report the improvement over the unimodal \"GR-Electra\" model, while for a \"ViT+Greek-BERT\" multimodal configuration, we report the improvement over \"Greek-BERT\". Possible negative values in this column indicate that the unimodal encoder outperforms that multimodal configuration. In Table 8, we present an ablation study for priority classification in different locations. For this experiment, we fine tuned the models on the training set and split the test-set observations in four location-specific test sets, namely, \"Construction\", \"Office\", \"Store\" and \"Warehouse\". We evaluated the model on each of the location-specific test sets. We used ViT as the visual encoder, GR-Electra or Greek BERT as the text encoder and, for multimodal configurations, we used CLSS for fusion. We report the unimodal model accuracy, the multimodal model accuracy, as well as the random baseline. The first observation is that the \"Construction\", \"Office\" and \"Warehouse\" sets are more imbalanced than the overall test set, with the random baseline accuracy being 59.95, 57.62 and 56.12, respectively. The \"Construction\" set statistics differ though, as the most frequent priority class is \"high\", while in the other sets the most frequent class is \"medium\" priority.