## 7 Experiments and Results
### Task 2: Observation Assessment  
Each observation is accompanied by metadata for the SO who created it and their experience level. In this section, we investigate the development of a system that can assign an effective SO experience score to incoming observations. Specifically, the system is tasked by being given an observation to guess the experience level of the SO who registered it. This system could be applied to the continuous training of SOs, where incoming observations can be quickly assessed with respect to their _effective experience score_. The training could be based on case studies with \"good\" and \"bad\" examples of existing observations, selected using the outputs of this system (it should be stressed for ethical concerns that the goal of this system is to perform observation assessment and not SO assessment. The desired application is to be used in the creation of case studies for SO training.). The underlying assumption is that more experienced SOs will register higher quality observations about more serious safety or compliance issues. The task was formulated as a 3-class classification problem for four different experience levels.  
Table 9 summarizes the results for the \"effective experience level\" classification task. We evaluate all model configurations. We observe that, for this task, the visual and text modalities contain complementary information, as the improvement between the best unimodal model (Greek BERT) and the best multimodal model (Greek BERT, Resnet18, CLSS) is larger than 5% in terms of accuracy. Moreover, we observe that the fusion approach is important. Most models that use simple concatenation for fusion show marginal, if any, improvement over their unimodal counterparts. In contrast, when we used CLSS fusion, we obtain a strong and consistent improvement for all configurations.  
\begin{table}
\begin{tabular}{c c c c c c} \hline
**Location** & **Modality** & **Text Encoder** & **Visual Encoder** & **Fusion Method** & **Accuracy (\%)** \\ \hline \multirow{4}{*}{Construction} & \(\mathcal{V}\) & - & ViT & - & 60.59 \\  & \(\mathcal{T}\) & Greek BERT & - & - & 59.81 \\  & \(\mathcal{T}\) & GR-Electra & - & - & 60.07 \\  & \(\mathcal{T}+\mathcal{V}\) & Greek BERT & ViT & CLSS & **64.13** \\  & \(\mathcal{T}+\mathcal{V}\) & GR-Electra & ViT & CLSS & 62.59 \\ \cline{2-6}  & \multicolumn{4}{c}{Random Baseline} & \multicolumn{4}{c}{59.95} \\ \hline \multirow{4}{*}{Office} & \(\mathcal{V}\) & - & ViT & - & 60.21 \\  & \(\mathcal{T}\) & Greek BERT & - & - & 59.6 \\  & \(\mathcal{T}\) & GR-Electra & - & - & 59.96 \\  & \(\mathcal{T}+\mathcal{V}\) & Greek BERT & ViT & CLSS & **63.46** \\  & \(\mathcal{T}+\mathcal{V}\) & GR-Electra & ViT & CLSS & 62.16 \\ \cline{2-6}  & \multicolumn{4}{c}{Random Baseline} & \multicolumn{4}{c}{57.62} \\ \hline \multirow{4}{*}{Store} & \(\mathcal{V}\) & - & ViT & - & 60.59 \\  & \(\mathcal{T}\) & Greek BERT & - & - & 60.52 \\  & \(\mathcal{T}\) & GR-Electra & - & - & 61.04 \\  & \(\mathcal{T}+\mathcal{V}\) & Greek BERT & ViT & CLSS & **64.13** \\  & \(\mathcal{T}+\mathcal{V}\) & GR-Electra & ViT & CLSS & 63.35 \\ \cline{2-6}  & \multicolumn{4}{c}{Random Baseline} & \multicolumn{4}{c}{45.54} \\ \hline \multirow{4}{*}{Warehouse} & \(\mathcal{V}\) & - & ViT & - & 60.23 \\  & \(\mathcal{T}\) & Greek BERT & - & - & 60.43 \\  & \(\mathcal{T}\) & GR-Electra & - & - & 60.81 \\  & \(\mathcal{T}+\mathcal{V}\) & Greek BERT & ViT & CLSS & **64.14** \\  & \(\mathcal{T}+\mathcal{V}\) & GR-Electra & ViT & CLSS & 63.23 \\ \cline{2-6}  & \multicolumn{4}{c}{Random Baseline} & \multicolumn{4}{c}{56.12} \\ \hline \end{tabular}
\end{table}
Table 8: Ablation regarding the priority classification accuracy for different issue locations. \(\mathcal{T}\): Text, \(\mathcal{V}\): Visual.