RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Acknowledgements
This research work has been funded by the German Federal Ministry of Education and Research and the Hessian Ministry of Higher Education, Research, Science and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE. This work has also been supported by the DYNAMIC center, which is funded by the LOEWE program of the Hessian Ministry of Science and Arts (Grant Number: LOEWE/1/16/519/03/09.001(0009)/98).
C.1 Frequency related parameters
Pitch : Logarithmic F0 on a semitone frequency scale, starting at 27.5 Hz. A semitone is the smallest music interval and is considered the most dissonant when sounded harmonically.

Jitter : Jitter measures the cycle-to-cycle variations of the fundamental frequency. It is a measure of frequency variability compared to the person’s fundamental frequency.

Formant 1, 2, and 3 frequency : Centre frequency of first, second, and third formant. Formants are distinctive frequency components of the acoustic signal produced by speech. They are used to identify vowels.

Formant 1 Bandwidth : Bandwidth of the first formant. The formant bandwidth affects the identification of vowels in competition with other vowels.
C.2 Energy/Amplitude related parameters
Shimmer : Shimmer measures the cycle-to-cycle variations of fundamental amplitude. The shimmer changes with the reduction of glottal resistance and mass lesions on the vocal cords and is correlated with the presence of noise emission and breathiness.

Loudness : an estimate of perceived signal intensity from an auditory spectrum.

Harmonics-to-noise ratio : Relation of energy in harmonic components to energy in noise-like components. HNR quantifies the relative amount of additive noise.
C.3 Spectral Parameters
Alpha Ratio : Ratio of the summed energy from 50-1000 Hz and 1-5 kHz.

Hammarberg Index : Ratio of the strongest energy peak in the 0-2 kHz region to the strongest peak in the 2–5 kHz region.

Spectral Slope 0-500 Hz and 500-1500 Hz : Linear regression slope of the logarithmic power spectrum within the two given bands.

Formant 1, 2, and 3 relative energy : The ratio of the energy of the spectral harmonic peak at the first, second, and third formant’s centre frequency to the energy of the spectral peak at F0.

MFCC 1-4 : Mel-Frequency Cepstral Coefficients 1-4.

Spectral flux : Difference of the spectra of two consecutive frames.
Figure 1: Proposed QuestMF framework to predict depression severity score. Here, Qx denotes Question number x in the questionnaire. MLP denotes Multilayer Perceptron, which is used as the classification head. Each question is scored among classes \{0,1,2,3\} . These scores are then added to get the total score \in\{0,1,2,...,3n\} .; Proposed QuestMF framework to predict depression severity score. Here, Qx denotes Question number x in the questionnaire. MLP denotes Multilayer Perceptron, which is used as the classification head. Each question is scored among classes \{0,1,2,3\} . These scores are then added to get the total score \in\{0,1,2,...,3n\} .
Figure 2: Architecture of single modality encoder models. We use a turn-based architecture to encode multi-turn dialogue data.; Architecture of single modality encoder models. We use a turn-based architecture to encode multi-turn dialogue data.
Figure 3: Architecture of two-modality fused models. We use cross-attention layers for interaction among modalities M1 and M2 . In cross-attention, X \rightarrow Y denotes that the Y modality encoding is used as the query and the X modality encoding as the key and value.; Architecture of two-modality fused models. We use cross-attention layers for interaction among modalities M1 and M2 . In cross-attention, X \rightarrow Y denotes that the Y modality encoding is used as the query and the X modality encoding as the key and value.
Figure 4: Architecture of the three-modality fused model. In cross-attention, X \rightarrow Y denotes that the Y modality encoding is used as the query and the X modality encoding as the key and value.; Architecture of the three-modality fused model. In cross-attention, X \rightarrow Y denotes that the Y modality encoding is used as the query and the X modality encoding as the key and value.
Figure 5: Validation CCC for each question with different modality models. Here, T refers to Text, A refers to Audio, and V refers to Video. An addition between the modalities denotes using a fusion of them. The video model for question 8 gives the same scores to all data points. Thus, its CCC is not valid and is not shown in the graph.; Validation CCC for each question with different modality models. Here, T refers to Text, A refers to Audio, and V refers to Video. An addition between the modalities denotes using a fusion of them. The video model for question 8 gives the same scores to all data points. Thus, its CCC is not valid and is not shown in the graph.
Results of QuestMF trained with ImbOLL function compared with ablation frameworks and prior works.
Results of QuestMF ( ImbOLL ) framework over 3 different seed runs compared with ablation frameworks.
Ablation results for using different combinations of modalities with different frameworks. Here, T refers to Text, A refers to Audio, and V refers to Video. An addition between the modalities denotes using a fusion of them. The CCC scores presented are the mean over 3 different seed runs.
ImbOLL experiments
OLL experiments
Results of experiments with LSTM output dimension for QuestMF Framework trained with ImbOLL function
Results of experiments with LSTM output dimension for QuestMF Framework trained with OLL function with \alpha=1
Results of experiments with LSTM output dimension for QuestMF Framework trained with OLL function with \alpha=1.5
Results of experiments with LSTM output dimension for QuestMF Framework trained with MSE loss function
Results of experiments with LSTM output dimension for Total Framework
Ablation results for using different combinations of modalities with different frameworks. The CCC, RMSE and MAE scores presented are the mean and standard deviation over 3 different seed runs.