Subjectivity and Its Prediction: a Deep Dive into Annotation Differences across Modalities in a Multimodal Emotion Dataset
Authors: Quanqi Du, Thomas Demeester, Veronique Hoste


Reviewer paper summaries: This paper investigates the annotation differences across modalities in a multimodal corpus annotated with emotions. The authors observe that the disagreement is often caused by the assessment of emotional intensities, especially on negative emotions. Then, the authors also propose a model to predict such disagreement among annotators.
Summary of strengths: * Detailed investigation and analyses of emotion annotation disagreement among annotators and across modalities for a multimodal corpus
* Predicting disagreement among annotators is interesting
Summary of weaknesses: * No further conclusion or suggestions for improving the annotations
* The analyses are limited to one particular corpus
Comments, suggestions and typos: Maybe the whole procedure could be summarized and used for other corpora analysis.
Overall assessment scores: 2