Teaching Interactively to Learn Emotions in Natural Language (2022)

Abstract: Motivated by prior literature, we provide a proof of concept simulation study for an understudied interactive machine learning method, machine teaching (MT), for the text-based emotion prediction task. We compare this method experimentally against a more well-studied technique, active learning (AL). Results show the strengths of both approaches over more resource-intensive offline supervised learning. Additionally, applying AL and MT to fine-tune a pre-trained model offers further efficiency gain. We end by recommending research directions which aim to empower users in the learning process.

## introduction we examine machine teaching (mt), an understudied interactive machine learning (iml) method under controlled simulation for the task of textbased emotion prediction @xcite @xcite @xcite @xcite . this problem intersects with affective computing @xcite @xcite , and a family of language inference problems characterized by human subjectivity in learning targets @xcite and semantic-pragmatic meaning @xcite . both subjectivity and the lack of data for learning to recognize affective states motivate iml techniques. here, we focus on resource efficiency. our findings from simulations provide directions for user experiments. human perception -and thus human annotators' interpretation -is influenced by human factors such as preferences, cultural differences, bias, domain expertise, fatigue, time on task, or mood at annotation time @xcite @xcite . generally, experts with long-standing practice or in-depth knowledge may also not share consensus @xcite . inter-subjective disagreements can reflect invalid noise artifact (detectable by humans) or ecologically valid differences in interpretation. holzinger (2016) define iml methods as algorithmic procedures that "can interact with agents and can optimize their learning behavior through these interactions [...]" (p. 119). in our study, the stakeholders in the learning process are models (learners) and humans (agent-users or agentteachers). @xcite posit that iml involves either active learning (al) or interactive machine teaching (mt), foot_0 based on humans' role in the learning loop. in al, the learning algorithm uses query strategies (e.g., triggered by uncertainty) to iteratively select instances from which it learns @xcite if licensed by a budget; with a human agent who annotates upon learner request. in contrast, in mt, the teacher (user) who possesses problem knowledge instead selects the instances to be labeled and uses them to train the learner @xcite . initial, foundational mt research focused on constructing a minimal, ideal set of training data, striving for optimality in the data the learner is presented with to learn from. interactive mt assumes human agent interaction with the learner @xcite , for enabling time-and resource-efficient model convergence. following the training by error criterion described in @xcite , if the learner is unable to predict the right answer, and the budget allows, the human teacher instructs the learner with the label. thus, al leverages measures to wisely choose instances for human labeling and subsequent learning, whereas mt capitalizes on the teacher's knowledge to wisely select training instances and proceed to learn when the criterion to teach is met (cf. figure 1 ). mt deals with a teacher designing a wellreasoned, ideally optimal, training set to drive the learner to the desired target concept/model @xcite . while there has been some progress in the use of mt, its application in nlp is present in its earliest form with little empirical exploration or refinement. mt has been explored mostly in computing security, where the teacher is a hacker/advisor who selects training data to adjust the behavior of an adaptive, evolving learner @xcite . @xcite reported that mt could greatly reduce the number of instances required, and even outperformed most al strategies. these findings are compelling and motivate exploring mt's potential in nlp, which, however, has some distinct characteristics, including high-dimensional data impacted by scarcity. mt's possibilities in nlp are thus as of yet largely unknown. we begin here by focusing on controlled experimental simulations to examine resource-efficiency and performance in text-based emotion prediction, whereas future work will take a step closer to ecological validity in interactive mt with real-time agent-teachers. overall, several prospects can be noted for nlp with interactive machine learning (iml): while iml is promising, issues include: ## mt/al for emotion prediction text-based emotion data are subject to variation and ambiguity, which adds to the difficulty in the annotation process, compounded with data scarcity for capturing many affective states. iml methods can be a means to deal with data limitations. in this study, we used a subset of the goemotions dataset @xcite which consists of emotion labels for reddit comments. we prioritized resource-efficiency as the primary experimental variable over exploring impact on target concept ambiguity. figure 2 shows the imbalanced distribution of emotion classes in this subset. the training and test sets comprised approximately 2800 and 700 instances respectively. in all experiments, the learner was trained initially with 10% of the training set while the remaining 90% was reserved as an unlabeled pool of data which were gradually added to the training set in each iteration. 2 the simulated 'user' had access to the labels of the instances from the unlabeled dataset whenever required via dataset lookup. ## discussion we showed that iml efficiently produces desired results for text-based emotion prediction. mt remains understudied and should be further explored for nlp tasks. fine-tuning a pre-trained model with al can leverage the strengths of both approaches with small datasets. in addition to experiments detailed above, we explored training the learner incrementally (online training) versus in a non-incremental setup (the learner is trained using accumulated training set up to the most recent query). the incremental approach experiences catastrophic forgetting but requires very little time for learner updating and can thus work well under low memory usage, e.g., for a life-long learning setting or edge devices. ## conclusion our study on text-based emotion prediction demonstrated the potential of both mt and al methods. we offered initial experimentation with mt and al for this problem, and based on promising results under controlled simulation, next steps will focus on real-time user/teacher interactions, a broader set of teaching criteria, and new forms of training instance selection. in addition, we are interested in exploring heavily understudied affective states, which are currently not covered sufficiently or not covered at all in annotated emotion corpora. we also suggest focused research on specialized teachers in nlp tasks toward better selection of training data. teachers who assess the learner and decide the right time to offer an adequate set of new information may also help create more robust or interpretable learners which evolve over time.