Effectiveness of Domain Adaptation Approaches for Social Media \POS~Tagging
Authors: Tobias Horsmann, Torsten Zesch (University of Duisburg-Essen, 2015)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
We compare a comprehensive list of domain adaptation approaches for POS tagging of social media data. We find that the most effective approach is based on clustering of unlabeled data. We also show that combining different approaches does not further improve performance. Thus, POS tagging of social media data remains a challenging problem.

Introduction
Part-of-Speech (PoS) tagging of social media data is still challenging. Instead of tagging accuracies in the high nineties on newswire data, on social media we observe significantly lower numbers. This performance drop is mainly caused by the high number of out-of-vocabulary words in social media, as authors neglect orthographic rules (Eisenstein, 2013). However, special syntax in social media also plays a role, as e.g. pronouns at the beginning of sentence are often omitted like in “went to the gym” where the pronoun ’I’ is implicated (Ritter et al., 2011). To make matters worse, existing corpora with PoS annotated social media data are rather small, which has led to a wide range of domain adaptation approaches being explored in the literature. There are two main paradigms: First, adding more labeled training data by adding foreign or machine-generated data (Daumé III, 2007; Ritter et al., 2011). Second, incorporating external knowledge or guiding the machine learning algorithm to extract more knowledge from the existing data (Ritter et al., 2011; Owoputi et al., 2013). The first strategy affects from which data is learned, the second one what is learned. Using more training data Usually there is only little PoS annotated data from the social media domain, so just using re-training on domain-specific data does not suffice for good performance. Mixed re-training adds additional annotated text from foreign domains to the training data. In case there is much more foreign data than social media data, Oversampling (Daumé III, 2007) can be used to adjust for the difference in size. Finally, Voting can be used to provide more social media training data by relying on multiple already existing taggers. Using more knowledge Instead of adding more training data, we can also make better use of the existing data in order to lower the out-ofvocabulary rate. PoS dictionaries provide for instance information about the most frequent tag of a word. Another approach is clustering which group words according to their distributional similarity (Ritter et al., 2011). In this paper, we evaluate the potential of each approach for solving the task.

Conclusion
In this paper, we analyzed domain adaptation approaches for improving PoS tagging on social media text. We confirm that adding more manually annotated in-domain data is highly effective, but annotation costs might often prevent application of this strategy. Adding more out-domain training data or machine-tagged data is less effective than adding more external knowledge in our experiments. We find that clustering is the most effective individual approach. However, clustering based on very large corpora did not further increase accuracy. As combination of strategies did only yield minor improvements, clustering seems to dominate the other strategies.