<figure><img src="image_13.png" /><figcaption>Figure 1: Results on *unknown* VVPPER word forms with various methods</figcaption></figure>

<table><caption>Table 2: $F_1$ on all and on out-of-vocabulary instances</caption><thead><tr><th rowspan="2">Setup</th><th rowspan="2"></th><th rowspan="2">All F1</th><th colspan="3">Out-Vocabulary</th></tr><tr><th>P</th><th>R</th><th>F1</th></tr></thead><tbody><tr><td rowspan="8">HunPos</td><td>Baseline</td><td>.78</td><td>.80</td><td>.38</td><td>.52</td></tr><tr><td>Downs. 75%</td><td>.78</td><td>.63</td><td>.48</td><td>.54</td></tr><tr><td>Downs. 50%</td><td>.79</td><td>.74</td><td>.41</td><td>.53</td></tr><tr><td>Downs. 25%</td><td>.79</td><td>.81</td><td>.40</td><td>.53</td></tr><tr><td>Overs. x2</td><td>.79</td><td>.78</td><td>.40</td><td>.53</td></tr><tr><td>Overs. x3</td><td>.79</td><td>.74</td><td>.41</td><td>.53</td></tr><tr><td>Annotated x2</td><td>.83</td><td>.80</td><td>.56</td><td>.65</td></tr><tr><td>Annotated x3</td><td><strong>.88</strong></td><td>.81</td><td>.70</td><td>.75</td></tr><tr><td rowspan="7">Two-Step</td><td>Baseline</td><td>.77</td><td>.95</td><td>.32</td><td>.48</td></tr><tr><td>Downs. 75%</td><td>.78</td><td>.85</td><td>.37</td><td>.51</td></tr><tr><td>Downs. 50%</td><td>.80</td><td>.96</td><td>.38</td><td>.55</td></tr><tr><td>Downs. 25%</td><td>.79</td><td>.92</td><td>.38</td><td>.53</td></tr><tr><td>Overs. x2</td><td>.77</td><td>.95</td><td>.32</td><td>.48</td></tr><tr><td>Overs. x3</td><td>.77</td><td>.95</td><td>.32</td><td>.48</td></tr><tr><td>Annotated x2</td><td>.81</td><td>.93</td><td>.43</td><td>.59</td></tr><tr><td></td><td>Annotated x3</td><td><strong>.85</strong></td><td>.92</td><td>.56</td><td>.69</td></tr></tbody></table>

<table><caption>Table 3: Results of the contextualised Two-Step</caption><thead><tr><th rowspan="2">Configuration</th><th rowspan="2">All F<sub>1</sub></th><th colspan="3">Out-of-Vocabulary</th></tr><tr><th>P</th><th>R</th><th>F<sub>1</sub></th></tr></thead><tbody><tr><td>Baseline</td><td>.81 (+.04)</td><td>.93 (+.02)</td><td>.41 (+.09)</td><td>.57 (+.09)</td></tr><tr><td>Annotated x3</td><td>.86 (+.01)</td><td>.89 (-.03)</td><td>.62 (+.06)</td><td>.73 (+.04)</td></tr></tbody></table>

and must now rely more on the word context.

**Results** In Table 3, we show the changes in performance of the contextualised Two-Step tagger. In parentheses, we show the differences to the not contextualized tagger in Table 2. For both setups we see an improved $F_1$, but especially the recall increases for out-of-vocabulary instances. The overall $F_1$ reached by HunPos (.88) in Table 2 is still superior but the trade off between precision and recall of Two-Step better supports the use case in which the tagger functions as a precise filtering tool with decent recall.

## 4. Field Trial in Social Media

So far, we have only simulated our use case of a linguist who uses a tagger as a filtering tool, while now, we turn to a real setting and apply a tagger to plain text Twitter messages for finding verb-pronoun contractions.

Working on plain text means that the ground truth of how many instances there are in the data is unknown, thus, the recall cannot be computed. Consequently, we focus on evaluating the precision of the tagging, and evaluate how many new instances are found. We choose the Twitter domain for its ease of obtaining data but also for its linguistic diversity that ranges from tweets using informal, interac- tional language to tweets that are close to the written standard. This domain provides us with a challenging test bed that should allow to determine a conservative, lower-bound performance for our approach. We will use the contextual- ized Two-Step tagger for its higher precision while providing a reasonable high recall.

**Twitter Data** We use a random subsample of 50k tweets (about 1.7 million tokens) crawled between 2011 and 2017 from the public Twitter API that we language-filtered for German. All occurrences of user-mentions, hashtags and URLs are replaced by a text constant and the tweets are tokenized by Gimpel et al. (2011)'s ArkTools tokenizer.

a rather similar overall performance. When looking at precision and recall for adding annotated data, highlighted in grey, we see that Two-Step is considerably more precise than HunPos, which has a better recall. Because oversampling showed barely any effect, we suspect that the added lexical knowledge is mostly accountable for the improvements, which also means that the word context seems to be neglected for making decisions. If the tagger focuses too much on lexical forms, it will find mostly instances known from training which is in particular a problem for finding new instances. Hence, an increased weighting of the local word context should support finding new instances and enable a better generalization.

## 3.3. Experiment: Forced Generalization

In this experiment, we try to improve generalization of the Two-Step tagger by forcing the tagger to rely more on the local word context and, thus, improve the recall. We chose Two-Step, as we have implemented this tagger ourselves which facilitates adaptation. We alter the feature space of the SVM and exclude all features that contain the lexical form of the *positive* instances. Thus, the SVM is not aware of any lexical forms that can occur with the tag VVPPER,
<table><caption>Table 4: Examples of tagged instances</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>Strict</td><td>Da lernste pragmatisch zu sein.<br>Ich sachs dir noch.</td></tr><tr><td>Relaxed</td><td>Wer häts gedacht.<br>Ich wills nicht ich will aber auch nicht [...]</td></tr><tr><td>All</td><td>Warum einfach, wenn's auch kompliziert geht ? URL<br>Ich beschränke mich auf's nicht im Weg stehen.</td></tr><tr><td>Frequent Confusion Cases</td><td>Und keiner weiss warum.<br>Ich weiss gar nicht, was du beruflich machst.</td></tr></tbody></table>

**Tagger Setup** We train the coarse model and the SVM on the full shared task data set including the additionally annotated data. To provide more lexical knowledge and increase the robustness when facing standard language text, we also add 100k tokens of the German newswire Tiger (Brants et al., 2004) corpus to both tagging steps.

**Evaluation setup** We evaluate the tagged instances with two annotators. The annotators make four distinctions: *strict*, *relaxed*, *all* and *none*. *Strict* are full verb contractions with personal pronoun, the exact phenomenon we intended to tag. *Relaxed* counts all verb contractions with personal pronoun as correct, this includes also modal and auxiliary verbs. *All* counts all contractions phenomena as correct, this additionally includes, for instance, contractions of conjunctions with personal pronouns. The remaining cases are no contractions and are, thus, false positives.

We will evaluate two setups. The first one selects the first 250 of all found instances, which will be the overall evaluation. The second evaluation focuses on out-of-vocabulary instances in which we remove all tagged instances that are known from the training set until we gather 250 instance and, thus, evaluate how reliably new instances are found.

**Results** In total, we found 1091 instances in 50k tweets tagged as *VVPPER*. The two annotators reached a perfect agreement on the subset of the first 250 instances that we evaluated manually. Figure 2a shows the precision of the overall evaluation. The *strict* result shows that the majority of found instances are the targeted full verb contractions. Including modal and auxiliary verbs in the *relaxed* mode, even three-quarter are verb contractions. Including also miscellaneous contractions in *all*, almost all instances are contractions.

In Figure 2b, we take a closer look on the performance of detecting new contractions, e.g. out-of-vocabulary instances. We focus our discussion on the *strict* results. The precision is drastically decreased to almost half the value that we reach when including all instances. We also computed the type/token ratio which is at 0.69 almost twice as high as in the overall evaluation in Figure 2a. This confirms that the tagger is able to recognize many new instances of the phenomenon. Furthermore, when ignoring the known instances almost every correct instance is a new lexical form.

**Discussion** Table 4 depicts examples of each of the three contraction classes (bold face) and additionally presents a

<figure><img src="image_14.png" /><figcaption>Figure 2: Results of manual evaluation</figcaption></figure>

frequent confusion case, which is erroneously tagged as contraction. Of the VVPPER training data we provided, many instances end on *s* or *'s*, which is a common morphological property of contractions in German. On the one hand, this bias introduces a substantial amount of false positives - for instance the verb *weiß* (*to know*) occurs frequently in a misspelled form *weiss* in social media. On the other hand, this enables the SVM to also tag similar contraction cases of other word classes in *relaxed* or *all*.

## 5. Conclusion

We presented experiments that investigated how a PoS tagger can be designed that works as a corpus querying tool to find instances of rare phenomena. We experimented with altering the frequency weight of rare instances but found that adding relatively small amounts of additionally labelled data is unavoidable. By machine tagging data in which only the phenomenon of interest is manually corrected, we keep the effort minimal but yet achieve considerable improvements on detecting the phenomenon. We showed how recall is easily improved when forcing a tagger to focus more on the local word context. In a field study on plain text, we confirmed that our tagger works well as corpus query tool which finds accurately instances of the phenomenon of interest including many new ones. For future work, we plan to improve our method and also study the applicability to other under-represented phenomena.
Beißwenger, M., Bartz, T., Storrer, A., and Westpfahl, S. (2015). Tagset und Richtlinie für das Part-of-Speech-Tagging von Sprachdaten aus Genres internetbasierter Kommunikation / Tagset and guidelibes for the PoS tagging of langauge data from genres of computer-mediated communication. In *EmpiriST guideline document (German and English version)*.

Beißwenger, M., Bartsch, S., Evert, S., and Würzner, K.-M. (2016). EmpiriST 2015: A Shared Task on the Automatic Linguistic Annotation of Computer-Mediated Communication and Web Corpora. In *Proceedings of the 10th Web as Corpus Workshop (WAC-X) and the EmpiriST Shared Task*, pages 44–56, Berlin, Germany.

Beißwenger, M. (2013). Das Dortmunder Chat-Korpus. In *Zeitschrift für germanistische Linguistik 41*, volume 1, pages 161–164.

Brants, S., Dipper, S., Eisenberg, P., Hansen, S., König, E., Lezius, W., Rohrer, C., Smith, G., and Uszkoreit, H. (2004). TIGER: Linguistic Interpretation of a German Corpus. pages 597–620. Journal of Language and Com-putation.

Brants, T. (2000). TnT: A Statistical Part-of-speech Tagger. In *Proceedings of the Sixth Conference on Applied Natural Language Processing*, ANLC '00, pages 224–231, Stroudsburg, PA, USA. Association for Computational Linguistics.

Brown, P. F., DeSouza, P. V., Mercer, R. L., Pietra, V. J. D., and Lai, J. C. (1992). Class-Based n-gram Models of Natural Language. *Computational Linguistics*, 18:467–479.

Daumé III, H. (2007). Frustratingly Easy Domain Adaptation. In *Conference of the Association for Computational Linguistics (ACL)*, pages 256–263, Prague, Czech Republic.

Gimpel, K., Schneider, N., O’Connor, B., Das, D., Mills, D., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., and Smith, N. A. (2011). Part-of-speech Tagging for Twitter: Annotation, Features, and Experiments. In *Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2*, HLT ’11, pages 42–47, Stroudsburg, PA, USA. Association for Computational Linguistics.

Halácsy, P., Kornai, A., and Oravecz, C. (2007). Hun-Pos: An open source trigram tagger. In *Proceedings of the 45th Annual Meeting of the ACL*, pages 209–212, Stroudsburg. Association for Computational Linguistics.

Hochreiter, S. and Schmidhuber, J. (1997). Long Short-Term Memory. *Neural Computation*, pages 1735–1780.

Horsmann, T. and Zesch, T. (2016a). Assigning Fine-grained PoS Tags based on High-precision Coarse-grained Tagging. In *Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers*, pages 328 – 336, Osaka, Japan. Dublin City University and Association for Computational Linguistics.

Horsmann, T. and Zesch, T. (2016b). LTL-UDE @ EmpiriST 2015: Tokenization and PoS Tagging of Social Media Text. In *Proceedings of the 10th Web as Cor-

pus Workshop (WAC-X) and the EmpiriST Shared Task*, pages 120–126, Berlin, Germany. Association for Computational Linguistics.

Lafferty, J. D., McCallum, A., and Pereira, F. C. N. (2001). Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In *Proceedings of the Eighteenth International Conference on Machine Learning*, ICML ’01, pages 282–289, San Francisco, CA, USA.

Plank, B., Søgaard, A., and Goldberg, Y. (2016). Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss. In *Proceedings of ACL2016*, pages 412–418, Berlin, Germany. Association for Computational Linguistics.

Schiller, A., Teufel, S., Stöckert, C., and Thielen, C. (1999). Guidelines für das Tagging deutscher Textcorpora mit STTS (Kleines und großes Tagset). Institut für maschinelle Sprachverarbeitung, University of Stuttgart, Germany.

Toutanova, K., Klein, D., Manning, C. D., and Singer, Y. (2003). Feature-rich Part-of-speech Tagging with a Cyclic Dependency Network. In *Proceedings of the 2003 NACCL: HLT*, pages 173–180, Stroudsburg, PA, USA. Association for Computational Linguistics.
# Defining a protocol and assessing natural language metadata for a Databank of
## Oral Teletandem Interactions (DOTI)

Paola Leone

University of Salento

P.zzetta A. Rizzo, 1

Lecce (Italy)

paola.leone@unisalento.it

## Abstract

The current study addresses the definition of a protocol for collecting, storing data and describing (in a simple and generic way) a repository. Particularly, the transparency of a form aimed at gathering information about the pedagogical context of oral telecollaboration for language learning named Teletandem (TT; Telles, 2006) will be tested before it is spread more widely. To uncover problems in submitting information, data-input-triggers quality and reliability have been tested interviewing professors and language instructors who will be involved in a preliminary phase of Teletandem corpus implementation. General goals of the study are to enlarge the research group, to increase data and to improve efficiency in data collection.

**Keywords:** corpora, data collection, metadata, telecollaboration, language learning, protocol

## 1. Teletandem as learning context

Teletandem (TT) is a learning context, in which pairs of native and non-native speakers of different languages talk alternatively in their L1 and L2 in order to learn each other's native language (www.teletandembrasil.org). The communication between partners is multimodal, via video calls and chat (e.g. by employing Skype, MSN). Students "virtually meet" during sessions that normally last one hour. From a pedagogical perspective, Teletandem has a positive impact on students' learning experience for different reasons: i) it is a form of learning from peers (Hanushek, Kain, Markman, & Rivkin, 2003), ii) it is extensible to perform and develop different forms of plurilingualism (e.g. to foster L2 production and/or to enhance interlocutors' ability to mutually understand each other when speaking different languages). Furthermore, since communication is enabled by VoIP technology, TT is a context for practicing computer mediated interactions which are becoming more and more used for job application and for university admission. The existence of websites which give "essential Skype interview tips" is a proof of the future relevance of this technology transmitted interaction. The positive impact of Teletandem in language learning (e.g. Leone & Telles, 2016) is a good basis for expecting a gradual, but sustained increase of its practice in higher education. This trend calls for further empirical research and implies a high demand for video/audio data.

## 2. Background information

Bearing the above in mind, two universities, in which for

several years TT has been experienced, have undertaken a project which aims at organizing already existing Teletandem data and at defining natural language metadata. Teletandem databank is currently named Databank of Oral Teletandem Interaction (DOTI; Aranha & Leone, 2016; forthcoming) and it is build out of video recordings of Teletandem sessions thus data includes participants' moving visual images as well as voice and chat texts. Metadata allow the description of the context in which video recordings have been done and they are organized hierarchically. They stem from computer mediated interaction standardized metadata (Chanier, et al., 2014) and from pedagogical research (Mangenot, 2008); the former displays general characteristics of computer mediated interaction, the latter concerns the learning context and are currently not yet in a standardized form.

Following the general framework of "interaction space" (Chanier, et al., 2014), which includes comprehensively guidelines for describing all CMC genres in a multimodal perspective, teletandem as learning context is characterized in terms of: a) participants (i.e. n.2); b) location, meaning online location that is b.1) how interactions are transmitted (e.g. via VoIP technology, via mail etc.), b.2) where data are originally recorded (e.g. university server), b.3) the place where teletandem sessions happen (i.e. at the university or outside the institution); c) time frame which shows the beginning and the end of each session, but also the length of the telecollaboration project with specific information (days of the week, year, month). Concerning the technological environment, Teletandem is multimodal - i.e. diverse communicative modalities are used: audio-video via
VOIP technology and Internet Relay Chat (IRC)- ; communication is synchronous; it is a dyadic exchange since two people are involved; it implies oral, written, gestural and iconic modes of communication; language used can be different.

For recording characteristics related to the pedagogical
implementation of Teletandem, the concepts of “learning
scenario” and task have been referred to. The hierarchical
organization of the learning scenario encompasses as well
concepts of macrotasks (e.g. described in terms of
number, typology) and microtasks.

3. Research

The general goal of the current research is to enlarge the
research group involved in data collection, to increase
data and to improve efficiency in data collection. Hinging
on the above mentioned studies (Aranha & Leone, 2016,
forthcoming), we aim at developing a protocol which
offers guidelines and rules for forthcoming data collection
(Aranha & Leone, forthcoming). In the protocol two main
implementation levels are laid down. Level 1 consists in
collecting Teletandem interactional video recordings as
“raw data”, storing them and describing video file
contents using natural-language definition metadata
(Aranha & Leone, 2016, forthcoming). Professors and/or
language instructors will carry out this activity uploading
video files in a cloud storage system and submitting as
well - in a digital form - information which highlights
properties of the specific TT learning context - i.e.
information range from the institutional profile for the
course in which TT has been institutionalized (Aranha &
Cavalari, 2014) to the tasks assigned for the whole
activity-. In level 1, researchers, professors and language
instructors must agree on which terms to be used for
property description and on which definition terms have.
In level 1 DOTI will be enriched. Level 2 encompasses all
those actions aimed to create an interoperational
databank. It consists in implementing metadata
interoperability by checking the compatibility of level 1
metadata with standardized metadata (e.g. Text Encoding
Initiatives and Dublin Core Metadata Initiative). In this
phase of corpora development, metadata will give access
to digital data and they will be targeted to applied
linguistic researchers.

The current research sticks into level 1 actions and it is a
test of the clarity of the form which will be used to collect
information about Teletandem as learning context and
then for describing Teletandem databank. Informants are
colleagues who already employed Teletandem in
teaching. Research questions are: Are language
instructors and professors familiar with terms employed
as natural language metadata for describing Teletandem
as a learning context in Higher Education? Does
terminology facilitate entry of information? How can
metadata be improved so to be more user friendly? The
methodology for collecting colleagues' opinions is an in
depth interview supported by a written questionnaire.
Interviews have been recorded. Informants have to fill the

form in (i.e. the assumption was “the form is well defined
if colleagues can submit proper information in the right
place”) and then discuss if each question and/or concept
description was/were clear (or not) and, if unclear, how it
could be improved. The form contains data-input-triggers
which are originated from the metadata provided for
describing already collected data.

4. Results and conclusion

A first analysis of the findings shows that for some form
fields (and subfields) there is coherence between required
and given information (e.g. for the concept of learning
scenario), whereas in other entries there was no match
between purposes and information (e.g. macrotask and
microtask are not easy to understand). To place interface
terminologies in context, results will be discussed
considering the kind of background informants show in the
field of foreign language teaching. Suggestions on how
natural language metadata could be improved and how the
form interface layout could be implemented will be given.

References

Aranha, S., & Cavalari, S. (2014). A trajetória do projeto Teletandem Brasil: da modalidade institucional não-integrada à institucional integrada. *The ESpecialist*, 35(2), 70-88.

Aranha, S., & Leone, P. (2016). DOTI: Databank of Oral Teletandem Interactions. In S. Jager, & Kurek, M. (Eds.), *New directions in telecollaborative research and practice: selected papers from the second conference on telecollaboration in higher education* (pp. 327-332). Research-publishing.net.

Aranha, S., & Leone, P. (forthcoming). State of the arts and DOTI-Databank of Oral Teletandem Interaction.

Chanier, T., Poudat, C., Sagot, B., Antoniadis, G., Wigham, C., Hriba, L., Seddah, J. (2014). The CoMeRe corpus for French: structuring and annotating heterogeneous CMC genres. *Journal for Language Technology and Computational Linguistics*, 2(29), 1-30.

Hanushek, E., Kain, J., Markman, J., & Rivkin, S. (2003). Does peer ability affect student achievement? *Applied econometrics*, 527-544.

Leone, P., & Telles, J. (2016). The Teletandem network. In T. Lewis, & R. O'Dowd (A cura di), *Online Intercultural Exchange: Policy, Pedagogy, Practice* (p. 243-248). London: Routledge.

Mangenot, F. (2008). La question du scénario de communication dans les interactions pédagogiques en ligne. *Jocair (Journées Communication et Apprentissage Instrumentés en Réseau)*, 13-26.

Telles, J. A. (2006). *Projeto Teletandem Brasil: Línguas Estrangeiras para Todos – Ensinando e Aprendendo línguas estrangeiras in-tandem via MSN Messenger*. Faculdade de Ciências e Letras de Assis, UNESP.
# The #Idéo2017 Platform

Julien Longhi¹, Claudia Marinica², Nader Hassine¹,², Abdulhafiz Alkhouli², Boris Borzic²

¹ University of Cergy-Pontoise, AGORA

² ETIS Lab UMR 8051 University of Paris-Seine, University of Cergy-Pontoise, ENSEA, CNRS
33 Boulevard du Port, 95000 Cergy-Pontoise, France

julien.longhi@u-cergy.fr, claudia.marinica@ensea.fr, nader2hassine@gmail.com,

abdulhafiz.alkhouli@ensea.fr, boris.borzic@ensea.fr

## Abstract

The #Idéo2017 platform allows citizens to analyze the tweets of the 11 candidates at the French 2017 Presidential Election. #Idéo2017 processes the messages of the candidates by creating a corpus in almost real time. By using techniques from linguistics supplied with tools, #Idéo2017 is able to provide the main characteristics of the corpus and of the employment of the political lexicon, and allows comparisons between the different candidates.

**Keywords:** NLP for social media, NLP applications, textometry, tweets mining

## 1. Introduction

Social networks are becoming an important source for citizen's information, concerning mainly their "consumption" of information (Mercier, 2014). Twitter, the most known micro-blogging platform (Kaplan and Haenlein, 2010), by allowing the publication of short messages (140 characters), gives to social networks a new dimension. Indeed, Twitter can be used to assess how users react to social (Longhi and Saigh, 2016), political (Longhi et al., 2014; Conover et al., 2011), or economic issues. Therefore, the textual data (messages) sent on Twitter can be used to extract emotions, feelings, opinions, etc., of the users (Johnson and Goldwasser, 2016).

The analysis of political tweets during the election campaigns, or specific events, is increasing and can be seen as a specific type of political discourse (Longhi, 2013). Among the studies on this subject, Roginsky and Cock (2015) propose a qualitative analysis of interactions on Twitter, but they are limited to "the discursive and communicational analysis of the types of expression on Twitter that we can observe, with a particular interest in the way studied actors present and put forward themselves". Johnson and Goldwasser (2016) propose a classification of positionings based on the most frequent words, while the analysis of Vidak and Jackiewicz (2016) focuses on emotions. Moreover, many studies have proposed approaches to predict the result of the presidential elections (or to explain why the prediction is not possible) by analyzing the tweets (Tumasjan et al., 2010; Gayo-Avello et al., 2011; Metaxas et al., 2011).

Thus, there is an extensive literature on the analysis of political tweets, but these works are difficult to gather because they come either from the computer sciences, either from the humanities and social sciences (communication sciences, linguistics). Moreover, despite the unquestionable interest in outlining political facts, these results are not accessible by citizens interested in this subject.

In this context, this article presents #Idéo2017, a new and innovative platform making analytic information available to citizens. #Idéo2017 proposes a tool for analyzing tweets and speeches (relayed on Twitter) of the 11 candidates at

the presidential election in France in 2017. #Idéo2017 analyzes the messages of the candidates by creating a corpus in almost real time (updated every 24 hours) with the tweets published in candidates' official accounts (from September 1st 2016 to May 7th 2017). Using techniques and metrics derived from linguistic tools, the new platform provides the main characteristics of the corpus and allows comparisons between the different candidates.

The rest of the paper is structured as follows. In Section 2., we provide a general description of the tool, and in Section 3. we present the analyses that can be carried out. Then, in Section 4. we detail the tool's development, as well as the technological choices that we made. Section 5. concludes the papers and provides a set of perspectives.

## 2. Description of #Idéo2017

#Idéo2017 is a web platform available online allowing to analyze the messages, posted on Twitter, related to political news (meetings, debates, television broadcasts, etc.). Its objective is to make available on the web for average citizens a set of statistical analyses and data visualization tools applied on the Twitter messages. The choice of a web platform rather than software to be installed comes from the fact that we want citizens that are non-specialists of tools and software to be able to have access to the analyses' results without going through the phases of corpus formation, tagging, etc. Thus, the citizens can make their own queries (based on linguistic and textometric criteria, more precisely, the most used words by political personalities, analyses of similarities, ALCESTE algorithm, etc.) and obtain comprehensible result.

The #Idéo2017 platform follows the processing chain shown in Figure 1: (1) retrieving the set of tweets of the candidates, (2) setting up a backup of tweets, (3) indexing tweets to facilitate the search process, (4) applying a set of linguistic analyses on tweets, (5) setting up a search engine on the tweets, and (6) displaying the results on a web page. In this processing chain, we are firstly interested in extracting candidates' tweets: we want to extract the tweets daily and to propose to the users to analyze the current database; for example, on April 4th, 2017 the users are able to analyze
<figure><img src="image_15.png" /><figcaption>Figure 1: Processing chain in the #Idéo2017 platform.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Figure 2: Analyses for the word work: under- / over- use.</figcaption></figure>

the tweets sent by the candidates until April 3rd, 2017. To this end, we took advantage of the work that we carried out in the context of the extraction of the *Polititweets* corpus (Longhi et al., 2014), which is available and documented on the Ortolang platform<fnref n="1" />.

In the tweets backup and indexing steps, we are interested in the issues of tweets storage, and, respectively, in the implementation of an indexing system. These two steps facilitate the access to the tweets and the development of an intelligent search engine.

In the linguistic analysis step, we propose to the user a set of analyses to be carried out on the set of tweets. These analyses, described in the next section, concern: the use of a specific word and its derivatives by the different candidates, the words associated with a specific word, the word cloud, themes, relations between words, and the specificities of the different candidates. In addition, we have developed an intelligent search engine based on the faceted search process that allows to the user to perform searches on tweets using complex filters.

**3. Analyses and Search Engine Description**

The #Idéo2017 platform, available at http://ideo2017.ensea.fr/plateforme/, proposes two types of analyses and the search engine. These three elements are described below.

**3.1. The Analysis “I Analyze the Tweets that Contain the Word [Word]”**

The analysis “I analyze the tweets that contain the word [word]” allows to the user to choose a word among the 13 words that are often used in political debates (Alduy, 2017).

Our choice on limiting the user to 13 words is related to the computation time of analyses and graphics which would be too high if performed in real time. So, all the computations for the 13 words are performed at the same time (during the night) and the results are kept on the drive and displayed when requested. In a new version of our platform we plan to improve this aspect. On the other side, if the user wish to search for a word in the tweets he/she can use the search engine.

The list of 13 selected words is: *France, state, Republic, people, law, work, freedom, democracy, security, immigration, terrorism, Islam and secularism*.

Once the word is chosen, the user has access to four analyses:

* The first analysis allows to identify the use of the chosen word by the different candidates, and the results are presented in the form of two graphs: one for the computation of specificities (the under- / over- use of the word by the candidates), and the other one for the frequency of use of the word by the candidates.

* The second analysis detects the words associated with the chosen word for all candidates. This analysis of co-occurrences is presented in the form of a graph of associated words.

* The third analysis consists in computing the use of the chosen word and its derivatives (on contrary to the first analysis) by the different candidates.

* The last analysis creates a word cloud that allows to display graphically the lexicon.

To exemplify these analyses, let us consider the word *work* (*travail* in French). We can see in Figure 2 the under- / over- use of the word *work* (computation of specificities),

<footnote n="1">¹https://repository.ortolang.fr/api/content/comere/v3.3/cmr-polititweets.html</footnote>
<figure><img src="image_17.png" /><figcaption>Figure 3: Analysis of the tweets of the candidate Emmanuel Macron: the themes (by lexical categories).</figcaption></figure>

and in Figure 5 in Appendix Section the frequency of this word in all candidates' tweets.

The Figure 6 in Appendix Section allows us to see the words associated with the word *work* (analysis of similarities), and the Figure 10 in Appendix Section the word cloud.

It is important to outline that the analyses are performed after lemmatizing each word (for example, for the lemma liberty, we can have the forms liberty, liberties, etc.).

## 3.2. The Analysis “I Analyze the Tweets of [Candidate]”

The analysis “I analyze the tweets of [candidate]” allows to the user to perform a set of analyses on each of the 11 candidates: N. Arthaud, F. Asselineau, J. Cheminade, N. Dupont-Aignan, F. Dillon, B. Hamon, J. Lassalle, M. Le Pen, E. Macron, J.-L. Mélenchon and P. Poutou.

Once a candidate is chosen, the possible linguistic analyses are the following:

* The first analysis allows to detect and analyze the words the most used by the candidate;

* The second analysis, called “Themes”, proposes, for the chosen candidate, to group together the words that are semantically close in order to outline the important themes discussed by the candidate;

* The third one analyzes the similarity between the words of each candidate in a graphical form;

* The fourth analysis displays the lexicon of the tweets in the form of a word cloud;

* The last one is devoted to the analysis of the tweets of all the candidates. It allows to identify the specific words and categories of the different candidates and to compare them.

For example, let us consider the candidate Emmanuel Macron; the Figure 8 in Appendix Section presents the most frequent words used in his tweets, and the Figure 3 exposes the themes of his tweets.

## 3.3. The Search Engine

In the previous 2 sections we presented two type of analyses available in the #Idéo2017 platform; the third feature of #Idéo2017 is an intelligent search engine in real time (the graphical interface is shown in Figure 9 in Appendix Section) which offers a faceted search over the tweets: by candidate, by hashtag or by mention. Moreover, it provides complete search flexibility allowing to the user to compare the candidates using complex queries, and it also allows to sort the results by the date or by the commitment.

Twitter includes already a classical search engine in its interface; in order to propose a richer search experience, we built our search tool as a hybrid system bringing together the results of real time queries on the tweets and the synthesis of several tweets by aggregating the information via several facets (filters) and linguistic computations or word clouds. Thus, for a specific word or theme, our goal is to provide an access to the original tweets for each candidate, but also to compute the exact distribution of tweets per candidate and per theme.

The tweets' distribution information permits to contextualize each query, because our objective is at the same time to build a search engine, but also to propose a business intelligence system allowing to study the communication strategies of the candidates. It is important to outline that the two features (search engine and business intelligence system) have completely different goals: meanwhile the search engine struggles against the noise (all the answers should be the most pertinent), a business intelligence system, as a benchmark, aims to reduce the silence (all pertinent tweet should be shown to the user). However, when we try to reduce the silence, we increase the noise, and the more we fight against the noise, the more the silence becomes loud. In computer science, this complexity is assessed by two complementary metrics named the precision and the recall. To overcome this challenge, we took advantage of the applications in business intelligence (BI), the tools in reporting and the systems of knowledge management. Generally dedicated to dashboarding or back-office tools, we propose to the citizens to extend the queries results with a synthetic information provided by the linguistic analyses and integrated visually and progressively.

## 4. Tool Development

For the development of the tool, we had to tackle different technological problems. We will present the solutions, shown in Figure 11 in Appendix Section, that we have chosen for each problem. First, we used the Twitter API to retrieve the tweets directly from the official accounts. Then, we stored these tweets in the MongoDB<fnref n="2" /> NoSql database; its advantage consists in a flexible, document-oriented structure that does not require complex queries to access the data. Then, we decided to use Elasticsearch to

<footnote n="2">²www.mongodb.com</footnote>
store the tweets; Elasticsearch (Kononenko et al., 2014) improves the response time of our tool especially when using the search engine.

Given that Elasticsearch's standard method performs a classical search without dealing with the derivatives of a word, before sending the data from MongoDB to Elasticsearch, we prepare a data index that takes into account the derivatives of the words. All the communication between these tools uses the Java language. For the analysis part, we compared several software packages for the linguistic analysis of corpus (Hyperbase, Lexico3, Trameur, TXM and Iramuteq). We studied their analyses but also their availability in open source and/or API. After our study, we decided to use several features of Iramuteq<fnref n="3" /> that are implemented in PHP and available in open source. To this end, several modifications were needed in the implementation of Iramuteq. We also used PHP Word Cloud<fnref n="4" /> for word clouds and pChart<fnref n="5" /> for graphics.

After the election, a second version of the platform was released with new features: the first one, for data visualizations (as shown in Figure 7 in Appendix Section), and the second one, for sub-corpus extraction from the complete corpus by the candidate and the period (as shown in Figure 4). The latter is very important and useful for the humanities and social sciences community. Indeed, researchers in this area are interested in specific political issues, but they do not have access to tools/platforms allowing them to extract and structure their corpus before usage.

<figure><img src="image_18.png" /><figcaption>Figure 4: Corpus generator feature.</figcaption></figure>

# 5. Conclusion and Perspectives

#Idéo2017 combines different technologies and inputs, which give to citizens the opportunity to grasp a part of the discursive issues of the election. This development, which can be enriched, allows to easily use a set of features usually accessible by software requiring different transformations of the data.

## 5.1. Creation of the #Idéo2017 Corpus

For the period from September 1st 2016 to May 7th 2017, 42290 tweets were extracted for the 11 candidates. These tweets were gathered in a collection that will be published in a TEI corpus in the standards of the Ortolang platform. The publication of this corpus under the requested standard is founded with the support of the CORLI consortium<fnref n="6" />. This process will follow the guidelines listed in the

acquisition report (Longhi, 2014) written when we released the Polititweets corpus. Concerning the juridical issues, the creation of this corpus is legal. The position of Twitter is the following:

* "Please review the Twitter Rules (which are part of these Terms) to better understand what is prohibited on the Service. We reserve the right at all times (but will not have an obligation) to remove or refuse to distribute any Content on the Services, to suspend or terminate users, and to reclaim usernames without liability to you. We also reserve the right to access, read, preserve, and disclose any information as we reasonably believe is necessary to (i) satisfy any applicable law, regulation, legal process or governmental request, (ii) enforce the Terms, including investigation of potential violations hereof, (iii) detect, prevent, or otherwise address fraud, security or technical issues, (iv) respond to user support requests, or (v) protect the rights, property or safety of Twitter, its users and the public."

* "Except as permitted through the Services, these Terms, or the terms provided on dev.twitter.com, you have to use the Twitter API if you want to reproduce, modify, create derivative works, distribute, sell, transfer, publicly display, publicly perform, transmit, or otherwise use the Content or Services."

Thus, Twitter does not disclose personally identifying information to third parties except in accordance with their Privacy Policy. Moreover, Twitter encourages and allows broad re-use of content. The Twitter API exists to enable this.

## 5.2. #Idéo2017 as a Prototype: the Reproducibility of the Platform

#Idéo2017 allowed to the French electors to analyze the discourse of the candidates by means of their tweets. But, the utility of this platform is not limited to the French election, because it can be modified to different needs and usages, and adapted to other contexts. Thus, after the presidential election, we released two other versions of the platform: #législatives2017 (http://ideo2017.ensea.fr/legislatives2017/) and #quinquennat (http://ideo2017.ensea.fr/quinquennat/) which allow, in the first case to analyze the tweets of the main political parties during the election of deputies to the French National Assembly, and in the second case to daily analyze the beginning of Emmanuel Macron's presidential mandate via the tweets of the current political protagonists.

A set of new features were also proposed such as statistical analyses of the hashtags and mentions with Kibana. As a perspective, the #Idéo2017 prototype could be adapted to be used as a competitive intelligence, a measurement and a visualization tool analyzing people's opinion on Twitter. We can imagine dealing with political, social or cultural subjects through the integration of influential accounts, but also individual accounts of the users.

<footnote n="3">³www.iramuteq.org</footnote>

<footnote n="4">⁴github.com/sixty-nine/PHP Word Cloud</footnote>

<footnote n="5">⁵www.pchart.net</footnote>

<footnote n="6">⁶https://corli.huma-num.fr/</footnote>
# 6. Acknowledgements

UCP Foundation, Vo tv/Vo News, IUT of Cergy-Pontoise,
University of Cergy-Pontoise, ETIS Lab, AGORA Lab.

# 7. References

Alduy, C. (2017). *Ce Qu’Ils Disent Vraiment. Decoder Le Discours Des Presidentiables*. SEUIL.

Conover, M., Ratkiewicz, J., Francisco, M., Gonçalves, B., Flammini, A., and Menczer, F. (2011). Political polarization on twitter. In *Proc. 5th International AAAI Conference on Weblogs and Social Media (ICWSM)*.

Gayo-Avello, D., Metaxas, P. T., and Mustafaraj, E. (2011). Limits of electoral predictions using twitter. In Lada A. Adamic, et al., editors, *ICWSM*. The AAAI Press.

Johnson, K. and Goldwasser, D. (2016). “all I know about politics is what I read in twitter”: Weakly supervised models for extracting politicians’ stances from twitter. In *COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11-16, 2016, Osaka, Japan*, pages 2966–2977.

Kaplan, A. M. and Haenlein, M. (2010). Users of the world, unite! the challenges and opportunities of social media. *Business Horizons*, 53(1):59–68.

Kononenko, O., Baysal, O., Holmes, R., and Godfrey, M. W. (2014). Mining modern repositories with elasticsearch. In *Proceedings of the 11th Working Conference on Mining Software Repositories*, MSR 2014, pages 328–331, New York, NY, USA. ACM.

Longhi, J. and Saigh, D. (2016). A textometrical analysis of french arts workers “fr.intermittents” on twitter. In *4th conference CMC and Social Media Corpora for the Humanities*, Ljubljana, Slovenia, September.

Longhi, J., Marinica, C., Borzic, B., and Alkhouli, A. (2014). Polititweets : corpus de tweets provenant de comptes politiques influents. In Chanier T., editor, *Banque de corpus CoMeRe*. Ortolang.fr.

Longhi, J. (2013). Essai de caractérisation du tweet politique. *L'information grammaticale*.

Longhi, J. (2014). Proposition pour l'acquisition d'un corpus de tweets (cmr-polititweets-teiv1-manuel.pdf). In J. Longhi, et al., editors, *Polititweets : corpus de tweets provenant de comptes politiques influents*. Banque de corpus CoMeRe. Ortolang.fr : Nancy. [cmr-polititweets-tei-v1].

Mercier, A. (2014). Twitter l’actualité : usages et réseautage chez les journalistes français. recherches en communication. *Recherches en communication 40*, pages 111–132.

Metaxas, P. T., Mustafaraj, E., and Gayo-Avello, D. (2011). How (not) to predict elections. In *SocialCom/PASSAT*, pages 165–171. IEEE.

Roginsky, S. and Cock, B. D. (2015). Faire campagne sur twitter. modalités d'énonciation et mises en récit des candidats à l'élection européenne. *Les Cahiers du numérique*, 11(4):119–144.

Tumasjan, A., Sprenger, T. O., Sandner, P. G., and Welpe, I. M. (2010). Predicting elections with twitter: What

140 characters reveal about political sentiment. In *Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media*.

Vidak, M. and Jackiewicz, A. (2016). Les outils multimodaux de twitter comme moyens d’expression des émotions et des prises de position. *Cahiers de praxématique*, (66).

# Appendix

<figure><img src="image_19.png" /><figcaption>Figure 5: Analyses for the word *work*: frequency.</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>Figure 6: Analyses for the word *work*: associated words.</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>Figure 7: Data visualization.</figcaption></figure>
<figure><img src="image_22.png" /><figcaption>Figure 8: Analysis of the tweets of the candidate Emmanuel Macron: the most frequent words.</figcaption></figure>

<figure><img src="image_23.png" /><figcaption>Figure 9: The user interface of the search engine.</figcaption></figure>

<figure><img src="image_24.png" /><figcaption>Figure 10: Analyses for the word work: word cloud.</figcaption></figure>

<figure><img src="image_25.png" /><figcaption>Figure 11: Schema of the tool.</figcaption></figure>
# Connecting Resources: Which Issues Have to be Solved to Integrate CMC Corpora from Heterogeneous Sources and for Different Languages?

Michael Beißwenger<fnref n="1" />, Ciara Wigham<fnref n="2" />, Carole Etienne³, Darja Fišer<fnref n="4" />,
Holger Grumt Suárez⁵, Laura Herzberg⁶, Erhard Hinrichs<fnref n="7" />,
Tobias Horsmann<fnref n="1" />, Natali Karlova-Bourbonus⁵, Lothar Lemnitzer⁸,
Julien Longhi<fnref n="9" />, Harald Lüngen¹⁰, Lydia-Mai Ho-Dac<fnref n="11" />,
Christophe Parisse¹², Céline Poudat<fnref n="13" />, Thomas Schmidt¹⁰,
Egon Stemle¹⁴, Angelika Storrer⁶, Torsten Zesch<fnref n="1" />

michael.beisswenger@uni-due.de, ciara.wigham@uca.fr, carole.etienne@ens-lyon.fr, darja.fiser@ff.uni-lj.si,
Holger.H.Grumt-Suarez@germanistik.uni-giessen.de, lherzber@mail.uni-mannheim.de,
erhard.hinrichs@uni-tuebingen.de, tobias.horsmann@uni-due.de, Natali.Karlova-Bourbonus@zmi.uni-giessen.de,
lemnitzer@bbaw.de, julien.longhi@u-cergy.fr, luengen@ids-mannheim.de, hodac@univ-tlse2.fr,
cparisse@u-paris10.fr, celine.poudat@unice.fr, thomas.schmidt@ids-mannheim.de,
egon.stemle@eurac.edu, astorrer@mail.uni-mannheim.de, torsten.zesch@uni-due.de

The paper reports on the results of a scientific colloquium dedicated to the creation of standards and best practices which are needed to facilitate the integration of language resources for CMC stemming from different origins and the linguistic analysis of CMC phenomena in different languages and genres. The key issue to be solved is that of interoperability – with respect to the structural representation of CMC genres, linguistic annotations metadata, and anonymization/pseudonymization schemas. The objective of the paper is to convince more projects to partake in a discussion about standards for CMC corpora and for the creation of a CMC corpus infrastructure across languages and genres. In view of the broad range of corpus projects which are currently underway all over Europe, there is a great window of opportunity for the creation of standards in a bottom-up approach.

**Keywords:** corpora, research infrastructures, annotation, anonymization

## 1. Background and Motivation

The paper reports on the results of a scientific colloquium (https://sites.google.com/view/dhcmc2017/) dedicated to the creation of standards and best practices which are needed as a prerequisite for the exchange, interconnection, and combined analysis of CMC corpora of different origins, for different languages and different genres. The goal of the colloquium which was held with funding from the French Embassy in Germany was to determine open issues which have to be solved to represent CMC corpus data (including metadata and annotations) using interoperable formats. From a wider perspective, the colloquium addressed not only issues of interoperability of CMC corpora between one another but also the interoperability of CMC corpora with corpora of other types, namely text corpora and spoken language corpora. To make the goal of interoperability and of the development of standards more tangible for the community of CMC researchers and corpus creators, the colloquium outlined the scenario of creating a multilingual and genre-heterogeneous demo corpus that would include samples from existing CMC corpora in different languages and on different CMC genres and which would also present CMC in the context of other discourse domains through the inclusion of samples from text and spoken language corpora.

In the following sections, we summarize results and remaining open issues towards the creation of standards and towards an interoperability of corpora as were determined during the colloquium. The overview is based on input from representatives of the following corpus projects and language resource infrastructure projects:

### a) CMC corpora:

* CoMeRe: a collection of 14 French corpora for 9 different CMC genres (including multimodal genres) represented in TEI (Chanier et al., 2014) and available for download (CC BY, OpenData) via ORTOLANG. (Longhi and Wigham, 2015)<fnref n="1" />

* DEREKO-News: Corpus of German Newsgroups in DEREKO, since 2013, 98 million tokens (Schröck and Lüngen, 2015).<fnref n="2" />

* DEREKO-Wikipedia: Wikipedia corpora in DEREKO: German language article, talk and user talk (Margaretha and Lüngen, 2014), 581 million tokens, available for online querying via COSMAS II.

* DiDi corpus: The CMC corpus from the DiDi project with 570.000 tokens of German, Italian and South Tyrolean Facebook posts and interactions, available

<footnote n="1">¹ University of Duisburg-Essen, Germany ² University Clermont-Auvergne, France ³ ICAR Laboratory Lyon, France</footnote>

<footnote n="4">⁴ University of Ljubljana, Slovenia ⁵ Justus-Liebig-Universität Gießen, Germany ⁶ University of Mannheim, Germany</footnote>

<footnote n="7">⁷ Eberhard-Karls-Universität Tübingen, Germany ⁸ Berlin-Brandenburg Academy of Sciences, Germany</footnote>

<footnote n="9">⁹ Université de Cergy-Pontoise, France ¹⁰ Institute for the German Language, Mannheim, Germany</footnote>

<footnote n="11">¹¹ Université Toulouse 2, France ¹² Université Paris Nanterre, France</footnote>

<footnote n="13">¹³ Université Nice Côte d'Azur, France ¹⁴ Eurac Research, Bolzano, Italy</footnote>

<footnote n="1">¹ http://hdl.handle.net/11403/comere</footnote>

<footnote n="2">² https://cosmas2 ids-mannheim.de/</footnote>
for online querying via ANNIS (Frey et al., 2016).<fnref n="3" />

* *Dortmund Chat Corpus 2.0:* corpus of German chat discourse represented in TEI, available as part of the CLARIN-D corpus infrastructure (Lüngen et al., 2016, Beißwenger et al., 2017).

* *DWDS blog corpus:* The blog corpus in the corpus collection of the DWDS project: 103 million tokens from CC-licensed, mainly German blog entries, available for online querying (Barbaresi, 2016).<fnref n="4" />

* *Gießen scienceblog corpus:* ongoing project at the University of Gießen; goal: creation and annotation of a corpus of German science blogs. (Grumt Suárez et al., 2016)

* *Janes corpus:* The Corpus of Nonstandard Slovene comprising >200 million tokens from tweets, forum posts, blogs, comments on news articles and Wikipedia discussions (Fišer et al., 2016, 2017).<fnref n="5" />

* *MoCoDa:* ongoing project at University of Duisburg-Essen (2017–), goal: creation of a database with a web frontend for repeated, donation-based collection of mobile CMC (whatsapp, sms & co.).

* *Wikiconflits:* TEI-CMC-encoded corpus of French Wikipedia talk pages associated to conflicts capturing 7 topics related to (pseudo-)science with 4456 posts (Poudat et al., 2017).

* *WikiTalk:* TEI-P5-encoded corpus of French Wikipedia talk pages (365,612 pages, >1M threads).

* *DiscoWiki:* corpus of an ongoing project with the goal of annotating relevant characteristics for conflict detection and description at the thread level; corpus based on a selection of talk pages extracted from *Wikiconflits* and *WikiTalk* (Ho-Dac and Laippala, 2017).

b) *Corpora of other types:*

* Text corpus collection at the BBAW, Berlin (DWDS corpora) (Geyken et al., 2017).

* German reference corpus at the IDS Mannheim (DEREKO) (Lüngen, 2017).

* The French spoken language corpora *Colaje* (Morgenstern and Parisse, 2012) and *Orfeo*<fnref n="6" />.

* Research and Teaching Corpus of Spoken German (FOLK) at the IDS Mannheim (Schmidt, 2016).<fnref n="7" />

c) *Corpus infrastructure projects:*

* CLARIN-D, the German national branch of the Eurpoean CLARIN initiative.<fnref n="8" />

* ORTOLANG, the French infrastructure for Open Resources and TOols for LANGuage.<fnref n="9" />

The objective of presenting this summary at CMC-Corpora17 is to convince more people with corpus

projects in the field to join the discussion about standards for CMC corpora and – probably – for transforming the idea of a CMC demo corpus into a cooperative project with the participation of a broad range of projects and researchers.

## 2. State of the art and open issues

### 2.1 Basic representation format

Since 2012, the special interest group (SIG) “computer-mediated communication” in the Text Encoding Initiative (TEI) has created three TEI extensions for the representation of CMC data and tested these extensions with different CMC genres and corpora for French and German (Beißwenger et al., 2012, Chanier et al., 2014, Lüngen et al., 2016). All three extensions are available in the form of RNG schemas and ODD documents and ready to be used for annotation in other projects.<fnref n="10" /> Current work of the SIG is focused on the transformation of the available extensions into a “feature request” which is necessary to make an official suggestion for extending the TEI guidelines with models for CMC. Discussions at the colloquium showed that for further dissemination of the TEI extensions for CMC is desirable

* to document practices, tools, and guideline documents from projects that have already converted raw data into TEI and make them available as Open Access resources to facilitate the conversion of corpus data into TEI for colleagues who have not worked with TEI before;

* to diffuse information/documentation concerning toolchains that can be used on CMC data as currently there is little support for users to help them process resources in one way or another.

One possible outlet for the diffusion of resources of this type would be the CLARIN Language resource switchboard (Zinn, 2016) that is currently being developed within CLARIN-PLUS as a means to link linguistic resources with the tools that can process them. It aims to create a single point of access where users can find the tools that fit their needs and their language resource.

### 2.2 Natural language processing

Different creators of CMC corpora are using a different types of linguistic annotations and different (e.g., language-specific) tagsets. For mapping annotations in existing corpus resources without the need to perform a complete re-annotation, the resources of the Universal Dependencies Initiative (UD)<fnref n="11" /> may provide CMC researchers with “a universal inventory of categories and guidelines to facilitate consistent annotation of similar constructions across languages, while allowing language-specific extensions when necessary”. Nevertheless, UD does not provide any tags for the

<footnote n="3">³ http://www.eurac.edu/didi</footnote>

<footnote n="4">⁴ https://www.dwds.de</footnote>

<footnote n="5">⁵ http://nl.ijs.si/janes/</footnote>

<footnote n="6">⁶ http://www.projet-orfeo.fr/</footnote>

<footnote n="7">⁷ http://agd.ids-mannheim.de/folk.shtml</footnote>

<footnote n="8">⁸ https://www.clarin-d.de/de/</footnote>

<footnote n="9">⁹ http://ortolang.fr</footnote>

<footnote n="10">¹⁰ These resources are available via the SIG space in the TEI wiki: https://wiki.tei-c.org/index.php?title=SIG:CMC</footnote>

<footnote n="11">¹¹ http://universaldependencies.org/introduction.html</footnote>
description of CMC-specific phenomena. In order to
determine the feasibility of mapping tagset extensions
used in CMC corpora in different languages onto each
other, the group agreed that it will be helpful to compare
how CMC-specific phenomena are treated in the DiDi,
CoMeRe, CLARIN-D and Janes (and other CMC)
corpora and check the extent to which they are already
compatible with each other / analyse the effort needed to
transform them into a compatible structure. This could be
the subject of a workshop or short-term project in the near
future. Further work on this topic should also take into
account the tagsets and resources both from the
PoSTWITA shared task on PoS tagging Italian social
media data<fnref n="12" /> and from the EmpiriST shared task on PoS
tagging of German CMC and web corpora data<fnref n="13" />.

## 2.3 Anonymization

In order for CMC corpora to conform to the restrictions of
national data protection rights (DPR), existing corpora
projects have developed different strategies and practices
for removing or masking personal data. From a DPR
perspective, it is required that the data included in corpora
which are made available to the scientific community
should be represented in a way that the expenses that one
would have to invest to identify a certain person are so
high that it seems unrealistic that anybody would invest
them. Different annymization approaches have been
adopted by the different projects. It may be a fruitful topic
for further investigation (i.e. in form of a workshop or
short-term project) to compare, in detail, the results of
anonymization vs. pseudonymization approaches adopted
in different projects to determine the best balance between
preserving as much of the semantics of the original data as
possible (which is an important resource especially for
qualitative analyses) while on the other hand staying as
feasible as possible in terms of time-cost factors. One
action point could be to make anonymization guidelines
from different projects available to other colleagues so
that procedures employed can be re-used. Examples for
best practices for anonymizing CMC corpora have been
developed and employed in the DiDi project (Frey et al.,
2015), in the CLARIN-D curation project ChatCorpus2
CLARIN (Lüngen et al., 2017) and in CoMeRe (Chanier
and Jin, 2013).

## 2.4 Metadata

The creation and representation of metadata for CMC
corpora and for single interactions preserved in them is a
huge and urgent open issue. One key point is for metadata
to include information about the version of the
communication platform, given the rate at which
communication platforms evolve. In the future, should
access to older version no longer be available, at least
prose descriptions of the communication platform at the

time of data collection would be practical for future
corpora end-users and mandatory to guarantee corpus data
sustainability. A comparison of the metadata captured in
different corpus projects (be it in the form of annotations,
be it in the form of prose descriptions) could be a fruitful
contribution to a more precise discussion of (i) what types
of metadata are needed in and for CMC corpora, (ii) to
what extent these metadata are specific to individual
CMC genres or for CMC in general, (iii) the preservation
of which types of metadata could be standardized, and (iv)
how a basic representation schema for CMC metadata
(e.g., in the TEI Header) could look like.

## 3. Outlook

In view of the broad range of corpus projects which are
currently underway all over Europe (Beißwenger et al.,
2017a), there is a great window of opportunity for the
creation of standards for CMC corpora in a bottom-up
approach. The discussions obtained on this issue at
CMCCorpora17 shall be included in the creation of a
white paper giving a more precise outline of future work
for the issues addressed in this paper. The creation of a
demo corpus including samples from different existing
CMC corpora could support the further investigation of
open issues and provide valuable feed-back for existing
best practices in the field. A prerequisite would be a
"critical mass" of resources and researchers who are
willing to contribute to the creation of such a corpus. As a
first step of preparatory work it is planned to set up a
platform for the exchange of tools, tips and case studies
between projects in order to facilitate the dissemination of
knowledge and best practices.

## 4. References

Barbaresi, A. (2016). Efficient construction of meta-data-enhanced web corpora. In *Proceedings of the 10th Web as Corpus Workshop, Association for Computational Linguistics*, pp. 7-16. https://hal.archives-ouvertes.fr/hal-01371704v2/document.

Beißwenger, M., Ermakova, M., Geyken, A., Lemnitzer, L., Storrer, A. (2012). A TEI Schema for the Representation of Computer-mediated Communication. *Journal of the Text Encoding Initiative* 3. http://jtei.revues.org/476 (DOI: 10.4000/jtei.476).

Beißwenger, M., Lüngen, H., Schallaböck, J., Weitzmann, J.H., Herold, A., Kamocki, P., Storrer, A., Wildgans, J. (2017, in press). Rechtliche Bedingungen für die Bereitstellung eines Chat-Korpus in CLARIN-D: Ergebnisse eines Rechtsgutachtens. In: M. Beißwenger (Ed.), *Empirische Erforschung internetbasierter Kommunikation*. Berlin/New York: de Gruyter (Empirische Linguistik / Empirical Linguistics).

Beißwenger, M., Chanier, T., Erjavec, T., Fišer, D., Herold, A., Lubešic, N., Lüngen, H., Poudat, C., Stemle, E., Storrer, A., Wigham, C. (2017a). Closing a Gap in the Language Resources Landscape: Groundwork and Best Practices from Projects on Computer-mediated Communication in four European Countries. In: L. Borin (Ed.), *Selected papers from the CLARIN Annual*

<footnote n="12">¹² http://corpora.ficit.unibo.it/PoSTWITA/index.php?slab=guidelines</footnote>

<footnote n="13">¹³ Tagset/guidelines: https://sites.google.com/site/empirist2015/; results: WAC-X/EmpiriST (2016).</footnote>
Conference 2016, Aix-en-Provence, 26-28 October 2016 (Linköping University Electronic Conference Proceedings 136), pp. 1-18. http://www.ep.liu.se/ecp/ contents.asp?issue=136

Chanier, T., Jin, K. (2013). *Defining the online interaction space and the TEI structure for CoMeRe corpora*. *Projet CoMeRe (Communication Médiée par les Réseaux)*. https://corpuscomere.files.wordpress.com/2014/01/tei-cmc-comere-interactionspace_131231.pdf

Chanier, T., Poudat, C., Sagot, B., Antoniadis, G., Wigham, C., Hriba, L., Longhi, J., Seddah, D. (2014). The CoMeRe corpus for French: structuring and annotating heterogeneous CMC genres. *Journal of language Technology and Computational Linguistics*, 29(2), pp. 1–30. http://www.jlcl.org/2014_Heft2/1Chanier-et-al.pdf.

Fišer, D., Erjavec, T., Ljubešić, N. (2017). The compilation, processing and analysis of the Janes corpus of Slovene user-generated content: In C.R. Wigham, G. Ledegen (Eds.), *Corpus de Communication Médiée par les Réseaux. Construction, structuration, analyse*. Paris: L’Harmattan (Humanités numériques), pp. 125–138.

Fišer, D., Erjavec, T., Ljubešić, N. (2016). JANES v0.4: Korpus slovenskih spletnih uporabniških vsebin. *Slovenščina 2.0*, 4(2), pp. 67–99.

Frey, J.C., Glaznieks, A., Stemle, E.W. (2015). The DiDi Corpus of South Tyr-olean CMC Data. In *Proceedings of the 2nd Workshop on Natural Language Processing for Computer-Mediated Communication / Social Media (NLP4CMC2015)*, Essen, Germany.

Frey, J.-C., Glaznieks, A., Stemle, E. (2016). The DiDi Corpus of South Tyrolean CMC Data: A Multilingual Corpus of Facebook Texts. In *Proceedings of the Third Italian Conference on Computational Linguistics (CLiC-it 2016)*. ceur-ws.org/Vol-1749/paper27.pdf.

Geyken, A., Barbaresi, A., Didakowski, J., Jurish, B., Wiegand, F., Lemnitzer, L. (2017, in press). Die Korpusplattform des „Digitalen Wörterbuchs der deutschen Sprache“ (DWDS). *Zeitschrift für germanistische Linguistik*, 45 (2).

Grumt Suárez, H., Karlova-Bour bonus, N., Lobin, H. (2016). Compilation and Annotation of the Discourse-structured Blog Corpus for German. In *Proceedings of the 4th Conference on CMC and Social Media Corpora for the Humanities (cmc-corpora2016)*, Ljubljana, Slovenia. http://nl.ijs.si/janes/wp-content/uploads/2016/09/CMC-2016_Grumt_et_al_Compilatio-and-Annotation.pdf.

Ho-Dac, L.-M., Laippala, V. (2017). Le corpus WikiDisc, une ressource pour la caractérisation des discussions en ligne. In C.R. Wigham, G. Ledegen (Eds.), *Corpus de Communication Médiée par les Réseaux. Construction, structuration, analyse*. Paris: L’Harmattan (Humanités numériques), pp. 107–124.

Longhi, J., Wigham, C.R. (2015). *Structuring a CMC corpus of political tweets in TEI: corpus features, ethics and workflow*. Poster at Corpus Linguistics 2015, Lancaster, United Kingdom. https://halshs.archives-

ouvertes.fr/halshs-01176061.

Lüngen, H. (2017). DEREKO – Das Deutsche Referenz-korpus. Schriftkorpora der deutschen Gegenwarts-sprache am Institut für Deutsche Sprache in Mannheim. *Zeitschrift für germanistische Linguistik*, 45 (1), pp. 161–170.

Lüngen, H., Beißwenger, M., Ehrhardt, E., Herold, A., Storrer, A. (2016). Integrating corpora of computer-mediated communication in CLARIN-D: Results from the curation project ChatCorpus2CLARIN. In *Proceedings of the 13th Conference on Natural Language Processing (KONVENS 2016)*, Bochum, Germany, pp. 156–164. https://www.linguistics.rub.de/konvens16/pub/20_konvensproc.pdf.

Lüngen, H., Beißwenger, M., Herzberg, L., Pichler, C. (2017). Anonymisation of the Dortmund Chat Corpus 2.1. In *Proceedings of the 5th Conference on CMC and Social Media corpora for the Humanities, Bolzano, Italy*.

Margaretha, E., Lüngen, H. (2014). Building Linguistic Corpora from Wikipedia Articles and Discussions. *Journal of language Technology and Computational Linguistics*, 29(2), pp. 59–82. http://www.jlcl.org/2014_Heft2/3MargarethaLuengen.pdf.

Morgenstern, A. & Parisse, C. (2012). The Paris corpus. *Journal of French Language Studies*, 22, pp. 7–12.

Poudat, C., Grabar, N., Paloque-Berges, C., Chanier, T., Juin, K. (2017). Wikiconflits: un corpus de discussions éditoriales conflictuelles du Wikipédia francophone. In C.R. Wigham, G. Ledegen (Eds.), *Corpus de Communication Médiée par les Réseaux. Construction, structuration, analyse*. Paris: L’Harmattan (Humanités numériques), pp. 19–36.

Schmidt, T. (2016). Good practices in the compilation of FOLK, the research and teaching corpus of spoken German. *International Journal of Corpus Linguistics*, 21(3), pp. 396–418.

Schröck, J., Lüngen, H. (2015). Building and Annotating a Corpus of German-Language Newsgroups. In *Proceedings of the 2nd Workshop on Natural Language Processing for Computer-Mediated Communication / Social Media (NLP4CMC2015)*, Essen, Germany, pp. 17–22. https://sites.google.com/site/nlp4cmc2015/program

[WAC-X/EmpiriST 2016] Proceedings of the 10th Web as Corpus Workshop (WAC-X) and the EmpiriST Shared Task. Stroudsburg: Association for Computational Linguistics (ACL Anthology W16-26). http://aclweb.org/anthology/W/W16/W16-26.pdf.

Zinn, C. (2016). The CLARIN Language Resource Switchboard. In *Proceedings of the CLARIN Annual Conference, Aix-en-Provence, France*, https://www.clarin.eu/sites/default/files/zinn-CLARIN2016_paper_26.pdf.
# “You’re trolling because...” – A Corpus-based Study of Perceived Trolling and Motive Attribution in the Comment Threads of Three British Political Blogs

Márton Petykó

Department of Linguistics and English Language, Lancaster University
County South, Lancaster University, Lancaster LA1 4YL, United Kingdom
E-mail: m.petyko@lancaster.ac.uk

## Abstract

This paper investigates the linguistically marked motives that participants attribute to those they call trolls in 991 comment threads of three British political blogs. The study is concerned with how these motives affect the discursive construction of trolling and trolls. Another goal of the paper is to examine whether the mainly emotional motives ascribed to trolls in the academic literature correspond with those that the participants attribute to the alleged trolls in the analysed threads. The paper identifies five broad motives ascribed to trolls: emotional/mental health-related/social reasons, financial gain, political beliefs, being employed by a political body, and unspecified political affiliation. It also points out that depending on these motives, trolling and trolls are constructed in various ways. Finally, the study argues that participants attribute motives to trolls not only to explain their behaviour but also to insult them.

**Keywords:** troll(ing), motive attribution, blog

## 1. Introduction

This corpus-based case study investigates a prominent social phenomenon of computer-mediated communication: trolling. It aims to identify the linguistically marked motives that participants attribute to those whom they call trolls in 991 comment threads. These threads were published on three British political blogs, *Guardian Politics Blog*, *Guido Fawkes*, and *LabourList*. The paper is also concerned with how these motives affect the way trolling and trolls are discursively constructed in the threads. Another goal of the paper is to examine to what extent the motives attributed to trolls in the academic literature correspond with those that the participants attribute to the alleged trolls.

The analysis focuses on 2,036 motivation-related metapragmatic comments taken from these 991 threads. In these comments, participants call other users trolls or identify comments as trolling and also discuss the possible reasons why the alleged trolls are trolling. The study first presents a taxonomy of the linguistically marked motives in these comments and then it applies this taxonomy to annotate the comments. Thus, it develops a discursive-pragmatic annotation system for linguistically marked motive attribution in computer-mediated interactions.

This study can be situated within the fields of corpus-based discourse analysis (Baker, 2006) and pragmatics (Culpeper & Hardaker, 2016). Beyond trolling, the paper has relevance to the pragmatics of computer-mediated communication (Herring, Stein, & Virtanen, 2013) and within that, to the study of metapragmatic comments in computer-mediated interactions (Tanskanen, 2007).

## 2. Literature Review

'Trolling' is usually described as a set of goal-driven behaviours, while 'troll' is deemed a behaviour-based identity (Hardaker, 2013). The most often mentioned goals attributed to trolls are: attracting other users' full attention (Hardaker, 2010), triggering intense unpleasant emotional reactions (Thacker & Griffiths, 2012), eliciting potentially offensive responses from others (Morrissey, 2010), causing,

perpetuating or escalating conflict (Galán-García et al., 2014), disrupting the ongoing interaction (Binns, 2012), and deceiving or manipulating others (Donath, 1999).

The discursive actions perceived as acts of trolling are: repeating the same utterance (Shachaf-Hara, 2010), posting irrelevant or meaningless information (Morrissey, 2010), posting misleading or factually incorrect information (Hardaker, 2010), disseminating bad and/or dangerous advice (Donath, 1999), ignoring, despising, rejecting or attacking the core values of the interaction (Utz, 2005), (hypo)criticising others (Hardaker, 2013), and directly insulting, threatening or otherwise attacking others (Herring et al., 2002).

Although the motives for trolling are also often mentioned in the literature, most studies do not attempt to empirically examine them but they instead treat them in a speculative manner (Hopkinson, 2013). This is a clear gap in the literature, to which this study is related.

Trolling is usually approached as an emotionally motivated individual behaviour. The most often mentioned motive is that trolls engage in this behaviour because they simply enjoy it or its consequences (Hardaker, 2010). Further emotional motives are also mentioned, such as boredom (Baker, 2001), a need for attention or achievement, revenge (Shachaf & Hara, 2010), loneliness, curiosity, malevolence (Fichman & Sanfilippo, 2015), a desire for control and self-empowerment, hate towards specific participants, and hostility to the purpose of the interaction (Herring et al., 2002). It is also suggested that trolls can be motivated by specific political goals and (political) ideologies (Özsoy, 2015). A key aim of this study is to examine whether the above-mentioned motives correspond with those that the participants attribute to the alleged trolls.

## 3. Data and Method

### 3.1. Data collection

The corpus consists of 991 comment threads of three British political blogs, *Guardian Politics Blog* (GP), *Guido Fawkes* (GF), and *LabourList* (LL). In this paper, a 'thread' refers to the comments of a blog post. These 991 comment
threads thus include 617,782 comments of 991 blog posts. The size of the corpus is around 21.9 million tokens.

GP is the political blog section of a major British newspaper, *The Guardian*. GP can be characterised as a liberal centre-left political blog with more permanent contributors and a highly diverse readership. The blog posts are written by professional journalists in a neutral manner while the commenters represent the entire political spectrum.

GF is an independent libertarian and anti-establishment political blog, which was founded by Paul Staines. Whilst GP has more authors, Staines most likely remains the main contributor. The blog posts are often sarcastic and overtly criticise or mock the major British political parties, such as the Conservative Party, the Labour Party, and the Liberal Democrats, and their leading politicians. Similarly to most political blogs, the commenters do not form a homogeneous community. However, many of them explicitly support the right-wing UK Independence Party (UKIP). This strongly relates to GF's anti-establishment stance as many perceive UKIP as an anti-establishment party.

LL overtly supports the centre-left Labour Party and aims to provide a forum for debate within the Labour Party. The blog posts are written by numerous contributors. While LL itself is said to be independent from the Labour Party, many of the contributors are Labours MPs or are otherwise affiliated with the Labour Party. Rather unsurprisingly, most commenters support the Labour Party and have left-wing leanings.

The threads were selected based on two criteria: (1) The thread had to be published on GP, GF or LL between 1 January and 31 December 2015. (2) The thread had to include at least one comment in which a participant called at least one other participant a troll and/or described at least one comment as an act of trolling at least once (hereon referred to as a 'troll comment'). That is, at least one participant had to use a word form of the lexeme *troll*, such as *troll*, *trolling* or *troller* to refer to another participant or comment as illustrated in example (1).

(1) [guardian_65_22345]
stop posting rubbish, troll!

Data collection included the following steps:

(1) A list of 50 British political blogs active in 2015 was compiled. I considered a blog to be any website appearing on a blog hosting platform, such as.blogspot.com and/or that called itself a blog. They were deemed to be active in 2015 if at least one post was published between 1 January and 31 December 2015. Finally, I classified political blogs as those whose main topic is politics, i.e. the acquisition, distribution and practice of power in human communities, societies and states. Four sources for collection were used:

(a) **Teads list of top 100 British political blogs in September 2015.** Teads is a French technology company expert in video advertising solutions. It regularly publishes a list of top 100 British political blogs on its website.

(b) **Vuelio list of top 10 UK political blogs in October 2015.** Vuelio is a leading global provider of PR and Political Services Software. It publishes a list of the top10

UK political blogs.

(c) **Google search.** The search terms were *British political blog*, "British political blog" 'British political blog as exact term', *UK political blog* and "UK political blog" 'UK political blog as exact search term'.

(d) **The political blogs recommended on the already collected ones** were also considered.

(2) I gathered all those threads from these 50 blogs in which at least one participant/comment was deemed to be a troll/trolling. I manually searched 26,804 threads from 2015 for the *troll* character string, and found 1,712 relevant threads. Then I saved each thread in a separate txt file.

(3) For the purposes of this case study, I selected the first three blogs, GP, GF, and LL since these had the highest number of qualifying threads. I decided to focus on only these three blogs in this paper since although the original list consisted of 50 political blogs that cover the entire political spectrum from far right to far left, 58% of the collected troll threads come from these three blogs. Thus, GP, GF, and LL are the key British political blogs for analysing perceived trolling in the British political blogosphere and their troll threads constitute an adequate sample of the more comprehensive corpus that includes all the 1,712 troll threads of the 50 blogs. Furthermore, the aim of this paper is not to draw general conclusions on perceived trolling in the British political blogosphere but to provide a context-sensitive analysis of the motives attributed to trolls on three British political blogs where participants call others trolls considerably more often than on other British political blogs.

(4) Four versions of the corpus were created. Version 1 consists of complete comment threads with blog posts and metadata (nicknames, dates, URLs etc.). Version 2 also includes complete comment threads but without the blog posts and any metadata. The troll comments (\<tc\>\<tc\>) and the troll tokens within them (\<tt\>\<tt\>) are also annotated in this version. Version 3 has only the troll comments while Version 4 contains all non-troll comments.

Table 1 includes the number of blog posts, comments, tokens, troll comments, and troll tokens in the second version of the corpus.

<table><caption>Table 1: Threads, comments, and tokens in the corpus</caption><thead><tr><th></th><th>Overall</th><th>GP</th><th>GF</th><th>LL</th></tr></thead><tbody><tr><td>Threads</td><td>991</td><td>167</td><td>391</td><td>433</td></tr><tr><td></td><td>100%</td><td>16.9%</td><td>39.5%</td><td>43.7%</td></tr><tr><td>Comments</td><td>617,782</td><td>374,604</td><td>170,610</td><td>72,568</td></tr><tr><td></td><td>100%</td><td>60.6%</td><td>27.6%</td><td>11.7%</td></tr><tr><td>Tokens (million)</td><td>21.9</td><td>14.5</td><td>3.9</td><td>3.5</td></tr><tr><td></td><td>100%</td><td>66.2%</td><td>17.8%</td><td>16%</td></tr><tr><td>Troll comments</td><td>4,477</td><td>1,738</td><td>900</td><td>1,839</td></tr><tr><td></td><td>100%</td><td>38.8%</td><td>20.1%</td><td>41.1%</td></tr><tr><td>Troll tokens</td><td>4,884</td><td>1,894</td><td>955</td><td>2,035</td></tr><tr><td></td><td>100%</td><td>38.8%</td><td>19.6%</td><td>41.7%</td></tr></tbody></table>

The majority of the data comes from GP as 60.6% of the comments and 66.2% of the tokens were published on this blog. However, LL has the most troll comments.
## 3.2. Data analysis

Data analysis involved a corpus-based qualitative-interpretative analysis of the collected troll comments:

(1) Using the concordance lines of the search term `<tt>*troll*</tt>` in AntConc (Anthony, 2016), I selected and annotated those troll comments from Version 2 in which the assumed motives for trolling were discussed (hereon referred to as 'troll motive comments'). This is illustrated in example (2).

(2) [labourlist_333_21]

**The Tories must be really panicking if they hired A to troll the way he does here. You just can't get decent staff these days.**

(2) I identified the linguistically marked motives that participants attributed to those they called trolls and created a taxonomy from them.

(3) I described how the different linguistically marked motives affect the discursive construction of trolling and trolls in the comments.

(4) To determine how often the participants explicitly attribute the identified motives to the alleged trolls, I used the motives as descriptive categories and provided each troll motive comment with motive-related annotations.

(5) To make this discursive-pragmatic annotation process more transparent and systematic, I studied the n-grams and collocates of the search term `<tt>*troll*</tt>` in Version 2 and the positive keywords in Version 3 against Version 4 as a reference corpus using AntConc. (Settings for n-grams: search term: both on the left and on the right, cluster size: between 2 and 6, min frequency: 5 and min range 1. Settings for collocates: window span: 5L5R, statistic: Mutual Information (MI), min MI score: 3.0, min frequency: 5. Settings for keywords: keyness statistic: log-likelihood (LL), min LL score: 3.84, min frequency: 5.) The aim of this step was to identify those words and expressions that mark a motive for trolling on their own.

(6) I summarised the quantitative results of the annotation.

# 4. Results

## 4.1. A taxonomy of the motives attributed to trolls

2,037 troll motive comments were identified in the corpus. 866 in GP, 279 in GF, and 892 in LL threads. Five motives for trolling emerged during the analysis of these comments: (1) various emotional mental health-related/social reasons, (2) financial gain, (3) unspecified political affiliation, (4) political beliefs, and (5) being employed by a political body.

The first motive covers various, often inter-related emotional states (e.g. boredom, loneliness or enjoyment), mental health issues, such as OCD, and social deprivation as reasons for trolling. When users suggest this motive, trolling is constructed as an emotionally motivated individual behaviour and trolls are portrayed as miserable individuals with emotional, mental health-related, and social problems.

(3) [guido_40_308]

*No wonder A keeps trolling here. He must be bored witless.*

The second motive refers to those cases where users imply that others are trolling because they are paid for it. However, it is not mentioned who pays the trolls and why. Here, trolling is constructed as a financially motivated individual activity and trolls are represented as rational but immoral and dishonest individuals.

(4) [guardian_48_3718]

*He/she might be an individual expressing their own opinion, legitimate in a democracy whether you or I agree with it. Whereas you could be described as a paid troll.*

The third motive represents those comments where users indicate that others are trolling due to their political affiliation. However, it remains unspecified whether the trolls merely support a political body or they work for it. Thus, the way trolling and trolls are constructed in these comments is ambiguous.

(5) [labourlist_432_1761]

**Tory troll hanging around Labour sites. Why?**

The fourth motive stands for those occasions when users imply that others are trolling since they support a political party or an ideology. Thus, trolling is constructed as an ideologically motivated individual activity and trolls are depicted as irrational political fanatics.

(6) [guido_90_573]

FFS we have an unusually high number of stupid socialist trolls in this thread. Are they seriously trying to tell us that Bin Laden wasn't a murderous butcher who had declared war on the western world? Keep it up you lefty trolls so everyone realises how vile and stupid you are.

The fifth motive is that certain users are trolling because a political body, namely a British political party, another country (Russia or Israel) or the European Union employs them and has ordered them to do so.

(7) [guardian_129_6462]

*Nice trolling from a Tory Party Central Office intern. Hopefully, come the 11th, you'll be signing on as unemployed.*

It is also repeatedly suggested that as part of their employment, these political bodies (5a) send the trolls to these blogs, (5b) tell them how to troll, (5c) sponsor their trolling and (5d) train them. Consequently, trolling is constructed as a financially and politically motivated and centrally organised collective activity while trolls are portrayed as unskilled and low-paid employees of low prestige who simply follow orders but do not necessarily support the political body that employs them.
## 4.2. Annotation of the Troll Motive Comments

The above-presented motives were used as descriptive categories to annotate the 2,037 troll motive comments in the corpus. Table 2 displays the n-grams and collocates of the troll tokens and the keywords in the troll comments that were used to make the annotation process more consistent.

<table><caption>Table 2: The n-grams, collocates and keywords marking a motive attributed to trolls</caption><thead><tr><th>N-gram</th><th>Collocate</th><th>Keyword</th><th>Motive tag</th></tr></thead><tbody><tr><td>paid troll(s)</td><td>paid</td><td>paid</td><td>2/5c</td></tr><tr><td>–</td><td>pay(ing)</td><td>–</td><td>2/5c</td></tr><tr><td>–</td><td>sponsored</td><td>sponsored</td><td>2/5c</td></tr><tr><td>–</td><td>funded</td><td>–</td><td>2/5c</td></tr><tr><td>Tory troll(er)(s)</td><td>Tory</td><td>Tory, torytroll</td><td>3/4/5</td></tr><tr><td>trolling Tory</td><td>Tory</td><td>Tory</td><td>3/4/5</td></tr><tr><td>–</td><td>conservative</td><td>–</td><td>3/4/5</td></tr><tr><td>Labour troll(s)</td><td>Labour</td><td>–</td><td>3/4/5</td></tr><tr><td>–</td><td>Corbynista(s), corbynite</td><td>–</td><td>3/4/5</td></tr><tr><td>–</td><td>Corbytrolls</td><td>Corbytroll(s)</td><td>3/4/5</td></tr><tr><td>–</td><td>Blairite</td><td>–</td><td>3/4/5</td></tr><tr><td>establishment troll</td><td>establishment</td><td>establishment</td><td>3/4/5</td></tr><tr><td>–</td><td>liblabcon</td><td>–</td><td>3/4/5</td></tr><tr><td>UKIP troll(er)(s)</td><td>UKIP</td><td>–</td><td>3/4/5</td></tr><tr><td>–</td><td>kipper</td><td>cyberkipper</td><td>3/4/5</td></tr><tr><td>Green (Party) troll</td><td>Green</td><td>Green</td><td>3/4/5</td></tr><tr><td>SNP troll</td><td>–</td><td>SNP</td><td>3/4/5</td></tr><tr><td>–</td><td>BNP</td><td>–</td><td>3/4/5</td></tr><tr><td>EU troll</td><td>EU</td><td>EU</td><td>3/4/5</td></tr><tr><td>right(-)wing troll(s)</td><td>right(-)wing</td><td>–</td><td>4</td></tr><tr><td>left(-)wing trolls</td><td>–</td><td>–</td><td>4</td></tr><tr><td>lefty/leftie troll</td><td>lefty, leftie(s)</td><td>leftie</td><td>4</td></tr><tr><td>leftard troll</td><td>leftard</td><td>leftard</td><td>4</td></tr><tr><td>Central Office Troll(s)</td><td>Central, office</td><td>central</td><td>5</td></tr><tr><td>CCHQ troll(s)</td><td>CCHQ, HQ</td><td>CCHQ, HQ</td><td>5</td></tr><tr><td>–</td><td>Lynton</td><td>Lynton</td><td>5</td></tr><tr><td>–</td><td>employed</td><td>–</td><td>5</td></tr><tr><td>troll army</td><td>army</td><td>–</td><td>5</td></tr><tr><td>–</td><td>Kremlin</td><td>Kremlin</td><td>5</td></tr><tr><td>Hasbara troll</td><td>Hasbara</td><td>Hasbara</td><td>5</td></tr></tbody></table>

Table 3 presents the proportion of those troll motive comments that were provided with a particular motivation-related tag. Note that as one comment could receive multiple tags, the sum of the percentages in the same column is not necessarily 100%.

<table><caption>Table 3: The proportion of troll motive comments provided with a particular motivation-related tag</caption><thead><tr><th>Motive</th><th>Tag</th><th>Overall</th><th>GP</th><th>GF</th><th>LL</th></tr></thead><tbody><tr><td>Emotional reasons</td><td>1</td><td>5.9%</td><td>6.8%</td><td>10%</td><td>3.8%</td></tr><tr><td>Financial gain</td><td>2</td><td>1.9%</td><td>2.3%</td><td>5.4%</td><td>0.3%</td></tr><tr><td>Unspecified political affiliation</td><td>3</td><td>65.3%</td><td>56.2%</td><td>38%</td><td>82.7%</td></tr><tr><td>Political beliefs</td><td>4</td><td>12.3%</td><td>15.1%</td><td>16.5%</td><td>8.2%</td></tr><tr><td>Being employed by a political body (PB)</td><td>5</td><td>17.7%</td><td>24.1%</td><td>33.7%</td><td>6.5%</td></tr><tr><td>Being sent by a PB to troll</td><td>5a</td><td>0.8%</td><td>1.3%</td><td>0.7%</td><td>0.3%</td></tr><tr><td>Being told by a PB how to troll</td><td>5b</td><td>1.3%</td><td>2.2%</td><td>1.4%</td><td>0.4%</td></tr><tr><td>Being paid by a PB to troll</td><td>5c</td><td>5.3%</td><td>6%</td><td>16.5%</td><td>1%</td></tr><tr><td>Being trained by a PB for trolling</td><td>5d</td><td>0.3%</td><td>0.2%</td><td>0.7%</td><td>0.2%</td></tr></tbody></table>

The results demonstrate that the most prevalent linguistically marked motive for trolling is an unspecified political affiliation, which is followed by being employed by a political body, and political beliefs. Meanwhile, emotional/mental health-related and social reasons as motives ascribed to trolls only occur in 5.9% of the troll motive comments.

The most striking difference in the distribution of the motives attributed to trolls between the three blogs is that unspecified political affiliation is much more prevalent whereas being employed by a political body is considerably less frequent on LL than on GP or GF. This is because there was a single commenter on LL who frequently used the expression *Tory troll* and consequently, his/her comments were provided with the unspecified political affiliation motive tag.

This shows that since only a small minority of the commenters call others trolls, the individual habits of those who do so can have a major impact on the general distribution of the motives on a blog. Thus, the quantitative differences between the blogs can be better explained by these context-dependent individual practices than by abstract variables, such as the political position of the blogs.

## 5. Conclusions

The main conclusions of this study are as follows:

(1) Although the relevant academic literature regards trolling as a chiefly emotionally motivated behaviour, in the context of online political discourse, participants attribute other motives to trolls as well, including financial gain, unspecified political affiliation, political beliefs, and being employed by a political body.

(2) In the examined corpus of comment threads from British political blogs, an unspecified political affiliation, being employed by a political body and political beliefs are more frequently mentioned motives for trolling than emotional reasons.

(3) A local conspiracy theory has been developed around trolling on the investigated blogs as some participants repeatedly suggest that various British political parties, other countries or the European Union secretly employ trolls. Thus, trolling is perceived as part of the online political warfare, a means that is believed to be used to manipulate public opinion.

(4) Whilst the concept of trolling can be constructed in different ways in the analysed troll motive comments, a
common trait of these comments is that the alleged trolls
are portrayed in a strongly negative manner. Thus, when
participants call others troll, they do not only attribute
motives to the trolls to explain their behaviour but also to
insult them.

6. References

Anthony, L. (2016). *AntConc* (Version 3.4.4) [Computer Software]. Tokyo, Japan: Waseda University. Available from http://www.laurenceanthony.net/

Baker, P. (2001). Moral Panic and Alternative Identity Construction in Usenet. *Journal of Computer-mediated Communication*, (7)1.

Baker, P. (2006). *Using Corpora in Discourse Analysis*. London: Continuum.

Binns, A. (2012). "Don't Feed the Trolls! Managing Troublemakers in Magazines' Online Communities." *Journalism Practice*, 6(4), pp. 547-562.

Culpeper, J. & Hardaker, C. (2016). Pragmatics. In P. Baker & J. Egbert (Eds.), *Triangulating Methodological Approaches in Corpus-linguistic Research*. New York/London: Routledge, pp. 124–137.

Donath, J.S. (1999). Identity and Deception in the Virtual Community. In P. Kollock & M.A. Smith (Eds.), *Communities in Cyberspace*. London/New York: Routledge, pp. 27–58.

Fichman, P. & Sanfilippo, M.R. (2015). The Bad Boys and Girls of Cyberspace: How Gender and Context Impact Perception of and Reaction to Trolling. *Social Science Computer Review*, 33(2), pp. 163–180.

Galán-García, P. et al. (2014). Supervised Machine Learning for the Detection of Troll Profiles in Twitter Social Network: Application to a Real Case of Cyberbullying. In Á. Herrero et al. (Eds.), *Proceedings of International Joint Conference SOCO’13-CISIS’13-ICEUTE’13*. New York: Springer, pp. 419–428.

Hardaker, C. (2010). Trolling in Asynchronous Computer-mediated Communication: From User Discussions to Academic Definitions. *Journal of Politeness Research. Language, Behaviour, Culture*, 6(2), pp. 215–242.

Hardaker, C. (2013). "Uh....not to be nitpicky,,,,,but...the past tense of drag is dragged, not drug.". An Overview of Trolling Strategies. *Journal of Language Aggression and Conflict*, 1(1), pp. 58-86.

Herring, S.C. et al. (2002). Searching for Safety Online: Managing „Trolling” in a Feminist Forum. *The Information Society*, 18(5), pp. 371–384.

Herring, S.C., Stein, D., & Virtanen, T. (Eds). (2013). *Pragmatics of Computer-Mediated Communication*. Berlin/New York: Mouton De Gruyter.

Hopkinson, C. (2013). Trolling in Online Discussions: From Provocation to Community-building. *Brno Studies in English*, 39(1), pp. 5–25.

Morrissey, L. (2010). Trolling Is an Art: Towards a Schematic Classification of Intention in Internet Trolling. *Griffith Working Papers in Pragmatics and Intercultural Communications*, 3(2), pp. 75–82.

Özsoy, D. (2015). Tweeting Political Fear: Trolls in Turkey. *Journal of History School*, 8(22), pp. 535–552.

Shachaf, P. & Hara, N. (2010). Beyond Vandalism: Wikipedia Trolls. *Journal of Information Science*, 36(3), pp. 357–370.

Tanskanen, S.-K. (2007). Metapragmatic Utterances in Computer-mediated Interactions. In A. Hübler & W. Bublitz (Eds.), *Metapragmatics in Use*. Amsterdam: John Benjamins, pp. 87–106.

Utz, S. (2005). Types of Deception and Underlying Motivation. What People Think. *Social Science Computer Review*, 23(1), pp. 49–56.
# Fear and Loathing on Twitter: Attitudes towards Language

Damjan Popič,* Darja Fišer*†

*Faculty of Arts, University of Ljubljana
Aškerčeva 2, SI-1000 Ljubljana, Slovenia

†"Jožef Stefan" Institute

Jamova cesta 39, SI-1000 Ljubljana, Slovenia

E-mail: damjan.popic@ff.uni-lj.si, darja.fiser@ff.uni-lj.si

## Abstract

The paper deals with the sociolinguistic concept of prestige imbued in the notion of standard language, and the social status connected to the inherent language skill (or lack thereof). To this end, we analyse Slovenian tweets pertaining to language use and the (in-)correctness of other users' use of language, propose a typology, especially in cases where language use is used as an argument against someone's qualifications or beliefs.

**Keywords:** orthography, (linguistic) prestige, computer-mediated communication

## 1. Introduction

The paper deals with a corpus-enhanced sociolinguistic analysis of attitudes towards language of Slovenian users on Twitter. In the Slovenian culture, the ability to produce linguistically correct texts has long been a test of a person's sophistication, education, and social class. This stance was further entrenched by the purist efforts, perpetuated by the country's constant struggle against imperialist forces, and the notion persists to this day.<fnref n="1" />

For this reason, we analyse the attitudes of Slovenian Twitter users towards language, and the types of discourse that language-related debates feature in. By using the keywords *comma*, *orthography*, *grammar*, *Slovene*, and *language* we isolate tweets in which users comment on language from the corpus JANES-Tweet v1.0 (Fišer et al. 2017). We analyse these examples from a sociolinguistic standpoint of prestige. This kind of attitude towards language is very common in linguistically conservative communities, and is often accompanied by a shared belief that the attitude towards language is a moral issue. This is visible in language use as (supposed) improper use of language is often used as an *ad hominem* argument aiming to suppress the relevance of that person's beliefs (on any matter).

Furthermore, recent developments have shed a new light on the attitudes towards language proficiency on social media all over the world, especially with Donald Trump. His tweets have been subject to much scrutiny, linguistic and otherwise.<fnref n="2" /> This shows that the language on Twitter, although supposedly informal in nature, is very much considered to be public.

## 2. Background

The Slovenian normative language tradition has been built around the German syntactic system and is highly prescriptive in nature. This means that the syntax in Slovene is highly structured, and this combination of a highly complex syntax and strict language rules imposed by the normative language guide (i.e. *pravopis*, from Ger. *Rechtsschreibung*) makes it notoriously difficult to write “correctly.” For instance, the dreaded comma is the bane of students' lives, with many recent studies showing that Slovenes in general are underperforming severely when it comes to commas (Popič et al. 2016).

## 3. Analysis

In this section, we present the results of the analysis of tweets containing at least one of the selected keywords: *comma*, *orthography*, *grammar*, *Slovene*, and *language*. The analysis involved a manual content analysis of the entire extracted sub-corpus and drafting a typology that allows us to cover the attitudes towards language imbued in the extracted tweets, for all the keywords. As the following tables indicate, the attitudes differ in number and tenor, however, scathing and disparaging tweets are present in each of the categories. Table 1 gives the frequencies of the selected keywords in the corpus.

<table><caption>Table 1: Keyword frequencies.</caption><thead><tr><td>Keyword</td><td>Frequency</td></tr></thead><tbody><tr><td>vejica (the comma)</td><td>1,978</td></tr><tr><td>slovnica (grammar)</td><td>930</td></tr><tr><td>pravopis (orthography)</td><td>432</td></tr><tr><td>jezik (language)</td><td>11,202</td></tr><tr><td>slovenščina (Slovene)</td><td>4,607</td></tr><tr><td>Total</td><td>19,149</td></tr></tbody></table>

As Table 1 demonstrates, by far the most frequent of the five keywords is *language*, with *Slovene* being second. However, the expression *language* is relatively general and also used in other senses (as in ‘tongue’ in the physical sense, in phrasemes, etc.). Considering the much narrower meaning of *vejica* (it almost exclusively

<footnote n="1">¹ This is perhaps best described by the latest development in which the Slovenian Parliament voted on amending the War Grave Act (with 54 ayes and 6 nays). Following the parliamentary conclusion, a (superfluous) comma will be removed from the first stanza of Oton Župančič's poem that will be inscribed on the memorial of national reconciliation.</footnote>

<footnote n="2">² See a comprehensive R-based statistical analysis of his tweets in relation to the device used for posting (Robinson 2016; http://varianceexplained.org/r/trump-tweets/).</footnote>
pertains to the comma as a means of punctuation), it is evident that the comma plays an immensely significant role with 1978 occurrences. For this reason, we devote the most attention to this keyword and deal with the rest of the keywords in pairs, focusing above all on the differences in the attitudes between the keywords.

## 3.1 Vejica ‘the comma’

The comma holds the most prestigious place in the Slovenian linguistic tradition, and performs several functions in discourse. Considering the examples extracted from our corpus, we can categorize the extracted examples in the categories given in Table 2 (including a free and standardised translation for each).

<table><caption>Table 2: Tweets relating to the comma.</caption><thead><tr><td>Attitude</td><td>Example</td></tr></thead><tbody><tr><td>inquisitive</td><td>@strankaDL Zakaj je v ministrovi izjavi vejica? ;-)</td><td>Šoltes: Aktualno politico<br/>Why is there a comma in the minister's statement?</td></tr><tr><td>informative</td><td>@drVinkoGorenak Spoštovani, manjka vam vejica za prvim ne. Prav je: "Ne, ne<br/>Dear Sir, you are missing a comma following the first 'ne'. It should read: "Ne, ne</td></tr><tr><td>lamenting</td><td>nekoc v eseju s "PS" dodal deset vejic in pripisal vstavi po potrebi<br/>I once put ten commas in the postscript in an essay, with a comment: "Insert where appropriate."</td></tr><tr><td>jocular</td><td>Če bi Krpan imel vejico v žepu, bi jo gotovo postavil<br/>If Krpan [a Slovenian folk hero] had a comma in his pocket, he'd certainly use it.</td></tr><tr><td>dismissive</td><td>@finance.si @NovaSlovenija Še vejice porihtajte. Da ne boste kot Pojbič<br/>Get your commas in order. So you aren't like Pojbič...</td></tr><tr><td>defensive</td><td>čaki mal.. A zarad vejic je pa človk nepismen?? ;-)</td><td>#svašta<br/>Wait a second... So [the misuse of] commas make[s] you illiterate? #whatever</td></tr><tr><td>apologetic</td><td>tudi jaz bi napisal z vejicami, če bi mi omejitev 140 znakov<br/>I'd use commas if I had more than 140 characters</td></tr><tr><td>idiomatic</td><td>Najboljši članek! Podpišem vse do vejice in pike! Desnica naj zakoplje<br/>The best article ever! I agree with every comma and full stop!</td></tr></tbody></table>

As Table 2 demonstrates, we have identified eight different attitudes towards the comma in our dataset. The inquisitive attitude pertains to actual questions regarding the use of the comma. The second, i.e. the informative attitude, is complimentary to the inquisitive one as it provides explanation(s) on the use of the comma and/or answers to questions on the use of the comma. The example we provided above includes a detailed (and quite possibly condescending) explanation directed at a Slovenian MP explaining why his use of the comma in his tweet was incorrect.

The lamenting attitude covers examples of exasperation over the perceived difficulty over the use of the comma. The example above laments about the school experience of a former pupil who asked his teacher to insert commas for him if required. The jocular attitude covers examples that apply irony on the Slovenian situation and the obsession with commas, whereas dismissive tweets use (alleged) misused commas to portray a person or institution as incompetent in general. For instance, in our example in Table 2, a person directs a comment at the Twitter account of the Slovenian conservative party NSi, saying that they should get their writing in order lest they should “go full Pojbič”, referring to a Slovenian MP famous for his poor grammar and spelling in his tweets. It is of note that the Slovenian politicians (especially conservative) and official institutions are under constant scrutiny when it comes to language (in social media), especially regarding the comma.

On the other hand, the apologetic and defensive attitudes aim to explain or justify one's “transgressions” with the comma, each in its own way. While the apologetic approach involves providing reasons for a particular example of comma misuse (either not knowing the rules or more pragmatic excuses like haste, typos, etc.), the defensive attitude conveys a stronger reaction, either against the user(s) exposing the misuse or against the relevance of the comma itself (normally in contrast to the meaning). The final category involves examples containing idiomatic references to the comma, i.e. using the comma as a metonymic expression for ‘language’ or ‘text’.

## 3.2 Slovenščina ‘Slovene’ and jezik ‘language’

As both keywords display very similar attitudes, we deal with them in a single section, with the obvious intention of displaying the attitudes relating to the “mother tongue”. A classification of tweets containing the two keywords is given in Table 3.

The tweets containing *Slovene* and *language* can be classified in five categories. The category General covers tweets providing general information and inquiries on both keywords. It is interesting to note that the example containing a purist tweet is again directed at the conservative MP Vinko Gorenak, who faced criticism, as recorded in Table 2, for his comma transgressions. This time he is the target of two purist comments, one lambasting him for improper grammar, and one for using a word that does not exist in Slovene on TV.
<table><caption>Table 3: Tweets relating to language and Slovene</caption><thead><tr><th>Attitude</th><th>Jezik 'language'</th><th>Slovenščina 'Slovene'</th></tr></thead><tbody><tr><td rowspan="2">General/inquisitive</td><td>A je slovenščina res najtežji jezik na svetu?</td><td>Kako prevedemo "small talk" v slovenščino ?</td></tr><tr><td>Is Slovene really the hardest language to learn in the world?</td><td>How do you say "small talk" in Slovene?</td></tr><tr><td rowspan="2">nationalist</td><td>s slovenščino, našim maternim jezikom, delamo kot svinja z mehom.</td><td>UPORABLJA ISKLJUČNO SLOVENSKE BESEDE. Slovenščina to smo SLOVENCI, ALI NIMAMO SVOJIH</td></tr><tr><td>We treat Slovene, our mother tongue, like dirt.</td><td>Use only Slovene words, we are Slovenes after all.</td></tr><tr><td rowspan="3">purist</td><td>@drVinkoGorenak Prosim za podnapise v slo jeziku: kaj pomeni "paložek"?<br/>@PlanetTV<br/>Subtitles please: what does "paložek" mean in Slovene?</td><td>@drVinkoGorenak "bodite točna" v slovenščini ne obstaja. Če že vikate, vikajte "Bodite točna" [i.e. the mixing of plural and singular forms] doesn't exist in Slovene. If you mean to address someone formally, stick to it.</td></tr><tr><td>casov "the day after yesterday". Jeziki ji ful laufajo</td><td>Ha odkar delam s kamionarji, mi slovenscina neki sepa.</td></tr><tr><td>"The day after yesterday." She's great with languages.</td><td>Ever since I started working with truckers, my Slovene has deteriorated.</td></tr><tr><td rowspan="2">idiotic</td><td>vidva takorekoč govorita isti jezik. In kaj bi bilo</td><td>vidim, da si bil zgolj v službi slovenščine. ČE ČM!</td></tr><tr><td>You speak the same language, as it were.</td><td>I see you were only in Slovene's service.</td></tr></tbody></table>

The tweets that employ a jocular perspective either expose the difficulties people have with Slovene or are meant as a ridicule of the Slovenian hard-line stance on language, whereas, in idiomatic tweets, the keyword *language* is very common, and *Slovene* much less so. The most significant difference in attitudes towards the comma and towards language/Slovene lies in the generality – the tweets belonging to the latter category are much more general (as well as less vicious and dismissive), but at the same time more nationalist.

### 3.3 Pravopis 'orthography' and slovnica 'grammar'

As in section 3.2, we combine the related keywords into a single analysis. This is especially relevant because most people confuse or substitute orthography and grammar, the latter most often being an umbrella term for all language-related matters, but in the Slovenian linguistic tradition, the two are distinct entities.

<table><caption>Table 4: Tweets relating to grammar and orthography</caption><thead><tr><th>Attitude</th><th>slovnica 'grammar'</th><th>pravopis 'orthography'</th></tr></thead><tbody><tr><td rowspan="2">purist</td><td>slovnica pa pravopis 1, vsebina 5</td><td>Vzgled? Pravopis pod vzglavnik, nekompetentnež</td></tr><tr><td>You get an F for grammar and orthography, and an A for content.</td><td>"Vzgled"? [an ungrammatical form of zgled] Put the orthography under your pillow, you incompetent...</td></tr><tr><td rowspan="2">apologetic</td><td>domače. Tle ni prostora za visoko slovnico. Če ne morem na #hodok, sem pa</td><td>tw je za vsebino, ne pravopis :) mene je bolj strah</td></tr><tr><td>This is no place for elevated grammar.</td><td>Twitter is meant for content, not orthography.</td></tr><tr><td rowspan="2">informative</td><td>slavisti. Ali pač? Po slovenski slovnici je desni prilastek pri kraticah</td><td>osebno se tudi zdi tako. Ampak pravopis je stvar konsenza. Greste na</td></tr><tr><td>The Slovenian grammar on modifiers with acronyms says that...</td><td>It may seem like that but orthography is all about consensus.</td></tr><tr><td rowspan="2">inquisitive</td><td>po ;-)</td><td>inovatorke. Ali cestarji obvladajo pravopis? via</td></tr><tr><td>Saj menda "greve" ni po slovnici?<br/>#samvprašam #slovnica<br/>Surely "greve" is ungrammatical?</td><td>Do road workers know orthography?</td></tr><tr><td rowspan="2">idiotic</td><td>Lukšič: Ko gre za slovnico političnega delovanja, je treba</td><td>le za demokracija, črn dan za pravopis! #krivopisje #domoljupi</td></tr><tr><td>When it comes to the grammar of political action...</td><td>Not just democracy, today is a sad day for orthography.</td></tr><tr><td rowspan="2">nationalist</td><td>slabo znanje slovnice se imenuje "čefurščina" ;)</td><td>slovenski narod, niti osnovnega pravopisa ne pozna. Kak veš, da si v</td></tr><tr><td>Poor grammar is indicative of čefurščina [a pejorative term for the dialect of Slovene spoken by immigrants from the countries of the former Yugoslavia]</td><td>Alas, the nation of Slovenia – they don't even know the basic orthography.</td></tr><tr><td rowspan="2">jocular</td><td>uvede policijska ura in odpravi slovnica. Just sat through Simeon ten</td><td>crknu?!? Kako se reče po noven slo pravopisu beseda KLON? PODOBNIK !</td></tr><tr><td>Let's implement curfew and abolish grammar altogether.</td><td>What is a clone called in Slovene? It's PODOBNIK [a surname of two Slovenian MPs, as well as a very literary word for a lookalike in Slovene].</td></tr></tbody></table>
The attitudes contained in the tweets relating to orthography and grammar are given in Table 4. The attitudes in tweets pertaining to orthography and grammar can be categorized in 7 different categories. As can be seen in our purist examples, users on Twitter employ a rather harsh tone when it comes to these keywords. The same seems to hold true for nationalist tweets, which, as in our examples, usually employ a hard-line stance or one of a exasperation over the use of language of people writing in Slovene.

With apologetic tweets, we can see an interesting trend that users themselves, upon being faced with accusations of having “poor grammar”, are trying to discredit Twitter as a means of “formal” communication and instead establish it as a means of fast, direct, and informal communication. The concepts of “orthography” and “grammar” are thus used as synonyms for “form”, whilst the significance of “meaning” is pointed out.

## 4. Conclusion

In this paper, we were interested in discovering whether or not the stereotypical attitudes towards written Slovene have been preserved and transformed in computer-mediated communication, in our case on Twitter. The analysis shows that, in spite of the belief that informality is an integral part of new media, all traditional notions pertaining to Slovene are still very much alive in the digital age. What is more, some forms of prejudice seem to be flourishing online due to the lack of oversight. For instance, personal attacks with dismissive tweets aiming to destroy one's credibility are very common, especially in regard to the comma, whereas more general topics attract more general attitudes, but just as vicious (purist and nationalist).

## 5. Acknowledgements

The work described in this paper was funded by the Slovenian Research Agency within the national basic research project "Resources, Tools and Methods for the Research of Nonstandard Internet Slovene" (J6-6842, 2014-2017).

## 6. References

Fišer, D., Erjavec, T., Ljubešić, N. 2017. The compilation, processing and analysis of the Janes corpus of Slovene user-generated content. Wigham, C.R. & Ledegen, G. (ur.). *Corpus de communication médiée par les réseaux: construction, structuration, analyse*. Collection Humanités Numériques. Paris: L'Harmattan (in print).

Popič, D., Fišer, D., Zupan, K., Logar, P. (2016). Raba vejice v uporabniških spletnih vsebinah. *Proceedings of the Language Technologies and Digital Humanities Conference*. Ljubljana, Slovenia: 149–153.

Robinson, D. (2016). Text analysis of Trump's tweets confirms he writes only the (angrier) Android half. (http://varianceexplained.org/r/trump-tweet)
# A Comparative Study of Computer-mediated and Spoken Conversations from Pakistani and U.S. English using Multidimensional Analysis

Muhammad Shakir and Dagmar Deuber

Department of English, University of Münster
Johannisstr. 12 - 20, 48143 Münster, Germany
muhammad.shakir@uni-muenster.de, deuber@uni-muenster.de

## Abstract

The present study compares four computer-mediated conversational registers (comments, Facebook (FB) groups, FB status updates and tweets), and spoken conversations from Pakistani and U.S. English using Biber's Multidimensional Analysis framework on three dimensions of variation, i.e. (i) Interactive versus Descriptive Explanatory Discourse, (ii) Expression of Stance, and (iii) Informational Focus versus 1st Person Narrative. Spoken conversations have a high score on dimension 2, while CM conversations show register and regional variation on dimension 1 and 3. FB groups are significantly different between both regional varieties, followed by FB status updates, comments and tweets. Pakistani FB groups discuss self-help related topics, and appear to be slightly interactive and highly informational, while the U.S. ones are interactive and narrative discussing community and political issues. Pakistani FB status updates and tweets use English mainly for informational purposes, while the U.S. counterparts have an interactive and personal orientation indicating a wider functional role of English.

**Keywords:** Register Variation, Multidimensional Analysis, World Englishes, Conversations

## 1. Introduction

Language users converse with each other to exchange news, views and ideas in an informal way (Oxford Online Dictionary, 2017). Traditionally conversations have been spoken only. With the advent of the internet, another medium has been added, i.e. the written medium. Spoken and newly emerging computer-mediated (CM) conversations are different in ways like turn-taking (Herring, 2011) or synchronicity (Bieswanger, 2016), but at the same time they are linguistically similar to each other (Jonsson, 2015). Though extensively studied, CM conversations need to be studied using a comparative and multi-dimensional approach (Herring, 2011) like Biber's (1988) Multidimensional Analysis (MDA) model, which combines an analysis of situational context with lexicogrammatical features, and interprets them functionally (Biber & Conrad, 2009). Present research paper aims to study emerging CM registers – i.e. comments, Facebook (FB) groups, FB status updates and tweets – in relation to spoken conversations. MDA studies on CM registers have, until now, largely focused either on U.S. English (Grieve et al., 2010) or native varieties of English (Biber and Egbert, 2016). Pakistani English is an outer circle variety in Kachru's (1992) three circle model, which is an important tool in the linguistic repertoire of Pakistani internet users, but not widely studied in relation to the internet. On the other hand, U.S. English, is an inner circle and globally dominant variety, which may be influencing varieties like Pakistani English due to contact and technological influence on the internet. Previous research (e.g. Hardy and Frigital, 2012) suggests that there might be differences between inner and outer circle varieties of English regarding the use of CM registers. Hence further aim is to combine the study of register and regional variation.

### 1.1 Previous Research

Spoken conversations are generally involved and interactive (Biber, 1988). However, later studies also show

additional dimensions like narrativity, informational focus (Biber, 2004), and expression of stance (Biber, 2006). Various types of CM conversations have been studied using MDA. Collot and Belmore (1996) applied Biber's (1988) MDA to study bulletin boards – an ancestor of today's FB groups – and found them nearer to public interviews in spoken conversations. FB status updates and tweets have been said to be CM equivalents of spoken conversations but quite different (Sardinha, 2014), and to be highly informational and descriptive instead of being involved and interactive (Titak & Roberson, 2013). Similarly, comments have been found to be involved, personal and past oriented (ibid). Lastly, studies using MDA to find out regional variation, e.g. Xiao (2009) and Coats (2016), do not involve the comparison of CM and spoken conversations.

## 2. Material and Methods

<table><caption>Table 1: Description of the corpus</caption><thead><tr><th rowspan="2">Categories</th><th colspan="2">Pakistani English</th><th colspan="2">U.S. English</th></tr><tr><th>Words</th><th>Texts</th><th>Words</th><th>Texts</th></tr></thead><tbody><tr><td>Comments</td><td>334,447</td><td>794</td><td>342,517</td><td>747</td></tr><tr><td>FB groups</td><td>163,940</td><td>502</td><td>163,158</td><td>426</td></tr><tr><td>FB S. U.</td><td>67,737</td><td>104</td><td>68,819</td><td>108</td></tr><tr><td>Tweets</td><td>58,771</td><td>115</td><td>62,086</td><td>103</td></tr><tr><td>Conv.</td><td>158,521</td><td>85</td><td>487,476</td><td>111</td></tr></tbody></table>

Table 1 describes the data for both varieties. Four registers were selected for CM conversations as they were publicly accessible on the internet. Comments were collected from various blogs (single- and multi-writer blogs, newspaper blog posts, and technology blogs) using the website下载器 software DarcyRipper and a custom software written in C#. The data for FB groups was manually copied and cleaned after identifying groups originating from Pakistan (mostly closed groups) and the U.S.A. (mostly open groups). Status updates were also manually collected by identifying user profiles from member lists of already scraped groups. Twitter profiles were identified from real-
time tweets originating from Pakistan and the U.S.A. The tweets were downloaded using a custom software. The CM conversations data was reviewed and edited for spam, automatic messages, Roman Urdu code switching, and non-standard spellings to facilitate the tagger. However, the spoken data was not reviewed. The data for Pakistani English was extracted from an under-development corpus of Pakistani English (ICE-PK), and for the U.S. variety from the Corpus of Contemporary American English (COCA). The entire data was collected for the time period of 2009-15.

The data was then tagged using Biber Tagger (Biber, 1988; 2006). The tagger tags only approximately 140 specific lexico-grammatical features. Hardy and Frigital (2012) have reported up to 93% accuracy of Biber Tagger on blog posts. Though desirable, such a manual verification of tagging accuracy was beyond the scope of the present research. A new MDA was conducted following guidelines provided in Biber and Gray (2013) and Egbert and Staples (forthcoming). The statistical software package R was used to perform Exploratory Factor Analysis. 61 lexico-grammatical features were selected by studying previous research on conversational registers (Biber, 2004; Titak & Roberson, 2013; Biber et al., 1999). After conducting multiple factor analyses with factor solutions from 2-7, a 3-factor solution with Principal Axis Factoring as factor extraction method and Promax as rotation method was deemed fit to describe the data. The details and descriptive statistics are provided in table 2.

<table><thead><tr><th>Factor</th><th>+/-</th><th>Linguistic Features with Loadings</th></tr></thead><tbody><tr><td rowspan="2">1</td><td>+</td><td>present tense 0.70, 2<sup>nd</sup> person pronouns 0.49, contractions 0.38, activity verbs 0.34, models of prediction 0.29, models of possibility 0.27 (1<sup>st</sup> person pronouns 0.34)</td></tr><tr><td>-</td><td>prepositions -0.40, attributive adjectives -0.38, nominalisations -.35 (word length -0.29)</td></tr><tr><td rowspan="2">2</td><td>+</td><td>that deletion 0.57, mental verbs 0.49, that clauses controlled by verbs 0.48, that clauses controlled by communication verbs 0.45, communication verbs 0.43, that clauses controlled by factive verbs 0.40, that clauses controlled by likelihood verbs 0.40 (communication verbs in other contexts 0.30)</td></tr><tr><td>-</td><td>(common nouns -0.38)</td></tr><tr><td>3</td><td>+</td><td>word length 0.53, common nouns 0.51, communication verbs in other contexts 0.40, process nouns 0.34, abstract nouns 0.27 (communication verbs 0.38)</td></tr></tbody></table>

<table><tr><td>-</td><td>1<sup>st</sup> person pronouns -0.38, adverbs of place -0.33, general adverbs -0.32, past tense -0.29 (contractions -0.37), (nominalisations -0.25)</td></tr></table>

Other Descriptive Statistics

<table><caption>Table 2: Results of Factor Analysis and other descriptive statistics</caption><tr><td>Total Variance Explained</td><td>22%</td></tr><tr><td>Variables in final FA</td><td>25</td></tr><tr><td>Cut-off</td><td>+/- 0.25</td></tr><tr><td>Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy</td><td>0.54 (Classification: Miserable)</td></tr></table>

The total variance explained and KMO values of the factor analysis are quite low. As Egbert and Staples (forthcoming) analysed in their study, total variance explained values have been generally low for MDA studies. This is especially the case with internet-based registers. Similarly, they also reported KMO value less than .60 for one of their previous studies. A possible reason might be that the lexico-grammatical variables depend on other variables not included in present analysis. Heterogeneity of the data could be another possible reason.

Each factor in the solution has feature groups with positive and negative loadings, which are mutually less likely to co-occur (Biber, 1988). High factor loading indicates the feature is salient, and vice versa. The features within brackets overlapped with the ones in other factors, hence they were used in the interpretation of factors to dimensions, but not for dimension score calculation. The dimension scores were calculated for each text by summing z-scores of positive as well as negative features, and finally by subtracting negative total score from positive total score. The mean scores for each register category were also calculated to compare the registers on each dimension. Parametric (One-way) or non-parametric (Kruskal-Wallis) ANOVA and respective post-hoc tests were used to check if corresponding registers had significant differences between the regional varieties.

## 3. Results and Discussion

### 3.1 Dimension 1: 'Interactive versus Descriptive Explanatory Discourse'

Dimension 1 has eleven features with 7 on positive and 4 on negative side. The features on positive side belong to an interactive discourse (Grieve et al., 2010). The less important features, such as possibility and prediction modals combine with human subjects and dynamic verbs to denote to intrinsic meaning (Biber et al., 1999). The texts with a high positive score are from FB groups. They discuss about present and future events, and are highly interactive.
<figure><img src="image_26.png" /><figcaption>Figure 1: Conversational registers on Dimension 1: ‘Interactive versus Descriptive Explanatory Discourse’ (One-way ANOVA: F(9, 3085) = 70.23, p < .001; Post-hoc Tukey HSD significant groups between varieties: Comments, FB groups, FB status updates)</figcaption></figure>

The features on negative side are prepositions, attributive adjectives, and nominalisations. Attributive adjectives indicate the presence of descriptive discourses. Prepositional phrases are “the most common type of post-modifiers” (Biber et al, 1999, p. 631). A look at high scoring texts from comments and FB groups show that the texts are descriptive and explanatory in general. Thus, combining positive and negative features the dimension 1 can be interpreted as ‘Interactive versus Descriptive Explanatory Discourse’.

Figure 1 shows comparison of register categories on dimension 1. Comments, FB groups and FB status updates in Pakistani English are significantly different from their counterparts in U.S. English, while tweets and conversations do not have significant differences. FB groups is by far the most interactive register, while the category of conversations has the highest inclination towards descriptive and explanatory side of the dimension.

## 3.2 Dimension 2: ‘Expression of Stance’

Dimension 2 has eight linguistic features on positive side, and only one feature on negative side. The positive features include communication verbs like *ask*, *shout*, *tell* etc., which show the activity of communication. Mental verbs like *think*, *know*, *love*, *want* etc. are used for cognitive meaning as well as to express attitudes of the speakers (Biber et al., 1999). That complement clauses controlled by communication and likelihood verbs are used to convey stance (Biber, 2006), or the presence of reported speech or activities (Titak and Roberson, 2013). An examination of texts with high positive scores from comments, conversations and FB groups show that they contain the elements of opinion or stance. Considering only positive features of this dimension, it can be interpreted as ‘Expression of Stance’.

<figure><img src="image_27.png" /><figcaption>Figure 2: Conversational registers on Dimension 2: ‘Expression of Stance’ (Kruskal-Wallis ANOVA: H = 206.0, p = 0; Post-hoc Conover-Iman Test significant groups between varieties: FB groups, FB status updates)</figcaption></figure>

Looking at the results in figure 2, expression of stance seems largely related to spoken conversations, which have the highest scores among all registers. Pakistani conversations have a higher score and a wider range as compared to the U.S. data, which is probably due to a wider variety of conversations (face-to-face, talk shows, interviews etc.). Comments do not have high mean scores, which indicates the possible presence of other stance marking devices like stance adverbs, nouns and adjectives as observed by Biber (2006, p. 92). FB groups, status updates and tweets are less stance oriented, though both FB related registers show significant differences between Pakistani and U.S. English.

## 3.3 Dimension 3: ‘Informational Focus versus 1st Person Narrative’

Dimension 3 contains twelve linguistic features with six features on either side. The positive features include various kinds of nouns and word length, which generally have a positive correlation with each other, i.e. a higher frequency of nouns indicates lengthier words. The majority of texts with high positive score on this dimension are from Pakistani FB groups, which either contain job ads followed by infrequent formulaic comments, or discussions related to study that include abstract and process nouns.

Among features on negative side, 1st person pronoun and contractions normally occur in informal texts with a personal focus. Past tense verbs have been found relevant to narrative texts (e.g. Biber, 1988). The texts with high negative score are from U.S. FB groups, which generally talk about events with the mention of places in 1st person using past tense. Combining both interpretations, dimension 3 can be labelled as ‘Informational Focus versus 1st Person Narrative’.
<figure><img src="image_28.png" /><figcaption>Figure 3: Conversational registers on Dimension 3: ‘Informational Focus versus 1st Person Narrative’ (Kruskal-Wallis ANOVA: H = 864.43, p = 0; Post-hoc Conover-Iman Test significant groups between varieties: Comments, FB groups, FB status updates, Tweets)</figcaption></figure>

Figure 3 elaborates mean dimension scores of all registers on dimension 3. Though all CM conversational registers show significant differences between Pakistani and U.S. English, spoken conversations do not show much variation except a slight orientation towards the narrative side of the dimension. The most obvious differences are between FB groups, status updates and tweets, where all Pakistani registers have information focused orientation, while U.S. registers incline towards 1st person narration.

## 4. Conclusion

The differences between spoken and CM conversations mainly appear to be on dimension 1 and dimension 2. On dimension 1, spoken conversations incline towards descriptive and explanatory discourse. The reason for U.S. English seems to be the selection of spoken register for this category, i.e. broadcast discussions, which are different from spontaneous face-to-face conversations. For Pakistani English, though spoken conversations come from more than one registers, for example interviews, talk shows, and student face-to-face conversations, it appears that even face-to-face conversations are generally descriptive and explanatory instead of being involved and interactive. Dimension 2 ‘Expression of Stance’ has been observed previously as well (Biber, 2004; 2006) for spoken conversations. The results apparently confirm that CM conversations are similar but quite different from spoken conversations (Titak & Roberson, 2013). Another possible reason could be the lesser representation of spoken registers in U.S. English and a smaller number of words in Pakistani English.

On the other hand, CM conversations show variation on all dimensions between registers as well as between regional varieties. Pakistani FB groups are generally related to study help, job, pet and game related talk, which makes them interactive as well as information oriented. However, U.S. FB groups are related to politics, community related issues, as well as pet and game related talk, so they are a little less

interactive but highly inclined towards personal narration. Pakistani comments are slightly more interactive due to comments from “diary type blogs” (Grieve et al., 2010), while U.S. comments are descriptive in contrast due to an abundance of political “commentary type blogs” (ibid). FB status updates and tweets are highly informational in Pakistani English partially due to the use of local languages to talk about personal issues, while that is not the case with U.S. English. To conclude, Pakistani CM conversations differ from U.S. counterparts, though a more representative data of spoken conversations would help to better understand the relation between both types of conversations.

## 5. References

Biber, D. (1988). *Variation across speech and writing*. Cambridge University Press.

Biber, D. (2004). Conversation text types: A multi-dimensional analysis. In *Le poids des mots: Proc. of the 7th International Conference on the Statistical Analysis of Textual Data*. Louvain: Presses universitaires de Louvain, pp. 15–34.

Biber, D. (2006). *University language: A corpus-based study of spoken and written registers, Vol. 23*. John Benjamins Publishing.

Biber, D. & Conrad, S. (2009). *Register, genre, and style*. Cambridge University Press.

Biber, D. & Gray, B. (2013). Identifying multi-dimensional patterns of variation across registers. In M. Krug & J. Schlüter (Eds.), *Research Methods in Language Variation and Change*. Cambridge University Press, pp. 402–420.

Biber, D., Johansson, S., Leech, G., Conrad, S. & Finegan, E. (1999). *Longman Grammar of Spoken and Written English*. Longman Publications Group. (ISBN: 0-582-23725-4.)

Bieswanger, M. (2016). Electronically-mediated Englishes: Synchronicity revisited. In L. Squires (Ed.), *English in Computer-Mediated Communication: Variation, Representation, and Change Vol. 93*. Walter de Gruyter GmbH & Co KG, pp. 281–300.

Coats, S. (2016). Grammatical feature frequencies of English on Twitter in Finland. In L. Squires (Ed.), *English in Computer-Mediated Communication: Variation, Representation, and Change Vol. 93*. Walter de Gruyter GmbH & Co KG, pp 179–209.

Collot, M. & Belmore, N. (1996). Electronic Language: A New Variety of English. In S. C. Herring (Ed.), *Computer-mediated communication: Linguistic, social, and cross-cultural perspectives Vol. 39*. John Benjamins Publishing, pp. 13–28.

Conversation. (n.d.). In *Oxford Living Dictionaries*. Retrieved May 15, 2017, from https://en.oxforddictionaries.com/definition/conversation.

Grieve, J., Biber, D., Frigital, E. & Nekrasova, T. (2010). Variation among blogs: A multi-dimensional analysis. In Alexander M., & S. Sharoff (Eds.), *Genres on the Web Computational Models and Empirical Studies*. Springer,
pp. 303—322.

Herring, S. C. (2011). Computer-mediated conversation
Part II: Introduction and overview. *Language@ internet
8(2), pp. 1—12. Retrieved from
http://www.languagetinternet.org/articles/2011/Herrin
g.

Egbert, J. & Staples, S. (forthcoming). Doing multidimensional analysis in SPSS, SAS and R. In T. B. Sardinha & M. V. Pinto (Eds.), *Multidimensional Analysis*. London: Bloomsbury.

Jonsson, E. (2015). *Conversational Writing-A Multidimensional Study of Synchronous and Supersynchronous Computer-Mediated Communication*. PETER LANG LTD International Academic Publishers.

Kachru, B. B. (1992). *The other tongue: English across cultures*. University of Illinois Press.

Sardinha, T. B. (2014). 25 years later Comparing Internet and pre-Internet registers. In T. B. Sardinha & M. V. Pinto (Eds.), *Multi-Dimensional Analysis, 25 years on A tribute to Douglas Biber Vol. 60*. John Benjamins Publishing Company, pp. 81—105.

Titak, A. & Roberson, A. (2013). Dimensions of web registers: an exploratory multi-dimensional comparison. *Corpora* 8(2), pp. 235–260.

Xiao, R. (2009). Multidimensional analysis and the study of world Englishes. *World Englishes* 28(4), pp. 421—450.
# Posters

## Contents

MoCoDa 2: Creating a Database and Web Frontend for the Repeated Collection of Mobile Communication (WhatsApp, SMS & Co) .................................................. 71

Michael Beißwenger, Marcel Fladrich, Wolfgang Imo and Evelyn Ziegler

Public Service News on Facebook: Exploring Journalistic Usage Patterns and Reaction Data .................................................................................................................. 72

Daniel Pfurtscheller

The graphic realization of /1/-vocalization in Swiss German WhatsApp messages................ 73

Simone Ueberwasser
# MoCoDa 2: Creating a Database and Web Frontend for the Repeated Collection of Mobile Communication (WhatsApp, SMS & Co.)

Michael Beißwenger<fnref n="1" />, Marcel Fladrich<fnref n="2" />, Wolfgang Imo<fnref n="3" />, Evelyn Ziegler<fnref n="1" />

¹ University of Duisburg-Essen <fnref n="2" /> University of Münster <fnref n="3" /> University Halle-Saale
michael.beisswenger@uni-due.de, m.fladrich@uni-muenster.de, wolfgang.imo@germanistik.uni-halle.de, evelyn.ziegler@uni-due.de

## Abstract

The poster reports about intermediate results of MoCoDa 2, an ongoing project funded by the Ministry for Innovation, Science, Research and Technology of the German federal state North Rhine-Westphalia in which we are developing a database and web frontend for the repeated, donation-based collection of CMC interactions from smartphone messaging apps like WhatsApp. The database shall serve as a resource not only for quantitative but also for qualitative approaches in the analysis of CMC. MoCoDa 2 builds on experiences from the preceding project MoCoDa<fnref n="1" /> which has collected a (relatively small) set of 2,198 interactions with 19,161 user posts or ~193,000 tokens since 2012. For MoCoDa 2 the database and web frontend will be re-implemented from the scratch and expanded with additional functions and features:

* A form for donating and editing the data, which involves the donators into the editing and anonymization process and assists them with capturing metadata on the context and topic of the donated sequences as well as on the interlocutors and their social relations. Anonymization will follow an anonymization guideline developed in the CLARIN-D curation project *ChatCorpus2CLARIN*.<fnref n="2" />

* Part-of-speech annotations which comply with the extended 'STTS 2.0' tagset for German CMC<fnref n="3" /> and which will be created using a toolchain provided by the Language Technology Lab (LTL) at the University of Duisburg-Essen<fnref n="4" />.

* A TEI export for the collected data on basis of the 'CLARIN-D TEI schema for CMC'<fnref n="5" />. Through adopting the STTS 2.0 tagset and a TEI-based export format the corpus data will be interoperable with corpora that are already part of the CLARIN-D corpus infrastructure at the Institute for the German Language (IDS) in Mannheim. To allow for comparative analyses of the MoCoDa 2 data with the discourse found in text corpora and in other CMC corpora, MoCoDa 2 will not only be made available as a standalone resource but also be integrated into the German Reference Corpus (DEREKO) at the IDS Mannheim.

**Keywords:** corpora, collection strategies, whatsapp, annotation

<footnote n="1">¹ The MoCoDa project (http://mocoda.spracheinteraktion.de/) is headed by Wolfgang Imo.</footnote>

<footnote n="2">² Lüngen, Harald; Beißwenger, Michael; Herzberg, Laura; Pichler, Cathrin (2017): Anonymisation of the Dortmund Chat Corpus 2.1. In: Proceedings of the 5th Conference on CMC and Social Media corpora for the Humanities. Bolzano, Oct 02-03, 2017.</footnote>

<footnote n="3">³ The tagset and annotation guidelines can be found at https://sites.google.com/site/empirist2015/home/annotation-guidelines</footnote>

<footnote n="4">⁴ http://www.ltl.uni-due.de/ (Cooperation partner at the LTL: Torsten Zesch)</footnote>

<footnote n="5">⁵ http://wiki.tei-c.org/index.php?title=SIG:CMC/clarindschema</footnote>
Public Service News on Facebook:
Exploring Journalistic Usage Patterns and Reaction Data

Daniel Pfurtscheller

University of Innsbruck

E-Mail: daniel.pfurtscheller@uibk.ac.at

Abstract

As social networking sites have become staples in everyday life an increasing number of people worldwide use social media as a source of news. To reach this audiences, news organizations and public service broadcasters have ventured on services such as Facebook, which in terms of news is by far the most important social networking site in many parts of Europe.

This poster presents an ongoing research project that explores the ways in which public service media from different European countries are delivering news on public Facebook Pages. In a first step, the Facebook Page of the Austrian news magazine "Zeit im Bild" is examined in a pilot study. The poster presents the project as work in progress and gives an overview of the planned corpus building process. The analysis is based on public data gathered from the public Facebook page operated by Austrian national broadcasting agency ORF (http://facebook.com/zeitimbild). The data are extracted using the public Facebook Graph API. The corpus contains all the posts and comments of the Facebook Pages as well as related metadata. No personally-identifiable information is collected.

The social media data are explored using the R software environment to identify and compare journalistic usage patterns and to visualize the interaction of Facebook users. This should provide an overview over the different forms of journalistic news content (i.e. types of posts) and the basic communicative practices that can be observed in the context of the Facebook Pages (i.e. number of comments, shares, likes and other "Reaction" types). To allow deeper insights an exploratory case-study approach is used. Drawing upon media linguistic research the focus is on the micro level of the media texts and their multimodal design. The in-depth analysis aims to characterize different forms of news reporting via Facebook and looks at the different usage of multimodal resources in the context of the Facebook posts and comments. This combination of qualitative and quantitative methods should allow a better understanding of how Facebook is used as a means of news distribution by public service media providers on a large scale and how technical affordances shape the design of news content and follow-up interactions. This knowledge is critical for the discussion of the emerging role of social media in the context of public opinion and political decision-making.

**Keywords:** Facebook, social media interaction, public service news, metadata, media linguistics
# The graphic realization of /l/-vocalization in Swiss German WhatsApp messages

## Simone Ueberwasser

University of Zurich

simone.ueberwasser@ds.uzh.ch

## Abstract

This Poster represents a corpus based study into /l/-vocalization in the German speaking part of Switzerland. Its main scientific objective is to show that isolated occurrences of /l/-vocalization can be found in speakers who do not originate in the geographical area for which this phenomena has traditionally been described. This result is relevant for the discussion of /l/-vocalization because the study does not look at the realization of the sound in specific words as they are produced in an artificial setting but rather at real data as it is written by informants in everyday WhatsApp communications and as such shows an extended use of the phenomena by a group of speakers hitherto ignored.

/l/-vocalization, i.e. the replacement of a lateral approximant [l] by the vowel [u] is a well described feature of some Swiss German dialects (GSW). Phonological (e.g. Haas (1983), who mentions an implicational scale of phonological factors that favor /l/-vocalization) and sociolinguistic (e.g. Christen (2001)) restrictions and promoters are well described. The *Linguistic Atlas of German-Speaking Switzerland* (Sprachatlas der deutschen Schweiz, 1962 to 2003), is the most renowned documentation of the GSW dialects representing regional variation in linguistic features. It describes /l/-vocalization as a feature roughly to be found between Berne and Lucerne. A more recent study (Leemann et al., 2014) found /l/-vocalization to progress to “... southeasterly, southerly, and westerly directions, but with much less success to the north and northwest, where the equally influential dialectal areas of Basel and Zürich seem to exert opposing influences” (Leemann et al., 2014, 191).

A large scale multilingual corpus (617 chats, 5,5 Mio tokens) of authentic WhatsApp messages was compiled in 2014 at the University of Zurich (cf. Ueberwasser and Stark (in print) and www.whatsup-switzerland.ch). The 45 chats from this corpus that are in GSW and for which informants provided information about their home town today and when they were in 5th grade (± 12 years old) were used for the study.

Focusing on the places where informants lived when in 5th grade, /l/-vocalization can, of course, be found in the expected area but also around Zurich and Basel in suburban places. However, in these areas, /l/-vocalization is not applied as consequently as in the core area but very sporadically. The tokens that are realized with /l/-vocalization follow different phonological patterns. Even though many of them come from the highest class in Haas' implicational scale, not all of them do. On the other hand, the lexemes to which /l/-vocalization is applied outside the expected area are mostly very frequent and thus salient in the GSW subcorpus and often interjections. These and more results will be presented in the poster.

**Keywords:** Swiss German dialects, /l/-vocalization, dialect use in WhatsApp messages

## References

Christen, H. (2001). Ein Dialektmarker auf Erfolgskurs: Die /l/-Vokalisierung in der deutschsprachigen Schweiz. *Zeitschrift für Dialektologie und Linguistik*, (1):16–26.

Haas, W. (1983). Vokalisierung in deutschen dialekten. In Werner Besch, et al., editors, *Dialektologie: Ein Handbuch zur deutschen und allgemeinen Dialektforschung. Vol II*, pages 1111–1116. de Gruyter, Berlin.

Leemann, A., Kolly, M.-J., Werlen, I., Britain, D., and Studer-Joho, D. (2014). The diffusion of /l/-vocalization in Swiss German. *Language Variation and Change*, 26(02):191–218.

Sprachatlas der deutschen Schweiz. (1962 to 2003). Francke, Bern/Basel.

Ueberwasser, S. and Stark, E. (in print). What's up, Switzerland? A corpus-based research project in a multilingual country. *Linguistik Online*.
# Appendix

## Contents

Author Index .................................................................................. 75

Keyword Index .................................................................................. 76
Author Index

A. Seza Doğruöz, 3
Abdulhafiz Alkhouli, 46
Aivars Glaznieks, 2
Angelika Storrer, 16, 52

Boris Borzic, 46

Céline Poudat, 52
Carole Etienne, 52
Cathrin Pichler, 21
Christophe Parisse, 52
Ciara Wigham, iii, 52
Claudia Marinica, 46

Dagmar Deuber, 65
Damjan Popič, 61
Daniel Pfurtscheller, 72
Darja Fišer, 4, 30, 52, 61

Egon Stemle, iii, 52
Erhard Hinrichs, 52
Evelyn Ziegler, 71

Harald Lüngen, 21, 52
Holger Grumt Suárez, 52

Julien Longhi, 46, 52

Laura Herzberg, 16, 21, 52

Lieke Verheijen, 6
Lisa Hilte, 11
Lothar Lemnitzer, 52
Lydia-Mai Ho-Dac, 52

Márton Petykó, 56
Marcel Fladrich, 71
Michael Beißwenger, 21, 39, 52, 71
Muhammad Shakir, 65

Nader Hassine, 46
Natali Karlova-Bourbonus, 52

Paola Leone, 44

Reinhild Vandekerckhove, 11

Simone Ueberwasser, 73
Stefania Spina, 25
Steven Coats, 35

Taja Kuzman, 30
Thomas Schmidt, 52
Tobias Horsmann, 39, 52
Torsten Zesch, 39, 52

Walter Daelemans, 11
Wilbert Spooren, 6
Wolfgang Imo, 71
Keyword Index

(linguistic) prestige, 61
/1/-vocalization, 73

annotation, 52, 71

anonymisation, 21, 52

bilingualism, 35

blog, 56

collection strategies, 71

conversations, 65

corpora, 21, 44, 52, 71

corpus linguistics, 35

cross-lingual CMC study, 16

data collection, 44

demonyms, 30

dialect writing, 2

discourse analysis, 30

emoticons, 25

Facebook, 2, 72

interaction signs, 16

language learning, 44

language modeling, 11

literacy, 6

machine learning, 3

media linguistics, 72

metadata, 44, 72

mixed-effects models, 25

motive attribution, 56

multidimensional analysis, 65

multilingualism, 3

nationalities, 30

NLP applications, 46

NLP for social media, 46

non-standardness, 11

orthography, 2, 61

part-of-speech, 39

protocol for data collection, 44

public service news, 72

quantitative methods, 35

rare phenomena, 39

register variation, 65

research infrastructures, 52

school writing, 6

Slovene, 30

small vs. large data sets, 3

social media, 6, 35, 39

social media data, 3

social media interaction, 72

Swiss German dialects, 73

teenage talk, 11

telecollaboration, 44

textometry, 46

troll(ing), 56

tweets mining, 46

Twitter, 25, 30, 35

WhatsApp, 6, 71

WhatsApp: dialect use in, 73

Wikipedia talk pages, 16

World Englishes, 65