<figure><img src="image_1.png" /></figure>

HAL
open science

# Handbook Twitter for Research, 2015 / 2016

Clement Levallois, Morgane Marchand, Tiago Mata, Andre Panisson, Sibele Fausto, Pascal Aventurier, Eglantine Schmitt, Michael Gauthier, Adrien Guille, Fabien Rico, et al.

<figure><img src="image_2.png" /></figure>

## To cite this version:

Clement Levallois, Morgane Marchand, Tiago Mata, Andre Panisson, Sibele Fausto, et al. (Dir.). Handbook Twitter for Research, 2015 / 2016. EMLYON Press, 2016, 978-1523263394. 10.5281/zen-odo.44882. hal-01257444

HAL Id: hal-01257444

https://hal.science/hal-01257444

Submitted on 17 Jan 2016

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Twitter Mix Days 2015

Edited by Clément Levallois, Morgane Marchand, Tiago Mata, André Panisson

Conference hosted in Lyon, France
by EMLYON Business School,

22-24 April 2015
<figure><img src="image_4.png" /></figure>
Foreword

Close to 2,500 research papers on Twitter have been published since 2006, the year the social networking service was founded (see Fausto & Aventurier, chapter 1 in this volume). Scientists have developed an interest in Twitter as an object of study itself, and also as a rich and convenient data source to conduct research on topics independent of Twitter – from the prediction of election results to new challenges in sentiment analysis. The volume and diversity of these scientific contributions would suggest that fruitful exchanges would take place between researchers sharing an interest in Twitter.

Yet, science being structured in separate fields, a consequence is that one researcher using Twitter data in her scientific domain might never cross the path of another scientist using the same type of data but in a different scientific domain. Specialists of business-to-business marketing rarely interact with students in literary studies, while sociologists of urban life seldom engage with the forefront of text mining research – even if they all use Twitter as an input to explore their respective research question. It is reasonable to think that scholars situated in vastly different research traditions would benefit from meeting and exchanging views on how they use Twitter, for which purpose and for what results.

**The conference « Twitter for Research » was organized on April 24, 2015 in Lyon, France with the ambition to fulfill this promise:** gathering scientists using Twitter in their research, from all disciplinary backgrounds, for a day of exchange, communication and networking. The present volume gathers 13 contributions which were presented this day (see the full program of the conference at http://tinyurl.com/twitter4research2015).

This Handbook is the first edition (2015/2016) of an annual series devoted to cover research contributions harnessing Twitter as a key input. To be considered for the next edition, we encourage you to submit a proposal for the next conference "Twitter for Research", due to be organized in Galway, Dublin on April 18-20, 2016.

Clement Levallois

@seinecle

levallois@em-lyon.com
# Organization

## Executive Committee

Conference Chair
Clément Levallois, Assistant Professor, EMLYON Business School
Morgane Marchand, Research Engineer at eXenSa
Tiago Mata, Lecturer in Science and Technology Studies, University College London
André Panisson, Researcher at the Data Science Laboratory of the Institute for Scientific Interchange in Turin, Italy

## Sponsoring Institutions

EMLYON Business School, Lyon-Ecully, France.

## Acknowledgements

We would like to thank all the participants for contributing to the success of the conference with a varied program and lively exchanges in panels and in informal settings. We extend our special gratitude to Dr. Jeff Kolb, Data Scientist at Twitter, for delivering a keynote in the spirit of the conference: interdisciplinary in scope, opening broad horizons.

Capgemini Consulting and all our sponsors also receive our sincere thanks for contributing their expertise to the conference and the 2 days preceding it (the “Twitter Mix Days”): DiscoverText, La Cuisine du Web, Learn Assembly, OnlyLyon, Synomia, VisiBrain, FrenchWeb.fr, Social Media Club France.

The events and communication team at EMLYON (Typhaine Grisard, Nadia Brakta, Mathieu Cottereau and Alex Panizzo) made sure that the conference run efficiently, and we are also thankful to them for organizing this day.

Finally, we would like to express our gratitude to Mark Callahan, Senior Platform Growth Manager at Twitter, who helped this initiative take off the ground and develop into a successful event. Thank you.

Clément Levallois
Morgane Marchand
Tiago Mata
André Panisson
# Table of Contents

## Twitter Mix Days 2015

Foreword.................................................... III

Organization ................................................ IV

Acknowledgements ........................................ IV

Scientific literature on Twitter as a subject research: Findings based on Bibliometric Analysis .................................................... 1

Sibele Fausto and Pascal Aventurier

1 Introduction.................................................... 1

2 Methods ........................................................ 3

3 Results ........................................................ 4

3.1 Quantitative indicators.................................... 4

3.2 Citations and textual analysis................................ 7

4 Discussion and concluding remarks........................ 10

Elements for an epistemology of instrumentation and collaboration in Twitter data research .................................................... 15

Eglantine Schmitt

1 Introduction.................................................... 15

2 Bibliometric data collection ................................ 17

3 Data analysis.................................................... 18

4 Improving research driven by Twitter data ................ 22

5 Conclusion ........................................................ 24

Text mining and Twitter to analyze British swearing habits ................................ 27

Michael Gauthier, Adrien Guille, Fabien Rico, Anthony Deseille

1 Introduction.................................................... 28

2 Corpus description ............................................ 29

2.1 What is swearing? ........................................ 29

2.2 Social media and swearing ................................ 31

3 Methodology .................................................... 31

3.1 Inferring gender on Twitter ................................ 32

3.2 Inferring age .................................................. 32

4 Analysis and results ............................................ 33

4.1 Distribution of the number of tweets per gender and age .................... 33

4.2 Distribution of the number of tweets per user .................... 34

4.3 Proportion of swearing tweets among women and men .................... 34
<table>
  <tbody>
    <tr>
      <td>4.4</td>
      <td>Proportion of swearing tweets by gender per million words</td>
      <td>35</td>
    </tr>
    <tr>
      <td>4.5</td>
      <td>Average ratio of swearing tweets per day by gender and age</td>
      <td>36</td>
    </tr>
    <tr>
      <td>4.6</td>
      <td>Named-entity recognition</td>
      <td>38</td>
    </tr>
    <tr>
      <td>4.7</td>
      <td>Event detection</td>
      <td>40</td>
    </tr>
    <tr>
      <td>5</td>
      <td>Limitations</td>
      <td>43</td>
    </tr>
    <tr>
      <td>6</td>
      <td>Conclusion</td>
      <td>43</td>
    </tr>
    <tr>
      <td colspan="2">Mapping the structure of the global maker laboratories community through Twitter connections</td>
      <td>47</td>
    </tr>
    <tr>
      <td colspan="2">Massimo Menichinelli</td>
      <td></td>
    </tr>
    <tr>
      <td>1</td>
      <td>Introduction</td>
      <td>47</td>
    </tr>
    <tr>
      <td>2</td>
      <td>Method</td>
      <td>49</td>
    </tr>
    <tr>
      <td>3</td>
      <td>Results</td>
      <td>50</td>
    </tr>
    <tr>
      <td>3.1</td>
      <td>General information</td>
      <td>50</td>
    </tr>
    <tr>
      <td>3.2</td>
      <td>Modularity measure: the structure of the sub-communities</td>
      <td>50</td>
    </tr>
    <tr>
      <td>3.3</td>
      <td>Centrality measures: importance and influence in the network</td>
      <td>52</td>
    </tr>
    <tr>
      <td>4</td>
      <td>Conclusions</td>
      <td>56</td>
    </tr>
    <tr>
      <td colspan="2">MapTwitter tribal language(s)</td>
      <td>63</td>
    </tr>
    <tr>
      <td colspan="2">Nelleke Oostdijk and Hans van Halteren</td>
      <td></td>
    </tr>
    <tr>
      <td>1</td>
      <td>Introduction</td>
      <td>63</td>
    </tr>
    <tr>
      <td>2</td>
      <td>Language variation</td>
      <td>64</td>
    </tr>
    <tr>
      <td>3</td>
      <td>Hashtag clusters</td>
      <td>65</td>
    </tr>
    <tr>
      <td>4</td>
      <td>Preliminary findings of syntactic variation and similarities</td>
      <td>67</td>
    </tr>
    <tr>
      <td>5</td>
      <td>Annotation</td>
      <td>67</td>
    </tr>
    <tr>
      <td>6</td>
      <td>Results and discussion</td>
      <td>70</td>
    </tr>
    <tr>
      <td>6.1</td>
      <td>Principal component analysis</td>
      <td>71</td>
    </tr>
    <tr>
      <td>6.2</td>
      <td>Language use and variation between the four clusters</td>
      <td>71</td>
    </tr>
    <tr>
      <td>6.3</td>
      <td>Language use and variation within each of the clusters</td>
      <td>74</td>
    </tr>
    <tr>
      <td>7</td>
      <td>Conclusion</td>
      <td>80</td>
    </tr>
    <tr>
      <td>A</td>
      <td>Clusters</td>
      <td>83</td>
    </tr>
    <tr>
      <td>B</td>
      <td>Annotation scheme</td>
      <td>85</td>
    </tr>
    <tr>
      <td colspan="2">#TweetCommeUneFille: Twitter as giveaway for stereotyping speech acts</td>
      <td>88</td>
    </tr>
    <tr>
      <td colspan="2">Camille Lagarde-Belleville and Michel Otell</td>
      <td></td>
    </tr>
    <tr>
      <td>1</td>
      <td>Introduction</td>
      <td>88</td>
    </tr>
    <tr>
      <td>2</td>
      <td>About the corpus: why "#TweetCommeUnMec" and "#TweetCommeUneFille"?</td>
      <td>88</td>
    </tr>
    <tr>
      <td>2.1</td>
      <td>Twitter: a micro-blogging platform where discourse is embedded in a maximum of 140 characters</td>
      <td>89</td>
    </tr>
    <tr>
      <td>2.2</td>
      <td>The hashtag: a tool for referencing tweets with keyWord-dependent tags</td>
      <td>89</td>
    </tr>
    <tr>
      <td>2.3</td>
      <td>#TweetCommeUnMec vs. #TweetCommeUneFille: an archetypal couple</td>
      <td>89</td>
    </tr>
    <tr>
      <td>3</td>
      <td>A few words on our theoretical background</td>
      <td>90</td>
    </tr>
    <tr>
      <td>3.1</td>
      <td>The praxis</td>
      <td>90</td>
    </tr>
    <tr>
      <td>3.2</td>
      <td>The dialectic of the same and the other</td>
      <td>91</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>3.3</td>
      <td>About dialogism in nomination</td>
      <td>91</td>
    </tr>
    <tr>
      <td>3.4</td>
      <td>Ready-made in language</td>
      <td>92</td>
    </tr>
    <tr>
      <td>3.5</td>
      <td>Lexicalisation</td>
      <td>92</td>
    </tr>
    <tr>
      <td>3.6</td>
      <td>Folk wisdom</td>
      <td>92</td>
    </tr>
    <tr>
      <td>3.7</td>
      <td>Praxical patterns (or praxical schemes)</td>
      <td>92</td>
    </tr>
    <tr>
      <td>3.8</td>
      <td>Value-oriented stereotyping systems</td>
      <td>92</td>
    </tr>
    <tr>
      <td>4</td>
      <td>Tendencies from our corpus</td>
      <td>93</td>
    </tr>
    <tr>
      <td>4.1</td>
      <td>Sexuality and Relationship</td>
      <td>93</td>
    </tr>
    <tr>
      <td>4.2</td>
      <td>Sports and video games</td>
      <td>94</td>
    </tr>
    <tr>
      <td>4.3</td>
      <td>Culture</td>
      <td>96</td>
    </tr>
    <tr>
      <td>4.4</td>
      <td>Avenues for Reflexion</td>
      <td>96</td>
    </tr>
    <tr>
      <td>5</td>
      <td>#TweetLikeAMan versus #TweetLikeAGirl: highlights from the corpus and discussion</td>
      <td>97</td>
    </tr>
    <tr>
      <td>5.1</td>
      <td>Commonplaces and gender-dependent topics</td>
      <td>97</td>
    </tr>
    <tr>
      <td>5.2</td>
      <td>Hetero- Versus Auto-Representation</td>
      <td>98</td>
    </tr>
    <tr>
      <td>5.3</td>
      <td>Retweets and confrontational conversations</td>
      <td>98</td>
    </tr>
    <tr>
      <td>5.4</td>
      <td>Associated Hashtags</td>
      <td>99</td>
    </tr>
    <tr>
      <td>5.5</td>
      <td>Metadiscursive tweets and gender supremacy</td>
      <td>100</td>
    </tr>
    <tr>
      <td>6</td>
      <td>Conclusion</td>
      <td>101</td>
    </tr>
    <tr>
      <td colspan="2">Managing your online identity through Twitter within business-to-business (B2B) organisations</td>
      <td>103</td>
    </tr>
    <tr>
      <td colspan="2">Lucill J. Curtis</td>
      <td></td>
    </tr>
    <tr>
      <td>1</td>
      <td>Introduction: Aims</td>
      <td>103</td>
    </tr>
    <tr>
      <td>2</td>
      <td>Constructing identity – can it ever be unified into a single narrative voice?</td>
      <td>104</td>
    </tr>
    <tr>
      <td>3</td>
      <td>Existing online identity theories as multiple constructs</td>
      <td>105</td>
    </tr>
    <tr>
      <td>4</td>
      <td>Constructing identity – can it ever be unified into a single narrative voice?</td>
      <td>106</td>
    </tr>
    <tr>
      <td>5</td>
      <td>Narrative identity</td>
      <td>107</td>
    </tr>
    <tr>
      <td>6</td>
      <td>Introducing the new concept of ‘Narrative Public Voice’</td>
      <td>108</td>
    </tr>
    <tr>
      <td>7</td>
      <td>Method</td>
      <td>109</td>
    </tr>
    <tr>
      <td>7.1</td>
      <td>Data collection</td>
      <td>109</td>
    </tr>
    <tr>
      <td>8</td>
      <td>Research design</td>
      <td>109</td>
    </tr>
    <tr>
      <td>8.1</td>
      <td>New advances in qualitative research methods</td>
      <td>109</td>
    </tr>
    <tr>
      <td>8.2</td>
      <td>Collecting data from ‘Living Story Spaces’</td>
      <td>111</td>
    </tr>
    <tr>
      <td>8.3</td>
      <td>Understanding the context of the ‘Living Story Space’ - The collection of tweet data</td>
      <td>114</td>
    </tr>
    <tr>
      <td>9</td>
      <td>Phase one – Antenarrative processes</td>
      <td>114</td>
    </tr>
    <tr>
      <td>9.1</td>
      <td>Living Story Spaces analysis - Netnographic observations of Twitter sites from five case-study B2B organisations</td>
      <td>114</td>
    </tr>
    <tr>
      <td>10</td>
      <td>Living story – Organisational and personal identity through story</td>
      <td>116</td>
    </tr>
    <tr>
      <td>11</td>
      <td>Abductive interpretivist approach - Data analysis, interpretation and presentation</td>
      <td>117</td>
    </tr>
    <tr>
      <td>11.1</td>
      <td>Using coding to identify themes from the tweet data</td>
      <td>117</td>
    </tr>
    <tr>
      <td>12</td>
      <td>Phase two – Antenarrative processes: Narrator – (Narrativist) paradigm</td>
      <td>119</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>12.1 Qualitative semi-structured interviews</td>
      <td>119</td>
    </tr>
    <tr>
      <td>12.2 Characteristics of the participants</td>
      <td>119</td>
    </tr>
    <tr>
      <td>12.3 Interviews: Sampling method, access and participation/purposiveness</td>
      <td>119</td>
    </tr>
    <tr>
      <td>12.4 Exploring themes from the tweets written by the interviewees to inform the interviews</td>
      <td>121</td>
    </tr>
    <tr>
      <td>13 Emergent thematic identification and reasoning – interpretive interview data Analysis</td>
      <td>121</td>
    </tr>
    <tr>
      <td>13.1 Coding the interview data</td>
      <td>122</td>
    </tr>
    <tr>
      <td>14 Emergent findings and initial discussion</td>
      <td>122</td>
    </tr>
    <tr>
      <td>15 Contribution of this study, a new theoretical construct</td>
      <td>125</td>
    </tr>
    <tr>
      <td>Twitter data for urban policy making: an analysis on four European cities</td>
      <td>132</td>
    </tr>
    <tr>
      <td><i>Marta Severo, Timothée Giraud, and Hugues Pecout</i></td>
      <td></td>
    </tr>
    <tr>
      <td>1 Introduction</td>
      <td>132</td>
    </tr>
    <tr>
      <td>2 Method</td>
      <td>133</td>
    </tr>
    <tr>
      <td>2.1 The variety of uses of Twitter</td>
      <td>133</td>
    </tr>
    <tr>
      <td>2.2 The small amount of geo-tagged Tweets</td>
      <td>134</td>
    </tr>
    <tr>
      <td>3 Results</td>
      <td>134</td>
    </tr>
    <tr>
      <td>3.1 Frist method: tweets about the city</td>
      <td>134</td>
    </tr>
    <tr>
      <td>4 Tweets inside the city</td>
      <td>142</td>
    </tr>
    <tr>
      <td>4.1 City's influencers on Twitter</td>
      <td>145</td>
    </tr>
    <tr>
      <td>5 Conclusion</td>
      <td>152</td>
    </tr>
    <tr>
      <td>Building a recommender system using multilingual multiscript tweets</td>
      <td>156</td>
    </tr>
    <tr>
      <td><i>Ritesh Shah, Christian Boitet, Pushpak Bhattacharyya</i></td>
      <td></td>
    </tr>
    <tr>
      <td>1 Introduction</td>
      <td>156</td>
    </tr>
    <tr>
      <td>2 Background</td>
      <td>157</td>
    </tr>
    <tr>
      <td>2.1 Twitter</td>
      <td>157</td>
    </tr>
    <tr>
      <td>2.2 Tweets</td>
      <td>158</td>
    </tr>
    <tr>
      <td>3 Preliminary experiments</td>
      <td>158</td>
    </tr>
    <tr>
      <td>3.1 Data collection</td>
      <td>158</td>
    </tr>
    <tr>
      <td>3.2 Code-mixing: a preliminary study</td>
      <td>159</td>
    </tr>
    <tr>
      <td>3.3 Geo-information: a preliminary study</td>
      <td>160</td>
    </tr>
    <tr>
      <td>4 Modules and methodology</td>
      <td>160</td>
    </tr>
    <tr>
      <td>4.1 Twitter data: Preprocessing, Extraction, Analysis and Storage (PEAS)</td>
      <td>162</td>
    </tr>
    <tr>
      <td>4.2 Morphology-based Processes and Presyntactic Analysis (MPPA)</td>
      <td>162</td>
    </tr>
    <tr>
      <td>4.3 Named Entities identification (NEi)</td>
      <td>163</td>
    </tr>
    <tr>
      <td>4.4 Interlingual Lexical Disambiguation (WSD)</td>
      <td>163</td>
    </tr>
    <tr>
      <td>4.5 Content Extraction (CE) and Polarity Determination (PD)</td>
      <td>164</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>Colors of the World Cup: visualizations of images shared on Twitter during the 2014 World Cup</td>
      <td>167</td>
    </tr>
    <tr>
      <td>Fábio Goveia, Lia Carreira, Lucas Cypriano, Tasso Gasparini, Johanna Honorato, Veronica Haacke, Willian Lopes</td>
      <td></td>
    </tr>
    <tr>
      <td>1 Introduction</td>
      <td>167</td>
    </tr>
    <tr>
      <td>2 Methodology</td>
      <td>168</td>
    </tr>
    <tr>
      <td>3 Analysis</td>
      <td>171</td>
    </tr>
    <tr>
      <td>4 Final Considerations</td>
      <td>172</td>
    </tr>
    <tr>
      <td>Why Larry is different than Narry: A linguistic study of shipping communities in the One Direction fandom</td>
      <td>179</td>
    </tr>
    <tr>
      <td>Grace A. Ruiter</td>
      <td></td>
    </tr>
    <tr>
      <td>1 Introduction</td>
      <td>179</td>
    </tr>
    <tr>
      <td>2 Methodology</td>
      <td>180</td>
    </tr>
    <tr>
      <td>2.1 Data collection procedures</td>
      <td>180</td>
    </tr>
    <tr>
      <td>2.2 Data analysis procedures</td>
      <td>181</td>
    </tr>
    <tr>
      <td>3 Results</td>
      <td>183</td>
    </tr>
    <tr>
      <td>3.1 Elouonor Sample</td>
      <td>183</td>
    </tr>
    <tr>
      <td>3.2 Zerrie Sample</td>
      <td>185</td>
    </tr>
    <tr>
      <td>3.3 Sophiam Sample</td>
      <td>186</td>
    </tr>
    <tr>
      <td>3.4 Larry Sample</td>
      <td>187</td>
    </tr>
    <tr>
      <td>3.5 Narry Sample</td>
      <td>188</td>
    </tr>
    <tr>
      <td>3.6 Ziam Sample</td>
      <td>189</td>
    </tr>
    <tr>
      <td>4 Discussion</td>
      <td>189</td>
    </tr>
    <tr>
      <td>5 Conclusions</td>
      <td>192</td>
    </tr>
    <tr>
      <td>A friend, not a phone call: Brands on Twitter and the future of customer service</td>
      <td>195</td>
    </tr>
    <tr>
      <td>Josephine Bromley</td>
      <td></td>
    </tr>
    <tr>
      <td>1 Introduction</td>
      <td>195</td>
    </tr>
    <tr>
      <td>1.1 The impact of social media</td>
      <td>196</td>
    </tr>
    <tr>
      <td>1.2 The data</td>
      <td>197</td>
    </tr>
    <tr>
      <td>1.3 Difficulties with the data</td>
      <td>198</td>
    </tr>
    <tr>
      <td>1.4 Methodology</td>
      <td>199</td>
    </tr>
    <tr>
      <td>2 Customer service</td>
      <td>199</td>
    </tr>
    <tr>
      <td>2.1 Customers and consumers</td>
      <td>200</td>
    </tr>
    <tr>
      <td>2.2 Grammar of apologies</td>
      <td>203</td>
    </tr>
    <tr>
      <td>3 Customer experience</td>
      <td>206</td>
    </tr>
    <tr>
      <td>3.1 Content of tweets</td>
      <td>206</td>
    </tr>
    <tr>
      <td>3.2 Pronoun use</td>
      <td>211</td>
    </tr>
    <tr>
      <td>4 Discussion</td>
      <td>214</td>
    </tr>
    <tr>
      <td>5 Conclusion</td>
      <td>216</td>
    </tr>
  </tbody>
</table>
The tweeting brand: When conversation leads to humanization 222

Andria Andriuzzi

1 Introduction 222

2 Theoretical framework: Anthropomorphism and brand personification 223

3 Method 224

4 Results 225

4.1 Conversational brands and anthropomorphism 225

4.2 Brand conversation effects on consumers 227

4.3 Four perceptions of conversational brands 227

4.4 A quality conversation 228

5 Discussion and conclusion 230
Scientific literature on Twitter as a subject research:
Findings based on Bibliometric Analysis

Sibele Fausto¹ and Pascal Aventurier²

¹ University of São Paulo (USP)
São Paulo, Brazil
sifausto@usp.br *

² Institut National de la Recherche Agronomique, Délégation Information Scientifique et Technique, Services Aéconcentrés for Services Déconcentrés (INRA-DIST-SDAR)
France
Pascal.Aventurier@paca.inra.fr

**Abstract.** Since its launch in 2006, Internet platform Twitter has rapidly expanded. As a phenomenon of the digital era, Twitter generates a new type of research data that has received a good deal of attention in the academic literature. It has turned into a popular subject research that has been widely investigated in the academic world in different fields ranging from the Social Sciences to Health Sciences, addressing various questions, methods approaches, and covering multiple data sets. This study provides some findings of a bibliometric study which was conducted to describe the scientific literature available on Twitter with descriptive, quantitative information and also in a qualitative approach, in addition to the previous studies and designed as a contribution to a broader picture of how the evolution of the current scientific literature about Twitter is related to bibliographic data sets. Results show a variety of findings that can provide a better comprehension of this social media platform which evolved from a data source for the research to, nowadays, being a research subject itself.

**Keywords:** Twitter, scientific literature, bibliometric analysis

# 1 Introduction

Since its launch in 2006, founded by Jack Dorsey and associates in San Francisco, California, USA, Internet platform Twitter (https://twitter.com) has rapidly expanded, “conceived as part of a long line of squawk media, dispatch, short messaging, as well as citizen communications services” (Rogers, 2014), with Dorsey trying to define Twitter as a new medium in itself, a public instant messaging system, meaning also a public information utility (Sarno, 2009).

* Correspondence concerning this article should be addressed to Sibele Fausto, Technical Department of the Integrated Library System, University of São Paulo, Rua da Biblioteca, s/n, Complexo Brasiliana, São Paulo, SP, CEP 05508-050 (Brazil). Email: sifausto@usp.br.
As a phenomenon of the digital era, Twitter generates a new type of research data related to this role as public information utility, emerging as an important source of the new digital paradigm, and has received a good deal of attention in the academic literature. It has turned into a popular research subject that has been widely investigated in the academic world in different fields, ranging from the Social Sciences to the Health Sciences, addressing various questions, method approaches, and covering multiple data sets, with early studies focusing on characterizing tweet types, determining how many of them are of value, and evaluating Twitter as, more or less, of interesting content (Rogers, 2014).

An analysis of the coverage of Twitter's first three years, performed by Arceneaux and Schmitz Weiss (2010), pointed out the prevalence of “food tweets” and the more general “mindless stream”, emanating from Twitter in its early years, leading Rogers (2014) to ask what value lies in breakfast and lunch tweets, and responding that “geo-located food tweets may be of interest to those studying the geography of taste and other questions of cultural preference” (p. xiii-xiv).

Thus, together with other social web platforms, such as Facebook, blogs, Wikipedia, YouTube, and others, Twitter becomes a new source of information for many researchers in different disciplines, providing a variety of data to extract (Metaxas & Mustafaraj, 2014). Despite being behind services like Facebook in numbers (with 1.3 billion active users) and WhatsApp (500 million), this platform currently has about 284 million active users worldwide, with 500 million tweets being sent out every day in more than 30 different languages (Twitter, 2015). In 2014 the eMarketer Consultancy predicted that Twitter should reach 300 million monthly active users by 2016, and pass the 400 million mark by the end of 2018.

Since its launch Twitter has incorporated growing facilities for sharing content, with options to add links, images and videos, and efficient mechanisms to aggregate specific topics by hashtags: labels assigned to a term, preceded by the hash symbol (#), making the term an active link that allows recovery of all tweets with the hashtag. Hashtags not only gather and record the tweets on the same subject but also make it possible to recover its track record, through monitoring or mining. Moreover, Twitter's dynamics, where users develop a virtual community made up of a public identified by common interests - followers, who share and replicate content freely with each other in an intense interaction through actions, such as the “re-tweet” (RT), generating affiliations around themes, topics and identities (Zappavigna, 2011). All these characteristics make Twitter a major data source to track the performance of personal and institutional microblogs, communities and the most diverse subjects through systems that monitor the interactions of real-time messages generated on the platform, targeting to various analyses, hence the growth of scientific literature on Twitter.

Bruns and Weller (2014) pointed out that scientific publications, with “Twitter” explicitly in the title, are as follows: 1,400 on Scopus® database, 470 on Web of Science® (WoS) database and another 10,000 available via Google Scholar. In turn, Zimmer and Proferes (2014) conducted a content analysis study with a set of 382 academic publications that used Twitter from 2006 to 2012. Also, Weller et al (2014) published a comprehensive study on Twitter, addressing aspects of this social media related to its concepts and methods, even as
its perspectives and practices in use in society, as those from popular culture, brand and
crisis communication, politics and activism, journalism, and in academia.

This study provides some more insights in this matter through a bibliometric study which was conducted to describe the scientific literature available on Twitter, with quantitative information and also in a qualitative approach, in addition to the previous studies and designed as a contribution to a broader picture of how the evolution of the current scientific literature about Twitter is related to bibliographic data sets.

## 2 Methods

The methodological approach chosen for this study is Bibliometric Analysis, an original area of study generally covering books and publications and whose principle, according to Tague-Sutcliffe (1992), is the analysis of scientific or technical activity through quantitative studies of publications. Bibliometric Analysis provides insights on which stage a given research area lies, and it is a methodology also used in several areas, with the objective of characterizing the published research output and the forecasting of trends, yielding results that are also useful to support decision-making (Rostaing, 1996).

Through Bibliometric Analysis it is possible to study some quantitative parameters of the publications, periodicals, authors, keywords, users and citations (Pao, 1989). But nowadays Bibliometric Analysis integrates the broader area of metric studies of the information, with different scopes and finalities in the discovery of information characteristics in general, not just about science, comprising and combining different methodologies and tools to provide a more comprehensive view of its objects of study (Cronin & Sugimoto, 2014).

This Bibliometric Analysis was carried out for the indexed literature in the period from 2006 (when Twitter was released) until 2014, and only included articles with the word "Twitter" in the title, abstract and/or keywords. Analysis was conducted on the results of the Scopus® database (from Reed-Elsevier), which has a wider coverage than WoS® (from Thomson Reuters). The 2,338 documents retrieved were analyzed on two levels. The first was a basic one related to quantitative indicators, according to the average growth in the number of publications per year, identifying main research areas that have published about Twitter, as well as journal titles, most prolific authors, institutions and countries. This was followed by a second level with a deeper analysis, considering citations and the 'hot topics' of the literature, through the combination of techniques and tools of lexical analysis and content about the corpora, with the software Sphinx Lexica®, that allows the researcher to differentiate in a semi-automatic form the main terms, identifying those that are 'hot topics' in the publications about Twitter, and CorText Manager<fnref n="3" />, that was used to discover the relationships between the different topics and countries involved, providing maps of heterogeneous networks as thematic and collaborative maps, sorted by term frequency per countries and by a chi-squared (χ²) distribution of the "hot topics".

<footnote n="3">³ CorText Manager: urlhttp://www.cortex.net/.</footnote>
# 3 Results

## 3.1 Quantitative indicators

The academic literature with Twitter as research subject has seen an average growth rate of 52.23% per year, from one sole article published in 2006 to the 720 published in 2014. Figure 1 shows that this growth was ongoing and has accelerated after 2009, reaching the peak in 2014. This result points out that in fact Twitter is becoming a consolidated object of research.

<figure><img src="image_5.png" /><figcaption>Fig. 1: Growth of the scientific literature about Twitter (2006-2014).</figcaption></figure>

As for the research areas which have published about Twitter, Figure 2 shows the dominance of certain disciplines with more than 100 publications, such as the Social Sciences, with 1,014 articles (33% of the total sample); and Computer Science, with 826 articles (27%), followed by Medicine, with 327 articles (11%). But we can observe that other disciplines also appear with relevant participation, such as Business Management and Accounting, and Engineering, which had 284 and 273 papers, respectively, both reaching 9% of the total sample; followed by Arts and Humanities, with 215 (7%) and Mathematics, with 105 (4%).

Related to journal titles with more than 10 publications in this matter, results were as follows: PLoS One (n = 46); Computers in Human Behavior (n = 37); Journal of Medical Internet Research (n = 37); First Monday (n = 36); Information Communication and Society (n = 35); Public Relations Review (n = 34); *Estudios Sobre El Mensaje Periodistico* (n = 19); Social Science Computer Review (n = 18); E-content (n = 17); Cutting Edge Technologies in Higher Education (n = 16); New Media and Society (n = 15); Expert Systems with Applications (n = 14); ACM Transactions on Intelligent Systems and Technology, Journal of Communication, Government Information Quarterly, and PC World San Francisco CA, 13 articles each; Journal of the American Society for Information Science and Technology, and
<figure><img src="image_6.png" /><figcaption>Fig. 2: Number of articles by disciplines.</figcaption></figure>

Online Wilton Connecticut with 12 articles each; American Behavioral Scientist (n = 11); Cyberpsychology Behavior and Social Networking, Proceedings of the ASIST Annual Meeting, and Studies in Computational Intelligence, with 10 articles each [Figure 3].

<figure><img src="image_7.png" /><figcaption>Fig. 3: Top journal titles with articles published about Twitter (2006-2014).</figcaption></figure>

The most prolific authors were H. W. Park with 13 articles and A. Bruns, with 12 ones, but others authors also figured with a good research output in this matter, such as J. J. Jung and T. Highfield, both with 9 articles; P. R. Spence and S. H. Burton, with 8 articles; and A. O. Larsson, I. Himelboim, J. K. Harris and M. Thelwall with 7 articles each [Table 1].
<table><caption>Table 1: Most prolific authors who published about Twitter (2006-2014)</caption><thead><tr><td>Author</td><td>n. Articles</td></tr></thead><tbody><tr><td>Park, H.W.</td><td>13</td></tr><tr><td>Bruns, A.</td><td>12</td></tr><tr><td>Jung, J.J.</td><td>9</td></tr><tr><td>Highfield, T.</td><td>9</td></tr><tr><td>Spence, P.R.</td><td>8</td></tr><tr><td>Burton, S.H.</td><td>8</td></tr><tr><td>Larsson, A.O.</td><td>7</td></tr><tr><td>Himelboim, I.</td><td>7</td></tr><tr><td>Harris, J.K.</td><td>7</td></tr><tr><td>Thelwall, M.</td><td>7</td></tr></tbody></table>

As for institutions with more than 15 contributions, results show Yeungnam University with the most number of articles published (30 contributions); followed by Queensland University of Technology (n = 22); Pennsylvania State University, the University of Oxford and the University of Toronto (n = 18 each); University of Texas at Austin (n = 17); University of Maryland and Indiana University (n = 16 each); Seoul National University, University College, (London), Carnegie Mellon University, University of New York State at Buffalo, with 15 articles each [Figure 4].

<figure><img src="image_8.png" /><figcaption>Fig. 4: Top institutions with more than 15 articles published about Twitter (2006-2014).</figcaption></figure>

The following countries had more than 50 articles: the United States (USA) with 912 publications (39% of the total sample); the United Kingdom (UK) with 216 (9.23%); Spain, 148 (6.33%); Australia, 119 (5.1%); South Korea, 116 (4.9%); China, 99 (4.23%); Canada, 98 (4.2%); Germany, 65 (2.8%); Japan, 63 (2.7%); and The Netherlands, 51 (2.2%) [Figure 5].

Ahead, in this analysis, we will show those same countries according to the topics most published by them, forming clusters by equivalent subject.
10 Main countries publishing about Twitter

<figure><img src="image_9.png" /><figcaption>Fig. 5: Top countries with more than 50 articles published about Twitter (2006-2014).</figcaption></figure>

**3.2 Citations and textual analysis**

**Most cited authors and journals.** In addition to the basic quantitative indicators shown before, we also examined the data sample to verify the most cited papers in order to get insights about the impact of that scientific literature with Twitter as subject research. Table 2 shows articles with more than 100 citations, summarized by authors, article titles, publication year, journal titles, number of citations received and the topics.

The most cited authors were Kaplan and Haenlein (2010), with an article published in the "Business Horizons" journal, which reached 1,296 citations. As the journal title points out, the main topic of this paper is Business. Next, the authors cited about 400 times were Jansen et al (2009), with an article cited 438 times, in the topic of Information in the "Journal of the American Society for Information Science and Technology;" and Bollen, Mao and Zeng (2011), cited 430 times, with a paper on the topic of Computation, in the "Journal of Computational Science".

The third group of most-cited authors, which reached about 200 citations, were Kietzmann et al (2011) with an article on the topic of Business, again in the "Business Horizons" journal, with 283 citations; Marwick and Boyd (2011) on the topic of Information in "New Media and Society," with 243 citations; and, finally, Yardi, Romero and Schoenebeck (2009), with an article in "First Monday" on the General topic that reached 239 citations.

The fourth and last group of most-cited authors spanned 116 to 176 citations and includes 14 journals, with topics like Science (3 occurrences), Health (5 occurrences), Information (2 occurrences), Computation (2 occurrences) and Business (1 occurrence) [Table 2].

Only four journals with most-cited articles are also in the top journals list with more than 10 articles published about Twitter, as follows: "PLoS One" has published 46 articles on Twitter and also figured among the most-cited with two articles, from Chew and Eysenbach
<table><caption>Table 2: Most prolific authors who published about Twitter (2006-2014)</caption><thead><tr><th>Author</th><th>Title</th><th>Year</th><th>Journal</th><th>n.</th><th>Citations</th><th>Topic</th></tr></thead><tbody><tr><td>Kaplan A.M.; Haenlein M.</td><td>Users of the world, unite! The challenges and opportunities of Social Media</td><td>2010</td><td>Business Horizons</td><td>1296</td><td></td><td>BUSINESS</td></tr><tr><td>Jansen B.J.; Zhang M.; Sobel K.; Twitter power: Tweets as electronic word of mouth</td><td>Twitter mood predicts the stock market</td><td>2009</td><td>Journal of the American Society for Information Science and Technology</td><td>438</td><td></td><td>INFORMATION</td></tr><tr><td>Chowdury A.</td><td>Kietzmann J.H.; Hermkens K.; McCarthy I.P.; Silvestre B.S.</td><td>2011</td><td>Business Horizons</td><td>283</td><td></td><td>BUSINESS</td></tr><tr><td>Bollen J.; Mao H.; Zeug X.</td><td>Social media? Get serious! Understanding the functional building blocks of social media</td><td>2011</td><td>Business Horizons</td><td>430</td><td></td><td>COMPUTATION</td></tr><tr><td>Marwick A.E.; Boyd D.</td><td>I tweet honestly. I tweet passionately: Twitter users, context collapse, and the imagined audience</td><td>2011</td><td>New Media and Society</td><td>243</td><td></td><td>INFORMATION</td></tr><tr><td>Yardi, S.; Romero, D.; Schoenebeck, G.</td><td>Detecting spam in a Twitter network</td><td>2009</td><td>First Monday</td><td>239</td><td></td><td>GENERAL</td></tr><tr><td>Chew C.; Eysenbach G.</td><td>Pandemics in the age of Twitter: Content analysis of tweets during the 2009 H1N1 outbreak</td><td>2010</td><td>PLoS One</td><td>176</td><td></td><td>SCIENCE</td></tr><tr><td>Hawn C.</td><td>Report from the field: Take two aspirin and tweet me in the morning: How twitter, facebook, and other social media are reshaping health care.</td><td>2009</td><td>Health Affairs</td><td>170</td><td></td><td>HEALTH</td></tr><tr><td>Boyd D.; Crawford K.</td><td>Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon</td><td>2012</td><td>Information Communication and Society</td><td>167</td><td></td><td>INFORMATION</td></tr><tr><td>Golder S.A.; Macy M.W.</td><td>Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures</td><td>2011</td><td>Science</td><td>161</td><td></td><td>SCIENCE</td></tr><tr><td>Signorini A.; Segre A.M.; Polgreen P.M.</td><td>The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic</td><td>2011</td><td>PLoS One</td><td>161</td><td></td><td>SCIENCE</td></tr><tr><td>Scanfeld D.; Scanfeld V.; Larson E.L.</td><td>Dissemination of health information through social networks: Twitter and antibiotics</td><td>2010</td><td>American Journal of Infection Control</td><td>147</td><td></td><td>HEALTH</td></tr><tr><td>Junco R.; Heiberger G.; Loken E.</td><td>The effect of Twitter on college student engagement and grades</td><td>2011</td><td>Journal of Computer Assisted Learning</td><td>143</td><td></td><td>COMPUTATION</td></tr><tr><td>O'Keeffe G.S.; Clarke-Pearson K.; Mulligan D.A.; et al.</td><td>Clinical report - The impact of social media on children, adolescents, and families</td><td>2011</td><td>Pediatrics</td><td>131</td><td></td><td>HEALTH</td></tr><tr><td>Thelwall M.; Buckley K.; Paltoglou G.</td><td>Sentiment in Twitter events</td><td>2011</td><td>Journal of the American Society for Information Science and Technology</td><td>131</td><td></td><td>INFORMATION</td></tr><tr><td>Ebner M.; Lienhardt C.; Rohs M.; Microblogs in Higher Education - A chance to facilitate informal and process-oriented learning?</td><td>Computers and Education</td><td>2010</td><td></td><td>131</td><td></td><td>COMPUTATION</td></tr><tr><td>Meyer I.</td><td>Social Internet Sites as a Source of Public Health Information</td><td>2009</td><td>Dermatologic Clinics</td><td>128</td><td></td><td>HEALTH</td></tr><tr><td>Vance K.; Howe W.; Dellavalle R.P.</td><td>Hennig-Thurau T.; Malthouse E.C.; et al.</td><td>2010</td><td>Journal of Service Research</td><td>121</td><td></td><td>BUSINESS</td></tr><tr><td>Lee Hughes A.; Palen L.</td><td>Twitter adoption and use in mass convergence and emergency events</td><td>2009</td><td>International Journal of Emergency Management</td><td>116</td><td></td><td>HEALTH</td></tr></tbody></table>
(2010) which received 176 citations, and from Signorini et al (2011), with 161 citations. The "First Monday" has published 36 articles on Twitter and received 239 citations in the article by Yardi, Romero and Schoenebeck (2009). The "Information Communication and Society" has published 35 articles and received 167 citations from Boyd and Crawford (2012). Finally, the "New Media and Society" journal published 15 articles on Twitter and received 243 citations to the Marwick and Boyd's article (2011).

So there isn't a clear relationship between the largest number of published articles on Twitter and a greater number of citations received, as only four (18,18%) from a total of 22 journals with more than 10 publications on Twitter received more than 100 citations.

**"Hot Topics" in the literature about Twitter.** Continuing our analysis, we proceeded in the mining of the most frequent topics into literature about Twitter, through applying the software Sphinx Lexica® in a semi-automatic lexical corpus analysis, after removing the term "Twitter" from it, identifying specific terms with more than 100 occurrences by year to observe their evolution in frequency during the analyzed period of 2006-2014. All terms were joined together in groups of equivalent ones.

Results show several most frequent topics, with "social media" leading in a total of 3,197 occurrences, with an ongoing growth in its frequency over the period, followed by the topic "social networks," with the total of 1,981 occurrences, also with an ongoing growth in its frequency over the same period.

"Hot topics" also very often include the topic of "learning" (human learning), with 360 occurrences; "health" (330); "political activity" (324), among others represented in Table 3.

Representative keywords were evolving rapidly. For example, "Second Life" was representative in the early years. In turn "big data" and "data mining" appear over the whole time period. It was observed that in general most of the topics had a continued growth in frequency since the year in which they first appeared in the literature on Twitter, and this evolution of their frequency can predict the trends of these topics.

This topic analysis also allowed the mapping of these topics by clusters, according to related lexical similarities, e.g. a topic cluster related to "health" groups terms such as "public health," "patient" and "health care," (represented in clear blue in Figure 6), as well as a topic cluster related to terms such as "political activity," "social movements," "public relations" and "mass communication" (represented in clear yellow in Figure 6, that also shows the other topic clusters that emerged in this analysis).

This "hot topic" analysis also allowed clustering topics related to their frequency by countries, as shown in Figure 6, and also pointing out the relations between countries by thematic collaborations, next in Figure 7.

In Figure 6 there are countries that are more present in each cluster, addressing related topics sorted by decreased term frequency, such as USA, China, UK, Spain, Australia, Canada, South Korea, Japan, etc. But in Figure 7 it is possible to observe the thematic collaborations between these countries and others that were checked through the institutional affiliations (by country) in the recovered articles, with a chi-squared ($\chi^2$) distribution of the "hot topics". It was observed that there are only 340 papers with international collaborations,
<table><caption>Table 3: “Hot topics” and their evolution in literature about Twitter (2006-2014)</caption><thead><tr><th>Topics</th><th>2006</th><th>2007</th><th>2008</th><th>2009</th><th>2010</th><th>2011</th><th>2012</th><th>2013</th><th>2014</th><th>TOTAL</th></tr></thead><tbody><tr><td>social_media</td><td>0</td><td>0</td><td>1</td><td>23</td><td>10</td><td>370</td><td>494</td><td>984</td><td>1211</td><td>3197</td></tr><tr><td>social_networks</td><td>0</td><td>4</td><td>19</td><td>50</td><td>128</td><td>221</td><td>373</td><td>592</td><td>593</td><td>1981</td></tr><tr><td>learning</td><td>0</td><td>0</td><td>0</td><td>1</td><td>10</td><td>38</td><td>61</td><td>93</td><td>157</td><td>360</td></tr><tr><td>health</td><td>0</td><td>0</td><td>0</td><td>4</td><td>11</td><td>30</td><td>35</td><td>122</td><td>127</td><td>330</td></tr><tr><td>political_activity</td><td>0</td><td>0</td><td>0</td><td>0</td><td>3</td><td>38</td><td>44</td><td>114</td><td>125</td><td>324</td></tr><tr><td>mobile</td><td>0</td><td>2</td><td>2</td><td>14</td><td>28</td><td>32</td><td>67</td><td>68</td><td>69</td><td>282</td></tr><tr><td>real_time</td><td>0</td><td>0</td><td>6</td><td>1</td><td>11</td><td>28</td><td>38</td><td>60</td><td>53</td><td>197</td></tr><tr><td>content_analysis</td><td>0</td><td>0</td><td>0</td><td>0</td><td>10</td><td>20</td><td>30</td><td>58</td><td>73</td><td>191</td></tr><tr><td>sentiment_analysis</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>12</td><td>25</td><td>56</td><td>89</td><td>182</td></tr><tr><td>marketing</td><td>0</td><td>0</td><td>0</td><td>5</td><td>8</td><td>22</td><td>34</td><td>61</td><td>45</td><td>175</td></tr><tr><td>twitter_users</td><td>0</td><td>0</td><td>1</td><td>2</td><td>2</td><td>11</td><td>29</td><td>48</td><td>69</td><td>162</td></tr><tr><td>case_study</td><td>0</td><td>0</td><td>0</td><td>5</td><td>5</td><td>11</td><td>30</td><td>47</td><td>57</td><td>155</td></tr><tr><td>opinion</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>13</td><td>23</td><td>45</td><td>66</td><td>147</td></tr><tr><td>public_health</td><td>0</td><td>0</td><td>0</td><td>4</td><td>5</td><td>18</td><td>21</td><td>40</td><td>50</td><td>138</td></tr><tr><td>facebook</td><td>0</td><td>0</td><td>0</td><td>3</td><td>9</td><td>24</td><td>17</td><td>25</td><td>48</td><td>126</td></tr><tr><td>news</td><td>0</td><td>0</td><td>1</td><td>1</td><td>4</td><td>22</td><td>24</td><td>32</td><td>41</td><td>125</td></tr><tr><td>qualitative_analysis_data</td><td>0</td><td>0</td><td>0</td><td>0</td><td>3</td><td>18</td><td>19</td><td>24</td><td>54</td><td>118</td></tr><tr><td>web_sites</td><td>0</td><td>0</td><td>1</td><td>4</td><td>19</td><td>19</td><td>23</td><td>25</td><td>23</td><td>114</td></tr><tr><td>big_data</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>27</td><td>30</td><td>55</td><td>112</td></tr><tr><td>large_scale</td><td>0</td><td>0</td><td>0</td><td>0</td><td>6</td><td>7</td><td>23</td><td>26</td><td>43</td><td>105</td></tr><tr><td>higher_education</td><td>0</td><td>0</td><td>0</td><td>1</td><td>10</td><td>18</td><td>16</td><td>36</td><td>22</td><td>103</td></tr><tr><td>userActivities</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>2</td><td>19</td><td>28</td><td>49</td><td>102</td></tr><tr><td>second_life</td><td>0</td><td>1</td><td>0</td><td>1</td><td>5</td><td>10</td><td>0</td><td>2</td><td>0</td><td>19</td></tr></tbody></table>

13% of the whole corpus, which is a very low percentage compared to other domains in international research.

These clusters, for example, show such themes as “collective action”, “big data”, “text and web mining”, “marketing”, “political activity” and “user generated content” and were common topics in the collaborations between Brazil, France, India, Italy, UK, and others. In turn, topics such as “crisis communication”, “corporate communication”, “traditional education”, “political activity” and “sentiment classification”, were most common in collaborations between Belgium, Switzerland, The Netherlands, and Turkey. This analysis is useful to realize which countries have collaborations in the research of specific topics about Twitter.

## 4 Discussion and concluding remarks

These findings indicate that there was a steady growth in the academic literature which includes Twitter as research subject in the period from its founding, in 2006, to 2014. The subject areas with most contributions were the Social Sciences, followed by Computer Science and Medicine. There is a long tail of journal titles that have published articles on this subject, with 22 titles in more than 10 publications, all of them in English, apart from a single title in
<figure><img src="image_10.png" /><figcaption>Fig. 6: “Hot topics” by countries sorted by decreased term frequency.</figcaption></figure>

Spanish. Two authors stand out in terms of the number of articles, and various institutions have made contributions, most of them being universities from North America (USA and Canada) and Europe (UK, Spain, Germany and The Netherlands), although Asia (China, Japan and South Korea) and Oceania (Australia) also stood out.

With a second-level analysis in a deeper manner, it was possible also consider citations and the “hot topics” of the literature about Twitter, in order to verify their impact, and to identify which topics were more discussed in this literature, as well as to find out with a mapping tool how these topics were distributed by subject groups and countries, including collaborations between these countries. It was observed that there is a group of papers which received more than 100 citations, but there was not a clear relation between the largest number of published articles on Twitter and a greater number of citations received.

There are several “hot topics” that were addressed in the literature on Twitter, with “social media” and “social networks” leading with 3,197 and 1,981 occurrences, respectively, and the majority of these topics had a continued growth in their frequency in the literature during the analyzed period. Twitter analysis is strongly related to social issues and enhanced problems linked to youth populations, as health (obesity), public health, international disasters, drug abuse, but also to the learning and education process. Three other important fields of analysis are social behavior (opinion), political activity and branding/media strategy.
<figure><img src="image_11.png" /><figcaption>Fig. 7: “Hot topics” by country collaborations sorted by $\chi^2$ distribution.</figcaption></figure>

It was interesting to see how these “hot topics” were organized under common subjects and how they were distributed between different countries, including collaborations between these countries in the research of certain topics about Twitter. There are a few international collaborations (13% of the corpus) and it is mostly delimited to traditional collaborator countries (between USA, UK, and those from Asia – China and South Korea).

All these findings add to and group together with previous studies about Twitter and aim at a contribution to a better comprehension of this social media platform which evolved from a data source for the research to, nowadays, being a research subject itself.

Future directions of the investigation of this role of Twitter as an object of study through Bibliometric Analysis are a continuous task that point to other approaches which can provide more detailed insights, such as how the research network is drawn from the cited references.
Bibliography

[1] Arceneaux, N., & Schmitz Weiss, A. (2010). Seems stupid until you try it: Press coverage of Twitter, 2006–9. *New Media & Society*, 12(8), 1262–1279. doi:10.1177/1461444809360773.

[2] Bollen, J., Mao, H., & Zeng, X. (2011). Twitter mood predicts the stock market. *Journal of Computational Science*, 2 (1), 1-8. doi:10.1016/j.jocs.2010.12.007.

[3] Boyd, D., & Crawford, K. (2012). Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. *Information, Communication & Society*, 15(5), 662-679. doi:10.1080/1369118X.2012.678878.

[4] Bruns, A., & Weller, K. (2014). Twitter Data Analytics – Or: the Pleasures and Perils of Studying Twitter. *Aslib Journal of Information Management*, 66 (3). DOI: http://dx.doi.org/10.1108/AJIM-02-2014-0027.

[5] Chew, C., & Eysenbach, G. (2010). Pandemics in the age of Twitter: content analysis of Tweets during the 2009 H1N1 outbreak. *PloS One*, 5(11), e14118. doi:10.1371/journal.pone.0014118.

[6] Cronin, B., & Sugimoto, C. R. (Eds.) (2014). *Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Scholarly Impact*. Cambridge, MA: MIT Press. 466 p.

[7] eMarketer Consultancy (2014). *Emerging Markets Drive Twitter User Growth Worldwide*. Accessed 9 Jun. 2015 from: http://www.emarketer.com/Article/Emerging-Markets-Drive-Twitter-User-Growth-Worldwide/1010874\#sthash.pxPo8DI3.dpuf.

[8] Jansen, B.J., Zhang, M., Sobel, K., & Chowdury A. (2009). Twitter power: Tweets as electronic word of mouth. *Journal of the American Society for Information Science and Technology*, 60 (11), 2169-2188. doi:10.1002/asi.21149.

[9] Kaplan A.M., & Haenlein, M. (2010). Users of the world, unite! The challenges and opportunities of Social Media. *Business Horizons*, 53(1), 59–68. doi:10.1016/j.bushor.2009.09.003.

[10] Kietzmann, J. H., Hermkens, K., McCarthy, I. P., & Silvestre, B. S. (2011). Social media? Get serious! Understanding the functional building blocks of social media. *Business Horizons*, 54(3), 241-251. doi: 0.1016/j.bushor.2011.01.005.

[11] Marwick, A. E. (2011). I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience. *New Media & Society*, 13(1), 114-133. doi:10.1177/1461444810365313.

[12] Metaxas, P. & Mustafaraj, E. (2014). Sifting the sand on the river bank: Social media as a source for research data. *IT-Information Technology*, 56 (5), 230–239. doi:10.1515/itit-2014-1047.

[13] Pao, M. L. (1989). *Concepts of information retrieval*. Englewood, Colorado: Libraries Unlimited, Inc. 285 p.

[14] Rogers, (2014). Debanalising Twitter: The Transformation of an Object of Study. In Weller, K., Bruns, A., Burguess, J., Mahrt, M., & Puschmann, C. (Eds). *Twitter and Society* (pp. ix- xxvi). New York: Peter Lang Publishing.
[15] Rostaing, H. (1996). *La bibliométrie et ses techniques*. Marseille: CRRM.135 p.

[16] Sarno, (2009). Twitter creator Jack Dorsey illuminates the site's founding document. Part I. *Los Angeles Times*. Accessed 15 Jun. 2015 from: http://latimesblogs.latimes.com/technology/2009/02/twitter-creator.html.

[17] Signorini, A., Segre, A. M., & Polgreen, P. M. (2011). The use of Twitter to track levels of disease activity and public concern in the US during the influenza A H1N1 pandemic. *PloS One*, 6(5), e19467. doi:10.1371/journal.pone.0019467.

[18] Tague-Sutcliffe (1992). An Introduction to Informetrics. *Information Processing Management*, 28 (1), 1-3. doi:10.1016/0306-4573(92)90087-G.

[19] Twitter Inc. (2015). *About*. Accessed 9 Jun. 2015 from: https://about.twitter.com/company.

[20] Weller, K., Bruns, A., Burguess, J., Mahrt, M., & Puschmann, C. (Eds) (2014). *Twitter and Society*. New York: Peter Lang Publishing. 486 p.

[21] Yardi, S., Romero, D., & Schoenebeck, G. (2009). Detecting spam in a twitter network. *First Monday*, 15(1). doi: http://dx.doi.org/10.5210/fm.v15i1.2793.

[22] Zappavigna, M. (2011). Ambient affiliation: A linguistic perspective on Twitter. *New Media & Society*, 3(5), 788-806. doi:10.1177/1461444810385097.

[23] Zimmer, M. & Proferes, N. J. (2014). A topology of Twitter research: disciplines, methods, and ethics. *Aslib Journal of Information Management*, 66 (3), 250-261. doi: 10.1108/AJIM-09-2013-0083.
# Elements for an epistemology of instrumentation and collaboration in Twitter data research

Eglantine Schmitt

Sorbonne Universités, Université de Technologie de Compiègne

Proxem, Compiègne

France

eglantine.schmitt@utc.fr

**Abstract.** Twitter has become the most studied online social network in academia, in social sciences as well as in other fields. It is commonly grasped through a collection and analysis of its own data. In this paper, I show through a bibliometric analysis that scholarly publications on this matter come equally from social and computer sciences, and from natural sciences to a lesser extent. Social scientists rely mostly on classical quantitative methods while computer scientists try to improve algorithms and techniques. Twitter data can take several epistemic values, from representing nothing to representing real-world social phenomena. Having observed the infrequency of interdisciplinary works, I make a few suggestions based on the history of science for future collaborative projects based on Twitter data.

**Keywords:** Twitter, epistemology, bibliometrics, instrument research, interdisciplinarity

## 1 Introduction

In a few years' time, Twitter has become the most studied online social network (OSN) in academia. For Tufekci (2014), it is to Web study what the *Drosophila melanogaster* is to biology: a model organism, chosen for convenience at first, then studied so much more than any other social network that it polarises research, giving scholars a well-documented reference where one can add cumulative knowledge. The choice of a model organism is never entirely arbitrary; there are usually good reasons to choose one in particular, but those may be more practical than scientific ones. In Twitter's case, the fact that most tweets are public, added to the availability of an application programming interface (API) which remained more or less stable over time, makes sufficient amounts of Twitter data available to researchers, which is basically the condition of possibility of any scientific investigation on the matter. This is probably why Twitter is, as of today, much more studied than Facebook, although the latter has much more users (1.441 billion versus 302 million monthly active users during Q1 2015, according to Statista 2015) and should then provide a richer and more detailed view of online sociability. What sets Twitter and Facebook apart, from a data collection point of view, is that, while Facebook also provides an API, it only shows the private data that one is allowed to see, that is, posts and comments from one's friends. As a consequence, full data sets analysis is possible only to Facebook R& D team and their potential partners, any
result based on Facebook data would be otherwise a result on a researcher's own Facebook friends' data.

While Twitter data can be easy to collect for a knowledgeable user, Twitter in itself is not just any online social network, it is one of a kind. It is a singular sociotechnical object that will not fit in a standard definition, from a media and communication point of view. Kwak, Lee et al. (2011) in particular, have shown that it is both a tool for sociability and information, hence a medium and a social network, and maybe fundamentally neither of the two. From a linguistics perspective, it can be seen as a repository of oral or journalistic language (Rogers 2013), a language of its own kind, or a mix of community 'dialects' (Bryden, Funk and Jansen 2013). Just like the Drosophilia melanogaster, what can be said of Twitter may not be true of any other online social network, nor networking in general. It would then be crucial to researchers to be well aware of the theoretical requirements for any generalisation from Twitter data analysis.

Describing and mapping a specific digital object is usually the role of social sciences and the humanities, and especially media and communication studies. As a new OSN arises, social scientists may want to explore whether the notion of sociability evolves through it, and this is made possible by Twitter since followers/followings and mentions form empirical networks which can actually be retrieved and visualized. In the same way, linguists would study how people talk or write on Twitter and compare those idiosyncrasies to general languages. From an epistemological point of view, this is regular, usual, social science and humanities, or, to phrase it in Kuhnian terms, normal science.

However, new kinds of data analysis based on Twitter have emerged from many other scientific fields related to computer science, machine learning and so on, with a much more systematic media coverage. According to those approaches, it is made possible to predict the success of Kickstarter campaigns (Etter, Grossglauser and Thiran 2013), box-offices (Asur and Huberman 2010) or the stock market (Bollen et al. 2011). With the required conceptualisation from the social sciences and humanities, we might be witnessing the emergence of a new scientific field and new ways to analyse society and language, that we might call ‘computational sociology’ (Hummon and Fararo 1995), ‘computational social science’ (Lazer, Pentland et al. 2009) or ‘social physics’ (Pentland 2014).

While articles claiming they can predict a social phenomenon through Twitter data analysis have been heavily press-covered, what matters to the study of the evolution of science is how much of that kind of work is actually published and how it is considered by other scientists. Computational study of Twitter might be completely anecdotal to less press-covered ‘normal’ social science, but may also be about to replace it. Objectifying and quantifying the balance between computer sciences and social sciences in the study of Twitter is the purpose of this paper.

In this respect, I studied how Twitter is used in academia, what it is aimed for and which theoretical status is conferred to Twitter data. I did so by collecting all scientific papers about Twitter for 6 months (a total of almost 300 papers), and registering for each one the field in which authors work, the subject of the paper, the methods they use and some other things such as the paper source (journal or conference), the researcher's university's country, the
<figure><img src="image_12.png" /><figcaption>Fig. 1: Publication by format.</figcaption></figure>

language in which they write and the publication date (Part 1). Through this bibliometric
study, I thus empirically demonstrate that, in the quantitative picture, research based on
Twitter (or research about Twitter) equally comes from computer sciences on the one hand,
and social sciences and the humanities on the other hand, with a lesser contribution from
natural sciences (Part 2). With that in mind, I discuss what ‘studying Twitter’ actually
means and what signification can be given to Twitter data. In the light of this reflection,
I suggest how the quality of research based on Twitter data analysis might be improved
through actual interdisciplinary collaborations (Part 3).

## 2 Bibliometric data collection

A list of papers was constituted through the keyword request ‘twitter’ using Google Scholar alerting system for six months. Other publication sources such as arXiv or PLOS One are known to be rather focused on certain fields, such as computer science or biology, and would have introduced a bias in my data collection since it was not meant to be focused on a specific field. Also, Google Scholar tends to reference fringe journals and conferences which might be under-represented in well-established scientific repositories. As more innovative approaches tend to appear on the frontiers of science rather than in core journals, the choice of Google Scholar was also a way to include in my set more of the ‘computational social science’ mentioned above. The keyword itself was non-ambiguous and hence usually brought good results, but publications not mentioning Twitter in the title or abstract were not included in the data set because of the method; however, while an empirical proof of that has not been made, I could not find a reason why this sample would not be representative of research about Twitter as a whole.

Data collection took place from July 26th, 2013 to January 27th, 2014. 282 articles were found, including 60% of journal article and 32.5% of conference papers. The remaining 7.5% constituted in Master's theses and pre-prints. [Figure 1] For each article, I manually retrieved the title, authors' names, publication date, university's country, journal or conference name and field, language used, and subject and method(s) used according to the abstract. Coding the scientific field and method was the most delicate task as there is a larger part left to subjectivity and no standard definition of what a field is. For a more accurate coding, I used a three-level typology consisting of 1) field as described by journal's or conference's name,
<figure><img src="image_13.png" /><figcaption>Fig. 2: Publication by language.</figcaption></figure>

author's lab or a similar information source, which was 2) grouped in field families and finally 3) in 'meta fields' which were computer sciences, social sciences and natural sciences. To code the method, I collected techniques and elements of methodology that were explicitly mentioned in the title or abstract with very little rephrasing, hence promoting papers which considered that the method mattered enough to appear in the abstract. While coding the method, I hid the scientific field column of my data set so that I would not be influenced or tempted to align certain methods with certain fields. It is worth mentioning that the distinction between tweet collection and tweet sampling can be difficult to make; as a rule of thumb I considered that, when the abstract suggested that tweets were manually annotated, the authors had probably worked on a sample of maybe hundreds of thousands of tweets rather than a set of millions of tweets. When several methods or algorithms were mentioned, I took them all; when nothing was mentioned, I left the field empty rather than projecting assumptions on a paper. On the whole, less than ten papers did not show any method at all.

## 3 Data analysis

A search on papers related to Twitter cannot be considered representative of academia in general; however, it provides geographical and linguistic information about this fragment of scientific research, which may or may not resemble the whole scholarly world. In our case, English is, by far and as expected, the most used language in the scientific community; other represented languages are Portuguese, Spanish and French. [Figure 2] In terms of country of affiliation, the United States is obviously the most productive; then come European and Asian countries. [Figure 3] The latter seem to be particularly active as far as computer sciences are concerned. This is rather consistent with the geography of science in general.

In terms of scientific fields, media and communication studies are the most represented, followed by social sciences in the broad sense and knowledge engineering. Rather new fields such as digital studies and social network analysis (SNA) also have quite a prominent place. Some less expected matters such as health or education seem to have a solid interest in studying Twitter. [Figure 4] When gathered in meta-fields as described above, subjects are quite well balanced between social sciences (46.1%) and computer sciences (41.8%), what remains being grouped in natural sciences (12.1%). [Figure 5] The latter are mainly
<figure><img src="image_14.png" /><figcaption>Fig. 3: Publication by researcher affiliation country.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 4: Publication by field.</figcaption></figure>

represented by health scientists and professionals wondering how Twitter can be used for a better understanding of the general public, epidemics detection and scientific communication, and physicists trying to apply the mathematical methods from their field to the analysis of Twitter data. In each of those meta fields, the conference/journal ratio can be quite different. Twitter research in social and natural science is much more published in journals, while computer scientists seem to write in journals and conferences in a most balanced way. [Figure 6] Researchers from those three fields informally confirmed to me that journals are more prestigious in natural and social sciences while computer scientists prefer to publish in conference proceedings. A way to interpret this distribution is to consider that, Twitter research being aligned with publication habits, Twitter as a subject is (already) a rather prestigious one.

However, my purpose was not to determine whether Twitter is studied in A-list journals but how scientific practices have taken a grasp on the subject. To this end, one of the main results of this paper is to provide a weighted list of the methods used and mentioned by researchers in their attempts to explore this rather new subject. Overall, collecting or sampling are the most frequently noted techniques, which is quite logical since any data analysis such as social network analysis or text mining relies on data acquisition. [Figure 7] More surprisingly, writing an essay or surveying people without collecting tweets seems to have remained quite a frequent way to address such a contemporary question; those
<figure><img src="image_16.png" /><figcaption>Fig. 5: Publication by meta field.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 6</figcaption></figure>

papers are about Twitter but they are not based on actual Twitter data. This is especially true in social sciences since other fields scarcely resort to those classical methods. [Figure 8] Social scientists’ approach can be quantitative, relying on a statistical analysis of some variables related to Twitter, but they are hardly computational, with the exception of social network analysis, which has become a prominent tool in the social sciences since Moreno’s seminal (and non-computational) work in 1934. Social network analysis (SNA) is also quite noticeable in computer sciences, amongst other text and data mining techniques such as time series analysis or Latent Dirichlet Allocation. [Figure 9] There seems to be no essays about Twitter written by computer scientists, who rely on data collection and analysis when contributing to academic research. When confronting those techniques to the researchers’ goal as presented in titles and abstracts, we observe that Twitter is basically used to provide data for regular mining tasks such as classification, event detection or sentiment analysis. Natural sciences resemble both other fields, with a strong presence of collection and sampling, but also a long tail of computational techniques [Figure 10].

In a nutshell, we can say as of now that there are two major attitudes towards Twitter in research:

- in social sciences, classical methods are called up to address a new research subject;

- in computer sciences, more modern techniques are applied on Twitter data to attain classical goals.
<figure><img src="image_18.png" /><figcaption>Fig. 7: Publication by method (appearing more than once).</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>Fig. 8: Methods in Social Sciences (appearing more than once).</figcaption></figure>

A closer observation of the publications' meta-data suggests that there are, at best, very few exchanged views between those attitudes, or in other words, very little interdisciplinary work between computer scientists and social scientists. It seems then that the observations of C.P. Snow (1990) regarding the two cultures of natural sciences and the humanities can be played again to describe the (non-) connections between computer and social scientists in the context of Twitter research.

In addition to a methodological comparison of scientific fields, we can adopt a more epistemological view of Twitter research. Through a more qualitative approach, I tracked how authors define the meaning they confer to Twitter data, or in other words, what they believe Twitter data to be a representation of. A selection of those definitions appears on Table 1. What stands out in those definitions is that Twitter data can have several epistemic values, depending on the field, subject and stakes of the paper, namely:

(1) Twitter data can be analysed for a better knowledge of Twitter itself or the way people behave, interact or communicate on Twitter;

(2) Twitter data can serve as any other data from any other source, to fuel the calibration of an algorithm, without consideration of what those data may represent;

(3) Twitter data can be used for a more ambitious purpose, where it is a proxy for real-world phenomena (such as stock markets or elections) as shown in Table 1.
<figure><img src="image_20.png" /><figcaption>Fig. 9: Methods in Computer Sciences (appearing more than once).</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>Fig. 10: Methods in Natural Sciences (appearing more than once).</figcaption></figure>

Several issues emerge from this typology. First, none of those points are unquestionable and each of them should probably be discussed independently within each affected field. Second, not all papers explicitly define which epistemic value they confer to Twitter and it sometimes changes within the paper itself. Lastly, some papers which are not written by social scientists make assumptions on social phenomena from Twitter or the ‘real world’: I recognize here what one may call the ‘hubris of big data’, where someone with computer skills begins to make assumptions belonging to social sciences, claiming for instance that they are able to predict the future through Twitter data (the truth of that claim being a different matter). For that matter, the epistemic values of Twitter as described above are somehow correlated to a field, but there are many exceptions to each case. However, as can logically be inferred from what has just been suggested, we can say with confidence that the ‘hubris of big data’ is more typical of computer sciences than social sciences.

# 4 Improving research driven by Twitter data

Social and computer sciences have shown different agendas in Twitter research, the former trying to understand what Twitter is and how it modifies what sociability is, the latter aiming at better algorithms that would accurately classify information and eventually predict phenomena. What is also happening is that new kinds of knowledge about social phenomena are emerging outside social sciences, anchored in what a social scientist would call the instrumentation of her scientific practice. In other words, computer sciences no longer play
<table><caption>Table 1</caption><thead><tr><td>Quote</td><td>Reference</td></tr></thead><tbody><tr><td>'The recent advent of social networking sites and the data within such sites now allow researchers to empirically validate these theories'</td><td>Cha and Haddadi 2010</td></tr><tr><td>'data from the social media services Foursquare and Twitter often reflects activities of users at real-world venues including restaurants, coffee shops, shopping venues and more.'</td><td>Grinberg et al. 2013</td></tr><tr><td>'The use of information gathered via Twitter messaging may provide a method to overcome some of these difficulties, allowing for the assessment of mood states in a set of users through analysing the contents of their textual communications in real time and on a large scale.'</td><td>Lampos et al. 2013</td></tr><tr><td>'Internet-based activities have demonstrated to be highly reflective of real-world situations.'</td><td>Ostrowski</td></tr><tr><td>'Although the Twitter stream contains much useless chatter, by virtue of the sheer number of tweets, it will still contain enough useful information for tracking or even forecasting behavior when extracted in an appropriate manner.'</td><td>Signorini et al. 2011</td></tr><tr><td>'Since the majority of tweets are conversational in nature, they form a good source for public opinion. As a result of its large, diverse, and growing user base, Twitter has emerged as an important source for online opinion and sentiment indexes.'</td><td>Hassan et al. 2013</td></tr></tbody></table>

an ancillary role for sciences and seem to have turned into something that is neither entirely a science yet nor really a technique any more. Through the case of Twitter research, we are possibly witnessing the emergence of a new scientific paradigm, or at least, a reconfiguration of the interrelation between the tools and the object of a given science.

While this reconfiguration is new indeed in the case of Twitter analysis, and of what we may loosely call ‘data sciences’, this is not the first one as regards the history of sciences and especially social sciences. In natural language processing, a field that was born and gradually freed itself from linguistics, it is a common saying that computer systems dedicated to language analysis improve every time a linguist is eliminated. In the last decade, Aurélien Bénel (2013), a researcher with a computer science background, has been looking for ways to improve interdisciplinary collaborations between engineers and social scientists where computer science is not purely an ancillary way to build a tool for research, but a field of interdisciplinary research and experiment in itself. Grossetti and Boë (2008) studied the history of speech analysis and how engineers grasped the subject by taking the question of speech in the broadest sense outside the field of phonetics and language sciences in general. At first, they were working for linguists but soon began to work with some open-minded linguists such as phonologist René Gsell, and finally became an independent research area. As a matter of fact, Terry Shinn (2008) has argued that this kind of research, neither purely
scientific nor purely academic, called ‘instrument research’, has existed in fact ever since the
birth of modern sciences in the 17th century.

In my view, what happened to linguistics and phonetics in the 20th century is currently
happening to social sciences in a broad sense. As Aurélien Bénel pointed out, social scientists
have better options than simply using computer science as a tool just like digital humanists
mostly did until now. The design and construction of computational tools for the collection
and analysis of digital data can be done through an actual collaboration between computer
and social scientists, without a sponsor-provider relationship and starting from the moment
research goals are set. More precisely, the success of the collaboration should be one of those
goals. I believe that, through this configuration, research based on Twitter data and digital
data as a whole can be improved with an actual conceptualisation of both the object of the
research (the social phenomena) and the tools to reach it and build new knowledge.

5 Conclusion

In this paper I have shown that Twitter has become an equally popular source of data for both
social and computer scientists. While their works are fed from the same source, they pursue
very different objectives, some of them being typical of their field, some more innovative.
On the one hand, social scientists conceptualise Twitter itself and try to understand how it
transforms our definition of sociability; on the other hand, computer scientists design tools
to analyse those data and make assumptions about real-world social phenomena. Twitter
data can therefore have several epistemic values, which are not systematically expressed by
the researchers themselves; all in all, everything points to the emergence of new grounds and
methods to study social phenomena. From my point of view, this evolution looks like what
happened to language sciences in the 20th century, which means that researchers from all
fields have the possibility to take inspiration from the history of science and improve their
scientific practices and results through interdisciplinary instrument research.
Bibliography

[1] Asur, S., & Huberman, B. A. (2010). Predicting the Future with Social Media. In 2010 *IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology*, Vol. 1, pp. 492–499. IEEE.

[2] Bénel, A. (2013). Quelle interdisciplinarité pour les « humanités numériques » ? *Les Cahiers Du Numérique*, 1, 1–23. doi:10.3166/LCN.9.1.25-38

[3] Bollen, J., Mao, H., & Zeng, X. (2011). Twitter mood predicts the stock market. *Journal of Computational Science*, 2(1), 1–8.

[4] Bryden, J., Funk, S., & Jansen, V. (2013). Word usage mirrors community structure in the online social network Twitter. *EPJ Data Science*, 1–9.

[5] Cha, M., & Haddadi, H. (2010). Measuring user influence in twitter: The million follower fallacy. *Proceedings of International AAAI Conference on Weblogs and Social*.

[6] Etter, V., Grossglauser, M., & Thiran, P. (2013). Launch hard or go home! In *Proceedings of the first ACM conference on Online social networks - COSN '13*, pp. 177–182. New York, New York, USA: ACM Press.

[7] Grinberg, N., Naaman, M., Shaw, B., & Lotan, G. (2013). Extracting Diurnal Patterns of Real World Activity from Social Media.

[8] Grossetti, M., & Boë, L.-J. (2008). Sciences humaines et recherche instrumentale: qui instrumente qui? *Revue d'Anthropologie des Connaissances*, 2, 1(1), 97.

[9] Hassan, A., Abbasi, A., & Zeng, D. (2013). Twitter Sentiment Analysis: A Bootstrap Ensemble Framework. *Proceedings of the ASE/IEEE*.

[10] Hummon, N. P., & Fararo, T. J. (1995). The emergence of computational sociology. *The Journal of Mathematical Sociology*, 20(2-3), 79–87.

[11] Lampos, V., Lansdall-Welfare, T., Araya, R., & Cristianini, N. (2013). Analysing Mood Patterns in the United Kingdom through Twitter Content. *arXiv Preprint*, 1–8.

[12] Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabasi, A.-L., Brewer, D., Van Alstyne, M. (2009). Social science. Computational social science. *Science* (New York, N.Y.), 323(5915), 721–3.

[13] Ostrowski, D. A. (n.d.). Identification of Trends in Consumer Behavior through Social Media.

[14] Moreno, J. (1934). *Who shall survive? A new approach to the problem of human interrelations*. Washington, DC: Nervous and Mental Disease Publishing Co.

[15] Pentland A. (2014). *Social Physics: How Good Ideas Spread-The Lessons from a New Science*. Penguin Press.

[16] Shinn, T. & Ragouet, P. (2000). Formes de division du travail scientifique et convergence intellectuelle. La recherche technico-instrumentale. *Revue Française de Sociologie*, 41(3), 447–473.

[17] Signorini, A., Segre, A. M., & Polgreen, P. M. (2011). The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic. *PloS One*, 6(5), e19467.
[18] Snow, C. P. (1990). The Two Cultures. *Leonardo*, 23(2), 169–173.

[19] Statista, “Number of monthly active Facebook users worldwide as of 1st quarter 2015 (in millions)”, May 2015, online: http://www.statista.com/statistics/264810/number-of-monthly-active-facebook-users-worldwide/ (checked on May 19th 2015)

[20] Statista, “Number of monthly active Twitter users worldwide from 1st quarter 2010 to 1st quarter 2015 (in millions)”, May 2015, online: http://www.statista.com/statistics/282087/number-of-monthly-active-twitter-users/ (checked on May 19th 2015)

[21] Tufekci, Z. (2014). Big Questions for Social Media Big Data: Representativeness, Validity and Other Methodological Pitfalls. In *ICWSM '14: Proceedings of the 8th International AAAI Conference on Weblogs and Social Media*.
# Text mining and Twitter to analyze British swearing habits

Michael Gauthier¹, Adrien Guille², Fabien Rico³, and Anthony Deseille⁴

¹ CRTT, University of Lyon 2
Lyon, France

michael.gauthier@univ-lyon2.fr

² ERIC, University of Lyon 2
Lyon, France

adrien.guille@univ-lyon2.fr

³ ERIC, University of Lyon 1
Lyon, France

⁴ University of Lyon 1
Lyon, France

**Abstract.** The way women and men speak and are expected to behave is frequently discussed. For example, women are sometimes described as speaking more than men, and men as swearing more than women. These stereotypes can alter people's expectations concerning the way we should behave. Indeed, if the idea that females generally swear less frequently than males is widespread, women who swear may be perceived as deviant from the norm, and thus be stigmatized. Clearly understanding what is true and what is not in these studies and reports is not an easy task, because there is a considerable amount of differing opinions on the topic. The way swear words are used by women and men is one of those topics which remains vague, but whose stake is great, since swearing is often considered as an act of power and a way of affirming oneself.

This article will introduce the data gathered from a corpus of tweets in order to shed a new light on new ways of analyzing specific sociolinguistic features like gendered uses of swear words on Twitter. Analyzing the linguistic behavior of users of these media can be an interesting way of generating a most contemporary corpus representative of general trends, and computational linguistics can represent a very accurate and powerful method of analyzing the different uses people can make of certain speech patterns. In order to carry out the study, we used several tools taken from both computer science and linguistics. These tools may represent innovative methods to analyze the effect of social parameters on speech patterns displayed in Twitter corpora. Thanks to this data, we analyze both quantitative, and qualitative instances of swear words in the corpus, to see how the linguistic gendered preferences may differ when swearing is used, but just as importantly, we see how comparable they can be. Indeed, very often when dealing with gender in corpus linguistics, small differences tend to be focused on, whereas they are actually minor compared to the similarities. As for every study, the methods used here also have certain limits that we present as well. Without pretending to be representative of interactions other than the computer-mediated ones present in this corpus, we hope that this data can shed an up-to-date
and neutral light on the way women and men use swear words on Twitter, and on the implications these results may have, as well as on new tools researchers can use in various areas of research. We believe that this study can also be useful to computational linguists/sociologists thanks to the methods used to access data not directly available and displayed by users (e.g. the age or the sex).

**Keywords:** Linguistics, swear words, gender

# 1 Introduction

The way women and men speak is a common source of discussions and debates, be it in academic research or in mainstream media. Many social attitudes and linguistic features have generally been attributed either to women or men; swearing is one of those topics traditionally associated with men. As Coates (2004) reported, “*the folklinguistic belief that men swear more than women and use more taboo words is widespread*”, consequently leading to the creation of stigmas preventing women or men from using a certain type of linguistic behavior without being stigmatized. These preconceived ideas also fuel societal stereotypes and may impact people’s standards concerning what is desirable from each gender. Moreover, swearing is often considered as an act of power (see Beers Fägersten, 2012; Murray, 2012; Lakoff, 2004; G. I. Hughes, 2006) and a way of affirming oneself. Thus, the fact that one gender may be perceived as more frequent users of swear words, or on the other hand as swear words eschewers, may have an impact on other qualities related to power we would inherently attribute to one gender or the other, whether or not these differences are real. A certain number of studies have shown that contrary to what has long been widely believed, women do not swear less than men, nor do they use a drastically different register (Baruch & Jenkins, 2007; Hammons, 2012; S. E. Hughes, 1992; Jay, 1992; Baker, 2014; Thelwall, 2008; Coates, 2004). Indeed, some of these surveys have shown that what generally differs between women’s and men’s use of swear words is not the rate at which they are used, but the context in which they are used, as well as the kinds of words women and men use. Some studies seem to indicate that the use of “strong” swear words (see below for a description of what “strong” swearing is) by women is increasing in certain contexts, and especially on social media (Murray, 2012; Thelwall, 2008), and in the United Kingdom. Thelwall predicted that “*a reversal in gender patterns for strong swearing will slowly become more widespread, at least in social network sites*”, and this seemed especially true for younger generations of users (users aged 16-19 in the case of Thelwall). Without pretending that our article would answer this question, this is a good example to illustrate how social media can be one way of highlighting new linguistic features, and try to better understand them. The importance and time we devote to social media sites is growing every year according to a study from Ofcom (see the 2013 Ofcom report), and it concerns people from all age groups, and all socioeconomic backgrounds (Smith & Brewer, 2012). It would also seem that on these media, and especially on Twitter, people tend to swear more than in face to face interactions (Wenbo et al., 2014). This paper will introduce the data gathered from a sample corpus of about one million tweets from about 18,000 users collected through Twitter in order to see
how data such as age and gender can be put in relation with more (socio)linguistic-related
features on Twitter, and how these features can be analyzed. Here, the goal of this pilot
study is more experimental and demonstrative, as we will try to suggest innovative ways of
empirically analyzing gender and language, rather than give definitive answers. In order to
carry out our investigation, we collected tweets localized in the UK, then we analyzed the
corpus with various tools (event detection programs, lemmatizers, named-entity recognition
etc). Information concerning the users (sex, age and location) was retrieved so that we could
draw statistics and study different social, contextual and linguistic uses of swear words.

Thanks to this data, we will analyze both quantitative, and qualitative instances of swear
words in the corpus, to see how linguistic gendered preferences may differ when we swear,
but just as importantly, we will see how comparable they can be. As Baker (2014) pointed
out, in a lot of studies dealing with gender in corpus linguistics, small differences often tend
to be focused on, whereas they remain minor compared to the similarities, thus giving the
erroneous idea that these represent proofs of an inherent divergence between sexes. To try
to avoid that, this paper will try to remain as neutral as possible, to see whether a clear
distinction can be made out of this corpus or not. As for every study, the methods used here
have certain limits that we will discuss here as well.

## 2 Corpus description

When carrying out a linguistic study of any kind, the researcher needs a corpus on which
they will base their analyses. The way this corpus is collected will directly impact the kind of
data gathered, as well as the implications, limitations, and the reliability of this data. Thus,
the corpus collection phase has to be carefully thought to have results matching the different
aspects the researcher wants to investigate.

### 2.1 What is swearing?

As mentioned in the introduction, certain studies seem to indicate that strong swearing
among women may be becoming more and more common on social media. However, being
able to determine what can be considered a swear word, and whether it is offensive or not is
not an easy task, as not everyone has the same latitude regarding swearing. Indeed, children
who swear will sometimes be severely reprimanded, whereas it may go unnoticed among
adults (Ladegaard, 2004). Also, people's own perceptions of swear words may influence
how offended they are by them (Jay, 1992; Stapleton, 2010), and thus not everyone will be
offended by the same words. The way swearing is perceived varies a lot between generations
for example (Harris, 1990), and some people may not even consider that certain words are
swear words, whereas others will (S. E. Hughes, 1992). For these reasons, it is hard to define
a clear and empirical list of swear words on which everyone will agree. In order to compile a
list of swear words that would be as objective and appropriate to our sample as possible, we
needed to have a standard, a list of words considered swear words by most British speakers.
In this regard, the study from Wenbo et al. (2014) seemed to be a good start, as they have
managed to put together a list of 788 English swear words and their variations. These words
were manually and independently annotated by two native speakers of English who both agreed that these words are “mostly used for cursing”. We decided to use the Wenbo et al. (2014) study as a standard on which we would base certain aspects of the methodology and analyses of our pilot study, because we deemed that their study can be considered as a reference point for our own investigation partly due to the fact that their research was carried out in 2014, so it is to this day one of the most contemporary on this topic. It is also very exhaustive, as their corpus is composed of 51M English tweets from all around the world, so their results are more likely to be representative of global trends on Twitter. One of the conclusions they came to is that of the 788 words they used to define swearing tweets, “*the top seven swear words - fuck, shit, ass, bitch, nigga, hell and whole cover 90.40% of all the curse word occurrences*” in their corpus. These seven words alone then represent the vast majority of the swear words repertoire of Twitter users in their sample.

Considering the afore-mentioned fact, and in order to be able to maximize the relevance of what we can consider as a swearing tweet in our sample, we chose to include the 20 most common swear words in the Wenbo et al. study. This alone should ensure a reliable representativeness of what can be used to differentiate a swearing tweet from a non-swearing one. However, it could be argued that this list may not necessarily be representative of a majority of native speakers of English, especially as it was sampled by only two native speakers. We mentioned earlier the fact that people's attitudes regarding swear words tend to vary a lot, so relying on two people only seems limited to be able to build a comprehensive list of swear words, especially when considering that for the sake of our study, we wished to focus on the UK only. Thus, what was significant in Wenbo et al.'s study may not be as significant in our own sample, as their study was based on a sample of the worldwide stream of tweets from a given period, whereas our corpus is much more localized, so there may also exist a geographical bias. In order to limit those bias as much as possible we also used the swear words mentioned in the editorial guidelines concerning the use of offensive language by the British Broadcasting Corporation (BBC) and which were not present in the list taken from Wenbo et al.. The BBC can be considered as representative of a standard in terms of what should be labelled as a swear word in the UK, especially as this concerns what is acceptable or not from audiences<fnref n="5" />. We deemed that this would represent a reliable addition we could use to create a list of widely accepted offensive words applicable to a British sample. In the end, the complete list of words we used to distinguish swearing tweets from the others is composed of *fuck, shit, ass, bitch, nigga, hell, whore, dick, piss, pussy, slut, tit, fag, damn, cunt, cum, cock, retard, blowjob, wanker, bastard, prick, bollocks, bloody, crap, bugger*. In other words, if a tweet contains any one, or more, of these words, it will be considered as a swearing tweet.

<footnote n="5">⁵ For more details on the guidelines regarding what the BBC considers as offensive language, see: http://www.bbc.co.uk/guidelines/editorialguidelines/advice/offensivelanguage/index.shtml</footnote>
## 2.2 Social media and swearing

It is inadvisable to presume that the speech patterns displayed on social media are accurately representative of trends present in face to face conversations, especially on Twitter, as users are faced with a limit of 140 characters which does not apply in face to face interactions. However, it can be interesting to compare the way people swear on social media to the way they swear in oral contexts to be able to better understand how these two modes of communication can be compared, and how representative of “real life” trends swearing on Twitter can be. As we discussed earlier, the time dedicated to social media sites like Twitter increases every year. In 2013, a study from Ofcom<fnref n="6" /> revealed that in 2012 in the UK, a vast majority of people from all age groups and socio-economic backgrounds used social media. A majority of these people also reported using social media more than once a day, which was not the case in 2011. This illustrates the growing importance that the Internet, and social media in particular, are gaining in our daily lives, consequently increasing the likelihood of daily speech patterns and evolutions of certain linguistic attitudes being present on social media and vice versa. According to Wenbo et al., “one out of 13 tweets contains curse words” (Wenbo et al., 2014). As corpora of tweets can be composed of a virtually unlimited number of tweets, the proportion of potentially interesting swear words to analyze thereby represents a very appealing way of generating data.

As we stated earlier, the way swear words or other linguistic resources are used and perceived by a specific community can have an impact on the way this community is considered. Conversely, the way swear words are used inside a group can be an indication of evolutions in the way this community identifies itself with regards to its status, or its power for example (Beers Fägersten, 2012; Lakoff, 2004; Murray, 2012; G. I. Hughes, 2006). Since women from the United Kingdom seemed to be the most likely to use strong swear words more than men in previous studies (Thelwall, 2008), we figured that studying the use of swear words of British women and men on Twitter may reveal more profound changes in people’s perception and use of swear words, at least in online communities present on Twitter.

## 3 Methodology

The main requirements we had in order to be able to carry out our study were thus the age of the informants (as younger generations of women seemed to be the most likely to experience this increase in swearing on social media sites), their gender, and they had to be localized in the UK, as this region seemed to be the most sensitive to the aforementioned phenomenon. Twitter’s API (Application Programming Interface) seemed to be the perfect solution in this regard, as it can offer access to every one of these parameters. In order to collect our corpus, we used the streaming API and only requested tweets from the United Kingdom by mentioning the corresponding geolocation. We let the collection of tweets run between 7 April and 15 May 2015 and got a total number of 961,186 tweets from 18,060 users.

<footnote n="6">⁶ See the Ofcom report on Adults’ media use and attitudes report, 2013.</footnote>
## 3.1 Inferring gender on Twitter

Users' gender is determined thanks to the name they provided. We created two repositories of female and male names given to British babies since the 1950s<fnref n="7" />, and they are composed of a total of about 30,000 gendered names. For every user whose tweet we collect, the program automatically checks whether the name provided is present in one file or the other (i.e., whether the name is present in the male or the female repository), and if it is present in one, and not the other, the user is attributed the corresponding gender. As a way to avoid any bias with ambiguous names (names which can be given both to women or men, like Robin for example), if the name is present in both files, the user is considered as undefined, and is rejected.

## 3.2 Inferring age

The age is determined thanks to the information provided by users in the description of their profiles. We have defined a list of patterns which allows the program to automatically identify a digit sequence in a description that corresponds to the age of the user (e.g. thanks to regular expressions, the program will identify 25 from "I'm 25 yo" for example). In order to maximize the accuracy of our results, we decided to split users according to their gender and age groups. We thus categorized users into six different age groups which will be referred to as described in Table 1.

<table><caption>Table 1: Table of notations</caption><thead><tr><td>Notation</td><td>Meaning</td></tr></thead><tbody><tr><td>C<sup>m</sup><sub>12-18</sub></td><td>Tweets published by males aged 12-18</td></tr><tr><td>C<sup>m</sup><sub>19-30</sub></td><td>Tweets published by males aged 19-30</td></tr><tr><td>C<sup>m</sup><sub>31-45</sub></td><td>Tweets published by males aged 31-45</td></tr><tr><td>C<sup>m</sup><sub>46-60</sub></td><td>Tweets published by males aged 46-60</td></tr><tr><td>C<sup>f</sup><sub>12-18</sub></td><td>Tweets published by females aged 12-18</td></tr><tr><td>C<sup>f</sup><sub>19-30</sub></td><td>Tweets published by females aged 19-30</td></tr><tr><td>C<sup>f</sup><sub>31-45</sub></td><td>Tweets published by females aged 31-45</td></tr><tr><td>C<sup>f</sup><sub>46-60</sub></td><td>Tweets published by females aged 46-60</td></tr></tbody></table>

The reason why we chose those age groups is because as many sociolinguistic studies have shown, people we spend a lot of time with can have an influence on the way we speak, especially among children (Eckert, 2008; Stapleton, 2010; Ladegaard, 2004). Since children spend most of their time at school, with peers of the same age, children of the same educational level are more likely to display similar speech patterns. Thus, until age 18, users

<footnote n="7">⁷ Sources: General Register Office, National Records of Scotland and Office for National Statistics.</footnote>
are classified according to the academic level they are the most likely to belong to in the
United Kingdom.

According to the Office for National Statistics, in 2013 the average age of mothers was 30 in England and Wales<fnref n="8" />, so age 30 will be used as a marker for two age groups. Indeed, studies suggest that parents who have children produce more standard forms than usual and avoid the use of taboo language (Stapleton, 2003; Mercury, 1995), so having babies is likely to influence the linguistic attitudes of people from these generations, hence the need to take it into account in our age classification. These age groups should allow the heterogeneousness of our sub-corpora to be limited as much as possible. Such groups also have the advantage of limiting the interference of problems caused by users who may not keep their profiles up to date for example, and who may claim to be 22 in their descriptions, whereas they would now be 23. The age reported would in this case not be the actual age of the user, but they would still belong to the most appropriate age group.

# 4 Analysis and results

To help us analyze this vast collection of tweets and gain insights into the contexts in which Twitter users swear, we leverage several data analysis tools developed in the field of machine learning. But first, some classic data concerning the demographics of our corpus will help us understand its composition.

## 4.1 Distribution of the number of tweets per gender and age

Table 2 and Figure 1 present basic data about the demographics of our corpus. Unsurprisingly, as shown in Figure 2, there is a huge imbalance in the representation of the different age groups taken into account, with a vast majority of our users reported as being between 12 and 30 years old. This was to be expected and this repartition also corresponds to the most represented age groups on Twitter as a whole.

Also, what a manual verification revealed is that for both the youngest and the oldest age groups (i.e. the 5-11 and the 61-99), most of the users' descriptions do not correspond to actual human users, or are representative of anomalous profiles, like pages dedicated to companies or pets (for the youngest age group), or clearly untrustworthy profiles (for the oldest age group). This is mainly due to the method we used to gather information concerning the age of our users, based on regular expressions, and which does not make a difference between a profile dedicated to a 4 year old dog, and a 4 year old boy, although a 5 year old boy is unlikely to have a Twitter profile. Our regular expressions are meant to look for profiles mentioning a number followed by "years old" in users' descriptions (or variations of "years old", like "yo", as it is a very common way to mention one's age on Twitter), among others, but does not take into account any mention of gender, or of being a human, as this is only processed thanks to the name provided. So, to prevent the potential interference of this dubious data, these age groups are never taken into account in the analyses we make, and are just presented here out of a concern for transparency.

<footnote n="8">⁸ See the 2014 report from the Office for National Statistics.</footnote>
<table><caption>Table 2: Basic corpus properties</caption><thead><tr><th></th><th>Male</th><th>Female</th><th>Total</th></tr></thead><tbody><tr><td># of users</td><td>10313</td><td>7747</td><td>18060</td></tr><tr><td># of tweets</td><td>579864</td><td>381322</td><td>961186</td></tr></tbody></table>

<figure><img src="image_22.png" /><figcaption>Fig. 1: Distribution of the number of tweets per gender and age.</figcaption></figure>

## 4.2 Distribution of the number of tweets per user

Figure 2 plots the distribution of the number of tweets per user on a log-log scale. We note that it follows a power-law, with one user contributing over 3700 tweets in the corpus, while a lot of users contribute to fewer than 100 tweets, thus showing that the interference of potential spam accounts producing a great number of tweets in a short amount of time is very limited.

## 4.3 Proportion of swearing tweets among women and men

In our corpus, 5.8% of the male tweets contained at least one swear word, compared to 4.8% for women. Figures 3 and 4 present the proportion of tweets containing the eleven most common swear words for women and men. However, as percentages of this kind do not provide much information about the specific use of each word, we normalized the frequency of each swear word on one million words for both women and men. The results are presented below in Table 3.
<figure><img src="image_23.png" /><figcaption>Fig. 2: Distribution of the number of tweets per user on a log-log scale.</figcaption></figure>

<figure><img src="image_24.png" /><figcaption>Fig. 3: Most common swear words found in swearing tweets published by male users.</figcaption></figure>

## 4.4 Proportion of swearing tweets by gender per million words

Table 3 presents the proportions of use of all the swear words we took into account for both genders. As we mentioned before, there is an imbalance in the number of male and female users, as well as in the number of tweets for each gender. Thus, raw percentages would have been useless in that they are not comparable in such situations. To be able to efficiently
<figure><img src="image_25.png" /><figcaption>Fig. 4: Most common swear words found in swearing tweets published by female users.</figcaption></figure>

compare the use of swear words by women and men, we calculated the number of instances of each swear word there would be in one million words. This then gives us an objective value on which to base our analyses. For each swear word, we calculated the log-likelihood (*LL*) score, which is based on the null hypothesis that there is no difference between the observed relative frequencies of a given swear word in the two corpora (i.e. female tweets and male tweets). We can reject the null hypothesis at the level of *p* < 0.01 when the *LL* value is greater than 6.63 (Rayson et al., 2004). In that case, we consider this word to be characteristic of men or of women. In Table 3, the three most statistically significant words for women and men are highlighted. These words are, in descending order of significance, *bitch*, *bloody* and *hell* for women, and *cunt*, *tit* and *fuck* for men. It would seem that some of the findings of McEnery (2006) are verified here, as in his study of the use of swear words of women and men on MySpace, he found that *fucking*, *fuck*, *jesus*, *cunt* and *fucker* were more typical of males, and *god*, *bloody*, *pig*, *hell*, *bugger*, *bitch*, *pissed*, *arsed*, *shit* and *piss* were more typical of females. However, although some of the most significant words for women and men in our sample were also significant for the MySpace users, the ranking of those words is different. *Cunt* is now the most significant word for men, and *bitch* the most significant for women, which may suggest an evolution in the gendered preferences of swear words. However, it may also be due to different ways of swearing and topical differences triggered by the two social media in question (MySpace and Twitter).

## 4.5 Average ratio of swearing tweets per day by gender and age

As mentioned earlier, Twitter's API enables us to collect many information along with tweets themselves. The time at which those tweets are published is one of them. Figures 5 and 6
<table><caption>Table 3: Frequency of swear words by gender per million words.</caption><tbody><tr><td><strong>fuck</strong></td><td><strong>1719.21</strong></td><td><strong>2105.07</strong></td><td>Men</td><td><strong>179.15</strong></td></tr><tr><td>shit</td><td>996.36</td><td>1198.84</td><td>Men</td><td>85.96</td></tr><tr><td>ass</td><td>63.48</td><td>63.48</td><td>Neither</td><td>0</td></tr><tr><td><strong>bitch</strong></td><td><strong>262.02</strong></td><td><strong>142.88</strong></td><td>Women</td><td><strong>168.1</strong></td></tr><tr><td>nigga</td><td>10.97</td><td>18.82</td><td>Men</td><td>9.51</td></tr><tr><td><strong>hell</strong></td><td><strong>344.57</strong></td><td><strong>309.89</strong></td><td>Women</td><td><strong>8.54</strong></td></tr><tr><td>whore</td><td>11.23</td><td>10.43</td><td>Neither</td><td>0.14</td></tr><tr><td>dick</td><td>124.34</td><td>133.64</td><td>Neither</td><td>1.54</td></tr><tr><td>piss</td><td>257.84</td><td>253.08</td><td>Neither</td><td>0.21</td></tr><tr><td>pussy</td><td>19.33</td><td>33.7</td><td>Men</td><td>17.92</td></tr><tr><td>slut</td><td>20.11</td><td>15.74</td><td>Neither</td><td>2.49</td></tr><tr><td><strong>tit</strong></td><td><strong>88.03</strong></td><td><strong>195.24</strong></td><td>Men</td><td><strong>187.47</strong></td></tr><tr><td>fag</td><td>18.02</td><td>30.28</td><td>Men</td><td>14.3</td></tr><tr><td>damn</td><td>117.81</td><td>102.84</td><td>Neither</td><td>4.73</td></tr><tr><td><strong>cunt</strong></td><td><strong>94.3</strong></td><td><strong>295.86</strong></td><td>Men</td><td><strong>487.84</strong></td></tr><tr><td>cum</td><td>8.62</td><td>17.79</td><td>Men</td><td>14.7</td></tr><tr><td>cock</td><td>82.02</td><td>93.08</td><td>Neither</td><td>3.22</td></tr><tr><td>retard</td><td>11.23</td><td>26.86</td><td>Men</td><td>29.71</td></tr><tr><td>blowjob</td><td>1.04</td><td>1.88</td><td>Neither</td><td>1.1</td></tr><tr><td>wanker</td><td>21.42</td><td>49.96</td><td>Men</td><td>52.82</td></tr><tr><td>bastard</td><td>68.44</td><td>110.54</td><td>Men</td><td>45.49</td></tr><tr><td>prick</td><td>47.02</td><td>84.36</td><td>Men</td><td>48.8</td></tr><tr><td>bollocks</td><td>14.62</td><td>37.3</td><td>Men</td><td>45.96</td></tr><tr><td>bugger</td><td>18.8</td><td>18.3</td><td>Neither</td><td>0.03</td></tr><tr><td><strong>bloody</strong></td><td><strong>227.01</strong></td><td><strong>173.51</strong></td><td>Women</td><td><strong>33.47</strong></td></tr><tr><td>crap</td><td>100.05</td><td>89.66</td><td>Neither</td><td>2.64</td></tr></tbody></table>

represent the average swearing ratio throughout the day for male and female users aged between 12-18 years old. The global patterns are the same for women and men for both age groups, the highest peak of swearing ratio being located in every case between 2am and 5am. In other words, this period is the one in which the proportion of swearing tweets compared to non-swearing tweets is the greatest.

During the day, the pattern seems to be the same for both genders from both age groups as well, since the swearing ratio for both genders keeps increasing throughout the day. As Wenbo et al. (2014) noted, we notice that the global pattern of swearing tweets corresponds to the standard activity of human life, as users start swearing between 6am and 7am, when people usually wake up, and gradually increases throughout the day. Interestingly, we observe
that there is a downfall in the swearing ratio around dinner time (around 7pm and 8pm), which suggests that people tweet, or swear less at that moment. However the ratio increases drastically after that period. As studies have shown, one of the main functions of swearing is to express strong emotions like anger, joy or sadness (Allan & Burridge, 2006; Jay & Janschewitz, 2008). Considering these interpretations, the fact that the ratio for men is constantly higher than women may suggest that they feel more comfortable expressing these kinds of emotions than women.

Thus, apart from the differences between genders, what these figures suggest is that both women and men have the same attitude regarding swearing throughout the day. Even if the way women and men swear can differ lexically or quantitatively (as shown in Table 3), some aspects of swear words usage are the same, and apparently the way women and men use swear words according to the time of the day is something which is common to both genders in our corpus.

<figure><img src="image_26.png" /><figcaption>Fig. 5: Swearing ratio versus time of day in $C_{12-18}^f$.</figcaption></figure>

## 4.6 Named-entity recognition

Named-entity recognition (NER) enables us to automatically locate specific elements in tweets, more precisely the names of people, organizations or locations. To perform this task, we use a software (Finkel et al., 2005) which implements a NER method that relies on classification rules based on features of the word sequences that constitute tweets.

Table 4 reveals that on average, men use named entities more than women. Also, for both women and men, users tend to mention named entities consistently more as they get older.
<figure><img src="image_27.png" /><figcaption>Fig. 6: Swearing ratio versus time of day in $C_{12-18}^m$.</figcaption></figure>

Figures 7 and 8 present detailed proportions of named entities per gender and age group in swearing tweets. This shows that whatever their age, both women and men majoritarily mention named entities referring to people when swearing. However, what differs is the fact that women from every age groups seem to favor locations over men, who prefer mentioning organizations. This method then highlights the fact that as far as swearing is concerned, context plays a big role. We suggest that these differences may point at gendered differences in the topics women and men focus on, at least when they swear, which may reveal the fact that the pragmatic functions of swear words for women and men of the same age groups may differ. However, more qualitative analyses would be necessary to be able to confirm this hypothesis.

<table><caption>Table 4: Proportion of tweets that contain named entities.</caption><thead><tr><th></th><th>[12; 18]</th><th>[19; 30]</th><th>[31; 45]</th><th>Average</th></tr></thead><tbody><tr><th>Women</th><td>10.28%</td><td>13.41%</td><td>14.60%</td><td>12.76%</td></tr><tr><th>Men</th><td>14.25%</td><td>19.67%</td><td>20.59%</td><td>18.18%</td></tr></tbody></table>
<figure><img src="image_28.png" /><figcaption>Fig. 7: Distribution of the types of named entities in swearing tweets published by men.</figcaption></figure>

<figure><img src="image_29.png" /><figcaption>Fig. 8: Distribution of the types of named entities in swearing tweets published by women.</figcaption></figure>

## 4.7 Event detection

In order to analyze the impact that real world events may have on Twitter discussions, we studied specific reactions on Twitter triggered by the most influential of these real world
events (e.g. the broadcast of a popular TV show, a political event, etc.) for users. We use Mention-Anomaly-Based Event Detection (MABED), a statistical method proposed by Guille & Favre (2015, 2014) for the detection of significant events from tweets. Thanks to this method, we are able to map both macro and micro levels of gendered reactions, as it describes each event it detects with a set of words, a time interval and a score that reflects the magnitude of impact of the event over users. Moreover, it is possible to analyze the tweets associated with these events, to understand their underlying composition, and the way swear words are used in our case for example.

Table 5 and Table 6 present as an example the ten most significant events detected by MABED for women and men aged 19-30. These events are numbered from 1 to 10 in decreasing order of significance. The ‘event’ column presents the keywords recognized as the most representative of the event, and the last column presents the percentage of tweets containing at least one swear word inside each event. Events marked as “spam” were events which were considered as such because one single user posted the same spam tweet very often, thus virtually generating keywords considered by MABED as relevant events. These spammers, though being a minority in our corpus as shown in Figure 1, create a considerable amount of noise for our event detection method and prevent a more accurate analysis of gendered events. It is however interesting to note that in this sample, spammers are twice as more present in the female corpus than in the male one, thus suggesting that spam accounts are more likely to adopt a female name.

<figure><img src="image_30.png" /><figcaption>Fig. 9: Evolution of the number of tweets containing #bbcdebate.</figcaption></figure>

Apart from spams, which have been manually identified as such, what the results from MABED reveal is that generally speaking, male events could be summarized by sports (boxing and soccer) and politics, and female events by media/entertainment (birth of the Royal baby, BGT (Britain's Got Talent)), sports (Grand National) and politics. Generally speaking, we observe that throughout those ten events, men use more swear words than women, and the event with the smallest number of swear words among men still contains
more of these than the event containing the greatest number of swear words among women. The events containing the most and least amount of swear words are highlighted for each gender, and this reveals that both women and men swear more when talking about politics.

What is interesting to notice is that on average, the proportion of swearing tweets in reaction to an event is higher (11.3% for men and 6.53% for women) than in classic interactions on Twitter (5.8% for men and 4.8%). It is also worth mentioning that the only common topic between women and men is the one containing the hashtag #bbcdebate. Figure 9 plots the evolution of the number of gendered tweets containing this hashtag. Though men tweet consistently more about that hashtag than women (but it must be reminded that this graph presents the raw number of tweets, so the gendered imbalance may be explained by the fact that we have more men than women in our corpus), we observe that the two patterns are very similar, and that the events are both detected roughly when the broadcast of the debate starts on television, and gradually decrease after the broadcast is over, as it triggers fewer and fewer reactions. The proportion of swearing tweets inside this common event does not differ much between women and men, which may again suggest that gendered differences in swearing are not triggered by gender alone, but by the context in which swearing occurs. In other words, women and men in the exact same context would not differ much in the linguistic attitudes they display. This would imply that swear words are not so much gendered as contextualized, which would correspond to other studies pointing to the fact that when considering gendered speech patterns, the context of use plays a greater role than gender alone (Eckert, 2008; Bamman et al., 2014; Baker, 2014; Holmes, 1995; Ladegaard, 2004). In our case, further qualitative research would however be needed to confirm or refute that hypothesis.

<table><caption>Table 5: Top 10 most impactful events detected from $C_{19-30}^m$</caption><thead><tr><th>Rank</th><th>Event</th><th>% of swearing tweets</th></tr></thead><tbody><tr><td>1</td><td><i>spam</i></td><td></td></tr><tr><td>2</td><td>fight maywheather he maypac watch pacquiao</td><td>13.00%</td></tr><tr><td>3</td><td>bbcdebate ed farage about natalie milliband</td><td>10.70%</td></tr><tr><td>4</td><td>messi ronaldo best goal ever boateng world lionel what</td><td>10.10%</td></tr><tr><td>5</td><td>snp seats have</td><td>9.60%</td></tr><tr><td><b>6</b></td><td><b>tories labour have more you</b></td><td><b>15.50%</b></td></tr><tr><td>7</td><td><i>spam</i></td><td></td></tr><tr><td>8</td><td>bournemouth league premier next all well play football</td><td>11.70%</td></tr><tr><td><b>9</b></td><td><b>exit polls ge2015 have wrong hope right lib</b></td><td><b>9.40%</b></td></tr><tr><td>10</td><td>seat his lost</td><td>10.40%</td></tr></tbody></table>
<table><caption>Table 6: Top 10 most impactful events detected from $C_{19-30}^f$.</caption><thead><tr><th>Rank</th><th>Event</th><th>% of swearing tweets</th></tr></thead><tbody><tr><td>1</td><td>spam</td><td></td></tr><tr><td>2</td><td>royalbaby princess kate girl charlotte diana baby name</td><td>2.50%</td></tr><tr><td>3</td><td>spam</td><td></td></tr><tr><td>4</td><td>bgt dog antanddec ant me max omg</td><td>6.60%</td></tr><tr><td>5</td><td>spam</td><td></td></tr><tr><td>6</td><td>baby royalbaby girl princess kate</td><td>4.40%</td></tr><tr><td>7</td><td>National grandnational bets</td><td>7.90%</td></tr><tr><td>8</td><td>spam</td><td></td></tr><tr><td>9</td><td>bbcdebate nigel up ed nhs would</td><td>9.20%</td></tr><tr><td>10</td><td>Grand national bets</td><td>8.60%</td></tr></tbody></table>

## 5 Limitations

This study presents certain limits. The first one concerns the way we categorized users according to their age. Though it has some advantages, it is not perfect, as some users will have children before age 30, or will leave school before age 18, so the linguistic patterns potentially influenced by those social phenomena may differ.

Another potential problem is that we did not include hashtags in our swear word detection methods, and hashtags often contain swear words, thus potentially limiting our data in this regard.

A manual verification of the information provided in the description of a lot of users in our sample reveals that many are students. Even if it sounds normal, as the most represented age group is the 19-30, there may thus exist a bias towards this category of users.

## 6 Conclusion

In this article, we tried to give hints about new methods which could be used to analyze specific sociolinguistic parameters on Twitter. For that purpose, we analyzed the data of a corpus of about one million tweets from users for whom we could infer both the age and the gender. Even if our data would need to be analyzed more thoroughly and qualitatively in order to draw more generalizable conclusions, our goal here was to show that by combining techniques from both computer science and linguistics, it is possible to provide innovative ways of studying the way women and men swear on Twitter. These tools showed that beyond mere quantitative data which could lead to erroneous impressions and generalizations on the reasons why women and men swear, contextual parameters are sometimes more important in being able to determine what is influential, as we concluded with the event detection and NER analyses. This work is then the continuation of prior studies which showed that gender is often enacted in subtil ways, hence the necessity to develop more tools to explore these
questions. We believe that some of the tools presented here can be used improved in future
research based on Twitter data, so that the analyses presented in this paper can be refined,
especially in order to be more qualitative.
Bibliography

[1] Allan, K., & Burridge, K. (2006). *Forbidden words: Taboo and the censoring of language*. Cambridge University Press.

[2] Baker, P. (2014). *Using corpora to analyze gender*. Bloomsbury Publishing.

[3] Bamman, D., Eisenstein, J., & Schnoebelen, T. (2014). Gender identity and lexical variation in social media. *Journal of Sociolinguistics*, 18(2), 135–160.

[4] Baruch, Y., & Jenkins, S. (2007). Swearing at work and permissive leadership culture: When anti-social becomes social and incivility is acceptable. *Leadership & Organisation Development Journal*, 28(6), 492–507.

[5] Beers Fägersten, K. (2012). *Who's swearing now? The social aspects of conversational swearing*. Cambridge Scholars Publishing.

[6] Coates, J. (2004). *Women, men and language: a sociolinguistic account of gender differences in language*. Pearson.

[7] Eckert, P. (2008). Variation and the indexical field. *Journal of Sociolinguistics*, 12, 453–476.

[8] Finkel, J. R., Grenager, T., & Manning, C. (2005). Incorporating non-local information into information extraction systems by Gibbs sampling. In *Proceedings of the 43rd annual meeting of the association for computational linguistics* (pp. 363–370).

[9] Guille, A., & Favre, C. (2014). Mention-anomaly-based event detection and tracking in Twitter. In *Proceedings of the IEEE/ACM international conference on advances in social network analysis and mining* (p. 375-382).

[10] Guille, A., & Favre, C. (2015). Event detection, tracking and visualization in Twitter: a mention anomaly based approach. *Social Network Analysis and Mining*, 5(1), 1-18.

[11] Hammons, J. W. (2012). *Wgaf: Swearing, social structure and solidarity in an online community* (Unpublished doctoral dissertation). Ball State University, Indiana.

[12] Harris, R. (1990). Lars porsena revisited. In *The state of the language* (p. 411-421). Faber.

[13] Holmes, J. (1995). *Women, men and politeness*.

[14] Hughes, G. I. (2006). *An encyclopedia of swearing: The social history of oaths, profanity, foul language, and ethnic slurs in the English-speaking world*. Sharpe.

[15] Hughes, S. E. (1992). Expletives of lower working-class women. *Language in Society*, 21(2), 291-303.

[16] Jay, T. (1992). *Cursing in America: a psycholinguistic study of dirty language in the courts, in the movies, in the schoolyards, and on the streets*. John Benjamins.

[17] Jay, T., & Janschewitz, K. (2008). The pragmatics of swearing. *Journal of Politeness Research. Language, Behaviour, Culture*, 4(2), 267–288.

[18] Ladegaard, H. J. (2004). Politeness in young children’s speech: context, peer group influence and pragmatic competence. *Journal of Pragmatics*, 36.

[19] Lakoff, R. (2004). *Language and woman's place*. Harper and Row.

[20] McEnery, T. (2006). *Swearing in English*. Routledge.
[21] Mercury, R. E. (1995). Swearing: A “bad” part of language; a good part of language learning. *TESL Canada Journal*, 13(1), 268–36.

[22] Murray, T. E. (2012). Swearing as a function of gender in the language of Midwestern American college students. In *A cultural approach to interpersonal communication: Essential readings* (pp. 233–241). Blackwell.

[23] Rayson, P., Berridge, D., & Francis, B. (2004). Extending the cochran rule for the comparison of word frequencies between corpora. In *Proceedings of the 7th international conference on statistical analysis of textual data* (p. 926-936).

[24] Smith, A., & Brewer, J. (2012). *Twitter use 2012* (Tech. Rep.). Pew Research Center.

[25] Stapleton, K. (2003). Gender and swearing: A community practice. *Women's and Language, 26*(2).

[26] Stapleton, K. (2010). Swearing. *Interpersonal pragmatics*, 289–305.

[27] Thelwall, M. (2008). Fk yea i swear: Cursing and gender in a corpus of MySpace pages. *Corpora*, 3(1), 83–107.

[28] Wenbo, W., Lu, C., Krishnaprasad, T., & Amit, P. S. (2014). Cursing in English on Twitter. In *Proceedings of the ACM conference on computer supported cooperative work and social computing* (p. 415-425).
# Mapping the structure of the global maker laboratories community through Twitter connections

Massimo Menichinelli

Aalto University

School of Art, Design and Architecture

Department of Media

Media Lab Helsinki

P.O.Box 31000,00076 Aalto, Finland.

massimo.menichinelli@aalto.fi

**Abstract.** The Maker Movement is currently considered an interesting and promising phenomenon with social and economic implications, especially through a series of locally implemented but globally connected laboratories for making. The social structure of such global community as a whole has not been examined yet. This research proposes therefore to analyze the connections of following and being followed among the Twitter accounts of such Maker laboratories, as a proxy of their pattern of trust and influence. The Twitter accounts were manually gathered in Twitter lists with the help of the search functionalities of Twitter, and the data was later accessed and reconstructed with a script based on the NetworkX library. The network was reconstructed from the ego-networks of each account on the examined lists, and accounts which were not part of the lists were removed. The resulting network shows a relevant polarity between roughly Fab Labs on one side and Hackerspaces, Makerspaces and TechShops on the others. Further analysis of the data revealed that the Fab Lab sub-community is much more diversified and less homogeneous than the other part, with further insights on the impact of the organization of such laboratories and their networks. Furthermore, the analysis of some centrality measures showed how few nodes bridge these polarities, while most of the nodes are very close to each other and the Fab Lab network has a more relevant and distributed influence. A possible direction for improving this research along multiple dimensions is proposed as a conclusion.

**Keywords:** Fab Lab, Makerspace, Hackerspace, TechShop, Makers, Social Network Analysis, Twitter, Community

## 1 Introduction

The Maker Movement is currently considered an interesting and promising phenomenon with social and economic implications (Anderson, 2012; Hatch, 2014). The “Maker” term is quite broad, and there are currently many discussions regarding how to clearly identify Makers. Generally, however, Makers are usually defined as a people who are interested in designing, making, repairing physical objects with digital tools, and that have the required skills for such activities and share their projects, experiences or time in local laboratories.
The emergence of Makers has been facilitated by three factors: (1) the growing accessibility of tools for manufacturing physical objects; (2) the evolution of web platforms from just discussion to even design, prototyping and manufacturing; (3) the growing number of local laboratories for individual and collaborative design and making. On one side, the development of manufacturing technologies and especially technologies based on digital fabrication (Gershenfeld, 2005, 2012) has been increasing the quality of manufacturing processes, it has been lowering the cost of such technologies and processes which consequently started to be more widespread available. Many open source software and hardware projects like the RepRap 3D Printer (Sells, Bailard, Smith, Bowyer, & Olliver, 2009) or the Arduino development board (Banzi, 2009) lowered the barriers to creating physical objects regarding the low price, the local availability, the easiness of building and repairing and the easiness of studying and developing projects on top of them. On another side, the emergence of the Web 2.0 or Social Web phenomenon (O'Reilly, 2005) generated multiple experiments of online platform for the emergence of collaborative communities, both non-profit and for-profit, that represented a further development of the first online communities, bringing them to online discussion to online software, hardware and design development, prototyping and manufacturing with platforms like GitHub<fnref n="1" />, Thingiverse<fnref n="2" />, UpVerter<fnref n="3" />, Shapeways<fnref n="4" /> and Ponoko<fnref n="5" /> (Anderson, 2012; Benkler, 2002; Howe, 2006; Tapscott & Williams, 2006). On a third side, the emergence of many different formats of Maker facilities like Hackerspaces, Makerspaces, TechShops and Fab Labs made these technologies and processes locally available in a growing number of localities (Anderson, 2012; Cavalcanti, 2013; Gershenfeld, 2005; Hatch, 2014; Maxigas, 2012; Tweney, 2009). The interactions among these three phenomena are creating the conditions for the emergence of the Open and Distributed Manufacturing scenario, where design and production are re-localized, supply chains redesigned and new business generated, all based on local communities which are connected globally (Bauwens, 2009).

Therefore, it is especially through a series of such local but globally distributed laboratories, that this phenomenon of the Maker Movement (i.e. the people) and of Open and Distributed Manufacturing (i.e. the technologies and the business) is taking place and growing with new members, technologies and businesses. Such laboratories have different names and formats like Hackerspaces, Makerspaces, Fab Labs, TechShops and so on. Each format has a different story, culture, typical business plan and tools for networking among them. Boundaries among these formats are not always well defined, as there are both places that may fit into different formats and Makers who participates in different formats. As a whole, these laboratories constitute the local dimension of the whole Make Movement. Currently, it is widely considered that such laboratories (and the whole Maker Movement) constitute a global community thanks to the use of ICT technologies for communication and coordination. The social dimension of the local community of some specific laboratory has been investigated, but

<footnote n="1">¹ https://github.com/</footnote>

<footnote n="2">² http://www.thingiverse.com/</footnote>

<footnote n="3">³ https://upverter.com/</footnote>

<footnote n="4">⁴ http://www.shapeways.com/</footnote>

<footnote n="5">⁵ https://www.ponoko.com/</footnote>
the global community as whole has not been examined yet. For example, some researchers analyzed the local community that gathers around Fab Lab Amsterdam (Ghalim, 2013; Maldini, 2014); others focused on the national community of Fab Labs, Hackerspaces and Makerspaces in Italy (Menichinelli & Ranellucci, 2015) or the national community of Makers in Italy (Bianchini, Menichinelli, Maffei, Bombardi, & Carosi, 2015). Thus there is no evidence yet that such a movement really constitutes a global community or just single laboratories or patterns of single laboratories and of more coordinated and connected laboratories.

The aim of this research is to shed a first light on the social dimension of the global phenomenon of Maker laboratories. The task of examining such a wide phenomenon can be extremely difficult because of its global and distributed nature, and because each format uses different tools and governance strategies for networking. This research therefore propose to analyze the Twitter accounts of such Maker laboratories, as a proxy of the global phenomenon. Twitter was not designed as a place for community building, but some investigations proved that it could actually give place to communities (Gruzd, Wellman, & Takhteyev, 2011). The goal of this research is of analyzing the patterns of connections among such Twitter accounts, in order to find which kind of social structures they have and which kind of community or sub-communities they form. In order to discover such structures, this research analyzes the connection among the Twitter accounts of such Maker laboratories and their connections of following / being followed, as a proxy of the disposition to communicate with other laboratories and of therefore the influence of laboratories in being listened to. Since such laboratories represents experiments in forms of networked and distributed organizations, such analysis could shed a first light on the role and distribution of influence and organization in such distributed systems.

## 2 Method

This research focused on analyzing the connections of following or being followed among the Twitter accounts of such Maker laboratories as a first step in analyzing such global community. The Twitter accounts were manually gathered with the help of the search functionalities of Twitter, in order for the researcher to be able to select those account that were really related to the context of the research: several Twitter accounts have a name that could suggest their belonging to the global community, but upon a closer examination they showed to just use the same keywords. The accounts that were considered were of laboratories, events or organizations related to the global Maker community, since these entities are strongly linked. The accounts were initially separated in three Twitter lists, but these lists were joined later during the analysis of the data, therefore constituting one single list.

The social network of following or being followed connections among the accounts was built with an ego-network of each account: for each account analyzed, Twitter accounts that followed or were followed by it were gathered. Before analyzing the data, all the Twitter accounts that did not belong to the lists were removed. In this way, only the Twitter accounts of the global Maker laboratories were finally analyzed. The data was downloaded and reconstructed into a graph with a custom Python script, based on the NetworkX library (Hagberg, Schult, & Swart, 2008) that accesses the lists trough the Twitter API (Menichinelli,
2015). The script has been released with a Free Software / Open Source license and its code and development can be accessed, commented and contributed to online on GitHub<fnref n="6" />. The obtained data was later analyzed and visualized with Gephi (Bastian, Heymann, & Jacomy, 2009) with the a Force-Directed layout (Jacomy, Venturini, Heymann & Bastian, 2014) and NetworkX with the Matplotlib and Seaborn Python libraries (Hunter, 2007; Michael Waskom et al., 2014). The resulting network was first analyzed in terms of the different sub-communities, in order to understand its structure, and then in terms of several centrality measures, in order to understand the distribution of influence among the nodes. Before exporting the final data, the Twitter accounts were anonymized; however the awareness of the identity of the Twitter accounts proved to be useful during the analysis, in order to understand the sub-communities present in the global network. The research can be replicated with the same software developed for it (Menichinelli, 2015) and one or more Twitter lists<fnref n="7" />.

# 3 Results

## 3.1 General information

The development of the script and the analysis of the data took place along several months during 2014 and 2015; one intermediary analysis was done on the 8th of January 2015 and the final analysis was done on the 26th of May 2015. The act of following or being followed on Twitter is asymmetric, therefore the collected graph was reconstructed as a directed graph. The full network of accounts who were followed or followed the Twitter accounts of the Maker laboratories consisted of 499,681 nodes and 1,098,373 edges: there were errors with accessing the information of only 10 Twitter accounts, who where most likely inaccessible as a consequence of a privacy settings. The final network consisting only of Twitter accounts of Maker laboratories consists of 946 nodes and 29,821 edges. The density of the network is quite low at 0.033: a common value for large networks. The average path length is 2.755, indicating that nodes are highly connected and information can flow quickly among them. The diameter is 10, indicating that there are 10 connections between the farthest nodes in the network: this is a small path, if we consider that the network has 946 nodes. These values could be considered a first measurement of the size of the global Maker laboratories: it is a difficult task to calculate the number of laboratories since there are several online platforms for mapping them. Furthermore, since most of these laboratories start as bottom-up initiatives, it is difficult for such platforms to keep track of the development of the global Maker laboratories community.

## 3.2 Modularity measure: the structure of the sub-communities

The most relevant feature of the network is how it is subdivided into two main polarities, which are only loosely connected by few nodes. These two polarities consists roughly of

<footnote n="6">⁶ https://github.com/openp2pdesign/Twitter-Lists-SNA-EgoNetworks</footnote>

<footnote n="7">⁷ The raw anonymized data can be downloaded here: http://dfn.link/ktq2</footnote>
Fab Labs on one side and mainly of Hackerspaces, Makerspaces and TechShops on the other side. In order to understand the different sub-communities that constitute such global phenomenon, the network was analyzed with a modularity detection. The measurement was done in in Gephi (Bastian et al., 2009), which uses as specific algorithm for modularity detection (Blondel, Guillaume, Lambiotte, & Lefebvre, 2008) with a resolution parameter for identifying more or less communities (Lambiotte, Delvenne, & Barahona, 2008). At first, the network was analyzed with a resolution of 1.0, and then with a resolution of 0.5, going then deeper into its sub-communities.

In the first case (Table 1), with a resolution of 1.0, 41 communities were found but only three big communities are relevant for their size: Hackerspaces, Makerspaces and TechShops together on the right, and Fab Labs on the left and a separated subset of French Fab Labs on the lower part of the left, pointing out a first subdivision of the Fab Lab community and instead a more homogeneous structure of Hackerspaces, Makerspaces and TechShops (Figure 1). As a whole, Makerspaces and Hackerspaces constitute 53.28% of the network, while Fab Labs are 42.07% of the network.

<table><caption>Table 1: Main sub-communities identified with modularity detection with a resolution of 1.0</caption><thead><tr><td>Size of the sub-community related to the global network</td><td>Sub-community identified</td></tr></thead><tbody><tr><td>53.28%</td><td>Makerspaces and hackerspaces</td></tr><tr><td>37.1%</td><td>Fab Labs</td></tr><tr><td>4.97%</td><td>A subset of French Fab Labs</td></tr><tr><td>&lt; 1.0%</td><td>Several smaller sub-communities</td></tr></tbody></table>

In the second case (Table 2), with a resolution of 0.5, the number of sub-communities rises to 54: on the right, we can clearly distinguish Hackerspaces (in green), Makerspaces (in yellow) and TechShops (in dark red) (Figure 2). Counting with a longer history, Hackerspaces constitute 32.66% of the network, while Makerspaces constitute 16.7% of the network. TechShops, which is the format that mostly related to a franchising of Maker laboratories, constitutes only 1.48% of the network. This is consistent with the common idea that these are connected but somehow different format of laboratories: at this resolution therefore we can identify some common formats. On the other side of the network on the left, at the same resolution, we find many more sub-communities, pointing out how the Fab Lab network is a much more diversified and articulated network of laboratories: a union of sub-communities more than a homogeneous community. While at this resolution we find the main Hackerspaces, Makerspaces and TechShops sub-communities, on the Fab Lab side there is a much more diversified structure. Few sub-communities on the Fab Lab side are related to a country (French, Italy, Spain), others are more mixed, but with a more prominent country (Netherlands, Japan). Finally, more sub-communities are constituted by laboratories from mixed countries.
<figure><img src="image_31.png" /><figcaption>Fig. 1: The main sub-communities found with a resolution of 1.0 (highlighted with a different color).</figcaption></figure>

<table><caption>Table 2: Main sub-communities identified with modularity detection with a resolution of 0.5</caption><thead><tr><td>Size of the sub-community related to the global network</td><td>Sub-community identified</td></tr></thead><tbody><tr><td>32.66%</td><td>Hackerspaces</td></tr><tr><td>16.7%</td><td>Makerspaces</td></tr><tr><td>6.66%</td><td>A subset of French Fab Labs, Hackerspaces, Makerspaces</td></tr><tr><td>6.13%</td><td>Most of Italian Fab Labs</td></tr><tr><td>5.71%</td><td>Mixed Fab Labs from all over the world</td></tr><tr><td>5.6%</td><td>Mixed Fab Labs from all over the world</td></tr><tr><td>3.7%</td><td>Most of Spanish Fab Labs</td></tr><tr><td>3.7%</td><td>Mixed Fab Labs from all over the world</td></tr><tr><td>3.38%</td><td>Mixed Fab Labs from all over the world</td></tr><tr><td>2.11%</td><td>Mixed Fab Labs from all over the world</td></tr><tr><td>2.11%</td><td>Mixed Fab Labs from all over the world, with especially Dutch laboratories</td></tr><tr><td>1.59%</td><td>Mixed Fab Labs from all over the world, with especially Japanese laboratories</td></tr><tr><td>1.59%</td><td>Mixed Fab Labs from all over the world</td></tr><tr><td>1.48%</td><td>TechShops</td></tr><tr><td>1.37%</td><td>Mixed Fab Labs from all over the world</td></tr><tr><td>&lt; 1.0%</td><td>Several smaller sub-communities</td></tr></tbody></table>

## 3.3 Centrality measures: importance and influence in the network

After having identified the structure of the sub-communities in the whole network, several centrality measurements were calculated in order to understand the health and the distribution
<figure><img src="image_32.png" /><figcaption>Fig. 2: The sub-communities found with a resolution of 0.5 (highlighted with a different color).</figcaption></figure>

of influence in all the network and the differences in the sub-communities. Degree centrality was calculated first: the number of incoming and outgoing connections for each node is a first measurement of the importance of each node. Since the analyzed network is a directed graph, in-degree and out-degree were calculated: however the results are quite similar, with only few nodes more prominent on out-degree than in-degree. The average degree is 31.523, with only few nodes have an outstanding degree: the network is quite homogeneous beside these exceptions, and the distribution of degree follows a common power-law distribution (Figure 3). However, such an average degree is quite high, meaning that on average each laboratory is connected to 31 other laboratories. A higher number of nodes with higher degree values can be found in the Fab Lab sub-community compared to the Hackerspaces, Makerspaces and TechShops sub-community (Figure 4). The nodes with a higher degree can be found in the Fab Lab sub-community but at the interface with the sub-community of Hackerspaces, Makerspaces and TechShops.

<figure><img src="image_33.png" /><figcaption>Fig. 3: Distribution of Degree centrality among the Twitter accounts gathered.</figcaption></figure>
<figure><img src="image_34.png" /><figcaption>Fig. 4: Distribution of degree centrality in the network (highlighted with a color and size related to the value).</figcaption></figure>

Betweenness centrality was then measured (Brandes, 2001) in order to understand the nodes who acts as a bridge among the many sub-communities found. Another common power-law distribution was found (Figure 5), but this time with a much higher number of nodes with a very low or close to zero value. Betweenness centrality is less uniformly distributed than degree centrality, with only few nodes in the Fab Lab sub-community (the same nodes noted before at the interface with the other sub-community) and then, with a lower value, few nodes in the Hackerspaces, Makerspaces and TechShops sub-community (Figure 6). Very few nodes act as a bridge among the two main sub-communities: 3-4 nodes on each sub-community interface with the other sub-community. Both main polarities are then connected by very few nodes.

<figure><img src="image_35.png" /><figcaption>Fig. 5: Distribution of Betweenness centrality among the Twitter accounts gathered.</figcaption></figure>
<figure><img src="image_36.png" /><figcaption>Fig. 6: Distribution of Betweenness centrality in the network (highlighted with a color and size related to the value).</figcaption></figure>

In order to understand how close the nodes are compared to all the other nodes in the network, closeness centrality was calculated (Freeman, 1978). Here instead we have a different distribution: there is a peak at zero, and then the distribution is a normal one (Figure 7). Therefore, except for several nodes (21.67% of the whole network) that are far from the other nodes in the network (as they are disconnected regarding in-degree or out-degree), most of the network (78.32% of the whole network) has a more uniform distribution of closeness, showcasing short distances among nodes and therefore a common ability to spread information quickly (Figure 8). Both main polarities are strongly connected, where all nodes can easily communicate with each other and information can spread quickly: most of the laboratories are uniformly closed to each other, as in a real global community.

<figure><img src="image_37.png" /><figcaption>Fig. 7: Distribution of Closeness centrality among the Twitter accounts gathered.</figcaption></figure>
<figure><img src="image_38.png" /><figcaption>Fig. 8: Distribution of Closeness centrality in the network (highlighted with a color and size related to the value).</figcaption></figure>

In order to measure the importance of each node related to its neighbors, Eigenvector centrality and PageRank centrality (Brin & Page, 1998; Page, Brin, Motwani, & Winograd, 1999) (with a probability of 0.85) were calculated. These values measure the influence of each node on its neighbors and the network. Both measurements presents a common power-law distribution (Figure 9-Figure 11). There are differences, however, in the distribution of the measured values in the main sub-communities. Regarding Eigenvector centrality, nodes get more importance according to the importance of the neighbors: in this case the Fab Lab sub-community has clearly almost all the higher values, compared to the other part of the network (Figure 10). Nodes in the Fab Lab sub-community are clearly more influential in the whole Maker laboratories global community than the other laboratories, and this influence takes place with a power-law distribution where nodes with an higher influence are concentrated in the Fab Lab part of the network. Regarding PageRank centrality, it represents the likelihood that a Twitter account will reach a particular Twitter account in the network following the connections among all of the accounts. In this case, the value measured shows a power-law distribution as well but is concentrated only in very few nodes instead, especially in the Hackerspaces, Makerspaces and TechShops sub-community, with few nodes with a lower value but clearly identifiable in the Fab Lab sub-community (Figure 12).

# 4 Conclusions

The data obtained with this research give first insights about the structure and the distribution of influence and trust in the global Maker laboratories community on Twitter. The sub-
<figure><img src="image_39.png" /><figcaption>Fig. 9: Distribution of Eigenvector centrality among the Twitter accounts gathered.</figcaption></figure>

<figure><img src="image_40.png" /><figcaption>Fig. 10: Distribution of Eigenvector centrality in the network (highlighted with a color and size related to the value).</figcaption></figure>

<figure><img src="image_41.png" /><figcaption>Fig. 11: Distribution of PageRank centrality among the Twitter accounts gathered.</figcaption></figure>
<figure><img src="image_42.png" /><figcaption>Fig. 12: Distribution of PageRank centrality in the network (highlighted with a color and size related to the value).</figcaption></figure>

communities obtained give insights about the social structure of the global Maker laboratories community, showing how Hackerspaces, Makerspaces, and TechShops are more homogeneous groups compared to Fab Labs, and how these two polarities are well separated, even if with few connecting nodes. Some sub-communities are geographically located, but more are a mix of laboratories from different places, showing the global nature of the community. The presence of more sub-communities in the Fab Lab part could represent the presence of more groups inside the global community that could grow apart in the future, or it could represent how the global format and community of Fab Labs is growing to a more local level or how it could bridge distant localities together. Except few nodes with high degree and betweenness with a common power-law distribution, closeness is more uniformly distributed for connected nodes. Power-law distributions take place also regarding influence and trust: while Eigenvector centrality is much more concentrated in several nodes in the Fab Lab community, and PageRank centrality is concentrated in very few nodes among Makerspaces and Hackerspaces. Therefore, trust and influence take a different meaning in Fab Labs and in Makerspaces and Hackerspaces, according to how they are measured. The whole network is therefore constituted by several identifiable sub-communities that are organized in two polarities, with few nodes bridging them with a stronger role and trust; however most of the nodes are very close to each other in a strong community. Since each of the format of Maker laboratories follows different design processes, organizational structures and policies, these data could be used in order to further study the impact of such elements on the social structure of a global community, in order to foster cohesion, communication and coordination.

This research represent a first analysis of the global Maker laboratories community on Twitter, as a proxy of the global connections among the laboratories. It focused only on the
connections of following or being followed by Maker laboratories Twitter accounts. Therefore,
no real interactions were analyzed in this research: future research should investigate this
global phenomenon along more multiple directions (Gruzd et al., 2011) in order to understand
if such patterns of connections constitutes a real community and not just a consequence
of activity or interest in Twitter. For example, further research could focus on the online
discussion among Maker laboratories accounts, and also among them and users, and only
among users, in order to understand the difference structure of the community as experienced
by laboratories and as experienced by users.
Bibliography

[1] Anderson, C. (2012). *Makers: The New Industrial Revolution*. Crown Business.

[2] Banzi, M. (2009). Getting Started with Arduino (1st edition). Sebastopol: O'Reilly Media.

[3] Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: An open source software for exploring and manipulating networks. In International AAAI Conference on Weblogs and Social Media (Vol. 2). Retrieved from http://gephi.org/publications/gephi-bastian-feb09.pdf

[4] Bauwens, M. (2009). The Emergence of Open Design and Open Manufacturing. WeMagazine, 02. Retrieved from http://www.we-magazine.net/we-volume-02/

[5] Benkler, Y. (2002). Coase's Penguin, or, Linux and The Nature of the Firm. The Yale Law Journal, 112. Retrieved from http://www.yalelawjournal.org/the-yale-law-journal/content-pages/coase\%27s-penguin,-or,-linux-and-the-nature-of-the-firm/

[6] Bianchini, M., Menichinelli, M., Maffei, S., Bombardi, F., & Carosi, A. (2015). Makers' Inquiry. Un'indagine socioeconomica sui makers italiani e su Make in Italy. Milano: Libraccio Editore. Retrieved from http://makersinquiry.org/

[7] Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008(10), P10008. http://doi.org/10.1088/1742-5468/2008/10/P10008

[8] Brandes, U. (2001). A Faster Algorithm for Betweenness Centrality. Journal of Mathematical Sociology, 25, 163-177.

[9] Brin, S., & Page, L. (1998). The Anatomy of a Large-scale Hypertextual Web Search Engine. In Proceedings of the Seventh International Conference on World Wide Web 7 (pp. 107-117). Amsterdam, The Netherlands, The Netherlands: Elsevier Science Publishers B. V. Retrieved from http://dl.acm.org/citation.cfm?id=297805.297827

[10] Cavalcanti, G. (2013, May 22). Is it a Hackerspace, Makerspace, TechShop, or FabLab? Retrieved May 30, 2015, from http://makezine.com/2013/05/22/the-difference-between-hackerspaces-makerspaces-techshops-and-fablabs/

[11] Freeman, L. C. (1978). Centrality in social networks conceptual clarification. Social Networks, 215.

[12] Gershenfeld, N. (2005). FAB: The Coming Revolution on Your Desktop-From Personal Computers to Personal Fabrication. Basic Books.

[13] Gershenfeld, N. (2012). How to Make Almost Anything: The Digital Fabrication Revolution. Foreign Affairs, 91, 43-57.

[14] Ghalim, A. (2013). Fabbing Practices: An Ethnography in Fab Lab Amsterdam (Master's Thesis). Universiteit van Amsterdam (New Media and Culture Studies), Amsterdam. Retrieved from http://www.scribd.com/doc/127598717/FABBING-PRACTICES-AN-ETHNOGRAPHY-IN-FAB-LAB-AMSTERDAM
[15] Gruzd, A., Wellman, B., & Takhteyev, Y. (2011). Imagining Twitter as an Imagined Community. American Behavioral Scientist, 55(10), 1294-1318. http://doi.org/10.1177/0002764211409378

[16] Hagberg, A. A., Schult, D. A., & Swart, P. J. (2008). Exploring network structure, dynamics, and function using NetworkX. In Proceedings of the 7th Python in Science Conference (SciPy2008) (pp. 11-15). Pasadena, CA USA. Retrieved from http://math.lanl.gov/~hagberg/Publications/hagberg-2008-exploring.shtml

[17] Hatch, M. (2014). The maker movement manifesto. Rules for innovation in the new world of crafters, hackers, and tinkerers. New York: McGraw-Hill Education.

[18] Howe, J. (2006, June). The Rise of Crowdsourcing. Wired, 14(6). Retrieved from http://www.wired.com/wired/archive/14.06/crowds.html

[19] Hunter, J. D. (2007). Matplotlib: A 2D graphics environment. Computing In Science & Engineering, 9(3), 90-95.

[20] Jacomy, M., Venturini, T., Heymann, S., & Bastian, M. (2014). ForceAtlas2, a Continuous Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi Software. PLoS ONE 9(6): e98679. doi: 10.1371/journal.pone.0098679.

[21] Lambiotte, R., Delvenne, J.-C., & Barahona, M. (2008). Laplacian Dynamics and Multiscale Modular Structure in Networks. arXiv:0812.1770 [physics]. Retrieved from http://arxiv.org/abs/0812.1770

[22] Maldini, I. (2014). Digital makers: an ethnographic study of the FabLab Amsterdam users. In A Matter of Design. Making Society through Science and Technology. Retrieved from http://www.stsitalia.org/conferences/ocs/index.php/STSIC/AMD/paper/view/58

[23] Maxigas. (2012). Hacklabs and Hackerspaces. Tracing Two Genealogies. Journal of Peer Production, (2). Retrieved from http://peerproduction.net/issues/issue-2/

[24] Menichinelli, M. (2015). Twitter-Lists-SNA-EgoNetworks: v0.2 stable. http://doi.org/10.5281/zenodo.18209

[25] Menichinelli, M., & Ranellucci, A. (2015). Censimento dei Laboratori di Fabbricazione Digitale in Italia 2014. Roma: Fondazione Make in Italy CDB. Retrieved from http://www.makeinitaly.foundation/censimento-dei-laboratori-fabbricazione-digitale-in-italia/

[26] Michael Waskom, Olga Botvinnik, Paul Hobson, John B. Cole, Yaroslav Halchenko, Stephan Hoyer, & Dan Allan. (2014). seaborn: v0.5.0 (November 2014). http://doi.org/10.5281/zenodo.12710

[27] O'Reilly, T. (2005, September 30). What Is Web 2.0 - O'Reilly Media. Retrieved March 31, 2012, from http://oreilly.com/web2/archive/what-is-web-20.html

[28] Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank citation ranking: Bringing order to the web. Retrieved from http://ilpubs.stanford.edu:8090/422

[29] Sells, E., Bailard, S., Smith, Z., Bowyer, A., & Olliver, V. (2009). RepRap: The Replicating Rapid Prototyper: Maximizing Customizability by Breeding the Means of Production. In F. T. Piller & M. M. Tseng, Handbook of Research in Mass Customization and Personalization (pp. 568-580). World Scientific Publishing Company. Retrieved from http://www.worldscientific.com/doi/abs/10.1142/9789814280280_0028
[30] Tapscott, D., & Williams, A. D. (2006). Wikinomics: How Mass Collaboration Changes Everything. Portfolio Hardcover.

[31] Tweney, D. (2009, March 29). DIY Freaks Flock to “Hacker Spaces” Worldwide. Retrieved March 31, 2012, from http://www.wired.com/gadgetlab/2009/03/hackerspaces/
MapTwitter tribal language(s)

Nelleke Oostdijk and Hans van Halteren

CLS

Radboud University

Nijmegen, Netherlands

{n.oostdijk; hvh}@let.ru.nl

**Abstract.** The highly variable nature of Twitter language use has been amply noted in a wide variety of studies, where quite often it is found to present a problem to existing tools used for the automatic processing of tweets. In this paper we investigate the nature of the variability, with a focus on the syntactic constructions that are used. In a first exploratory study we cluster hashtags in Dutch tweets on the basis of the users contributing to those hashtags. On further investigation of four larger coherent groups of hashtags (related to school, employment, politics, and appreciation), we find that there are clear differences in the overall language use in the various groups, also at the syntactic level. We discuss the clustering, specific language use patterns for the hashtag groups, and the implications for the automatic processing of tweets for research.

**Keywords:** Twitter, tweets, language variation, syntactic structure, hashtag clustering

# 1 Introduction

These days Twitter is widely used for various purposes, from ‘taking the world’s pulse’, i.e. using Twitter data to discover what is going on, what is occupying people’s minds, etc., to gathering all sorts of information which is extremely valuable for example for marketing purposes. So far, however, most linguists appear to shun away from Twitter as a data source, possibly because the amount of data is rather daunting, but surely also because the metadata used in sociolinguistic research is usually missing. Another factor may well be that the language use found in the tweets is extremely diverse: it shows a great deal of variation (there does not appear to be such a thing as one single Twitter language), and it is unclear how it relates to the standard language as we know it and which over the past decades has been the traditional object of study in linguistics. Most studies that have investigated the language used in tweets have focused on vocabulary use and spelling variation. Beyond that, as far as we are aware, there has been no attempt to make for example a systematic inventory of the syntactic constructions that are used.

In this paper we set out on developing an approach that will eventually enable us to develop a rule-based syntactic parser for analyzing Dutch tweets. As a first step, we try to harness the wide variation by identifying the various ‘tribes’, i.e. communities of users
who share an interest in specific topics<fnref n="1" />. Our assumption is here that – as in the standard language – language variation can be attributed to two main parameters: the language user and the situational context in which he/she finds him/herself. Therefore, we cluster the tweets on the basis of the hashtags and the users that are using these hashtags. We expect that where variation in language use occurs, the variation within a cluster will be less than the variation between clusters. Next, a more detailed analysis is made of a few clusters with the aim to establish whether they lend further support to the idea that the variation observed with Twitter data can be attributed to different tribes and to uncover some of the distinctive syntactic features that characterize different tribal languages.

The remainder of this paper is structured as follows: In Section 2 we give a brief overview of related research. Then, in Section 3, we describe our approach in which we cluster the Twitter data using hashtags and users, and select four clusters for further analysis. In Section 4 we first look at what surface statistics can tell us about the saliency of the selected clusters, before proceeding (in Section 5) with the annotation of the data. In Section 6 we present the results and discuss the variation observed *between* and *within* clusters. Section 7 concludes this paper with a summary of the main findings and suggestions for further research.

## 2 Language variation

Research into variation in language use has a long tradition in linguistics. Numerous studies have investigated the nature of the variation and the linguistic and extra-linguistic parameters that can be associated with it. Already in early work by, for example, Gregory (1967) and Crystal & Davy (1969), it was observed that groupings of linguistics features can be correlated with recurrent situational features (cf. Gregory, 1967: 178). These situational features relate to both reasonably permanent characteristics of the user and to the recurrent characteristics of the user's use of the language. The former include the user's individuality, his temporal, geographical and social provenance, and his range of intelligibility<fnref n="2" />. The latter pertain to the purposive role, the medium relationship, and the addressee relationship<fnref n="3" />. While most previous studies before him analyzed linguistic variation in terms of a single parameter, Biber (1988) was among the first to undertake a computerized, corpus-based study using a multi-feature/multi-dimensional approach in which co-occurring linguistic features were identified empirically and interpreted in terms of their situational, functional, and processing constraints. Subsequent studies have since investigated linguistic variation both between and within genres, registers, and text types. Biber's approach has also successfully been applied to variation within texts. To date, however, most linguistic variationist studies involve more traditional data. Only more recently studies have begun to appear that include data from

<footnote n="1">¹ The term ‘tribe’ was taken from an article in the Guardian Datablog (15 March 2013), ‘Twitter users forming tribes with own language, tweet analysis shows’.</footnote>

<footnote n="2">² In linguistics these user-related variants are known as idiolect, temporal dialect, geographical dialect, and standard/sub-standard/non-standard language.</footnote>

<footnote n="3">³ In linguistics these underlie the distinction between the notions of field, mode and tenor of discourse.</footnote>
the social media (e.g. Bamman, Eisenstein, & Schnoebelen, 2014; Puschman, 2010; Imo, 2010; Crystal, 2006).

In the field of natural language processing researchers have massively turned to the social media, as it can provide access to vast amounts of data that can be mined for all sorts of information. Often, however, the data does not reach its full potential as researchers perceive the data to be so noisy that it hinders automatic processing. They struggle with issues of encoding and syntax (in the technical sense), and mostly focus on vocabulary use, spelling, the use of emoticons, hashtags, and URLs. Variants tend to be viewed as erroneous instantiations of some standard form, usually the standard upheld in written edited published texts. Thus, Kaufmann and Kalita (2010) as regards Twitter data observe that “[T]his offers new and exciting opportunities, and there is much useful information that can be learned from meaningful analysis of the data. But the quality of the data is so poor that standard NLP tools are unable to process it.”<fnref n="4" /> According to Kaufmann and Kalita “[T]weets often contain highly irregular syntax and nonstandard use of English”<fnref n="5" />. In NLP, unlike in linguistics, there seem to be relatively few studies so far which consider language used on Twitter as a variety in its own right.

Studies from the field of communication studies at large have been quick to pick up on the potential of social media. However, when it comes to variation, we find that they generally look at variation from a different perspective: they are not so much interested in the linguistic features that co-occur under specific conditions, rather they focus on the communicative setting and the communicative effect of what is being communicated. The vast majority of specialized studies that target computer-mediated communication (e.g. Bieswanger, 2013; Danet, 2010; Tagg, 2009; Danet, 2001) are mostly concerned with orthography and spelling. Some studies focus on medium-specific features., like for example Shapp (2014) who investigates variation in the use of Twitter hashtags.

Community membership showing in language has been demonstrated, for example, in a recent study by Bryden, Funk, & Jansen (2014). They looked at communication on Twitter and found that “the network emerging from user communication can be structured into a hierarchy of communities, and that the frequencies of words used within those communities closely replicate this pattern” (2014:1). In their study, Bryden et al. take word usage as a proxy of variation in language. In the present study we build upon and extend the idea as we investigate the use of syntactic constructions by different communities or ‘tribes’.

## 3 Hashtag clusters

In a first experiment we conducted, we used the TwiNL data collection, which according to the creators contains about 40% of Dutch tweets (Tjong Kim Sang & van den Bosch, 2013). We restricted ourselves to the 478,509 users who posted at least 1,000 tweets over the period January 2011 to June 2013. We then selected the 2,081 hashtags that were used by a minimum of 478 users, i.e. at least one in a thousand. We determined the pointwise mutual

<footnote n="4">⁴ Cited from Kaufmann & Kalita (2010), Section II. Motivation.</footnote>

<footnote n="5">⁵ Cited from Kaufmann & Kalita (2010), Abstract.</footnote>
information (PMI) of these users and hashtags and created a 478,509-dimensional vector for each hashtag in which each element was either the PMI value when that was positive, or 0 otherwise. We then determined a dissimilarity matrix, using 1 minus the cosine of each vector pair as the distance measure, and used this matrix to cluster the hashtags.<fnref n="6" />

Inspection of the output of the automatic clustering revealed that the clustering indeed appears to yield clusters of hashtags that belong to specific topics of discourse. In addition, we see clear influences of user communities. Thus we find two clusters that both have to do with ‘housing’. One cluster is formed by hashtags such as #Tekoop (‘For sale’), #Appartement (‘Apartment’), and #Commercieel (‘Commercial’), which point towards the ‘selling side’ of the housing market. The other cluster is formed by hashtags such as #huren (‘to rent’), #makelaar (‘real estate agent’), and #wonen (‘to live’), which point towards the ‘buying side’. Similarly we see that, in the clustering of hashtags of political parties, there is a first grouping of #CU, #SGP, #GL, #Groenlinks, #D66, #PVV, #CDA, #VVD, #PvdA and #SP, and another of #sp, #groenlinks, #d66, #pvda, #vvd, #pvv and #cda before these go on to form a single larger grouping.

We selected four clusters, related to ‘appreciation’ (cluster A), ‘employment’ (cluster E), ‘politics’ (cluster P), and ‘school’ (cluster S) for further inspection. The clusters were selected on an intuitive basis: rather than making use of some threshold level, for each cluster we established the point up to where the clustered hashtags were judged to be related to a specific topical domain without including too much noise. Obviously, the intuitive judgment about the saliency of a cluster was troubled by the fact that some hashtags are more ambiguous than others. For example, #kots can be used in its literal sense (‘puke’), but it can also be an expression of negative sentiment. The same is true for #blond, which literally means ‘fair-haired’ but is quite frequently used in the sense of ‘dumb’ or ‘not too smart’, often used as a comment on the way someone acts. Yet other tags may be used in a sarcastic fashion, e.g. #bedankt (‘thanks’). Still, we were fairly confident that the clusters we selected comprised sets of hashtags that were sufficiently coherent to serve as a basis for our investigation.

Cluster A which we labelled ‘appreciation’ comprises 74 hashtags and is by far the largest of the four clusters<fnref n="7" />. The binding factor for the hashtags in this cluster seems to be that they all express some kind of sentiment or opinion, such as #gaap (‘yawn’), #oneerlijk (‘unfair’), #onzin (‘nonsense’), #spannend (‘exciting’).

Cluster E (‘employment’) comprises 21 hashtags, all of which seem to refer to things having to do with employment: #vacature (‘vacancy’), #CV (‘CV’), #carriere (‘career’), etc. but also type of education or educational level (#MBO, i.e. medium level professional education).

The 32 hashtags that make up cluster P (‘politics’) refer to political parties (e.g. #CDA, #PvDA, #PVV), elections (#tk2012 and #TK2012 i.e. the 2012 election for the Tweede

<footnote n="6">⁶ Using the function hclust in R (R Development Core Team, 2008).</footnote>

<footnote n="7">⁷ See also Appendix A where a more detailed overview is given of the composition of each of the four clusters.</footnote>
Kamer ‘Dutch parliament’), and political issues (e.g. #pensioen ‘pension’, #crisis ‘crisis’, and #euro ‘euro’).

In cluster S (‘school’) 18 hashtags can be found. These refer to school-related matters and range from subjects taught at school (#ak ‘geography’, #gs ‘history’, #engels ‘English’, #wiskunde (‘maths’), to #huiswerk and #hw (both ‘homework’), #wrts (a much used app for practicing vocabulary) and even social media and devices popular among pupils (#sms, #skype, #ipod).

## 4 Preliminary findings of syntactic variation and similarities

Before embarking on a costly annotation process, we first wanted to measure automatically whether there were any signs that the use of syntactic constructions indeed varied between the clusters. Actually measuring syntactic complexity was impossible, as there is no syntactic parser for Dutch tweets as yet. Instead, we used two word-related measures, that strictly speaking are not about syntactic structure, but that do indirectly point towards the syntactic realization of the tweets. The first measure is the tweet length in tokens. As syntactic possibilities with tweets with short lengths are limited, we can expect a higher syntactic complexity to be correlated with a greater tweet length. The second measure, the average U-score, stems from previous research (van Halteren & Oostdijk, 2015). For every word, the U-score (short for ubiquitous-score) expresses on a scale of 0 to 1 whether the word is restricted to specific topics/domains or can be used equally well with any text. As the words with high U-scores tend to be function words or other words that in the standard language are used to build larger syntactic constructions, a high average U-score over all words in a text (or here sample of tweets) indicates the presence of syntactic structures also observed in the standard language, whereas a low value may indicate highly reduced or otherwise deviating structures. In Figure 1, we plot the tweet length against the average U-score. We see clearly that the user-derived clusters, which we already validated for relation in content, also appear to show similarities in the linguistic composition of the corresponding tweets, suggesting that we can indeed proceed to the annotation of syntactic constructions.

## 5 Annotation

The four clusters were subjected to closer inspection. To this end we selected 20 hashtags (highlighted in Figure 1):

<table><tr><td>Appreciation (A):</td><td>#apart, #au, #blond, #gezellig, #jammer, #lekker, #leuk, #vreemd</td></tr><tr><td>Employment (E):</td><td>#baan, #carriere, #jobs, #productie</td></tr><tr><td>Politics (P):</td><td>#CDA, #crisis, #euro, #Europa</td></tr><tr><td>School (S):</td><td>#ak, #huiswerk, #hw, #wrts</td></tr></table>

The selection was made such that for each cluster it included hashtags spread over the cluster.
<figure><img src="image_43.png" /><figcaption>Fig. 1: Placement of hashtags from the clusters 'school' (green), 'employment' (dark blue), 'politics' (light blue), and 'appreciation' (red) with regards to tweet length and a syntax-related measurement. The hashtags highlighted with a yellow background are those for which a sample has been annotated for syntactic structure.</figcaption></figure>
<table><caption>Table 1: Parse units and single unit tweets per sample. Note that the number of hashtags from which we sampled 100 tweets per hashtag is different for cluster A. There we have 8 hashtags, whereas for each of the other three clusters (E,P and S) there are only 4 hashtags.</caption><thead><tr><th>hashtag</th><th>parse units<br>(N)</th><th>single unit<br>(%)</th><th>hashtag</th><th>parse units<br>(N)</th><th>single unit<br>(%)</th></tr></thead><tbody><tr><td>#apart</td><td>140</td><td>63.0</td><td>#jammer</td><td>160</td><td>59.0</td></tr><tr><td>#au</td><td>151</td><td>64.0</td><td>#lekker</td><td>138</td><td>73.0</td></tr><tr><td>#blond</td><td>191</td><td>48.0</td><td>#leuk</td><td>174</td><td>56.0</td></tr><tr><td>#gezellig</td><td>151</td><td>65.0</td><td>#vreemd</td><td>167</td><td>49.0</td></tr><tr><td><strong>Overall A (800)</strong></td><td>1272</td><td>59.6</td><td></td><td></td><td></td></tr><tr><td>#baan</td><td>164</td><td>69.0</td><td>#jobs</td><td>285</td><td>9.0</td></tr><tr><td>#carriere</td><td>265</td><td>21.0</td><td>#productie</td><td>322</td><td>2.0</td></tr><tr><td><strong>Overall E (400)</strong></td><td>1036</td><td>25.3</td><td></td><td></td><td></td></tr><tr><td>#CDA</td><td>207</td><td>23.0</td><td>#euro</td><td>173</td><td>46.0</td></tr><tr><td>#crisis</td><td>179</td><td>44.0</td><td>#Europa</td><td>183</td><td>48.0</td></tr><tr><td><strong>Overall P (400)</strong></td><td>742</td><td>40.3</td><td></td><td></td><td></td></tr><tr><td>#ak</td><td>119</td><td>86.0</td><td>#hw</td><td>155</td><td>67.0</td></tr><tr><td>#huiswerk</td><td>132</td><td>79.0</td><td>#wrts</td><td>137</td><td>71.0</td></tr><tr><td><strong>Overall S (400)</strong></td><td>543</td><td>75.8</td><td></td><td></td><td></td></tr></tbody></table>

For each hashtag, a sample of 100 tweets was manually annotated. As a first step in the annotation process, tweets were (manually) split into parse units. Table 1 gives an overview of the number of parse units per sample, and for each sample, the percentage of tweets which consist of a single unit.

Next, each parse unit was annotated with information pertaining to its (surface) syntactic structure. Thus in our annotation scheme (cf. Table 2) we distinguished between various syntactic categories, viz. sentences, clauses, and phrases, as well as syntactic relations such as coordination and juxtaposition. Three additional labels were introduced. Two of these were needed to provide for parse units that were clipped or only contained some discourse item (a responsive phrase such as ‘yes’ or ‘indeed’, a greeting such as ‘hi’, thanks, or such like item), while the third one was used to cover any remaining realizations. While annotating the parse units for syntactic information, user names, emoticons, hashtags, and URLs were ignored, unless they were an integral part of the syntactic structure, as for example when a hashtag featured as a word in a sentence or phrase.

We also annotated for the occurrence of dependent clauses. Apart from recording their frequency of occurrence, dependent clauses were labelled for their syntactic function: utterance (UTT), subject (SU), notional subject (NOSU), direct object (OD), adverbial (A), preposi-

<footnote n="8">⁸ The annotation scheme is described in some more detail in Appendix B. There also examples are given of how the labels apply.</footnote>
<table><caption>Table 2: Annotation scheme⁸</caption><thead><tr><td>label</td><td>denotation</td></tr></thead><tbody><tr><td>coordS</td><td>coordinated sentences</td></tr><tr><td>juxtS</td><td>juxtaposed sentences</td></tr><tr><td>S decl</td><td>declarative sentence</td></tr><tr><td>S decl,-su</td><td>declarative sentence with subject missing</td></tr><tr><td>S decl,-su,-aux</td><td>declarative sentence with subject and auxiliary missing</td></tr><tr><td>S decl,-su,-cop</td><td>declarative sentence with subject and copular verb missing</td></tr><tr><td>S decl,-su,-v</td><td>declarative sentence with subject and verb missing</td></tr><tr><td>S decl,-v</td><td>declarative sentence with verb missing</td></tr><tr><td>S decl,-od</td><td>declarative sentence with direct object missing</td></tr><tr><td>S int</td><td>interrogative sentence</td></tr><tr><td>S int,-su,-aux</td><td>interrogative sentence with subject and auxiliary missing</td></tr><tr><td>S int,-cop</td><td>interrogative sentence with copular verb missing</td></tr><tr><td>S decl/int</td><td>declarative or interrogative sentence (undecided)</td></tr><tr><td>S excl</td><td>exclamatory sentence</td></tr><tr><td>S imp</td><td>imperative sentence</td></tr><tr><td>CL</td><td>clause</td></tr><tr><td>NP</td><td>noun phrase</td></tr><tr><td>AJP</td><td>adjective phrase</td></tr><tr><td>AVP</td><td>adverb phrase</td></tr><tr><td>PP</td><td>prepositional phrase</td></tr><tr><td>VP</td><td>verb phrase</td></tr><tr><td>DISC</td><td>discourse item (word or phrase)</td></tr><tr><td>CLIPPED</td><td>clipped uni</td></tr><tr><td>REST</td><td>other (none of the above)</td></tr></tbody></table>

tional complement (PC), noun phrase postmodifier (NPPO), adjective phrase postmodifier (AJPO), or adverb phrase postmodifier (AVPO)<fnref n="9" />.

## 6 Results and discussion

In this section we discuss the nature of the variation that we see in the use of syntactic constructions in our data. We first show that the four clusters are again recognizable by considering all constructions together by way of a principal component analysis (Section 6.1). We then look at some individual constructions, discussing the variation observed both between clusters (Section 6.2) and within clusters (Section 6.3).

But before moving on to the investigation of the clusters, we would like to point out that the observed constructions, except for CLIPPED (7.2%) and some of REST (3.8%), are all part of the inventory of constructions found in the standard language. It is merely their use that is different. However, the constructions are often strung together without using

<footnote n="9">⁹ For examples, see Appendix B.</footnote>
the punctuation that we would expect in written language. This means that, if we are to
build a syntactic parser for Dutch tweets, we will have to either find a way to identify the
individual parse units automatically, or have our grammar describe sequences of parse units.
For standard written texts, the latter option can be expected to lead to major computational
problems, but, given the length restriction on tweets, we think it could be applied in this
case.

## 6.1 Principal component analysis

We performed a principal component analysis (PCA; Pearson, 1901; Hotelling, 1930)<fnref n="10" /> on the thirteen characteristics for each hashtag shown in the biplot in Figure 2. These characteristics are mostly the frequencies for the constructions listed in Table 2. However, we merged some rare classes. The labels S_def_ellip and S_int_ellip cover all types of declarative and interrogative sentences with ellipted elements respectively, and the label phrase covers all phrase types: NP, AJP, AVP, PP and VP. Similarly, we did not distinguish between various types of dependent clauses, but included all in a single characterictic DEPCL. Finally, we added the characteristic NUMUNITS, which represents the number of parse units per 100 tweets, as listed in Table 1.

In the biplot, our selected clusters are again concentrated to some degree in specific areas of the plot. The first component differentiates mostly between the employment cluster (dark blue) and the other three, with the employment direction linked to larger numbers of parse units, realized mostly by phrases, and higher frequencies of clipped tweets, whereas the other three show lower numbers of parse units and more clausal constructions. On the second component, we see a high placement of the politics cluster, showing remarkably high frequencies of interrogative and imperative sentences, and a low placement of the school cluster, with ellipsis as its most prominent characteristic. The appreciation cluster is rather spread out in these two dimensions, but clearly on the left side with politics and school.<fnref n="11" />

## 6.2 Language use and variation between the four clusters

When we look at how the four clusters compare to each other (cf. Table 3 and Figure 3), we
find that

- In three of the four clusters (A, P and S) the declarative sentence is used quite frequently. As shown in Table 3, declarative sentences account for 65.8% (cluster A), 49.9% (cluster

<footnote n="10">¹⁰ A principal component analysis rotates the data space in such a way that the variance in the data is shown as much as possible in the earlier dimensions, i.e. the first dimension covers the highest possible proportion of variance, the second the highest possible proportion within the remaining variance, etc. After ordering the dimensions in this way, we can choose to represent the data by a lower number of dimensions while keeping as much information as possible (dimensionality reduction). A plot of the first two dimensions (principal components) is thus assumed to show the most interesting aspects of the data. In such a plot, we can also show where the originally measured characteristics would be placed; this leads to a so-called biplot (e.g. Figure 2).</footnote>

<footnote n="11">¹¹ The appreciation cluster is separated from politics and school only in the third and fourth principal component.</footnote>
<figure><img src="image_44.png" /><figcaption>Fig. 2: Biplot showing the placement of the hashtags and the individual features in relation to the first two principal components. The hashtags are again marked with the cluster colours: 'school' (green), 'employment' (dark blue), 'politics' (light blue), and 'appreciation' (red).</figcaption></figure>

P) and 70.5% (cluster S) of the units. Cluster E with a mere 7.4% of the units realized by a declarative sentence stands out.

- What is striking is that in cluster S the majority of the declarative sentences is elliptic (50.8% vs 19.7%), whereas in cluster P most declarative sentences are full sentences. In cluster A, full and elliptic sentences occur equally frequently (32.9%).

- Interrogative sentences are most frequent in cluster P (10.4%) and S (6.1%).

- Imperative sentences on the whole are rare, except for cluster P where they account for 6.3% of the units.
<table><caption>Table 3: Linguistic syntactic structure: Four clusters. Figures denote the percentage of parse units that were assigned these labels.<fnref n="12" /></caption><thead><tr><th>Cluster</th><th>S(dec)</th><th>S(dec,ellip)</th><th>S(int)</th><th>S(imp)</th><th>phrase</th></tr></thead><tbody><tr><td>Appreciation</td><td>32.9</td><td>32.9</td><td>3.5</td><td>1.3</td><td>10.3</td></tr><tr><td>Employment</td><td>4.5</td><td>2.9</td><td>1.3</td><td>1.4</td><td>67.4</td></tr><tr><td>Politics</td><td>34.5</td><td>15.4</td><td>10.4</td><td>6.3</td><td>16.6</td></tr><tr><td>School</td><td>19.7</td><td>50.8</td><td>6.1</td><td>0.6</td><td>9.9</td></tr></tbody></table>

- In cluster E the largest proportion of units by far (67.4%) is realized by a phrase; in none of the other clusters are phrases that frequent.

- To the extent that dependent clauses are used, they mainly occur in clusters A and P, more specifically as direct object, adverbial and noun phrase postmodifier.

<figure><img src="image_45.png" /><figcaption>Fig. 3: Boxplots showing the frequencies of specific constructions (Panes 1 to 6) in each cluster: 'school' (green), 'employment' (dark blue), 'politics' (light blue), and 'appreciation' (red). Note that the vertical axis has a different scale in Panes 1 to 3 (0-100%) from that in Panes 4 to 6 (0-20%).</figcaption></figure>

<footnote n="12">¹² The table only includes the main distinctive features.</footnote>
## 6.3 Language use and variation within each of the clusters

As we saw in Section 6.2, the four clusters are quite different from each other. There is, however, also considerable variation within the clusters (cf. Figure 3). Below we discuss each of the clusters in some more detail.

**Appreciation.** For all of the 8 hashtags we inspected in the appreciation cluster, the most common realization of a unit is the declarative sentence (cf. Table 4). There is, however, quite some variation among these hashtags. In the case of #gezellig and #lekker, elliptic sentences appear to be favored over full declarative sentences (56.7 and 61.6 percent of elliptic sentences respectively), whereas with #blond, #apart and #vreemd the opposite is true: here we find only 12.1, 12.9 and 13.9 percent of elliptic sentences respectively. In this cluster, only for #vreemd do we find quite a large proportion of units that appear to be clipped (7.8%). These are typically found in tweets where news facts are commented upon as in examples [1] and [2].

[1] #vreemd Britse premier Cameron vergeet dochter in pub: De Britse premier David Cameron heeft zijn 8-jarige docht... http://t.co/Ekvx7wME
EN: British prime minister Cameron forgets daughter in pub. The British prime minister David Cameron has [...] his 8-year-old daught...
(hashtag: odd)

[2] #vreemd Vrouw stript voor geld langs kant van de weg: AMSTERDAM - In de Amerikaanse staat Pennsylvania is de 35-... http://t.co/71teCBIa
EN: Woman strips for money by the side of the road: AMSTERDAM — In the American state Pennsylvania the 35-...
(hashtag: odd)

Most tweets in the appreciation cluster are about expressing some sentiment or opinion on some stated fact or observation (hence the frequent use of declaratives). The overall proportion of interrogatives then may seem somewhat surprising. However, as the figures in Table 4 show, interrogatives are especially frequent with #blond. In this cluster typically questions are asked about how things work or to have something clarified, using the ‘being blond’ as an explanation for (some)one’s ignorance (exs. [3]-[4]).

[3] waarom doet me #htc geen internet zonder wi-fi? en hoe zet ik dat goed #blond #durftevragen
EN: why doesn't my #htc have internet without wi-fi? and how can I correct that
(hashtags: not very smart, dare to ask)

[4] Waarom hebben ze het over een 'definitieve prognose'? #tk2012 #blond?
EN: Why are they talking about a 'definitive prognosis'?
(hashtags: parlement2012, not very smart)

In other cases where #blond is found, it is used to comment on someone (quite often the user him- or herself) doing something stupid (exs. [5]-[6]).
<table><caption>Table 4: Linguistic syntactic structure: Appreciation cluster. Figures denote the percentage of parse units that were assigned these labels. In each column the highest and lowest percentages are indicated using bold face type (underlined and italics respectively).</caption><thead><tr><th>Hashtag</th><th>S(dec)</th><th>S(dec,ellip)</th><th>S(int)</th><th>S(imp)</th><th>phrase</th></tr></thead><tbody><tr><td>#apart</td><td>46.4</td><td>12.9</td><td>2.9</td><td>0.0</td><td>10.7</td></tr><tr><td>#au</td><td>35.8</td><td>37.1</td><td>2.6</td><td>0.0</td><td>5.3</td></tr><tr><td>#blond</td><td>39.8</td><td>12.1</td><td>11.5</td><td>3.1</td><td>12.6</td></tr><tr><td>#gezellig</td><td>16.6</td><td>56.7</td><td>1.3</td><td>1.3</td><td>13.3</td></tr><tr><td>#jammer</td><td>33.8</td><td>34.4</td><td>1.3</td><td>0.0</td><td>10.3</td></tr><tr><td>#lekker</td><td>15.2</td><td>61.6</td><td>0.7</td><td>1.4</td><td>11.6</td></tr><tr><td>#leuk</td><td>25.9</td><td>42.5</td><td>1.1</td><td>3.4</td><td>14.9</td></tr><tr><td>#vreemd</td><td>47.3</td><td>13.2</td><td>4.8</td><td>0.6</td><td>4.2</td></tr></tbody></table>

[5] Ik heb me hoesje verkeert er omheen gedaan #blond
EN: I put on the cover the wrong way round
(hashtag: not very smart)

[6] Bedenk me net dat ik vergeten ben uit te checken met de ov-chipkaart. #handig #duurgeintje #blond #gelukkigstaaternietzoveelmeerop
EN: I just realized that I forgot to check out with the Oyster card.
(hastags: smart, costly affair, not very smart, fortunately there was not much money on it any more)

Noteworthy for #blond is also the large proportion (6.8%) of juxtaposed sentences. A typical examples are given in [7] and [8]. With all other hashtags (in this cluster but also in the other clusters) juxtaposed sentences were quite rare.

[7] ik dnek wanneer begint mn liedje nou, staat mn geluid uit ! #blond ?
EN: I'm wondering when will my song start, my sound is switched off !
(hashtag: not very smart)

[8] ik ben bekant bij de sportschool. staat edwin achter me. ik was mn id vergeten. #blond?
EN: I'm almost at the sports school. Edwin turns up behind me. I'd forgotten my ID.
(hashtag: not very smart)

Finally, whereas coordinated sentences are rare in all clusters and hashtags, #apart, #jammer, and #vreemd have exceptionally large proportions of coordinated sentences: 12.1, 10.6, and 12 percent of the units respectively are coordinated sentences.

**Employment.** The tweets in the employment cluster are mostly from professional/commercial users. In this cluster #baan is typically the odd one out: #baan has by far the largest proportion of tweets (69/100) that are composed of a single unit.<fnref n="13" /> Moreover, tweets that occur with #baan stand out in this cluster as they, compared to the other hashtags, contain sentences relatively more frequently (cf. Table 5). Typical examples from #baan are [9] to [11].

<footnote n="13">¹³ Cf. #carriere 21/100, #jobs 9/100, and #productie 2/100 (Table 1).</footnote>