Georgia Institute of Technology
From the SelectedWorks of Diana Hicks

August 23, 2017

The unbearable emptiness of tweeting—Aboutjournal articles
Nicolas Robinson Garcia
Rodrigo Costas
Kimberley Isett
Julia Melkers, Georgia Institute of Technology - Main Campus
Diana Hicks

Available at: https://works.bepress.com/diana_hicks/51/

The Unbearable Emptiness of Tweeting –about journal articles
Nicolas Robinson-Garcia1, Rodrigo Costas2, Kimberley Isett3, Julia Melkers3, and Diana Hicks3
1

INGENIO (CSIC-UPV), Universitat Politècnica de València, Spain
Centre for Science and Technology Studies (CWTS), Leiden University, The Netherlands
3
School of Public Policy, Georgia Institute of Technology, United States
2

Nicolas Robinson-Garcia, Rodrigo Costas, Kimberley Isett, Julia Melkers, and Diana Hicks, The
Unbearable Emptiness of Tweeting – about journal articles, PLOS ONE 12(8):e0183551.https://doi.org/10.1371/journal.pone.0183551.
[This version contains footnotes and uses the Harvard referencing style, neither of which PLOS ONE
accommodates]

Abstract
Enthusiasm for using Twitter as a source of data in the social sciences extends to measuring the impactof research with Twitter data being a key component in the new altmetrics approach. In this paper, we
critically examine tweets containing links to research articles in the field of dentistry to assess the extent
to which tweeting about scientific papers signifies engagement with, attention to, or consumption ofscientific literature. The main goal is to better comprehend the role Twitter plays in scholarly
communication and the potential value of tweet counts as traces of broader engagement with scientific
literature. In particular, the pattern of tweeting to the top ten most tweeted scientific dental articles
and of tweeting by accounts is examined. The ideal that tweeting about scholarly articles represents
curating and informing about state-of-the-art appears not to be realized in practice. We see much
presumably human tweeting almost entirely mechanical and devoid of original thought, no evidence of
conversation, tweets generated by monomania, duplicate tweeting from many accounts undercentralized professional management and tweets generated by bots. Some accounts exemplify the ideal,
but they represent less than 10% of tweets. Therefore, their influence on any conclusions drawn fromtwitter data is swamped by the mechanical nature of the bulk of tweeting behavior. In light of theseresults, we discuss the compatibility of Twitter with the research enterprise as well as some of thefinancial incentives behind these patterns.

Introduction
Twitter, though not the most successful social media company (Economist, 2016), has seen uniquesuccess in enabling several public movements – the Arab spring, the Spanish 15M, Black Lives Matterand various mass expressions of consumer anger over corporate missteps. Such high visibility social
media activity over the past decade combined with younger generations’ social media addiction has left
1

mature professionals worried about being left behind. The professional literature has therefore seen asteady stream of exhortations to join social media accompanied by suggested professional use cases.
The health professions have shown a genuine interest in the use of information technology to improvehealth care (Eysenbach, 2008). So it was not surprising to find doctors exhorting other doctors to join
Twitter (Hawn, 2009). While voicing concerns with maintaining privacy and professionalism on social
media, proponents described largely idyllic scenarios emphasizing Twitter’s potential for aiding careerdevelopment (McGinnigle et al., 2016), connecting with colleagues, building relationships with patients(Alpert & Womble, 2016), enabling virtual journal clubs and scientific conferences (Roupret &Misrai,2015), complementing traditional teaching methods (Nason et al., 2015; Forgie et al., 2013),
critically appraising and reviewing research (Maclean et al., 2013), or all of the above (Roupret & Misrai,2015). Some have argued that Twitter promised to speed knowledge transfer and bridge thecommunication gap between researchers and stakeholders (Grande et al., 2014) leading others tosuggest strategies and tips to improve the dissemination of research findings (Schnitzler et al., 2016),
and to engage readers to follow journals’ Twitter accounts (Cardona-Grau, 2016).
This paper focuses narrowly on one suggested use of Twitter – tweeting about journal articles. Twitter
being touted as “a great source for both disseminating and discovering information” (Fuller & Allen,2016) where “one can share information on research and papers from peers and colleagues that isinteresting or cutting-edge” (Thompson et al., 2015). Across many research communities Twitter has
been seen as a promising channel for increasing visibility and reaching broader audiences interested in
discussing scientific literature (Haustein et al., 2014a; Letierce et al., 2010). Twitter is the predominantsource of altmetric indicators, that is, mentions in social media platforms because it contains the mostmentions of scientific papers (Costas, Zahedi & Wouters, 2015; Robinson-Garcia et al., 2014). The bestcovered literature in Twitter is biomedical (Sugimoto et al., 2016; Thelwall et al., 2013) and social
science (Haustein, Costas & Larivière, 2015). Therefore, Twitter provides a key component of social
media data that analysts hope to use to monitor the spread of scientific information outside academia.
This paper critically examines tweets containing links to research articles in the field of dentistry toassess the extent to which tweeting about scientific papers signifies engagement with, attention to, or
consumption of scientific literature. The main goal is to better assess the proposal that counts of tweets
trace of engagement of broader audiences with scientific literature (Haustein et al., 2016c).

Background
The argument that tweeting about papers is a good idea led naturally to the suggestion of counting
those tweets and using the result as indicators of impact outside the academy (Eysenbach, 2011;
Piwowar & Priem, 2016). The Altmetric Manifesto (Priem et al., 2010) promoted the importance ofsocial media for accelerating the assessment of journal articles. Immediacy tempted many to see Twitter
mentions of scholarly papers as traces of fast-moving conversations about scientific literature:
Twitter citations are also uniquely conversational, reflecting a broader discussion
crossing traditional disciplinary boundaries. Twitter citations are much faster than
traditional citations, with 40% occurring within one week of the cited resource’s
publication. Finally, while Twitter citations are different from traditional citations, ourparticipants suggest that they still represent and transmit scholarly impact. . .
offering faster, broader, and more nuanced metrics of scholarly communication to2

supplement traditional citation analysis. For example, up-to-date metrics including
Twitter citations might augment a tenure or promotion portfolio. (Priem & Costello,
2010, p. 4)
In turn, this advocacy fed back to the professions. The work by Priem and Costello (2010) has been aresource for doctors recommending Twitter use (Choo et al.,2015; Schnitzler et al., 2016).
The vision seemed rapidly to become reality with the creation of Impactstory.org (formerly
TotalImpact), a nonprofit corporation aimed at ‘helping to build a new scholarly reward system thatvalues and encourages web-native scholarship’ (https://www.impactstory.org/about). In parallel,
Altmetric.com was founded by Euan Adie to provide data concerning mentions of research articles insocial and news media (Robinson-Garcia et al., 2014). Providers and advocates of altmetrics haveadmitted that there ‘has been some confusion over what is being claimed’ (Adie, 2014, p. 351). Still,
their commercial interest dictated marketing efforts directed at convincing librarians of the value ofsocial media and the virtues of integrating altmetrics into assessment of scholarship (Konkiel,
Madjarevic & Rees, 2016). Librarians have become an important ally in the promotion of altmetrics, anda gap has opened between what librarians recommend researchers use and what researchers actuallyuse (Kramer, 2017). Twitter and social media in general are now perceived as promising sources ofevidence for assessing research (Wilsdon et al., 2015).
However, early hopes that tweets would be faster versions of citations and so would predict citation
counts have been dashed. Much research has examined the correlation between tweet and citation
counts; a meta-analysis of this work found the correlation to be negligible (Bornmann, 2015; Costas etal., 2015). Haustein and colleagues (2014b) stated that “the high number of tweets did not seem to becaused by their intellectual contribution or scientific quality” and the diversity of motivations for
tweeting a paper made the value of tweeting papers inconclusive.
The lack of knowledge about who tweeted scientific literature (Ke et al., 2017) or the purposes that
motivated them to do so, undermined the potential value of counts of tweets (Zahedi, Costas &Wouters, 2014). Tweets may have originated from publishers or corporations promoting their content,
or from bots, both of which compromised any claim that tweeting of papers indicates attention or
impact (Haustein et al., 2016b). Recognition of the heterogeneity, volatility and inconsistency of tweets
created calls for a more secondary role complementing traditional metrics rather than becoming a true
alternative (Torres-Salinas, Cabezas-Clavijo & Jiménez-Contreras, 2013). Researchers studying the use ofaltmetrics for research evaluation purposes have shown great skepticism as to their reliability andconsistency when used to inform science policy makers and evaluators (Haustein, 2016; Sugimoto, et al.,2016, Wouters & Costas, 2012). The growth in studies using twitter data in research evaluation as wellas methodological cautions mirror broader trends in use of social media as a source of social science
data (Tufekci, 2014; Zimmer & Proferes, 2014).

Data and methods
This work was part of a larger project addressing dissemination channels and clinical information
sources in dentistry in the United States. In 2016, Web of Science included 84 journals in its category
Dentistry, Oral Surgery & Medicine. In addition, PubMed indexed 47 journals that were active and were
published in the United States. References to papers in these journals were downloaded from Web of
Science and PubMed. Both databases provide PubMed IDs and these were used to find tweets about3the papers in the Altmetric.com database housed at CWTS, Leiden University that indexes the full text oftweets that contain a URL linking to a paper. Altmetric.com is one of the main providers of altmetricdata. It tracks mentions and links to scientific literature from a selection of social media platforms, news
media coverage and policy documents. Twitter is the predominant source of social media data in
Altmetric.com (Robinson-Garcia et al., 2014). Altmetric.com uses the Twitter API to collect tweets,
retweets and quoted tweets that contain a direct link to a scholarly article from a publisher that they
track (Altmetric.com, 2016). Altmetric.com converts the URL contained in the tweet into a unique
identifier: PubMed ID, DOI or ArXiv ID.
For each tweet, the text tweeted, account name, and time posted were obtained. The data also flagged
retweets. To this was added the cited journal article’s title and bibliographic reference. Because thelarger project is examining the US dental profession, this Twitter data was restricted to accounts
originating in the US. The final dataset contained 2,202 US based accounts that sent 8,206 tweets about
4,358 dental papers between June 2011 and June 2016. Tweets were counted, as opposed to links
within tweets to papers. So for example, even a tweet containing more than one URL added just one tothe counts. We refer the reader to the data set analyzed (Robinson-Garcia et al., 2017).
In what follows we discuss tweeting about dental papers based on a reading of the tweets, on looking at
the spacing of tweets over time, and on examining the characteristics of the accounts posting thetweets. The analysis is organized into two sections. The first explores tweets about the top 10 most
tweeted papers. The second explores how accounts engage with the content of tweeted papers, inparticular how mechanical or human was the content of an account’s tweets.

Results - Top 10 most tweeted papers
First the pattern of tweeting that brought the top 10 most tweeted scientific dental papers toprominence is explored. There were three patterns: single issue tweeters, professional social media
account management and broader tweeting, though with little original content. None of the patterns
evidenced engagement with the journal article. What follows describes each of these three patterns.

Single issue campaigners
Monomania seemed to underpin high tweet counts 1. The most tweeted paper in this set was:Aminoshariae, A. and Khan, A., 2015. Acetaminophen: old drug, new issues. Journal of
Endodontics, 41(5), pp.588-593.
The paper was tweeted 264 times by US based tweeters. The paper’s altmetric score was 25:“In the top 5% of all research outputs scored by Altmetric.” 2 However, a closer look raised doubts. 193(73%) of the tweets came from one account, @autismepi who included a link to this paper 65 times intweets that read: “Paracetamol research: [URL]” and 33 times in tweets that read: “#Acetaminophen‘may not be considered a safe drug in #pregnancy’-offspring behavioral disorders, hormone disruption[URL]”.

1
2

Monomania - exaggerated or obsessive enthusiasm for or preoccupation with one thing.
Source: https://www.altmetric.com/details/3753910/Twitter, accessed March 14, 2017.

4

A second less extreme account, @jen_in_TX, tweeted this paper 58 times. The accounts retweeted eachother sometimes. The account profiles convey the content of the tweets about this paper: Tylenol may
not be considered a safe drug during pregnancy, which was an accurate conclusion to draw from thispaper. Discarding all but one of the tweets from these two accounts, the paper would be left with 15
tweets.
The same phenomenon was visible in the second most tweeted paper:
Hujoel, P., 2009. Dietary carbohydrates and dental-systemic diseases. Journal of Dental Research,
88(6), pp.490-502.
@AnnChildersMD tweeted this paper 49 times, her tweets were retweeted 19 times. Twice Childers
retweeted retweets of her tweets. Most tweets were the paper title and URL. There were some unique
tweets such as these:
Does it cause tooth decay or gum disease? Avoid [URL] #LCHF
Is your diet good for your teeth and gums? [URL]This began in 2011 and continued through 2016. Although this paper was tweeted 70 times, only 17
accounts were involved.
A third top 10 tweeted paper earned its status thanks to a single-issue campaigner:
Yu, V. et al. 2016. Electronic cigarettes induce DNA strand breaks and cell death independently of
nicotine in cell lines. Oral Oncology, 52, pp.58-65.
But here there was evidence of broader interest given that 34 accounts tweeted about this paper 39 times,
including 30 retweets. The tweeting began a week after the article was posted online, with a tweet andretweet of the title. One month later, on 28 December 2015 the AAAS news service put out a press releaseabout the article that closed with the quote:"Based on the evidence to date," [the investigator] says, "I believe [e-cigarettes] are no better thansmoking regular cigarettes."
The next day retweeting of Clive Bates began. Bates is a British campaigner whose blog – TheCounterfactual - is headlined:
What's the right thing to do? Analytical advocacy – getting beyond the rhetoric of campaigners(www.clivebates.com)
Bates took issue with the press coverage resulting from the press release in a letter to the Telegraphnewspaper, a blog post about the letter, and a five times retweeted tweet:
oddly, the press release didn't mention cig smoke findings. #tooeasilyplayed
Over the next week the title was tweeted/retweeted 5 times, 1 tweet/retweet pair pointed to the fulltext, 1 tweet took credit for funding the study and 5 tweets were about ecig vapor damaging DNA.
On January 5, 2016 the AAAS posted a clarification/correction to the original press release:5Contrary to what was stated or implied in much of the news coverage resulting fromthis news release, the lab experiments did not find that e-cigarette vapor was asharmful to cells as cigarette smoke. In fact, one phase of the experiments, notaddressed in the news release, found that cigarette smoke did in fact kill cells at amuch faster rate. However, because similar cell-damage mechanisms were observed
as the result of both e-vapor and regular cigarette smoke, Dr. Wang-Rodriguez
asserts, based on the evidence from the study, that e-cigarettes are not necessarily ahealthier alternative to smoking regular cigarettes. As stated in the journal paper andthe news release, further research is needed to better understand the actual longterm health effects of e-cigarettes in humans.
https://www.eurekalert.org/pub_releases/2015-12/varc-chs122815.php,
accessed 3.23.2017
Five months later, Clive Bates posted a critique on the PubMed page of the paper and tweeted about his
post. 19 retweets followed.

Social Media Managers
Because tweets tend to appear contemporaneously with the paper’s publication and Twitter launched in
2006, only 6% of papers were published earlier. One of these was tweeted 51 times:
Lamberts, D.M., Wunderlich, R.C. and Caffesse, R.G., 1982. The Effect of Waxed and Unwaxed
Dental Floss on Gingival Health: Part I. Plaque Removal and Gingival Response. Journal of
Periodontology, 53(6), pp.393-396.
The first tweet about this paper was at 7:20pm, April 1, 2016:
Does wax make a difference in the effectiveness of dental floss? Check it out: [URL]
Four hours and 24 minutes later, 48 identical tweets from 41 accounts had accumulated, differing onlyin their shortened URL, meaning they were not retweets.
A similar pattern appeared for another early paper that received 39 tweets, from 39 accounts:
Gerabek, W.E., 1999. The tooth-worm: historical aspects of a popular medical belief. Clinical Oral
Investigations, 3(1), pp.1-6.
38 of the tweets read:You've probably heard of bookworms, but what about tooth worms? [URL]
These tweets appeared sporadically and are probably still appearing.
The most likely explanation for these repeated, identical tweets is that the dentists behind theseaccounts hired the same company to run their Twitter accounts. That this happened was suggested by
the identical recent tweets from accounts that have tweeted the bookworm text. The attraction of
hiring a service to produce a well-maintained stream of relevant content is obvious. The service did notneed to worry much about overlap in followers because dental practices maintain Twitter accounts
mainly to communicate with patients who are geographically localized and so unlikely to follow two
accounts from the same account manager. However, the service did need to be fulfilling promises of6

providing original content, so it was not going to be retweeting. If the going rate for accountmaintenance is $200 per month, then the bookworm tweeting pattern suggests that someone isgrossing at least $93,600 per year from producing social media content for these 39 dentists. 3

Broader tweeting
A pattern more convincingly demonstrating interest in a paper was seen in the 59 tweets from 41 accounts
about:
Burt, B.A., 2002. Fluoridation and social equity. Journal of Public Health Dentistry, 62(4), pp.195200.
The first 14 tweets, on 1/23/15, report not the paper title, but the conclusions, in English and Spanish:
Study: Fluoridation is "the most effective and practical way" to reduce dental disparities [URL]#factsfavorfluoridation
Sporadic tweeting continues until a slightly mutated version is tweeted 29 times on 4/14/16:
Researcher: Fluoridation is "the most effective and practical" way to reduce dental disparities[URL] #OralHealthEquity
A 2007 paper tweeted 54 times by 33 accounts started off slowly. The title of the paper:
Watt, R.G., 2007. From victim blaming to upstream action: tackling the social determinants of oral
health inequalities. Community dentistry and oral epidemiology, 35(1), pp.1-11.
Was tweeted once in 2013 and again in 2015. Then on February 4, 2016, @WelshMountain tweeted theconclusion rather than the title:
Greater focus is needed on the social determinants that shape a child’s or family’s oral health[URL] #Month4Smiles
Five identical tweets followed that day. Sporadic repeats continued until the hashtag was changed to#OralHealthEquity on April 7, 2016 after which the tweet appeared 35 times in English and Spanish from
22 accounts on April 14, 2016. #OralHealthEquity was the only frequently used hashtag in this datasetthat was neither so basic as to be almost a stop word (dentist, dental, teeth) nor used by one accountonly. It was used by many accounts tweeting about four papers. These two cases suggest people
followed this hashtag, as its application to a tweet seemed to result in more retweeting.
Three more papers filled out the top ten most tweeted. One was a paper in the British Dental Journal(BDJ) that was tweeted by the BDJ several times, and each time was retweeted. The other two papers
were tweeted by 25 accounts tweeting 7 and 2 text variants respectively. The notable characteristic ofthese last five cases is the low number of text variants tweeted. To calculate the number of variants
URLs and account names were removed, and text strings were counted as the same if they differed onlyby the addition of a punctuation mark, deletion of characters at the end to accommodate the additionof an account name at the start, addition of something like: “study” at the beginning or Spanishtranslation. Excluding the BDJ paper, four papers were tweeted 176 times, of which 62 were retweets,
3$200 per month figure obtained from aldersocial.com, April 2017, minimum publicly listed package, includes
managing a Facebook, Twitter and Google+ account.

7

leaving 114 notionally original tweets. Yet there are just 16 true text variants among these tweets. The
tweets were generated not from the paper, but from previous tweets referencing the paper, and yet did
not register as retweets. Interpreting more than 16 of these tweets as representing any kind ofengagement with the paper seems quite dubious.

Summary of tweeting of top 10 papers
This examination of the top ten most tweeted papers provided insight into the nature of tweeting about
research articles. Table 1 summarizes the discussion, listing the papers examined, a short reminder ofthe discussion above, the number of citations from Web of Science, number of tweets about the paper,
number of accounts tweeting about each paper, number of text variants represented in the tweets andnumber of tweets that begin with “@”, which possibly represents conversation and is addressed below.
These ten cases accounted for 8.4% of the tweets in this dataset. There were three cases of a single
account repeat tweeting one paper that concerned harm to humans. That such papers attracted single
issue campaigners has been found in other studies (Vainio & Holmberg, 2017). In one case, high levelsof tweeting were driven not by interest in the paper per se, but by interest in a dispute over how thepaper was portrayed in the press. Including this case, three cases were underpinned by determinedsingle-issue campaigners. BDJ self-tweeting and associated retweeting explained another case. Finally,four cases evinced more convincing signs of broader interest in a journal article. Each of these articles
addressed dental public health, and two benefited from association with the popular #oralhealthequity
hashtag. Many accounts tweeted, and the papers both are quite well cited in the research literature.
Nevertheless, engagement with the paper was not indicated by the tweeting because the tweets werelargely identical, even if not retweets. The same phenomenon was more strikingly illustrated in twocases of identical tweeting from a range of accounts, not retweeting. These two papers differed in topic
as well. The rest of the top ten papers were about harm to humans or health inequality. The two othercases were about floss and a historical curiosity and so seem plausibly targeted at dental patients.
These tweets were likely sent by professional social media account management services.

8

Accounts

Tweet variants

2015

9

264

15

71 103

2009
2002

36
42

70
59

17
41

30
4

14
0

#oralhealthequity

2007

159

54

33

3

0

Social media
manager

1982

17

51

44

2

0

2016

12

39

34

13

2

1999

NA

39

39

3

0

Duplicate tweets
Retweets of BDJ
tweets

2010

47

35

25

7

0

2013

6

29

18

13

0

Duplicate tweets

2009

103

28

25

2

0

Explanation
Single-issue
campaigner
Single-issue
campaigner#oralhealthequity

Largely single-issue
campaigner
Social media
manager

Figure 1 illustrates these results. The largest circle represents the number of tweets about these 10
papers – 668. This number decreases to 291 if accounts that tweeted about these 10 papers arecounted. Finally, counting unique texts that is non-duplicate tweets, further decreases the total to 148.

9

Begins with @TweetsAcetaminophen Old Drug, New Issues.
Dietary Carbohydrates and Dental-Systemic
Diseases
Fluoridation and social equity.
From victim blaming to upstream action:
tackling the social determinants of oral healthinequalities
The Effect of Waxed and Unwaxed Dental Flosson Gingival Health: Part I. Plaque Removal and
Gingival Response
Electronic cigarettes induce DNA strand breaks
and cell death independently of nicotine in celllines
The tooth-worm: historical aspects of a popular
medical belief.
Oral Health Literacy among Female Caregivers:
Impact on Oral Health Outcomes in Early
Childhood
Why do GDPs fail to recognise oral cancer? The
argument for an oral cancer checklist
Beyond the DMFT: the human and economic
cost of early childhood caries.

Citations

Paper title

Publication
year

Table 1 Top 10 most tweeted dental papers

Figure 1 Metrics for the group of 10 most tweeted papers

Results - Analysis of Twitter accounts
That single accounts played such a large part in the stories of highly tweeted papers points to theimportance of characterizing tweeting accounts. 2,202 US based accounts tweeted about dental papers
between 2011 and 2016. These accounts posted more than tweets about dental papers, but here only
their tweets about dental papers will be characterized. That bots are a substantial presence on Twitteris well known (Chu et al., 2012; Haustein et al., 2016b), suggesting that it is important to determine thepresence of bots in this data. The analysis focuses on this, characterizing the tweeting pattern ofaccounts as mechanical or human.

What does a tweeting bot look like?
Bots tweeted about dental papers. One bot almost landed a paper in the top 10:

10

Chou, J.et al., 2014. Bioresorbable zinc hydroxyapatite guided bone regeneration membrane forbone regeneration. Clinical Oral Implants Research 27(3), 354-360.
The first red flag was the topic of the paper, which was highly technical in comparison to the cases
examined above. Despite its topic, the paper was tweeted 22 times over two days in 2014 by the account@semantic_bot. All the tweets had the form: “True or false?” followed by one word, then two words orphrases in parentheses then a URL for the paper. For example:
True or false? release(#ZnMem, #Pb) [URL] #Collagen
True or false? release(100 thick, buffer solution) [URL] #Collagen
The first sentence of the results section of the abstract was:The synthesized Zn-MEM (100 μm thick) showed no zinc ions released in the phosphate buffer
solution (PBS) buffer, but zinc was observed under acidic conditions.
Clearly, this was a Twitter bot grabbing words from the abstract to fill out a formula. This bot tweeted 27
dental papers in 192 tweets over the course of 1 month. The account is gone, a sign that either Twitter
deleted it for violation of terms of service or its owner decided to remove it. A second account,@bionavist, tweeted 8 times about dental papers, all were copied @semantic_bot tweets that did notregister as retweets. The account racked up 44.2k tweets resending @semantic_bot tweets for about
three months in 2014 and has not been deleted.
Another type of bot generated text looks almost, but not quite human (Freitas et al.,2015). For example,
this 2014 tweet from @gary_gshafe:
this is a mouth wash that CAN eliminate bad bacteria LIKE stop cavity's type of good thing? Yes it
has bin approve…
The illogical nature of the tweets is mirrored in the word salad provided as a profile for @gary_gschafe:
Futurist - science & technology- Energy-Transportation- Environment- HEALTH- genetics- seeing
Micro-Nano Atomic- Following MR Elon Musk Giving =WORLDA trio of accounts rounded out the set of bots. @MarkWilliams672, @EmilieThompso15 and@AvaWalker473 appeared in the data with the same one retweet. The last tweet from each accountappeared to link to papers but instead linked to a Twitter notice that the link is blocked because it isharmful. The profiles were again word salads:•••

Nerd ✦ Playful learning and Jesus ✦ Outdoor enthusiast ✦ Fishing guru
Psychology and English student ✶ Self-proclaimed foodie ✶ Chocolatier ✶ Happiness
Promoter ✶ ChristianArt photographer | Joyfully at your service | Triathlete | Animal lover | Sailor

Beyond all this, the features that distinguished these accounts and which signaled their bot identity
were the large number of accounts they followed and the high friend/follower ratio (i.e. number they
follow divided by number who follow them). These features figure prominently among accountcharacteristics found to be useful in identifying fake Twitter accounts, in particular accounts sold as fake
followers (Cresci et al., 2015). @Bionavist, @gary_gschafe, @MarkWilliams672, @EmilieThompso15
and @AvaWalker473 each follow between 1450 and 2000 accounts. Their friend to follower ratio
11

ranged from 6 to 13. On both metrics they scored among the highest in the dataset. It is easy andcheap to buy Twitter followers (buyTwitterfollowersreview.org). If you can sell an account 1,500 time orso (more might arouse Twitter’s suspicions), mounting 1,000 fake accounts could earn you $15,000 at
the going rate of $10 per 1,000 followers, more if you dropped customers after a certain amount of timeand resold the follow.
These identified bots accounted for 0.14% of accounts and 2.5% of tweets about dental journal articles.

What does a human tweeting like a bot look like?
Distinguishing humans from bots on Twitter is not easy and is an active area of research (Chu et al.,2012; Cresci et al., 2015; Freitas et al., 2015). It is particularly difficult when examining tweets about
journal articles. Truthy, a website that when given an account name rates its likelihood of being a bot,cannot distinguish bots from humans tweeting about papers (Haustein et al., 2016b), because one
marker of a bot is a high share of tweets containing URLs and this also characterizes tweets about
papers.
Chu et al. (2012) offered several criteria distinguishing humans from bots on twitter: originality,
evidence of intelligence and specificity 4. Most accounts’ tweets about dental papers violated two of Chu
et al.’s criteria for identifying a human because they lack original and intelligent content, for example
only posting retweets, or repeatedly posting duplicate tweets (Chu et al., 2012 p. 814). For example,
38% of accounts, generating 27% of tweets only retweeted. If humans governed these accounts, their
tweeting could easily have been automated, and thus they behaved like a bot. Another 29% ofaccounts, generating 47% of tweets tweeted only paper titles and URLs. There was no original content
in their tweets about dental papers. This result aligns with sentiment analysis of tweets about journal
articles that find nothing, leading the authors to conclude that tweeting is ‘unlikely to give insights into
the reception of articles by readers’ (Thelwall et al., 2013; Friedrich et al., 2015). The title and URL of apaper can be tweeted by clicking on an icon on a paper’s page. Again, this is easily automated andexcessive automation was another of Chu et al.’s criteria for being a bot. Humans behaving like botsthus accounted for 67% of accounts and 74% of tweets about dental papers.
Duplicate tweets, that is about the same paper from one account, were not uncommon or confined to a
few accounts. Not only does this encompass retweets, but also identical tweets that do not register as
retweets. Grabbing the text of a person’s tweet would be an easy way for a bot to avoid gobbledygook.
When people engage in this variety of easily automated behavior, they again behave like bots. At
minimum, they do not engage with the paper which must cast doubt on interpretations of tweet counts
as representing engagement with papers. 216 accounts, or about 10% of the 2,202 examined, tweeteda paper more than once. 52 of these accounts did this for more than one paper.

What does a human tweeting about papers look like?
A human activity that Twitter might facilitate would be conversations about papers, signaled by tweets
beginning with @ and an account name. 5 Of 343 such tweets, 21 were sent in seven minutes by@JeffNedelman. The tweets read:4

Bots tend to be vague. “I like it” as opposed to “I like the large screen”.
Tweets tagged as replies were also identified. Replies overlapped almost completely with tweets beginning withan account name.

5

12

New study suggests erythritol may be breakthjrough [sic] approach approach [sic] in oral health.
[URL]
Each tweet was prefaced with a different account name, which is perhaps explained by the account profile
which states:
Helping large and small companies communicate nutrition, health & wellness science to top-tierinfluencers and national media, for more than 25 years.
Presumably the account names were clients and the tweets were part of the service @JeffNedelman
was providing to each. In a somewhat similar gambit, one morning @autismepi decided to tweet 71
times “@[account name] paracetamol research:” followed by three paper URLs. Excluding the 135
tweets that began with @ and were sent by these two accounts and @AnnChilders, 2.5% of tweets here
could possibly have been somewhat conversational. This very low percentage indicated thatconversation was not driving tweeting about dental papers.
This is not a surprise. The spewing character of much of the tweeting about dental papers bodes ill for
potential human readers. Few things so clearly portend information overload than bot imitating, or bot
generated text. And information overload means potential human readers retreat, developing filteringand avoidance mechanisms to reduce exposure to a manageable level. Bots of course are unaffected,
they can count all night. So the very nature of the tweet flood means nobody is home to enter into
conversation.
Our experience working with Twitter data in other domains suggests that even without the volume,
Twitter would be ill suited to conversing about papers. In forums or blogs, the concept of a thread
applies; a focus is expected. On Twitter, rich conversation invariably signals a veering away from theoriginal tweet with reactions based on how tweeters feel at the time, not exactly stream of
consciousness because that implies pure thought, but stream of judgement, attitude or feeling. The ecigarette case exemplified this in being not about the paper, but rather representing a stream ofendorsements for a judgement passed on the press release about a paper.
In search of the human element in tweeting about dental papers, we looked for accounts tweeting morethan five times that were definitively human in their tweeting pattern. The leading example was@endofactologist, with 637 tweets, at least 120 referencing papers and 98 tweets about dental papers.
Its profile reads: “Endodontic factoids. Get your daily dose”. The account follows 47 and has 312
followers. 6 All the numbers characterizing this account are human scale.
@endofactologist tweeted the conclusions of papers, not the titles. And the topics were quite technical.
For example:••

Endo surgery has better initial success, but ReTx offers a more favorable long-term outcome.
There is a dose-response relationship btw cigarette smoking the risk of RCT.

What was human about this account? First basic facile use of language and syntax removed suspicion of
bot generated tweets. Second, the consistent topic interest suggested purpose and intent behindchoice of articles creating a larger context of the account’s interest in all things endodontic. This
6

Since the minimum purchase of fake followers can be $5, or 500, these followers are not fake.

13

exemplified the ideal of curation Priem and colleagues proposed in the altmetric manifesto (Priem et al.,2010). Tweeting the conclusions is possible after someone has read and comprehended a paper, or atleast the abstract. This capsule summary approach would offer value to readers in providing more thanthey could get by glancing at the titles of recent articles in a journal. Overall, 1% of accounts producing
6% of tweets looked like humans actively engaged with the dental literature on Twitter. 31% of theaccounts, accounting for 18% of tweets tweeted one or just a couple of links to papers. These accounts
may be active dental tweeters, just not consistently interested in journal articles.

Summary of account tweeting behavior
Examining the tweeting concerning dental journal articles from 2,202 U.S based accounts provided
further insight into the nature of tweeting about research articles. Bots are prevalent on Twitter, andbot accounts tweeted about dental papers. But bots only accounted for 2.5% of tweets. The bulk of
tweets about dental papers were sent by accounts seemingly run by people but whose dental journal
article tweeting could be easily automated. Thus, they chose to behave like bots in only retweeting or
tweeting titles and URLs of papers, easily done by clicking on the Twitter icon on a paper’s webpage.
Such tweets lacked the originality that demarcates humans from bots. In contrast, a small percentage oftweets were sent by accounts whose tweets were original, often summarizing the result of a paper.
Such accounts have modest metrics – numbers of tweets, followers, friends – as would be accumulated
by a person not employed full time to tweet. Figure 2 summarizes these findings.
Figure 2 – Share of tweets by account typeDiscussion
A multi-year campaign has sought to convince us that counting the number of tweets about papers hasvalue. Yet, reading tweets about dental journal articles suggested the opposite. This analysis found:obsessive single issue tweeting, duplicate tweeting from many accounts presumably under centralized
professional management, bots, and much presumably human tweeting duplicative, almost entirely
mechanical and devoid of original thought. Advocates of tweeting about the research literature and of14

altmetrics have extoled an ideal of curating and informing about the literature. Some accounts
exemplified this, but they posted less than 10% of tweets about dental papers. Finding these accounts
or seeing their influence on Twitter data about dental papers would be like looking for a needle in ahaystack.
One response to these findings would be to improve tweet counts by filtering out problematic data.
First, the best available algorithms to identify bots and fake follower accounts could be applied toremove such accounts from the data. Analysts of tweeting about papers could develop an open source
list of such accounts to be used by everyone so that all analyses meet the same standards of rigor.
Second, instead of counting tweets, accounts tweeting about a paper could be counted. This would
eliminate the outsized influence of obsessive duplicate tweeting. 7 Altmetric.com does this. The dental
dataset would be reduced by 18% from 8,247 tweets, to 6,784 unique account-paper pairs. Third,
tweets of the title could be excluded, on the grounds that they lack evidence of engagement with thepaper. Finally, if one were concerned to register only evidence of engagement with the paper, uniquetext variants could be counted. This would eliminate retweets, but as our data revealed, retweets
account for only a portion of identical tweets. Cleaning tweets to remove URLs and account names andcounting remaining tweet variants for each paper reduces the dental tweet count by 21% to 6,487.
Unifying further, across minor text variations such as adding “study” or “researchers” to the beginningof a tweet, changing punctuation or varied number of characters dropped from the end of a tweet whenan account name is added to the beginning, would further decrease the unique tweet count.
Rather than using these results to learn how to distill data that aligns with our ideal of what tweeting
about papers should be, a second response to these results would be to focus on the part of the datathat violates our ideal vision. 74% of tweeting about dental papers was produced by people behaving
like bots. If, as one author suggests, “the war for our humanity is upon us” (Philp, 2017, p. 277), dental
tweeting behavior suggests that humanity is losing the war. To understand why people choose to
behave like bots requires attending to the affordances of the medium, or the actions Twitter is designed
to make as easy as possible. Tweeting is an activity created by a company whose value derives fromgrowth in that activity. The platform is designed to lower barriers to tweeting because the easier it is to
tweet, the higher the tweet count and so the higher is the company’s value. Hence the ease of
mechanically retweeting and mechanically clicking on a Twitter icon on a page of a journal article. Thesedesign features encourage quick, non-human-like action that generates tweets that increase measuresof engagement with the platform just as much as a unique, thoughtful, funny piece of text.
In the context of tweeting about journal articles, enrolling people in an algorithmic exercise to maximize
tweet counts contrasts with the deeply human activity of advancing knowledge. Even narrowing the
focus to the documents involved, journal articles attain their value through a rich association of people
working together, expending the resources they have gathered, using equipment and materials,
engaging in conceptual and manual labor, writing, and revising. The document implicates people,
institutions that support them, materials, equipment, theory and past results on which they build therefore all can be identified by reading the journal article. Knowledge advances slowly, and endures.
7

One might imagine that some duplicate tweeting occurred because one account was engaged in several different
discussions about a paper and this would be missed if duplicates were eliminated from the count. However, wefound no evidence of this. Neither were duplicates prefaced with different @ and account names nor did they
have different hashtags, both of which would suggest different discussions.

15

It takes time to identify the truly novel advances and incorporate them into further advances (Wang etal., 2016). References of course are integral to the narrative in a journal article, functioning to support
the argument. By enabling past contributions to be used as resources in building current arguments,
references create links across time. The value of references, and their inversion - citations, derives fromtheir entanglement in this nexus.
In contrast, Twitter’s question is: what’s happening? Twitter excels as a medium for the contemporary,
immediate and ephemeral. Its forte is relating current news and opinion and enabling asymmetricalconnections between individuals and organizations. Its speed favors expressing feelings in the moment
rather than thoughtful reflection. 8 Twitter’s immediacy and speed are mismatched to the slow pace ofknowledge advance. Its thinned out, 140 character missives contrast with the thick and rich texts ofresearch and scholarship. Twitter’s valuing of volume contrasts with the thoughtfulness of high quality
scholarship. The isolation of the tweet and the lack of larger purpose, contrast with the connection
indicated by references that serve the larger purpose of the narrative in which they are embedded. And
yet, altmetrics was premised on tweets being “faster, broader and more nuanced” (Priem & Costello,
2010, p. 4) and so superior to references when in fact tweets and references are incommensurable
because the logic of social media and the institutional logic of the research enterprise differ so radically(van Dijck and Poell, 2013).

Limitations
We acknowledge the limits of this study. First, one discipline was studied, and things might differ inother areas of science or social science. For example, if a major dental school had been an active
tweeter about the research it published, the results would look different, and this could easily happen inanother area of research. Second, the analysis of US accounts might not be representative of tweeting
behavior in other countries. In particular, the British Dental Journal (BDJ) produced by far the most
tweets about dental journal articles and published an article analyzing tweets about dental papers(Kolahi and Khazaei, 2016), perhaps because it is in the same corporate family as Altmetric.com. Third,
tweeting behavior is constantly evolving and our results will date quickly.

Conclusions
This paper examined 8,000 tweets, from 2,000 US-based accounts referencing 4,000 papers published indental journals. Examining tweets to the 10 most tweeted papers found that counts would reduce by
56% if accounts tweeting rather than tweets were counted. If unique, non-duplicate tweets werecounted, the number would reduce by another 50%, reducing counts to 22% of their original number.
Examining tweeting accounts established that bots accounted for 2.4% of tweets, accounts only
retweeting for 27%, and accounts just tweeting titles for 47%. In sum, 77% of tweets were mechanicalin nature. 23% of tweets showed a more human, creative character though most such accounts
tweeted about dental papers only a few times. 6% of accounts were human in character and sustained
their tweeting about dental papers.
The results suggest that analysts who would use counts of tweets either to evaluate research or as asource of data should exercise caution. At minimum, bots and fake followers should be eliminated and8Though its ephemeral nature is something of an illusion presented to its human users; to the algorithms that turnits traces into counts, tweets endure for purposes of data analytics.

16

accounts tweeting rather than tweets should be counted. Better would be to identify and discountaccounts with identical tweets. Even better would be to count true unique text variants. Of course,
doing this would reduce Twitter data to a shadow of its current self, which would not be in the interestsof the advocates currently pushing for its use. However, it would be in the interests of analysts to
identify the tweets and tweeters that are informative, relevant and indicative of true discussion ofresearch.
Any evaluative regime risks establishing incentives that do not align with the desired result, in this caseresearch excellence. Research evaluation by tweet count seems to promise particular dangers of thiskind, above and beyond the standard litany of problems associated with metrics regimes applied toresearch. Evaluation by tweet count would present scientists with incentives to work on the kind of
oddball topics that attract obsessive single issue campaigners or are useful to social media accountmanagers. Focusing on tweet counts might lead to impatience with the stately, all-too-human, pace atwhich knowledge advances. Finally, tweet counts are all-too-easily inflated, enticing people to allocate
limited time to serving the bot agenda, a service that will always be better provided by bots themselves.
What is more, the dangers extend to librarians who recommend readings to students and professionals
based on such metrics in the belief that they are truly providing an added value. Almetric.com’s famousdoughnut displaying the altmetric score of papers, appearing on ever more journal websites, is largely
driven by Twitter counts. Its capacity to focus attention in a time when scientific literature increases
exponentially, might harm those tempted to confound well-conducted and highly rigorous research with
inflated numbers of tweets the origin and meaning of which is, to say the least, obscure. As altmetrics
gain in legitimacy and influence, the danger to the research enterprise grows.
Simplistic and naïve use of social media data risks damaging the scientific enterprise, misleading both
authors and consumers of scientific literature. All users of Twitter data need to be mindful of its limits
and consider whether it is fit for the purposes for which they propose to use it.

Acknowledgments
Nicolas Robinson-Garcia has received support from a Juan de la Cierva-Formación postdoctoral
fellowship from the Spanish Ministry of Economy and Competitiveness. He also received funding theFulbright Commission and José Castillejo. This work was partially supported by NIH grant U19-DE-22516
and NSF award number 1445121. We are grateful for discussions and help with data from Rakshit
Trivedi. We acknowledge Altmetric.com for facilitating access to their data. Opinions and assertions
contained herein are those of the authors and are not to be construed as necessarily representing theviews of the respective organizations or the National Institutes of Health.

References
Adie, E. (2014). Taking the alternative mainstream. El profesional de la información, 23(4), 349-351.
Altmetric.com (2016) How does Altmetric track Twitter?
https://help.altmetric.com/support/solutions/articles/6000157183-how-does-altmetric-track-twitter-,accessed July 17, 2017.
Alpert, J.M., & Womble, F.E. (2016). Just what the doctor tweeted: Physicians’ challenges and rewards of using
Twitter. Health Communication, 31(7), 824-832.
Bornmann L. Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics. Scientometrics.
2015;103:1123–1144.

17

Cardona-Grau, D. (2016). Join the discussion on Twitter! Journal of Pediatric Urology, 12(6), 334.
Choo, E.K., Ranney, M.L., Chan, T.M., Trueger, N.S., Walsh, A.E., Tegtmeyer, K., McNamara, S.O., Choi, R.Y. andCarroll, C.L., (201)5. Twitter as a tool for communication and knowledge exchange in academic medicine: a guidefor skeptics and novices. Medical teacher, 37(5), pp.411-416.
Chu, Z., Gianvecchio, S., Wang, H. and Jajodia, S., 2012. Detecting automation of Twitter accounts: Are you a human,bot, or cyborg?. IEEE Transactions on Dependable and Secure Computing, 9(6), pp.811-824.
Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison ofaltmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information
Science and Technology, 66(10), 2003-2019.
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A. and Tesconi, M., 2015. Fame for sale: efficient detection of fake
Twitter followers. Decision Support Systems, 80, pp.56-71.
Economist (2016) Twitter in retweet, September 17.
Eysenbach, G. (2009). Medicine 2.0: Social networking, collaboration, participation, apomediation, and openness.
Journal of Medical Internet Research, 10(3), e22.
Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation withtraditional metrics of scientific impact. Journal of Medical Internet Research, 13(4), e123.
Forgie, S.E., Duff, J.P., & Ross, S. (2013). Twelve tips for using Twitter as a learning tool in medical education. Medical
Teacher, 35(1), 8-14.
Freitas, C., Benevenuto, F., Ghosh, S. and Veloso, A., 2015, August. Reverse engineering socialbot infiltration
strategies in Twitter. In Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social
Networks Analysis and Mining 2015 (pp. 25-32). ACM.
Friedrich, N., Bowman, T. D., Stock, W. G., & Haustein, S. (2015). Adapting sentiment analysis for tweets linking toscientific papers. arXiv preprint arXiv:1507.01967.
Fuller, M.Y. and Allen, T.C., (2016). Let's Have a Tweetup: The Case for Using Twitter Professionally. Archives of
Pathology & Laboratory Medicine, 140(9), 956-957.
Grande, D., Gollust, S.E., Pany, M., Seymour, J., Goss, A., Kilaru, A., et al. (2014). Translating research for healthpolicy: Researchers’ perceptions and the use of social media. Health Affairs, 33(7), 1278-1285.
Haustein, S. (2016). Grand challenges in altmetrics: Heterogeneity, data quality and dependencies. Scientometrics,
108(1), 413-423.
Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014a). Coverage and adoption of altmetrics
sources in the bibliometric community. Scientometrics, 101(2), 1-19.
Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014b). Tweeting biomedicine: An analysis of
tweets and citations in the biomedical literature. Journal of the Association for Information Science andTechnology, 65(4), 656-669.
Haustein, S., Bowman, T.D., Holmberg, K., Tsou, A., Sugimoto, C.R., & Larivière, V. (2016b). Tweets as impact
indicators: Examining the implications of automated “bot” accounts on Twitter. Journal of the Association for
Information Science and Technology, 67(1), 232-238.
Haustein, S., Costas, R., & Larivière, V. (2015). Characterizing social media metrics of scholarly papers: The effect of
document properties and collaboration patterns. Plos One, 10(3), e0120495.
Haustein, Tsou, A., Minik, V., Brinson, D., Hayes, E., Costas, R., et al. (2016c). Identifying Twitter user communities in
the context of altmetrics. In 3:AM Conference, Bucharest, Romania. doi: 10.6084/m9.figshare.3980142.v1
Hawn, C. (2009). Take two aspirin and tweet me in the morning: How Twitter, Facebook, and other social media arereshaping Health Care. Health Affairs, 28(2), 361-368.
Kolahi, J. and Khazaei, S., 2016. Altmetric: Top 50 dental articles in 2014. British dental journal, 220(11), pp.569-574.
Konkiel, S., Madjarevic, N., & Rees, A. (2016). Altmetrics for librarians: 100+ tips, tricks, and examples. Altmetric LLP.
Ke, Q., Ahn, Y.-Y., & Sugimoto, C.R. (2017). A systematic identification and analysis of scientists on Twitter. Plos One,12(4), e0175368.
Kramer, B. (2017). #osc2017 What % of researchers use (and librarians recommend) #altmetrics tools/platforms cf.
traditional metrics? 101innovations.wordpress.com. https://Twitter.com/i/web/status/844539651083919361

18

Letierce, J., Passant, A., Decker, S., & Breslin, J.G. (2010). Understanding how Twitter is used to spread scientificmessages. In Proceedings of the Web Science Conference, Raleigh, NC, USA.
Maclean, F., Derek, J., Carin-Levy, G., & Hunter, H. (2013). Understanding Twitter. British Journal of Occupational
Therapy, 76(6), 295-298.
McGinnigle, E., McAloon, C.J., Warriner, D.R., Francis, R., & Chohan, B.C. (2016). Analysis of cardiologists’ use of
Twitter. European Heart Journal, 37(S1), 1264.
Nason, G.J., O’Kelly, F., Kelly, M.E., Phelan, N., Manecksha, R.P., Lawrentschuk, N., et al. (2015). The emerging use of
Twitter by urological journals. BJU International, 115(3), 486-490.
Philp, D. (2017). A $500 House in Detroit, Scribner.
Piwowar, H., & Priem, J. (2016). RLM: Investigating metrics at the researcher level. In 3:AM Conference, Bucharest,
Romania. doi: 10.6084/m9.figshare.3980091.v1
Priem, J. and Costello, K.L. (2010). How and why scholars cite on Twitter. Proceedings of the American Society for
Information Science and Technology, 47(1), pp.1-4.
Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. http://altmetrics.org/manifesto
Robinson-Garcia, N., Costas, R., Isett, K., Melkers, J., Hicks, (2017). Underlying data to the study: The unbearable
emptiness of tweeting – about journal articles. Figshare. https://doi.org/10.6084/m9.figshare.5195122.v1
Robinson-Garcia, N., Torres-Salinas, D., Zahedi, Z., & Costas, R. (2014). New data, new possibilities: Exploring theinsides of Altmetric.com. El professional de la informacion, 23(4), 359-366.
Roupret, M., & Misrai, V. (2015). Exponential use of social media in medicine: Example of the interest of Twitter ©
in Urology. Progress en Urologie, 25(1), 11-17.
Schnitzler, K., Davies, N., Ross, F., & Harris, R. (2016). Using Twitter ™ to drive research impact: A discussion ofstrategies, opportunities and challenges. International Journal of Nursing Studies, 59, 15-26.
Sugimoto, C.R., Work, S., Larivière, V., & Haustein, S. (2016). Scholarly use of social media and altmetrics: a reviewof the literature. arXiv Preprint arXiv:1608.08112
Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C.R. (2013). Do altmetrics work? Twitter and ten other social
web services. Plos One, 8(5), e64841.
Thelwall, M., Tsou, A., Weingart, S., Holmberg, K., & Haustein, S. (2013). Tweeting links to academic articles.
Cybermetrics, 17(1), 1.
Thompson MA, Mijhail NS, Wood WA, Perales M-A, Chaboissier M. (2015). Social media and the practicing
hematologist: Twitter 101 for the busy healthcare provider. Curr Hematol Malig Rep., 10(4), 405–412.
Torres-Salinas, D., Cabezas-Clavijo, Á., & Jiménez-Contreras, E. (2013). Altmetrics: New indicators for scientificcommunication in Web 2.0. Comunicar, 41, 53-60.
Torres-Salinas, D., & Milanés-Guisado, Y. (2014). Presencia en redes sociales y altmétricas de los principales autoresde la revista El profesional de la información. El profesional de la información, 23(4), 367-372.
Tufekci, Z. (2014). Big questions for social media big data: Representativeness, validity and other methodologicalpitfalls. Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media, 505-514, arXiv
preprint arXiv:1403.7400.
Van Dijck, J., & Poell, T. (2013). Understanding Social Media Logic. Media and Communication, 1(1), 2-14.
Vainio, J., & Holmberg, K. (2017). Highly tweeted science articles: who tweets them? An analysis of Twitter user
profile descriptions. Scientometrics, 1-22.
Wang, J., Veugelers, R., & Stephan, P. (2016). Bias against novelty in science: A cautionary tale for users ofbibliometric indicators (No. w22180). National Bureau of Economic Research.
Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., et al. (2015). The Metric Tide: Report of the
independent review of the role of metrics in research assessment and management. doi:
10.13140/RG.2.1.4929.1363
Wouters, P., & Costas, R. (2012). Users, narcissism and control: Tracking the impact of scholarly publications in the
21stcentury, http://www.surf.nl/nl/publicaties/Documents/Users%20narcissism%20and%20control.pdf
Zahedi, Z., Costas, R., & Wouters, P. (2014). How well developed are altmetrics? A cross-disciplinary analysis of thepresence of ‘alternative metrics’ in scientific publications. Scientometrics,101(2), 1491-1513.

19

Zimmer, M., & Proferes, N. J. (2014). A topology of Twitter research: disciplines, methods, and ethics. Aslib Journalof Information Management, 66(3), 250-261.

20