## 3.4 Reciprocity

Do speakers tend to reciprocate the choice of other speakers who pay attention to them? Reciprocity is a known powerful driver of human behavior and an antidote to status hierarchies. According to Gould (2002, p. 1151), "Someone who pays less attention to you than you pay to her implicitly asserts that she is superior to you in status. If you do not respond by withdrawing your attention, you have implicitly agreed". He submits that people are less willing to demonstrate esteem toward those who do not return the favor and conversely prefer to receive such demonstrations without reciprocating. When it prevails, the desire to have one's gestures reciprocated curbs the self-reinforcing character of status attributions.

Reciprocity may sometimes be replaced by group-based mechanisms (such as clustering) that sustain one-sided ties without sharpening status hierarchies. Put differently, a social group may provide a forum for interaction that mitigates status even when ties are not reciprocated (Block, 2015).

In addition to direct reciprocity, there may be a form of indirect reciprocity, which according to Nowak and Sigmund (2005), comes in two flavors. Upstream indirect reciprocity occurs when a speaker B who has received attention from speaker A, feels motivated to demonstrate esteem to someone too – and chooses to do so toward speaker C. Downstream indirect reciprocity is based on reputation and is observed when speaker A has paid attention to B and therefore receives attention from C. With indirect reciprocity, speakers who pay attention to many others, also receive many choices: this is another mechanism that may mitigate status hierarchies.

If speakers (directly) reciprocate attention choices, then the resulting relational structure will tend to be symmetric and can be represented as a graph where most ties are mutual (bottom right panel of figure 2). Indirect reciprocity will give rise to a network structure with high correlation between in-degrees (number of choices received) and out-degrees (number of choices directed to others).

# 4 Empirical setting and data

I explore these questions with data from the fourth OuiShare Fest, which took place on 18-20 May 2016 in Paris. I use back-office data from the event's software application, "Sched", and the register of participants kept by the organizers. Sched is an online and mobile tool to present oneself to others, connect, and choose a personalized program: each participant had the opportunity to create a profile (with name, photo, organization, job title, self-description), import contacts from social media such as Twitter and LinkedIn (from among those who were also attending the event), and choose a schedule (from among up to seven simultaneous parallel sessions going on at any one time during the event). The latter function was particularly attractive as no printed version of the program was available to participants, in line with the "Zero-Waste" policy that OuiShare promotes. Sched is designed to
suit the needs of a population that makes intense use of, and is comfortable with, digital technologies.

App data are an ideal source of information to address people’s attention-paying choices as discussed above (section 2). By taking the time to choose their agenda on Sched, participants devoted attention to the program and expressed interest in a selection of sessions and their speakers. Of note, their choices were visible to all online, being displayed on the Sched personal page of the choosing participant, on the page of the chosen session and the pages of all its speakers (also including moderators, facilitators etc.). They are publicly-visible endorsements, akin to the “Like” function of some social media platforms.

Sched data are traces of an attention exercise that participants did online prior to the event, based solely on their interests. They do not mirror practical factors such as time clashes: the application let users choose sessions that took place at the same time in different rooms (session-hopping was allowed), and there was no check-in at the entrance of sessions to monitor actual behaviors.

Of the 1830 registered participants in 2016 (of whom 1505 actually attended the event at least for one day), about two fifths have completed Sched pages. Importantly, this includes all the 258 speakers, whose profiles had been pre-filled by the organizers in advance, noting the sessions in which they were involved as speakers, panelists or moderators. 88 of these speakers chose a program of sessions to attend beyond the one(s) in which they were scheduled to be involved. 475 non-speakers did so. A logistic regression (Annex) shows that controlling for time spent at the event, participation in previous editions, place of residence, and gender, speakers were less likely than others to choose a schedule on Sched. As suggested in subsection 3.3, paying attention to others has a “cost” that not all are prepared to pay, whilst such cost is lower (or perhaps non-existent) for non-speaking participants, whose main reason to attend the event is the program. Only press representatives were less likely than speakers to use Sched – presumably because they had narrow reporting goals rather than an overarching interest in the Fest.

## 4.1 Dependent variable

The dependent variable *Y* is the S2S network of speakers paying attention to other speakers. It is a dyadic variable, whose values are defined for each pair of the 258 OuiShare Fest 2016 speakers. Disregarding self-attendance, it corresponds to 258×257 = 66306 dyads. Each value *Y*<sub>ij</sub> (with *i*, *j* = 1, ..., 258, *i* ≠ *j*) is equal to the number of times speaker *i* chose to pay attention to speaker *j* by choosing one of their sessions on Sched. *Y*<sub>ij</sub> will differ from *Y*<sub>ji</sub> whenever the choice of *i* to pay attention to *j* is not reciprocated.

The S2S network is almost fully connected, with a main component of 255 nodes and 3 isolates – speakers who neither send out nor receive choices. Figure 3 illustrates differences between speakers depending on the number of attention choices they receive from, and direct to, other speakers, also known as their in-degrees (left
panel) and out-degrees (right panel). Both distributions are right skewed, consis-
tently with most empirical social networks: positive skewness indicates higher vari-
ation in relational activities than would be produced by a random process, and points
to differences in the outgoing attitudes of individuals (in the case of out-degrees) and
differences in status, prominence or prestige in the case of in-degrees (Ahuja et al.,
2009; Pallotti et al., 2013).

<figure><img src="image_11.png" /><figcaption>**Fig. 3** Degree distributions. In-degrees (left panel) and out-degrees (right panel) in the S2S network of 258 speakers. Histograms (gray bars) and density plots (red curved lines).</figcaption></figure>

In what follows, for analytical convenience and without significant loss of infor-
mation, the values of *Y* are binarized (that is, each *Y*<sub>*ij*</sub} > 0 is set to 1).

**4.2 Independent variables**

Independent variables are also defined at dyadic level, with one value for each of
the 66306 pairs of speakers. They measure the different social mechanisms that
may affect the network structure Y.

**4.2.1 Thematic affinity**

To remain as close as possible to the experience of Sched users, I map speakers
to themes, or areas of expertise, using the categorization proposed by the event or-
ganizers, who had assigned each session to either a general category (for sessions
such as Fest Opening and Closure), a satellite event, or one of six following themes:
“Building Enterprises for the Digital Age”, “Digital Institutions and The City”, “Ed-
ucation and Personal Development”, “Power and Capital in the 21st Century Orga-
nization”, “The (Present) Future of Work”, “Understanding Decentralization and
The Blockchain” (subsection 2.1).

Figure 4 represents the extent to which speakers who are competent in each of the six themes (rows) pay attention to speakers in theirs and other themes (columns)<fnref n="2" />. It appears clearly that most choices come from speakers involved in sessions on “Education and Personal Development”, “Power and Capital in the 21st Century Organization” and to a lesser extent “Building Enterprises for the Digital Age” and “Understanding Decentralization and The Blockchain”, while the themes that receive most choices are “Power and Capital in the 21st Century Organization”, “Digital Institutions and The City”, and to a lesser extent “Building Enterprises for the Digital Age” and “Understanding Decentralization and The Blockchain”. On the one hand, these choices differ from those of the 475 non-speaking attendees who have a Sched schedule, whose preferred theme is “Digital Institutions and The City” (22.9%), followed by “Building Enterprises for the Digital Age” (20.7%) and “Power and Capital in the 21st Century Organization” (17.6%). Thus, speakers’ choices have specificities that are worth exploring. On the other hand, same-theme ties are overall rarer than cross-theme ones. This is enough to rule out the possibility that the network is split into separate thematic components.

Formally, this is a dyadic, binary, symmetric variable *TA* (table 1) whose value *TA*<sub>ij</sub> = *TA*<sub>ji</sub> is 1 if speakers *i* and *j* (with *i* ≠ *j*) have at least one theme in common, 0 otherwise.

4.2.2 Positional similarity

Attention choices not only result from exogenous factors such as thematic affinity, but are also partly self-sustained through an endogenous tendency to reproduce existing network structures. As discussed in sub-section 3.2, I interpret competition between speakers as the extent to which they attract attention from the same participants, whether they be themselves speakers or simple attendees (positional similarity). This requires examining the attendance choices of all OuiShare Fest participants who used Sched to build a personalized online program, in the expectation that they will inflect speakers’ choices of whom, among other speakers, to pay attention to. Technically, this means calculating an indicator that reflects similarity in incoming ties within the A2S2S network structure (right panel of figure 1), as a potential determinant of S2S (central panel of figure 1).

A commonly used measure of positional similarity is the Jaccard index which, for
any two speakers, is defined as the number of common participants (intersection),

<footnote n="2">² Thematic expertise is attributed to speakers based on the theme(s) of the session(s) in which they speak. Each speaker can thus be expert in one or more themes. I follow Everett and Borgatti (2017) in representing these “overlapping categories” as a person-by-category affiliation matrix $B = (b_{ij})$, of dimensions 258x6. Matrix $C = AB$, where $A$ is the 258x258 square matrix of speakers’ attention-paying choices, is a speaker-by-theme matrix that measures speakers’ access to different themes not through their own expertise, but through their attention-paying choices. The heatmap of figure 4 represents the product matrix $B^T C = B^T AB$.</footnote>
<figure><img src="image_12.png" /><figcaption>**Fig. 4** Heatmap representing the extent to which speakers with expertise in each of the six themes of OuiShare Fest 2016 (rows) pay attention to speakers in theirs and other themes (columns). Darker colors represent higher numbers. Interpretation: the highest number of choices are from speakers in the “Education and Personal Development” theme toward speakers in the “Power and Capital in the 21st Century Organization” theme.</figcaption></figure>

divided by the cardinality of the union (sum of the participants of each of the two speakers, both separately and jointly). Scaling by the size of the union rules out the possibility that similarity scores are affected by the tendency of more prominent speakers to receive more choices than lesser-known speakers (Borgatti, 2012). This dyadic variable $J_{ij} = J_{ji}$ can be computed for each pair of speakers, is symmetric and varies between 0 and 1 (table 1).

### 4.2.3 Status

What matters here is the status of the receiver, rather than the sender, of attendance choices – the potential object of others’ attention. Hence for each of the 66306 pairs of speakers [i, j], focus is on the status of j (see table 1). As discussed in subsection 3.3, some dimensions of status are exogenous, and I retrieve them from speakers’ Sched profiles and from registration information. Variable $St_j^1$ is binary and indicates whether speaker j was the founder of a sharing economy start-up, think-tank, charity or other type of organization. Founders are entrepreneurs who invest their talent and resources to advance the sharing economy, and are highly respected in this community that values initiative, innovation and dynamism. Variable $St_j^2$ is also
binary and indicates whether speaker *j* had already participated in the 2015 and/or 2014 OuiShare Fest, to denote experience in, and familiarity with, this milieu.

As discussed above, some status cues that can be used to infer quality when it
cannot be objectively measured, depend on the network structure itself. Again, I
need to include status indicators that reflect the A2S2S network structure, under the
expectation that they affect S2S. One of these indicators is the number of attention
choices that each speaker received through Sched – a proxy for popularity (sub-
section 3.3). To retrieve this information, I sum over the attendance choices of all
participants who had a Sched profile (variable $St_j^3$, taken on A2S2S). A speaker could
be chosen more than once by the same participant.

The other indicator that can be computed from the A2S2S network structure is
the number of followers that speakers had on social media, notably Twitter and
LinkedIn, insofar as it was imported into, and made visible through, Sched. Social
media are relevant because Fest participants are heavy technology users, interact-
ing online as much as face-to-face. OuiShare has always actively used Twitter and
LinkedIn to communicate, creating a large online network around itself, and all its
events were live-tweeted. Being part of the conversation signals online notoriety and
legitimacy within this community. Technically, I count the number of times each
speaker appears as an online contact in the Sched pages of OuiShare Fest partici-
pants, summing over all those (speakers and non-speakers in the A2S2S structure)
who had linked their Sched profiles to their social media accounts (variable $St_j^4$).

4.2.4 Control variables

Other potentially relevant factors are participation in the previous edition of the
event, 2015, which may facilitate integration of participants (variable *T1*), place
of residence (distinguishing, for simplicity, France where 50% of participants and
about 40% of speakers came from, and other countries: variable *Fr*) and gender
 variable *Ge*). They are originally monadic and taken from the organizers’ register.
To adapt them to the dyadic structure of the data, I compute an exact match indicator
for each variable, to identify pairs of speakers in the same category – for example,
both from France (table 1).

5 An Exponential Random Graph Model of attention choices

Attendance choices are driven both by exogenous and endogenous factors. I have
already introduced the number of choices received by a speaker as a measure of
status, and the Jaccard index of positional similarity – two measures that I take on
the multi-level relational structure A2S2S, and that I use to explain the speaker-to-
speaker (S2S) relational structure. The presence of such systematic dependencies
between network ties requires explicit modeling and cannot be left to standard sta-
<table><caption>**Table 1** Dyadic variables included in empirical model specification: definitions and descriptive statistics. Those in italics are endogenous to the multi-level A2S2S network.</caption><thead><tr><th>Variable</th><th>Construct</th><th>Min</th><th>Max</th><th>Mean</th><th>St.Dev</th></tr></thead><tbody><tr><td>Speaker-to-speaker attendance</td><td>Dependent variable</td><td>0</td><td>1</td><td>0.059</td><td>0.235</td></tr><tr><td>TA: Common themes</td><td>Thematic affinity</td><td>0</td><td>1</td><td>0.234</td><td>0.538</td></tr><tr><td>St1: Founder of organization</td><td rowspan="2">Status (receiver)</td><td>0</td><td>1</td><td>0.31</td><td>0.463</td></tr><tr><td>St2: Experience</td><td>0</td><td>2</td><td>0.527</td><td>0.753</td></tr><tr><td>St3: Number choices</td><td rowspan="2">Status (receiver)</td><td>4</td><td>500</td><td>105.7</td><td>72.407</td></tr><tr><td>St4: Number online followers</td><td>0</td><td>63</td><td>4.77</td><td>10.642</td></tr><tr><td>Ja: Jaccard index</td><td>Positional similarity</td><td>0</td><td>1</td><td>0.145</td><td>0.119</td></tr><tr><td>T1: Participation previous year</td><td rowspan="3">Control (match)</td><td>0</td><td>1</td><td>0.314</td><td>0.465</td></tr><tr><td>Fr: Residence in France</td><td>0</td><td>1</td><td>0.376</td><td>0.485</td></tr><tr><td>Ge: Gender</td><td>0</td><td>1</td><td>0.616</td><td>0.487</td></tr></tbody></table>

tistical inference approaches which assume independence of observations (Pallotti et al., 2013).

Recently developed Exponential Random Graph Models (ERGM) allow specifying the dependence among relations that such micro-mechanisms entail, in addition to modeling the effects of exogenous covariates (Robins et al., 2009; Snijders et al., 2006). These models address a fundamental issue that arises with network data: the presence of dependencies across observations as one node may be involved in multiple relationships with other nodes. They do not limit themselves to “controlling for” such dependencies, but go as far as representing them directly, uncovering specific local (sub-graph) configurations of ties implied by dependence mechanisms, and testing their statistical significance (Amati et al., 2018).

With ERGM, one may go as far as exploring effects that are fully endogenous to the S2S network structure (table 2). Reciprocity, a potentially important determinant of choices (sub-section 3.4), is one such effect: only speakers can reciprocate an attention choice by other speakers. Direct reciprocity is the tendency of the network structure to be symmetric, and concerns 7% of ties in this network. Indirect reciprocity is the tendency of outgoing and incoming ties to co-occur (technically, a mixed-two-paths configuration, which can represent both upstream and downstream indirect reciprocity), which entails a tendency of in-degrees and out-degrees to be correlated.

Some status-related effects are also fully endogenous to the S2S network (sub-section 3.3). They capture differences in the propensity of individual speakers to be attended by other speakers – independently of the other choices they might receive from the broader population of participants. Such differences may create status disparities regardless of other individual qualities. To account for this possibility, I
include so-called in-degree effects, which measure popularity interpreted as the extent to which attention choices are more likely to be directed at speakers who already have incoming ties, all other things equal.

Specifically, I use two-stars, three-stars and geometrically-weighted in-degree. The meaning of two- and three-star effects is intuitive: the more edges on a node, the more two- (or three-) stars an additional edge will create (table 2). The in-degree effect is an higher-order effect, suitable for cases in which the in-degree distribution is uneven. It estimates the change in tie likelihood given the in-degree of the nodes involved, and it is “geometrically weighted”, that is, with marginally decreasing weighting as degree increases. The weights avoid degeneracy, a common problem with ERGM estimations<fnref n="3" />.

<table><caption>**Table 2** Structural dependencies to be modeled, representing endogenous feedback effects to the S2S network.</caption><thead><tr><th>Variable</th><th>Construct</th><th>Configuration</th></tr></thead><tbody><tr><td>DR: Reciprocity</td><td>Direct reciprocity</td><td><pre>  ■   ■
  |   |
</pre></td></tr><tr><td>IR: Mixed-two-paths</td><td>Indirect reciprocity</td><td><pre>  ■   ■
  |   |
</pre></td></tr><tr><td>2I: 2-instar</td><td></td><td><pre>  ■   ■
  |   |
</pre></td></tr><tr><td>3I: 3-instar</td><td>In-degree effects</td><td><pre>  ■   ■
  |   |
</pre></td></tr><tr><td>gwI: geometrically weighted in-degree</td><td></td><td><pre>  ■   ■
  |   |
</pre></td></tr></tbody></table>

Despite requiring a major departure from the way data structures are commonly conceptualized in the social sciences, ERGM provide the same approach to inference, whereby an outcome variable is predicted by several independent variables. Parameter values help identify a probability distribution for all graphs of the same

<footnote n="3">³ Degeneracy occurs when the ERGM estimation process produces graphs that are complete, or empty, or with all edges concentrated in a small region of the graph, or otherwise irrelevant (Handcock, 2003; Snijders et al., 2006).</footnote>
size, and estimates aim at finding the parameter values that best match the observed
network structure. This is obtained via Markov Chain Monte Carlo Maximum Like-
lihood (MCMCML) or other simulation-based techniques; interested readers may
find more statistical details in Lusher et al. (2013).

6 Results

Table 3 reports ERGM estimates of a model in which the dependent variable $Y = S2S$ is estimated on the basis of the exogenous covariates and endogenous network dependencies outlined above. Among endogenous configurations, I do not include transitive closure, widely used in the literature but not salient in this context (meaning that two speakers who pay attention to the same third speaker are also likely to choose each other). Neither do I model out-degree effects: rather, I constrain the estimation process to preserve observed out-degrees (so that only networks whose out-degrees are the same as those in the data have non-zero probability). Indeed, speakers’ overall reluctance to make online attendance choices (Annex) suggests that the specificity of the few Sched-using speakers is not an incidental occurrence but a constitutive element of the relational structure under study. The model is estimated with R package ‘ergm’ in the Statnet suite (Handcock et al., 2003; Hunter et al., 2008).

6.1 Estimates

Let us first comment on quality of estimations. The MCMC diagnostics (not re-
ported here) indicate good convergence: the MCMC sample statistics vary randomly
around the observed values at each step and the difference between the observed and
simulated values of the sample statistics have a roughly bell-shaped, zero-mean dis-
tribution indicating stationarity.

Goodness-of-fit can be assessed following the approach of Hunter et al. (2008) who suggested that the network statistics measured in the original data should be reproduced reasonably well on networks simulated from the fitted model (Li and Carriere, 2013). The observed statistics can be plotted against the simulated statistics to assess whether ERGM fits the data well. The Annex includes plots of the statistics commonly used to measure goodness-of-fit, showing that the model represents key patterns in the data rather well, though it slightly amplifies the skewness of the in-degree distribution.
<table><caption>**Table 3** ERGM Maximum Likelihood estimates. Dependent variable: network of attention choices S2S, independent variables: covariates and endogenous structural dependency statistics outlined in Tables 1 (upper section) and 2 (lower section).</caption><thead><tr><th>Variable</th><th>Estimate</th><th>Std. err.</th><th>p-value</th><th>Signif.<sup>a</sup></th><th>Odds-ratio</th></tr></thead><tbody><tr><td>Thematic affinity</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mutual (TA<sub>ij,ji</sub>)</td><td>0.9118</td><td>0.1963</td><td>&lt; 1e-04</td><td>***</td><td>2.4888</td></tr><tr><td>Upper (TA<sub>ij</sub>)</td><td>0.2904</td><td>0.0734</td><td>&lt; 1e-04</td><td>***</td><td>1.3369</td></tr><tr><td>Lower (TA<sub>ji</sub>)</td><td>0.4507</td><td>0.075</td><td>&lt; 1e-04</td><td>***</td><td>1.5694</td></tr><tr><td>Positional similarity</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mutual (Ja<sub>ij,ji</sub>)</td><td>-0.2875</td><td>0.0472</td><td>&lt; 1e-04</td><td>***</td><td>0.7502</td></tr><tr><td>Upper (Ja<sub>ij</sub>)</td><td>0.038</td><td>0.2657</td><td>0.8861</td><td></td><td>1.0388</td></tr><tr><td>Lower (Ja<sub>ji</sub>)</td><td>-0.1027</td><td>0.2756</td><td>0.7094</td><td></td><td>0.9024</td></tr><tr><td>Status</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Receiver founder</td><td>0.0606</td><td>0.0375</td><td>0.106</td><td></td><td>1.0624</td></tr><tr><td>Receiver experience</td><td>-0.057</td><td>0.0286</td><td>0.0463</td><td>*</td><td>0.9446</td></tr><tr><td>Receiver No. attendees</td><td>0.0098</td><td>0.0009</td><td>&lt; 1e-04</td><td>***</td><td>1.01</td></tr><tr><td>Receiver No. followers</td><td>-0.0033</td><td>0.0022</td><td>0.1334</td><td></td><td>0.9967</td></tr><tr><td>Controls</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Co-presence past year</td><td>-0.1128</td><td>0.0431</td><td>0.0089</td><td>**</td><td>0.8934</td></tr><tr><td>Both from France</td><td>0.2248</td><td>0.0413</td><td>&lt; 1e-04</td><td>***</td><td>1.2521</td></tr><tr><td>Same gender</td><td>0.0437</td><td>0.0417</td><td>0.2938</td><td></td><td>1.0447</td></tr><tr><td>Structural dependencies</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Reciprocity</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Direct reciprocity</td><td>0.127</td><td>0.174</td><td>0.4655</td><td></td><td>1.1354</td></tr><tr><td>Indirect reciprocity</td><td>0.0001</td><td>0.0008</td><td>0.873</td><td></td><td>1.0001</td></tr><tr><td>Indegree effects</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>2-instar</td><td>0.0804</td><td>0.0091</td><td>&lt; 1e-04</td><td>***</td><td>1.0838</td></tr><tr><td>3-instar</td><td>-0.0034</td><td>0.0004</td><td>&lt; 1e-04</td><td>***</td><td>0.9966</td></tr><tr><td>Gwidegree<sup>b</sup></td><td>-1.6607</td><td>0.3274</td><td>&lt; 1e-04</td><td>***</td><td>0.19</td></tr></tbody></table>

## 6.2 Interpretation

Interpretation of specifications and outcomes closely resembles that of a logistic regression: generally, a large positive (negative) parameter indicates that the corresponding configuration is observed in the network under study more (less) frequently than what would be expected by chance alone, conditional on the presence of configurations associated with other effects in the data.

<footnote><sup>a</sup> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘’ 1</footnote>

<footnote><sup>b</sup> Geometrically-weighted in-degree (section 5).</footnote>

<footnote>N = 66306, AIC: -2133, BIC: -1969</footnote>
**6.2.1 Thematic affinity**

Thematic affinity does not go as far as splitting the community into separate islands, but it has a positive and significant effect: speakers are more likely to give and receive choices from others with similar interests, and this is true whether their choice is reciprocated ("mutual" effect) or not (ij and ji combinations respectively). Their behavior is highly cooperative provided there is a shared interest: they give back when they receive, but are also happy to give without a return. As suggested in sub-section 3.1, this may be because choosing like-minded others may reinforce one's thematic interests and contribute to legitimizing them. In this sense, Sched attendance choices are a way for speakers to promote their ideas beyond their own talks, by publicly displaying interest in those similar to them.

**6.2.2 Competition**

Competition, interpreted as positional similarity (sub-section 3.2) has a negative ef-
fect only on mutual (reciprocal) ties. Put differently, if speakers *i* and *j* attract largely
the same attendees, then they are less likely to choose each other, but their compet-
itive position does not significantly affect the likelihood of a non-reciprocated tie
between them (*ij* or *ji*). Speakers actively counter reciprocity when they compete
for attention of the same third parties – hence, for the same resources. Gould (2002)
theorizes that people are torn between the desire to receive demonstrations of esteem
from others without returning the favor – because this raises their status – and the
desire to see their own demonstrations of attention to others reciprocated – because
this avoids lowering their status (sub-section 3.4). The finding reported in table 3
suggests that the former tendency prevails under competitive conditions.

**6.2.3 Status**

Regarding the observable dimensions of status (sub-section 3.3), being founder of an
organization does not matter, while surprisingly, experienced speakers are slightly
less likely to receive attention from other speakers. This result is best understood in
conjunction with the effect of one control variable, co-participation the year before,
which is negative and significant. Together, they mean that all other things equal, a
returning speaker is less likely than a newcomer to attend the talk of another return-
ing speaker. This attitude is in line with OuiShare's explicit orientation towards "a
willingness to experiment, step out of one's comfort zone" (section 2). Interestingly,
preference for novelty seems to counter some dimensions of status – those, at least,
that can be associated to durable involvement in the community. A trade-off appears
between novelty/openness and continuity/fixed anchors.

The model includes several proxies for the unobservable dimensions of status,
notably popularity (sub-section 3.3). They are operationalized in the A2S2S net-
work by counting number of online followers and number of attendees: the former
has no effect while the latter is positive and significant. That this finding resonates
with the literature on herding behavior does not mean that the choice to attend the
most attended should be interpreted as mindless imitation. In an international event
that hosts multiple views and standpoints, a large number of attendees may denote a
diverse audience. Thus, following their choices may signal (again) a quest for nov-
elty, a sense of curiosity for what others like. If the preference for novelty counters
some dimensions of status, as noted above, it is consistent with others.

Other popularity indicators are in-degree effects, all significant: two-in-star is positive, meaning that ties are more likely to be directed at nodes that already have an incoming tie, while three-in-star and geometrically weighted in-degree distribution, a higher-order effect, are negative, meaning that after accounting for all other model terms, nodes with large numbers of ties are less likely to receive more ties. Taken together, these results indicate that a speaker does not want to be the first to pay attention to another speaker, but does not follow mere popularity either. Distinction from peers is valued, but nobody would go as far as taking the risk of setting entirely new trends. That speakers value distinction while being reluctant to overly differentiate themselves suggests a form of conformity as often observed in middle-status individuals Phillips and Zuckerman (2001).

6.2.4 Reciprocity

Neither direct reciprocity nor indirect reciprocity (sub-section 3.4) are significant. Paying attention to someone else does not increase one's odds of receiving attention. Further, using Sched to choose one's own agenda does not affect the chances of receiving attention choices by others, so there is no tendency toward correlation of in- and out-degrees. These tendencies are better understood in light of the results already outlined: if in general, reciprocated and unreciprocated ties are equally likely to be observed, speakers are more likely to form mutual ties when there is the comfort of thematic affinity between them (sub-section 6.2.1), while they are less likely to do so when there is the discomfort of competing for the same attendees (sub-section 6.2.2). Put differently, collaborative contexts favor reciprocity, while competitive conditions hinder it.

The effects of reciprocity (or lack of it) are important because they inevitably
spill over onto status (sub-section 6.2.3). Reciprocity within same-theme network
niches may mitigate status effects, but lack of an overall tendency to reciprocity
in the network (and active resistance to it among competitors) tend to reinforce
them. Overall, the interplay of these opposing social mechanisms results in mild
status hierarchies. This is consistent with a setting such as the sharing economy
whose definition, scope and boundaries were still moving when the research took
place, and there were no strict social norms to regulate how participants should pay
attention to, and recognize the value of, one another.

Overall, this setting is one in which actors have multiple forms of status that do
not systematically strengthen one another. Some interactions that generate, maintain
or exacerbate status hierarchies, coexist with others that mitigate them. Newcomers are welcome, while struggles to gain prominence occur: actors compete for attention and make choices instrumentally to discover novelties and to promote their themes of interest.

# 7 Conclusions

The motivating question of this paper was, what shapes people's choice to pay attention to others in conference-like settings. The choice of whom to pay attention to is one of the mechanisms through which values (broadly interpreted as themes, orientations and areas of activity) gain or lose legitimacy within a community. The speakers who give and receive attention in a conference carry with them values that are essentially precarious, whose survival depends on the support they receive (Selznick, 1957), especially in a nascent area of activity such as the sharing economy in 2016.

The above analysis suggests two main answers. The first is a quest for novelty: people pay attention to those who offer new experiences, who get them out of their comfort zone as OuiShare put it. The desire for novelty may or may not lead to the choice of high-status others: in particular when status involves some form of rigidity, whether real or perceived, novelty prevails (sub-section 6.2.3). At first sight, this result seems at odds with the literature which, focusing mostly on trade fairs (section 1), finds that ties are more likely to be formed after a history of co-participation (Brailly et al., 2016). But the structure of interactions differs. In trade fairs, researchers map networks of informal contacts through which participants prepare future commercial transactions: connecting with previous acquaintances is a way for them to reduce uncertainty and facilitate negotiations. Instead in conferences, where the key interaction is attendance at sessions and the goal is to gain knowledge or insights rather than contracts, new speakers are more likely to be non-redundant. This is why novelty is appreciated. Research in social network analysis has made clear that when the stakes are of informational or cognitive nature, redundancy is an issue, while access to non-redundant ideas brings advantages (Burt, 1992, 2004). Thus, one way in which speakers' attention-paying choices create value in an event like the OuiShare Fest is through their orientation toward novel, non-redundant content.

The second answer is a strategic use of popularity and reciprocity. Participants fine-tune their use of popularity cues to affirm their standing while limiting the risks of status loss that might arise if they were the first to pay attention to someone else, especially under conditions of uncertainty (sub-section 6.2.3). Similarly, and perhaps ironically in a festival formed around the premise of sharing, people use reciprocity (sub-section 6.2.4) as a flexible instrument to manage their position in the social structure, depending on whether their goal is to secure support for their thematic niche (sub-section 6.2.1) or to stay afloat in situations of competition. It would be too hasty to conclude that these results challenge received wisdom that
popularity is always (more or less) self-reinforcing, and that dislike of symmetry is uncommon. Rather, the insight they provide is that popularity and reciprocity themselves are too general principles to directly explain behaviors. They subtly combine with other relevant social mechanisms (here, thematic homophily and competition) and play out differently depending on the resulting mix of factors. In the specific case of the OuiShare Fest, limited and selective use of reciprocity contributes to generating a status hierarchy, which creates value by providing signals of quality when other information is lacking. Conversely, circumspect reliance on popularity keeps this hierarchy mild, and creates value by avoiding a rigidity that would hamper creativity and innovation. Together, they manage status in this community so that its above-discussed drive for novelty does not entirely exclude some degree of fixed anchoring, and its homophily does not split the community.

Future research will establish how general these empirical findings are. For example, it could be that competitive events such as awards display sharper status differences, based on different combinations of reciprocity, popularity, or homophily. A specific limitation of my study stems from the design of the digital application used to make attendance choices. Perhaps, modified software features might trigger more (or less) intense use of Sched: for example, one might ask whether a more user-friendly application could lower the cost of making attendance choices.

What is general beyond my setting is the importance of considering the joint effects of multiple relational mechanisms (reciprocity, popularity, homophily and so on), each interfering with the others in complex ways. Within the narrower literature on tournaments of values (also called field-configuring events or large-scale professional gatherings s discussed earlier), my results suggest the need of distinguishing between types of interactions, as cognitive-oriented attention choices and (for example) trade-oriented informal networking do not rely on the same social processes. Given the wide diffusion of attention-based events such as conferences and festivals, and their potential impact on the communities that they gather, the ideas discussed here may be of broader interest than the specific setting considered. Attempts to replicate the results reported above and extend them to different settings will stimulate further reflection on the determinants and consequences of attention-paying choices.

# 8 Annex

## 8.1 Oui Share Fest session themes, 2013-17
<table><caption>**Table 4** OuiShare Fest session themes, with number of sessions per theme, 2013-17. Non-thematic sessions (such as Opening and Closure) and scheduled leisure and breaks (coffee and lunch breaks, conclusive “love” party etc.) not included.</caption><thead><tr><th colspan="2">2013</th><th colspan="2">2014</th><th colspan="2">2015</th><th colspan="2">2016</th><th colspan="2">2017</th></tr><tr><th>Themes</th><th>N. sessions</th><th>Themes</th><th>N. sessions</th><th>Themes</th><th>N. sessions</th><th>Themes</th><th>N. sessions</th><th>Themes</th><th>N. sessions</th></tr></thead><tbody><tr><td>Business &amp; Design</td><td>7</td><td>Co-Create</td><td>21</td><td>Capital</td><td>7</td><td>Building Enterprises for the Digital Age</td><td>23</td><td>APIs for new urban services</td><td>1</td></tr><tr><td>Collaborative Consumption</td><td>17</td><td>Connect</td><td>17</td><td>Cities</td><td>5</td><td>Digital Institutions &amp; The City</td><td>25</td><td>Fighting climate change</td><td>10</td></tr><tr><td>Economics &amp; Society</td><td>17</td><td>Debate</td><td>28</td><td>Culture</td><td>10</td><td>Education &amp; Personal Development</td><td>30</td><td>Future of work</td><td>9</td></tr><tr><td>Finance &amp; Money</td><td>8</td><td>Inspire</td><td>43</td><td>Decentralization</td><td>9</td><td>Power and Capital in the 21st Century Organization</td><td>14</td><td>Living together</td><td>49</td></tr><tr><td>Makers &amp; Open Manufacturing</td><td>13</td><td>Play</td><td>12</td><td>Experience &amp; Other</td><td>39</td><td>The (Present) Future of Work</td><td>14</td><td>New technologies for impact</td><td>25</td></tr><tr><td>Open Knowledge</td><td>8</td><td></td><td></td><td>Makers</td><td>9</td><td>Understanding Decentralization &amp; The Blockchain</td><td>15</td><td>Presence &amp; consciousness</td><td>12</td></tr><tr><td>UnConference</td><td>12</td><td></td><td></td><td>Open Knowledge Organizations</td><td>8</td><td></td><td></td><td>Social justice</td><td>12</td></tr><tr><td></td><td></td><td></td><td></td><td>Politics</td><td>23</td><td></td><td></td><td>Taking power back</td><td>18</td></tr><tr><td></td><td></td><td></td><td></td><td>Sharing economy</td><td>9</td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td>Social impact</td><td>7</td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td>Sustainability</td><td>9</td><td></td><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td>Work</td><td>9</td><td></td><td></td><td></td><td></td></tr></tbody><tfoot><tr><td><strong>Total</strong></td><td>82</td><td></td><td>121</td><td></td><td>162</td><td></td><td>121</td><td></td><td>136</td></tr></tfoot></table>
## 8.2 Logistic regression model of posting attendance choices

Table 5 reports the results of a logistic regression model of posting attendance choices on Sched. The dependent variable takes values 1 = Yes, 0 = No. N = all persons registered to OuiShare Fest 2016. Covariates are Presence (number of days spent at the event, between 0 and 3); Experience (number of Fest participations in previous two years, between 0 and 2); residence in France (1 = Yes, 0 = No); Gender (1 = Male, 0 = Female); and type of participant (Attendee as reference category; other types: Attendee 1 day, Press, Speaker, Team).

<table><caption>Table 5 Results of a logistic regression model of posting attendance choices on Sched<sup>a</sup>.</caption><thead><tr><th></th><th>Estimate</th><th>Std. Error</th><th>z-value</th><th>Pr(&gt;|z|)</th><th>Signif.<sup>b</sup></th><th>Odds-ratios</th></tr></thead><tbody><tr><td>Presence</td><td>0.8514</td><td>0.0667</td><td>12.759</td><td>&lt; 0.0000</td><td>***</td><td>2.343</td></tr><tr><td>Experience</td><td>-0.008</td><td>0.1009</td><td>-0.079</td><td>0.9368</td><td></td><td>0.992</td></tr><tr><td>France</td><td>-0.4283</td><td>0.1224</td><td>-3.499</td><td>0.0005</td><td>***</td><td>0.6516</td></tr><tr><td>Gender</td><td>-0.0295</td><td>0.1123</td><td>-0.262</td><td>0.793</td><td></td><td>0.971</td></tr><tr><td>Attendee</td><td>ref.</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Attendee 1 day</td><td>-0.2798</td><td>0.1946</td><td>-1.438</td><td>0.1505</td><td></td><td>0.7559</td></tr><tr><td>Press</td><td>-0.6036</td><td>0.2859</td><td>-2.112</td><td>0.0347</td><td>*</td><td>0.5468</td></tr><tr><td>Speaker</td><td>-0.5516</td><td>0.1648</td><td>-3.348</td><td>0.0008</td><td>***</td><td>0.5761</td></tr><tr><td>Team</td><td>0.0855</td><td>0.1705</td><td>0.501</td><td>0.6162</td><td></td><td>1.0892</td></tr><tr><td>(Intercept)</td><td>-1.85</td><td>0.1892</td><td>-9.78</td><td>&lt; 0.0000</td><td>***</td><td>0.1572</td></tr></tbody></table>

## 8.3 ERGM goodness-of-fit diagnostics

In figure 5, the observed network is compared to networks simulated from the fitted ERGM in terms of standard summary statistics: in-degree (number of incoming ties of a node), edge-wise shared partners (nodes connected to both sender and receiver of a tie), and minimum geodesic distance, or path length (minimum number of connected ties, by which two nodes are linked). The black solid lines are the observed statistics in the actual network and the box plots reflect networks simulated from the model. Fit is good if they overlap as much as possible.

Figure 6 plots the model's observed statistics as quantiles of the simulated sample. In a good fit, the observed statistics are near the sample median (0.5).

<footnote><sup>a</sup> N = 1830, AIC: 1942.1</footnote>

<footnote><sup>b</sup> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘’ 1</footnote>
<figure><img src="image_13.png" /><figcaption>Fig. 5 The diagnostics plots of Hunter et al. (2008).</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Fig. 6 Observed statistics as quantiles of the simulated sample.</figcaption></figure>

# 9 Acknowledgments

This paper is part of the research project "Sharing Networks". I gratefully acknowledge logistical support from the OuiShare Fest 2016 organization team, in particular Francesca Pick (Chair), Khushboo Balwani (Communications officer), Camille
Chapuis (Project manager), and additionally OuiShare founders Arthur de Grave, Diana Filippova and Benjamin Tincq. I thank Sarah Benmarhnia for research assistance in preparation of the data, and Lise Mounier for advice on a draft of this paper. Preliminary results were presented at the DYMLNA workshop at Sciences Po, Paris (March 2017), a seminar of the RITM research group at Université Paris Sud, Sceaux (May 2017), the XXXVII INSNA Sunbelt conference in Beijing (June 2017), the 3rd EUSN conference in Mainz (September 2017), the 2018 Research Day of the Society & Organizations (S & O) research group at HEC Paris, and the 2018 International Conference on Complex Networks and their Applications, Cambridge, UK (Tubaro, 2018). I am grateful to all participants and reviewers for very helpful and constructive feedback, and I am particularly indebted to Julien Brailly, Emilio J. Castilla, Guillaume Favre, Emmanuel Lazega, Alessandro Lomi and Tom A.B. Snijders for their insightful suggestions. Any remaining errors or inconsistencies are exclusively mine.

## References

Ahuja, G., Polidoro Jr., F., and Mitchell, W. (2009). Structural homophily or social asymmetry? The formation of alliances by poorly embedded firms. *Strategic Management Journal*, 30(9):941–958.

Amati, V., Lomi, A., and Mira, A. (2018). Social network modeling. *Annual Review of Statistics and Its Application*, 5(1):343–369.

Anand, N. and Jones, B. C. (2008). Tournament rituals, category dynamics, and field configuration: The case of the Booker Prize. *Journal of Management Studies*, 45(6):1036–1060.

Anand, N. and Watson, M. (2004). Tournament rituals in the evolution of fields: the case of the Grammy Awards. *Academy of Management Journal*, 47(1):59–80.

Anderson, B. (2006 [1983]). *Imagined Communities: Reflections on the Origin and Spread of Nationalism*. Verso.

Appadurai, A. (1986). Introduction: Commodities and the politics of value. In Appadurai, A., editor, *The social life of things*, pages 3–63. Cambridge University Press.

Arcodia, C. and Whitford, M. (2006). Festival attendance and the development of social capital. *Journal of Convention & Event Tourism*, 8(2):1–18.

Aspers, P. and Darr, A. (2011). Trade shows and the creation of market and industry. *The Sociological Review*, 59(4):758–778.

Blau, P. (1964). *Exchange and Power in Social Life*. Wiley.

Block, P. (2015). Reciprocity, transitivity, and the mysterious three-cycle. *Social Networks*, 40:163–173.

Borgatti, S. (2012). Two-mode concepts in social network analysis. In Myers, R., editor, *Computational Complexity: Theory, Techniques, and Applications*, pages 2912–2924. Springer.
Botsman, R. and Rogers, R. (2010). *What's Mine Is Yours: The Rise of Collaborative Consumption*. HarperBusiness.

Brailly, J. (2016). Dynamics of networks in trade fairs—a multilevel relational approach to the cooperation among competitors. *Journal of Economic Geography*, 16(6):1279–1301.

Brailly, J., Favre, G., Chatellet, J., and Lazega, E. (2016). Embeddedness as a multilevel problem: A case study in economic sociology. *Social Networks*, 44:319–333.

Burt, R. (1992). *Structural Holes: The Social Structure of Competition*. Harvard University Press.

Burt, R. S. (1987). Social contagion and innovation: Cohesion versus structural equivalence. *American Journal of Sociology*, 92(6):1287–1335.

Burt, R. S. (2004). Structural holes and good ideas. *American Journal of Sociology*, 110(2):349–399.

Centola, D., González-Avella, J. C., Eguíluz, V. M., and Miguel, M. S. (2007). Homophily, cultural drift, and the co-evolution of cultural groups. *Journal of Conflict Resolution*, 51(6):905–929.

Delacour, H. and Leca, B. (2011). A salon's life. Field configuring event, power and resistance in a creative field. In Moeran, B. and Pedersen, J., editors, *Negotiating Values in the Creative Industries: Fairs, Festivals and Competitive Events*, pages 36–58. Cambridge University Press.

Everett, M. and Borgatti, S. (2017). Dealing with overlapping categorical attribute data. Paper presented at the XXXVII Sunbelt Conference of the International Network for Social Network Analysis (INSNA) 30 May–4 June 2017 in Beijing, China.

Favre, G. and Brailly, J. (2015). Salons et définition de normes marchandes : Le cas de la distribution de programmes de télévision en Afrique sub-saharienne. *L'Année sociologique*, 65(2):425–456.

Fitzmaurice, C. J., Ladegaard, I., Attwood-Charles, W., Cansoy, M., Carfagna, L. B., Schor, J. B., and Wengronowitz, R. (2018). Domesticating the market: moral exchange and the sharing economy (online first). *Socio-Economic Review*.

Forrest, M. (2017). Establishing the complexity of obesity: The conference as a site of understanding obesity as a medical condition. In Høyer Leivestad, H. and Nyqvist, A., editors, *Ethnographies of Conferences and Trade Fairs: Shaping Industries, Creating Professionals*, pages 87–105. Springer.

Goldhaber, M. H. (1997). The attention economy and the Net. *First Monday*, 2(4).

Gould, R. V. (2002). The origins of status hierarchies: A formal theory and empirical test. *American Journal of Sociology*, 107(5):1143–1178.

Handcock, M. (2003). Assessing degeneracy in statistical models of social networks. Mimeo.

Handcock, M. S., Hunter, D. R., Butts, C. T., Goodreau, S. M., and Morris, M. (2003). *statnet: Software tools for the Statistical Modeling of Network Data*. Seattle, WA.
Hardy, C. and Maguire, S. (2010). Discourse, field-configuring events, and change in organizations and institutional fields: Narratives of DDT and the Stockholm convention. *The Academy of Management Journal*, 53(6):1365–1392.

Hunter, D., Handcock, M., Butts, C., Goodreau, S., and Morris, M. (2008). ergm: A package to fit, simulate and diagnose exponential-family models for networks. *Journal of statistical software*, 24(3):1–29.

Lampel, J. and Meyer, A. D. (2008). Guest editors’ introduction. *Journal of Management Studies*, 45(6):1025–1035.

Lazega, E. and Snijders, T. (2016). *Multilevel Network Analysis for the Social Sciences: Theory, Methods and Applications*. Springer.

Leivestad, H. and Nyqvist, A. (2017). *Ethnographies of Conferences and Trade Fairs: Shaping Industries, Creating Professionals*. Palgrave Macmillan.

Li, Y. and Carriere, K. (2013). Assessing goodness of fit of exponential random graph models. *International Journal of Statistics and Probability*, 2(4):64–74.

Lorrain, F. and White, H. C. (1971). Structural equivalence of individuals in social networks. *The Journal of Mathematical Sociology*, 1(1):49–80.

Lusher, D., Koskinen, J., and Robins, G. (2013). *Exponential Random Graph Models for Social Networks: Theory, Methods, and Applications*. Cambridge University Press.

Manzo, G. and Baldassarri, D. (2015). Heuristics, interactions, and status hierarchies: An agent-based model of deference exchange. *Sociological Methods & Research*, 44(2):329–387.

McPherson, M., Smith-Lovin, L., and Cook, J. M. (2001). Birds of a feather: Homophily in social networks. *Annual Review of Sociology*, 27(1):415–444.

Mizruchi, M. S. (1993). Cohesion, equivalence, and similarity of behavior: a theoretical and empirical assessment. *Social Networks*, 15(3):275–307.

Moeran, B. (2011). The book fair as a tournament of values. In Moeran, B. and Pedersen, J., editors, *Negotiating Values in the Creative Industries: Fairs, Festivals and Competitive Events*, pages 119–144. Cambridge University Press.

Moeran, B. and Pedersen, J. (2011). Introduction. In Moeran, B. and Pedersen, J., editors, *Negotiating Values in the Creative Industries: Fairs, Festivals and Competitive Events*, pages 1–35. Cambridge University Press.

Nowak, M. and Sigmund, K. (2005). Evolution of indirect reciprocity. *Nature*, 437:1291–1298.

Pallotti, F., Lomi, A., and Mascia, D. (2013). From network ties to network structures: Exponential Random Graph Models of interorganizational relations. *Quality and Quantity*, 47(3):1665–1685.

Panitz, R. and Glückler, J. (2017). Rewiring global networks at local events: congresses in the stock photo trade. *Global Networks*, 17(1):147–168.

Phillips, D. J. and Zuckerman, E. W. (2001). Middle-status conformity: Theoretical restatement and empirical demonstration in two markets. *American Journal of Sociology*, 107(2):379–429.

Podolny, J. and Phillips, D. (1996). The dynamics of organizational status. *Industrial and Corporate Change*, 5(2):453–471.
Raafat, R. M., Chater, N., and Frith, C. (2009). Herding in humans. *Trends in Cognitive Sciences*, 13(10):420-428.

Ravenelle, A. (2019). *Hustle and Gig: Struggling and Surviving in the Sharing Economy*. University of California Press.

Robins, G., Pattison, P., and Wang, P. (2009). Closure, connectivity and degree distributions: Exponential random graph (p*) models for directed social networks. *Social Networks*, 31(2):105-117.

Rosenblat, A. and Stark, L. (2016). Algorithmic labor and information asymmetries: A case study of Uber's drivers. *International Journal of Communication*, 10(0).

Sauder, M., Lynn, F., and Podolny, J. M. (2012). Status: Insights from organizational
sociology. *Annual Review of Sociology*, 38(1):267-283.

Schor, J. and Cansoy, M. (forthcoming 2020). The sharing economy. In Wherry, F. and Woodward, I., editors, *The Oxford Handbook of Consumption*. Oxford University Press.

Schor, J. B. and Attwood-Charles, W. (2017). The “sharing” economy: labor inequality, and social connection on for-profit platforms. *Sociology Compass*, 11(8):e12493.

Schüssler, E., Rüling, C.-C., and Wittneben, B. B. F. (2014). On melting summits: The limitations of field-configuring events as catalysts of change in transnational climate policy. *Academy of Management Journal*, 57(1):140-171.

Selznick, P. (1957). *Leadership in Administration: A Sociological Interpretation*. University of California Press.

Slee, T. (2015). *What's Yours is Mine: against the sharing economy*. Scribe Publications.

Smits, R. (2018). Tournaments of values at the European Film Market. *Arts and the Market*, 8(1):64-79.

Snijders, T. A. B., Pattison, P. E., Robins, G. L., and Handcock, M. S. (2006). New specifications for exponential random graph models. *Sociological Methodology*, 36(1):99-153.

Suchman, M. C. (1995). Managing legitimacy: Strategic and institutional approaches. *The Academy of Management Review*, 20(3):571-610.

Torlò, V. J. and Lomi, A. (2017). The network dynamics of status: Assimilation and selection. *Social Forces*, 96(1):389-422.

Tubaro, P. (2018). Emergent relational structures at a “sharing economy” festival. In Aiello, L., Cherifi, C., Cherifi, H., Lambiotte, R., Lió, P., and Rocha, L., editors, *Proceedings of the 7th International Conference on Complex Networks and Their Applications*, volume 2, pages 559–571.

Urry, J. (2003). Social networks, travel and talk. *The British Journal of Sociology*, 54(2):155-175.
<figure><img src="image_15.png" /></figure>
# Whose results are these anyway?
## Reciprocity and the ethics of “giving back” after social network research

Paola Tubaro

## Abstract

Bridging the social networks, field methods and ethics literatures, I make the case that the process of reporting research findings is an ethical issue, and recommends elevating it in the research design. I draw on a reflective account of three research experiences with settings in, respectively, online health communities, economic organizations, and the mainstream media. I proceed in steps, discussing release of personal network results to individual participants, of whole network results to the researched community, and finally of general results to wider audiences, under a unifying idea that a reciprocity obligation underlies the reporting process. I claim that communication should follow an iterative rather than a linear approach to reach all relevant stakeholders, thereby mitigating the vulnerabilities that arise from research.

**Key words:** Research ethics, reporting results to study participants, reciprocity, communication outside academia, social network research, network visualization.

## 1 Introduction: communicating research results as an ethical problem

What ethical issues may arise in the process of communicating the results of a social network study? I faced this question when I was researching the online and face-to-face social networks of persons living with anorexia nervosa and other eating disorders. The study looked at the linkages between their health condition and their social environment, showing that relationships offer support complementary

Laboratoire de Recherche en Informatique, Centre National de la Recherche Scientifique, Paris Saclay, e-mail: paola.tubaro@lri.fr
to treatment, and that web-based groups could be particularly helpful (Tubaro and Mounier, 2014). One day I received an invitation to speak at a popular TV show, whose success relied on a sensationalist approach – juxtaposing people at opposite ends of the body weight spectrum and chastising their unhealthy eating habits. The producers had heard about “pro-ana” websites and expected me to warn viewers about the potential risks of online networking and social influence allegedly triggering disordered eating behaviors. Accepting would certainly earn wide visibility for my study, but would also indirectly endorse the show’s problematic concept – while I knew from my research that judgment and stigma could be far more harmful for sufferers than any contact with (and even influence from) peers. After some hesitation, I eventually declined the invitation, explaining my reasons in a long letter to the show’s producers<fnref n="1" />.

This is just an instance of a more general, but largely overlooked, research ethics problem: where and in what contexts should we, as researchers, draw on our research results to provide feedback to members of the social group we investigated, and more generally to the public? Standard ethical guidelines provide relatively scarce guidance on such issues. Surely, there are general principles that recommend dissemination Iphofen (2011, chap. 12), and several research institutions, publishers, and learned societies have created organizations and adopted codes of conduct (such as COPE, Committee on Publication Ethics) that cover matters such as authorship, originality, acknowledgement of sources, declaration of conflicting interests. Nevertheless, most of these initiatives concern scientific publication, while communication with stakeholders outside the world of research is seldom discussed. Ethical approval procedures usually do require researchers to take into account these stakeholders, but they are undertaken before a study starts and focus on the early stages: key concerns are anonymization, informed consent, and any distress that might occur during data collection, but it is more difficult to anticipate reporting and communication issues that might arise long afterward. Neither is it practical to go back to ethics review boards later for one-off advice: their procedures are too lengthy for media requests that typically come at very short notice, or for unanticipated stakeholders’ reactions that require rapid responses. Further, board members may have limited knowledge of the specific setting and circumstances. In the end, the researcher is to make a decision alone, often in no time.

Reporting social network research outputs is especially tricky insofar as, in recent times, ubiquitous online social networking services and the blurring of boundaries between “public” and “private” spheres (Tubaro et al., 2014b) make people’s relationships widely visible. In the wake of internet-mediated communication at both local and global levels, people have progressively learned to recognize their connectedness and to attach values and status to it (Christakis and Fowler, 2011; Rainie and Wellman, 2012). The potential benefits of an extended range of social contacts are now widely understood, and are among the motivations for using social platforms and applications, so much so that people sometimes overestimate the extent and

<footnote n="1">¹ I made public an anonymized version of the letter: https://web.archive.org/web/20131217023109/http://www.anamia.fr/reponse-a-un-journaliste-tele-ayant-sollicite-une-interview-sur-les-troubles-alimentaires-et-les-sites-pro-ana/</footnote>
reach of their online social capital (Dunbar, 2016). Thus, any research results that make people aware of the relative richness (or poverty) of their social networks are likely to have an effect on their personal life satisfaction and well-being (D'Angelo and Ryan, 2019).

Another reason why in present times, reporting research outcomes is a particu-
larly sensitive part of social network research is that stakeholders who hold setting-
specific knowledge may well re-identify study participants and misuse results. In
their influential contribution to the *Social Networks* special issue on ethical dilem-
mas, Borgatti and Molina (2005) made precisely this point, in the case of organi-
zational network research that managers may use to reward or penalize employees.
Today, increased visibility of relationships through digital technologies extends this
challenge beyond workplaces. Using the above example, websites on eating disor-
ders might expose to unwanted viewers (including potential future employers) not
only their authors, but also their readers.

A related challenge stems from the common confusion between social networks as our research community construes them – as structures of relationships, interactions or interdependencies between individuals and groups – and social networks popularly interpreted as digital tools for social networking (or social media). This is a common source of misunderstandings, especially for those of us who study non-technology-mediated networks, and calls for very careful word choice at every step. Kadushin (2005, p. 139)'s point that "the social network field may have become a victim of its own successes" is still, and perhaps increasingly, true: we benefit from improved visibility but struggle more to get our message through.

This paper aims to advance social network scholars’ endeavor to comprehend the conditions for responsible return of research results. It draws on a reflective account of three cases from my social network research activity. All three concern communication to laypersons: thus, scientific presentations and peer-reviewed publications fall outside the scope of my analysis. I share my field experiences as a step towards a collective effort to co-construct an appropriate ethical framework, in the belief that researchers can (and should) take an active role.

To achieve this, I set out to bridge different literatures, in addition to research
ethics strictly interpreted. Particularly in field methods, there is a large and mature
tradition of reflecting on, and developing best practices for, returning results to the
community where the researcher originally collected the data, and on which those
results might exert an effect. At the crossroads of these different lines of thought, I
frame return of results as a reciprocity obligation – a way of “giving back” to society
in exchange of the resources (be they informational, financial or other) that it made
available to research. Accordingly, return of results addresses not only informants in
the field, but also a wider range of stakeholders, including funders and promoters of
research projects, partners, policymakers and the general public. Instead of a linear
progression in which reporting is just the final stage of research after data collection
and analysis, the process I envision is a feedback loop between stakeholders and
researchers that forms an integral part of research design since inception.

I argue that social network research is an ideal setting to develop these ideas.
While other domains, especially biomedical research, sometimes notoriously shaped
the key principles of ethics for all disciplines, the social sciences may benefit from fresh perspectives that take into account both their substantive and methodological specificities. Social network research is particularly well suited to serve as a terrain of experimentation as, unlike clinical trials, its focus is not on individuals taken in-dependently of one another but on interpersonal relationships that link individuals and groups. It thus introduces fundamental dependencies across people's behaviors and is capable of providing unique insights into society seen as a cohesive whole, rather than as a sum of separate entities.

In addition, social network research helps answer a fundamental question that the literature often leaves open: what is the “community” to which researchers should return results. If it is not limited to the narrow circle of study participants, where should one draw the boundaries? Network studies suggest that a community may be understood as the set of social relationships that are meaningful to a focal indi-vidual or group (Wellman et al., 1988), and prove that mapping such relationships very precisely and assessing how actors make sense of them, can be very helpful. For example in my above-mentioned study of the networks of persons with eating disorders, doctors and other health professionals appeared as alters together with friends and family members, and were therefore targets of my communication as part of participants’ “personal communities” (Chua et al., 2011).

## 2 Bridging literatures

While frequently ignored in established research ethics guidelines, presentation of results has begun to receive a greater deal of attention in recent times, notably in the literature on clinical trials. Returning results to patient participants becomes a greater priority as part of the ongoing shift in healthcare toward participant engage-ment and “patient-centered care” (World Health Organisation, 2007). At the very least, this means communicating aggregate research results to general audiences, in summary form and using language that is understandable and usable by non-specialists. Some regulators have already adopted policies and processes to this end: both the US Food and Drug Administration, and the EU Clinical Trials regulation, require publication of summaries of results for laypersons.

There is also an emergent, though less consensual need to give clinical study participants access to their individual research results, that is, the outcomes of any tests (such as blood tests, scans etc.) performed on them individually. The hope to get such results is a primary motivation for many patients to participate in a clin-ical study at all, especially in a context in which digital self-tracking devices and the services of commercial companies such as 23andMe have accustomed people to easily accessing, handling and visualizing their own health data (Wong et al., 2018). Ethically, return of individual results is a way to ensure participants receive the greatest possible benefit from their contribution to the study. Research bodies such as MRCT in the United States (2017) are issuing guidelines to support release of individual outcomes, although there are disagreements over the optimal timing,
the extent to which results can place a burden on patients, and the need to with-
hold findings in some circumstances. One contested issue is report to the family,
including after the death of the research participant, notably in the case of genomic
research (Wolf et al., 2015).

Calls for open science and open access reinforce the expectation of accessibility of research results to even wider communities – not limited to research participants. Recent debates on open access rely upon the moral framing that requires the outputs of publicly funded research to be freely available to those who paid for it – the taxpayers. This is an argument about obligations of reciprocity (Parker, 2013), whereby researchers receive (financial) resources from the community, and give knowledge to it in exchange. In this perspective, dissemination of knowledge is a key responsibility of researchers and grounds their social legitimacy. The public is implicitly supposed to participate in the process, although this current of thought focuses mostly on scientific publications as such (with issues such as replicability), rather than on outreach to lay audiences (Cahill and Torre, 2007).

These approaches have the merit of replacing presentation of results at the center-
stage of research ethics, shifting from a focus on data collection and data manage-
ment to a more comprehensive view that covers all phases of a scientific study. Yet
report of research results is not an unprecedented outcome of patient-centered care,
as the idea that researchers have a duty to engage in conversations with wider audi-
ences predates the current movement toward open science. Traditional approaches
to the social sciences, not always framed as ethics, have long taken strong normative
stances in this matter. Pierre Bourdieu (2015) believes researchers have a duty to ex-
pose the power relationships that shape society, while Michael Burawoy (2005a,b)'s
“public sociology” advocates dialogue between researchers and diverse audiences
beyond academia. However, these arguments are often cast in too general terms to
help answer specific, practical questions: what is the relevant community? What
results can be made available? What is the appropriate format?

More to the point, ethnography and field methods have long reflected on “giv-
ing results back” to the community where researchers originally collected the data.
Field study design encompasses not only entry into, but also exit from, the target
community, so that sharing of results is naturally constitutive of it. While firmly
grounded in methodology, this literature raises issues that have a strong bearing on
ethics. For example, what is the place (if any) of the researcher’s autobiographical
experience and emotions in the field (Ellis, 1995)? Do participants have a say on nar-
ration of results concerning them? What information can intermediaries, community
leaders, and any other non-participant stakeholders expect to receive? How should
the researcher address any criticism or disagreement? Especially within participa-
tory and advocacy research (MacKenzie et al., 2015), it is increasingly recognized
that researchers have an ethical obligation to share results with the community un-
der study, in order for it to reap a maximum of benefits from participation. Indeed
the potential effects of research on stakeholders may vary from shedding light on
a problem they are facing, to giving them more voice in the policy process, or to
facilitating work towards a solution (Reyes-García and Sunderlin, 2011).
Schurmans et al. (2014) broaden the concept of reporting results, to address not only informants in the field but all who may be concerned, including funders and promoters of research, partners, and fellow researchers. One may also add policy-makers, for their role in steering both research and social action, as well as associations and other civil society bodies, according to the teachings of public sociology (Burawoy, 2005b; Jeffries, 2011). This approach requires re-framing report of findings not as the last stage of a linear process that starts with data collection and terminates with scientific publication, but as a cyclical one with multiple feedback loops. Continuing dialogue with a range of relevant audiences enables researchers to validate their work, builds trust with participants and other stakeholders, and may lead to maintaining relationships over time, possibly through multiple projects (Chavis et al., 1983; MacKenzie et al., 2015). Some social science traditions like action research already operate in such a cyclical mode. In this long-run perspective, social network research may benefit from its capacity to identify the relevant community of stakeholders, as discussed above, to establish what results to return to each of its different components at what time.

These approaches draw on different perspectives and types of disciplinary expertise, but resonate with, and complement, each other. Both the field methods and open science currents build on a principle of reciprocity. Bringing them together enlarges the range of stakeholders to consider: ideally, all those relationships that shape the social environment in which the research is undertaken, and on which it may have an impact. At the crossroads of ethical, scientific and political issues, such a comprehensive view accounts for the multiple commitments that researchers have to diverse actors and the difficulties that arise in the effort of reconciling diverging agendas (Tubaro, 2017).

## 3 Social networks research and case-based reasoning

So far, the social networks literature has not addressed these issues explicitly. The 2005 Social Networks special issue on ethical dilemmas touches on them only indirectly and partially, in terms for example of potential misuses of results and of commitments to problematic sponsors such as the military. Yet social network research exemplifies at best some of the ethical issues that the above discussed literatures highlight. For example, there are specific concerns regarding report of individual results to participants, insofar as revealing the set of ties surrounding a person may jeopardize confidentiality and more generally trigger consequences. Social network research also raises specific problems as a consequence of the power and growing popularity of visualizations, which constitute an effective tool to communicate results but may be deceptive or distressing under some conditions. Further, the common confusion between social networks and online social media adds to the difficulty of communicating results clearly and unambiguously.

Conversely, social network research provides excellent material to work toward, and test the efficacy of, possible solutions that draw on the integrated approach pro-
posed above. Return of individual (ego) networks is already practiced and, with
appropriate ethical safeguards, may pioneer extensions of this practice to the social
sciences at large. Similarly, work on visualizations may improve researchers' ca-
pacity to communicate to audiences beyond academia – and again, social networks
may be a test for solutions of general applicability. Also as mentioned above, net-
work research can help us identify more precisely the community of stakeholders
– who have an interest in the research or on whom the research may have an effect
– whom researchers should interact with.

In what follows, I explore these ideas with the help of three example cases from
my own research experience: one about use of visual sociograms to return individ-
ual results, another about whole network representations of the researched commu-
nity/organization, the third expanding on the TV show dilemma introduced earlier.
A case study approach is most suitable to a situation characterized by limited prelim-
inary evidence and a research context that is not yet clearly specified, in an emerg-
ing domain of investigation (Yin, 2009). I do not limit my inquiry to the exploratory
level, though, and move forward to using the cases at hand to extract more general
arguments, the conclusions of which can be actionable. Thus I make an instrumental
use of my cases, seeing them as concrete instances of the broader issue of interest,
into which they can provide insight (Baxter and Jack, 2008; Stake, 1995). Cases
require in-depth study of particular empirical contexts in which they are embedded,
in order for the researcher to distinguish the general from the specific, to propose
interpretations, and to devise new ideas (Ragin, 1992).

As Crossley and Edwards (2016) maintain, social network research is essentially
a case-study method, where focus is exclusively upon a given, well-demarcated so-
sial group, especially in whole-network designs. Contrary to classical statistical in-
ference, there is no assumption that the case or cases under consideration are repre-
sentative, in a statistical sense, of any wider population. This is one reason why so-
sial network research has caught on in studies of specific, historically or geograph-
ically limited worlds such as organizations and social movements. In principle, the
case study requires engagement with all possible sources of data, methods and ana-
lytic strategies that may shed light on the phenomenon of interest. In practice, it is
often associated with qualitative research, except in social network studies that draw
on a notoriously strong quantitative tradition. Recent approaches lead to greater con-
vergence, whereby qualitative case study specialists use more quantitative methods
(Mills et al., 2010) and network researchers increasingly draw upon multiple data
sources, triangulate observations and build richer pictures of their cases (Crossley,
2010; Dominguez and Hollstein, 2014; Ryan and D'Angelo, 2018).

4 Giving personal network results to individuals

Social network research makes visible patterns of relationships that would otherwise
escape human gaze. Therefore, giving individual-level results back to participants
provides them with unique, otherwise inaccessible information about themselves.
Should we, then, systematically provide study participants with maps of their own networks – representing their “personal communities” in the case of personal network research, or extractions of their ego-centered subgraphs in the case of whole network research?

This practice is already spreading with the help of network visualizations, which are being integrated into different stages of research, from data gathering to presentation of results (D’Angelo et al., 2016). Researchers who have shown participants visualizations of their ego networks note that they shifted their role from being observed to becoming observers themselves (Molina et al., 2014) and felt the need to explain and justify what they saw (Bellotti, 2016). But if people understand visuals, they may take some type of action as a consequence. This is especially likely in a society that values connectedness (Christakis and Fowler, 2011), and in which individuals are aware of the benefits that might accrue from their relationships (Rainie and Wellman, 2012), so that they will have a sense of themselves as more or less successful networkers. Wide availability of social media visualizations until about 2013 has contributed to increasing people’s appetite for this type of information, although most digital platforms have recently restricted access to these data (Hogan, 2018). The potential existence of such effects involves a duty for the researcher to anticipate them and as far as possible, to prevent any drawbacks.

A case from my research illustrates some of the ensuing dilemmas. As part of the above-mentioned study of the social networks of eating disorder sufferers, I used a graphical concentric-circles sociogram structure embedded in a web survey to elicit personal networks (Tubaro et al., 2014a; Tubaro and Mounier, 2014; Tubaro et al., 2016). Participants had to fill in two such sociograms, for their face-to-face and for their computer-mediated ties. The top panels of figure 1 are two examples of these sociograms, representing a face-to-face personal network (left), and an online personal network (right). From these images, drawn by participants themselves, were extracted edgelists and tables with alters’ attributes, for use in analysis. My team and I had initially planned to offer respondents an optimized visualization of their full (face-to-face and online) personal networks at the end of the survey, to reward them and acknowledge their participation. The bottom panel of figure 1 is the layout of one early option we considered (and eventually dismissed), representing online and face-to-face networks as concentric circles as before, but placing them together in the same image as perpendicular plans intersecting in a three-dimensional space (horizontal: face-to-face, vertical: online).

But there were risks. If participants liked the visualization, they would export it and perhaps post it onto their personal pages and social media profiles. If the visualization included the names of their contacts (alters), anonymity would bejeop-ardized; but without names (as is the case in this figure), the visuals might appear confusing and little meaningful, especially with sizable networks and/or major over-laps between online and face-to-face contacts. Another risk would arise if our tool confronted participants to their isolation and loneliness – utterly visible in a graphical representation. At the time (2010), there was no discussion of such a risk in the social networks literature, and the rare previous experiences of sociogram use positively acknowledged participants’ comments on how “interesting their personal
<figure><img src="image_16.png" /><figcaption>**Fig. 1** Graphical, concentric-circle sociograms for data collection (top) and planned return to participants (bottom) in a web survey. Top left panel: a face-to-face personal network, right panel: an online personal network, as could be drawn by a participant. The bottom panel is an unfinished (and anonymized) project to aggregate these same data into one image that participants could export at the end of the survey. Horizontal plan, blue: face-to-face network; vertical plan, green: online network. All the graphs reproduced in this figure are from technical tests and do not represent real persons.</figcaption></figure>

networks look” (Hogan et al., 2007, p. 137). Yet ours was a self-administered web survey where no interviewer’s mediation was available to offer reassurance to participants if needed. We were particularly concerned because we were surveying persons living with eating disorders, whose experiences of illness and hospitalization are known to limit their inter-personal interactions and to disrupt their social functioning (Levine, 2012; Patel et al., 2016). We eventually dropped the plan to give individual results back to participants and never developed the tool depicted in the bottom panel of figure 1.

Later research provides hard evidence of the issues we intuitively anticipated – even outside the health field and in interviewer-led studies. Ryan et al. (2016) interviewed highly-skilled French professionals in London, many of whom appeared quite self-conscious about the visual representations of their personal networks.
Those who thought of themselves as successful, confident networkers experienced a sense of unease when presented with, for example, lack of diversity in their relationships – as a woman who had described her network as “cosmopolitan”, but realized that her visual included almost only co-nationals. Their attitude betrayed a form of vulnerability – that of migrants whose relatively high economic status did not ensure full integration.

In sum, results may reveal (possibly untold) vulnerabilities and trigger reactions that are not observed in more classical participant-researcher interactions. For all their power and appeal, visualizations can be double-edged swords. This is not to say that personal network results should never be shared with study participants. Rather, the lesson learned is that researchers should try and anticipate the conditions that might in some cases lead to distress, and adapt their use of visualizations accordingly.

# 5 Giving whole network results to communities

Let us now turn to a second case study, in the context of a research about networking patterns at an international festival of the so-called “sharing economy”, the annual OuiShare Fest, which took place in Paris from 2013 to 2017. At the 2016 edition, I distributed a questionnaire to participants, with name generators that asked them about their relationships pre-dating the event, the ties formed during its course, and those expected to give rise to further contacts or collaborations in the near future. Almost one year after that, and about one month before the following edition of the festival, I approached the organizers with some preliminary results. I did so to give them confidence that the study was producing relevant outcomes, to renegotiate access to the event and re-run the survey again, and to offer them insights – which I expected to matter to them as professional networking was a watchword of the OuiShare experience. Another motivation was the interest of study participants, which I clearly perceived during fieldwork. Asked in 2016 if she would like to see my (aggregate) results, one of them had replied enthusiastically:

> I think it will be really interesting. The fact that somebody is interested in studying what's happening, provides the feeling that what's happening is really happening, and that it's big enough to be studied [...] So I think if you do that, it will be really huge (resp. 26, 2016).

Indeed OuiShare members were thrilled with the Gephi visuals that I produced from my data. One graph showed that relationships between participants that pre-existed the 2016 OuiShare Fest formed clusters largely disconnected from one another, with many isolates; another revealed that through the new ties formed during the event, most people had become part of a main, densely-knitted connected component.

OuiShare leaders immediately produced an animated gif from my graphs and shared it widely on Twitter. Two of them went as far as printing my visualizations on T-shirts that they wore at the following (2017) edition of the event, labeling them
as “before” and “after”. Interestingly, they advertised themselves to attendees as “human tinders” – the match-makers who, like the well-known app, connect people “in real life” (figure 2).

This particular episode happened not to harm anyone, and it was even rather funny; but it alerted me about potential issues. One brings me back to visualization and its power. It was indeed just those two graphs that caught the attention of OuiShare, not the other visuals I provided, nor the explanations I gave on the side. Those two graphs alone were so successful, partly because I deliberately chose a layout that made very apparent the outcome that OuiShare members were so keen to see – the effectiveness of their event in connecting people. The effect was not spurious, but other graphical choices, such as colors or layouts, could have made it much less visually striking. Although there was no deception here, graph visualizations may lead to selective appropriation by stakeholders, and researchers need to be very careful about their use to communicate results.

Another reason for reflecting on this case is that the key users of my results were not study participants individually, but the organizers of the event where the research took place: among the latter, few actually responded to my questionnaire, while many of them appeared in the network as alters (that is, they were nominated by other respondents). This confirms the tenet of the field methods literature that reporting results to the researched community does not simply mean getting back to participants narrowly defined, and that a more holistic perspective is needed. Network analysis can help refine and extend this approach through its capacity to map who matters to whom, so as to account for the full range of relevant stakeholders, whether they are study participants themselves or not. The fact that my graphs first circulated through Twitter, a social networking service, ensured targeted diffusion to stakeholders within the OuiShare community much more effectively that any publication in a generalist outlet.

When results are shared with a whole social setting rather than with single participants, additional issues may arise depending on the power structure and influence channels within that setting. OuiShare includes prominent community leaders, and their flagship event has played an important role in shaping the “sharing economy”, an emerging and still loosely defined field of activity. They asked me to what extent the results I reported were attributable to their members: were they instrumental in linking participants at the event? From the point of view of their internal management, this would be useful information: “connector” is an official role in their organization and they expect members to be effective networkers. But from a researcher’s standpoint, this was a difficult decision to make. Even if OuiShare prides itself to be a flat, collegial organization of (mainly) freelancers and independent professionals interacting via high-tech digital devices, fear of introducing or exacerbating power relationships prevented me from deanonymizing network nodes to identify the best connectors. Because invisible, informal dominance structures might still be in place behind any formal shape (Krackhardt and Hanson, 1993), with potential consequences for individual members, the same issues might arise as those highlighted by Borgatti and Molina (2003, 2005) in the context of more traditional, hierarchical organizations. As Howard Becker (1978, p. 330) first noted, full
<figure><img src="image_17.png" /><figcaption>**Fig. 2** T-shirts printed by OuiShare members in 2017, with my graphs of networks of ties between participants at the previous edition of the event. The “before” graph refers to ties that pre-existed the event; the “after” one to ties formed during it. I drew the graphs in Gephi using questionnaire data I collected in 2016. The labels, the “Human Tinder” title, and the “Connecting people in real life since 2013” caption were added by OuiShare.</figcaption></figure>

consensus does not exist in any community or organization, and the social scientist may end up pleasing one faction while disserving another, so that “the problem is not to avoid harming people but deciding which people to harm”. Even anonymized results would not be neutral, producing potential tensions if, for example, they revealed that it was outsiders rather than insiders who mostly contributed to linking participants.

In a sense, there was also a risk for the researcher. I gave my findings to this community at an early stage to build trust, receive feedback and negotiate a return to the field. But the unexpectedly wide publicity could have backfired, in that any unhappy members could have obstructed my second wave of data collection. In addition, because my authorship was not acknowledged on the T-shirts (although it was in the animated gif that circulated via Twitter), I feared my re-use of such material in future publications would need to be very carefully thought out. In essence, researchers are to be included among those potentially affected, that an ethical approach should take into account – as will be discussed more extensively later. Extending reporting
efforts beyond individual participants is thus a challenge. Visuals can serve as effec-
tive and easy-to-understand tools, but they may be distressing or deceptive. When
are difficulties most likely to arise? Extant literature (Borgatti and Molina, 2003,
2005) already emphasized the risks that arise in organized settings, and my case ap-
pears as an extension of these ideas to looser organizational forms. Even mild power
or influence relationships produce specific opportunities and threats that researchers
should anticipate as far as possible. My case study also resonates with recent evi-
dence that people attach values and status to connectedness (Christakis and Fowler,
2011; Rainie and Wellman, 2012) – and are therefore keen to demonstrate their role
as actors of this connectedness, possibly using research results to this purpose.

The key is to ensure reciprocity criteria are met, at least to some extent: the
fact that they were in the OuiShare Fest case (the organizers gave me access to
the field, I gave results to them), kept potential risks under control and gave me a
second access to the field, and even further logistical support, one year later (after
which I also returned results to them as I had done before). But in other cases, such
conditions are not met and return of results is more problematic, as discussed in the
next section.

6 Beyond networked communities: sharing results with the media and wider audiences

What about communicating general results to publics that are larger than the specific
social setting in which a study was run (such as the OuiShare community)? Research
institutions increasingly encourage outreach to wider non-academic audiences who
are indirect financial contributors (notably through public funding schemes) and/or
potential beneficiaries (for example, families, neighborhoods or communities of in-
terest). Ethical issues arise as sensitive decisions must be made, such as choice of
sufficiently accurate wording when technical language cannot be used. So far, only
a few institutions and learned societies have published ethical guidelines for respon-
sible press communications (for example the Society for Neuroscience<fnref n="2" />).

As before, I draw on my own research experience to stimulate a discussion. I
already mentioned a case in which I declined to speak at a TV show to avoid risks
of distortion of my findings on online networks and eating disorders. This is a far-
too-common trade-off between two ethical imperatives, one that urges researchers
to reach out to the public, and one that invites to caution in light of the potential
negative consequences of any misrepresentation of results, notably when they have
health implications. Other challenges related to this case speak expressly to the so-
sial network research community. The producers of the show implicitly endorsed
the idea that online relationships are somehow less valuable than face-to-face ones.
As network scholars, we know this is a more complex matter. A long-lasting de-

<footnote n="2">² See https://www.sfn.org/Membership/Professional-Conduct/Guidelines-for-Responsible-Conduct-Regarding-Scientific-Communication, accessed on 27 July 2019.</footnote>
bate has opposed those who stress the “internet paradox” (Kraut et al., 1998) of a communication technology that drives people apart and undermines human propensity towards face-to-face conversations (Turkle, 2012), and those who emphasize seamless integration of the two (Wellman and Gulia, 1999; Wellman and Haythornthwaite, 2002). Even young people are not all savvy with technologies, and down-to-earth inequalities persist online (Hargittai, 2010).

What's more, the show producers took for granted that social influence through peers on the web triggers imitation of unhealthy eating behaviors and is therefore undesirable. Beyond its neglect of influences from non-computer-mediated relationships (Valente, 2010), this view puts the blame on personal networks, without considering the responsibilities of mainstream media that can reach out to much larger audiences. My own research showed that personal networks mediate the reception of media contents, sometimes attenuating their effects, sometimes amplifying them, with different strength and significance depending on individuals' body size (Pallotti et al., 2018). This very insight from my social networks research suggested that, with little time to elaborate and no control on the social environments of the show's publics, it would be wiser not to communicate widely on such sensitive matters.

Alternatively, I could have chosen to attend the show and explain myself; but even if I had been given sufficient time to make my point, my approach could be seen as lecturing others, as if I were in a position of higher standing, rather than just engaging in a mutually beneficial conversation.

This brings me to the reciprocity argument I introduced above. Broadly defined, dialogue with stakeholders is a way for researchers to give back to their social environment in exchange of what they received from it – not just the financial resources they ultimately obtain from taxpayer money, and not just responses from participants to a study, but any form of information and social support. The problem is that the conditions for reciprocity must be in place for it to happen. In the case of this TV show, I could not ensure appropriate “giving back”, as the risk of distortions and sensationalism was too high. Besides, reciprocity was made difficult by the lack of common language and shared understandings of the matter at stake.

# 7 Discussion: dual vulnerability and reciprocation

Building on the thought of Emmanuel Levinas, philosopher Chardel (2013), who was a partner in my eating disorders study, describes the mirroring of instability and risks in respondents (or other stakeholders) and in researchers as dual vulnerability. First, vulnerability concerns study participants: in our research on eating disorder networks, it was mainly due to their health condition, exposure to social stigma, and relatively young age. The second vulnerability is that of researchers who take human, scientific and ultimately legal responsibility when they handle sensitive information which participants (especially if in situations of illness or distress) entrust them with. Chardel's notion of researcher vulnerability stems specifically from French data protection law, which had stringent requirements even before entry into
force of the GDPR in May 2018, all the more so when data contained sensitive information (e.g. on health).

Yet this notion can be easily transposed to regulations in place in other countries, where researchers are subject to Review Boards, with further constraints coming from the legal and risk management services of their institutions (Molina and Borgatti, 2019). Chardel’s view has the advantage of taking into account various sources of researcher vulnerability, deriving from the broad range of ethical and even scientific risks that arise from interactions with multiple stakeholders. These extra sources of vulnerability are looser and less recognizable, but their effects are no less forceful.

Some aspects of my case studies can be understood in these terms. The first (return of personal networks to individuals) suggests that not only participants, but also researchers are vulnerable if presentation of results fails to protect anonymity, because their status and reputation in the profession may suffer. Perhaps in a subtler way, both parties are vulnerable if researchers are unable to reassure participants when findings generate discomfort. Researchers may lack the necessary skills, training, or information to provide support, especially in difficult cases that would require specialized clinical or psychological expertise. Deciding whether to try and give support is in itself a challenge, insofar as social science research is not usually meant to have therapeutic value.

The second case study (return of whole networks to communities) hints at dual vulnerability arising from potential misuse of research results, especially in organized settings. Local leaders may leverage these results for commercial purposes, and even use them to promote their own advantage, possibly to the detriment of weaker members. The researched community is vulnerable to the potentially destabilizing effects of such actions. Researchers are vulnerable too, insofar as they may have unwittingly contributed to developing tools for purposes that do not fit with their values.

The third case (communication of generic results to the public at large) is probably the one where the vulnerability of all parties is at its highest. Misrepresentation through highly-visible popular media may unduly stigmatize the researched population or community while also de-legitimizing researchers if they fail to strike the right balance between the complexity of the problem under study and the need to use simplified language. This is difficult to manage as researchers typically lack control over the final media product (content of the article or report) and its framing (insertion in a broader story, choice of title and accompanying images). Further, researchers are rarely prepared to handle exposure to public debates (although universities and learned societies increasingly offer training and support for press communications).

In all three cases, a reciprocation process may mitigate the emerging dual vulnerability. I argued earlier that a reciprocity obligation underlies the reporting process notably in the context of social network research. Now, it can be added that reciprocation emerges in the mirroring of reduced vulnerability as stakeholders develop trust in researchers (and vice versa) when communication to relevant audiences proceeds safely. The cases presented above show that when the conditions for reci-
procity are met, researchers are fully able to “give back” to the communities that supported their work, while keeping vulnerability low for all parties.

## 8 Conclusions

Researchers' presence in the field is far from neutral and naturally involves some form of feedback to those affected. In what precedes, I have argued that the act of reporting results is an important part of the ethical reflection that should permeate a research project at all stages. Some preliminary plan to share results with participants should be made at the very beginning, even though not all circumstances can be anticipated, and some adjustments will need to be made along the way.

A thorny issue is who to return results to. Under some conditions, research participants may expect to receive their own individual results – notably the structure and composition of their personal network, possibly though not necessarily in visual form. Using the case of a study of users of health-related websites, I highlighted some circumstances in which this is not suitable. Visuals offer an intuitive, effective means for the social network researcher to communicate, but may be misleading or unhelpful in some cases.

Whole network results may be of interest to the researched organizations, groups or communities. Using the case of an event whose attendees I surveyed, I highlighted several ethical risks such as appropriation (and possibly, misuse) of results by stakeholders in a position of power or leadership. These influential members may sometimes raise awareness of a research in a given community, encourage participation, and provide domain-expert feedback on preliminary results. It is thus essential that researchers identify such leaders at an early stage and inform and involve them in all phases of the fieldwork. In cases in which leadership is emergent rather than determined by formal structures, social network research may detect individuals in key positions and help understand which relationships to leverage.

Generalized research results (such as the fact that people's reception of media contents is mediated by their social contacts) are also relevant for the public at large, and researchers are increasingly encouraged to communicate to the generalist press. While this serves laudable transparency and accountability purposes, the risk of misrepresentation of results is often high. As discussed above, it is wiser in some cases to renounce the visibility that media presence offers. This is by no means a call to shy away from the press: journalists can be allies who help find a larger audience for research and may be instrumental to pass the message especially when it can have policy impact. Ideally, researchers should be able to include journalists in their social networks, establishing long-term relationships that help create the conditions for "giving back" as discussed above.

These considerations prompt a reflection on *when* to return results. The Society for Neuroscience, one of the few to have released guidelines for ethical communication with the press, recommends waiting until a finding is peer-reviewed and accepted for publication before announcing it to the public at large. However
in the social sciences, this should not rule out the possibility of earlier release to researched and/or affected communities, as long as the preliminary or descriptive nature of findings is clear. I construe report of findings as a cyclical process involving dynamic, mutual interchange between researchers and relevant stakeholders. I shared my preliminary OuiShare Fest findings with the organizers before showing them to academic audiences. By so doing, I both offered a reward to OuiShare and benefited my research, earning the possibility to go back to the field for a new wave of data collection, and receiving feedback that helped me refine my interpretation. Later, as peer reviews increased my confidence in the results, I communicated more broadly through the OuiShare radio and newsletter. In short, communication with field actors both preceded and followed dissemination to fellow researchers, and each step involved larger circles. While this is just one case and different sequences may be envisioned in other settings, the common challenge is to identify relevant stakeholders and decide what to share with each of them at what time.

Overall, these considerations suggest that there is insight to gain by sharing researchers' first-hand experience and the lessons learned from it. While ethical guidelines may include some general principles of how to communicate with the press, they cannot anticipate all possible issues, and some solutions are difficult to transfer beyond disciplinary boundaries. Learning from one another and working together to create the conditions for appropriate "giving back" is a form of "indirect" reciprocity that defines a more promising way forward.

We could set the example as social network scholars, while also engaging in a collective effort to rethink the way we have practiced our research and branded ourselves since the advent of ubiquitous digital technologies. Some of the difficulties I have discussed derive from an ambiguity in the way outsiders see our work. To be sure, as a group we have benefited from the increasing popularity of social media, as it has turned out very soon that we have the right analytical tools and concepts to analyze them. Our very methods – our points-and-lines graphs, our metrics, and our vocabulary – are more widely understood today than in the pre-Facebook era. And yet, we suffer from the unprecedented imbalance of economic and political power between us – a niche scientific community – and the tech giants that own online social networking sites. It is perhaps time to engage in a more upfront discussion of the matter, to position ourselves more clearly in regard to our objects of study and our methods. It is not only a matter of identity or legitimacy toward our institutions and our funders: we also face consequences in terms of clarity of our message to the public, and capacity to illuminate broad societal issues.

Such considerations do not represent an end-point to this discussion. Our views on how to responsibly communicate our research results are bound to evolve, especially as researchers develop solutions that mitigate the forms of dual vulnerability to which they and their stakeholders are subject. It is important not to leave this to top-down regulators: we need to creatively and collectively contribute to devising an appropriate ethical framework.
## 9 Acknowledgements

This paper is under consideration for publication in the journal *Social Networks*, as part of a special issue on "Recent Ethical Challenges in Social Network Analysis". This is the revised version, submitted to the journal after a request for minor modifications in July 2019. A preliminary draft was presented at the workshop RECSNA2017 that took place in Paris on 5-6 December 2017, organized by BSA-SNAG (British Sociological Association - Social Network Analysis Group) with support from Maison des Sciences de l'Homme Paris-Saclay and Institut d'Etudes Avancées, and in collaboration with RT 26 (Réseaux sociaux) of Association française de sociologie. I am grateful to the whole RECSNA team and attendees, and I am especially indebted to Bruce Cronin, Louise Ryan and two anonymous reviewers of *Social Networks* for their helpful feedback. Any remaining errors or inconsistencies are exclusively mine.

## References

- Baxter, P. and Jack, S. (2008). Qualitative case study methodology: Study design and implementation for novice researchers. *The Qualitative Report*, 13(4):544–559.

- Becker, H. (1978). Problems in the publication of field studies (1st ed. 1964). In Bynner, J. and Stribley, K., editors, *Research Design: the Logic of Social Inquiry*, pages 325–340. Routledge.

- Bellotti, E. (2016). Qualitative methods and visualizations in the study of friendship networks. *Sociological Research Online*, 21(2):2.

- Borgatti, S. P. and Molina, J. L. (2003). Ethical and strategic issues in organizational social network analysis. *The Journal of Applied Behavioral Science*, 39(3):337–349.

- Borgatti, S. P. and Molina, J. L. (2005). Toward ethical guidelines for network research in organizations. *Social Networks*, 27(2):107–117.

- Bourdieu, P. (2015). *Questions de sociologie* (1st ed. 1980). Les Editions de Minuit.

- Burawoy, M. (2005a). 2004 American Sociological Association Presidential address: For public sociology. *The British Journal of Sociology*, 56(2):259–294.

- Burawoy, M. (2005b). The critical turn to public sociology. *Critical Sociology*, 31(3):313–326.

- Cahill, C. and Torre, M. (2007). Beyond the journal article: representations, audience, and the presentation of Participatory Action Research. In Kindon, S., Pain, R., and Kesby, M., editors, *Participatory Action Research Approaches and Methods*, page 196–205. Routledge.

- Chardel, P. (2013). Réseaux numériques et pratiques de soi: le cas des communautés ‘ana-mia’. Réinterroger l’articulation du droit et de l’éthique. *Ethique publique*, 15(2).
Chavis, D., Stucky, P., and Wandersman, A. (1983). Returning basic research to the community: A relationship between scientist and citizen. *The American Psychologist*, 38(4):424–434.

Christakis, N. and Fowler, J. (2011). *Connected: The Surprising Power of Our Social Networks and How They Shape Our Lives*. Back Bay Books.

Chua, V., Madej, J., and Wellman, B. (2011). Personal communities: the world according to me. In Scott, J. and Carrington, P., editors, *The Sage Handbook of Social Network Analysis*, pages 101–115. Sage.

Crossley, N. (2010). The social world of the network: Combining qualitative and quantitative elements in social network analysis. *Sociologica*, 1.

Crossley, N. and Edwards, G. (2016). Cases, mechanisms and the real: The theory and methodology of mixed-method social network analysis. *Sociological Research Online*, 21(2):13.

D'Angelo, A. and Ryan, L. (2019). The presentation of the networked self: Ethics and epistemology in social network analysis (online first). *Social Networks*.

D'Angelo, A., Ryan, L., and Tubaro, P. (2016). Special section on “Visualization in mixed-methods research on social networks”. *Sociological Research Online*, 21(2).

Dominguez, S. and Hollstein, B. (2014). *Mixed Methods Social Networks Research: Design and Applications*. Cambridge University Press.

Dunbar, R. (2016). Do online social media cut through the constraints that limit the size of offline social networks? *Royal Society Open Science*, 3(1):150292.

Ellis, C. (1995). Emotional and ethical quagmires in returning to the field. *Journal of Contemporary Ethnography*, 24(1):68–98.

Hargittai, E. (2010). Digital na(t)ives? variation in internet skills and uses among members of the “net generation”. *Sociological Inquiry*, 80(1):92–113.

Hogan, B. (2018). Social media giveth, social media taketh away: Facebook, friendships, and apis. *International Journal of Communication*, 12:592–611.

Hogan, B., Carrasco, J. A., and Wellman, B. (2007). Visualizing personal networks: Working with participant-aided sociograms. *Field Methods*, 19(2):116–144.

Iphofen, R. (2011). *Ethical Decision-Making in Social Research*. Palgrave Macmillan.

Jeffries, V. (2011). *Handbook of Public Sociology*. Rowman & Littlefield Publishers.

Kadushin, C. (2005). Who benefits from network analysis: ethics of social network research. *Social Networks*, 27(2):139–153.

Krackhardt, D. and Hanson, J. (1993). Informal networks: the company behind the chart. *Harvard Business Review*, 71(4):104–111.

Kraut, R., Patterson, M., Lundmark, V., Kiesler, S., Mukopadhyay, T., and Scherlis, W. (1998). Internet paradox. a social technology that reduces social involvement and psychological well-being? *The American Psychologist*, 53(9):1017–1031.

Levine, M. (2012). Loneliness and eating disorders. *The Journal of Psychology*, 146(1-2):243–257.
MacKenzie, C. A., Christensen, J., and Turner, S. (2015). Advocating beyond the academy: Dilemmas of communicating relevant research results. *Qualitative Research*, 15(1):105–121.

Mills, A., Durepos, G., and Wiebe, E. (2010). *Encyclopedia of Case Study Research*. Sage.

Molina, J. and Borgatti, S. (2019). Moral bureaucracies and social network research (forthcoming).

Molina, J., Maya-Jariego, I., and McCarty, C. (2014). Giving meaning to social networks: methodology for conducting and analyzing interviews based on personal network visualizations. In Dominguez, S. and Hollstein, B., editors, *Mixed-Methods Social Networks Research*, pages 305–335. Cambridge University Press.

Pallotti, F., Tubaro, P., Casilli, A., and Valente, T. (2018). “You see yourself like in a mirror”: The effects of internet-mediated personal networks on body image and eating disorders. *Health Communication*, 33(9):1166–1176.

Parker, M. (2013). The ethics of open access publishing. *BMC Medical Ethics*, 14(16).

Patel, K., Tchanturia, K., and Harrison, A. (2016). An exploration of social functioning in young people with eating disorders: A qualitative study. *PLOS ONE*, 11(7):e1–23.

Ragin, C. (1992). Cases of “What is a case?”. In Ragin, C. and Becker, H., editors, *What is a Case? Exploring the Foundations of Social Inquiry*, pages 1–17. Cambridge University Press.

Rainie, L. and Wellman, B. (2012). *Networked: The New Social Operating System*. MIT Press.

Reyes-García, V. and Sunderlin, W. (2011). Why do field research? In Angelsen, A., Larsen, H., Lund, J., Smith-Hall, C., and Wunder, S., editors, *Measuring Livelivinghs and Environmental Dependence: Methods for Research and Fieldwork*. Earthscan.

Ryan, L. and D’Angelo, A. (2018). Changing times: Migrants’ social network analysis and the challenges of longitudinal research. *Social Networks*, 53:148–158.

Ryan, L., Mulholland, J., and Agoston, A. (2016). Talking ties: Reflecting on network visualisation and qualitative interviewing. *Sociological Research Online*, 19(2):16.

Schurmans, M., Charmillot, M., and Dayer, C. (2014). Introduction du dossier “la restitution des savoirs”. *SociologieS*.

Stake, R. (1995). *The Art of Case Study Research*. Sage.

Tubaro, P. (2017). Les tensions entre sociologie et politique à l'aune d'une tentative de législation des sites web sur les troubles alimentaires. *SociologieS*.

Tubaro, P., Casilli, A., and Mounier, L. (2014a). Eliciting personal network data in web surveys through participant-generated sociograms. *Field Methods*, 26(2):107–125.

Tubaro, P., Casilli, A., and Sarabi, Y. (2014b). *Against the Hypothesis of the End of Privacy: An Agent-Based Modelling Approach to Social Media*. Springer.

Tubaro, P. and Mounier, L. (2014). Sociability and support in online eating disorder communities: Evidence from personal networks. *Network Science*, 2(1):1–25.
Tubaro, P., Ryan, L., and D'Angelo, A. (2016). The visual sociogram in qualitative and mixed-methods research. *Sociological Research Online*, 21(2).

Turkle, S. (2012). *Alone Together: Why We Expect More from Technology and Less from Each Other*. Basic Books.

Valente, T. (2010). *Social Networks and Health*. Oxford University Press.

Wellman, B., Carrington, P., and Hall, A. (1988). Networks as personal communities. In Wellman, B. and Berkowitz, S., editors, *Social Structures: A Network Analysis*, pages 130–184. Cambridge University Press.

Wellman, B. and Gulia, M. (1999). Net surfers don't ride alone: Virtual communities as communities. In Wellman, B., editor, *Networks in the Global Village*, pages 167–194. Westview Press.

Wellman, B. and Haythornthwaite, C. (2002). *The Internet in Everyday Life*. Wiley-Blackwell.

Wolf, S., Branum, R., Koenig, B., Petersen, G., Berry, S., Beskow, L., Daly, M., Fernandez, C., Green, R., LeRoy, B., Lindor, N., O'Rourke, P., Radecki Breitkopf, C., Rothstein, M., Van Ness, B., and Wilfond, B. (2015). Returning a research participant's genomic results to relatives: analysis and recommendations. *The Journal of Law, Medicine & Ethics*, 43(3):440–463.

Wong, C. A., Hernandez, A. F., and Califf, R. M. (2018). Return of Research Results to Study Participants: Uncharted and Untested. *Journal of the American Medical Association (JAMA)*, 320(5):435–436.

World Health Organisation (2007). People-centred health care: A policy framework. Report, World Health Organization, Geneva.

Yin, R. (2009). *Case Study Research: Design and Methods (4th ed.).* Sage.
<figure><img src="image_18.png" /></figure>
# The trainer, the verifier and the imitator:
Three ways in which human platform workers
support automation

Paola Tubaro, Antonio A. Casilli, and Marion Coville

## Abstract

This paper aims to shed light on the role of digital platform labor in the development of today's artificial intelligence (AI), predicated on data-intensive machine learning algorithms. Focus is on the specific ways in which outsourcing of data tasks to myriad "micro-workers", recruited and managed through specialized platforms, powers virtual assistants, self-driving vehicles and connected objects. Using qualitative data from multiple sources, we show that micro-work performs a variety of functions, between three poles that we label, respectively, "AI preparation", "AI verification", and "AI impersonation". Because of its wide scope of application, it is a structural component of automation and an essential part of contemporary AI production processes – not an ephemeral form of support that may vanish once the technology reaches maturity stage. Through the lens of micro-work, we prefigure the policy implications of a future in which automation does not replace human workforce but implies its marginalization and precarization.

**Key words:** Digital platform labor, micro-work, datafied production processes, artificial intelligence, machine learning

<footnote>Paola Tubaro
Laboratoire de Recherche en Informatique, Centre National de la Recherche Scientifique, Paris
Saclay, e-mail: paola.tubaro@lri.fr</footnote>

<footnote>Antonio Casilli
Telecom Paris, e-mail: antonio.casilli@telecom-paris.fr</footnote>

<footnote>Marion Coville
IAE Poitiers, e-mail: mcoville@poitiers.iae-france.fr</footnote>
# 1 Introduction

Recent spectacular progress in research on artificial intelligence (AI) has revamped concerns that date back to the early nineteenth century, when the idea that machines may supersede human labor first spread among scholars, policymakers and workers. A well-publicized prospective literature emphasizes potential job losses due to automation (Brynjolfsson and McAfee, 2014; Chui et al., 2016; Frey and Osborne, 2017). While some scientific research corroborates these predictions (Acemoglu and Restrepo, 2018), other studies highlight historical dynamics of complementarity rather than substitution between human labor and machinery (Autor, 2015; Bessen, 2017), with more complex outcomes such as polarization between high- and low-skilled workers (Autor and Dorn, 2013).

However divergent these analyses may be, they share a quasi-exclusive focus on the expected spillovers of AI to other economic sectors, inferred from long-run industry trends and from known effects of previous waves of labor-saving mechanization. Less commonly discussed is the place of labor in the production of AI itself. Are there any distinctive effects of AI as it is conceived and implemented at the present moment?

The present paper builds on the assumption that the specificity of today's AI production processes, and their unique effects on labor, reside in their role in a "datafied" economy (Mayer-Schönberger and Cukier, 2013). Contemporary AI solutions are predicated on machine learning algorithms with a voracious appetite for data, despite a history of diverse approaches and visions (Domingos, 2017). Viewed as part of a broader drive to accumulate and leverage data resources (Kitchin, 2014), today's AI production reveals its need for human help not only to design cutting-edge algorithms (highly-qualified engineers and computer scientists) but also at a much more basic level, to produce, enrich, and curate data. This is the role of "micro-workers" (Irani, 2015a): barely visible and poorly compensated contributors, who operate remotely online from their computer or smartphone to flag inappropriate web content, label images, transcribe or translate bits of text, read aloud and record short sentences. These are activities that humans can do quickly and easily (whence the "micro" adjective), yet more efficiently than computers. They are fragments of large-scale data projects that support machine-learning algorithms.

Against popular discourses, the very existence of micro-work suggests that today's AI industry is in fact labor-intensive, although under less-than-ideal working conditions – with low pay, no employment contract, and absence of social protection (Berg et al., 2018). Even whilst automation is still in the making and has not yet been deployed at large scale, its demand for micro-tasks is already transforming the daily practices, experiences and career trajectories of thousands of workers worldwide (Gray and Suri, 2019). In the language of (Ekbia and Nardi, 2017), this is an instance of "heteromation", a neologism that stresses how, against a myth of automation capable of liberating people from the need to toil, labor demand is still high but humans operate on the margin of machines and computerized systems. Gray and Suri (2017) call this mixed configuration of machinery and human activ-
ity the “paradox of automation’s last mile”, the incessant creation of residual human
tasks as a result of technological progress.

This (still scant) literature provides only the global picture, though, without look-
ing deeper into the specific functions of micro-work in today’s AI industry. At what
stage(s) of the production chain are humans needed? That is, what micro-tasks sup-
port what processes? Which of these tasks, if any, are temporary solutions to fill
workflow gaps that will be probably resolved in future? Which ones, instead, fulfill
structural needs and are therefore likely to be permanently needed? Are there any
tasks or processes where human intervention is more embarrassing for AI produc-
ers – and which, therefore, are more likely to obfuscate its role? Answering these
questions is important to unpack the nature of the linkages between artificial and
human intelligence, and the re-organizations of labor that data-powered technolo-
gies induce. A better understanding of these processes is also important to inform
policy action: if demand for micro-work is a transitory phenomenon, short-term
measures will suffice, but if not, a more profound re-thinking of labor protection
will be needed.

In this paper, we review a wide range of micro-working activities and show that they perform not just one, but a continuum of crucial functions, between three poles that we label, respectively, “AI preparation”, “AI verification”, and “AI impersonation”. We conclude that because of its wide and diverse scope of application, micro-work is a structural component of automation and an essential part of data-intensive AI production processes – not an ephemeral form of support that may vanish once the technology reaches maturity stage. In short, it is here to stay.

Our findings invite platforms and regulators alike to take concrete steps in regard
to the working conditions, remunerations and career prospects of the people who toil
behind the successes and promises of present-day AI. This requires a major effort to
raise awareness and change mindsets, insofar as micro-work has remained largely
out-of-sight so far – not explicitly considered even in otherwise laudable attempts
to develop ethical principles for AI (Jobin et al., 2019).

2 Micro-work as an instance of digital platform labor

Let us start with some background on micro-work. Like other forms of digital labor (Casilli, 2019), it is an outcome of the emergence of platforms as devices to coordinate economic activity between service providers and clients – both construed as independent businesses that endeavor to make a one-off deal, rather than parts of a long-term employer-employee relationship. Platforms support large-scale outsourcing by enabling client companies to access workforce on demand, at a fraction of the cost of salaried staff, and usually with quicker turnaround times. They advertise themselves to clients as AI-service vendors, and to workers as providers of online earnings opportunities.

The most famous micro-work platform is Amazon Mechanical Turk, originally
an internal service that the company developed to remove duplicates from its cata-
logue. It turned out that distributing tasks to a broad set of people working in paral-
lel, was much more effective than any attempt to solve the problem algorithmically.
Toward the mid-2000s, Amazon opened its service to external users, and is now an
intermediary in a two-sided market (Rochet and Tirole, 2003) of workers and “re-
questers”. Many more platforms, such as Microworkers at international level, and
Foule Factory<fnref n="1" /> in France, adopt this very business model today, with minor vari-
ants. Few micro-work platforms serve a single monopsonist, usually a major tech
company: TryRating for Apple, RaterHub for Google, and UHRS (Universal Hu-
man Relevance System) for Microsoft. There are also mixed models: the German
Clickworker offers both a marketplace like Amazon and a managed service for more
demanding clients, as part of which it recruits workers for, and administers access
to, UHRS.

Additionally, micro-work platforms differ in size and scope. Some are tiny start-
ups, others have grown to multi-nationals, such as Appen, a publicly-traded com-
pany headquartered in Sydney which has lately acquired other major players, no-
tably the Americans Leapforce in 2017 and Figure Eight (formerly called Crowd-
Flower) in 2019. Finally, some platforms such as Mechanical Turk and Foule Fac-
tory cater to a diverse range of corporate needs in addition to AI, others such as the
French ISAHit propose a narrower set of services including an explicit AI offer, and
a growing number specialize exclusively in AI services (Schmidt, 2019). The latter
often involve alliances between a company that specializes in sales to AI companies,
and another that manages micro-workers: for example Figure Eight and Clixsense,
Mighty AI (acquired by Uber in June 2019) and Spare5, Wirk and Foule Factory.

In the typology proposed by Schmidt (2017), and re-elaborated by Berg et al. (2018), platform micro-work is an instance of “cloud-work”, performed remotely online. It differs from the other main variety of cloud-work, web-based freelancing, which concerns creative work (such as graphic design and software development), involves qualified professionals, and entrusts them with whole, relatively long projects rather than single short tasks. Examples of freelancing platforms include Upwork and Freelancer.com. Another way to see the difference between micro-work and freelancing is that the former is dispersed to an undefined set of anonymous, replaceable contributors via the platform, rather than assigned in full to a selected, identified contractor. For this reason, micro-work is sometimes referred to as “crowd-work” or “crowdsourcing”, contraction of “crowd” and “outsourcing” – a term that, however, wipes away the fragmented nature of the tasks and the small remunerations they attract (as low as one or two cents).

Both forms of cloud-work differ from “gig” labor where platform-mediated ser-
vices are performed offline, even though coordination occurs online: for example,
driving for Uber, delivering food for Glovo, cleaning for Helpling. Yet one form
of gig-work, which Schmidt (2017, p. 7) calls “local micro-tasking”, is similar to
micro-work insofar as it consists of small tasks (such as taking pictures of products
in shops) given to an unspecific set of providers. Local micro-tasking platforms with

<footnote n="1">¹ “Foule” means crowd in French.</footnote>
operations in varied European countries are Roamler and BeMyEye, while Click-worker offers both online and local micro-tasking.

Some of the tasks that micro-workers do are similar to the unpaid contributions of many users of digital technologies who unwittingly feed into tech companies' databases with their everyday routines. The reCAPTCHA algorithm that Google deployed to tell apart people from bots famously had the accessory purpose of leveraging human computation to recognize words and characters that software could not read (von Ahn et al., 2008). The difference is that micro-work platforms pay – and even if rates are low, they are enough to affect users' motivations and clients' expectations.

The nascent literature on platform labor has often conflated micro-work with other forms of platform-based digital labor, especially freelancing. Indeed some (online-only) micro-work platforms occasionally make available more qualified tasks, such as translations or text-writing, and conversely some freelancing platforms happen to publish simpler, lower-compensated tasks. This approach has been useful in a recent valuable effort to assess the size and growth rate of the whole online (or cloud-based) global labor market (Kässi and Lehdonvirta, 2018). However, this obscures the linkages with AI production. There is now a need to decouple micro-work from other forms of platform labor, which serve a range of economic and societal needs – from urban transportation to companies' branding – which do not necessarily tie in to automation. How do paid micro-tasks precisely affect the data economy behind automation, and how does their role in this particular production chain distinguish them from platform work more generally? For greater clarity and focus, we address this question in reference to online-only micro-work, leaving aside local micro-tasking.

# 3 The labor-for-data needs of machine learners

AI producers are companies, start-ups and research labs that use machine learning to develop applications ranging from chatbots and hands-free vocal assistants, to automated medical image analysis, self-driving vehicles and drones. Let us first review the basic functioning of machine learning, and derive preliminary conjectures on where and when it may need human intervention.

## 3.1 How data fuel machine-learning algorithms

At the crossroads of informatics and statistics, machine learning “teaches” computers to find solutions from data, without each step having to be explicitly programmed (Alpaydin, 2014, 2016). Its quality can get progressively better over time, depending not only on the algorithm but also on the data given to it. For example, development
of a vocal assistant requires huge audio datasets with examples of potential user requests (like “turn on the light in the kitchen”, “call mum”, etc.).

So-called “supervised” machine learning algorithms, the most widely used in both research and industry to date, need not only high-quantity, but also high-quality data, that is, complete with annotations. Supervised machine learning aims to infer a function that maps an input to an output based on exemplary input-output pairs (“training” dataset). The learned function must be able to assess new cases in “test” datasets. For example, to teach a computer to distinguish between images of dogs and other animals, one would need a training dataset that associates each image (input) to an annotation, such as a tag that says whether the image shows a “dog” or “other” (output); after having been exposed to many tagged images, the algorithm will be able to classify new, untagged images and determine whether they represent dogs. The more accurate the tags in the training dataset, the more the solution can be fine-tuned and generalized to a wide range of real-world cases.

Despite its apparent simplicity, the image recognition classification algorithm just outlined has far-reaching implications, for example in the development of self-driving cars, which need to recognize objects such as a dog crossing the street, before they can make decisions. A state-of-the-art example of the supervised family is “deep” learning, which analyzes data through a layered structure of algorithms inspired by the neural network of the human brain, leading to more effective learning.

Less demanding in terms of data quality is “unsupervised” machine learning, where data have no labels, and the algorithm is left to find its underlying structure based on common patterns. Two main types of algorithms can be distinguished, dimensionality reduction which consists in mapping a multidimensional dataset into more interpretable two-dimensional structures, and clustering, which groups observations into coherent classes. Today’s unsupervised learning brings to the next level some quantitative techniques traditionally used in social science, namely factorial analysis and hierarchical clustering (Boelaert and Ollion, 2018). Like these older tools, it is often used when the objective is unclear, or for exploratory analysis. It is unlikely to wipe out the supervised variant, because there are still many tasks it cannot do. Besides, interpretation of results can be problematic due to lack of objective standards to judge algorithmic performance.

A third family is reinforcement learning, a formalized version of human trial-and-error which uses mapping between input and output like supervised learning, but unlabeled data like unsupervised learning. It includes a feedback loop that gives the algorithm positive and negative signals, so that it adjusts accordingly. Because of its massive data needs, reinforcement learning is often applied to domains such as games, where simulated data are relatively easy to source.
## 3.2 A structural demand of micro-work for AI data preparation?

The above summary suggests that AI companies depend heavily on data resources, including not only raw data but also annotations that add extra meaning by associating each data point, such as an image, with relevant attribute tags. To account for these dual aspects, we propose to distinguish data generation and data annotation<fnref n="2" /> as two separate sub-processes in the production chain of AI, even though they have much in common. Both are part of the preliminary phase, the first stage of the AI production chain – which we label “AI preparation”. The problem is that both are challenges for AI producers, despite a widespread rhetoric of “data deluge” (Anderson, 2008): the right data are not always available or accessible, and when they are, they often lack suitable annotations, and need some interventions before they can be used. On this basis, we can formulate a preliminary expectation, to guide our empirical analysis. It is that micro-work caters precisely to these unmet data needs: it contributes to AI preparation, both in terms of data generation and data annotation. Micro-work is an input to AI in the current data economy.

Our other expectation is that micro-work is a structural rather than a temporary input to AI production. We are wary of the views of some technology enthusiasts who believe that data generation and annotation tasks will ultimately be fully automated. The “heteromation” paradigm (Ekbia and Nardi, 2017) implies instead that some essential tasks will always be directed to humans as indispensable though hidden providers, any solution resulting from a division of labor between the two. Our above review of machine learning techniques corroborates this line of thought, suggesting that their huge data needs, which grow with every new challenge, will keep demand for micro-work high in the foreseeable future.

## 4 Insights from fieldwork

Our expectations are not hypotheses to be tested stricto sensu. We take them simply as the starting points with which we enter the research setting, the prior assumptions that derive from the still limited social research on AI and platform labor, and from contextual knowledge of the basic linkages between AI, machine learning, and data. Being reflexively aware of these initial hunches is a guide toward comparing them to stakeholders’ discourses and their actual experience, but does not engage us to stick to them. Rather, we use empirical evidence to enrich and substantiate these preliminary expectations, to complexify them, and if necessary to revise them, in an iterative, emergent process.

The data we use are from DiPLab (“Digital Platform Labor”), a mixed-methods study of tech companies and of the day-to-day routines of platform workers that took

<footnote n="2">² For the purposes of this paper, it is unnecessary to distinguish between training, validation and test data as is often done in machine learning: all data need to be generated and, in the case of supervised learning, annotated.</footnote>
place in 2017-18 in France (Casilli et al., 2019). A country with traditionally high technological and scientific development, France is currently the second European country by number of AI start-ups (MMC Ventures, 2019), after significant public and private investments (French Government (Ministry for the Economy, Ministry of Education and Ministry of Digital Technologies), 2017; Villani et al., 2018). In addition to its inherent interest, focus on France enables to extend our gaze beyond the high-profile platforms, particularly Amazon Mechanical Turk, which have been overrepresented in the literature to date, despite being little used outside the United States. With its numerous, competing players, some of which operate only within national boundaries, France is exemplary of a trend toward diversification and specialization of micro-work platforms, attracting an ever-wide range of users.

We combine, on the one hand, insight from AI producers and micro-work platforms, and on the other hand, the views of people who perform micro-tasks online. We triangulate information obtained from these different stakeholders in order to reach greater consistency and completeness, to cross-check findings and to corroborate them. We do so because each of these stakeholders has a different perspective and, taken in isolation, would provide only a partial and incomplete view. Leaders and staff of AI companies have the best understanding of technology and its needs, but as we will show, they are often unwilling or unable to disclose their use of human workers. In turn, micro-work platforms that mediate between AI companies and workers, are best positioned to know the structure of the market, but their need to attract clients and investors may bias their communication strategies. Finally, workers can share the unique, concrete experience of doing micro-tasks, but they are not always aware of their purposes and final uses.

To uncover the viewpoint of AI producers and of the platforms/vendors that supply data laborers to them, we use primarily an inventory of micro-work platforms and related AI data service vendors. Although focus is on France (Wirk - Foule Factory, IsAHit), the inventory also includes information on international platforms whose scope of activity is global and includes France (like Appen, Clickworker, Li-onbridge, Mechanical Turk, Microworkers), and for comparison purposes, it adds a few AI start-ups with more limited penetration in France (like Mighty AI). We compiled this inventory using desk research. We explored industry reports, newspaper articles, and most importantly the websites, press kits, and other communication tools of the platforms and companies concerned. To validate our findings from this material, and to gain more insight into less publicized aspects, we use in-depth interviews with 3 French clients and platform owners.

To account for micro-workers' perspective, we rely on a survey that we distributed in 2018 as a paid task on Foule Factory, collecting 908 unique, complete responses. For the purposes of the present paper, we use only one particularly relevant open-ended question, which asked micro-workers to describe in their own words the last task done online. We coded responses independently and cross-checked our categories for greater reliability. Here, we analyze results qualitatively, to identify common patterns; a quantitative description can be found in Casilli et al. (2019). We also use in-depth interviews with micro-workers. We invited a sub-sample of 72 questionnaire respondents to a follow-up interview of 30 - 60 minutes. We did
another set of interviews (60 - 120 minutes) with 14 French micro-workers active on varied international platforms such as Appen, Clickworker and Lionbridge, and with 3 African micro-workers who do tasks for French requesters through IsAHit. All interviews have been audio-recorded and accompanied by a written report by the interviewer(s).

# 5 Micro-work for AI preparation

Let us start by looking at the role of micro-work platforms in the provision of data generation and annotation services for AI companies. Appen says openly that effectively harnessing the power of machine learning requires human skills<fnref n="3" />. Lionbridge AI sells “Machine intelligence, powered by humans”<fnref n="4" />. We first look at these value propositions through platforms’ communication materials, before turning our attention to the views of the underlying workforce.

## 5.1 Platforms’ offer of AI preparation services

An example of data generation that most platforms advertise to their clients is audio utterance collection, important to train voice-controlled devices. Platforms can leverage their contributor base to gather this data with a variety of vocal timbres, regional accents, uses of slang, and contexts (such as background noise). Platforms that operate at global level can replicate the data collection in different languages – Appen boasts over 180, Lionbridge 300. Platforms that operate at national level also have advantages: a vocal assistant to be sold in France, for example, must be trained in the country to learn French accents, the names of French cities and personalities, and other local specificities. The market seems small, but the process can scale: a producer of vocal assistant software that we interviewed, has built an application allowing users to customize the assistant to their needs. It integrates a “data generation” functionality through which users can request bespoke datasets: the company manages the order by passing it to a standard platform such as Mechanical Turk or Foule Factory, monitoring execution and ensuring delivery<fnref n="5" />.

Platforms present data annotation as their core offer to clients. With sound or text data, they propose services such as categorization of topics in a conversation, determination of emotions behind a statement, classification of intents, and identification

<footnote n="3">³ Appen website, consulted on 15 March 2019, URL: https://appen.com/why-human-annotated-data-key/</footnote>

<footnote n="4">⁴ Lionbridge website, consulted on 14 March 2019, URL: https://www.lionbridge.com/artificial-intelligence/</footnote>

<footnote n="5">⁵ In passing, we noted that this is a challenge for start-up engineers who, trained in technology, often feel unprepared to take on the community management or HR roles they would need to engage with human workers.</footnote>
of parts of speech. With images and videos, the offer includes assignment of im-
ages to categories, detection of objects within images with dedicated tools such as
bounding boxes (rectangles around the objects of interest), cuboids (3-D bounding
boxes) or polygons (precise drawings around objects of interest, possibly of irreg-
ular shape), addition of in-image tags to each object, and labeling of anatomical or
structural points of interest (like eyes in faces) with so-called “landmark annota-
tion”.

Technology moves fast, and computers are now pretty good at tasks that seemed unsurmountable even just a few years ago, such as (to use the above example) telling apart a dog from another animal. Human capacity is now in demand to recognize details and nuances, indispensable to increase the precision of software for sensitive applications such as autonomous vehicles and medical image analysis. A state-of-the-art technique is semantic segmentation, much more precise than those mentioned above because it involves painstakingly separating every pixel of an image into the parts that a computer vision algorithm will have to recognize. On Lion-bridge’s blog, a machine learning specialist speculates that pixel-accurate annotation is becoming the new norm, while rougher tools such as bounding boxes may eventually disappear<fnref n="6" />.

Such accuracy would be impossible if workers had to draw shapes with the func-
tionalities of standard software. Micro-work platforms such as Appen and Lion-
bridge compete fiercely to develop cutting-edge tools, themselves based on ma-
chine learning, that increase the precision, scale and speed of human data annotators.
Some AI start-ups have also joined the race, usually focusing on technological de-
velopment and using one or more standard micro-work platforms to access human
contributors. One solution consists in having workers manually label a sub-set of
data, and then letting an algorithm learn those annotations and applying them to the
rest of the dataset. Another relies on an automated tool that roughly pre-annotates
objects (for example, by forming lines around cars in a traffic image), so that the
worker only needs to adjust the details. Figure Eight’s “active learning” distributes
labor between humans and machines:

Computers can automate a portion but not all of the data, thus requiring a human-in-the-loop workflow. In this environment, computers can complete the high confidence rows and humans the lower confidence<fnref n="7" />.

To summarize, technological progress has not eliminated the need for micro-
tasking, but transformed it, integrating humans and computers more tightly. These
evolutions accompany the growth of the business of AI preparation: the industry
think-tank Cognilytica estimates the worldwide market for what we call data genera-
tion and annotation at over $500M in 2018, expecting it to rise to $1.2B by 2023. As
part of this trend, the market for third-party data solutions attained $150M in 2018,
and will exceed $1B by 2023 (Cognilytica, 2019). According to Lukas Biewald,

<footnote n="6">⁶ Lionbridge website, consulted on 17 August 2019, URL: https://lionbridge.ai/articles/why-pixel-precision-is-the-future-of-image-annotation-an-interview-with-ai-researcher-vahan-petrosyan/</footnote>

<footnote n="7">⁷ Figure Eight website, consulted on 14 March 2019, URL: https://success.figure-eight.com/hc/en-us/articles/202703295-Getting-Started-Ideal-Jobs-for-Crowdsourcing</footnote>
founder of Figure Eight, the recent rise of deep learning has boosted demand, be-
cause its complex algorithmic structures require much larger (labeled) data sets than
other machine learning techniques:

Deep learning has been fantastic [...] We began noticing deep learning when we started having customers who would ask for tens of millions of data rows right off the bat.<fnref n="8" />

5.2 Micro-workers’ experience of AI preparation

The concrete experience of micro-workers broadly confirms preliminary insight
from platforms' communication. Our online survey provides evidence of data gener-
ation in the form of voice recordings: many participants reported having read aloud
a few short sentences in French and audio-recorded them. Variants of this task in-
clude requests to record, say, five ways to ask a virtual assistant about the weather.
Some micro-workers understood that this was "to help design intelligent virtual as-
sistants controlling connected objects" (L.<fnref n="9" />). This task requires large numbers of
participants to ensure sufficient variety, and it is unlikely to change much over time,
in that linguistic skills and local knowledge cannot be easily replaced or outsourced
to offshore providers.

Data annotation tasks are also common. Workers are often asked to classify ob-
jects such as DVD titles, photos and “virtual avatars” (R.). Sometimes, they have
to associate images and names of commercial products as found in multiple online
marketplaces – clearly to teach computers to recognize essential similarities (the
product is always the same) despite dissimilar contexts (the websites differ). CV
anonymization, reported by almost one fifth of respondents, was understood to be
about “removing all distinctive marks that could be discriminatory” (C.). Of note,
workers also had to tag the spaces in the document where names, birth dates and
addresses were placed originally – arguably to help some recruitment algorithm to
understand the structure of a CV. This is another task that requires local knowledge
insofar as job application standards vary across countries.

In terms of image annotation, some respondents mentioned a task they called
"motocross" where they had to identify different roads and tracks in photographs
and to indicate the nature of the ground (pebbles, road, sand etc.). Some thought
it was for a video game, others for a census of racetracks. This is because, as we
soon realized, requesters vary widely in the extent to which they provide detailed
information on their tasks, and on the purposes they serve, leaving workers often
confused. A more dramatic example of the consequences of erratic information from
clients is a task that asked micro-workers to tag vegetables (tomatoes, carrots, etc.)
in pictures of salads. M. (30 years old, married, resident of a mid-sized city, full-
time teacher and micro-worker in her spare time) found this task "silly" but simple

<footnote n="8">⁸ Interview of L. Biewald with B. Lorica, O'Reilly Data Podcast, 4 May 2017, URL: https://www.oreilly.com/ideas/data-preparation-in-the-age-of-deep-learning</footnote>

<footnote n="9">⁹ Names of respondents have been changed to initials for confidentiality.</footnote>
and adequately paid for the limited effort it required. She grasped that it served to develop some software application for nutrition. But D. (25 years old, unmarried, living in a rural area, unemployed) could not make sense of it:

> They tell you: draw a circle around a tomato. We don't know why. I think everyone knows what a tomato is, I hope [...]. Then I think to myself: if it's there, it must be useful to someone, for something, but... Why, I don't know.

A type of task that did not surface in the questionnaire, but was mentioned in interviews, consists in categorizing videos, text and images circulating on the web. Flagging violence and pornography serves to train algorithms for content moderation. After the attacks of 2015-16, A. checked “monstrous” terrorist content for several weeks, 30 hours a week, as clients “were panicking”. Exposure to this content can be distressing (Roberts, 2019), although A. assures that she has found ways not to be personally affected. Only a small part of content moderation can be automated: any new types of data (here, videos) first require micro-workers to train future automated solutions.

In sum, micro-workers' experience confirms their important contribution to data generation and annotation for AI, although they are not always aware of it, and suggests that this role is neither temporally nor spatially concentrated.

## 6 Micro-work for “AI impersonation”

Our prior expectations about the linkages between micro-work and AI did not factor in scandals, yet examples abound. In 2019, investment firm MMC Ventures reviewed over 2,800 purported AI start-ups across Europe, and found evidence of AI consistent with their value proposition in about 60% of them. The newspapers that covered the story were eager to stress that, well, a whopping 40% of these start-ups do not do AI (see for example Ram (2019)).

The year before, we had heard similarly outraged voices – not from micro-workers, who often lack awareness of the ultimate goals that their activity is serving (subsection 5.2) and are therefore ill-positioned to judge whether an alleged AI is genuine. We interviewed K., a Parisian entrepreneur and start-up founder who blamed his competitors for their claim to do AI while, instead, they outsource all work to humans recruited through platforms overseas. He went as far as to claim that “Madagascar is the leader in French artificial intelligence”. Even more upset was S., a student who did an internship in an AI start-up that offers personalized luxury travel recommendations to the better-off. His company’s communication strategy emphasized automation, with an alleged recommender system based on users’ preferences extracted from social media. But behind the scenes, it outsourced all its processes to micro-providers in Madagascar. It did no machine learning, and the intern could not gain the high-tech skills he had dreamt of.

Why do start-ups cheat? One hint comes from the prices that data vendors post online (only a few of them do, while the majority propose only personalized quotes
to individual clients). Top-quality data are expensive: semantic segmentation costs a few dollars per image, compared to bounding boxes that are priced less than a dime, and simple categorizations that are available for one or two cents. Costs go further up if accuracy of results is sought, for example by having each data point annotated by multiple workers. To this, start-ups should add the costs of powerful hardware and the salaries of highly qualified computer scientists. Under pressure to perform, they may find it cheaper to fragment the work into micro-tasks and sub-contract them to low-paid workers through platforms.

If so, there can be another role for micro-work in addition to AI preparation, and we propose to label it “AI impersonation”. It happens when humans, so to speak, steal computers’ jobs. After all, this is the very idea behind Amazon Mechanical Turk, the platform that first popularized micro-work. Its name is that of a fake chess-playing machine built in the late eighteenth century and dressed in seemingly oriental clothes, but in fact operated by a human player hidden inside. That Amazon dubbed its creation “artificial artificial intelligence” is also indicative of its intent: filling the gap of what artificial intelligence is expected but unable to do (Irani, 2015b).

Seen in this way, impersonation is not just about fraud, and indeed Amazon has always been upfront about it. It is an interpretation of the “human-in-the-loop” that makes workers hardly distinguishable from algorithms. Amazon’s original idea was to allow programmers to seamlessly integrate the two into their processes, whereby managing a task for “Turkers” would be similar to sending a remote request for an algorithm to execute. Today, the typical system has some sort of self-monitoring capability, which allows it to recognize situations in which it has doubts, or in which it cannot autonomously bring an activity to completion: in these cases, it hands control over to a human operator. This is the approach followed, among others, by Google Duplex, a conversational assistant that makes restaurant reservations, where up to 25% of calls are made by humans as of May 2019 (Chen and Metz, 2019). An apparent deception, it is nevertheless a way to gradually train the assistant<fnref n="10" />. In some cases, impersonation may involve qualified labor rather than micro-work. The creators of Julie Desk, a French start-up producing an email-based scheduling assistant, initially did the job by hand, in place of an algorithm that had yet to be coded:

> We worked as assistants ourselves for a period of 8 months, and manually answered all the requests we received! It allowed us to understand what the recurrent patterns in the meeting scheduling process were and then, with the help of data scientists, we coded them to give birth to Julie<fnref n="11" />.

Micro-work platforms do provide human labor force to meet these needs, but tend not to explicitly advertise these roles: they arguably negotiate impersonating tasks individually with clients as part of their managed service packages. Humans

<footnote n="10">¹⁰ Google AI Blog, accessed 15 March 2019, URL: https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html</footnote>

<footnote n="11">¹¹ Julie Desk blog, accessed on 15 March 2019, URL: https://www.juliedesk.com/blog/artificial-intelligence-empathy/</footnote>
are always in the loop, but they are even less visible here, than when they do “prepa-
ration” tasks. To take these aspects into account, it is important to include imperson-
ation in our framework: it is not just a temporary strategy to keep afloat an insuffi-
ciently funded start-up, but part of the “heteromated” system that increases demand
for human workers with every new problem to be solved, while keeping them in
marginal and unrecognized roles.

7 Micro-work for AI verification

Even after inclusion of AI impersonation together with AI preparation, it appears
from our interviews there are more services that micro-workers provide to AI. In
spring 2019, public outcry followed the revelation in the news that human workers
listen to users' conversations with smart assistants (see for example Hern (2019)).
The year before, we had interviewed J., a transcriber who worked for six months
to improve the quality of the French version of one of these virtual assistants, pro-
duced by a tech multinational and already in commerce. Her job was to check that
the virtual assistant correctly understood what its users said. She listened to au-
dio recordings (usually short tracks, averaging between 3 and 15 seconds), then
compared her understanding to the automated transcription produced by the virtual
assistant. If the transcription was inaccurate, she had to correct it: any misunderstand-
ing, conjugation or spelling mistakes had to be highlighted. Another part of
her work consisted in adding tags to the transcribed text indicating any sounds or
events that could explain the virtual assistant's performance – why some sentences
were well understood, some not. J. knew that fellow transcribers were doing these
tasks in other European countries and languages, all following the same guidelines.

Because the role of J. was undisclosed to users, her case reminds of imperson-
ation, but the difference is that she was not replacing a failing algorithm: the one she
checked for quality was up and running. She realized that the results of her work
would help engineers and computer scientists to ensure the virtual assistant would
not make the same mistakes in future. In this sense, the case of J. also reminds of
what we called AI preparation, with the difference that she was intervening “post-
robot”: she produced training data from the amended outputs of an already-trained
algorithm. Therefore, we propose to call this case, similar yet not identical to the
other two, “AI verification”.

We found other examples of AI verification in our fieldwork. A., the micro-
worker who moderated violent content (section 5.2), also did relevance scoring.
This type of tasks consists in assessing the extent to which the outputs of search en-
gines or conversational agents are relevant to a user's request, and sending feedback
to help computer scientists improve their model. Another post-robot task consists in
checking the results of optical character recognition (OCR) software. For example,
a firm that aims to digitize its invoices may scan the original documents, use OCR
to convert the resulting images into character codes, and get human help to look at
the outcomes, fill the gaps, and make corrections if necessary. We interviewed three
African micro-workers who did such tasks for French clients through the Paris-based platform IsAHit. In both cases, humans intervene well after the preparatory phase, when the AI solution has already been trained, tested and brought to market.

Micro-work platforms do advertise services such as relevance scoring and trans-
scription checking to their AI-producing clients, but do not group them together in a
separate category corresponding to our AI verification. Partly, this is due to the same
reasons that keep them quiet about impersonation: any discovery that a supposedly
automated solution is at least partly hand-made, may be seen as a form of deception.
Additionally, output checks performed by humans sometimes involve privacy leaks
(to the discomfort of our interviewee J.) that may damage the reputation of the com-
pany or platform. Even more than in the other cases, micro-workers' contribution is
surrounded by silence.

We make space for AI verification in our analysis, because of its potentially wide applicability. Checks of the accuracy and quality of algorithmic solutions will always be needed, regardless of whether supervised, unsupervised or reinforcement learning is used. By the same token, verification is not a temporary need but a recurrent one. As the sales of AI-based tools increase and concern a more diverse range of users, there will be a growing need to ensure that outputs meet expectations.

8 Discussion and conclusions

To map the linkages between AI and micro-work in our datafied economies, we started from the expectations that micro-work contributes to the preliminary, input phase of the AI production process, and that its contribution is structural rather than temporary. The former is in line with the communication strategies of platforms, and their insistence on the value of data produced and annotated with a “human touch”; the latter is at odds with the opinions of tech enthusiasts who anticipate full automation of data generation and annotation, but resonates with industry reports that the global market for human-powered data services for AI is growing.

We reflectively thought through our expectations in light of empirical evidence
from desk research, responses to an online questionnaire and in-depth interviews.
We compared and contrasted the voices of all stakeholders to probe and refine
our ideas. This material corroborated our initial assumption of an important role
of micro-work in AI preparation. But we also noted some anomalies that led us
to broaden the set of roles that micro-work may play in AI production. Industry
actors brought us to identify AI impersonation, which occurs whenever humans
outperform computers, so that it is advantageous to use them instead of (parts of)
algorithms. In turn, micro-workers' accounts of interventions to check the outputs
of an automated system – that is, at the end of the production chain – prompted us
to single out AI verification.

The result of this analysis is a typology, summarized in Figure 1. The process of
AI production starts with preparation (left panel), which includes both data gener-
ation and annotation. This may concern image, text, sound, video or other types of
data, and it is largely outsourced to online micro-workers. The data that they pro-
duce or enrich feed an algorithm that learns a model (central panel) which in turn,
returns an output with some degree of certainty. For example, the output of an im-
age classification algorithm can be “it is 90% likely to be a dog, 10% likely to be
another animal”. If impersonation occurs at all, it is at this stage. Humans replace
part of the algorithm (when they step in to complete a task that, say, Google Duplex
struggles to achieve) or all of it (when they entirely simulate an algorithm that has
not yet been coded, as in the early days of Julie Desk). AI verification (right panel)
is the process through which outputs are sent to micro-workers to be checked for
accuracy and if necessary, corrected.

<figure><img src="image_19.png" /><figcaption>**Fig. 1** The three main functions of micro-work in the development of data-intensive, machine-learning based AI solutions of today. Source: authors’ elaboration based on Casilli et al. (2019).</figcaption></figure>

To be sure, there are possible overlaps between our three cases of AI preparation, AI verification and AI impersonation. On the one hand, both impersonation and verification may be first steps toward developing datasets that can be subsequently used for preparation purposes. On the other hand, the boundaries between verification and impersonation become fuzzy when humans intervene to correct errors in real time. There is in fact a continuum of functions for micro-work in AI, many real-world cases being positioned in-between the three main types that we have singled out.

There might even be cases that do not fit with any of these types. In France, we
did not find so-called click-farms, where workers have to “like” (or dislike, share
etc.) the webpages of brands, products, celebrities, sports teams or politicians. Usu-
ally even less paid than standard micro-work, and less often disclosed than the cases
of impersonation mentioned here, these tasks are commonly outsourced to providers
in low-income countries. Because they artificially inflate the indicators of quality
and popularity used by (among others) search engines, they have lately been blamed
for contributing to the diffusion of fake news. In our perspective, they may be said
to perform some kind of “AI disruption”. Future research may be able to assess their
scope and consequences.

Refining our initial idea to add more roles of micro-work, at different stages of
the AI production process, helps us find a comprehensive answer to the question
of the extent to which micro-work is a temporary or structural component of AI. If
we had focused exclusively on AI preparation, as in the discourse of most micro-
work platforms, we might have thought that the need to accumulate labor-powered
data is specific to the current times, in which AI is growing fast but has not reached
maturity, and the less data-demanding unsupervised learning has not made enough
progress. But in our data economy, this is unlikely to happen. Data availability will
never reach a steady state: most use cases for machine learning require ongoing
acquisition of new sources to continuously adjust to changing conditions, resulting
in a steadily-growing need for humans to produce data for more accurate, more
precise, and more profitable results. The discovery of AI verification strengthens
this idea, in that some of the data used to re-train an existing algorithm and adapt
it to new circumstances, come from the quality checks routinely done by humans.
Taking into account AI verification, not just preparation, also contributes to dismiss
the related idea that progress in unsupervised learning might eliminate the need for
humans: even then, verification tasks will have to be performed.

Similarly, impersonation should be understood in light of the other two types. A
single, perhaps highly mediatized case may suggest that micro-work is a transitory
phenomenon, to disappear as companies accumulate the necessary data, skills, and
computational capacity. But our typology supports the idea that impersonation is
systemic and will always be present to some degree, because it ensures the necessary
connection between AI preparation and AI verification, supplementing algorithms
when they fail. Impersonation also demonstrates that the durability of demand for
micro-work depends not only on technological, but also on economic factors. As
long as there are humans who can perform tasks more cheaply than AI, perhaps (but
not necessarily) because they reside in countries where the cost of labor is low, it
will be advantageous to substitute them for machines. Overall, our typology hints
that full automation is not to be expected any time soon, and that micro-work will
continue to play an important role in keeping the industry going.

That micro-tasking is not a transitory phenomenon, and can occupy multiple
roles in the AI production process, may make platforms pride themselves on coun-
tering the gloomy predictions of AI-induced job losses, by creating earning oppor-
tunities that would not exist without the technology. But difficult questions must
be asked about the conditions under which micro-work is performed. Although a
detailed analysis of working conditions is outside the scope of this paper, the in-
terviews we used hint that executing un-challenging tasks such as labelling images
for an unknown purpose can be destabilizing; that involuntarily accessing personal
data of other people, or witnessing grossly deceptive forms of impersonation, brings
ethical dilemmas; and that exposure to violent web content may generate distress
(Casilli et al., 2019). Because platform labor occurs in a legal grey area (Prassl,
2018), no systematic ways to alleviate these problems exist.
We repeatedly noted the silence surrounding micro-work. Platforms tell clients that human contribution has value, but not who these humans are and in what conditions they work. As a result, clients know little about micro-workers – just as the latter are often unaware of the purposes of their tasks – and may find it difficult to interact with them as mentioned above. Ironically, the full extent of human intervention is unclear even to key industry actors. The incentive to obscure the role of human contributors is highest when the credibility of full automation promises is at stake. As a general tendency and beyond one-off revelations, this contributes to keeping micro-work far from the gaze of the general public and from the agenda of policy-makers. Out of the reach of institutional regulations, subject only to the forces of a market plagued by an excess supply of workers (Graham and Anwar, 2019), remunerations remain structurally low.

Micro-work is the already-visible tip of an iceberg of transformations brought about by data-driven automation. AI is not the end of human labor, but risks depriving it of the quality, meaning and social status that it had acquired over time. There is a need for ambitious, long-term policies that frame the further development of AI by taking into account the concrete conditions of its production, in light of ongoing debates on digital platform labor and its shortcomings – from low remuneration and precariousness to lack of social security (Graham and Shaw, 2017). Put differently, credible commitment to socially responsible AI requires the definition of labor standards in the processes that underpin it. More transparency is needed, toward workers as well as the general public, to ensure the full extent of human participation is understood and recognized for what it is worth.

## 9 Acknowledgments

This paper presents results of the “Digital Platform Labor” (DiPLab) project, co-funded by Maison des Sciences de l’Homme Paris-Saclay; Force Ouvrière, a workers’ union, as part of a grant from Institut de recherches économiques et sociales (IRES); and France Stratégie, a service of the French Prime Minister’s office. The platform FouleFactory/Wirk offered logistical support, and Inria provided complementary funding. For more information: http://diplab.eu/

The authors would like to thank current and former DiPLab team members, notably Maxime Besenval, Odile Chagny, Clément Le Ludec, Touhfat Mouhtare, Lise Mounier, Manisha Venkat and Elinor Wahal.

Preliminary versions of this paper were presented at the Reshaping Work Conference in Amsterdam, October 2018, and at the annual Research Day of the Society and Organizations (S&O) Center of HEC Paris, May 2019. Many thanks to discussants, organizers and participants for constructive feedback. We are especially grateful to Mitali Banerjee, Magali Delmas, Vili Lehdonvirta, and Anoush Margaryan for helpful suggestions. Any remaining errors or inconsistencies are exclusively ours.
References

Acemoglu, D. and Restrepo, P. (2018). The race between man and machine: Implications of technology for growth, factor shares, and employment. *American Economic Review*, 108(6):1488–1542.

Alpaydin, E. (2014). *Introduction to Machine Learning* (3rd edition). MIT Press.

Alpaydin, E. (2016). *Machine Learning: the New AI*. MIT Press.

Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. *Wired*.

Autor, D. H. (2015). Why are there still so many jobs? The history and future of workplace automation. *Journal of Economic Perspectives*, 29(3):3–30.

Autor, D. H. and Dorn, D. (2013). The growth of low-skill service jobs and the polarization of the us labor market. *American Economic Review*, 103(5):1553–97.

Berg, J., Furrer, M., Harmon, E., Rani, U., and Silberman, M. (2018). Digital labour platforms and the future of work: Towards decent work in the online world. *ILO Report*.

Bessen, J. (2017). Automation and jobs: When technology boosts employment. *Boston Univ. School of Law, Law and economics research paper, no. 17-09*.

Boelaert, J. and Ollion, E. (2018). The great regression: Machine learning, econometrics, and the future of quantitative social sciences. *Revue française de sociologie*, 59(3):475–506.

Brynjolfsson, E. and McAfee, A. (2014). *The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies*. Norton.

Casilli, A. (2019). *En attendant les robots: Enquête sur le travail du clic*. Seuil.

Casilli, A., Tubaro, P., Le Ludec, C., Coville, M., Besenval, M., Mouhtare, T., and Wahal, E. (2019). Le micro-travail en France. Derrière l'automatisation, de nouvelles précarités au travail ? Report of the Digital Platform Labor (DiPLab) project.

Chen, B. and Metz, C. (2019). Google's duplex uses a.i. to mimic humans (sometimes). *The New York Times*.

Chui, M., Mayika, J., and Miremadi, M. (2016). Four fundamentals of workplace automation. *Mckinsey q. 2016* (1).

Cognilytica (2019). Data engineering, preparation, and labeling for ai 2019. Report n. cgr-de100.

Domingos, P. (2017). *The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World*. Penguin.

Ekbia, H. and Nardi, B. (2017). *Heteromation, and Other Stories of Computing and Capitalism*. MIT Press.

French Government (Ministry for the Economy, Ministry of Education and Ministry of Digital Technologies) (2017). France intelligence artificielle – rapport de synthèse. Report.

Frey, C. and Osborne, M. (2017). The future of employment: How susceptible are jobs to computerisation? *Technological Forecasting and Social Change*, 114(C):254–280.
Graham, M. and Anwar, M. (2019). The global gig economy: Towards a planetary labour market? *First Monday*, 24(4).

Graham, M. and Shaw, J. (2017). *Towards a Fairer Gig Economy*. Meatspace Press.

Gray, M. and Suri, S. (2017). The humans working behind the AI curtain. *Harvard Business Review*, (9):2–5.

Gray, M. and Suri, S. (2019). *Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass*. Houghton Mifflin Harcourt.

Hern, A. (2019). Apple contractors 'regularly hear confidential details' on Siri recordings. *The Guardian*.

Irani, L. (2015a). The cultural work of microwork. *New Media & Society*, 17(5):720–739.

Irani, L. (2015b). Difference and dependence among digital workers: The case of Amazon Mechanical Turk. *South Atlantic Quarterly*, 114.

Jobin, A., Ienca, M., and Vayena, E. (2019). The global landscape of AI ethics guidelines. *Nature Machine Intelligence*, 1:389–399.

Kitchin, R. (2014). *The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences*. Sage.

Kässi, O. and Lehdonvirta, V. (2018). Online Labour Index: Measuring the on-line gig economy for policy and research. *Technological Forecasting and Social Change*, 137:241–248.

Mayer-Schönberger, V. and Cukier, K. (2013). *Big Data: A Revolution That Will Transform How We Live, Work, and Think*. Houghton Mifflin Harcourt.

MMC Ventures (2019). The state of AI 2019: Divergence. Report.

Prassl, J. (2018). *Humans as a Service. The Promise and Perils of Work in the Gig Economy*. Oxford University Press.

Ram, A. (2019). Europe's AI start-ups often do not use AI, study finds. *Financial Times*.

Roberts, S. (2019). *Behind the Screen: Content Moderation in the Shadows of Social Media*. Yale University Press.

Rochet, J.-C. and Tirole, J. (2003). Platform competition in two-sided markets. *Journal of the European Economic Association*, 1(4):990–1029.

Schmidt, C. (2017). Digital labour markets in the platform economy: Mapping the political challenges of crowd work and gig work. Report, Friedrich-Ebert-Stiftung.

Schmidt, F. (2019). Crowdproduktion von Trainingsdaten: Zur Rolle von Online-Arbeit beim Trainieren autonomer Fahrzeuge. Report, Hans-Böckler-Stiftung.

Villani, C., Schoenauer, M., Bonnet, Y., Berthet, C., Cornut, A., Levin, F., and Ron-depierre, B. (2018). Donner un sens à l'intelligence artificielle: pour une stratégie nationale et européenne. Report, France Premier Ministre.

von Ahn, L., Maurer, B., McMillen, C., Abraham, D., and Blum, M. (2008). re-CAPTCHA: Human-based character recognition via web security measures. *Science*, 321(5895):1465–1468.
Counting “micro-workers”:
Methodological and societal challenges around
new forms of labor

Paola Tubaro, Clément Le Ludec, and Antonio A. Casilli

Abstract

Standard statistics struggle to apprehend the workforce that performs tasks and gigs on digital platforms. Particularly difficult to measure are "micro-work" platforms that allocate fragmented data tasks to remote online providers, for remuneration as low as a few cents, with hardly any form of contractual or social protection. As a first step toward filling this gap, we develop an innovative mix of qualitative and quantitative tools (online surveys, in-depth interviews, capture-recapture techniques, web traffic analytics) to count micro-workers in an industrialized country, France. We identify three distinct worker populations, corresponding to different levels of engagement. A group of about 15,000 "very active" individuals, most of whom use these platforms at least once a week; a second featuring over 50,000 "regular" workers, overall more selective and present at least once a month; and a third circle of approximately 260,000 "occasional" workers, more heterogeneous and who often alternate inactivity and various levels of work practice. These figures urge a re-thinking of some key principles of labor statistics, and invite a broader discussion on the place of labor in our society.

**Key words:** Micro-work, digital platforms, labor statistics

<footnote>Paola Tubaro
Laboratoire de Recherche en Informatique, Centre National de la Recherche Scientifique, Paris
Saclay, e-mail: paola.tubaro@lri.fr</footnote>

<footnote>Clément Le Ludec
MSH Paris Saclay</footnote>

<footnote>Antonio Casilli
Telecom Paris, e-mail: antonio.casilli@telecom-paris.fr</footnote>
# 1 Introduction

The rise of “platform work” in the wake of the 2008 financial crisis undermines the accuracy of the instruments classically used to observe employment, unemployment and inactivity, such as the Labor Force Surveys that national statistical institutes conduct in European countries. Based on international classifications that allow comparisons across time and space, these surveys have traditionally well served the need of measuring population activity levels and of describing jobs. Yet they struggle to apprehend platform-based "gigs", "rides" and "tasks" that challenge established definitions of salaried employment.

Lack of consensual definitions is a major obstacle to measuring this phenomenon. The surveys conducted so far, whether by official statistical agencies (for example, by adding a module or a few questions to an existing survey) or smaller-sized research organizations (for example Huws et al. (2017)), diverge in scope (which activities to include? which platforms, beyond the most high-profile ones?) and periodization (include all workers who used the platforms in the last week, as per Labor Force Surveys traditional protocols? or extend to the last month or even year? or perhaps include anyone who used platforms once in life?). There are also sampling problems: in large surveys fielded to the general population, endemic non-response (sometimes due to unclear questions) leads to underestimations (Eichhorst et al., 2017), while small, more focused surveys are unrepresentative and lead to over-estimations. Depending on their design, extant studies obtain very divergent estimates ranging from 0.4% to 22% of the size of Western economies (Frenken, 2018), although many of them are in the range of about 0.5% to 2% (O'Farrel and Montagnier, 2019).

Particularly difficult is the measurement of platform work that is performed entirely online. Some estimates, such as the one featured in ter Weel et al. (2018), even exclude it from scope. Unlike "geographically sticky" (Ojanperä et al., 2017) car rides and bike delivery services, such activities are often executed remotely, at home behind a computer or on a mobile phone while commuting. They even lack visual clues such as the pieces of equipment and corporate logos that gig workers wear, punctuating urban traffic. A recent study at the Oxford Internet Institute has taken an important step toward measuring online work, advancing that its global volume has increased by a whopping 25% between 2016 and 2018 (Kässi and Lehdonvirta, 2018).

However, this otherwise remarkable result does not distinguish between the two main configurations of online work. On the one hand, there are freelancing platforms that cater to companies' need for on-demand talents such as graphic designers, software developers, and strategy consultants. These are selected, sometimes very well-paid, professionals. On the other hand, “micro-work” platforms entrust much smaller and simpler tasks to indistinct masses of largely anonymous, and often little qualified, providers<fnref n="1" />. Micro-work is the most elusive form of work in the

<footnote n="1">¹ We prefer the term “micro-work” to the alternative “crowdsourcing”, first proposed by the *Wired* magazine in 2006 and resulting from contraction of the words “crowd” and “outsourcing”. Crowd-</footnote>
platform economy. It encompasses tasks such as identifying or labeling objects on images, transcribing invoices, translating bits of text, moderating content (such as videos), sorting or ranking photographs, recording sentences spoken aloud. Standardized and repetitive, these tasks usually take only a few minutes, are typically paid on a piecework basis, and attract compensations that can be as low as a few cents.

Online micro-work is a by-product of today’s "datafied" economy (Mayer-Schönberger and Cukier, 2013) because it supports the digital transformation of companies – for example, transcription of scanned invoices may serve the creation of a paperless archive. Disassembling bigger "projects" into tiny "tasks", and re-assembling workers’ outputs at the end, comes with a promise of speed and cost-effectiveness. By mobilizing many micro-workers simultaneously for a few hours rather than a single subcontractor for a longer time, client companies conjugate process scalability and workforce flexibility. More prominently, micro-work feeds into the scientific and industrial development of artificial intelligence (Gray and Suri, 2017; Irani, 2016). Labels that identify objects in natural settings can be used to train machine learning models. For instance, labelled images of cars in road traffic are necessary to operate autonomous vehicles (Schmidt, 2019; Tubaro and Casilli, 2019), conversation transcripts help improve automatic speech recognition of smart speakers, and sets of predefined questions and answers can be used to develop virtual assistants (Casilli et al., 2019).

Like other forms of online platform labor, the size of this workforce is unknown. Platforms themselves claim that their user-base is large: if Amazon Mechanical Turk, the most widely-known micro-working service, boasted 500,000 “Turkers” as early as 2014², the Chinese giant Witmart alone is said to exceed 12 millions<fnref n="3" />. However as we will see, researchers have challenged these figures and come to much more conservative estimates using a range of methods (section 2.3). Also, these figures refer to single platforms without taking into account the variety of players that co-exist, both globally and in specific geographies. Thus, they are of limited help in orienting local policy actions.

As a first step toward filling this gap, this paper leverages a range of methods to estimate the number of people micro-working in one single country, namely France. This case is inherently interesting because France is a pioneer in information technologies and a heavy investor in artificial intelligence research. Nevertheless, the country is little documented in the (still scant) literature on micro-work, and its residents are relatively less present in international platforms such as Amazon Mechanical Turk. Partly for language reasons, France has its own micro-work platforms, the

<footnote>sourcing includes both micro-tasking and some parts of freelancing, notably with platform designs that rely on spec work and tournament job where workers bid competitively against one another to win a client – so that the "crowd" does the job, but only one gets paid.</footnote>

<footnote n="2">² “500,000 Workers from 190 countries”, cited in M. Harris, “Amazon’s Mechanical Turk workers protest: ‘I am a human being, not an algorithm’”, *The Guardian*, December 3, 2014.</footnote>

<footnote n="3">³ *Les Echos*, "L'actu tech en Asie : Female founders veut percer le plafond de verre", June 19, 2015.</footnote>
most popular of which is Foule Factory<fnref n="4" />, whose webpage displays 50,000 contributors.

To estimate the size of the French micro-working population, we draw on publicly available sources combined with the results of DiPLab ("Digital Platform Labor"), a comprehensive study of micro-work in the French-speaking world<fnref n="5" />. Using three methods to collect and analyze, respectively, public data sources, experimental data and web traffic data, we identify three distinct pools of micro-workers and we link them to diverse levels of engagement. They comprise approximately 15,000 "very active", 50,000 "regular" and 260,000 "occasional" online platform micro-workers respectively.

To the best of our knowledge, ours is the first attempt to estimate the number of micro-workers in a specific country across micro-work platforms, considering a range of different uses and levels of engagement. It demonstrates the need to go beyond the most high-profile cases and to take into account local conditions in addition to global trends. In this sense, our results may inspire researchers interested in re-applying our methodology to other countries.

## 2 The challenges of measuring platform labor (especially micro-work)

Micro-work does not fit neatly into the categories ordinarily used to measure the labor force. Some of the difficulties are common to other forms of platform labor, others are specific to it. In what follows, we explore them and discuss how different tools (from general and ad-hoc surveys to publicly-available company data, and non-standard methods) may or may not help. We come to conclusions as to the best method-mix for our case.

### 2.1 Why surveys and administrative data fail to detect micro-work

Official statistical surveys are usually designed to report the main occupation of respondents, while the flexibility afforded by online platforms allows arranging work as a secondary activity, to earn a supplement of income. This is one reason why early studies such as that of Katz and Krueger (2019), which still followed standard statistics in tracing only people who perform platform work as a main job, obtained low estimates (0.5% of the adult population). A related issue is the possibility to perform platform work irregularly – an aspect that, again, Labor Force Surveys miss

<footnote n="4">⁴ “Foule” is the French for “Crowd”.</footnote>

<footnote n="5">⁵ As part of DiPLab, we built an inventory of micro-work platforms and mobile applications being used in France, collected online questionnaire data, and interviewed in-depth a selection of micro-workers, clients, platform managers and other stakeholders. The data collection was conducted in 2017-18 (see Section 3.3 for more details).</footnote>
insofar as they assume that those who answer “yes” to the question of whether they
did any paid work in a given reference period, did so with some regularity.

Measuring platform work cannot be content with such binary questions and addi-
tionally requires assessing regularity, intensity and significance (Urzì-Brancati et al.,
2019). If these issues are common to many forms of platform labor, micro-work am-
plifies them. The “unbundling” (Pesole et al., 2018) of large data jobs into tiny tasks
makes them more likely to be part of a secondary or occasional activity rather than
a main, stable occupation, at least in Western countries. Because these tasks are
mostly simple, short and sometimes even “gamified”, and their purposes are not
always disclosed, workers sometimes fail to even recognize them as labor. As a re-
sult, more than other platform-based activities, micro-work may go under-reported
in general surveys.

While the language and structure of official statistical agencies slowly adapts
to the fast-moving world of the platform economy, several groups of academic re-
searchers, unions and think-tanks have attempted to launch smaller, dedicated sur-
veys at country or cross-country levels (O'Farrel and Montagnier, 2019). The major-
ity have attempted to estimate the number of platform workers in general, without
specifically targeting micro-workers. However, a new wave of a European compar-
ative survey (Huws et al., 2017) run by the University of Hertfordshire in collabo-
ration with other research bodies has released more detailed results for Spain (Fun-
dación Felipe González and University of Hertfordshire, 2019), where allegedly
17% of the adult population do platform work at least once a week, and 40% of
them, amounting to over 3 million people, do online clerical work, small tasks
and “click-work” – that is, a set of activities that is broader than our definition of
micro-work, but includes most of it. Other studies enable indirect estimations. The
COLLEEM survey, fielded in 14 European countries in 2017, estimates that slightly
less than 10% of the 16-74 years old have ever done any platform work (Pesole et al.,
2018), with differences across countries (12.5% in Spain, 7.6% in France). Of them,
24.7% are said to have performed micro-tasks at least sporadically (Urzì-Brancati
et al., 2019). Applying these ratios to Spain would give a number of micro-workers
just below one million, while for France, this would amount to roughly 900,000.

While these surveys are valuable tools to situate micro-work in the broader eco-
nomic context, differences in how they operationalize the notions of secondary and
irregular activity produce widely divergent results, and invite to caution in inter-
pretation. The numbers for Spain in the two studies just cited differ markedly, and
even the lower figure may be an overestimate insofar as the COLLEEM survey pro-
duced systematically higher values than a comparable one, the 2018 Eurobarometer
(O'Farrel and Montagnier, 2019, p. 14)<fnref n="6" />.

In other settings, administrative data have proven to be a valuable resource that
compensates for the limitations of surveys. Here, administrative data may also open

<footnote n="6">⁶ The Flash Eurobarometer 467 "The use of the collaborative economy" of 2018 estimates that 11% of the adult French population have ever offered services through platforms. 5% of them (about 300,000 people) have performed online services such as IT and accounting, while 7% (about 420,000) have offered "other" online services: neither category corresponds precisely to micro-work, but both cover parts of it.</footnote>
new avenues if for example the fiscal authorities require platforms to report users'
earnings for tax collection purposes (O'Farrel and Montagnier, 2019). However,
a consequence of irregularity is that some workers earn very little, so that their
platform incomes fall below the given tax thresholds. Thus, the extent to which
their presence will be reported at all, will depend on the details of the specific legal
requirements that countries might impose.

## 2.2 Why data reported by micro-work companies cannot be taken at face value

In the absence of official statistical data, many studies so far have simply chosen to
rely on the information that platforms themselves release publicly, when available.
On micro-tasking specifically, a widely cited 2015 World Bank report summed the
number of registered users on the main international platforms then known (essen-
tially Amazon Mechanical Turk and Crowdflower<sup>7</sup>), coming to an estimate of 5.8
million micro-workers worldwide (Kuek et al., 2015). In France, user registration
data from platforms are featured in an influential report of the French General In-
spectorate of Social Affairs (Amar and Viossat, 2016).

However, this method is problematic for at least three reasons. First, the number
of sign-ups says nothing of people's actual levels of engagement, and it is there-
fore an incomplete information (unless the platform actively enforces a systematic
deregistration policy for inactive users). Second, this method over-emphasizes high-
profile players such as Mechanical Turk while under-researching the large number
of smaller local and international platforms, which release incomplete or no infor-
mation at all about their user basis. To the extent that these less visible platforms
attract different users (or function differently under some other dimension), it may
produce unreliable results. Third, this approach neglects the practice of register-
ing with multiple platforms. Extensively analyzed in the economics of two-sided
markets ("multi-homing", see Athey et al. (2018)) but curiously little explored in
research on platform labor, multi-activity can partly be seen as another consequence
of the accessory and/or irregular nature of micro-work. Insofar as workers sign up
to multiple services, the sheer number of registered users can be leveraged only to
analyze one single platform, but it will count some workers more than once if it
sums registrations across platforms.

A specificity of micro-work, notably its remote nature and its geographical "un-
stickiness", adds an additional difficulty in that even platforms that provide global
figures do not always give the breakdown of the number of registered workers by
country. The researcher, then, must rely on educated guesses. Existing evidence
on (mainly) freelancing online labor platforms suggests that most of the demand
is located in North America and Europe, and most of the supply in South and

<footnote n="7"><sup>7</sup> The platform Crowdflower re-branded as Figure Eight in 2018 and was acquired by another platform, Appen, in 2019.</footnote>