dataesr/scientific-paragraphs-categorization huggingface dataset:
These sectors in the Camargue, and other elements (e. g. age, family composition, origin) could be of importance for park management, but were also not identified. This could be due to biases in the data (e. g. the elderly may not upload their photos) or due to biases in use of the region (e. g. elderly people may not go into the Camargue). Extracting information from Flickr users profiles may give some information on demographics, but was not attempted in this study as all images from Flickr were used and not filtered for the content, or user metadata. Not all visitors will take and upload photos onto a social media platform, hence sampling bias is inherent in Flickr and social media data  (Levin, Lechner and Brown, 2017; Walden-Schreiner et al. , 2018a). Flickr data is biased by factors that are subject to continuous change including the popularity of the platform, user groups and geography  (Sessions et al.
For future expansion on this research, the role of biotic and abiotic factors could be assessed, with the inclusion of remotely sensed data to monitor the impact of seasonal events and larger temporal events, such as temporal ponding on the different visitor groups. Additionally, we could separate users by place of origin, hence be able to distinguish between recreation or tourism or local/domestic and foreign visitors as demonstrated by  Juutinen et al. (2011)  to investigate the differences in ecological and recreation preferences in Oulanka National Park in Finland. As this paper does not distinguish between the types of photos taken, future research could also filter for indoor/outdoor photos with the filtering of Google Cloud Vision image annotations or photo metadata directly from Flickr.
It is clear that park managers will very likely not be able to use raw social media data themselves directly and would need a tool developed to facilitate user-friendly harvesting and interpretation of data, but once in place, this could be a much more effort and cost-effective method than doing surveys in the field. This study has identified information which has been received by managers in the Camargue as very interesting. In particular, knowing when and where bird watchers and nature lovers wander in the Camargue is considered as original knowledge because these tourists often go undetected while touring in the Camargue. Using the obtained maps, we asked the representatives of the park management whether, and how, they would use the obtained information. They indicated that the maps confirmed important assumptions on tourism in the area, such as the limited use that religious and beach tourists make of the wider Camargue region.
, 2010; Juutinen et al. , 2011; Wood et al. , 2013; Siikamäki et al. , 2015; Millhäusler et al. , 2016). The utilisation of techniques that allow different and/or unique beneficiary groups to be analysed separately will allow more nuanced and dynamic management strategies to be developed for recreational areas. Social media data can be harnessed to better understand the area where visitors place value. Geo-referenced images coupled with content analysis allow a greater understanding of not only where users visit, but what especially they find attractive in the environment. By harnessing the power of LSA in this study, we have been able to demonstrate how visitors can be grouped to visualise spatial and temporal patterns of visitation. With increasing pressure on PAs, this type of analysis can allow park managers and decision makers to see how proposed management may impact respective beneficiary groups.
Data and methods We draw on data pertaining to the setup in itself. But to analyze any participatory setup, it is necessary to set it in its wider political context. We therefore also turn to data relative to how this consultation was received by major stakeholders (media coverage and documents produced by vaccine critics). The analysis is based on a mix of qualitative methods commonly used in political science and of quantitative methods developed in the field of computational linguistics. To interpret our results, we drew on our previous work on French vaccine-related controversies and policy-making  (Anonymous, 2015 (Anonymous, , 2016 (Anonymous, , 2018)). Our main focus will be on the two datasets that concern the setup in itself: the documents describing its organization and the public's comments posted on its website.
The database of all contributions published on the consultations' internet platform was provided by Santé Publique France, a public agency under the supervision of the French Minister for Health in charge of monitoring the French population's health and promoting health-inducing behaviors. This agency was in charge of the logistics of the citizen consultation including the construction and administration of the consultation's website. They were therefore responsible for guaranteeing that the contributions posted in the comments section of the website complied with French legal and ethical standards regarding internet publication (anonymization, filtering of violent and libelous discourse, etc. ). This led them to delete 1404 contributions and to authorize the publication of a total number of 10 435 contributions.
This choice of themes also reflects the fact that we were already aware before launching the study that this consultation led to the decision to extend mandatory vaccination and that it was criticized for not taking on the issue of the safety of adjuvants. Assessing the proportion of comments that took on these issues and the opinion they expressed was therefore crucial in the analysis of how the public's opinion was integrated in this participatory setup. Using the software WEKA  (Eibe et al. , 2016) , one of the authors trained an artificial intelligence to recognize texts according to those variables, with our sample as training corpus. To that end, he implemented two models based on multinomial logistic regression with Tikhonov regularization (Le Cessie and Van Houwelingen, 1992)based on n-grams (4-grams to 7-grams) of characters.
Punctuation marks were retained as they proved to be meaningful for the categorization of the texts. For instance, in our corpus, the use of exclamation marks is significantly associated with vaccine-critical texts. The texts have not been lemmatized as this would have concealed important information. Speaking of one « vaccine » and not of « vaccines », of one's « doctor » and not of « doctors » in general etc. is significant in the context of our study. Outcomes of the model were generated with an acceptable precision for the first question (0. 71, superior to the acceptability threshold of 0. 68 defined by Bradley  (Bradley, 1997) , as well as for the second question (0. 82). Relevant lexical entities were isolated using CQL requests with the software TXM  (Heiden et al. , 2010). To set the results from the previous analyses in context, we will also occasionally draw on the qualitative content analysis of two datasets that concern the reception of the consultation in the media and by vaccine critics.
This was meant to assess whether this consultation favored consensus-building and acceptability of its recommendations and whether it favored the wide dissemination of vaccinecritical arguments in the media. We gathered all articles pertaining to the consultation published in 7 of France's most prominent national newspapers (Libération, Le Parisien-Aujourd'hui en France, Le Monde, Le Figaro, La Croix, L'Humanité, Les Echos). These newspapers were accessed through the database Europresse which allows keywords searches (see www. europresse. com). We selected all articles containing the fragment "débat" or "Concertation" and the fragment "vacc" published between June 1 st , 2015 and February 1 st , 2017. The search generated 48 articles (after exclusion of duplicates and errors). Two of the authors applied a coding scheme focused on identifying critics and defenders of the consultation and their arguments.
To analyze the reception of the consultation among vaccine critics we turned to a database of the contents 299 French-speaking vaccine-critical websites. This sample comprises all prominent French vaccine-critical organizations and activists who had a website or a blog at the time. It was crawled using the software Hyphe, developed by the Medialab at Sciences Po Paris. We analyzed through close-reading all the pages of these websites that evoked this citizen consultation using the same keywords searches as for the media articles. Analysis was performed by two of the authors and the coding scheme focused on identifying the arguments for and against this consultation and the chronology of their appearance. Since all the data used in this article are public, this research did not require the approval of an ethics committee in France.
Introduction Mobile crowdsourcing platforms and applications (or apps) are being widely used to collect datasets in the field for both industrial and research purposes  [2, 6, 31]. By relying on a crowd of user devices, mobile crowdsourcing delivers an engaging solution to collect insightful reports from the wild. However, the design of such platforms presents some critical challenges related to the management of users, also known as workers. In particular, the privacy of the workers is often underestimated by the crowdsourcing platforms and it often fails to be addressed effectively in practice  [25]. While data anonymization is commonly achieved a posteriori on the server side  [7, 16, 20, 22] , this approach is subject to adversarial attacks, even when protocols for the communication and the data storage are claimed to be secured  [11, 12].
Furthermore, the workers may be reluctant to share Sensitive Personal Information (SPI) with third parties (e. g. , students contributing to a crowdsourcing campaign initiated by a professor). Gaining the confidence of workers is extremely difficult and we argue in this paper that the adoption of a priori data anonymization mechanisms contributes to delivering a trustable component to better mitigate privacy leaks in the data shared by workers. For example, the worker's location is not only the most requested but also the most sensitive data collected by mobile crowdsourcing platforms  [3]. Our scheme therefore explores the physical proximity of workers to agree on a dissemination strategy for reporting the crowdsourced data. By altering the link between workers and data consumers on the server, our approach intends to mix data contributed by several workers within a collaborative data flow that exhibit similar crowd-scale properties and without discarding any SPI.
In particular, we propose a system-level service that acts as a proxy within the mobile device for sharing crowdsourced data and from which workers can control their privacy settings. Fougere is our implementation of this anonymization scheme and is available as an open source library  4  that can be used by legacy mobile crowdsourcing apps. We illustrate the benefits of Fougere by integrating it within the state-of-the-art MobiPerf mobile crowdsourcing app as well as the APISENSE mobile crowdsourcing platform. We evaluate the effectiveness and the impact of our anonymization scheme on these two mobile crowdsourcing systems by deploying and orchestrating a crowd of 15 emulated mobile devices. More precisely, we replay the SfCabs cab mobility traces  [30]  and we show that Fougere defeats state-of-the-art privacy attacks  [18, 24, 26]  with little impact on the quality of the resulting datasets.
Related Work Thanks to the wide adoption of mobile devices, mobile crowdsourcing has emerged as a convenient approach to gather meaningful and scalable environmental datasets by involving citizens in the process of performing measurements in the wild  [2, 6, 22, 31]. While the development of mobile crowdsourcing apps is clearly leveraged by the Software Development Kits (SDK) made available by Android and iOS, mobile crowdsourcing platforms are bringing another level of abstraction to ease the design and the deployment of mobile crowdsourcing campaigns  [4, 8, 13, 20]. As depicted in Figure  1 , mobile crowdsourcing campaigns typically consist of several stages: i) the description of the data to be crowdsourced, ii) the deployment and the gathering of the dataset in the wild, iii) the aggregation and storage of datasets in the Cloud, iv) the processing and v) publication of the campaign results.
However, along all these stages, SPI can be conveyed by the platform and potentially be subject to attacks from adversaries, therefore motivating the development of a better privacy support. Location privacy protection mechanisms (LPPMs) are particularly interesting to limit user privacy leaks  [5]. A large body of the related work has been devoted towards the latest stages of mobile crowdsourcing campaigns by improving the privacy properties of datasets once uploaded to remote servers  [23, 28, 35]. These techniques contribute to preserving the privacy of workers while limiting the impact on the quality of the resulting dataset. However, raw datasets stored on a remote server may be leaked through security breaches. Collaborative privacy-preserving location-based services. In the domain of Location-Based Services (LBS), some privacy protection mechanisms can be adapted to mobile crowdsourcing platforms.
In particular, we consider the solutions where users collaborate to hide information from the server  [10, 29, 32, 33] , which share similarities with Fougere. In particular, Show et al. [10]  use communication over peer-to-peer (P2P) protocols, Shokri et al. [32]  use WiFi Access Point connection, and finally, Shokri et al. [33]  and Peng et al. [29]  propose to use Wi-Fi Direct communications. Yet, such approaches are not widely adopted by LBS solutions as they fail to demonstrate their effectiveness in a realistic deployment. Privacy in mobile crowdsourcing platforms. Mobile crowdsourcing platforms are actively working on privacy protection mechanisms  [3, 19]. In particular, Cornelius et al. [13]  have proposed Anonysense: a mobile platform for opportunistic sensing. Because the server hosting the collected dataset can trace the worker's wireless access points, they propose to use an anonymization network to hide worker locations, they rely on a third-party server for routing the data.
Anonysense also supports reporting data with a statistical guarantee of k-anonymity. The workers' data are blurred and combined before being reported to the remote server. While their approach hides workers from the server, it exposes them to a third-party server that has to be trusted by the workers. Thus, introducing a single point of failure. Das et al. [14]  present PRISM: a platform for remote sensing. They use a sandbox to prevent mobile apps from using mobile sensors. Adversaries can still collect geotagged data from workers and apply privacy de-anonymization attacks on the dataset. As discussed in  [14] , both Anonysense and PRISM suffer from similar privacy leaks as the mobile app collects data local sensors made available by their mobile device, allowing data to be linked to the worker identifier.
Privacy Threats in Mobile Crowdsourcing Systems This section discusses the potential threats in mobile crowdsourcing systems along 3 axes: the system model, the sensitive personal information, and the known location-based attacks. Mobile crowdsourcing system model. The architecture we consider is a mobile crowdsourcing campaign that involves three components, namely, mobile devices, crowdsourcing apps, and storage servers. We consider that the mobile crowdsourcing apps can be trusted as we believe that the owner of the mobile crowdsourcing app or platform is interested in gathering insightful datasets with the consent of workers, especially if this mobile app is open sourced. However, we consider that the storage server can be compromised and reveal some sensitive personal information on behalf of the owner and the workers.
) enlists a task in charge of uploading a crowdsourced data sample to the remote server when the TTL expires; send(. ) delegates the dissemination of a crowdsourced data to Fougere. Opportunistic dissemination. The current implementation of the Fougere dissemination module builds on the WiFi-Direct technology to discover nearby devices. When a mobile crowdsourcing app forwards a message, Fougere triggers the configured LPPMs and accumulates the data in the forwarding queue. For each data accumulated in the forwarding queue, Fougere picks a random peer that has never received this data and forwards it. If the message reaches the configured number of device hops (ttl = 0), then the forwarded data is placed in an uploading queue, which will be emptied as soon as the remote mobile crowdsourcing app runs by invoking the upload handler registered by the app.
We use mobility datasets that are publicly available to control the emulated devices and we collect their interactions to trace their actions a posteriori. The crowdsourced dataset collected on the remote server are evaluated by the LPM 2 toolkit  [32]  to evaluate the preservation of workers' privacy. By adopting such an empirical validation, we can evaluate real applications integrating Fougere and we can observe the impact of changing the parameters of Fougere (number of hops, LPPMs' specific parameters). In the remainder of this section, we select the legacy MobiPerf  [22]  mobile app as the mobile crowdsourcing app that we considered to assess Fougere.
While the choice of such a mobility dataset might be challenging depending on the category of mobile crowdsourcing app, we use the epfl/mobility dataset that is publicly available from CRAWDAD  [30]  to emulate 15 workers who are performing network measurements with the MobiPerf mobile app. The crowdsourced dataset contains network measurements reported every 5 minutes by the workers moving in the San Francisco bay area. Attacking crowdsourced datasets. To evaluate the impact of Fougere on the privacy of workers, we use the LPM 2 toolkit  [32] , which is a state-of-the-art tool for measuring location privacy. In particular, LPM 2 covers the evaluation of the LPPMs that are supported by Fougere, like the obfuscation mechanisms including perturbations (adding noise), reducing precision, location hiding.
Empirical Evaluation In this section, we instantiate the above experimental protocol to assess Fougere as a practical support to improve the location privacy of workers. Experimental Setup In particular, thanks to the AndroFleet  [27]  emulation platform, we can reproduce the execution of a deployment of 15 mobile instances emulating a one-week crowdsourcing campaign, thus proposing a realistic input dataset to evaluate Fougere. Then, we compare the behaviors of 6 configurations of the MobiPerf app: 1-Vanilla refers to the reference implementation of the MobiPerf Android app, as it can be downloaded from http://www. mobiperf. com. This configuration is used to demonstrate the vulnerability of legacy mobile crowdsourcing apps with regards to potential privacy threats. It is also used as a witness to evaluate the benefits of the other configurations including Fougere; 2-Fougere with no LPPM refers to the extension of MobiPerf with the Fougere library.
1, 0. 05 , thus reducing the location precision by 1 digit with a probability of 0. 1 and possibly removing the location with a probability of 0. 05. Time noise is set to 30, 0. 1, 0. 05 , thus reducing the time precision to half an hour with a probability of 0. 1 and possibly removing the timestamp with a probability of 0. 5, and finally k-anonymity is set to 2 , meaning that at least 2 samples should be produced in the same area to be forwarded; (b) strong privacy profile configured with location noise = 2, 0. 2, 0. 1 , time noise = 60, 0. 2, 0. 1 and k-anonymity = 4 as privacy settings. None of these configurations includes a privacy filter as these LPPMs are expected to be used to hide the living and working places of workers and the input dataset does not include this information. Furthermore, this paper does not aim at evaluating the efficiency of individual LPPMs, but rather demonstrating the benefit of combining them in an open framework like Fougere.
Increasing this data traveling distance with Fougere contributes to better shuffle crowdsourced data produced by a crowd of workers. Figure  4  reports on this distance traveled by the crowdsourced data before being uploaded back to the remote storage server. In particular, the default configuration of Fougere maximizes the traveled distance with 20 % of data that traveled at least 10 km (6. 2 miles), thus ensuring that the data was conveyed by Fougere as far as possible from the location where it has been produced. Utility analysis. While Fougere aims at improving the location privacy of workers, the utility of the resulting dataset should not be neglected. Figure  5  reports on the tradeoff between utility and anonymity of the configurations we considered. While the vanilla configuration (1) offers the highest utility with no anonymity, one can observe that the integration of Fougere seriously improves the anonymity of workers without seriously impacting the utility of the resulting dataset.
As mentioned in Figure  3 , the loss of 20 % utility is mainly due to crowdsourced data in sparsely populated areas that were retained by Fougere. Furthermore, adding some LPPMs (configurations 3-a and 3-b) strongly increase the anonymity of workers. Interestingly, one can observe that the weak privacy profile offers a good privacy/anonymity tradeoff compared to the strong privacy profile, which seriously harms the dataset utility without bringing any further improvement over anonymity. Robustness analysis. Regarding the effective privacy support offered by Fougere, we used the LPM 2 toolkit to evaluate the robustness of crowdsourced datasets that are uploaded through Fougere. We randomly select the crowdsourced data reported by one of the workers as the adversary knowledge required by LPM 2 to apply location privacy attacks and we depict in Figure  6  the reported robustness for 14 workers.
While LPM 2 successfully defeats worker 3 (used as the adversary knowledge), the other 14 workers clearly benefit from the integration of Fougere. In particular, we can observe that the integration of LPPMs complements efficiently our opportunistic dissemination scheme by supporting workers who are not located in a dense area and by offering similar privacy guarantees. Successfully location privacy attacks requires to combine different strategies to cope with the profile of workers. While Fougere offers the worker the possibility to manually adjust her privacy settings, one of the perspectives of this work consists in leveraging this configuration process by delivering privacy risk feedback that would guide her settings accordingly. By recommending the privacy settings of Fougere, we aim at maximizing the individual privacy of workers, while preserving the overall utility of the crowdsourced dataset (cf.
Thus, we are not assuming that our results can be used to generalize the impact of a specific LPPM on privacy. However, we believe that this paper contributes to prove that there is a clear positive impact for the privacy threats we considered. Reliability validity focuses on the possibility of replicating our experiments and results. We attempt to provide all the necessary details to replicate our study and our analysis. Furthermore, the reference implementation of Fougere, the input datasets, case studies and testing environment are made available online to leverage its reproduction by the research community. Construct validity has been covered by considering the convergent validity of privacy and utility properties. We observed that these two properties are related in practice, as the application of LPPMs tends to decrease the utility of the crowdsourced dataset.