dataesr/scientific-paragraphs-categorization huggingface dataset:
This means that the combined effect of detector-to-detector differences in gain (in e -ADU -1 ) [ADU: analogue-to-digital unit] and in overall quantum efficiency (QE) are removed. The unit of the counts in the flatfielded science images is termed "gain-normalised ADU". If QE differences are ignored, these counts can indeed be converted to electrons using a single gain valid for all detectors; this gain is about 4. 2 electrons per gain-normalised ADU. -Sky background correction or sky subtraction, performed by subtracting a sky image. The sky images are made by splitting the time sequence of science images into blocks; for this dataset blocks of 6 images were used. The objects in the images are masked (for this dataset using a mask we provided, which was based on existing K s and i-band data), and then these 6 object-masked images are combined to form the sky frame.
g. 0. 01 for J. The term does not correct for Galactic extinction as such. Processed data were made available to us by CASU on 2010 July 21. This included reduced individual images, calibration frames (darks, flats and sky frames) and _st stacks. We process these data further, as described in Sect. 3. 4. The essence is that we undo the CASU sky subtraction in the individual images, apply our own sky subtraction and stack these images, masking fake sources ("persistent images", see Sect. 3. 3) at the same time. We do not use the _st stacks, except that we use the photometric calibration contained in their headers. Reduced individual images were provided for all 258 obtained NB118 exposures, but only for 152 of the 164 obtained J-band exposures. The latter problem seems to be due to the J-band OBs being interrupted and restarted at two points due to minor technical (software) problems, which due to an unfortunate interplay between ESO's automatic image grading and the CASU pipeline made some images be rejected.
As the second step, we apply our own sky subtraction. We first undo the CASU sky subtraction. Specifically, to each individual reduced image from CASU we add the sky frame from CASU for that image. A new sky subtraction was performed with the following three key features: (1) masking of objects 7 http://youpi. terapix. fr/ was based on the above-mentioned first-pass stack of all the data plus a mask generated from the objects detected in the image itself (thus including artefacts not found in the stack, e. g. satellite trails and cosmics). For comparison, the CASU sky subtraction for this dataset used masking based on a mask we provided, which was based on existing K s and i-band data. (2) For each image a sky frame was constructed from the images (excluding the image itself) in a window of ±20 min centered on the image in question (also known as running sky or sliding sky).
Since the NB118 images typically are spaced by 4. 9 min, typically 8 images were used to create the sky frame, namely 4 images before and 4 after. For the images at the start or end of a sequence (an OB), the window was made asymmetric so that it would still contain 8 images (this differs from the UltraVISTA DR1 processing). For comparison, the CASU sky subtraction for this dataset consisted of creating a sky frame based on fixed groups of 6 NB118 images (corresponding to a time span of about 30 min) and using that sky frame to sky-subtract all 6 images. When inspecting the individual images as sky subtracted by CASU, we noted that the large-scale sky background was less flat for the images at the ends of such a 6-image sequence than for the images in the middle of the sequence. (3) Large-scale gradients were fitted and subtracted using SExtractor 8   (Bertin & Arnouts 1996) , with the objects being masked using the object masks also used in the sky subtraction process.
Finally an additional destriping was done. After the sky subraction, the weight maps were updated, assigning a value of zero to pixels where no sky frame could be computed; this happens when the given pixel is masked in all the images used to create the sky frame. Also the catalogues needed for SCAMP (see below) were remade. We note in passing that an earlier version of the TERAPIX sky-subtraction code had a bug involving images sometimes are shifted by 1 pixel. That bug has been fixed in this work. The bug affected the NB118 UltraVISTA DR1 stack. For each image the astrometric and photometric solutions were computed using SCAMP 9   (Bertin 2006). The photometric calibration is based on that provided by CASU. As described in Sect. 3. 2, a photometric zeropoint is only derived for the _st stacks, of which there are 18 for the NB118 data.
The zeropoint is listed as corrected to airmass unity, and we undo this correction, making the zeropoint applicable to the actual airmass. As the intial guess of the zeropoint for a given individual image we use the zeropoint of the _st stack in which the image is part. Using a catalogue for each image, SCAMP then compares the object fluxes between images and for each image computes a flux scale factor that will bring all images to agree. The absolute zeropoint is computed as the average over all the initial zeropoints: in the language of SCAMP all images were classified as photometric. While not all images may be photometric, the CASU zeropoints were computed from stacks of all the images, and therefore all images should be used to derive overall zeropoint. A conversion from Vega to AB was also included, using the same offsets as  McCracken et al.
9 nm. NDC has in April 2013 provided additional information about the filters based on recent measurements done by NDC on filter parts still in their possession, using more accurate equipment than was available originally. NDC now predicts that the central wavelength of the filters would move down 5. 2 nm with cooling and cone angle. We will now derive this shift in wavelength for the 16 NB118 filters installed in VIRCAM. When using narrow-band interference filters in fast convergent beams under cryogenic temperature, several considerations have to be taken into account (e. g. Reitmeyer 1967; Parker & Bland-Hawthorn 1998; Morelli 1991). First, the passband of the filters is temperature dependent. In VIRCAM there is no temperature sensor on the filters, only on the filter wheel hub. This latter temperature is reported in the headers of the individual images, and for this dataset the median value is 101.
4 K, with a range of 101. 1-101. 7 K. Paranal Science Operations estimate that (a) the typical filter temperature is 90 K ± 5 K, and (b) the filters typically are 5-10 K colder than the filter wheel hub, which for this dataset would imply a filter temperature of 91-96 K. We will assume a filter temperature of 90 K in our calculations. The temperature difference First row: collimated beam measurement at room temperature as provided by the manufacturer NDC. Second row: shift as expected due to the temperature difference. Third row: theoretical conversion to the convergent beam. Fourth row: response function including all theoretical filter transmittance, mirror reflectivity, detector QE, and atmospheric transmission (PWV = 1 mm, airmass = 1). In the plot, the 16 filters are separated into subsets of four filters corresponding to the four columns or stripes of the combined image (cf.
Particularly detectors 2 and 3 show a large standard deviation, meaning that they behave less like the other detectors (a behaviour that can also be seen e. g. in Fig. 11e ). Panel (d) shows the same for the J band, and here the variation is more than an order of magnitude smaller. The large detector-to-detector variation in the NB118 (but not in J-band) sky brightness means that for the above-mentioned standard star observations done at least 90 min into the night, detector 1 has a sky brightness in NB118 of 17. 8 mag/arcsec 2 that is on average 0. 9 mag/arcsec 2 darker than that in J, whereas detector 13 has sky brightness in NB118 of 16. 8 mag/arcsec 2 that is essentially identical to that in J. The CASU QC database contains 1080 observations of standard stars where the same field was observed in H, K s , J, Y, Z and NB118 (in that order) within 15 minutes.
In each image the median value over the image, i. e. the background, was subtracted. For each pixel, the fluxes f 0 and f 1 in the two images were measured. The figure shows a plot of these, binned in f 0. The persistence (i. e. detector memory effect) is seen by the fact that f 1 becomes positive above a certain threshold in f 0. The black solid line is a fit to the data (Eq. (A. 1)). The red dashed line is the threshold in f 0 at which the fit reaches a value of 3 ADU above the background, and this is used in our masking algorithm as the threshold for masking a given pixel (i. e. for excluding it when stacking the data). For detector 5 a lower threshold was manually set, as indicated by the blue dot-dashed line. The used thresholds are given on the panels and listed in Table  A. 1. The f 0 axis is logarithmic and runs from 10 to 70 000.
The algorithm uses a set of parameters, which we derived from a set of VISTA data with different background levels in order to make the algorithm more generally applicable. Specifically, we used data from this work, meaning NB118 (DIT = 280 s, NDIT = 1) and J (DIT = 30 s, NDIT = 4), and NB118 data from the VISTA SV programme 23  , with DIT ranging from 270 s to 860 s, all with NDIT = 1. The SV data were also chosen since some of these OBs also had an unfortunate nesting, meaning that masking of persistence is needed to make a clean stack. We proceeded as follows. We first created a list of images taken right after each other. These images are the individual images as reduced by CASU (Sect. 3. 2). We focused on a given image somewhere in the list, called image 0, and we then investigated how the pixel values in the 3 subsequent images (called image 1, image 2 and image 3) were correlated with the pixel values in image 0.
To reduce the noise we have binned the data in f 0 , and we have combined the different datasets that we analysed (see below). The persistence effect is clearly seen for all 16 detectors: when a certain threshold in f 0 is reached, f 1 becomes non-zero and increases with f 0. The datasets that we analysed consisted of 7 sets of images, which spanned the two filters (NB118 and J) and a range of DIT values. In each set of images we analysed two subsections. We also made versions of One difference is that the blue dot-dashed lines show the adopted masking threshold (given on the panels and in Table  A. 1), which is 30% lower than the threshold derived from the fit (marked by the red dashed lines). Some detectors do not show detectable persistence in image 2, and for these no threshold was derived.
A. 3 and A. 4 and Table A. 1. The algorithm works both for NDIT = 1 and NDIT > 1 images. In the case of NDIT > 1, the FITS file contains the sum of the NDIT sub-exposures. The pixel values simply need to be divided by NDIT. Care has to be taken that CASU sets the BSCALE header keyword to 1/NDIT, which makes some astronomical software (but not all) silently scale the data by this factor. An IDL implementation of the algorithm is available upon request. Our persistence masking algorithm was also used in the stacking of the UltraVISTA NB118 season 1 data  (McCracken et al. 2012 ).
While scientific communities extensively exploit simulations to validate their theories, the relevance of their results strongly depends on the realism of the dataset they use as an input. This statement is particularly true when considering human activity traces, which tend to be highly unpredictable. In this paper, we therefore introduce APISENSE, a distributed crowdsensing platform for collecting realistic activity traces. In particular, APISENSE provides to scientists a participative platform to help them to easily deploy their sensing experiments in the wild. Beyond the scientific contributions of this platform, the technical originality of APISENSE lies in its Cloud orientation and the dynamic deployment of scripts within the mobile devices of the participants. We validate this platform by reporting on various crowdsensing experiments we deployed using Android smartphones and comparing our solution to existing crowdsensing platforms.
Not only focusing on computing or communication capabilities, modern mobile devices are now equipped of a wide range of sensors enabling scientist to build a new class of datasets. Furthermore, the generalization of app stores or markets on many mobile phone platforms leverages the enrollment of participants to a larger scale than it was possible previously. Using cellphones to collect user activity traces is reported in the literature either as participatory sensing  [4] , which requires explicit user actions to share sensors data, or as opportunistic sensing where the mobile sensing application collect and share data without user involvement. These approaches have been largely used in the multiples research studies including traffic and road monitoring  [2] , social networking  [15]  or environmental monitoring  [16].
However, developing a sensing application to collect a specific dataset over a given population in not trivial. Indeed, a participatory and opportunistic sensing application needs to cope with a set of key challenges  [6, 12] , including energy limitation, privacy concern and needs to provide incentive mechanisms in order to attract participants. These constraints are making difficult, for scientists non expert in this field, to easily collect realistic datasets for their studies. But more importantly, the developed ad hoc applications may neglect privacy and security concerns, resulting in the disclosure of sensible user information. With regards to the state-of-the-art in this field, we therefore observe that current solutions lack of reusable approaches for collecting and exploiting crowd activity traces, which are usually difficult to setup and tied to specific data representations and device configurations.
We therefore believe that crowdsensing platforms require to evolve in order to become more open and widely accessible to scientific communities. In this context, we introduce APISENSE, an open platform targeting multiple research communities, and providing a lightweight way to build and deploy opportunistic sensing applications in order to collect dedicated datasets. This paper does not focus on user incentive challenges, which we already addressed in  [10]  to provide an overview of appropriate levers to encourage scientists and participants to contribute to such sensing experiments. The remainder of this paper is organized as follows. We provide an overview of the APISENSE platform by detailling the server-side infrastructure as well as the clientside application (cf. Section 2). Then, we report on the case studies we deployed in the wild using APISENSE (cf.
Distributed Crowdsensing Platform The APISENSE platform distinguishes between two roles. The former, called scientist, can be a researcher who wants to define and deploy an experiment over a large population of mobile users. The platform therefore provides a set of services allowing her to describe experimental requirements in a scripting language, deploying experiment scripts over a subset of participants and connect other services to the platform in order to extract and reuse dataset collected in other contexts (e. g. , visualization, analysis, replay). Technically, the server-side infrastructure of APISENSE is built on the principles of Cloud computing in order to offer a modular service-oriented architecture, which can be customized upon scientist requirements. The latter is the mobile phone user, identified as a participant.
Server-side Infrastructure The main objective of APISENSE is to provide to scientist a platform, which is open, easily extensible and configurable in order to be reused in various contexts. To achieve this goal, we designed the server-side infrastructure of APISENSE as an SCA distributed system (cf. Figure  1 ). The Service Component Architecture (SCA)  6  standard is a set of specifications for building distributed application based on Service-Oriented Architectures (SOA) and Component-Based Software Engineering (CBSE) principles. All the components building the server-side infrastructure of APISENSE are hosted by a Cloud computing infrastructure  [17]. The Scientist Frontend and Participant Frontend components are the endpoints for the two categories of users involved in the platform. Both components define all the services that can be remotely invoked by the scientists or the participants.
For example, once authenticated, the scientist can create new experiments, follow their progression, and exploit the collected dataset directly from this web interface. Crowdsensing Library. To reduce the learning curve, we decided to adopt standard scripting languages in order to ease the description of experiments by the scientists. We therefore propose the APISENSE crowdsensing library as an extension of the JavaScript, CoffeeScript, and Python languages, which provides an efficient mean to describe an experiment without any specific knowledge of mobile device programming technologies (e. g. , Android SDK). The choice of these host languages was mainly motivated by their native support for JSON (JavaScript Object Notation), which is a lightweight data-interchange format reducing the communication overhead.
The APISENSE crowdsensing library adopts a reactive programming model based on the enlistment of handlers, which are triggered upon the occurence of specific events (cf. Section 3). In addition to that, the API of APISENSE defines a set of sensing functions, which can be used to retrieve specific contextual data form sensors. The library supports a wide range of features to build dataset from built-in sensors proposed by smartphones technologies, such as GPS, compass, accelerometers, bluetooth, phone call, application status (installed, running) in the context of opportunistic crowdsensing, but also to specify participatory sensing experiments (e. g. , surveys). Privacy Filters. In addition to this script, the scientist can configure some privacy filters to limit the volume of collected data and enforce the privacy of the participants.
Client-side Library Although our solution could be extended to other Operating Systems, the APISENSE mobile application is currently based on the Android operating system for the following reasons. First, the Android operating system is popular and largely adopted by the population, unit sales for Android OS smartphones were ranked first among all smartphone OS handsets sold worldwide during 2011 with a market share of 50. 9% according to Gartner. Secondly, Android is an open platform supporting all the requirements for continuous sensing applications (e. g. , multitasking, background processing and ability to develop an application with continuous access to all the sensors), while for example, iOS 6 does no permit a continuous accelerometer sampling. A participant willing to be involved in one or more crowdsensing experiments proposed by a scientist can download the APISENSE mobile application by flashing the QR code published on APISENSE website, install it, and quickly create an account on the remote server infrastructure.
Once registered, the HTTP communications between the mobile device of the participant and the remote server infrastructure are authenticated and encrypted in order to reduce potential privacy leaks when transferring the collected datasets to the APISENSE server. From there, the participant can connect to the Experiment Store, download and execute one or several crowdsensing experiments proposed by scientists. Figure  2  depicts the APISENSE software architecture. Building on the top of Android SDK, this architecture is mainly composed of four main parts allowing i) to interpret experiment scripts (Facades, Scripting engine) ii) to establish connection with the remote server infrastructure (Network Manager), iii) to control the privacy parameters of the user (Privacy Manager), and iv) to control power saving strategies (Battery Manager).
Scripting Engine. Sensor Facades bridge the Android SDK with the Scripting Engine, which integrates scripting engines based on the JSR 223 specification. We build a middleware layer Bee. sense Scripting, which exposes the sensors that can be accessed from the experiment scripts. This layer covers three roles: a security role to prevent malicious calls of critical code for the mobile device, a efficiency role by including a cache mechanism to limit system calls and preserve the battery, and an accessibility role to leverage the development of crowdsensing experiments, as illustrated in Section 3. Battery Manager. Although the latest generations of smartphones provides very powerful computing capabilities, the major obstacle to enable continuous sensing application refers to their energy restrictions.
Therefore, in order to reduce the communication overhead with the remote server, which tends to be energy consuming, datasets are uploaded only when the mobile phone is charging. In particular, the battery manager component monitors the battery state and triggers the network manager component when the battery starts charging in order to send all the collected datasets to the remote server. Additionally, this component monitors the current battery level and suspends the scripting engine component when the battery level goes below a specific threshold (20% by default) in order to stop all running experiments. This threshold can be configured by the participant to decide the critical level of battery she wants to preserve to keep using her smartphone. Privacy Manager. In order to cope with the ethical issues related to crowdsensing activities, the APISENSE mobile application allows participants to adjust their privacy preferences in order to constrain the conditions under which the experiments can collect data.
Revealing Users' Identity from Mobility Traces This first experiment aimed at identifying the potential privacy leaks related to the sporadic disclosure of user's locations. To support this experiment, we developed a APISENSE script, which reports every hour the location of a participant, known as Alice. Listing 1. 1 describes the Python script we used to realize this experiment. This script subscribes to the periodic scheduler provided by the time facade in order to trigger the associated lambda function every hour. This function dumps a timestamped longitude/latitude position of Alice, which is automatically forwarded to the server. While this periodic report can be considered as anonymous since no identifier is included in the dataset, this study has shown that the identity of Alice can be semiautomatically be inferred from her mobility traces.
To do so, we built a mobility model from the dataset we collected in order to identify clusters of Alice's locations as her points of interest (POI). By analyzing the size of the clusters and their relative timestamps, we can guess that the largest POI in the night relates to the house of Alice. Invoking a geocoding service with the center of this POI provides us a postal address, which can be used as an input to the yellow pages in order to retrieve a list of candidate names. In parallel, we can identify the places associated to the other POIs by using the Foursquare API, which provides a list of potential places where Alice is used to go. From there, we evaluate the results of search queries made on Google by combining candidate names and places and we rank the names based on the number of pertinent results obtained for each name.