dataesr/scientific-paragraphs-categorization huggingface dataset:
56 E. g. Borella et al. (2006). 57 For the biobank entry/exit points. 58 Articles 99-102, 121-122 LIL. 59 https://www. cnil. fr/fr/les-labels-cnil. Since March 2018, personal data protection labels are no longer issued by the CNIL itself but by certified organisations. 60 Article L. 1111-8 CSP. See Agence Française de la Santé Numérique (ASIP) website: https:// esante. gouv. fr/labels-certifications/hebergement-des-donnees-de-sante. Since 2006, the CNIL has followed the GDPR approach based on operators' accountability and modified the declaration/authorisation system to create simplified procedures intended to ensure data subjects' protection while favouring research, innovation and competitiveness. The CNIL adopted referentials (Méthodologies de Référence, MR) specifying data protection rules in research contexts.
Processing within a MR scope can be implemented after a commitment of compliance with the CNIL. We will concentrate on MR001, 61 MR003 62 and MR004 63 which are particularly relevant for biobanks. The use of a particular MR depends on the qualification of the research activity. In any cases, samples and data collection must be justified. The MRs articulate the LIL and the PHC. Processing falling outside the MRs' scope need CNIL authorisation. 64 Biobanks receiving samples and data will be checkpoints. For each selected MR, Table  1  summarises the data protection rules impacting depositors to biobanks and biobanks' users.
A renewal of an individual's consent will only be necessary in case of the procurement of new samples for genetic analyses. The scope of this PHC article can be questioned as it does not explicitly cover the reuse of genetic databases without attached samples. We favour a broad interpretation with the same opt-out process for the reuse of genetic data. Table  2  provides an overview of the 3 MRs data protection principles and individual rights to be respected by depositors and access applicants to biobank 61 Délibération n°2018-153 du 3 mai 2018, JORF 13 juillet 2018, texte n°108. 62 Délibération n°2018-154 du 3 mai 2018, JORF 13 juillet 2018, texte n°109. 63 Délibération n°2018-155 du 3 mai 2018, JORF 13 juillet 2018, texte n°110. 64 Article 66(III), 76 LIL. 65 CNIL. Conformité RGPD: comment recueillir le consentement des personnes? See: https:// www.
Exceptions Regarding Data Subjects' Rights for Personal Data Processing in Research Biobanking necessitates special measures to be able to process personal data over the long-term for future, quite broadly defined, uses. The GDPR has made available several means for Member States to derogate from individual rights for the benefit of scientific developments under certain conditions. The GDPR's flexibility is preserved in French law. The CNIL created a software, open source and free, available in English and 18 languages, to perform and manage DPIA in compliance with the GDPR: https://www. cnil. fr/en/pia-software-20-available-and-growth-pia-ecosystem Regarding in particular the right to access, the LIL includes a special derogation 'where the personal data are stored in a form that manifestly excludes any risk as regard to privacy and data protection'.
68 o, these exceptions could be invoked either before including the indirectly collected data in the biobank or after, at the time of accessing the bioresources, for the reuse of existing biobanks' samples or databases in research, including for genetic research. The data subject can also decide not to be informed where it would lead to reveal a diagnosis or prognosis. 69 Such exceptional circumstances necessitate justification and could trigger, for the research promoter, the CNIL authorisation procedure, the MRs requiring data subjects to be informed, and REC approval for reuses in RIPH. Article 110 LIL allows derogations to the right to oppose to a processing where this latter answers to a legal obligation imposed to the controller or processor or where it is explicitly planned by the act authorising the processing.
Recently, lawyers criticised 70 the way the GDPR forces communication of the research promoter's DPO contact details within information notices provided to data subjects in clinical trials, claiming that DPO involvement could breach medical secret of which the sole investigator is the guarantor. Furthermore, they claim that the Clinical Trial Regulation is the special law that makes the investigator the only contact of the participants for exercising their rights. Therefore, in their opinion, DPO contact should not be provided. To date, the CNIL has not gone against GDPR. The LIL refers to the GDPR provisions regarding the implementation of other rights, in particular regarding the right to limit processing, the right to data portability, the right to oppose and the data breach notification process.
5. 1). 75 Outside archiving, the public interest purpose is not mentioned to exempt from the storage limitation principle in a research context. Data controllers involved in archiving in the public interest can derogate 76 from the rights established under Articles 15, 16 and 18 to 21 of the GDPR. But this blurry notion is problematic, in particular where competent authorities (CNIL and INDS) can refuse data processing requests based on this criterion. In 2016, under the auspices of the INDS, a legal interpretation 77 of the notion enabled the identification of useful specifications for avoiding misunderstandings. This expertise provides that 'public interest' is a synonym of 'general interest' and 'collective benefit'. Therefore, any uses essentially motivated by private purposes or aiming at re-identifying patients, or targeting prescription behaviours of health professionals for commercial purposes (e.
French Specificities French law integrates the GDPR and further develops individual rights on several points of interest for researchers. First, the LIL states that personal data stored as research results are only accessible and modifiable by persons authorised by the data controller, in the respect of deontology. Personal data as research results must be anonymised before communication to thirds, except where the third's interest in the communication overweigh data subject's ones. In this regard the CNIL can approve anonymisation mechanisms. 78 Then, anonymised data are no longer subject to the LIL. Second, while the GDPR excludes its application to deceased persons, the LIL ensures privacy protection after a data subject's death with a new right to write and record advanced directives on personal data management.
Perspectives Regarding Research and Biobanking While a decree about national derogations from certain data subjects' rights under Article 89 GDPR is expected, France launched in 2016 its national plan for genomicmedicine 80 with the aim of completing every year 200,000 human genome sequences. This will necessitate efforts in terms of samples and data storage and processing capacities but also a clear and appropriate legal and ethical framework. These new activities, plus the current revision of the bioethics law, highlight the new challenges for research biobanking. A first set of challenges relates to the development of new techniques in genomics and the future capability to store and use bigger sets of genomic data in the form of Whole Genome/Exome sequencing. The CCNE 81 and the State Council 82 have taken a position on this matter, both favouring the practice of 'enlarged informed consent' or 'consent by delegation' based on the monitoring functions of competent and independent trusted third parties 83 for genetic research, notably for the purpose of reuse of data.
See p. 157. 83 E. g. Research ethics committees. Biobanks' internal independent review mechanisms could qualify. committees or an independent trusted third party in conjunction with supplying a high level of information of participants. However, the Council rejected solutions such as broad unspecified consent and dynamic consent because of their legal or technical implications (see footnote 10). A second set of issues relates to research on human embryos, notably the creation, in research, of transgenic, chimeric embryos; the use of induced pluripotent stem cells and their ethical impact on the use of 'natural' human embryos; and the need to recognise by law the 14-day deadline limiting embryo cultures in research. A final challenge is the upcoming debate from a collective perspective of the implications of the production and use of large datasets through artificial intelligence or genetic testing/sequencing to be routinely provided in the health care system and/or commercially.
Conclusions Currently, the French regulatory environment for research biobanking remains complex and fragmented due to the fragmented nature of the legislation to comply with. Data protection law is a common feature of any type of health research and biobank processing of personal data, whether the latter are attached to, or generated from, a biological sample, with a risk-based approach for identifying requirements to be met by researchers and biobankers. The GDPR has been fully implemented, with the potential for further developments offered by its Article 89. We acknowledge the efforts made by the CNIL to provide operators with explanatory and practical toolkits that ease both procedures and GDPR-compliance. CNIL action is pragmatic and proactive, which are good qualities that can be used by DPO networks for the purpose of boosting the understanding and adoption of a data protection culture, and will lead to innovations in data protection.
Participants Participants were recruited through call for volunteers announcements posted on students, retirees and gamblers Facebook groups. No financial compensation nor any recompense was offered in exchange for participation. Data were collected online, using the Qualtrics platform, between 29 January 2017 and 31 March 2017. Six hundred and eighty-nine people took part in the study; 399 fully responded to the online protocol (socio-demographic questionnaire, game, and illusion of control and TP questionnaires). We excluded eight participants as they left the game stage before the first round, leaving data from 391 participants; 47 per cent gamble at least occasionally (for gambling habits by age, see Figure  1 ). Table  1  shows the participants' socio-demographic information. Participants were divided into eight groups according to age (emerging adults, young adults, middle-aged adults, seniors) and sequence of outcomes (ascending: losses then gains versus descending: gains then losses).
g. known periodic variable stars and Solar System objects). This stage produces a list of candidate alerts (see Section 2. 4 and after for details). All data are handled by the GSA PostgreSQL database which make use of the Quad Tree Cube (Q3C) software  (Koposov & Bartunov 2006). Further filtering removes the alert candidates that are probably due to interference effects from neighbouring sources (Section 2. 7). Human inspection (i. e. eyeballing) identifies those candidates suitable for publication (Section 2. 8). Finally, the chosen alerts are published immediately to the World Wide Web via the Alerts Website, TNS entries and VOEvents (see Section 2. 9). Each alert is published with a timestamp corresponding to the observation time by Gaia (in barycentric coordinate time, TCB) as well as the time of publication of the alert (in Coordinated Universal Time, UTC).
As an example, in the first half of operations during 2016, we employed a source-density map of the sky to reject all transients found in the most crowded regions (the density map was constructed from the GSA database). Once more thorough environmental filters were developed and tested, the use of the density map was discontinued (i. e. by June 2016). In the following sections we detail the most common types of false positive, which are either trapped and rejected in AlertPipe, or in two cases, flagged for inspection by the eyeball team. The most common types of automatically rejected alerts are summarised in Figure  6. The two classes of candidate false alert that require a human decision are (i) Solar System objects (SSO), and (ii) variable star in Gaia DR2 (Section 2. 7. 6). The first case is very rare, and almost always is unrelated to the alert (e.
2014 ). 2. Transients which are spatially-coincident with externally reported events. We maintain a comprehensive list of events discovered in other surveys within the GSA database (see Section 2. 8 for more details). 3. An additional and independent filter (LWfilter) was brought into operation in July 2018. LWfilter uses auxiliary data from other surveys to classify the source (star, galaxy, AGN). Additionally, each alerting lightcurve is fit with a microlensing model  (Paczynski 1996 ) in order to identify potential microlensing events. BP-RP colour is also used to identify blue flares (e. g. CVs and Be stars) and very red variables (e. g. long-period variables, such as Miras). Alerts are then inspected visually (by the Warsaw team) and added into the list for eyeballing. Until the end of 2019 (IDT run 4724, i.
2014)  to the raw BP/RP spectra, and shared the results with the eyeballers. GS-TEC takes a Bayesian approach to model observed spectra, using a constructed reference spectral library and literature-driven priors. GS-TEC can classify SN, AGN and stars down to G=19, however the classifier was disabled due to its significant execution time. Auxiliary data are parsed from a variety of sources, and presented to the eyeballers, to help understand the context of a Gaia transient detection. -To allow a visual inspection of the alert's location, the Eyeballing App shows the Aladin Lite  (Boch & Fernique 2014)  and SDSS finding charts. -Results of positional queries to the Simbad  (Wenger et al. 2000) , NED and VSX databases 7 , to determine whether it is an already-known transient or variable object.
-The list of YSOs described in Section 2. 7. 7. -To aid the rejection of spurious transients arising from contamination by Solar System objects, we also display data on nearby planets, their satellites, and minor planets. -These data also contain classification information for large numbers of transient events which are shared with the eyeballer, and used at the point of publication. Classifications often arrive to the database after publication of an alert. As part of the publisher app, these can be viewed and the alert record updated (at the discretion of the operator). The bulk of classifications are reported via TNS (supernovae for the most part), but we also receive classifications on microlensing events from the Warsaw group through the publisher app (see  Wyrzykowski et al. 2020 ).
Photometry All transit photometry is calibrated on-the-fly via a database function. The parameters used by the function are derived from the Photometric One Day Calibration (PODC, see Appendix B), and are generated with a one-day cadence. The operation to build the calibration is run roughly monthly, thus at the time of alert, the calibration can be a month out of date (and sometimes more). To test the precision of PODC, we selected a random sample of 184 000 sources which lie in the SDSS DR7 footprint (avoiding the most crowded regions of the Galactic plane). We required the sources to have a minimum of 10 Gaia field-of-view transits, and we used the median of the per-CCD PODC calibrated fluxes as the per-transit CCD flux. We used the standard deviation of multiple transits for a source as a measurement of the precision of a single Gaia transit in the Alerts system.
Future Gaia data releases, based on more observations, will remove the apparent parallaxes of QSOs included in GDR2. Another application of the alert colour information is demonstrated in Figure  17 , where we show a subsample of sources and follow their evolution through the diagram as their prop-A&A proofs: manuscript no. gsa   2019 ), which includes a large sample of periodic and non-periodic variables. We have colour coded sources by classification. Note that some sources show a large spread in BP-RP, indicating that our simple data curation may be insufficient, and a proper spectral calibration is required for more detailed analysis; fully calibrated epoch BP and RP spectra will be released in DR4. Nevertheless, we note that CVs and XRBs have bluer colours in outburst, as expected. We can also see the reddening of emission that occurs in novae between 10 and 100 days post-peak (e.
When an alert is published, all available internally calibrated Gaia G band photometry, and uncalibrated BP/RP spectra, of that object becomes public. This includes subsequent measurements of the same object, with the data added to the alert webpages when processed. We currently publish alerts at a rate of approximately 12 events day -1 , and almost 25 per cent of them are ultimately classified. The published classifications are dominated by SNe, but we show that this is biased by the extensive supernova followup campaigns. The bulk of our unclassified alerts reside in the Galactic plane, and are therefore likely to be Galactic in origin. As an experiment, we built a simple probabilistic alert classifier using uncalibrated BP-RP colour and Gaia DR2 parallax (where available), leading to the identification of 638 new candidate CVs and 202 new candidates YSOs.
We also show that per-transit data (G-band and uncalibrated BP and RP colours) for alerting sources with available DR2 parallaxes can be used to trace the evolution of the transient's position in the colourmagnitude diagram, revealing its nature without the need for spectroscopic confirmation. We investigated the astrometry of GSA and showed that the accuracy of individual alert detections is 55 milliarcseconds when compared to Gaia DR2, and is independent of source magnitude. The photometry of our alerts has a precision of 1 per cent for sources around G = 13, falling to around 10 per cent at G = 20. 7. Most alerting sources must reach G = 19, where the median standard deviation is 0. 031 mag. GSA suffers very low levels of contamination from false positives. We showed that the minimum purity of the survey in uncrowded regions for sources with G < 17 is 93 per cent.
Ancillary data is harder to find in the Galactic plane, and for fainter sources, but we find no evidence of additional sources of contamination at faint magnitudes, or in crowded regions. We also investigated the completeness of the survey, by comparing Gaia discoveries to classified supernovae published via TNS (across all sub-types). We measure an overall completeness of 46 per cent, and find that the Gaia scanning law, combined with our requirement for two Gaia detections, dominates our ability to detect a supernova. Our completeness for supernovae with two or more scans is 79 per cent, unless within 3 arcseconds of the nucleus of the host galaxy, where it drops significantly. Finally, we note that a total of 2612 alerts spanning observations taken between 25 July 2014 and 28 May 2017 will be included in Gaia DR3 in a supplementary table.
S. T. Hodgkin et al. : Gaia Photometric Science Alerts server holds the primary copies of the databases and an identical machine holds a secondary, read-only copy of each database, kept current by synchronous replication. These replica databases were originally installed as protection against data loss, a role that has never been exercised. They now serve to increase performance by spreading the query load across two servers. Each database server has 176 TB of bulk-data space, arranged as a RAID 6. The alerts computers are interconnected, and connected to DPCI, by an Infiniband network. The alerts pipeline is divided into: a batch application for ingestion of data, and primary data-reduction; a batch application for filtering of candidate alerts; a web application for human assessment of candidates surviving automatic filtering; and further web-applications for publishing selected alerts.
The data reduction application, which handles the greatest volume of data in each run, is assigned its own 64-core server. An identical server is held in readiness to take over should the first one fail. In practice, this reserve server is used for testing, for exploration of new algorithms, and occasionally for large-scale rearrangements of the main database. Each data-reduction server has 22 TB of data space to receive new data from DPCI. This holds roughly six months of data and allows reprocessing to correct errors or improve results over this span of the mission. The other applications run in virtual machines hosted on a pair of smaller servers. There is also a disc-server for back-ups of the main databases, with a capacity of 284 TB. The main data-reduction application was written in Java, for performance and to conform to DPAC standards.
The web applications were written in Python, using the Django framework, for ease of development. The batch-filtering application was also written in Python. The most notable feature of the computing design is that the bulk of the data is stored in the PostgreSQL database. Compare this with the more common arrangement where only metadata are stored in the RDBMS, while the bulk is stored in flat files. Ingesting both the full input data-set, and the results of processing, reduces significantly the amount of code needed to simply move data, and allows extreme flexibility in writing the applications that consume, analyse and display the results. However, it is challenging to obtain sufficient throughput from the relational architecture. Many design choices in the database and applications are forced by the need to reduce a day's data in six hours or less.
At the time of writing, the PostgreSQL system occupies 110 TB of its storage (with some unknown and variable fraction of this reclaimable for new data), almost all of which space is taken up in ingesting and recording the transit data. This is exceptionally large for a database that is extended daily and in which roughly 5% of the contents must be scanned for daily processing. The data-set is far too large to fit comfortably into a single table for each record type, and is therefore partitioned by position of the sources into level-5 (n side = 32) HEALpixels; there are 12,288 tables for each record type that deals with individual transits or sources. This partitioning is much coarser than the level-12 mesh used elsewhere by DPAC to group Gaia sources into catalogues. The level-5 mesh was chosen because it is the closest match to the width of the Gaia field of view.
We knew from pre-mission tests that the application was not the bottleneck and could be given sufficient threads of execution to saturate the database engine. Later experiments showed that the database performance, for this application, is limited by CPU power on the database server, not by I/O capacity; the complexity of the queries dominates. Therefore, the ideal arrangement is to have one application thread accessing the database per CPU core on the database server: 32 such threads in our system. The datareduction servers have 64 CPU cores and to use these we run two second-level threads per HEALpixel to consume the raised data from the memory of the data-reduction server. We feel that we have reached a practical limit in the size of database that can be hosted on a single RDBMS-server.
Any major expansion (beyond routine addition of daily data for the remaining mission) or rearrangement of the database would drive us to a multiserver solution with partitioning across computers. This would be massively more expensive and we are happy that we have managed to do the work with single servers. We note that while it is possible to tune the daily data-flow for acceptable performance, whole-database operations are uncomfortably slow. This includes data-model changes that add columns; weekly backups of the database; and especially changes to the IDT workingcatalogue that force us to update the transit-source mapping for the whole data-set. The latter operation requires down-time of at least one week. When using PostgreSQL at this scale, any routine operation becomes a significant campaign that requires careful planning.