dataesr/scientific-paragraphs-categorization huggingface dataset:
, 2011). The two European studies we manage require a tailored PIx infrastructure. Existing frameworks neither completely handle the diversity of our PIx requirements and project life cycle nor provide efficient tools to collect, check quality, and publish evolving data. Additional developments were required for building such complete infrastructure. We based these developments on a more general framework than the dedicated applications described above. In collaboration with Logilab company (Logilab SA, Paris, France), we developed three highly adaptive web services, based on the CubicWeb (CW) pure-Python framework, aimed at creating a (1) multi-center upload platform, (2) collaborative quality assessment platform, and (3) publication platform with massive-download features  (Logilab, 2000).
These developments were originally instituted for IMAGEN and EU-AIMS projects in order to host their data about mental health in adolescents  (Schumann et al. , 2010)  and autism  (Murphy and Spooren, 2012) , respectively. The corresponding studies require key features such as upload/browse published data from the web, dynamic selection and filtering of displayed data, support for flexible download operations, high-level request language, multilevel access rights, remote data access, remote user access rights management, collaborative QC, and interoperability.
MATeRIAlS AnD MeThoDS The three services described in the introduction were handled in distinct developments. Section 2. 1 presents the CW framework capabilities, Sections 2. 2 and 2. 4 introduce the upload and publication web services through which the tailored requirements of PIx studies are satisfied. Furthermore, section 2. 3 describes a collaborative rating web service that helps users to assess the data quality, and section 2. 5 describes a Python API that remotely queries these web services.
cubicWeb overview All the implemented services are based on the CW framework  (Logilab, 2000). We choose a high level pure-Python framework that bridges web technologies and database engines. This choice was also based on the expertise and experience of people from our laboratory and a tight collaboration with Logilab  (Michel et al. , 2013; Papadopoulos Orfanos et al. , 2015). CW distribution is organized in a core part and a set of basic Python modules, referred to as cubes, which can be used to efficiently generate web applications. The core of the CW framework, developed under the LGPL license, is constructed from well-established technologies (SQL, Python, web technologies such as HTML5 and Javascript). The main characteristics of the CW framework are given as follows: 1. CW defines its data model with Python classes and automatically generates the underlying database structure.
5. Thanks to the semantic nature of CW, all developments inherit the possibility to follow existing or emerging ontologies, thereby facilitating sharing, access, and processing. 6. CW has a security system that grants fine-grained access to the data. This system is similar to the row-level security and policies available in the most recent versions of PostgreSQL, and links access rights to entities/relations in the schema. Each entity type has a set of attributes and relations, and permissions that define who can add, read, update, or delete such an entity and associated relations. 7. CW may run either as a standalone application or behind an Apache front server. We refer to both settings as a data sharing service (DSS) (cf. Figure  1 ). 8. CW can be configured to run with various database engines.
For the best performance, PostgreSQL is recommended. Starting from the basic CW distributions, our suite of services is composed of an assembly of Python modules, also referred to as cubes. The Python language is widely used in scientific communities and facilitates interfacing with major or emerging processing tools such as Nipype  (Gorgolewski et al. , 2011) , Biopython  (Chapman and Chang, 2000) , Nilearn  (Abraham et al. , 2014) , and Morphologist  (Fischer et al. , 2012). Application cubes, built over system cubes, and business logic cubes can be distinguished. The system cubes ensure interactions with the operating system and middlewares. For example, they connect to LDAP for user credentials and information or invoke  FUSE (2002)  as a module to construct virtual file systems in a user repository for downloading.
The business logic cubes essentially provide the database schema and the application cubes define the access rights and the web interface. Among the available Python-based frameworks, we chose CW. A major advantage of CW is the RQL language which brings end users a query interface adapted for PIx data sharing. It simplifies and improves the user experience in searching for custom datasets. RQL also avoids the use of a complicated object relational mapper (ORM), is focused on browsing relations, and allows requesting several DSS at once. The semantic nature of this request language requires the user to know only about the used data model defined as a graph (nothing about the underlying low-level relational model). This data model simplification and the expressiveness of RQL help users writing custom requests, while most of existing DSS do not expose a query language but offer a limited predefined number of operations that can be carefully designed to be efficient (e.
Structured Data Upload Service In PIx studies, massive and complex data are gathered from multiple data acquisition centers or devices (topic (1)). Each collected dataset must be mapped with definitions that follow consensus representation rules. Those definitions are grouped in data dictionaries that ideally follow standards  (Rockhold and Bishop, 2012) , but they are mainly manufacturer and/or site specific. Thus, an efficient and versatile tool is required for mapping the different data dictionaries during the collection process. Leveraging those ideas, we implemented a flexible upload mechanism, a system cube named rql_upload 1 and provided a web frontend by integrating this cube with the application cube named PIWS 2 (Population Imaging Web Service, cf. Figure  1 ). Based on a CW feature that allows database completion through online HTML forms, these two cubes were developed to collect, in a DSS, both raw data and metadata.
CW also enables the customization of triggers that determine the integrity of the uploaded data: synchronous and asynchronous validation filters can be specified and applied to each upload dataset. The upload proceeds as follows (cf. Figure  2 ): 1. Synchronous validations are applied to each form field (e. g. , to check the extension of a file or the structure of an Excel table ). If the validation filtering fails, then the web form is refreshed and an adapted feedback is displayed. 2. After synchronous validation, all the uploaded raw data/ metadata are stored in generic entities and a "Quarantine" status is set. To avoid cluttering of the database and to ease file manipulation, files are stored in the central repository but remain accessible through the database. File hashes are automatically computed and indexed in order to assess data integrity.
3. To update the upload status from "Quarantine" to "Rejected"/"Validated, " automatic asynchronous validations can be configured in the service as looping tasks. Those validation filters are project and/or data and/or upload specific and generate adapted feedbacks for users and data managers. Moreover, any entity or relation may be endowed with access permission rules  (Logilab, 2000). Based on the CW security mechanisms, a customized security model was implemented for our upload DSS (it can be extended later). Only specific groups have the authorization to upload, and users can only access the uploads, which they are interested in. The customization of these 1 http://neurospin. github. io/rql_upload. 2 http://neurospin. github. io/piws. core features allowed the creation of an upload web service that is completely described in a single JSON file.
collaborative Quality control Service Owing to the large amount of data gathered/analyzed in PIx studies, we must consider more sophisticated operating procedures than simple quality controls (QCs), where datasets are usually only rated once by a handful of individuals. This issue can be addressed by implementing a web-based collaborative quality control process that will also remove the bias introduced by isolated raters (topic (2)). Moreover, for the studies we manage, we also added controlled vocabulary description to the ratings. We achieve these goals by implementing a flexible collaborative rating mechanism, i. e. , an application cube named zeijemol. 3 s in section 2. 2, a collaborative quality control DSS is entirely described in a single JSON file. This file consists, on the one hand, of the list of elements that will be rated (e.
g. , a Nifti image, a FreeSurfer segmentation, or a motion curve in a diffusion sequence of an individual) and, on the other hand, related quality indicators (e. g. , binary good/bad, controlled vocabulary, scaled rating). Each element is displayed by one of the embedded viewers such as triplanar view or mesh rendering (cf. Figure  3 ). The QC results are stored directly in the database. The emergence of such DSS will allow machine learning techniques to learn new classifiers to automatized the quality control task. The QC scores may also be directly used as prior knowledge during the analysis stage.
publication Service In PIx studies, data collection and QC are followed by data anonymization, ordering, and analysis. Ultimately, data are made available to the acquisition partners or the scientific community (topic (3)). While browsing the database content through the web interface, users expect to be able to download the displayed files as well as the data description and rich links between the data, also referred to as metadata. An intuitive and reliable sharing mechanism is therefore crucial as large amounts of heterogeneous evolving data must be provided. Furthermore, for the studies we manage, access rights are split along time points, scan types, questionnaires, or questions to match the consortia multilevel access permissions. Therefore, we implemented a system cube named rql_download 4  and provided a web frontend by integrating this cube with PIWS 5  whenever it was used in a publication service (cf.
Section 2. 4. 3 describes two approaches of rql_download, based on two basic softwares (FUSE or Twisted), that give users access to their saved searches. This section also discusses the pros and cons of both. Section 2. 4. 4 presents a suitable strategy for setting user rights from the CW security system. Finally, section 2. 4. 5 presents a descriptive data insertion mechanism, as a set of Python scripts.
A Dedicated Structure for Imaging Genomics Questionnaire Data The database schema was developed for handling multi-time point/multimodal datasets in the brainomics business logic cube. 6  This schema supports general information such as subject data and associated metadata (age, handedness, sex, …), acquisition center definitions, multimodal imaging datasets, clinical/ behavioral records, processed data, and some genomic concepts 6 https://github. com/neurospin/brainomics2. (including chromosomes, genes, SNPs, or genomic platforms). An excerpt of the produced schema is shown in Figure  4.
The major advantage of this approach is the use of the standard sFTP port. However, additional system level configurations are required during the installation of the DSS in order to set the user home directories and system accounts. 2. Twisted server: This approach is characterized by a Python process that creates a Twisted 8  event-driven networking server, retrieves all the searches in the database, and exposes the files via sFTP through the created server. Again, this process acts as a filter on the central repository where a user only sees a subset of files/directories. In this case, the authentication and file transfers are directly operated by CW. The major advantage of this strategy is that no system level configuration is required. However, listening on a non-default sFTP port, which could lead to firewall issues, is sometimes required.
The Unified Insertion Procedure A unified insertion module is provided as a set of Python scripts to insert neuroimaging, genomic, and clinical data such as scans, genomic measures, questionnaires, and processing steps. These scripts were helpful in efficiently managing the large amount of evolving data in our projects. The indexed data are uniformly organized according to the schema structure and thus take advantage of all the aforementioned developments (e. g. , shopping cart mechanism cf. section 2. 4. 2, security model cf. section 2. 4. 4, and common renderings cf. Figure  6 ). Generating such a DSS with these scripts can be performed without specific CW knowledge. Indeed, only a rich description of the data to be published is required as a set of Python dictionary objects.
A Transverse python Module to Remotely connect a cubicWeb DSS With the aforementioned capabilities of the DSS, a user manually selects and downloads data through graphical interfaces in order to analyze them locally (cf. sections 2. 4. 2 and 2. 4. 3). In the case of an evolving DSS, the downloaded data must be regularly updated, and this manual process becomes time consuming and error prone when large and heterogeneous data are considered. Moreover, the metadata, such as quality scores, used to specify the dataset to download are also likely to change. Therefore, to achieve the analysis of up-to-date data stored in a DSS, direct programmatic interaction with the DSS is recommended. In the neuroimaging and neuroscience communities, data are typically analyzed by using Python scripts. Classically, the systems provide RESTful web services such as XNAT, with a Python API  (Schwartz et al.
, 2012). Inheriting from the RQL request language, our publication DSS (cf. section 2. 4) offers a rich interface to access the data. We provide a regular Python module, named cwbrowser, 9  that implements a Python API to connect and send RQL to a remote DSS based on the CW framework. This module is completely independent of CW (no CW installation required) and similar to the CW distribution cwclientlib cube. A publication DSS, as described in section 2. 4, can be requested by the cwbrowser module that embeds the previously described data selection and shopping cart capabilities. It automatically fills and saves a shopping cart from a custom RQL request, downloads the associated virtual directories onto the local file system, and returns the complete requested dataset. The returned dataset contains metadata stored in the DSS such as subject sex or quality scores, and the path to the downloaded directories.
ReSUlTS Our laboratory operates several DSS for the IMAGEN project about mental health in adolescents  (Schumann et al. , 2010)  and the EU-AIMS project about autism  (Murphy and Spooren, 2012). Other DSS are currently under development to support new and ongoing initiatives. Note that the access to both IMAGEN and EU-AIMS datasets is (to date) restricted. In the IMAGEN project, 2,000 subjects are monitored on at least two visits (the third follow-up is underway). T1, T2, FLAIR, DWI, B0, task fMRI, resting-state fMRI scans are acquired, as well as clinical/behavioral records, genotyping, gene expression, and methylation. A publication DSS at https:// imagen2. cea. fr/database enables us to more than 37,000 scans, 32,000 processing results, and 16 million distinct variables. In the near future, an upload DSS will allow us to collect a new time point.
In the EU-AIMS project, 1,500 subjects (from 6 months to 30 years old) are monitored on several visits through two distinct studies. T1, T2, FLAIR, DWI, B0, task fMRI, resting-state fMRI, and spectroscopy scans are acquired, as well as clinical/behavioral records, EEG, eye-tracking, gene expression, and methylation. An upload DSS at https://eu-aims. cea. fr/database provides the means for collecting this data from 10 centers across Europe. In addition, a collaborative quality check DSS at https://eu-aims. cea. fr/qc allows us to assess the uploaded data quality, and a publication DSS at https://eu-aims. cea. fr/data_repository enables us to share more than 13,000 scans, 12,000 processing results, and 15 million distinct variables. DIScUSSIon AnD conclUSIon.
lightweight Solution for Data Sharing We developed a novel and lightweight PIx software infrastructure exclusively based on the CW framework. We offer a suite of CW tools that facilitates the creation of a DSS. The system delivers the data to users based on the principle of "what you see is what you get": users define their datasets of interest by browsing the database. Thanks to the RQL and the developed Python API, remote query of a DSS is easy and intuitive. In this environment, core features such as the schema definition, the web rendering of the database content, and the semantic request language are provided by a few Python codes at the heart of the CW framework. Our DSS can use any database engine, offers an access permission mechanism, and can be smoothly integrated with the standard Apache environment.
Future Directions Our developments inherit the web semantic capabilities embedded in the CW framework. Thanks to this key feature, numerous problems of interoperability can be efficiently tackled using emerging ontologies and standards in neuroinformatics, neurosciences, and bioinformatics, such as the NIDM standard for data exchange  (Keator et al. , 2013) , the Cognitive Atlas Ontology  (Poldrack et al. , 2011), and OntoNeuroLOG (Gibaud et al. , 2011)  for data annotation, or the Bio2RDF for the federation of large datasets using open-source semantic web technologies  (Dumontier et al. , 2014). The annotation of our datasets, with respect to these ontologies, is ongoing. Ultimately, should all DSS follow standard ontologies, RQL would provide new crossprojects querying possibilities. Although the CW framework is already used successfully in several commercial applications, it would be interesting to evaluate the CW framework performances on our DSS with Logilab dedicated tools.
FIgURe5| Illustration of the download process via the proposed shopping cart mechanism. (A) facet filter bar when all the scans ("Scan" entities) are requested (as highlighted in bold, the user has selected only the "FU2" time point and the diffusion MRI "DTI" scans), (B) the view corresponding to the filtered dataset, (c) add this new search to the cart (by activating these filtering options, the save RQL path search will be automatically updated), (D) a new search has been created, and (e) the download of the search and associated files as presented in FileZilla.
STATE MACHINES AND UML EVENTS This section presents overview of using UML State Machines for modeling and designing reactive software applications. A state machine is used for describing the behavior of either a class in objectoriented design or a component in component-based design. In the following, we commonly use the term class. The state machine processes external and internal events. UML defines four event types: CallEvent, SignalEvent, TimeEvent, ChangeEvent. A call event is associated with an operation/method and emitted if the operation is invoked. The processing of call events is synchronous meaning that it runs within the thread of the operation caller. The processing of other events is asynchronous meaning that these events received by the class are stored in an event queue which is maintained by the class at runtime for later processing.
For example, the Enp entry point in Fig. 1  allows the S5 sub-state of the S1 composite state to be active instead of S3 by the default entering way. Event Support: Our tool promotes four UML event types and event deference mechanism, which are able to express synchronous and asynchronous behaviors and exchange data between components/classes. UML-conformance: A recent specification formalizing the Precise Semantics of UML State Machine (PSSM) is under standardization of the OMG. It defines a test suite with 66 test cases for validating the conformance of runtime execution of code generated from UML State Machines. We have experimented our tool with the test suite. Traced execution results of 62/66 test cases comply with the standard and are, therefore, a good hint that the execution is semantically correct.
Event API: Generated code in our tool provides APIs for environment code to invoke operations or send data signals to reactive classes. The invocations and sending will automatically fire events for state machines to process. Concurrency: Concurrency aspects in state machines including doActivity of states, orthogonal regions, event detection, and event queue management are handled by the execution of multiple threads. Currently, we use POSIX threads for concurrency. Portability: Currently, our tool generates C++ code. The generated code can run on POSIX systems such as Ubuntu without installing any additional libraries to be able to compile and execute the code. Our code generation pattern and tool can be extended to generate code in other programming languages such as Java which supports threads and mutexes for multi-thread synchronization.
For example, entry in the listing implements all of the state action codes. State doActivitys, as specified by UML, are run concurrently. Each doActivity is then run within a permanent thread and a mutex is created for controlling it. Listing 2 shows a code segment for doActivity threads. The method doActivityThread takes as input a state id to use and call the appropriate mutex and doActivity, respectively. The method does nothing and stays in a waiting point if the state corresponding to the input parameter state identifier is inactive (line 5). If the state is active, a start signal is sent to this thread method to start the execution of doActiv- ity. The generated code typically follows the common paradigm in POSIX threads  (Butenhof, 1997).
Event Similar to the approach in  (Niaz et al. , 2004) , one method is generated for each event. An event enumeration EventId is created whose children are event identifiers associated with events. The event list of a state machine contains explicitly defined events and a special event called completion event, which is implicitly implemented. A completion event is fired when either the execution of the doActivity of simple/atomic state completes or all regions of a composite state have reached final states. For each event type, the pattern is realized as followings: CallEvent: When its associated operation is called, the event processing waits and locks the main mutex protecting the run-to-completion semantics as previously mentioned, and executes the event processing (see 4. 2). SignalEvent: An API sendSignal is created for environment code to interact and send an instance of the signal associated with the event by calling it.
Transitions Each event triggers a list of transitions. We suppose T trig (e) is the transition list triggered by the event e, and S trig (e) is a depth-ordered (from innermost to outermost) set of the source states of the transitions in T trig (e). Algorithm 1 describes how to generate the body of an event method. It first finds the innermost active states which are able to react e by orderly looping over S trig (e). This is to ensure that, in case of multiple transitions triggered by the event, the generated code for the transitions outgoing from innermost states will be executed. For each transition from an innermost state, code for active states and deferred events, guard checking, and transition code segments are generated by GEN CHECK, GEN GUARD(t) and GEN TRANS, respectively. If the identifier of e is equal to one of the deferred event list of the corresponding state (not shown in this paper), GEN CHECK generates code, which checks whether the event to be deferred and pushes the event to a deferred event queue managed by the runtime main thread.