dataesr/scientific-paragraphs-categorization huggingface dataset:
Curved Detector The fundamental component of the new prototype instrument is the curved detector. The first step is to determine the target radius of curvature best approximating the curvature of the studied brain. We first develop the prototype for rhesus macaque primates, for which there is a lot of experience. Magnetic resonance imaging (MRI) scans were used to estimate the curvature of the macaque visual cortex. To extract the surface of the cortex from the MRI data, we used Macapype  [15] , an open source pipeline that performs segmentation of the different brain tissues from primates's MRI scans. The region of interest (the area of the brain that is optically imaged) was then manually identified with BrainVISA (an open-source neuroimaging software platform). This area corresponds to the visual cortex accessible through the A c c e p t e d M a n u s c r i p t optical chamber surgically implanted in the primate's skull (mostly the first two visual cortical areas V1 and V2, illustrated in figure  2 ).
A horizontal dotted line indicate the radius of curvature that was selected for the new instrument and which corresponds to the macaque brain that the instrument is tested on (Ziggy, left hemisphere). The corresponding point is encircled, and the inset image shows in red the region of interest on one hemisphere that was considered for the fit. The surface is visualized with the Anatomist tool from the BrainVISA software, and the red axes show the reference frame (as defined by the position of the macaque head in the MRI machine when the scan was performed). The detector selected for curving is a large CMOS sensor (AMS CMV12000: monochromatic, 12 bit depth, 4096×3072 square pixels with a size of 5. 5 µm). It was curved with a technique based on substrate elasticity which currently does not permit to curve a detector to our target radius of 29 mm.
Lenses The use of lenses is required to form an image and, as described in Section 2. 2. 1, to account for the remaining brain curvature that cannot be corrected by the curved detector. The magnification of 0. 89 was chosen so as to resemble the conditions of the original instrument while also optimally exploiting the available surface of the detector (22. 5 mm x 16. 9 mm) and of the optical chamber. The selected optics combine a double Gauss design with a projection lens with aspherical corrector, whose shape is determined by the field of view and the residual curvature to be corrected. Figure  2  shows a cross section of the final optical design, along with the lens mount (described in more details in Section 2. 3. 1). The optical design was done with the software Zemax Optics Studio to optimize the image quality over the entire field of view.
Acknowledgments Portions of this work were presented at the  SPIE Optical Systems Design Digital Forum in 2021, Proc. SPIE 11876, Optical Instrument Science, Technology , and  Applications II ,  118760A  ( 12 September 2021 ); https://doi. org/10. 1117/12. 2597067Salvatore  Giancani conducted many of the neuroscience experimental procedures. Xavier Degiovanni  and  Joel Baurberg  performed the machining and electronics jobs, respectively. Manon Bourbousson  was involved in the project's early stages. David Meunier  and  Olivier Coulon  offered assistance with the IRM segmentation and analysis. Thibault Behaghel curved the detectors for this project, and  Emmanuel Hugo  and  Kelly Joaquina  helped with the maintenance and repairs of our detectors. Jean-François Sauvage  provided ideas, help and guidance, especially at the start of the project.
This step is formally equivalent to training a one hidden layer CNN. We implement a greedy procedure, optimizing each new filter over the residuals of the previously entered ones, using a gradient descent method initialized at the k-mer with the best score. To that end, we enumerates the k-mers contained in X using the DSK software  (Rizk et al. , 2013)  and compute their scores s(•, •). (iii) SEISM splits the set of sequence motifs into meshes according to the input parameter. This step leads to the definition of a set of null hypotheses and of a selection event E, i. e. the set of outcomes y ′ that would have led to the selection of motifs within the same meshes as the ones selected in (ii), namely the sequence of meshes (M i1 ,. , M iq ). Formally, the selection event reads E := y ′ ∈ Y : ∀j ∈ [q] , arg max z∈Z s(z, P j y ′ ) ∈ M ij , (1) for some projection matrix P j , to be defined later.
Greedy optimization It is common to solve (4) by stochastic gradient descent (SGD) jointly over the q filters. More generally, this approach for training a neural network with a single, large hidden layer is known to find a global optimizer at the large q limit under some assumptions  (Soltanolkotabi et al. , 2019). Our objective here is slightly different: we do not necessarily aim at approximating a continuous measure with a large number of particules, but we aim at selecting a small number of particules lending themselves to a biological interpretation. Furthermore, the number of relevant motifs on a given dataset is generally unknown. In this context, it is known that jointly optimizing the convolution filters leads to irrelevant PWMs, with some actual motif split across several filters and other duplicated  (Koo & Eddy, 2019).
Without this projection, iterating (8) would return the same z. Of note, joint optimization procedures of the q filters don't face this issue, and forward selection procedures over finite sets of features work around the problem by iteratively removing the selected elements from the set over which selection if performed  (Slim et al. , 2019). This sequential strategy combined with the testing procedure introduces in Section 4 provides a data-driven mean to choose the number q of relevant motifs. In practice, we solve (8) with a standard gradient descent algorithm, initialized at the k-mer with the best association score. The k-mer list is obtained using the DSK software  (Rizk et al. , 2013). We work on a less constrained set than Z (2) and don't enforce the positivity constraint during optimization.
Dealing with selection events over a continuous set of features Formally, our selection event E cont. is the set of outcomes y ′ that would have led to the selection of the same set of motifs Z = {z 1 ,. , z q } than the one selected using y from the real dataset, when applying the same selection procedure: E cont. := y ′ ∈ E : ∀j ∈ {1,. , q} arg max z∈Z s(z, P j y ′ ) = z j , ( 11 ) where P j is the orthogonal projection onto Span ℓ<j 1, ϕ z ℓ ,X ⊥. A simple rejection approach to sample from the null (10) conditioned to E cont. would be to sample y in E under (9, 10) and only retain those in E cont. Unfortunately, E cont. belongs to a strictly lower-dimensional vector space of R n and is therefore a null set for the Lebesgue measure on R n. For s HSIC and s ridge , and noting that a maximum is also a critical point, we indeed obtain: y ′ ∈ E cont.
SEISM performs as well as state-of-the-art de novo motif discovery methods In order to compare the accuracy of our selection step with existing motif discovery algorithms, we use the 40 ENCODE Transcription Factors ChIP-seq datasets from K562 cells  (ENCODE Project Consortium, 2004) , each of which contains a known TF motif, denoted m * , derived using completely independent assays  (Jolma et al. , 2013). STREME  (Bailey, 2021)  and MEME  (Bailey et al. , 2006)  are state-of-art bioinformatics methods for de-novo motifs discovery tasks. STREME identifies motifs that maximize a Fisher score of association between the presence of the motif and the binary class of sequences. By looking for maximum likelihood estimates of the parameters of a mixture model -made up of a background distribution and a model for generating k-mers at some positions -that may have produced a particular dataset using an expectation maximisation technique, MEME finds enriched motifs in this dataset.
Finally CKN-seq  (Chen et al. , 2017)  is a one-layer CNN tailored to small scale datasets. We set up STREME, MEME and SEISM to select 5 sequence motifs. SEISM is run with a regularization parameter λ = 0. 01. CKN-seq jointly optimizes its filters, which notoriously leads to poor performances when few filters are used. We train it consequently over 128 filters. We measure these accuracy of all methods by comparing the motifs they discover with the known motif corresponding to the transcription factor m *. We rely on the Tomtom method  (Gupta et al. , 2007) , which quantifies the probability that the euclidean distance between a random motif and m * is lower than the distance between the discovered motif and m *. More precisely for each method we use the lowest Tomtom p-value between the known TF motif m * and any of those discovered by the method.
The Tomtom score is then defined aslog 10 of the Tomtom p-value. We define the accuracy of the method as the proportion of experiments where the Tomtom score between its best match and the true TF motif was higher than some threshold. Figure  4  (right panel) shows that SEISM performs slightly worse than STREME and MEME for high thresholds on the Tomtom scores. This suggests that the matrix z that SEISM identifies is close enough to the PWM matrix of the true motif, but farther away than the matrices identified by STREME or MEME. This discrepancy reflects a different usage of z to parameterize a distribution of k-mers. In practice, we observe that on a given dataset, the p-values of the best motifs discovered by SEISM and STREME are not separated by more than 2 orders of magnitude, which leads to minor differences in the motifs, as illustrated in Table  1.
Statistical validity and performances In order to assess the statistical validity and of the SEISM procedure with the different strategies, we simulate datasets under the null hypothesis. To that end, we draw one sequence motif z with length k = 8 for each simulated dataset using a uniform distribution on Z restricted to motifs with an information level fixed at 10 bits. Then, we draw a set of n = 30 biological sequences X as follows: all sites are generated according to a uniform distribution over A, C, T, G for all sequences, and for half of the sequences one k-mer is drawn according to the categorical model parameterized by z. The phenotypes y are drawn from N (0, σ 2 C n ) to generate data under the null hypothesis for calibration experiments, and from N (ϕ z,X , σ 2 C n ) to generate data under the alternative for experiments on statistical power, with σ = 0.
1 in both cases. We then run the SEISM procedure to select and test two sequence motifs. For both the data-split strategy and the hypersphere direction sampling one, the distribution from which the replicates are drawn uses the empirical variance from y as variance parameter. Although any choice for this parameter leads to a valid procedure, as described in Section 4. 6, we make this choice for numerical stability considerations. For the data-split strategy, we sample 1000 replicates under the null hypothesis to compute the p-value. For SEISM, we sample 50, 000 replicates under the conditional null hypothesis using the hypersphere direction sampler, after 10, 000 burn-in iterations. Figure  5  (top) shows the Q-Q plot of the distribution of quantiles of the uniform distribution against the p-values obtained across 1000 datasets under the null hypothesis for the data-split strategy and 100 datasets for the hypersphere direction sampling one.
All the data points are well-aligned with the diagonal, which confirms the correct calibration of both the data-split and hypersphere direction sampling strategies, either considering the best motif or the center of the mesh and regardless of the size parameter. Figure  5  (bottom) shows the same Q-Q plot on data generated under the alternative hypothesis. From this figure, we observe that on small datasets, the post-selection strategy is more powerful than the datasplit one, regardless of the size of the mesh considered or the choice concerning the definition of the null hypothesis. The variance observed on the curves associated with the selective inference procedure is due to the presence of a weak residual signal after the first motif as a result of an imperfect selection step. Testing it with the best motif in the mesh captures this signal, resulting in curves under the diagonal.
This formalization also highlights the specific way by which CNNs with exponential activation functions parameterize the distribution of k-mers at a binding site. Although the PWM returned by most bioinformatics models represents a categorical distribution-probability to draw each letter at each site, trained convolution matrices parameterize a Gaussian distribution. In practice, this difference leads to discrepancies between the trained convolution filters and PWMs learned using categorical likelihoods-including those offered by databases and often used as ground truth. This observation also suggests alternative sets of constraints for convolution filters-e. g. , each column of the pointwise exponential of the filter should belong to the simplex. By providing an inference procedure for features extracted by the trained model, our work also connects neural networks for genomic sequences to GWAS.
Acknowledgements This work has been supported by  ANR  grants ( FAST-BIG  project  ANR-17-CE23-0011-01  and  PIECES  project  ANR-20-CE45-0017 ) and was performed using the computation facilities of the LBBE/PRABI. We thank  François Gindraud ,  Jean-Philippe Rasigade ,  Lotfi Slim  and  Dexiong Chen  for the insightful discussions and support.
and syntactic analysis. Therefore, now they can be carried out as software, like Software as a Service (SaaS) in cloud computing  [2]. Cloud computing is a model to allow flexible on-demand network access to a shared pool of configurable computing resources that can be accessed and delivered quickly with minimal management effort. This paradigm is based on many existing technologies such as the internet, the resource for virtualization, grid computing, and web services. Hence, Cloud Computing combines software as a service and utility computing. Cloud computing is designed to provide flexible and low-cost on-demand computing infrastructure with good service reliability  [3, 4]. Nowadays, cloud services for NLP analysis are becoming very popular among scientists and all users interested in the field.
It allows researchers to deploy, share, and use components and resources for language processing, following the paradigms of data as a service and software as a service. However, there are only a few reviews of natural language processing services in the cloud are available. A few examples of prominent natural language processing APIs and cloud-based services are Amazon Comprehend [5], Microsoft Azure Cognitive Services  [6] , Google Cloud Natural Language  [7] , and third-party options. Amazon Comprehend (AWS) service uses machine learning to extract key phrases and identify the language in a given text. Amazon Comprehend works with any AWSsupported application, and it has features such as sentiment analysis, tokenization, and automated text file organization. The Microsoft Azure Cognitive Services is a portfolio of natural language processing tools broken out into several different, more targeted services and uses.
For example, if developers want to build applications that can analyze the sentiment or identify the language of a given text, Azure Text Analytics API can be used. Alternatively, Azure Language Understanding Intelligent Service can understand things such as user intent. It is precious when developers build chatbots, voice-powered products, and even customer service platforms. Google Cloud Natural Language also emphasizes entity extraction, sentiment analysis, syntax analysis, and categorization. However, this API differs because it is powered by Google's own in-depth learning modules -the same ones that drive the query comprehension behind Google Search and the language understanding system behind Google Assistant. Third-party options are many natural language processing APIs and services in the market.
Similarly, different big data techniques and cloud computing paradigms are presented. The concept of this review and the main components of NLP, cloud computing, and big data are provided in Fig. 1. The organization of the paper is as follows: "Natural language processing" section presents the various important terminologies of NLP and its applications. "Cloud computing" section discusses cloud computing and presents its architectural layers and taxonomy, followed by presenting cloud services for NLP and towards the NLP platform as a service: requirements and methodology. "Big data in cloud" section introduces big data in the cloud, methods, and technologies related to NLP, and big data are discussed, followed by presenting machine learning and NLP in big data. Moreover, information extraction via NLP techniques within big data is introduced.
Natural language processing The field of Artificial Intelligence and linguistics aims to make computers understand statements or words written in human languages. NLP was developed to ease the user's work and satisfy the desire to communicate in natural language with the computer. Since all users may not be well versed in the machine-specific language, NLP supports those users who do not have enough time to learn or perfect new languages. A language can be described as a set of rules or a set of symbols. Symbols are combined and used for conveying information or broadcasting the information. NLP can essentially be divided into two components, natural language understanding (NLU) and natural language generation (NLG), which evolve around the task of understanding and generating the text.
Levels of linguistic analysis for NLP Linguistics is a science that comprehends the meaning of language, the linguistic context, and various forms of language. Language levels are one of the most analytic approaches to representing the NLP, helping to produce the NLP text by performing the content planning, sentence planning, and surface implementation phases. Various key terminologies of the NLP are divided into seven levels, from the simplest to the most difficult  [10] , namely: morphology, lexicon, syntax, semantics, phonology, discourse, and pragmatics. Morphology, the term itself is a Greek word derived from "morph" and "ology". The term "morph" means to form or structure, while "ology" means to study something. Therefore, morphological analysis can be defined as the scientific study, identification, analysis, and description of the structure or form of words in a language and the relationship of the words to one another.
Applications of NLP Natural language processing provides both theory and implementations for a range of applications. Any application that utilizes text is a candidate for NLP. The most common applications utilizing NLP include information retrieval and summarization. NLP can be applied in various areas such as machine translation, information extraction, summarization, and question answering among other applications. Information Retrieval, according to Copestake  [14] , information retrieval is the process of returning a set of documents in response to a user query. A typical example of information retrieval is search engines like Google to match a user's query to a collection of documents and then return the most similar or relevant documents. The major drawback of information retrieval is that the set of documents returned is not organized, and thus it becomes difficult for the end-users to navigate the documents.
Over recent years, the exploration of information has become an important area of research. Knowledge discovery research uses various techniques to retrieve helpful information from source documents, such as part-of-speech tagging, chunking or shadow parsing, stopwords and stemming. The former has good accuracy but a higher cost of implementation, while the latter has a lower cost of implementation but is generally insufficient for information retrieval: compound and statistical phrases index multi-token units instead of single tokens. Word sense disambiguation is the task of understanding the correct sense of a word in context. When used for information retrieval, terms are replaced by their senses in the document vector. Its extracted information can be used for various purposes, such as preparing a summarization, building databases, defining keywords, classifying text objects by specific predefined categories, among other purposes.
Question-Answering is a discipline of computer science in the fields of AI and NLP, focusing on building systems that automatically answer the questions posed by human beings in their natural language. Question-answering returns a set of relevant documents in response to a user's query  [24]. A computer system that understands the natural language has the capability of a program system to translate human-written sentences into an internal representation to produce valid answers. The exact answers can be given by doing syntax and semantic analysis of the questions. Lexical gap, ambiguity, and multilingualism are some of the challenges for NLP when it comes to building a good question answering system. The exact answers can be generated by doing syntax and semantic analysis of the questions.
It offers eight APIs with different functionalities such as article extraction, concept extraction, entity extraction, summarization, classification, semantic labeling, image tagging, sentiment analysis, hash-tag suggestion, language detection, and microformat extraction. These functionalities are available in six languages: English, German, French, Italian, Spanish and Portuguese. One interesting feature of Aylien is a text analysis add-on for Google Spreadsheets that allows users to run the API functionality through a familiar interface. Text Summarization API  [37]  provides a professional text summary service, which relies on advanced natural language processing and machine learning technologies. It can be used, to sum up short essential texts from the URLs or documents provided by users.
CoreNLP's analysis provides the foundational building blocks for higher-level and domain-specific text understanding applications. Text-Processing API  [42]  offers functionalities such as review of emotions, stemming and lemmatization, part of speech tagging and chunking, phrase extraction, and named entity recognition. The NLPTools API is created by Atrilla  [43]  for a text processing framework to analyze Natural Language by performing operations and tasks on corpus data. Hence, this approach focuses on the statistical track of NLP. It is primarily focused on text classification and sentiment analysis. Stemmer API  [44] , this API takes a paragraph and returns the text with each word stemmed using porter stemmer, snowball stemmer, or UEA stemmer. The WebKnox Text-Processing API  [45]  can process natural languages and detect text language, quality of writing, find entity mentions, tag part of speech, extract dates, locations, and determine the sentiment of the text.
It provides features such as language identifier, keyword extractor, named entity recognizer, part of speech tagger, syntactic analyzer, tokenizer, and a text summarizer. Semantria  [50]  provides a variety of NLP services based on the Salience Lexalytics engine but can only be accessed via an API or Microsoft Excel. In this work, we are interested in natural language processing tasks and their solutions using SaaS/PaaS. Analysis of available services is needed to clarify the boundaries of their functionality and the quality of performance of existing options. It is also essential to understand the policy of using these services. Cloud services for NLP provide features such as named entity recognition (NER), part of speech tagging (POS), sentiment analysis, stemming, lemmatization, categorization, among others  [51].