dataesr/scientific-paragraphs-categorization huggingface dataset:
Statistical Analysis Descriptive analyses will be conducted to characterize the sociodemographic and clinical characteristics of the initial sample as well as the retention at the end of the study. Receiver operating characteristics analysis will be applied to calculate the concordance ratio of the pairs (ie, the area under the curve)  [53]. Given that patients have started using the app from the baseline visit, the clinical data collected at this time point will not be included in the concordant pairs' analysis. Compliance will be calculated as the proportion of the enrolled patients completing the first 15 and then 30 and 90 consecutive days of ratings. The app completion rate would be completing at least 70% of the momentary assessments during the time a participant has continued in the study.
Although the Toi Même project is not the first to implement a smartphone-based self-monitoring tool for patients with bipolar disorder, this app was designed as an independent self-assessment tool targeting mainly the fundamental dimensions of behavior (eg, activity, energy, cognition speed, affective responses) rather than as an augmenting intervention measuring essentially mood symptoms. The Toi Même clinical trial has carefully followed the ethical and legal issues around providing and obtaining adequate informed content from participants, personal privacy, and safety of data collection/storage. The same clinical psychiatrist who was blinded to smartphone data collected during the study assessed the patients by using standardized rating scales. The Toi Même app automatically integrates behavior data collection (eg, physical activity, user's response time) and 2 gamified modules targeting cognitive speed (QUiCKBRAIN) and affective response to images (PLAYiMOTIONS).
Acknowledgments This work was supported by the  Perception and Memory Unit  and the  Department of Information Systems of the Pasteur Institute (Paris). AAD is supported by a fellowship grant from the  Laboratory of Excellence Program 'Revive' and the life insurance company AG2R La Mondiale. The funding agencies had no role in the conduct or publication of the study. We thank both the  Departments of IT and of Legal/Industrial Affaires at the Pasteur Institute  for their technical and administrative support. We also thank the patients and the staff of the  Centre Thérapeutique de Jour Troubles Bipolaires (Clinique Bellevue)  for their contributions.
We implement a decision procedure for linear mixed integer arithmetic and formally verify its soundness in Isabelle/HOL. We further integrate this procedure into one application, namely into CeTA, a formally verified certifier to check untrusted termination proofs. This checking involves assertions of unsatisfiability of linear integer inequalities; previously, only a sufficient criterion for such checks was supported. To verify the soundness of the decision procedure, we first formalize the proof that every satisfiable set of linear integer inequalities also has a small solution, and give explicit upper bounds. To this end we mechanize several important theorems on linear programming, including statements on integrality and bounds. The procedure itself is then implemented as a branch-and-bound algorithm, and is available in several languages via Isabelle's code generator.
Instead of repeatedly performing certification tasks that require immense amounts of data and computational effort, it may be more fruitful to formally verify the soundness of a solver once, so that it can then be trusted without instance-by-instance certification of its output. The implementation of such a solver, together with a formal proof of its soundness, is the goal of the present work. Specifically, we use Isabelle/HOL  [19]  to implement and prove the correctness of a branch-and-bound algorithm  [21, Chapter 24. 1], and then use Isabelle's code generator  [11]  to obtain verified executable code. Along the way, we also give the first formalized proofs for several important results on integer programming. A concrete example of an application for our solver comes from termination analysis, where a program is given as input to a termination tool that tries to determine whether the given program terminates on all inputs.
Since termination tools get patched and improved repeatedly, maintaining an up-to-date formal proof of soundness would be extremely difficult. Therefore, the approach that is typically used is to have the (unverified) termination tool output a certificate for its analysis, which can then be checked by a verified certificate checker. One such certificate checker is CeTA  [5, 24]. It has been verified in Isabelle/HOL, so that whenever it accepts a proof of termination for some program, the formal proof of CeTA's soundness ensures that the program does indeed terminate. As an example, consider a program to compute the binary logarithm. int log2(int x) { int n := 0; while (x > 0) { x := x div 2; n := n + 1; } return n; } This program can be translated into an integer transition system and termination can be proved by showing that the value of x is decreased by at least 1 in every loop iteration.
The overlap between our work and  [1]  consists of formalizations of basic facts concerning cones and polyhedra, the fundamental theorem of linear inequalities, and Farkas' lemma. However, whereas in  [1]  a simplex algorithm for optimization problems is implemented in order to be used in constructive mathematical proofs, we formalize theorems concerning integer programming, including bounds on the size of solutions, and use these together with the previously Isabelle-verified simplex algorithm to obtain formally verified, yet efficient, software. There is also a formalization of theorems about polyhedra in HOL Light, due to Harrison  [12] , but it contains neither a formalization of the simplex algorithm nor does it cover integer programming. Cooper's algorithm has been formalized by Nipkow  [18]  in Isabelle/HOL.
The proof is completed by showing that an infinite execution cannot occur. The above argument could in principle be formalized in Isabelle by defining a function that incorporates the algorithm, and then proving that the function is well-defined (which implies the termination of the algorithm on all inputs). However, we are really only interested in the algorithm's termination; the fact that some input is mapped to a certain output is irrelevant for the proof of the theorem. Furthermore, we only need that the algorithm terminates when the set of input vectors is valid (i. e. , of the right cardinality and linearly independent), but, due to the limitations of the Isabelle function-package  [13] , the domain of a function cannot be restricted in this manner. Consequently, we formalize the proof without modeling the algorithm as an Isabelle function.
Thus, we obtained a dataset of 1192 benchmarks, comprising 18% of the 6489 benchmarks in QF LIA. (More specifically, the following sub-folders were fully converted to a format that is readable by our solver: 20180326-Bromberger, miplib2003, pb2010, pidgeons, prime-cone, and slacks. ) All solvers were tested on this dataset, on the same hardware, with a 60s-timeout per benchmark. Z3 version 4. 4. 0pre-2, CVC4 version 1. 5-4, and the 2019-05-09 release of SMT-LIB were used. The only other verified LIA solver that we are aware of is an Isabelle formalization of Cooper's algorithm in the AFP. This algorithm solves a more general problem than linear integer arithmetic (namely linear arithmetic with arbitrary quantifiers over integer variables). We obtained an implementation with minimal changes to make code generation possible (just as we produced executables for our own solver).
Evaluation. Our branch-and-bound implementation solves 37% of the dataset with incremental simplex as a sub-routine, and 31% with non-incremental simplex (Table  1 ). Since we have only implemented a naive branch-and-bound algorithm, without any additional heuristics for pruning the search space, it is unsurprising that its performance cannot match that of more mature solvers. Somewhat surprising is the fact that some benchmarks are solved by our solvers but not by Z3 or CVC4: of the benchmarks solved by incremental bnb, 28 are not solved by Z3, 29 are not solved by CVC4, and 8 are solved by neither Z3 nor CVC4. Interestingly, the non-incremental simplex-based solver can handle a few instances that the incremental simplex-based solver does not. Although using an incremental simplex leads to better overall results, it appears that reusing valuations from previous simplex runs can sometimes lead the search astray in such a way that simple solutions are missed.
Conclusion and Future Work We have developed a verified solver for linear mixed integer arithmetic, and have formalized important results on linear integer programming that were needed in order to prove the soundness of the solver. To the extent of our knowledge, the main mathematical theorems of which we formalized proofs had not been previously verified in any formal system, and our solver is the first verified LIA solver that is also usable in practice. The two parts of our formalization amount to 9813 lines of Isabelle code and took roughly 10 person-months to implement. Currently, our solver is essentially "proof of concept" software, and there are a number of known optimizations that could improve it, e. g. , preprocessing of constraints, integration of cutting planes, unit-cube-tests, etc.
As the binding affinity highly depends on the stoichiometry of the interaction, we selected a set of significant, peptide-based interactions for isothermal titration calorimetric (ITC) measurements. This way, we validated the interactions that were originally detected by the FP assay and determined the binding stoichiometry in all instances. All determined K d values correlated well with the data provided by the orthogonal FP measurements (Table  2 , Fig. S15 ). Symmetric interactions were found with CapZ, NCX1, SIP, TRPM4, and MDM4. In cases of CapZ and MDM4, the experimental data were fitted by a two binding site model indicating slightly different affinities and a complex relationship between the S100 monomers. In contrast, asymmetrical interactions were detected with p53, RSK1, C-ERMAD, NMIIA, and FOP.
These findings confirmed the expected binding stoichiometry in all cases and clarified the binding mode of TRPM4 and FOP. We hypothesize that the binding mode of close paralogs should be identical (symmetric or asymmetric); therefore, asymmetric binding was assumed for MNK1, MK2 (based on RSK1), and FOR20 (based on FOP). We performed these ITC measurements in parallel with the FP experiments, and based on the refined stoichiometry, monomer or dimer S100 concentrations were used during the FP data evaluation.
For such purpose, we decided to create a robust phenogram  [31] , representing the functional relationships within the human S100ome, using hierarchical clustering (UPGMA)  [32]. This analysis separated the S100ome into two groups, in which the first group contains S100 proteins generally lacking significant interactions (termed here as 'orphan' S100 proteins) and the second group comprises generally good binders (termed here as 'promiscuous' S100 proteins) (Fig. 5 ). While promiscuous S100 proteins showed significant binding Fig. 3. The S100 protein family. (A) Multiple short linear motifs are recognized by S100 family members; however, no consensus-binding motif can be defined for the protein family, as indicated here by the sequence alignment of several S100 binding motifs by ClustalW algorithm (UniProt accession codes: CapZ: P52907, MDM4: O15151, NCX1: P32418, C-ERMAD: P15311, FOP: O95684, FOR20: Q96NB1, TRPM4: Q8TD43, NMIIA: P35579, p53: P04637, SIP: Q9HB71, RSK1: Q15418).
Though it is noteworthy that hydrophobic residues (green) are preferred, basic residues are also welcome in some cases. (B) S100 proteins act as dimers and are capable of interacting in two distinct ways with other proteins. On the left, a symmetric complex is shown, where one S100 dimer interacts with two peptides (S100B-CapZ, PDB code: 1MWN,  [17] ). By contrast, a single interacting partner can bind to one S100 dimer asymmetrically, as it is shown on the right side (S100A4-NMIIA, PDB code: 3ZWH,  [18] ). PYMOL (Schr€ odinger, New York, NY, USA) was used for visualization and for preparing molecular images. to at least a few (4-5) of the tested interaction partners, orphan S100 proteins showed either no sign of partner binding or a weak binding to a single partner.
Fitting of FP data The K d of the direct and competitive FP experiment was obtained by fitting the measured data with quadratic and competitive equation, respectively  [7]. For automatic fitting, we used an in-house developed, Python-based program, called ProFit, which is freely available from GitHub. The program is capable to process multiple experimental data at once, evaluate direct competitive experimental data series pairs, and estimate the variance of the deduced parameters (e. g. , dissociation constants) through a Monte Carlo approach. It produces ready-to-use figures for publications, as well as a report sheet for evaluation.
ITC measurements Titrations were carried out either at 310 or at 298 K in a buffer containing 150 mM NaCl, 20 mM HEPES pH 7. 5, 1 mM CaCl 2 , 0. 5 mM TCEP, using a MicroCal PEAQ-ITC instrument (Malvern Panalytical, Malvern, UK). The acquired data were fitted by PEAQ-ITC analysis software using the model 'One Set of Sites' for most of the experiments; however for S100B-CapZ and S100B-MDM4, this model provided unsatisfactory fits and the model 'Two Sets of Sites' were applied instead. Note that we used the minimal interacting region (696-735) of RSK1 instead of the larger construct which was used in the direct FP assay.
Acknowledgements We would like to thank  D aniel Knapp  from the  Department of Plant Anatomy, ELTE , for the help in bioinformatics analyses. We would also thank  Gitta Schlosser  and D avid Papp from the  Department of Analytical Chemistry, ELTE , for the help in mass spectrometry. This work was supported by the  National Research Development and Innovation Fund of Hungary  ( K119359  to  LN K120391  to JK). MAS and GG were supported through the  New National Excellence Program of the Hungarian Ministry of Human Capacities  ( UNKP-18-2 and UNKP-18-3, respectively). We also acknowledge the  FIEK_16-1-2016-0005  and  VEKOP-2. 3. 3-15-2016-00011grants  and Project No. 2018-1. 2. 1-NKP-2018-00005  implemented with the support provided from the  National Research, Development and Innovation Fund of Hungary , financed under the  FIEK_16 ,  VEKOP-2.
D. > 200 N. D. > 200 N. D. > 200 N. D. E. A. E. A. > 200 S100P 0. 93 AE 0. 24 > 200 > 200 N. D. > 200 N. D. S100Z > 200 N. D. > 200 N. D. > 200 N. D. > 200 N. D. N. D. N. D. > 200 > 200 0. 20 AE 0. 04 0. 15 AE 0. 04 > 200 N. D. 2. 8 AE 0. 47 1. 2 AE 0. 86 3. 2 AE 0. 73 > 200 1. 2 AE 0. 53 4. 5 AE 0. 47 > 200 2. 7 AE 0. 47 2. 5 AE 0. 38 0. 066 AE 0. 0049 phenomena, such as protein aggregation. To standardize and automatize data handling and to eliminate subjective factors, we developed a Python-based universal program, called ProFit, for fitting all direct and competitive experimental data (freely available at https://github. com/GoglG/ProFit).
Choosing which studies to include in a meta-analysis can be challenging. In principle, studies can be manually annotated as carefully as one likes. However, manual meta-analyses are not Table  1 : Diversity of vocabularies: there is no established lexicon of neuroscience, even in hand-curated reference vocabularies, as visible across CognitiveAtlas  [Poldrack and Yarkoni, 2016] , MeSH  [Lipscomb, 2000] , NeuroNames  [Bowden and Martin, 1995] , NIF  [Gardner et al. , 2008] , and NeuroSynth  [Yarkoni et al. , 2011]. Our dataset, NeuroQuery, contains all the terms from the other vocabularies that occur in more than 5 out of 10 000 articles. "MeSH" corresponds to the branches of PubMed's MEdical Subject Headings related to neurology, psychology, or neuroanatomy (see Appendix E. 1). Many MeSH terms are hardly or never used in practicee.
NeuroQuery is easily usable online, at neuroquery. org, and the data and source code can be freely downloaded. We start by briefly describing the statistical model behind NeuroQuery in Section 2. 1, then illustrate its usage (Section 2. 2) and show that it can map new combinations of concepts in Section 2. 3. In Section 2. 4 and 2. 5, we conduct a thorough qualitative and quantitative assessment of the new possibilities it offers, before a discussion and conclusion. 2 Results: The NeuroQuery tool and what it can do.
Using them helps meta analysis: it would require hundreds of studies to detect a pattern in locations reported for "aphasia", e. g. in lesion studies. But with the text of a few publications we notice that it often appears close to "language", which is indeed a related mental process. By leveraging this information, NeuroQuery recovers maps for terms that are too rare to be mapped reliably with standard automated meta-analysis. Using Non-negative Matrix Factorization (NMF), we compute a low-rank approximation of word co-occurrences (the covariance of the TFIDF features), and obtain a denoised semantic relatedness matrix (details are provided in Appendix B. 4). These word associations guide the encoding of rare or difficult terms into brain maps. They can also be used to explore related neuroscience concepts when using the NeuroQuery tool.
In addition, NeuroQuery uses an estimate of the voxel-level variance of association (see methodological details in Appendix B), and reports a map of Z statistics. Note that this variance represents an uncertainty around a prediction for a TFIDF representation of the concept of interest, which is treated as a fixed quantity. Therefore, the resulting map cannot be thresholded to reject any simple null hypothesis. NeuroQuery maps have a different meaning and different uses than standard meta-analysis maps obtained e. g. with ALE. Group-level map from the IBC dataset for the contrast "Read pseudo-words vs. consonant strings" and contour of NeuroQuery map obtained from this query. The NeuroQuery map was obtained directly from the contrast description in the dataset's documentation, without needing to manually select studies for the meta-analysis nor convert this description to a string pattern usable by existing automatic meta-analysis tools.
Illustration: using NeuroQuery for post-hoc interpretation After running a functional Magnetic Resonance Imaging (fMRI) experiment, it is common to compare the computed contrasts to what is known from the existing literature, and even use prior knowledge to assess whether some activations are not specific to the targeted mental process, but due to experimental artifacts such as the stimulus modality. It is also possible to introduce prior knowledge earlier in the study and choose a Region of Interest (ROI) before running the experiment. This is usually done based on the expertise of the researcher, which is hard to formalize and reproduce. With NeuroQuery, it is easy to capture the domain knowledge and perform these comparisons or ROI selections in a principled way. As an example, consider again the contrast from the RSVP language task  [Pinho et al.
, 2018 , Humphries et al. , 2006]  in the Individual Brain Charting (IBC) dataset, shown in Fig. 2. It is described as "Read pseudo-words vs. consonant strings". We obtain a brain map from NeuroQuery by simply transforming the contrast description, without any manual intervention, and compare both maps by overlaying a contour of the NeuroQuery map on the actual IBC group contrast map. We can also obtain a meta-analytic map for the whole RSVP language task by analyzing the free-text task description with NeuroQuery (Appendix C. 1).
By combining term similarities and an additive encoding model, NeuroQuery can accurately map rare or difficult terms for which standard meta-analysis lacks statistical power, as visible on Fig. 4. Quantitatively comparing methods on very rare terms is difficult for lack of ground truth. We therefore conduct meta-analyses on subsampled corpora, in which some terms are made artificially rare, and use the maps obtained from the full corpus as a reference. We choose a set of frequent and well-mapped terms, such as "language", for which NeuroQuery and NeuroSynth (trained on a full corpus) give consistent results. For each of those terms, we construct a series of corpora in which the word becomes more and more rare: from a full corpus, we erase randomly the word from many documents until it occurs at most in 2 13 = 8912 articles, then 2 12 = 4096, and so on.
Each tool is trained on its own dataset, which is why the full-corpus occurrence counts differ. right: convergence of maps toward their value for the full corpus, as the number of occurrences increases. Averaged over 13 words: "language", "auditory", "emotional", "hand", "face", "default mode", "putamen", "hippocampus", "reward", "spatial", "amygdala", "sentence", "memory". On average, NeuroQuery is closer to the full-corpus map. This confirms quantitatively what we observe for the two examples "language" and "reward" on the left. Note that here convergence is only measured with respect to the model's own behavior on the full corpus, hence a high value does not indicate necessarily a good face validity of the maps with respect to neuroscience knowledge. The solid line represents the mean across the 13 words and the error bands represent a 95% confidence interval based on 1 000 bootstrap repetitions.
Quantitative evaluation: NeuroQuery is an accurate model of the literature. Unlike standard meta-analysis methods, which compute in-sample summary statistics, NeuroQuery is a predictive model, that can produce brain maps for out-of-sample neuroimaging studies. This enables us to quantitatively assess its generalization performance. Here we check that NeuroQuery captures reliable links from concepts to brain activity -associations that generalize to new, unseen neuroimaging studies. We do this with 16-fold shuffle-split cross-validation. After fitting a NeuroQuery model on 90% of the corpus, for each document in the left-out test set (around 1 300), we encode it, normalize the predicted brain map to coerce it into a probability density, and compute the average log-likelihood of the coordinates reported in the article with respect to this density.
The procedure is then repeated 16 times and results are presented in Fig. 7. We also perform this procedure with NeuroSynth and GCLDA. NeuroSynth does not perform well for this test. Indeed, the NeuroSynth model is designed for single-phrase meta-analysis, and does not have a mechanism to combine words and encode a full document. Moreover, it is a tool for in-sample statistical inference, which is not well suited for out-of sample prediction. GCLDA performs significantly better than chance, but still worse than a simple ridge regression baseline. This can be explained by the unrealistic modelling of brain activations as a mixture of a small number of Gaussians, which results in low spatial resolution, and by the difficulty to perform posterior inference for GCLDA. that NeuroSynth is not designed to make encoding predictions for full documents, which is why it does not perform well on this task.
NeuroQuery is an openly available resource NeuroQuery can easily be used online: https://neuroquery. org. Users can enter free text in a search box (rather than select a single term from a list as is the case with existing tools) and discover which terms, neuroimaging publications, and brain regions are related to their query. NeuroQuery is also available as an open-source Python package that can be easily installed on all platforms: https://github. com/neuroquery/neuroquery. This will enable advanced users to run extensive meta-analysis with Neuroquery, integrate it in other applications, and extend it. The package allows training new NeuroQuery models as well as downloading and using a pre-trained model. Finally, all the resources used to build NeuroQuery are freely available at https://github.