dataesr/scientific-paragraphs-categorization huggingface dataset:
Comparison of CNV callers GIP was compared to CNVnator (version 0. 4. 1) (  27 ) and cn. MOPS (version 1. 36. 0) (55)  to predict gene CNVs in the L. infantum dataset (Supplementary Table  S1 ). To run CN-Vnator the commands '-tree', '-his', '-stat', '-partition' and '-call' were used. Following the authors' recommendations, the 'binSize' parameter was benchmarked using the '-eval' command optimizing the ratio of average read depth signal to its standard deviation. A common optimal 'binSize' value of 150 was selected considering the read length and the read depth of the sequencing libraries. Cn. MOPS was run using a windows length (WL) of 150. The WL value was chosen accounting for read depth differences between samples to have approximately an average of 50-100 reads per segment. To account for ploidy differences between samples cn.
GIP and giptools running time and computational resources The '-with-timeline' Nextflow option was used to render HTML timelines for all GIP executed processes. The '-withreport' Nextflow option was used to summarize GIP computational resources. A detailed description of each of the two options is available from Nextflow documentation at https://www. nextflow. io/docs/latest/tracing. html#. The giptools computational requirements were estimated using the Linux 'time' command (URL https://man7. org/linux/manpages/man1/time. 1. html) together with the '-v' option to obtain a verbose output.
The GIP workflow GIP is a tool for scientific investigation compatible with Linux systems, requiring minimal configuration and dis-Downloaded from https://academic. oup. com/nar/advance-article/doi/10. 1093/nar/gkab1237/6470687 by Institut Pasteur user on 05 January 2022 tributed as a self-contained package. Our integrative approach provides a broad range of genomic data analyses and visualizations, and combines new and existing bioinformatic methods (see Materials and Methods). GIP consists of three files: the Nextflow pipeline code, the configuration and the Singularity container files. The pipeline container conveniently provides a working environment with 19 offthe-shelf applications for genomic analyses and 40 R packages listed at https://gip. readthedocs. io/en/latest/software/ index.
For SNVs, calling GIP relies on Freebayes, a popular tool that resolves the issue of having multiple potential ambiguous alignments between the read and the homologous genomic region by examining the whole haplotype of the read independently of the precise alignment positions. It was shown that compared to alternative variant calling software Freebayes demonstrates good performance across different aligners and Illumina platforms  (56). For SV detection GIP runs DELLY, a method combining short insert paired-ends, long-range mate-pairs and split-read alignments to precisely define balanced and unbalanced forms of genomic rearrangements at single-nucleotide resolution. DELLY's prediction accuracy was previously validated by PCR  (28)  and its performance compared to other structural variant calling algorithms on simulated data under different sequencing parameter settings  (28).
Applying giptools on a Leishmania infantum case study GIP permits the batch analysis of a set of individual samples, where each sample is considered separately and compared only with respect to the provided reference genome assembly. As a consequence, all variants and copy number alterations detected in a sample merely reflect the differences between the sequenced and the reference genomes. While this application may be sufficient in some circumstances, research projects often involve downstream comparison between samples. Examples include the comparison of gene or chromosome copy variation number between   1 ). All modules in giptools are fully embedded in the Singularity container and they are provided with their own documentation. To illustrate the type of exploratory data analyses and the biological questions that can be addressed, we tested giptools on a previously analyzed dataset of seven clinical L.
infantum isolates from Tunisia  (31). Leishmania is the etiological agent of leishmaniasis, a life-threatening human and veterinary disease affecting 12 million people worldwide  (57). Parasites were derived from seven patients affected with visceral leishmaniasis, expanded in cell culture and their genomes were sequenced. This dataset includes four Glucantime drug susceptible isolates and three isolates from relapsed patients, and their comparison may inform on genetic factors resulting in treatment failure. Giptools allowed the detection and visualization of pervasive intra-chromosomal CNVs across the thirty-six Leishmania chromosomes (Figure  2A ). Additionally, giptools enables targeted comparison of normalized genomic bin sequencing coverage of sample pairs. We used giptools' 'binCNV' module to compute the ratio between corresponding genomic bins of the strains LIPA83 over ZK43, which correspond to a first-episode and a relapse leishmaniasis isolate, respectively.
Benchmarking of prediction tools GIP uses Freebayes and DELLY to perform SNVs and SVs calls while providing additional options to filter and reformat their output for downstream analyses. Freebayes and DELLY predictions were already extensively benchmarked against other tools and validated in previous reports  (28, 56, 58). GIP implements its own approach to quantify, normalize, compare and visualize genomic bins and genes. One gene CNV predicted using GIP was experimentally tested in a separate study where we confirmed by PCR the deletion of a NIMA-like kinase gene  (59). In the following we used the L. infantum dataset to compare the gene CNV predictions returned by GIP with the ones of two popular CNV detection tools, CNVnator  (27)  and cn. MOPS  (55). The three methods use sequencing cover-age depth information extracted from DNA read alignments.
39236 Gigabytes in the 'geInteraction' giptools step to evaluate the copy number variant genes. CNVnator and cn. MOPS require 2. 727132 and 0. 791924 Gb respectively. The cumulative running time to execute the GIP 'covPerGe' step on each individual sample and to execute the 'geInteraction' giptools step is of 18 min and 59. 06 s. For comparison CNVnator employs 12 min and 46. 53 s, while and cn. MOPS takes 6 min and 21. 04 s. Altogether, these results show that to compute gene CNVs on the L. infantum dataset our pipeline has similar memory requirements but longer execution times compared to CNVnator and cn. MOPS. The analysis confirms that the CNV gene predictions produced by our pipeline largely recapitulate the ones produced by CNVnator or cn. MOPS. Furthermore, GIP provides the added benefit of a higher robustness when predicting intrachromosomal CNVs in the event of chromosomal aneuploidy.
Comparative genomics of a Plasmodium vivax WGS dataset We next applied GIP and giptools on other biological systems to demonstrate its broad applicability outside the Leishmania field, including the human apicomplexan parasite P. vivax. Plasmodium vivax is a protist parasite and a human pathogen causing malaria. Plasmodium vivax gives rise every year to 130 million clinical cases  (60) , and it is estimated that 2. 5 billion people are at risk of infection worldwide  (61) (62) (63). We applied GIP and giptools to inves-tigate genomic variations across a sizeable dataset of 222 P. vivax genomes isolated from clinical samples of 14 countries worldwide  (44, 52)  (Supplementary Table  S1 ). The GIP quantification analysis was able to recover 5 out of the 11 previously reported CNV regions and predicted 73 new large (>3 kb) copy number variant areas (Supplementary Figure  S5 , Supplementary Table  S5 ).
Gene CNV analysis of Candida albicans evolutionary adapted strains We next applied GIP and giptools to the human fungal pathogen C. albicans, an opportunistic yeast exhibiting major genome plasticity (74-83) and causing hun-dreds of thousands of severe infections each year  (84). Candidemia, a bloodstream infection with Candida, are often associated with high rates of morbidity and mortality (15-50%) notwithstanding existing antifungal treatments  (85, 86). We applied GIP and giptools to a C. albicans WGS dataset described in a recent study that covers five different progenitor strains (P75063, P75016, P78042, SC5314, AMS3050) and investigates CNVs driving tolerance and resistance to anti-fungal azole drugs (87) (Supplementary Table  S6 ). We analyzed nineteen samples, including (i) four clinical isolates (P75063, P75016, P78042, SC5314), (ii) seven strains selected in vitro against the anti-fungal drug fluconazole (FLC) (AMS4104, AMS4105, AMS4106, AMS4107, AMS4397, AMS4444, AMS4702), (iii) four isogenic colonies adapted to the drug miconazole (AMS3051, AMS3052, AMS3053 and AMS3054) together with their progenitor (AMS3050) and (iv) three colonies derived from a miconazole-adapted population and isolated on a rich medium (AMS3092, AMS3093 and AMS3094)  (87) (88) (89).
Running times and computational resources We used the Nextflow built-in options to render the process execution time and the computational resources of each task executed by GIP for each sample considered in this study. As exemplified for the cancer cell line dataset, the GIP read mapping process accounts for the longest execution time for all three samples (Supplementary Figure  S10A ) and the highest I/O (Input/Output) data access, with peaks of read and written bytes of 773. 4 and 598. 8 Gb, respectively (Supplementary data 4). The process requiring the highest physical memory is 'bigWigGenomeCov' (mean usage 37. 18 Gb) (Supplementary Figure  S10B ). This step runs deepTools2 and bedGraphToBigWig  (91)  to create wiggle (wig) type files representing continuous-valued sequencing coverage data in indexed binary format (bigWig).
The time line and the computational resources usage including CPU, memory, job duration and I/O of all datasets are provided (Supplementary data 4). The computational resources and process execution statistics to run all giptools commands described in this study are provided (Supplementary Table  S10 ). The longest running time (474 578. 57 and 4 417. 61 s in user and system time, respectively) and the maximum required RAM size (9 465 100 Kb) were measured for the estimation of P. vivax phylogeny (Supplementary Table  S10 ), and largely explained by the execution of IQtree2.
The State of Alethe Alethe combines two major ideas whose roots reach back ten years to the first PxTP workshop in 2011  [7, 9]. It was proposed as an easy-to-produce format with a term language very close to SMT-LIB  [5] , the standard input language of SMT solvers, and rules with a varying level of granularity, allowing implicit proof steps in the proof and thus relying on powerful proof checkers capable of filling the gaps. Since then the format has been refined and extended  [2]. It is now mature, supports coarse-and fine-grained proof steps capturing SMT solving for the SMT-LIB logic UFLIRA  2  and can be reconstructed by the proof assistants Coq  [1, 11]  and Isabelle  [12, 14]. In particular, the integration with Coq was also used as a bridge for the reconstruction of proofs from the SMT solver CVC4  [3]  in Coq, where its proofs in the LFSC format  [15]  were first translated into the veriT format before reconstruction.
Finally, the format will also be natively supported in the upcoming cvc5. solver  3 On the one hand, Alethe uses a natural-deduction style calculus driven mostly by resolution  [7]. To handle first-order reasoning, dedicated quantifier instantiation rules are used  [9]. On the other hand, it implements novel ideas to express reasoning typically used for processing, such as Skolemization, renaming of variables, and other manipulations of bound variables  [2]. While the format was always inspired by the SMT-LIB language, we recently  [12]  changed the syntax of Alethe to closely resemble the command structure used within SMT-LIB. When possible Alethe uses existing SMT-LIB features, such as the define-fun command to define constants and the :named annotation to implement term sharing. The following proof fragment gives a taste of the format.
To rectify this, we studied veriT's source code and wrote an independent document with a list of all rules and a clear mathematical definition of each rule. We chose a level of precision for these descriptions that serves the implementer: precise enough to clarify the edge case, but without the details that would make it a fully formal specification. We are now extending this document to a full specification of the format. This specification is speculative in the sense that it will not be cast in stone. It will describe the format as it is in use at any point in time and will develop in parallel with practical support for the format within SMT solvers, proof checkers, and other tools. Flexible Rules. The next solver that will gain support for the Alethe format is the upcoming cvc5 solver. Implementing a proof format into another solver reveals where the proof format is too tied to the current implementation of veriT.
Powerful Tooling. We believe that powerful software tools may greatly increase the utility of a proof format. Towards this end we have started implementing an independent proof checker for Alethe. In contrast to a proof-assistant-based reconstruction, this checker will not be structured around a small, trusted kernel, and correct-by-construction extensions. Instead, the user would need to trust the implementation does not lead to wrong checking results. Instead, its focus is on performance, support for multiple features and greater flexibility for integrating extensions and refinements to the format. The Isabelle checker is currently not suited to this task -one major issue is that it does not support SMT-LIB input files. 5  This independent checker will also serve as a proof "elaborator".
Acknowledgment We thank  Bruno Andreotti  for ongoing work on the proof checker and  Hanna Lachnitt  for ongoing work on the cvc5 support. We are grateful for the helpful comments provided to us by the anonymous reviewers. The first author has received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation program  (grant agreement No. 713999 , Matryoshka). The second author is supported by the  LIT AI Lab  funded by the  State of Upper Austria.
Recent research considers few-shot intent detection as a meta-learning problem: the model is learning to learn from a consecutive set of small tasks named episodes. In this work, we propose PROTAUGMENT, a meta-learning algorithm for short texts classification applied to the intent detection task. PROTAUG-MENT is a novel extension of Prototypical Networks  (Snell et al. , 2017)  that limits over-fitting on the bias introduced by the few-shots classification objective at each episode. It relies on diverse paraphrasing: a conditional language model is first fine-tuned for paraphrasing, and diversity is later introduced at the decoding stage at each meta-learning episode. The diverse paraphrasing is unsupervised as it is applied to unlabelled data and then fueled to the Prototypical Network training objective as a consistency loss.
, 2021). First, comparative studies involve simple and limited datasets in terms of number and separability of classes (SNIPS  (Coucke et al. , 2018) , a very popular dataset, includes only 7 classes, with the current best model performing over 99% accuracy  (Cao et al. , 2020) ). Second, as we further better understand  (Niven and Kao, 2019) , fine-tune  (Liu et al. , 2019b; Hao et al. , 2020)  and refine  (Khetan and Karnin, 2020)  BERT-derived models, it is not clear if the different meta-learning frameworks can be considered state-of-the-art due to their architecture or due to the improvements of available text encoders at the time of conception. (Dopierre et al. , 2021)  concludes that Prototypical Networks  (Snell et al. , 2017)  (that were using LSTM-based text encoders when introduced in NLP) are actually the state-of-the-art for intent detection when equipped with a fine-tuned BERT text encoder model.
Paired with paraphrasing user utterances and its consistency loss incorporated in Prototypical net-works, our model is the best method for intent detection meta-learning on 4 public datasets, with neither extra labeling efforts nor domain-specific conditional language model fine-tuning. We also show that PROTAUGMENT, having access to only 10 samples of each class of the training data, still significantly outperforms a Prototypical Network which is given access to all samples of the same training data.
PROTAUGMENT In this section, we present our semi-supervised approach PROTAUGMENT. Along with the labeled data randomly chosen at each episode, this approach uses U unlabeled data randomly drawn from the whole dataset -that is, data from training, validation, and test labels. We first do a data augmentation step from this unlabeled data, where we obtain M paraphrases for each unlabeled sentence. The m th paraphrase of x will be denoted xm. Then, given unlabeled data and their paraphrases, we compute a fully unsupervised loss. Finally, we combine both the supervised loss L (the Prototypical Network loss using labeled data) and unsupervised loss (denoted L) and run back-propagation to update the model's parameters.
Generating augmentations through paraphrasing The BART  (Lewis et al. , 2020)  model is a Transformer-based neural machine translation architecture that is trained to remove artificially corrupted text from the input thanks to an autoencoder architecture. While it is trained to reconstruct the original noised input, it can be fine-tuned for taskspecific conditional generation by minimizing the cross-entropy loss on new training input-output pairs  (Bevilacqua et al. , 2020). In PROTAUGMENT, we fine-tune a pre-trained BART model on the paraphrasing task. The paraphrase sentence pairs we use for this task are taken from 3 different paraphrase detection datasets 1  : Quora  (Sharma et al. , 2019) , MSR  (Zhao and Wang, 2010) , and Google PAWS-Wiki  (Yang et al. , 2019; Zhang et al. , 2019). Those datasets have different sizes, and the largest one -Quora -consist of 149,263 pairs of duplicate questions.
To balance turns of sentences (questions/non questions paraphrases), 50% of our fine-tuning paraphrase datasets is made of Quora, 5. 6% of MSR and 44. 4% PAWS-Wiki. This yields 94,702 sentence pairs to train the model on the paraphrasing task. We include both code and data on our github repository 2. Using this fine-trained paraphrasing model, we can generate paraphrases of unlabeled sentences, hopefully having paraphrases representing the same intents as the original sentences. To add some diversity in the generated paraphrases, we use Di-  BART is pre-trained for the paraphrasing task on three datasets: Quora  (Sharma et al. , 2019) , MSR  (Zhao and Wang, 2010)  and Google PAWS-Wiki  (Yang et al. , 2019; Zhang et al. , 2019). The paraphrase model is used to paraphrase unlabeled samples but equipped with diversity strategies (back translation being proposed as a baseline).
Datasets We consider the DialoGLUE benchmark  (Mehri et al. , 2020) , a set of natural language understanding benchmark for task-oriented dialogue, which contains three datasets for intent detection: Banking77, HWU64 and Clinic150the three datasets were already available prior the release of DialoGLUE. Additionally, we also consider the Liu57 intent detection dataset, as it contains the same order of magnitude of intent classes and is user-generated as well. All datasets are public and in English.
Banking77 The Banking77 dataset  (Casanueva et al. , 2020)  classifies 13, 083 user utterances related to into 77 different intents. This dataset i) is specific to a single domain (banking) and ii) requires a fine-grained understanding to classify due to intents being very similar. Following  (Mehri et al. , 2020)  and contrary to  (Casanueva et al. , 2020) , we designate a validation set along a training and a testing set for that dataset (Table  1  resented only in one set of labels. This ensures the model learns to discriminate between both intents and domains. Clinic150 This dataset  (Larson et al. , 2019)  classifies 150 user intents in perfectly equallydistributed classes. This chatbot-like style dataset was initially designed to detect out-of-scope queries, though, in our experiments, we discard the out-of-scope class and only keep the 150 labeled classes to work with, as in  (Mehri et al.
Experimental settings Conditional language model and language model. For the BART fine-tuning process, we used the defaults hyper-parameters reported in  (Lewis et al. , 2020) , and we fine-tuned the BART model for a single epoch (two hours on a Titan RTX GPU). Increasing the number of epochs for fine-tuning BART degrades performances on the intent detection task: the downstream diverse beam search struggles to find diverse enough beam groups since the model perplexity has been lower with further fine-tuning (this is also hinted in  (Bevilacqua et al. , 2020) ). Our text encoder f φ is a bert-base model, and the embedding of a given sentence is the last layer hidden state of the first token of this sentence. For each dataset, this model is fine-tuned on the masked language modeling task for 20 epochs.
Then, the encoder of our meta learner is initialized using the weights of this fine-tuned model. Datasets From a dataset point-of-view, we create two data profiles: full (all the training dataset is available, the usual meta-learning scenario) and low (only 10 samples are available for each training class, an even more challenging meta-learning scenario in which a model meta-learns on very few samples per training class). All experimental setups are run 5 times. For each run, we randomly select training, validation, and testing classes, as well as the samples for the low setting. We train the few-shot models for a maximum of 10, 000 C-way K-shots episodes, evaluating and testing every 100 episodes, stopping early if the evaluation accuracy has not progressed for at least 20 evaluations. We evaluate and test using 600 episodes, as in other few-shot works  (Snell et al.
Evaluation of paraphrase diversity We evaluate the diversity of paraphrases for each method, and report results for two representative datasets in Table  3  (due to space limitations, the report for all datasets is given in appendix B). For each paraphrasing method and each dataset, metrics are computed over unlabeled sentences and their paraphrases. To assess the diversity of paraphrases generated by the different methods, the popular BLEU metric in Neural Machine Translation is a poor choice  (Bawden et al. , 2020). We use the bi-gram diversity (dist-2) metric as proposed by  (Ippolito et al. , 2019) , which computes the number of distinct 2-grams divided by the total amount of tokens. We also report the average similarity (denoted use) within each sentence set, using the Universal Sentence Encoder as an independent sentence encoder.
Those results are consistent across the different datasets, except for Clinic for which accuracies are all very high, making all methods hardly separable. The dataset is not challenging enough, or in other words, metalearning is robust to unbalanced short text classification problems given the nature of that dataset. These results illustrate the need for unsupervised paraphrasing and show that using diverse paraphrases provide a significant performance leap. In the 1-shot (resp. 5-shot) scenario, our best meta-learner improves prototypical networks by 5. 27 (resp. 2. 85) points on average. Remember that these improvements are made in an unsupervised manner hence at no additional cost. Slightly different from to  (Xie et al. , 2020) , we do not find statistical differences depending on the rate at which L is annealed in PROTAUGMENT loss (α ∈ {0.