dataesr/scientific-paragraphs-categorization huggingface dataset:
When several hits were found, we kept the best one for the considered protein. The corresponding phrog number and its functional annotation were then transferred to our newly predicted protein. When no hit with the PHROG database was found, or the affiliation to an existing phrog was too distant (probability < 80%, Evalue > 1 × 10 -4 ), the ORF was annotated as a singleton of unknown function according to the PHROG guidelines. The entire list of the best hit for each ORF for each phage before manual curation is supplied in Table  S5.
PDB-Based Functional Annotation In order to improve the PHROG-based functional annotation for protein still annotated "unknown function", we performed with HHblits a comparison of each predicted protein sequence HMM profile with the Protein Data Bank pdb70 database (accessed on 28 September 2021) to detect homologies with known protein structures. We kept the first hit for each protein sequence for further consideration when the probability was greater than 80% and the Evalue less than 1 × 10 -3. When these criteria were met, we also manually checked the prediction coverage to avoid transferring a PDB annotation based on a predicted structural similarity between a small portion (a domain or a sub-domain) of the PDB hit and the phage protein sequences. After all these considerations, we manually checked for phage ORF ascribed to a phrog of unknown function if a reliable structural prediction from the pdb70 could be used instead.
Prediction of Anti-CRISPR Proteins (Acr) Until very recently, searching for Acr was tedious, essentially relying on "guilt by association" with anti-CRISPR-associated (Aca) proteins containing "helix-turn-helix" motives. However, not all Acr are associated with an Aca. Thanks to the new approaches based on machine learning, it is now possible to predict Acr found in undescribed genomic environments. AcrHub is a platform that incorporates state-of-the-art Acr predictors and three analytical modules (similarity analysis, phylogenetic analysis, and homology network analysis)  [27]. The AcrHub database contains 339 experimentally validated, and 71,728 predicted Acr proteins. Validated Acr are mostly short proteins (89% between 60 and 160 aa). We ran our phage proteomes in AcrHub on two different Acr prediction algorithms (PaCRISPR  [75]  and AcRanker  [76] ).
Meaningful score thresholds are over 0. 5 for PaCRISPR and over -5 for AcRanker. We then used the AcrHub "Similarity analysis" module to relate our predicted Acr with experimentally validated Acr proteins in the AcrHub database. We excluded overlong proteins (>200 aa) and those whose query cover was below 40%. When pertinent, we compared the genomic surrounding of our predicted acr gene with an experimentally validated acr gene product.
The first one is that by combining the results of four different gene callers, including Phanotate especially designed for phage genomes (but that nevertheless tends to over-predict ORFs), we believe we could predict as many ORFs as is reasonably possible. However, we also showed that manual inspection and curation are still needed to get rid of obvious false positive predictions. The second point concerns the improvement of functional annotation with PHROG, a database dedicated to viral proteins at large (bacterial and archaeal viruses and their prophages). We believe this is a promising approach for functional annotation as it relies on viral protein orthologs clustering based on remote homology detection. PHROG capitalizes on decades worth of past work of many research teams that have led to the experimental validations of many functions, and PHROG also benefits from the manual annotation of various experts in the field (we could propose in this study a functional annotation to 24 phrog clusters previously annotated with unknown function).
Acknowledgments: We thank  Artemis Kosta  and  Hugo Le Guenno  from the Microscopy facility at the  Institut de Microbiologie de la Méditerranée (IMM ,  FR 3479 ,  CNRS-Aix-Marseille Université, Marseille, France)  for TEM grids preparation and image acquisition. We are very thankful to  Marie-Agnès Petit  ( INRAE, Jouy-en-Josas, France ) for fruitful discussions on functional prediction and phage genome annotation, discussions we had personally but also during dedicated workshops organized by the  French bacteriophage network  (www. phages. fr,accessed on 17 June 2022). We thank  François Enault  and  Eric Olo Ndela  ( LMGE, Université Clermont Auvergne, Clermont-Ferrand, France ) for their help with the PHROG database. We are very grateful to the  "Phage Cycle and Bacterial Metabolism " team members at the  Laboratoire de Chimie Bactérienne  ( UMR 7283 ,  CNRS-Aix-Marseille Université, Marseille, France)  for their help and support.
This research was funded by the  Agence Nationale de la Recherche of France , grant number  ANR-16-CE12-0029 , the  Ministerio de Ciencia e Innovación of Spain (Agencia Estatal de Investigación) , grant number  BIO2016-75235-P , and the  European Regional Fund , grant number  PID2020-116995RB-I00. Institutional Review Board Statement: Not applicable. Note applicable.
Data Availability Statement: Raw sequencing reads and Genbank files for the 10 newly isolated salmonella phages studied here are openly available at NCBI under the BioProject accession numbers PRJNA767534 and PRJEB37792. Input and output files pertaining to vContact2 analyses are publicly available since 17 June 2022 at figshare (https://doi. org/10. 6084/m9. figshare. 20088356). Output files pertaining to VICTOR analyses are publicly available since 17 June 2022 at figshare (https://doi. org/ 10. 6084/m9. figshare. 20088416 for Kuttervirus and https://doi. org/10. 6084/m9. figshare. 20088473 for Jersyvirus).
•Figure 3. 3 Figure 3. Interaction network between Caudoviricetes bacteriophages based on shared protein ortholog families. The interaction network was built using vContact2 and visualized with Cytoscape. Each dot corresponds to a single phage. (a) Network comprising all the Caudovirictes viruses in the vContact2 database. Viruses belonging to the Ackermannviridae, Siphoviridae, and Podoviridae families are highlighted in blue, red, and green, respectively. The rectangle indicates the region in the network where the 10 phages studied here are positioned. (b) Sub-network of the region underlined in (a) restricted to Ackermannviridae, Podoviridae, and Siphoviridae surrounding our 10 new phages. Viruses belonging to the Chivirus genus (VC_20), the Ackermannviridae family (VC_34), the Guernseyvirinae sub-family (VC_75), and the Lederbergvirus genus (VC_243) are highlighted in orange, blue, red, and green, respectively.
Our platform is built with Python 3. 9. 0 on the Flask micro framework (https://flask. palletsprojects. com/). The platform relies on a couple of distinct elements to allow its versatility: the database of perils, interactions, and catastrophe models, and the application backbone handling data management and serving the web interface as well as some distinct graphical elements (the main one being the interactive adjacency matrix). Note that by catastrophe model, we mean the adjacency matrix associated with a specific scenario. A SQL database hosts the data filled in the adjacency matrix by the users (see below). It is composed of distinct tables corresponding to the application data model: perils, interactions between perils, and catastrophe models. The three tables are connected as follow: perils are linked with interactions in two ways depending on whether they act as a trigger (or source) or a target and they can be part of multiple catastrophe models (i.
e. , individual matrices). Interactions have source and target perils which are taken from the peril table. The application backbone is based on the micro framework of Python Flask. The core element of the backbone is the application model where perils, interactions, and catastrophe model objects are set. Communication between the application model and the database is relatively straightforward thanks to the Flask-SQLAlchemy module (https://github. com/pallets/flasksqlalchemy). The basis of the web application is then built with Jinja Template Engine incorporated to Flask that enables the generation of HyperText Markup Language (HTML) files with embedded JavaScript (JS) and Cascading Style Sheets (CSS) for interactive plotting which is needed for presentations and online content (see some screenshot examples in Results).
The interactive adjacency matrix is made with D3. js which is a JS library for manipulating documents based on data (https:// d3js. org/). Interactivity with users is made possible by some elements on the HTML page. The first one is the peril button which allows the user to add some new perils that are then incorporated into the matrix. Afterward, the user can create an interaction between two perils by clicking on the matching matrix cell. Finally, the Catastrophe Model Flowchart is dynamically built with Mermaid. js which is a JS based diagramming and charting tool (https://mermaid-js. github. io). Addition of further functionalities and changes in the design can be expected for future versions of the platform once user feedback has been received (see Future Tests of the Platform).
Let us review the cascades shown in Figure  4B  as they indicate some limits that will need to be addressed in future tests. The five most likely cascades contain the 3-event chain 'earthquake' → 'flood' → 'industrial accident' which was the one presented during the seminar, leading to some obvious overweight. In future applications, the impact of background information will have to be carefully assessed. Although it seems rather difficult to avoid the bias of historical events and background information from impacting decisions, aiming at the gradual development of a comprehensive and uniform database of interactions should limit overweighting linked to recent observations and fashionable trends. Moreover, two interactions proposed by the participants appear to be dubious: 'volcanic eruption' → 'earthquake' and 'mass slide' → 'earthquake'.
Future Tests of the Platform We plan to test and enrich the proposed platform with crisis practitioners and experts within the context of two types of events, in crisis exercises and crisis laboratories: Crisis exercises offer a double opportunity: on the one hand, they allow the testing of the platform's functionalities. Does it give an appropriate picture of the situation? Does it facilitate policy makers' understanding of the situation? Are the suggested crisis scenarios relevant to the situation? Answering these questions makes the functionalities more adapted to the crisis decision makers' needs. On the other hand, these exercises might be opportunities to fill the platform with data provided by participants. Indeed, the database development is the most critical aspect of the platform alongside the user interface.
Recently, semi-supervised learning (SSL) methods, in the framework of deep learning (DL), have been shown to provide state-of-the-art results on image datasets by exploiting unlabeled data. Most of the time tested on object recognition tasks in images, these algorithms are rarely compared when applied to audio tasks. In this article, we adapted four recent SSL methods to the task of audio tagging. The first two methods, namely Deep Co-Training (DCT) and Mean Teacher (MT), involve two collaborative neural networks. The two other algorithms, called MixMatch (MM) and FixMatch (FM), are single-model methods that rely primarily on data augmentation strategies. Using the Wide ResNet 28-2 architecture in all our experiments, 10% of labeled data and the remaining 90% as unlabeled, we first compare the four methods' accuracy on three standard benchmark audio event datasets: Environmental Sound Classification (ESC-10), UrbanSound8K (UBS8K), and Google Speech Commands (GSC).
I. INTRODUCTION S EMI-SUPERVISED learning (SSL) aims to reduce the dependency of deep learning systems on labeled data by integrating non-labeled data during the learning phase. It is essential since the conception of a large labeled dataset is expensive, dependent on the task to be learned, and timeconsuming. On the contrary, the acquisition of non-labeled data is cheaper and quicker regardless of the learning task. Using unlabeled data while maintaining high performance can be done in three different ways: Consistency regularization [1],  [2] , which encourages a model to produce consistent prediction whereas the input is perturbed; Entropy minimization [3]-  [5] , which encourages the model to output high confidence predictions on unlabeled files, and Standard regularization by using weight decay  [6] ,  [7] , mixup  [8]  or adversarial examples  [9].
The most direct approach for SSL is pseudolabeling  [5] , but since then, many new and better approaches came out such as Mean Teacher  [10] , Deep Co-Training  [11] , MixMatch  [12]  and FixMatch  [13]. In this work, we adapted these four SSL methods to the task of audio tagging. One difficulty lies in choosing which audio data augmentation techniques to use, that work for different types of sound events and spoken words. We compare the methods' accuracy on three audio datasets with different scopes and size: Environmental Sound Classification 10 (ESC-10) Dataset  [14] , with audio event categories such as human noise not related to speech, natural ambient noise, UrbanSound8k (UBS8K)  [15] , more specific to urban noises, and Google Speech Commands (GSC) Dataset v2  [16]  containing spoken words exclusively.
In the MixMatch algorithm, a successful data augmentation technique called mixup  [8]  is used. It consists of mixing pairs of samples, both the data samples and the labels with a random coefficient. We propose to add mixup to the three other SSL approaches, namely MT, DCT and FM, which do not already use it. The results reported in this article will highlight the positive impact of mixup in almost all our experiments. This paper's contributions are two-fold: i) the application and comparison of recent SSL methods for audio tagging on three different datasets, ii) the modification of these methods with the integration of mixup resulted in systematic accuracy gains. We shall see that in most cases, MixMatch outperformed the other methods, closely followed by FixMatch+mixup. The structure of the paper is as follows.
B. Deep Co-Training (DCT) DCT has been recently proposed by Qiao and colleagues  [11]. It is based on Co-Training (CT), the well-known generic framework for SSL proposed by Blum and colleagues in 1998  [18]. The main idea of Co-Training is based on the assumption that two independent views on a training dataset are available to train two models separately. Ideally, the two views are conditionally independent given the class. The two models are then used to make predictions on the non-labeled data subset. The most confident predictions are selected and added to the labeled subset. This process is iterative, like pseudo-labeling. DCT is an adaptation of CT in the context of deep learning. Instead of relying on views of the data that are different, DCT makes use of adversarial examples to ensure the independence in the "view" presented to the models.
The second difference is that the whole non-labeled dataset is used during training. Each batch is composed of a supervised and an unsupervised part. Thus, the non-labeled data are directly used, and the iterative aspect of the algorithm is removed. Let S and U be the subsets of labeled and unlabeled data, respectively, and let f and g be the two neural networks that are expected to collaborate. The DCT loss function is comprised of three terms, as shown in Eq. 6. These terms correspond to loss functions estimated either on S, U, or both. Note that during training, a mini-batch is comprised of labeled and unlabeled samples in a fixed proportion. Furthermore, in a given mini-batch, the labeled examples given to each of the two models are sampled differently. L = L sup + λ cot L cot + λ diff L diff (6) The first term, L sup , given in Eq.
A. Datasets and evaluation metrics Environmental Sound Classification 10 (ESC-10)  [14]  is a selection of 400 five-second-long recordings of audio events separated into ten balanced categories. The dataset is provided with five uniformly sized cross-validation folds that will be used to perform the evaluation. The files are sampled at 44 kHz and are converted into 431 × 64 log-Mel spectrograms. UrbanSound8k (UBS8K)  [15]  is a dataset composed of 8742 files between 1 and 4 seconds long, separated into ten balanced categories. The dataset is provided with ten crossvalidation folds of uniform size that will be used to perform the evaluation. The files are zero-padded to 4 seconds, resampled to 22 kHz, and converted to 431 × 64 log-Mel spectrograms. Google Speech Commands Dataset v2 (GSC)  [16]  is an audio dataset of spoken words designed to evaluate keyword spotting systems.
The dataset is split into 85511 training files, 10102 validation files, and 4890 testing files. The latter is used for the evaluation of our systems. We ran the task of classifying the 35 word categories of this dataset. The files are zero-padded to 1 second if needed and sampled at 16 kHz before being converted into 32 × 64 log -Mel spectrogram. In all cases, the 64 Mel-coefficients were extracted using a window size of 2048 samples and a hop length of 512 samples. For ESC-10 and UBS8K, we used the official cross-validation folds. We report the average classification Error Rate (ER) along with standard deviations.
B. Models We used the Wide ResNet 28-2  [17]  architecture in all our experiments. This model is very efficient, achieving SOTA performance on the three datasets when trained in a 100% supervised setting. Moreover, its small size, comprised of about 1. 4 Million parameters, allows to experiment quickly. Its structure consists of an initial convolutional layer (conv1) followed by three groups of residual blocks (block1, block2, and block3). Finally, an average pooling and a linear layer act as a classifier. The residual blocks, composed of two BasicBlock, are repeated three times and their structure is defined in Eq. 21. The number of channels of the convolution layers is referred as l, BN stands for Batch Normalization and ReLU  [21]  for the Rectified Linear Unit activation function. We used the official implementation available in PyTorch  [22].
In MM, the maximum value of λ u is 1. In MM, we use two augmentations (k = 2), the sharpening temperature T is set to 0. 5. In FM, we use a threshold τ = 0. 8 on ESC-10 and GSC datasets, and τ = 0. 95 for UBS8K. For MM and FM, on ESC-10, the batch size is 60 because ESC-10 is a small dataset of 400 files only. During training, only four folders are used, that is, 320 files. In a 10% configuration and due to the whole division's restrictions, this represents only 30 supervised files in total. Each mini-batch must contain as many labeled as unlabeled files, hence the batch size of 60. Moreover, because of this small number of files, the training phase only lasts for 2700 iterations, and therefore, warm-up ends prematurely. For our proposed variants, which include mixup, we kept the same configurations and parameter values.
01% ER, and Weak+mixup yielded the best ER of 6. 58% It represents an augmentation of 3. 43 points, 34. 26% relative improvement. In the 100% setting, the Strong+mixup reached an ER of 2. 98%, outperforming the 4. 94% ER from the supervised model by 1. 96 point, 39. 68% relative improvement. Overall, we observe that in a supervised setting, if using a single augmentation improved the accuracy of our systems, the combination of mixup with a weak or a strong augmentation is systematically better than using one of those augmentations individually, except in the ESC-10 dataset.
B. Semi-supervised learning In this section, we report in Table V the results of the SSL methods with and without the mixup augmentation. For MM, mixup is already used in the original method, thus, we compare MM to MM without mixup (MM-mixup). For the three other methods, we denote, for instance FM+mixup the FM algorithm augmented with mixup. In all the three datasets, the four SSL methods brought ER decreases compared to the 10% supervised learning setup, when no augmentation is performed. Only MM and FM performed better than the best supervised training result, that used the weak augmentations. Furthermore, MM and FM significantly outperformed MT and DCT in all cases, showing that using single-model SSL methods is more efficient than two-model-based methods, at least on these three datasets.
C. Impact of mixup Given that the best SSL method so far was MM, and that mixup is used in MM and not in the three other methods, we decided to try to add mixup to MT, DCT and FM, in different ways for each method as explained in Section III-E. In Table  V , we reported the results when adding mixup to MT, DCT and FM, (MT+mixup, DCT+mixup, FM+mixup). We also give the ER when removing mixup from MM, in the row named MM-mixup. As a first comment, MM-mixup is always worse than MM. For instance, on USB8K, ER increased from 18. 02% to 20. 42%. With the other SSL methods, adding mixup brought performance improvements on all the datasets tested. The only counter-example observed is FM on ESC-10, which went from an ER of 13. 33% to an ER of 14. 67%. Similarly, FM on UBS8K went from 21. 44% ER without mixup to 18.
24% with mixup. On GSC, MM once again presented the most significant improvement with 4. 49% and 3. 25% ERs without and with mixup. It is also important to note that using mixup allowed to get ER values very close to the ones obtained with a fully (100% setting) supervised training using augmentations, on UBS8K and GSC. This is observable with MM and FM+mixup, which, compared to a Weak+mixup 100% supervised, have only 0. 06 and 0. 31 points difference on UBS8K, and 0. 25 and 0. 31 points difference on GSC. When we look at our supervised training performance, we can observe that an improvement in performance does not systematically follow the use of weak or strong augmentation. However, when combined with mixup, ER is frequently improved. This can be partly explained by the fact that audio augmentations are often difficult to choose and that their impact is often dependent on the dataset and the task at hand  [24].
VI. CONCLUSION In this article, we reported monophonic audio tagging experiments in a semi-supervised setting on three standard datasets of different sizes and content, the very small-sized ESC-10 with generic audio events, urban noises with Urban-Sound8K, and speech with Google Speech Commands, using only 10% of the labeled data samples and the remaining 90% as unlabeled samples. We adapted and compared four SSL existing algorithms for this task, two methods that use two neural networks in parallel: Mean Teacher and Deep Co-Training, and the two single-model methods MixMatch and FixMatch, that rely in particular on data augmentation. All the four methods brought significant gains compared to a supervised training setting using 10% of labeled data. They performed better than supervised learning without augmentation, and MixMatch and FixMatch were even better than supervised learning with augmentation.
Adding mixup systematically brought performance gains, which allowed us to get closer to supervised learning performance when using 100% of the labeled data available, and when using with data augmentation. Using FixMatch with mixup, the difference was 0. 31 points (1. 70%) only for UrbanSound8K and Google Speech Commands (10. 33%). In conclusion, if we were to recommend a method out of the ones tested in our work, we would recommend MixMatch, and FixMatch+mixup also, with very similar performances. In future work, we plan to adapt these SSL methods to multi-label audio tagging, for instance on Audioset  [25]  or FSD50K  [26]. In particular, we would have to adapt the sharpen method in MixMatch, and the thresholding operations in FixMatch. In DCT, the adversarial example generation would also need some changes.