dataesr/scientific-paragraphs-categorization huggingface dataset:
g. , SARC) or outperforming them (e. g. , COAD, OV, LGG). Students that penalized clinical variables tended to underperform both their teachers and the clinical-only model on cancers on which the clinical-only model performed particularly well (e. g. , COAD, OV), which was likely part of the reason for their overall underperformance, relative to their teachers (Figure  S1 ). We did not find a clear link between student fidelity as measured by Spearman's ρ and the gap between teacher and student performance. On COAD for example, where all students outperformed their teachers in terms of median test concordance (Figure  1 ), student fidelity was almost as high as on BLCA (Figure  S2 ), a dataset on which teachers and students performed almost identically in terms of test concordance. Further, for comparisons to selected reference methods, we restricted our attention to clinically guided students (Equation (3)) due to their improved test fidelity and discriminative performance.
Sparse linear students outperformed sparse linear models trained on survival targets Our student models consistently outperformed the considered sparse reference methods, in addition to the clinical-  only Cox PH model (Figure  2  and Figure  S3 ). None of the reference methods was able to outperform the clinical-only Cox PH model significantly as measured by a Wilcoxon ranked sign test across datasets, a feat that all of our students achieved. In addition, our student models achieved similar (for the RBF and BF teachers) or somewhat higher (for the HSAE teacher) sparsity levels across modalities (Table  2 ), compared to the reference models. Further, we noticed that while prioritylasso selected modalities somewhat uniformly, resulting in similar sparsity levels across modalities, our students seemed to focus much more on important modalities such as gene expression.
4 ± 9. 4 LASSO FAVORING 9. 4 ± 7. 4 11. 7 ± 13. 8 12. 0 ± 23. 6 9. 9 ± 13. 2 0. 2 ± 0. 5 0. 7 ± 1. 2 2. 8 ± 6. 1 by teachers trained on the full multi-omics data and distilled their knowledge to students only using a set of two input modalities. We considered clinical data in addition to each available genomic input modality. Reference methods were fit on the same modalities as the students but were trained using the survival targets instead of the teacher knowledge. Across all modalities, we again saw our students outperform the reference models consistently (Table  3 ). One of the students performed best in terms of pooled mean concordance in all settings and the reference methods performed significantly worse than the best student in all settings. Further, while all of our students performed significantly better than the clinical-only Cox PH model in at least three settings, the reference methods did not significantly outperform the clinical-only model in any setting.
Conclusion Our work has shown that knowledge distillation can in principle be effectively applied to survival models more broadly, and for the sparsification of black-box multi-omics models on TCGA data more specifically. Moreover, our results suggested that building sparse linear survival models for multi-omics integration could be improved by sparsifying already well-performing black-box models, as opposed to attempting to fit sparse linear models from scratch. We acknowledge several limitations of our work. First, we limited ourselves to a subset of the TCGA datasets, meaning that our results cannot be guaranteed to generalize to other cancers or databases. In addition, the teachers considered in our work had previously been shown to perform particularly well on TCGA. Further, we restricted our attention to non-pan-cancer models, that is, models that did not leverage transfer learning across multiple cancer types, a limitation that future work should certainly address  (Cheerla & Gevaert, 2019; Vale-Silva & Rohr, 2021).
Sponges ( phylum Porifera) are early-branching animals, whose outwardly simple body plan is underlain by a complex genetic repertoire. The transition from a mobile larva to an attached filter-feeding organism occurs by metamorphosis, a process accompanied by a radical change of the body plan and cell transdifferentiation. The continuity between larval cells and adult tissues is still obscure. In a previous study, we have produced polyclonal antibodies against the major protein of the flagellated cells covering the larva of the sponge Halisarca dujardini, used them to trace the fate of these cells and shown that the larval flagellated cells transdifferentiate into the choanocytes. In the present work, we identified the sequence of this novel protein, which we named ilborin. A search in the open databases showed that multiple orthologues of the newly identified protein are present in sponges, cnidarians, flatworms, ctenophores and echinoderms, but none of them has been described yet.
We considered the S-antigen identified in our study to be a new protein, since we found no described orthologues in the literature, and we named it ilborin. There are several unidentified homologues of ilborin in the protein databases of different invertebrates. We revealed the domain structure and the expression of ilborin as well as its diversity among Metazoa. We assume that this protein is involved in the energy metabolism regulated by calcium and that its function is conserved in several groups of marine invertebrates.
This band was subjected to trypsin digestion and MS/MS accompanied by searching in proteomic databases based on transcriptome assembly of H. dujardini (electronic supplementary material, file S1). The search yielded a 577 aa residue protein sequence. The attempt to find homologous proteins by tBLASTn against the NCBI Nucleotide database showed the lack of reliable similarity with any known proteins. We assumed that the protein was novel and named it ilborin. An additional search in the transcriptome revealed three more paralogues of ilborin, and we indicated them by letters from A to D (GenBank accession numbers MT559266-MT559269). Each of the ilborin transcripts has distinct 5 0 and 3 0 untranslated regions (electronic supplementary material, file S2). The existence of four transcripts was confirmed by PCR with primers designed to untranslated regions (UTRs).
Sequence analysis ilborin paralogues (Hdu_ilborin A, B, C, D) share 78% identitical amino acid positions. The four hduilborin gene ORFs comprise 577-583 aa residues. Hdu_ilborins are acidic proteins with a mean theoretical molecular mass of 66. 386 kDa (table  1 ). They have no signal peptide targeting them to any special cell compartment or secretion, neither do they have any predicted transmembrane domains. Based on this, Hdu_ilborins were considered to have cytoplasmic localization. Hdu_ilborin A was analysed further as a representative of the four paralogues. Protein backbone disorder prediction showed an overall rigidity of the sequence (S2 > 0. 8) with three inner disordered regions (S2 < 0. 7; figure  3 , dashed line). Two regions belonging to the superfamily aldolase-type triosephosphate isomerase (TIM) barrel (IPR013785; 22-354 aa residues) and EF-hand domain pair (IPR011992; 458-540 aa residues) were predicted (figure  3 ).
The aldolase-type TIM barrel superfamily contains many proteins with enzymatic activity possessing a characteristic beta/alpha barrel fold. Ca-binding proteins usually have an EF-hand domain. A highly conserved Ca-binding site from 514 to 526 aa residues (IPR018247) present in the Hdu_ilborin A EF-hand domain (figure  3 ) was identified in other invertebrates as well as in the sequence of all three other paralogues. It is described below and in electronic supplementary material, file S8.
Phylogeny of ilborin homologues The search against transcriptomic and genomic databases of different organisms, from choanoflagellates to vertebrates, with ilborin sequence as a query produced a number of unidentified homologues. They were found in Porifera, Cnidaria, Ctenophora, Platyhelminthes and Echinodermata (figure  6a ; electronic supplementary material, files S10 and S11). No homologues were observed in Choanoflagellata or chordates. Two bacterial proteins with the same domain structure and a significant e-value were found in the cyanobacteria Moorea producens and Okeania hirsuta. In Bayesian and maximumlikelihood (ML) analyses, the bacterial sequences fall into a sister clade to the metazoan ones and were used to root the tree (figure  6a ). Ilborin homologues are present in three classes of Porifera: Demospongiae, Calcarea and Homoscleromorpha.
The well-studied sponges A. queenslandica (Demospongiae) and Sycon ciliatum (Calcarea) have three paralogues each. Among cnidarians, the orthologues were present in Hydrozoa (Hydra vulgaris, Clytia hemisphaerica), Scyphozoa (Aurelia aurita) and numerous Anthozoa (N. vectensis, Orbicella faveolata, Pocillopora damicornis and others). Interestingly, both ctenophore sequences were placed inside a shared cnidarian branch. Our search in numerous datasets of planarians, nematodes and trematodes did not reveal any homologues. The only protein identified in Lophotrochozoa was found in Macrostomum lignano, a new model species of Platyhelminthes  [26]. Surprisingly, it was placed inside the Porifera clade with high support. The lengths of all orthologous proteins (560-610 aa residues), their conserved domain composition, and their relative positions were congruent with ilborin structure (figure  6b ).
EF-hand domain analysis EF-hand domain is a characteristic feature of all ilborin orthologues. We generated HMM LOGO of the consensus EF-hand domain (figure  7 ) and compared it with the Ca-binding sites of known EF-hand proteins. The relative position of aspartate, glutamate and phenylalanine residues in the consensus corresponded to that of amino acid residues in EF-hand responsible for calcium binding (residues 1, 3, 5 and 12 in terms of standard numeration of residues in EF-hand) (figure  7 )  [27, 28]. A large part of the molecule is occupied by the TIM barrel domain. We suggested that this part of the ilborin sequence might be similar to other EF-hand-containing proteins. In order to check this hypothesis, we compared fulllength EF-hand-containing proteins from the Swiss-Prot database and ilborin orthologues by all-versus-all basic local alignment search tool (BLAST).
Ilborin structural features At the first step of the S-antigen identification, BLAST search through all databases did not show any similarity with any described proteins. We found a number of previously undescribed orthologues in different invertebrate phyla but not in ancestral choanoflagellates. These orthologues have not been annotated, the information about them being limited to the name of the organism from which the transcriptome or the genome of the sequence was isolated. Halisarca dujardini ilborins and all the sequences collected from public databases possess an identical domain architecture: they contain an EF-hand domain and an aldolase TIM barrel domain (figure  3 ). They also show a high level of sequence similarity. Many of the species studied have more than one paralogue in their genome: in 13 species, 25 different ilborins were found.
The large number of similar sequences suggests that they belong to a newly defined family of metazoan proteins. Substantial differences in 5 0 and 3 0 UTRs seem to indicate that four ilborin paralogues are encoded by different genes. Other species also have more than one ilborin transcript or even gene (for species for which a sequenced genome is available). These features point to different genes rather than allele variants or products of differential splicing. While Hdu_ilborin A-D cDNAs have been cloned and sequenced, the sequences from other invertebrates are hypothetical mRNA/ protein-coding sequences (CDSs) that are yet to be characterized. Still, we suppose that these RNAs arise from different genes. Two putative conserved domains were identified in the amino acid sequence based on comparison with the databases: the aldolase TIM barrel domain and the EF-hand domain.
Microscopy For transmission electron microscopy (TEM) specimens were fixed with 2. 5% glutaraldehyde and post-fixed with 1% OsO 4. Specimens were embedded in Araldite (Sigma-Aldrich). Ultrathin sections (60-80 nm) were cut with an royalsocietypublishing. org/journal/rsob Open Biol. 12: 210336 Ultramicrotome PowerTome XL, equipped with a Drukkert 45°diamond knife, and contrasted with uranyl acetate. Ultrathin sections were studied under a Zeiss-1000 (Carl Zeiss) transmission electron microscope. The images were processed using ImageJ software (FiJi).
In silico sequence analysis and phylogeny The average molecular mass and isoelectric point were calculated with ProtParam  [59]  on the ExPASy server (https:// www. expasy. org/). Subcellular location was predicted with SignalP 5. 0, SCL-Epred  [60]  and WoLF PSORT  [61]. Protein backbone dynamics were predicted with DynaMine  [62]. Attribution to protein superfamilies and the Ca-binding site were predicted with InterPro 79. 0  [21]. For remote similarity search we used both HHblits  [63]  and HHpred  [64]  algorithms. In HHblits alignment of the four paralogues, the sequence of ilborin from H. dujardini (Hdu_ilborin) was queried in the UniRef30_2020_03 database. For HHpred search, all related proteins were aligned with MAFFT (v. 7. 402) and corrected manually. Part of the alignment 1-355 aa based on Hdu_ilborin A corresponding to the proposed enzymatic domain was queried in the PDB database (PDB_mmCIF70_29_May).
Positions of active site residues were assessed based on experimentally validated interaction with the substrate or a metal ion  [31, 65]. Unannotated homologues were identified in genomic and transcriptomics datasets with tBLASTn at the NCBI server, Fungal genome resource of the Joint Genome Institute (https://mycocosm. jgi. doe. gov/mycocosm/home), Ensembl Metazoa server (https://metazoa. ensembl. org), Compagen server (http://www. compagen. org/), M. lignano genome project (http://www. macgenome. org/), Caenorhabditis elegans server (https://wormbase. org/), Platynereis server (http:// pdumbase. gdcb. iastate. edu/platynereis/) and Planarians database server (http://planmine. mpi-cbg. de/). Conserved domains in identified sequences were determined with Inter-ProScan (v. 5. 44-79. 0). Proteins of other invertebrates were aligned with those from H.
dujardini using MAFFT (v. 7. 402) and corrected manually. Maximum-likelihood phylogenetic trees were estimated using the IQ-TREE web server  [66]. Statistical support was estimated by performing 100bootstrap replicates. Bayesian analyses were performed with MrBayes v. 3. 2. 7, using the Le & Gascuel (LG) + Γ+I model of evolution, with four chains, a subsampling frequency of 100, and four parallel runs, each 500 000 generations. The best-fit model for our set of proteins was chosen using ModelTest-NG (v. 0. 1. 5). Bayesian posterior probabilities were used to assess the confidence values of each bipartition. The analysis was carried out using computational resources provided by the Resource Centre 'Computer Centre SPbU' (www. cc. spbu. ru). Trees were visualized with FigTree 1. 4. 4 software.
Ilborin sequences were compared with other EF-hand proteins as follows. Annotated EF-hand proteins were obtained from the Swiss-Prot database (Prosite: PS50222). Ilborin orthologue sequences were joined to the Swiss-Prot entry, and all-by-all BLAST was performed in order to obtain the similarities between the sequence pairs and to calculate edge values for the generation of the SSN. Network files were created with EFI-EST (Enzyme Similarity Tool,  [29] ) under default settings, as the node for each entry and the edge for similarity or identity between entries. The network was visualized and analysed using Cytoscape 3. 8. 0 [67]. Markov clustering (MCL) was performed with cluster-Maker (v. 1. 11), a Cytoscape plug-in in the Markov Clustering Algorithm  [68]. HMM LOGO of the EF-hand domain was generated with the hmmsearch web-tool based on one ilborin paralogue from each analysed species owing to low sequence divergence.
Total white blood cells count is an important diagnostic parameter in both human and veterinary medicines. State-of-the-art is performed by flow cytometry combined with light scattering or impedance measurements. Spectroscopy point-of-care has the advantages of miniaturization, low sampling, and real-time hemogram analysis. While white blood cells are in low proportions, while red blood cells and bilirubin dominate spectral information, complicating detection in blood. We performed a feasibility study for the direct detection of white blood cells counts in canine blood by visible-near infrared spectroscopy for veterinary applications, benchmarking current chemometrics techniques (similarity, global and local partial least squares, artificial neural networks and least-squares support vector machines) with self-learning artificial intelligence, introducing data augmentation to overcome the hurdle of knowledge representativity.
Chemometrics or AI models that rely on data intrinsic correlations and do not hold causal relationships to the constituent should be used with caution, as high bias may occur if an unknown sample is out of the spectral characteristics of this subset  [14]. Spectral POC hemogram analysis was developed for measuring RBC, Hgb and HTC in dog and cat blood  [31]. The following MAPE metrics were achieved: (i) Dog blood: RBC (6. 39%), Hgb (7. 14%), hematocrit (HTC) (4. 43%); (ii) cat blood: RBC (5. 67%), Hgb (4. 08%) and HTC (1. 69%). RBC, Hgb and HTC are absorbance dominant. These are very well detected by Vis-SWNIR by spectral POC, allowing an accurate diagnosis at both high and low boundaries of the reference interval. MAPE at the higher and lower boundaries of the RI are 4% to 11%, allowing an accurate diagnosis  [31].
Thus, the information contained in compositional changes and spectral bands is the same, only expressed into different basis and units (e. g. , signal intensity at each wavelength vs. WBC concentration). This relationship is vectorial and the eigenstructure is unidimensional, being described by the molar extinction coefficient of the constituent, where concentrations are proportional to this vector basis  [13, 14, 31]. In complex samples, e. g. , blood, multi-scaled interferences arise from overlapping bands and distortions due to matrix effects (e. g. , pH, scattering). Quantitative and qualitative interference information is continuous and spreads along all wavelengths. As biological variance is significantly large, the covariance of large/representative datasets is unstable and presents high dimensionality.
This makes it necessary to unscramble the different types of interference that accurately relate the quantitative information of a particular constituent in the context of their interferents by searching for the CovM it belongs to  [14]. The CovM is given by a group of samples that provide the same information between spectral interference features and constituent concentration, isolating a particular interference mode present in the dataset. Each CovM sample has stable covariance between spectra (X) and constituents (Y) information. Such also implies that the information is similar but expressed on a different basis (wavelengths and concentrations). Therefore, the two information blocks exhibit latent structural similarity (t ∼ u), where t and u are derived independently from singular value decomposition of X and Y, where: X = TP t and Y = UQ t ; being P and Q the orthogonal basis of T and U, respectively.
Validation All models were constructed and validated using a two-step approach: (i) crossvalidation to optimize model parameters within the training set; and (ii) prediction for hold-out samples (HO) to estimate the error. Cross-validation (CV) is a hypothesis test to the null hypothesis, that is, the sample being present in the model dataset or not being present, a statistically similar result is expected or the effect is null. By leaving several samples out of model estimation, CV provides the error estimated for each sample in the training set if this sample is unknown, allowing one to decide which are the optimal parameters of each model that best depict the representative features of the dataset and not particularities of each sample (aka overfitting). If the dataset and model are representative, the null effect is expected when using the model to predict hold-out datasets, holding similar error results to the training dataset.
CV is used to avoid over-optimization to the dataset (overfitting), and hold-out samples (HO) for null hypothesis testing, determining the generalization of the chosen model. Non-optimal models are more robust to generalization at the cost of accuracy, being an important trade-off when dealing with data scarcity. Models are chosen for optimal generalization at minimum prediction error of CV (e. g. , LV in PLS, ANN architecture). In the case of local methods (Local-PLS and CovM search), the leave-one-out CV and one HO sample were used due to the lower amount of data of the group of samples. Method performance was evaluated by computing the standard deviation (SE), the mean absolute percentage error (MAPE), and the Pearson correlation (R) as a metric of linearity between predicted and measured values.
Spectral Data Augmentation Data augmentation increases the knowledge base diversity for improving model prediction accuracy  [47, 48]. It is especially relevant for spectroscopic blood analysis, because the high biological variance is difficult to be fully characterized by proof-of-concept experimental designs, as these are limited to a low number of samples. We refer to the experimental dataset as the real-world knowledge base dataset (RWD). Herein, we introduce the concept of an 'in silico' synthetic spectroscopy dataset (SSD) as an augmentation technique for improving the spectral quantification of WBC. The SSD is computed using the random mixture of spectra and the WBC of two random realworld samples, producing an average spectrum and WBC as synthetic information for the SSD. This procedure is equivalent to mixing two samples physically, because under an ideal mixture assumption spectral information would have direct correspondence to WBC.
Mixture samples are non-naturally occurring samples. For example, the blood of an animal never has the properties of the mixture of the blood of two different animals. By mixing 'in silico' the information of real samples, the knowledge base has new samples that cover spectral gradients, providing the covariance information between spectral features and blood composition which otherwise would not be present in the RWD. A total of 500 SSD samples were obtained by mixing random pairwise RWD samples of spectra (X) and hemogram (Y), where X SSD = 1 2 (Xi + Xj) and Y SSD = 1 2 (Yi + Yj), being i and j random blood samples (Figure  S1 ). SSD is an independent dataset from RWD, where the information about RWD spectral gradients are expected to be preserved. Thus, models optimized using solely the SSD should be resentative of RWD covariance.
WBC Quantification SIM was optimized using three neighboring samples, taking the Euclidean distance in the 3PC scores space, totaling 90. 74% of spectral variance (Figure  2c ). SIM has a low correlation and high error values (R = 0. 4503, MAPE = 37. 10%) (Table  1 , Figure  3a ). There is a high discrepancy between real and mixture datasets in terms of Pearson correlation coefficient (R). This low performance is because the Euclidean distance in the T space does not directly correspond solely to WBC information, and spectral variance (X t X) is not directly related to covariance (X t Y). Results for spectral POC hemogram of RBC, Hgb, and HTC  [31]  also demonstrated that spectral similarity cannot represent the first principles of the BLL. PLS has significant correlation (R = 0. 6069) but high prediction errors (MAPE = 31.