dataesr/scientific-paragraphs-categorization huggingface dataset:
Method In this work, we focus on the problem of mammography binary breast cancer classification in the context of data with heterogeneous annotations. First, we have a limited number of well-annotated samples with gold-standard-confirmed ground truth. Second, the available samples also have labels of other classes, such as density (i. e. , BI-RADS), cancer probability (i. e. , ACR), and view angle. Third, we have access to an extended dataset with images having labels of other classes only, without confirmed breast cancer classification. We aim at maximizing the sources of knowledge using most of the available data for the training of an image classifier. We address the problem of poorness and uncertainty of the ground truth labels with an MTL approach. Let I ∈ R H×W be an input mammography and let each image I have at most T labels (one per classification task) leading to a target vector y = [y 1 ,.
Implementation Details The code was implemented using Keras  (62)  and Tensorflow  (63). For the training, we used Adam optimizer with the learning rate of 1e -3, taking into account the sum of five losses used to train the network. The numbers of epochs in different experiments (see section 3. 3) vary and mainly depend on whether the training is performed from scratch or fine-tuned. We set the dropout rate to 0. 5. In all experiments, except for fine-tuning (see section 3. 3. 5), the neural network is trained from scratch and the weights initialized with Xavier method  (64). No additional hyperparameter was used to balance the different loss terms, i. e. , balancing factors are fixed to 1.
Datasets The experimental validation relies on three datasets coming from different populations, locations (countries), and mammography systems' vendors. First, we use a private multi-vendor dataset composed of 2,520 Full Field Digital Mammography (FFDM) images from four different vendors, namely Fujifilm, GE, Hologic, and Planmed. It contains 1,271 benign and 1,249 malignant mammograms. All the images were annotated with the labels for the four considered classification tasks: binary cancer classification, cancer probability, density, and view angle classes. We excluded cases with implants and clips, as well as post-surgical malignant cases. In the following, we refer to this dataset as HMI and the images of this dataset as I ∈ D HMI. Second, we use the publicly available INBreast dataset (30) composed of 410 FFDM images from a Siemens mammography system.
Similar to the HMI dataset, the images were annotated with the labels for the four considered classification tasks. We  count 100 malignant images and 310 benign images (including ACR3 class). This dataset is denoted as INB and its images as I ∈ D INB. Finally, we use data from the Susan G. Komen Tissue Bank at the IU Simon Cancer Center  (65). This dataset is composed of ≈16K donated mammograms coming mainly from healthy (i. e. , negative biopsy patients). The images do not systematically have labels of all four tasks, often missing cancer classification and sometimes missing the probability of cancer or the density class. However, most of the cases are labeled with density and/or cancer probability. The images come mainly from Hologic mammography systems, ≈10% from Fujifilm and GE systems, and the remaining from other lower represented systems (under 5%).
Since our bottleneck is of size We apply several augmentation techniques at train time, including (i) random horizontal and vertical translation, (ii) random zoom, and (iii) random vertical flipping. Each image can be modified with none, one, or several augmentations with a probability of 0. 5 for each technique. Our main training dataset (i. e. , D HMI train ) contains limited number of samples (see Table  1 ). Hence, to prevent the overfitting we observed during our first experiments, we applied data augmentation techniques. We rely on the state-of-the art augmentation techniques described in  (16, 17). However, we avoid shearing to prevent artificial deformations, and we did not use horizontal flipping to keep the overall shape consistency as in  (18).
Experimental Setup To explore and evaluate our multitask method we perform several experiments and ablation studies that show the interest of each task in improving the joint latent representation and consequently, the overall classification performance. We use the HMI dataset for training and testing, split with a 80/20 ratio. We separated the images breast-wise, i. e. , images of the same breast belong to the same subset. We note that several the patients had only one view available for a malignant breast. We refer to the train images as D HMI train and to the test images as D HMI test. Similar to  (16) , we use the INB dataset for validation only; that is, we do not train the networks on the INB dataset. Finally, we use the VTB dataset for the fine-tuning of our network. These datasets were fully or partially annotated for the 4 classification tasks (breast bin , ACR, BIRADS, view), as mentioned in section 2.
3. An illustration of samples and their multitask classes is given in Figure  5. We evaluate the performance of the breast classification and the view angle with the tasks with Area Under the Receiver Operator Curve (AUC). The ACR and BI-RADS classification tasks are evaluated with weighted accuracy metrics to compensate for the class imbalance. Unless stated otherwise, the training is performed for 100 epochs from scratch. We calculate the average and the standard deviation of the metrics over the 10 top-performing epochs on the D HMI test dataset.
State-of-the-Art Comparison To compare our method to the state-of-the-art, we evaluate the performance of several approaches on the D HMI test and D INB datasets following the same experimental setup. First, we set as baseline network a ResNet22, denoted as "Baseline, " similar to the encoder E(•) but without multi-level feature fusion and trained for a single binary classification task. We train this baseline model from scratch on D HMI train. We also compare the ResNet22 implementation from (18), denoted as "Wu et al. " Unlike ours, this implementation uses regular convolutions and contains additional dense layers after the GAP. We used the pre-trained image-wise model, made publicly available by the authors 2  after training on ≈1M images. Then, we compare to an ensemble approach, denoted as "Ensembles.
" The ensemble combines the top-performing models of the baseline ResNet22 architecture from 5 runs of training from scratch with 5 different random seeds. Finally, we compare to the Monte-Carlo (MC) dropout method (66), denoted as "MC-Drop, " running 10 forward passes of the baseline network with a dropout of 0. 5 at test time and computing the average over the predictions. We report the results on the two test datasets (D HMI test and D INB ). We also report results for the auxiliary tasks (density and view-angle), each computed over the relevant portions of the test datasets.
Fine-Tuning With Uncertainty Scoring Finally, we explored the proposed uncertainty-based loss weighting in the fine-tuning scenario. As an initial model, we used the top-performing model pre-trained on D HMI train. For fine-tuning, we relied on the D VTB dataset, which has heterogeneous labels, as described in section 2. 3. In this dataset, most of the highly-graded ACR cases do not have a histopathology follow-up. Hence, there is an underlying uncertainty in the breast classification outcome, preventing the straightforward training of the binary classification task. To cope with the uncertainty, we, first, created synthetic breast bin labels using the available ACR labels as follows: for the scores ACR ∈ {1, 2} we set breast bin = 0, for ACR ∈ {4, 5} we set breast bin = 1, ACR = 3 are ignored.
Then, we defined the uncertainty scores u t as described in section 2. 1 for each sample. For simplicity of the experiments, we set the same uncertainty score u 1 const for all of the synthetic labels. We ran fine-tuning with several values of u 1 const ∈ {0. 0, 0. 25, 0. 5, 0. 75, 0. 9} on D VTB dataset and evaluated on D HMI test and D INB (denoted FT u • ). For the labels of other tasks we set the uncertainty scores u 2. 4 const = 0. 5. We collected metrics for all the classifications tasks. In this experiment we also applied class reweighing in the loss function, to compensate for the significant under-representation of ACR4+ samples (see Table  1 ). We set the weight to 20 for all samples having ACR ∈ {4, 5}, and to 1 for all the other classes. We run 20 epochs of fine-tuning.
RESULTS With the experiments described in the previous section, we evaluate our proposed MTL method and report hereafter the results. When comparing to the state-of-the-art methods (see Tables  2, 3 ) we achieve higher binary breast-cancer classification scores on both datasets, i. e. , AUC = 80. 46 and AUC = 78. 13 on D HMI test and D INB , respectively, compared to the state-of-theart methods. Noteworthy, our method yields more consistent classification performances in the stratified groups: there is less variation between binary classification scores when the evaluating performance on a subset of samples for for a given type of breast. Interestingly, our method outperforms (18) on both datasets (AUC = 80. 46 vs. AUC = 71. 26 and AUC = 78. 13 vs. AUC = 74. 04), while being trained on a smaller dataset.
We note that we used the "image-only" method proposed by  (18) , which claims lower performances compared to other methods proposed by the authors. We hypothesize that our dataset might contain wider variety of mammography system vendors, favoring generalization. We rely on the AUC metric to summarize the trade-off between the true-positive and the false-positive rates for different probability thresholds. The AUC is the most frequently reported score in the state-of-the-art  (16) (17) (18) 32). Other metrics, such as sensitivity and specificity, require choosing an operating point. As an illustration, for the D INB dataset, we obtain a sensitivity of 80. 00% and a specificity of 49. 03% when fixing the operating point at a malignancy probability of p > 0. 5. In comparison, the method of (  18 ) reaches a sensitivity of 2.
00% and a specificity of 100. 00% for the same operating point; while moving the operating point to p > 0. 02 leads to scores of 75. 00 and 53. 87, respectively. We observe a similar behavior on the D HMI test dataset. Using the same operating point (p > 0. 5), our method obtains scores of 79. 92 and 68. 63, comparable to those for D INB. However, for the method from  (18) , the operating point of p > 0. 02 leads to scores of 65. 84 and 62. 45. We also explored the Matthews correlation coefficient (MCC) for its better fitness to the unbalanced-dataset scenario, such as D INB  (70). In our case, we obtained the highest MCC = 0. 53, at Sensitivity of 51. 00 and Specificity of 94. 52. The baseline method yields MCC = 0. 43 at  The AUC score is reported. Sensitivity of 53. 00 and Specificity of 88.
09 ± 1. 06 with a 7. 4% gain), followed by the ACR classification tasks (AUC = 77. 60 ± 0. 26 with a 6. 91% gain). Nevertheless, the view and BIRADS classes also contribute, allowing to achieve scores  We evaluated the results of the MTL when reducing the number of samples vs. excluding only the breast bin labels. We note that keeping the auxiliary labels allows maintaining higher performances (see Table  5 ). Remarkably, the performance of the baseline model (see Tables  2, 3 ), trained on the full D HMI train dataset with complete labels is lower than our MTL with only 50% of samples retained. In the same set of experiments, we also compare to two state-of-the-art methods (see section The evaluation is performed on the D HMItest dataset. The evaluation is performed on the D INB dataset.
For the unseen distribution (i. e. , D INB ), the scores sometimes decrease as in case of density and view-angle classification. That is, we obtain an accuracy of 31. 70 vs. 48. 75 for the density classification and AUC = 84. 12 vs. AUC = 97. 28 for the view-angle classification. In this work, we report weighted accuracy for ACR and density multi-class classification. We additionally explored other metrics, such as the F1-score for these tasks observing similar trends. For the D HMI test dataset, we obtain  for the D INB dataset, we obtain F 1 = 32. 11 ± 1. 87 and F 1 = 48. 92 ± 1. 07 for the MTL and mono-tasks trainings, respectively. Our fine-tuning experiments show the potential of our proposed approach to learn from datasets with scarce or noisy labels. The performance increase is particularly relevant for the experiment with the unseen dataset (D INB ), where an improvement w.
The design of our deep neural network architecture contains three main components: (i) a feature fusion block combining features from multiple levels of a ResNet-like encoder; (ii) a light-weight decoder for image reconstruction playing the role of implicit regularizer; and (iii) 4 classification outputs from the bottleneck of the auto-encoder. All three components contribute concurrently to the improvement of the performances. Moreover, our uncertainty-weighted training strategy has also shown the potential to improve the classification performance while training on heterogeneous data. In this case, the improvement is conditioned to a high uncertainty label (i. e. , 0. 90). We attribute this phenomenon in part to the significant data imbalance (see Table  1 ) and in part to the eventual presence of noisy labels (false positives and false negatives) in the dataset.
However, it can be part of future work. We note that our MTL method does not reach the state-ofthe-art performances claimed by (  16 ) and (  17 ), i. e. , AUC = 95. 0 on the INBreast dataset. However, both of these methods rely on additional labels provided by experts in the form of ROI around malignant/suspicious regions. These ROI are used for region detection in case of (  16 ), and for patches extraction in case of  (17). Moreover, the metrics claimed by  (17)  are collected only on a portion of the dataset, unlike our metrics computed on the entire dataset. Furthermore, as our method is designed to produce an image-wise prediction, we compute image-wise metrics. If we instead follow the evaluation protocol in (  16 ), computing breast-wise predictions (i. e. , considering the highest malignancy probability for two views of a breast), we obtain an AUC = 85.
23. The MTL approach also contributes to the explainability of the method. Indeed, the simultaneous training allows for bettershaped latent representation at the auto-encoder bottleneck (see Figure  6 ). We, therefore, expect that samples fall closer to anatomically-and physiologically-related images, even if misclassified or uncertain. We also observe better performances with the multi-task model when evaluating specific types of breast densities or view angles, while the network is trained on the entire dataset (see Tables  2, 3 ). Our approach is relatively straightforward, allowing for the introduction of other tasks, e. g. , classification upon the presence of different findings, such as masses, calcifications, etc. This offers a potential of performance improvement at little cost. Thanks to the MTL approach, our method can ease learning in environments where all the labels are not easily available, for example (if the regulation permits) in a federated learning setup  (71, 72).
CONCLUSION This work is a contribution to the field of breast cancer classification research. The proposed method aims to provide a reliable prediction of breast cancer using mammography images as input. To improve the quality of the prediction, we propose to rely on multi-task learning, introducing several tasks, including in particular density and view angle classification. We observe the superiority of our method compared to other state-of-the-art approaches when evaluating on two independent datasets. Moreover, we note a more meaningful representation of the images in the latent space supporting the explainability of the method. Despite the performance gain, our method still offers room for improvement. First, there is still a gap w. r. t. to state-of-the-art methods using finer levels of annotations [AUC = 95.
The improvements of classification performances may be achieved, for example, in ensembles models setup, which could be part of future explorations. Future work could also include a stronger uncertainty modeling, in particular relying on the prediction uncertainty as in  (55) , instead of the prior knowledge on the dataset. Another possible direction is the extension of the auxiliary tasks performed by the network by introducing, for example, a segmentation task in a self-supervised scenario as in  (28). Moreover, in this work simplified the loss balancing by multiplying each task loss with the factor of 1. Other balancing weights, based on experiments or learned dynamically, could be studied in the future. Finally, the explainability could be improved with a more explicit shaping of the latent space enforced through modeling constraints.
DATA AVAILABILITY STATEMENT Two publicly available datasets were used in this work. Data from the Susan G. Komen Tissue Bank at the IU Simon Cancer Center were used in this work. This data is available online http:// virtualtissuebank. iu. edu/. We used a subset of the dataset. The list of used cases can be obtained by the sending a request to the corresponding author. Data from the INBreast dataset  (30)  were also used. The data can be accessed by the sending a request to the authors. Furthermore, a multi-site private dataset was used in this work. Private agreements were signed between the institutions and Hera-MI. Institutional board approvals were obtained from each of the sites. This dataset can be shared upon justified requests and subsequent rights-holder approvals. Requests are to be addressed to the corresponding author.
CoV-2 pseudotypes For pseudotyping, murine leukemia virus pseudo-typed particles (PP) containing the spike of SARS-CoV-2 or derived mutants were produced according to a published protocol  [15]. Briefly, HEK-293TT cells (10 6 cells per P6 well) were transfected with plasmids encoding GAG-POL, F-LUC and SARS-CoV-2 spikes. Supernatants containing the pseudo-typed particles were harvested at 48 hours after transfection, pooled and filtered through 0. 45 μm pore-sized membranes. Five different PP were produced, a first containing the spike of the SARS-CoV-2 type (Genebank accession number: MN908947), and the four others containing the spike with RBD mutations representatives of the  variant (N501Y substitution), the  variant (N501Y, K417N and E484K mutations), the  variant (N501Y, K417T and E484K substitutions), the / variant (substitutions L452R and E484Q) and the ο variant (S371L, S373P, S375F, K417N, N440K, G446S, S477N, T478K, E484A, Q493K, G496S, Q498R, N501Y, Y505H substitutions).
Gene expression quantification by RT-qPCR. Total RNA was extracted from frozen olfactory mucosa and lungs using the Trizol method. Oligo-dT first strand cDNA were synthesized from 5 μg total RNA by iScriptAdv cDNA kit (Biorad, # 1725038) following the manufacturers recommendations. qPCR was performed using cDNA templates (5 μL) added to a 15 μL reaction mixture containing 500 nM primers (sequences in Table  1 ) and iTaq Universal SYBR Green Supermix (Biorad, # 1725124) using a thermocycler (Mastercycler ep realplex 2 , Eppendorf). The expression levels of target genes were measured using the Eppendorf realplex 2 software. A dissociation curve was carried out at the end of the PCR cycle to verify the (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission.
Statistical analysis Data shown as the means ± SEM. All statistical comparisons were performed using Prism 8 (GraphPad). Quantitative data were compared across groups using two-way ANOVA test for pseudovirus assay, weight, nasal swab and virus titre evolution. All other parameters were  Screening an αReps phage library allowed the identification of several binders specific of the RBD of the SARS-CoV-2 spike protein. Their binding affinity for the S1 domain was measured by biolayer interferometry. The neutralization activity of selected αReps was evaluated using a pseudo-typed S SARS-CoV-2 neutralization assay and a SARS-CoV-2 infection assay. Competitive binding assays were carried out by BLI to identify Reps recognizing nonoverlapping binding sites. Then, αRep derived constructs followed the same characterization steps than their single counterparts.
Virus strains. SARS-CoV-2 strain BavPat1 was obtained from Pr. C. Drosten through EVA GLOBAL (https://www. european-virus-archive. com/) and contains the D614G mutation. SARS-CoV-2 Alpha, (201/501YV. 1) was isolated from a 18 years-old patient. The full genome sequence has been deposited on GISAID: EPI_ISL_918165. The strain is available through EVA GLOBAL: UVE/SARS-CoV-2/2021/FR/7b (lineage B 1. 1. 7, ex UK) at https://www. european-virus-archive. com/virus/sars-cov-2-uvesars-cov-22021fr7b-lineage-b-1-1-7-ex-uk. SARS CoV-2 Beta (SA lineage B 1. 351) was isolated in France in 2021, The strain is available through EVA GLOBAL: UVE/SARS-CoV-2/2021/FR/1299-ex SA (lineage B 1. 351) at https://www. european-virus-archive. com/virus/sars-cov-2-uvesars-cov-22021fr1299ex-sa-lineage-b-1351. Sars-Cov-2 Gamma (SARS-CoV-2/2021/JP/TY7-503 lineage P.
1, ex Brazil) was isolated in Japan in January 2021. The full genome sequence has been deposited on GISAID: EPI_ISL_877769. The strain is available through EVA GLOBAL at https://www. european-virus-archive. com/virus/sars-cov-2-virus-strain-sars-cov-22021jpty7-503-lineage-p1-ex-brazil. SARS-CoV-2 Delta, (India lineage B. 1. 617. 2): the full genome sequence has been deposited on GISAID: EPI_ISL_2838050. The strain is available through EVA GLOBAL: SARS-CoV-2/2021/FR/0610 (Lineage B 1. 617. 2, Delta) at https://www. european-virus-archive. com/virus/sars-cov-2-virus-strain-sars-cov-22021fr0610lineage-b-16172-delta. To prepare the virus working stocks, a 25cm2 culture flask of confluent VeroE6 TMPRSS2 cells growing with MEM medium with 2. 5% FCS was inoculated at a multiplicity of infection (which was not certified by peer review) is the author/funder.
* is research is a part of a project that have received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation programme Grant Agreement 714704  ( M. Pilipczuk). Eun Jung Kim  is supported by the grant from  French National Research Agency  under JCJC program (ASSK:  ANR-18-CE40-0025-01 ). A preliminary version of this work was presented at  STOC 2022 [KKPW22a ].
(c) Simulation of SR in software SR can be simulated in software in a straightforward fashion by relying on high precision floatingpoint arithmetic. The computation whose result is to be rounded stochastically is performed using higher precision and then rounding the high-precision result using (5. 1), where q(x) in (5. 2) is based on the (higher-precision) approximation to x rather than on its exact value. This approach is easy to implement as long as higher-than-working-precision arithmetic is available, be it in hardware, for instance when emulating binary32 rounding using binary64 arithmetic, or in software, through arbitrary precision libraries such as the GNU Multiprecision Library (GNU MPFR)  [34]. In practice, once the high precision solution has been computed, the rounding step can be performed in several ways.