dataesr/scientific-paragraphs-categorization huggingface dataset:
CNN Design and Implementation The two CNNs have identical architecture and they are based on a state-of-the-art 3D CNN  [45]. Architecture version 6 from reference 60 was selected because it is a relatively small CNN, so it can be trained with small datasets, yet it can also learn effective multiscale spatio-spectral segmentation features in HSI data. This was demonstrated by its good performance for segmenting remote sensing HSIs with limited data, The CNN is organized such that in early layers, 3D spatio-spectral feature maps are produced, which are then reduced to1D feature vectors, and then finally processed by a fully connected layer. Network parameters are significantly reduced using ideas from SqueezeNet (https: //arxiv. org/abs/1602. 07360, accessed on 1 March 2021)  [46] , using a small number of filters combined with 1D convolutions along the spectral dimension in the pooling phase.
We illustrate the architecture in Supplementary Figure  S3. Layer 1 is a 3D convolution with 20 output channels. Layer 2 is a 1D convolution along the spectral dimension with pooling using a stride of 2. Layers 3 and 4 replicate layers 1 and 2 with 35 output channels. Layers 5 and 6 are 1D convolutions with pooling to further reduce the spectral dimension. Layer 7 is a fully connected layer where the number of output neurons is the number of classes (in our case 2). Layers 1, 3, 5, and 6 are followed by ReLU activation and layer 7 is followed by sigmoid activation. The CNN has a total of 32,444 trainable parameters and it was implemented in Pytorch 1,4. 0 (https://pytorch. org/, accessed on 1 March 2021) with Python 3. 7. 4 (https://www. python. org/, accessed on 1 March 2021). The CNN was organized into 7 layers, where each layer consisted of a spatial convolution followed by downsampling and max-pooling operations.
AI Training Processes The organ segmentation CNN was trained from segmentation masks generated from HS images by a skilled human operator. The operator demarcated the spatial extent of the liver in each HS image, for all pigs and at times 0 min, 30 min, 60 min, and 90 min (42 images). An interactive segmentation software was used (GIMP v. 2. 8), using an RGB image simulated from each HSI image. As an example, we illustrated a demarcated image (Figure  2b, f ). The operator segmented both the interior of the liver (positive class) and all regions surrounding the liver (negative class). The organ segmentation CNN was then trained using supervised learning with the segmentation masks provided by the human operator as ground truth. Special care was taken to ensure that the ability of the organ segmentation CNN to generalize to novel subjects could be tested.
To achieve this, LOPOCV was performed as follows. Five organ segmentation CNNs were trained, with one CNN for each pig. Each CNN was trained using image data from all pigs except for one (the held-out pig). The predictive performance of the CNN was then evaluated using images from the held-out pig only. LOPOCV was necessary to eliminate an elevated performance bias, which occurs if a CNN is trained and evaluated on data from the same subject. The training was performed and implemented in Pytorch using as follows. The HSI images used for training were first concatenated vertically to form a single HSI image, denoted by the HSI image I. The training label images were similarly concatenated to form a single training label image L. The values in L were either +1 (indicating liver), -1 (indicating non-liver), or 0 (indicating a spatial location that was close so close to the border of the liver that its label could not be determined by the annotator thanks to optical blurring around the liver border.
optim" package with a learning rate of 0. 01 and a weight decay (L2 regularization) of 0. 0005. The training was terminated after 2000 epochs. The time to train was approximately 13 h using a server (DGX 1 equivalent, Nvidia cooperation). The time to train all 5 CNNs for each cross-validation was approximately 65 h. The tissue characterization CNN was trained in practically the same manner as the organ segmentation CNN. Specifically, all training parameters and processes were identical, and the sole difference was the construction of the training data. LOPOCV was also used where for each animal, one tissue characterization CNN was trained using HSI data from all other animals (4 images per animal corresponding to times 0 min, 30 min, 60 min, and 90 min). Its predictive performance was then evaluated on the 4 images from the held-out animal.
Blood Analysis Blood for systemic lactate and liver function was sampled through a central catheter placed in the jugular vein (6 French IV catheter). Capillary lactate was analyzed by puncturing Glisson's capsule with a randomized selection of liver segments. Capillary and systemic lactates were measured using a strip-based portable lactate analyzer, which presents a margin error of 0. 35 mmol/L (EDGE, ApexBio, Taipei, Taiwan). The correlation analysis of the data was performed between HSI parameters and capillary lactate concentration. The surgical intervention was monitored to rule out any bias in the hepatic ischemic phase by means of systemic blood gas analysis (BGA) with the epoc Blood Analysis System (Siemens Healthineers, Henkestr, Germany) to measure pO 2 , pCO 2 , pH, glucose, creatinine, urea, and BUN.
Statistical Analysis Statistics were performed using GraphPad 8. 3 (GraphPad Software, San Diego, CA, USA). A Spearman's and Person's rho were analyzed to perform the correlation between optical and biological data. All data were expressed as means ± s. d. One-way ANOVA with Dunnett's multiple comparisons was performed for parametric tests to calculate differences in continuous paired variables. The Friedman test with Dunn's multiple comparison test was applied for paired non-parametric tests. A two-tailed p-value < 0. 05 was considered statistically significant. One-tailed significant p-value < 0. 05 was applied to the confusion matrix in the correlation between the AI score and biological data, considering that the correlation among these values was physiologically possible only in one direction.
The HS data showed more visible and well-defined changes as compared to the RGB camera. Oxygenation measured with StO 2 % and NIR% significantly decreased in both ischemic and reperfusion phases. The last HS image of pig 4 that died after 2 h of reperfusion showed that the liver was not perfused, and H&E confirmed parenchymal and microvascular disruption. Overall, after 5 h of reperfusion, the perivenular region appeared necrotic, probably due to the ischemic damage with a detachment of hepatocytes from the adjacent sinusoidal reticular support. Hepatocytes were reduced in volume with acidophilic cytoplasm and pyknotic nuclei. A mix of inflammatory cells such as lymphocytes and polymorphonuclear elements were visible in the parenchyma (Figure  4i ). Grouping pigs 1, 2, and 3, following the AI score, the statistical analysis showed that the HS image quantification of StO 2 % and NIR% presented a significant decrease as compared to the control (p = 0.
Dataset size is a very important factor when applying a deep learning approach. This relatively small dataset prohibits end-to-end training of large CNNs with a large number of trainable parameters. However, the test performances give strong empirical evidence that a deep learning approach works well with this dataset despite its small size using the proposed CNNs. We emphasize that using LOPOCV, data from the held-out animal was never used for training. The strong results, particularly liver classification with a sensitivity of 0. 993 and a specificity of 0. 997, show that the CNNs have learned well because they generalize to data from animals not present in the training set. This has been possible because the CNNs have been trained with mild regularization (weight decay), but more importantly, the CNNs have a relatively small number of trainable parameters  (32, 444).
This greatly reduces the potential to overfit and it is much smaller compared to larger CNNs typically used for image classification with RGB images such as ResNet50 [61] with over 23 million trainable parameters. Recall that with RGB data, spatial context is extremely important for good classification, which requires filters acting over relatively large spatial distances. In contrast, in HSI, the rich reflectance information reduces the need for spatial context. As a future work, we aim to study the benefit of enlarging the capacity of the CNNs with a larger dataset. This can be both in terms of spatial window (to add more spatial context information) and in terms of network depth (to add more abstract spatio-spectral feature representations into the CNNs). Concerning the CNN model, we believe that a simpler classifier could also be used to perform well on the segmentation task.
Funding: This study was funded by the ARC Foundation for Cancer Research (9 Rue Guy Môquet, 94803 Villejuif, France) through the ELIOS (Endoscopic Luminescent Imaging for precision Oncologic Surgery) grant and partly by the ANR (National Research Agency) through the Liver SURG grant (ANR-18-CE19-0026). This work was also partially supported by French state funds managed within the "Plan Investissements d'Avenir" and by the ANR (reference ANR-10-IAHU-02).
Study patients Patients aged ≥18 years with a diagnosis of AD, as defined by the American Academy of Dermatology, at least 12 months before screening, were included. Eligible patients had a documented history of inadequate response to topical medications and thus met criteria for systemic therapy. Additional details on BREEZE-AD7 study design and patients, including settings and location where data were collected, blinding, randomisation and inclusion/exclusion criteria, are presented elsewhere. 20.
Study design BREEZE-AD7 (NCT03733301) was a multicentre, randomised, double-blinded, placebo-controlled Phase 3 clinical trial. Patients were randomised 1 : 1 : 1 to receive oral placebo, baricitinib 2-or 4-mg plus TCS once daily for 16 weeks (Fig. S1 ). At baseline, patients were instructed to apply moderate potency TCS until lesions were clear or almost clear and then to switch to mild potency TCS for 7 days and then stop. If lesions reappeared, patients could resume the regimen. Patients experiencing unacceptable or worsening symptoms after 2 weeks of treatment may have been rescued with high-or ultra-high potency TCS or oral systemic agents. Patients who required rescue using oral systemic agents discontinued study treatment.
Treatment comparisons for PBI scores were analysed using an analysis of variance model, where treatment, region and baseline vIGA-AD were included as factors. The P-values for treatment comparisons were not adjusted for multiplicity. Data collected after first rescue therapy or permanent study drug discontinuation were considered missing. Non-responder imputation was used to impute missing values for categorical variables. No explicit imputations were conducted for continuous measures; MMRM analysis was performed to mitigate the impact of missing data because it yields valid inferences assuming that missing observations are missing-at-random. Observed data were used to analyse continuous PBI scores as the postbaseline data were collected only once during the treatment period. Per the study protocol, WPAI-AD outcomes were prespecified as secondary objectives, and DLQI, PROMIS and PBI outcomes were prespecified as exploratory objectives.
received grants, personal fees or non-financial support from  AbbVie , sponsored by  Abbott/AbbVie ,  ALK Scherax ,  Almirall , received grants from  Galderma ; MT has been a co-investigator for AbbVie, Boehringer,  Eli Lilly and Company, Galderma ,  Janssen-Cilag ,  Leo Pharma ,  Pierre Fabre ,  Sanofi-Regeneron , and  Eli Lilly and Company , under licence from  Incyte Corporation. The studies were funded by  Eli Lilly and Company. All costs associated with the development of this manuscript including article processing charges and medical writing services, provided by  Lydia Morris, PhD , of  Syneos Health (Raleigh, NC ), were funded by  Eli Lilly and Company.
Methods: We systematically searched Trip Medical Database, PubMed and Guidelines International Network from January 2012 up to December 2020 using the following search strings: 'cardiovascular disease' , 'prevention' , combined with specific cardiovascular disease risk factors. The identified records were screened for relevance and content. We methodologically assessed the selected guidelines using the AGREE II tool. Recommendations were summarized using a consensus-developed extraction form. Results: After screening, 27 clinical practice guidelines were included, all of which were developed in Western countries and showed consistent rigor of development. Guidelines were consistent about the benefit of regular, moderateintensity, aerobic physical activity. However, recommendations on strategies to achieve and sustain behavior change varied.
Methods We applied standard systematic review methodology as outlined by the Cochrane Collaboration  [12]  and we used the PRISMA 2 checklist  [13]  (Additional file 1) for selfevaluation of the overall standards and quality requirements for reporting a systematic literature review. All authors contributed to the development of the research protocol prior to the study. Between September 2017 and January 2018, NA, PVR and HB carried out a systematic search on Trip Medical Database and International Guidelines Library of the Guidelines International Network (G-I-N) to reach a broad scope of CPGs. An additional systematic search was subsequently carried out on G-I-N and PubMed in December 2020, with the aim of updating the results of this review with the most recently published guidelines. Suitable search strategies were developed for each database, using multiple combinations of free text, MeSH terms, word variants, Boolean operators and truncation for: 'cardiovascular disease' , 'prevention' , 'risk' , 'lifestyle' , 'physical activity'.
In order to ensure accuracy of data extraction for this literature review, an author-team consensus-based data extraction form was determined, comprising of three phases. NA extracted the data, regularly conferring with the senior research team (PVR, LP, PVB, GM, HB). Firstly, we listed all included CPGs and extracted publication year (or year of latest update), country, developing organization, language and title. Secondly, we made an inventory of all PA recommendations, stand alone or as a component of a broader lifestyle recommendation, in order to get an overview of the relevant content of each included CPG with regards to our research questions. If reported, the following characteristics were extracted from each recommendation and its scientific underpinning: strength of recommendation and level of evidence, intervention description and outcomes, implementation strategies, evidence gaps.
Funding This work was funded by the  European Commission , as part of a  Horizon 2020  project grant for the  SPICES  project. Project  ID :  733356 , Funded under:  H2020-EU. 3. 1. 6-Healthcare provision  and integrated care. Funding bodies had no role in the design of the study; the collection, analysis and interpretation of the data; nor the decision to approve publication of the final manuscript.
In this paper, we report experiments in which we aim to automatically classify primate vocalizations according to four primate species of interest, plus a background category with forest sound events. We compare several standard deep neural networks architectures: standard deep convolutional neural networks (CNNs), MobileNets and ResNets. To tackle the small size of the training dataset, less than seven thousand audio files, the data augmentation techniques SpecAugment and MixUp proved to be very useful. Against the very unbalanced classes of the dataset, we used a balanced data sampler that showed to be efficient. An exponential moving average of the model weights allowed to get slight further gains. The best model was a standard 10-layer CNN, comprised of about five million parameters. It achieved a 93.
We use the audio dataset provided by the INTER-SPEECH 2021 Computational Paralinguistics Challenge  [9, 11, 10]. Besides the four primate species of interest, a fifth class is considered in this classification task: natural background noise. In real applications, this "reject" class would be very interesting to have in a classifier if we aim at spotting out primate calls within lengthy audio field recordings. Besides comparing different models, we also investigate the use of audio data augmentation, as it is key for training networks. We will report significant performance gains using SpecAugment  [15]  and MixUp  [16]. The best system reported by the challenge organizers is based on extracting feature representations from a recurrent autoencoder in an unsupervised fashion and then use these embeddings, named "auDeep", as input to an SVM classifier.
Feature extraction The primate vocalization dataset  [10]  consists of three subsets ("train", "devel" and "test") of comparable size: 6915, 6918 and 6923 audio files of variable duration. We padded all the files to reach three seconds duration, as it corresponds to the largest recording duration of all the files. 64 log-mel spectrograms were extracted with a 1024-sample-long Hanning window and 320 samples hop size. We ran experiments with the original 16 kHz sampled signal and a downsampled 8 kHz version to speed up model comparison. With the 16 kHz version, spectrograms are of size 150 × 64 in time and frequency.
Experimental setup The models were trained to optimize binary cross-entropy, when MixUp was used, and categorical cross-entropy otherwise. In all our experiments, we used the Adam  [23]  optimizer to train the models with a learning rate of 1e-3. The size of the minibatches was set to 32. We trained our model for 60k iterations. Except WideResNet, all the model architectures are the ones proposed in  [17]  for multilabel audio event detection and we largely used their open-sourced implementation 1. Our PyTorch  [21]  code is available  2. We used Exponential Moving Average (EMA) to obtain our final models: during the training of any model, we create a separate model which is the EMA variant of the model being trained. We used a 0. 995 decay and EMA updates were initiated after 15k training iterations.
It may happen that the 1e-3 learning rate is not the best one for all the models, for instance. Nevertheless, the fact that a standard CNN is the best model in these experiments, is in line with the results obtained by Kong and colleagues on AudioSet  [17]. We tested CNNs with more layers (CNN14) on our task, but CNN10 proved to be better, probably because CNN14 is too deep to be correctly trained with our small dataset. In Table  2 , we also report three UAR values obtained on the official challenge test set. CNN10-16k reached a 92. 5% UAR on the test set, showing that it generalized well on these new data. It outperformed the ELM model  [10]  at 70. 7%, and the 86. 6% auDeep-SVM baseline provided by the challenge organizers  [11]. Figure  1  shows the confusion matrix on the devel subset, obtained with our best model.
Ablation study Table  3  shows the impact of several components: balanced sampler, data augmentation and EMA. This ablation study was performed with ResNet22-8k and CNN10-16k. The first two lines compare the use of a random sampler (no-bal) and a balanced sampler (bal). Using a random sampler is worse than a balanced one in UAR, as expected. In the case of ResNet22, there is a two points difference in mAP and 4 points in UAR. This is because the background class is better recognized with no-bal, and it is by far the most represented class in the dataset (50% of the samples). The next three lines show the impact of data augmentation: specAugment alone (specA), MixUp alone and specAugment and MixUp together. We can see that both techniques boost mAP and UAR significantly: about 3 points absolute in mAP and 5.
Conclusions We reported experiments aiming to classify primate vocalizations into four primate species of interest, plus a background category with forest sound events. We compared standard DNN architectures: standard deep convolutional neural networks (CNNs), MobileNets, ResNets and WideResNets. To tackle the relative small size of the training dataset, less than seven thousand audio files, the data augmentation techniques SpecAugment and MixUp proved to be very useful. Against the very unbalanced distribution of the classes, we used a balanced data sampler, which was efficient with a ResNet but not with our standard CNN. An exponential moving average of the model weights allowed to get slight further gains. The best model was a standard 10-layer CNN, comprised of about five million parameters, that achieved a 92.
Acknowledgements This work was partially supported by the  French ANR agency  within the  LUDAU  project ( ANR-18-CE23-0005-01 ) and the  French "Investing  for the  Future -PIA3" AI Interdisciplinary Institute ANITI (Grant agreement   ANR-19-PI3A-0004 ). We used HPC resources from  CALMIP  (Grant  2020-p20022 ).
Data Source The CIBMTR collects consecutive detailed HCT data from a volunteer network of >450 transplantation centers worldwide. These patient data, including the information from yearly longitudinal observations, are reported to a centralized statistical center of the CIBMTR research headquarters located at the Medical College of Wisconsin and the National Marrow Donor Program. The observational studies conducted by the CIBMTR meet the compliance requirements of all applicable federal regulations to protect all human research subjects. The Medical College of Wisconsin Institutional Board and the Privacy Officer granted a waiver of informed consent for this study, which was conducted in accordance with Health Insurance Portability and Accountability Act regulations.
Patient Selection Adult patients with AML or MDS who were 40 to 65 years old at their first HCT (between 2009 and 2015) and thereby potentially eligible to receive either MAC or RIC were included in this analysis. Peripheral blood, bone marrow, and umbilical cord blood (UCB) graft sources and all related donor (except identical twin) and unrelated donor (URD) types  [24]  were included. We excluded patients from embargoed centers and patients who were missing conditioning intensity, cytogenetic, or pre-HCT disease status information or comprehensive research data or informed consent forms. CIBMTR consensus criteria were used to define the conditioning intensity  [25]. Cytogenetic risks of AML  [18]  and MDS  [26]  were classified as described previously and considered for the revised DRI classification  [17].