Miracle-dz/UniFAR-FTUs huggingface dataset:
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling

Ensembling BERT models often significantly improves accuracy, but at the cost of significantly more computation and memory footprint. In this work, we propose Multi-CLS BERT, a novel ensembling method for CLS-based prediction tasks that is almost as efficient as a single BERT model. Multi-CLS BERT uses multiple CLS tokens with a parameterization and objective that encourages their diversity. Thus instead of fine-tuning each BERT model in an ensemble (and running them all at test time), we need only fine-tune our single Multi-CLS BERT model (and run the one model at test time, ensembling just the multiple final CLS embeddings). To test its effectiveness, we build Multi-CLS BERT on top of a state-of-the-art pretraining method for BERT (Aroca-Ouellette and Rudzicz, 2020). In experiments on GLUE and SuperGLUE we show that our Multi-CLS BERT reliably improves both overall accuracy and confidence estimation. When only 100 training samples are available in GLUE, the Multi-CLS BERT_Base model can even outperform the corresponding BERT_Large model. We analyze the behavior of our Multi-CLS BERT, showing that it has many of the same characteristics and behavior as a typical BERT 5-way ensemble, but with nearly 4-times less computation and memory.
Integral field unit spectroscopy of 10 early-type galactic nuclei - I. Principal component analysis Tomography and nuclear activity

Most massive galaxies show emission lines that can be characterized as LINERs. To what extent this emission is related to AGNs or to stellar processes is still an open question. In this paper, we analysed a sample of such galaxies to study the central region in terms of nuclear and circumnuclear emission lines, as well as the stellar component properties. For this reason, we selected 10 massive ($\sigma$ > 200 km/s) nearby (d < 31 Mpc) galaxies and observed them with the IFU/GMOS (integral field unit/Gemini Multi-Object Spectrograph) spectrograph on the Gemini South Telescope. The data were analysed with principal component analysis (PCA) Tomography to assess the main properties of the objects. Two spectral regions were analysed: a yellow region (5100-5800 A), adequate to show the properties of the stellar component, and a red region (6250-6800 A), adequate to analyse the gaseous component. We found that all objects previously known to present emission lines have a central AGN-type emitting source. They also show gaseous and stellar kinematics typical of discs. Such discs may be co-aligned (NGC 1380 and ESO 208 G-21), in counter-rotation (IC 1459 and NGC 7097) or misaligned (IC 5181 and NGC 4546). We also found one object with a gaseous disc but no stellar disc (NGC 2663), one with a stellar disc but no gaseous disc (NGC 1404), one with neither stellar nor gaseous disc (NGC 1399) and one with probably ionization cones (NGC 3136). PCA Tomography is an efficient method for detecting both the central AGN and gaseous and stellar discs. In the two cases (NGC 1399 and NGC 1404) in which no lines were previously reported, we found no evidence of either nuclear or circumnuclear emission, using PCA Tomography only.
Race Bias Analysis of Bona Fide Errors in face anti-spoofing

The study of bias in Machine Learning is receiving a lot of attention in recent years, however, few only papers deal explicitly with the problem of race bias in face anti-spoofing. In this paper, we present a systematic study of race bias in face anti-spoofing with three key characteristics: the focus is on analysing potential bias in the bona fide errors, where significant ethical and legal issues lie; the analysis is not restricted to the final binary outcomes of the classifier, but also covers the classifier's scalar responses and its latent space; the threshold determining the operating point of the classifier is considered a variable. We demonstrate the proposed bias analysis process on a VQ-VAE based face anti-spoofing algorithm, trained on the Replay Attack and the Spoof in the Wild (SiW) databases, and analysed for bias on the SiW and Racial Faces in the Wild (RFW), databases. The results demonstrate that race bias is not necessarily the result of different mean response values among the various populations. Instead, it can be better understood as the combined effect of several possible characteristics of the response distributions: different means; different variances; bimodal behaviour; existence of outliers.
f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation

Diffusion models (DMs) have recently emerged as SoTA tools for generative modeling in various domains. Standard DMs can be viewed as an instantiation of hierarchical variational autoencoders (VAEs) where the latent variables are inferred from input-centered Gaussian distributions with fixed scales and variances. Unlike VAEs, this formulation limits DMs from changing the latent spaces and learning abstract representations. In this work, we propose f-DM, a generalized family of DMs which allows progressive signal transformation. More precisely, we extend DMs to incorporate a set of (hand-designed or learned) transformations, where the transformed input is the mean of each diffusion step. We propose a generalized formulation and derive the corresponding de-noising objective with a modified sampling algorithm. As a demonstration, we apply f-DM in image generation tasks with a range of functions, including down-sampling, blurring, and learned transformations based on the encoder of pretrained VAEs. In addition, we identify the importance of adjusting the noise levels whenever the signal is sub-sampled and propose a simple rescaling recipe. f-DM can produce high-quality samples on standard image generation benchmarks like FFHQ, AFHQ, LSUN, and ImageNet with better efficiency and semantic interpretation.
Optimal wireless rate and power control in the presence of jammers using reinforcement learning

Future wireless networks require high throughput and energy efficiency. This paper studies using Reinforcement Learning (RL) to do transmission rate and power control for maximizing a joint reward function consisting of both throughput and energy consumption. We design the system state to include factors that reflect packet queue length, interference from other nodes, quality of the wireless channel, battery status, etc. The reward function is normalized and does not involve unit conversion. It can be used to train three different types of agents: throughput-critical, energy-critical, and throughput and energy balanced. Using the NS-3 network simulation software, we implement and train these agents in an 802.11ac network with the presence of a jammer. We then test the agents with two jamming nodes interfering with the packets received at the receiver. We compare the performance of our RL optimal policies with the popular Minstrel rate adaptation algorithm: our approach can achieve (i) higher throughput when using the throughput-critical reward function; (ii) lower energy consumption when using the energy-critical reward function; and (iii) higher throughput and slightly higher energy when using the throughput and energy balanced reward function. Although our discussion is focused on 802.11ac networks, our method is readily applicable to other types of wireless networks.
What Can the Neural Tangent Kernel Tell Us About Adversarial Robustness?

The adversarial vulnerability of neural nets, and subsequent techniques to create robust models have attracted significant attention; yet we still lack a full understanding of this phenomenon. Here, we study adversarial examples of trained neural networks through analytical tools afforded by recent theory advances connecting neural networks and kernel methods, namely the Neural Tangent Kernel (NTK), following a growing body of work that leverages the NTK approximation to successfully analyze important deep learning phenomena and design algorithms for new applications. We show how NTKs allow to generate adversarial examples in a ``training-free'' fashion, and demonstrate that they transfer to fool their finite-width neural net counterparts in the ``lazy'' regime. We leverage this connection to provide an alternative view on robust and non-robust features, which have been suggested to underlie the adversarial brittleness of neural nets. Specifically, we define and study features induced by the eigendecomposition of the kernel to better understand the role of robust and non-robust features, the reliance on both for standard classification and the robustness-accuracy trade-off. We find that such features are surprisingly consistent across architectures, and that robust features tend to correspond to the largest eigenvalues of the model, and thus are learned early during training. Our framework allows us to identify and visualize non-robust yet useful features. Finally, we shed light on the robustness mechanism underlying adversarial training of neural nets used in practice: quantifying the evolution of the associated empirical NTK, we demonstrate that its dynamics falls much earlier into the ``lazy'' regime and manifests a much stronger form of the well known bias to prioritize learning features within the top eigenspaces of the kernel, compared to standard training.
Improving Dense Contrastive Learning with Dense Negative Pairs

Many contrastive representation learning methods learn a single global representation of an entire image. However, dense contrastive representation learning methods such as DenseCL (Wang et al., 2021) can learn better representations for tasks requiring stronger spatial localization of features, such as multi-label classification, detection, and segmentation. In this work, we study how to improve the quality of the representations learned by DenseCL by modifying the training scheme and objective function, and propose DenseCL++. We also conduct several ablation studies to better understand the effects of: (i) various techniques to form dense negative pairs among augmentations of different images, (ii) cross-view dense negative and positive pairs, and (iii) an auxiliary reconstruction task. Our results show 3.5% and 4% mAP improvement over SimCLR (Chen et al., 2020a) andDenseCL in COCO multi-label classification. In COCO and VOC segmentation tasks, we achieve 1.8% and 0.7% mIoU improvements over SimCLR, respectively.
Phase I of the DiaVACCS screening trial: Study design, methods, population demographics and baseline results.

BACKGROUND Human papillomavirus (HPV)-based primary screening guidelines are based on screening test performance and prevalence data generated in high-resource areas with low HIV infection rates. There is an urgent need for local data on infection and disease prevalence, as well as screening test performance, among both HIV-positive and HIV-negative South African (SA) women, in order to inform updated screening guidelines. Objectives. This study describes the baseline characteristics of participants in the cross-sectional phase of the multicentric DIAgnosis in Vaccine And Cervical Cancer Screen (DiaVACCS) screening trial. The objective was to determine the prevalence of positive screening and pre-invasive disease using different tests and strategies in the SA HIV-positive and HIV-negative population. METHODS A total of 1 104 women aged 25 - 65 years and eligible for screening were included, 465 HIV positive and 639 HIV negative. Visual inspection and molecular and cytological screening tests were done on self-sampled and healthcare worker-collected specimens. All participants who screened positive and 49.1% of those who screened negative were invited for colposcopy and biopsy, and those qualifying for treatment were recalled for large loop excision of the transformation zone as part of the trial. The worst histology result for each participant was used, and for untested women, multiple imputation was used to estimate verification biasadjusted histology values. RESULTS Visual inspection was positive in 50.4% of HIV-positive v. 20.9% of HIV-negative women, cytology (atypical squamous cells of undetermined significance) in 39.9% v. 17.0%, and high-risk HPV DNA in 41.2% v. 19.6%. Overall, high-grade squamous intraepithelial lesion-positive cytology peaked in the age group 30 - 39 years at 16.7%. After adjustment for verification bias, histological diagnosis of cervical intraepithelial neoplasia (CIN)2+ was suspected in 44.7% v. 23.5% and CIN3+ in 23.3% v. 10.2% of HIV-positive and negative women, respectively. Invasive cancer was diagnosed in 15 women (1.95% of histological studies performed), and verification bias adjustment suggested 20 cases (1.8% of the study population). CONCLUSION The baseline findings from the DiaVACCS trial confirm a high prevalence of HPV-related cervical pathology in the SA HIV-negative screening population, showing a clear need to reach these women with a screening programme. Among HIV-positive women, prevalence values were almost doubled. The prevalence of existing invasive cervical cancer was 1 - 2% of all women. Further analysis of the performance of single and multiple screening tests between the two subgroups will contribute to the choice of the most effective strategies to identify women at risk of developing invasive cancer.
Functional characterization of MANNOSE-BINDING LECTIN 1, a G-type lectin gene family member, in response to fungal pathogens of strawberry

Abstract The mannose-binding lectin gene MANNOSE-BINDING LECTIN 1 (MBL1) is a member of the G-type lectin family and is involved in defense in strawberry (Fragaria × ananassa). Genome-wide identification of the G-type lectin family was carried out in woodland strawberry, F. vesca, and 133 G-lectin genes were found. Their expression profiles were retrieved from available databases and indicated that many are actively expressed during plant development or interaction with pathogens. We selected MBL1 for further investigation and generated stable transgenic FaMBL1-overexpressing plants of F. ×ananassa to examine the role of this gene in defense. Plants were selected and evaluated for their contents of disease-related phytohormones and their reaction to biotic stresses, and this revealed that jasmonic acid decreased in the overexpressing lines compared with the wild-type (WT). Petioles of the overexpressing lines inoculated with Colletotrichum fioriniae had lower disease incidence than the WT, and leaves of these lines challenged by Botrytis cinerea showed significantly smaller lesion diameters than the WT and higher expression of CLASS II CHITINASE 2-1. Our results indicate that FaMBL1 plays important roles in strawberry response to fungal diseases caused by C. fioriniae and B. cinerea.
Mechanisms underlying interoceptive exposure: belief disconfirmation or extinction? A preliminary study

ABSTRACT Interoceptive exposure, or exposure to one’s feared physical sensations, has been shown to be an important technique in cognitive behavioral therapies for anxiety disorders and related constructs, such as anxiety sensitivity (AS). The current study sought to further clarify the underlying cognitive-behavioral mechanisms of interoceptive exposure in a lab-based, analog study with individuals high in AS. Participants (n = 59) were randomized into three groups: a cognitive-behavioral intervention emphasizing belief disconfirmation (CbI), a behavioral intervention emphasizing exposure (BI), and a control condition. Self-report measures assessing AS, catastrophizing of bodily sensations, and subjective units of distress (SUDS) were collected before, during and after the intervention. Participants also completed online questionnaires at a one-month follow-up. Following the CbI but not BI, a decrease was observed in both AS and catastrophizing interpretations. Furthermore, only the CbI group exhibited a decrease in SUDS ratings, whereas the BI group exhibited a significant increase. Notably, these effects were not maintained at a one-month follow-up. Findings suggest that cognitive interventions without repeated behavioral exposure may be sufficient in reducing self-reported anxiety-related symptoms and catastrophic misinterpretations, though not at maintaining them. This raises questions regarding the role of pure behavioral mechanisms in exposure.
Assessment of poultry rearing practices and risk factors of H5N1 and H9N2 virus circulating among backyard chickens and ducks in rural communities

Background The avian influenza virus (AIV) causes significant economic losses by infecting poultry and occasional spillover to humans. Backyard farms are vulnerable to AIV epidemics due to poor health management and biosecurity practices, threatening rural households’ economic stability and nutrition. We have limited information about the risk factors associated with AIV infection in backyard poultry in Bangladesh. Hence, we conducted a cross-sectional survey comprising epidemiological and anthropological investigations to understand the poultry rearing practices and risk factors of AIV circulation among backyard poultry in selected rural communities. Methods We sampled 120 poultry from backyard farms (n = 30) of the three selected communities between February 2017 and January 2018. We tested swab samples for the matrix gene (M gene) followed by H5, H7, and H9 subtypes using real-time reverse transcriptase-polymerase chain reaction (rRT-PCR). We applied multivariable logistic regression for risk factor analysis. Furthermore, we conducted an observational study (42 hours) and informal interviews (n = 30) with backyard farmers to record poultry-raising activities in rural communities. Results We detected that 25.2% of the backyard poultry tested positive for AIV, whereas 5% tested positive for H5N1 and 10.8% tested positive for H9N2. Results showed that scavenging in both household garden and other crop fields has higher odds of AIV than scavenging in the household garden (AOR: 24.811; 95% CI: 2.11–292.28), and keeping a cage inside the house has higher odds (AOR:14.5; 95% CI: 1.06–198.51) than keeping it in the veranda, cleaning the cage twice a week or weekly has a higher risk than cleaning daily (AOR: 34.45; 95% CI: 1.04–1139.65), dumping litter or droppings (AOR: 82.80; 95% CI: 3.91–1754.59) and dead birds or wastage (AOR: 109.92, 95% CI: 4.34–2785.29) near water bodies and bushes have a higher risk than burring in the ground, slaughtering and consuming sick birds also had a higher odd of AIV (AOR: 73.45, 95% CI: 1.56–3457.73) than treating the birds. The anthropological investigation revealed that household members had direct contact with the poultry in different ways, including touching, feeding, slaughtering, and contacting poultry feces. Poultry is usually kept inside the house, sick poultry are traditionally slaughtered and eaten, and most poultry raisers do not know that diseases can transmit from backyard poultry to humans. Conclusions This study showed the circulation of H5N1 and H9N2 virus in backyard poultry in rural communities; associated with species, scavenging area of the poultry, location of the poultry cage, the practice of litter, wastage, droppings, and dead bird disposal, and practice of handling sick poultry. We suggest improving biosecurity practices in backyard poultry and mass awareness campaigns to reduce incidences of AIV in household-level poultry farms in rural communities in Bangladesh.
THE GROWTH OF DARK MATTER HALOS: EVIDENCE FOR SIGNIFICANT SMOOTH ACCRETION

We study the growth of dark matter halos in the concordance ΛCDM cosmology using several N-body simulations of large cosmological volumes. We construct merger trees from the Millennium and Millennium-II Simulations, covering the ranges 109–1015 M☉ in halo mass and 1–105 in merger mass ratio. Our algorithm takes special care of halo fragmentation and ensures that the mass contribution of each merger to halo growth is only counted once. This way the integrated merger rate converges and we can consistently determine the contribution of mergers of different mass ratios to halo growth. We find that all resolved mergers, up to mass ratios of 105:1, contribute only ≈60% of the total halo mass growth, while major mergers are subdominant, e.g., mergers with mass ratios smaller than 3:1 (10:1) contribute only ≈20% (≈30%). This is verified with an analysis of two additional simulation boxes, where we follow all particles individually throughout cosmic time. Our results are also robust against using several halo definitions. Under the assumption that the power-law behavior of the merger rate at large mass ratios can be extrapolated to arbitrarily large mass ratios, it is found that, independent of halo mass, ≈40% of the mass in halos comes from genuinely smooth accretion of dark matter that was never bound in smaller halos. We discuss possible implications of our findings for galaxy formation. One implication, assuming as is standard that the pristine intergalactic medium is heated and photoionized by UV photons, is that all halos accrete >40% of their baryons in smooth “cold” T ≳ 104 K gas, rather than as warm, enriched, or clumpy gas or as stars.
Efficacy and safety of topical treatments for seborrheic keratoses: a systematic review

Abstract Seborrheic keratoses (SKs) are benign epidermal neoplasms presenting as waxy, brown to black papules and plaques. Patients often seek removal for cosmetic reasons or irritation. The objective of this systematic review is to assess the efficacy and safety of topical treatments for SKs. Studies involving any topical medication indicated for SK removal were retrieved from Embase, Scopus, PubMed, and Cochrane. The final search was conducted on November 9, 2021, and 26 reports met inclusion criteria. A quality rating scheme was utilized to assess evidence quality. Heterogeneity of treatments and outcome measures precluded meta-analysis. Topical treatments that yielded a good-to-excellent response include hydrogen peroxide, Maxacalcitol 25 µg/g, BID Tazarotene 0.1% cream, 5% potassium dobesilate cream, 1% diclofenac sodium solution, urea-based solution, and 65% and 80% trichloroacetic acid. Local skin reactions were often mild and transient. Topical hydrogen peroxide showed the greatest evidence for clinical clearance of SKs, although there are no studies to our knowledge that directly compared hydrogen peroxide to current first-line treatments (e.g. cryotherapy or shave excision). The results of this review suggest viable and safe treatment of SK with topical therapies; however, there remains demand for topical treatments that reliably equate or exceed the efficacy of current first-line therapies. Key Points Question: Are safe and efficacious topical treatments for seborrheic keratoses available? Findings: Topical treatments for seborrheic keratoses yield different responses and may be associated with local skin reactions. Topical hydrogen peroxide shows the greatest evidence for clinical clearance of seborrheic keratoses and may be a viable option for patients requesting noninvasive removal. No studies to our knowledge directly compare hydrogen peroxide to current first-line treatments. Meaning: There remains demand for topical treatments that reliably equate or exceed the efficacy of current first-line therapies.