Miracle-dz/UniFAR-FTUs huggingface dataset:
Interlocking Backpropagation: Improving depthwise model-parallelism

The number of parameters in state of the art neural networks has drastically increased in recent years. This surge of interest in large scale neural networks has motivated the development of new distributed training strategies enabling such models. One such strategy is model-parallel distributed training. Unfortunately, model-parallelism suffers from poor resource utilisation, which leads to wasted resources. In this work, we improve upon recent developments in an idealised model-parallel optimisation setting: local learning. Motivated by poor resource utilisation, we introduce a class of intermediary strategies between local and global learning referred to as interlocking backpropagation. These strategies preserve many of the compute-efficiency advantages of local optimisation, while recovering much of the task performance achieved by global optimisation. We assess our strategies on both image classification ResNets and Transformer language models, finding that our strategy consistently out-performs local learning in terms of task performance, and out-performs global learning in training efficiency.
Precise Task Formalization Matters in Winograd Schema Evaluations

Performance on the Winograd Schema Challenge (WSC), a respected English commonsense reasoning benchmark, recently rocketed from chance accuracy to 89% on the SuperGLUE leaderboard, with relatively little corroborating evidence of a correspondingly large improvement in reasoning ability. We hypothesize that much of this improvement comes from recent changes in task formalization---the combination of input specification, loss function, and reuse of pretrained parameters---by users of the dataset, rather than improvements in the pretrained model's reasoning ability. We perform an ablation on two Winograd Schema datasets that interpolates between the formalizations used before and after this surge, and find (i) framing the task as multiple choice improves performance by 2-6 points and (ii) several additional techniques, including the reuse of a pretrained language modeling head, can mitigate the model's extreme sensitivity to hyperparameters. We urge future benchmark creators to impose additional structure to minimize the impact of formalization decisions on reported results.
Adaptive Self-training for Few-shot Neural Sequence Labeling

Neural sequence labeling is an important technique employed for many Natural Language Processing (NLP) tasks, such as Named Entity Recognition (NER), slot tagging for dialog systems and semantic parsing. Large-scale pre-trained language models obtain very good performance on these tasks when fine-tuned on large amounts of task-specific labeled data. However, such large-scale labeled datasets are difficult to obtain for several tasks and domains due to the high cost of human annotation as well as privacy and data access constraints for sensitive user applications. This is exacerbated for sequence labeling tasks requiring such annotations at token-level. In this work, we develop techniques to address the label scarcity challenge for neural sequence labeling models. Specifically, we develop self-training and meta-learning techniques for few-shot training of neural sequence taggers, namely MetaST. While self-training serves as an effective mechanism to learn from large amounts of unlabeled data -- meta-learning helps in adaptive sample re-weighting to mitigate error propagation from noisy pseudo-labels. Extensive experiments on six benchmark datasets including two massive multilingual NER datasets and four slot tagging datasets for task-oriented dialog systems demonstrate the effectiveness of our method with around 10% improvement over state-of-the-art systems for the 10-shot setting.
A Step Toward Model Comparison: Connecting Electroweak-Scale Observables to BSM through EFT and Bayesian Statistics

Recognizing the potential of effective field theories to posit multiple BSM scenarios in similar footing, with a possibility to compare them, we inspect the effects of 11 single scalar-multiplet extensions of the SM on the combined set of electroweak precision observables and Higgs signal strength data, by systematically integrating out the heavy multiplets and computing the resulting SMEFT operators and Wilson coefficients (WCs) up to one-loop level. Noting that multiple BSM models give rise to a degenerate set of WCs, we then perform Bayesian statistical inference both directly on the BSM parameters and on the associated set of independent WCs. Using the posteriors of the BSM parameters, we infer the respective (correlated) WC-distributions and compare both the model-independent and dependent analyses by overlaying the 2-D marginal WC-posteriors from both processes, thus laying the ground for a data-driven attempt to compare diverse BSM theories of different origins, and hopefully, a possible way to approach the intractable inverse problem. We also demonstrate, with an example model, the crucial role of theoretical constraints to rule out large chunks of BSM parameter spaces. The entirety of numerical results is available in GitHub.
JWST Noise Floor. I. Random Error Sources in JWST NIRCam Time Series

James Webb Space Telescope (JWST) transmission and emission spectra will provide invaluable glimpses of transiting exoplanet atmospheres, including possible biosignatures. This promising science from JWST, however, will require exquisite precision and understanding of systematic errors that can impact the time series of planets crossing in front of and behind their host stars. Here, we provide estimates of the random noise sources affecting JWST Near-Infrared Camera (NIRCam) time-series data on the integration-to-integration level. We find that 1/f noise can limit the precision of grism time series for two groups (230–1000 ppm depending on the extraction method and extraction parameters), but will average down like the square root of N frames/reads. The current NIRCam grism time-series mode is especially affected by 1/f noise because its GRISMR dispersion direction is parallel to the detector fast-read direction, but could be alleviated in the GRISMC direction. Care should be taken to include as many frames as possible per visit to reduce this 1/f noise source: thus, we recommend the smallest detector subarray sizes one can tolerate, four output channels, and readout modes that minimize the number of skipped frames (RAPID or BRIGHT2). We also describe a covariance-weighting scheme that can significantly lower the contributions from 1/f noise as compared to sum extraction. We evaluate the noise introduced by pre-amplifier offsets, random telegraph noise, and high dark current resistor capacitor (RC) pixels and find that these are correctable below 10 ppm once background subtraction and pixel masking are performed. We explore systematic error sources in a companion paper.
Revisiting Generalized Two Higgs Doublet Model in the Light of Muon Anomaly and Lepton Flavor Violating Decays at HL-LHC

One of the main motivations to look beyond the SM is the discrepancy between the theoretical prediction and observation of anomalous magnetic moment of muon. Alleviating this tension between theory and experiment and satisfying the bounds from lepton flavor violation data simultaneously is a challenging task. In this paper, in view of generalised Two Higgs Doublet Model, we explore muon anomaly and lepton flavor violation along with constraints coming from B-physics, theoretical constraints, electroweak observables and collider data that can restrict the model parameter space significantly. We find that within the framework of generalised Two Higgs Doublet Model it is possible to obtain regions allowed by all constraints, that can provide an explanation for the observed muon anomaly and at the same time predicts interesting signatures of lepton flavor violation. Furthermore, we consider the flavor violating decay of low-mass CP-odd scalar to probe the allowed parameter space at future runs of the LHC. With simple cut-based analysis we show that part of that parameter space can be probed with significance $>5 \sigma$. We also provide Artificial Neural Network analysis which definitely improves our cut-based results significantly.
Discriminatively-Tuned Generative Classifiers for Robust Natural Language Inference

While discriminative neural network classifiers are generally preferred, recent work has shown advantages of generative classifiers in term of data efficiency and robustness. In this paper, we focus on natural language inference (NLI). We propose GenNLI, a generative classifier for NLI tasks, and empirically characterize its performance by comparing it to five baselines, including discriminative models and large-scale pretrained language representation models like BERT. We explore training objectives for discriminative fine-tuning of our generative classifiers, showing improvements over log loss fine-tuning from prior work . In particular, we find strong results with a simple unbounded modification to log loss, which we call the "infinilog loss". Our experiments show that GenNLI outperforms both discriminative and pretrained baselines across several challenging NLI experimental settings, including small training sets, imbalanced label distributions, and label noise.
Learning to Fuse Sentences with Transformers for Summarization

The ability to fuse sentences is highly attractive for summarization systems because it is an essential step to produce succinct abstracts. However, to date, summarizers can fail on fusing sentences. They tend to produce few summary sentences by fusion or generate incorrect fusions that lead the summary to fail to retain the original meaning. In this paper, we explore the ability of Transformers to fuse sentences and propose novel algorithms to enhance their ability to perform sentence fusion by leveraging the knowledge of points of correspondence between sentences. Through extensive experiments, we investigate the effects of different design choices on Transformer's performance. Our findings highlight the importance of modeling points of correspondence between sentences for effective sentence fusion.
Rapid and sensitive detection of SARS-CoV-2 with functionalized magnetic nanoparticles.

The outbreak of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) threatens global medical systems and economies, and rules our daily living life. Controlling the outbreak of SARS-CoV-2 has become one of the most important and urgent strategies throughout the whole world. As of October, 2020, there have not yet been any medicines or therapies to be effective against SARS-CoV-2. Thus, rapid and sensitive diagnostics is the most important measures to control the outbreak of SARS-CoV-2. Homogeneous biosensing based on magnetic nanoparticles (MNPs) is one of the most promising approaches for rapid and highly sensitive detection of biomolecules. This paper proposes an approach for rapid and sensitive detection of SARS-CoV-2 with functionalized MNPs via the measurement of their magnetic response in an ac magnetic field. Experimental results demonstrate that the proposed approach allows the rapid detection of mimic SARS-CoV-2 with a limit of detection of 0.084 nM (5.9 fmole). The proposed approach has great potential for designing a low-cost and point-of-care device for rapid and sensitive diagnostics of SARS-CoV-2.
Volumes of 5-free hyperbolic 3-manifolds.

Let $M$ be a closed, orientable hyperbolic $3$-manifold. We show that if $\pi_1(M)$ is $5$-free then ${\rm vol}\ M>3.77$. As an application, we show that if ${\rm vol}\ M\le3.77$ then ${\rm dim}\ H_1(M;{\bf F}_2)\le10$.
A Novel Derivation of Rigorous Macroscopic Limits from a Micro-Meso Description of Signal-Triggered Cell Migration in Fibrous Environments

In this work we upscale a prototypical kinetic transport equation which models a cell population moving in a fibrous environment with a chemo- or haptotactic signal influencing both the direction and the magnitude of the cell velocity. The presented approach to scaling does not rely on orthogonality and treats parabolic and hyperbolic scalings in a unified manner. It is shown that the steps of the formal limit procedures are mirrored by rigorous operations with finite measures provided that the measure-valued position-direction fiber distribution enjoys some spacial continuity.
A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks

Autoregressive language models pretrained on large corpora have been successful at solving downstream tasks, even with zero-shot usage. However, there is little theoretical justification for their success. This paper considers the following questions: (1) Why should learning the distribution of natural language help with downstream classification tasks? (2) Why do features learned using language modeling help solve downstream tasks with linear classifiers? For (1), we hypothesize, and verify empirically, that classification tasks of interest can be reformulated as next word prediction tasks, thus making language modeling a meaningful pretraining task. For (2), we analyze properties of the cross-entropy objective to show that $\epsilon$-optimal language models in cross-entropy (log-perplexity) learn features that are $\mathcal{O}(\sqrt{\epsilon})$-good on natural linear classification tasks, thus demonstrating mathematically that doing well on language modeling can be beneficial for downstream tasks. We perform experiments to verify assumptions and validate theoretical results. Our theoretical insights motivate a simple alternative to the cross-entropy objective that performs well on some linear classification tasks.
Partial wave correlations with uncertainties in the Two Nucleon System.

The NN scattering problem is usually analyzed in terms of partial waves and the corresponding coupled channel phase-shifts and mixing angles, but the available experiments induce correlations amongst the corresponding channels with different quantum numbers. Based on the Granada-2013 database we analyze the meaning and impact of those correlations taking into account both the purely statistical ones reflecting the primary experimental data uncertainties as well as the systematic ones exhibiting the ambiguities in the form of the potential representing the unknown nuclear force for distances below 3fm. We also ponder on the implications of fixing nuclear potentials by direct fits to the phase-shifts stemming from a full fledged partial wave analysis (PWA) inferred from experimental data.
Should egg size and clutch size decrease with age

We analyse a model in which females gather resources for eggs in a pre-reproductive phase, and are then, in the reproductive phase, subject to mortality during the production of successive clutches of offspring. We find that if clutch size is constrained, egg size should decline with maternal age. Where egg fitness shows diminishing returns with egg size, and maternal mortality is age-independent, the decline in egg size will be linear, and the rate of decline will be proportional to maternal mortality. All females with the same survival prospects follow the same trajectory of decline, but larger females enter the trajectory at a larger egg size. Similarly, if egg size is constrained, clutch size declines with maternal age, the rate of decline increases with maternal mortality, and larger females lay larger clutches initially. Support for the models is found in a wide variety of species, and in particular in a number of recent studies of butterflies.
Classification of Speech with and without Face Mask using Acoustic Features

The understanding and interpretation of speech can be affected by various external factors. The use of face masks is one such factors that can create obstruction to speech while communicating. This may lead to degradation of speech processing and affect humans perceptually. Knowing whether a speaker wears a mask may be useful for modeling speech for different applications. With this motivation, finding whether a speaker wears face mask from a given speech is included as a task in Computational Paralinguistics Evaluation (ComParE) 2020. We study novel acoustic features based on linear filterbanks, instantaneous phase and long-term information that can capture the artifacts for classification of speech with and without face mask. These acoustic features are used along with the state-of-the-art baselines of ComParE functionals, bag-of-audio-words, DeepSpectrum and auDeep features for ComParE 2020. The studies reveal the effectiveness of acoustic features, and their score level fusion with the ComParE 2020 baselines leads to an unweighted average recall of 73.50% on the test set.
5G Network Slice Isolation with WireGuard and Open Source MANO: A VPNaaS Proof-of-Concept

The fifth-generation (5G) mobile networks aim to host different types of services on the same physical infrastructure. Network slicing is considered as the key enabler for achieving this goal. Although there is some progress in applying and implementing network slicing in the context of 5G, the security and performance of network slicing still have many open research questions. In this paper, we propose the first OSM-WireGuard framework and its lifecycle. We implement the WireGuard secure network tunneling protocol in a 5G network to provide a VPN-as-a-Service (VPNaaS) functionality for virtualized network functions. We demonstrate that OSM instantiates WireGuard-enabled services up and running in 4 min 26 sec, with potential the initialization time to go down to 2 min 44 sec if the operator prepares images with a pre-installed and up-to-date version of WireGuard before the on-boarding process. We also show that the OSM-WireGuard framework provides considerable enhancement of up to 5.3 times higher network throughput and up to 41% lower latency compared to OpenVPN. The reported results show that the proposed framework is a promising solution for providing traffic isolation with strict latency and throughput requirements.
Potencial antibacteriano de fungos endofíticos de cactos da Caatinga, uma floresta tropical seca no Nordeste do Brasil

O potencial biotecnologico de fungos endofiticos tem sido amplamente verificado em todo o mundo. Este estudo teve como objetivo verificar o potencial antibacteriano de fungos endofiticos isolados dos cactos Cereus jamacaru DC. subsp. jamacaru, Opuntia ficus-indica (L.) Mill. e Pilosocereus gounellei (F.A.C. Weber) Byles & G.D. Rowley subsp. gounellei da Floresta Tropical Seca brasileira (Caatinga) contra bacterias fitopatogenicas e patogenicas ao homem. O teste foi realizado com 60 endofitos e 21 (35%) demonstraram atividade contra pelo menos uma bacteria patogenica ao homem, com destaque dos endofitos isolados de C. jamacaru subsp. jamacaru. Trichoderma longibrachiatum Rifai inibiu sete das 10 bacterias utilizadas, seguido por Gibberella fujikuroi (Sawada) Wollenw. (6 bacterias) e Lecythophora decumbens (J.F.H. Beyma) E. Weber, Gorke & Begerow (5 bacterias) com halos variando de 5-30 mm. Micrococcus luteus (UFPEDA100) foi inibida por 11 dos 2 endofitos testados. Os endofitos nao apresentaram atividade no teste com bacterias fitopatogenicas e com o extrato bruto. Endofitos isolados de cactos crescendo na Caatinga possuem potencial para producao de compostos com potencial antibacteriano.
INVITED REVIEW SERIES: RESPIRATORY SLEEP DISORDERS

Asthma, chronic obstructive pulmonary disease (COPD) and obstructive sleep apnoea (OSA) are the most common respiratory disorders worldwide. Given demographic and environmental changes, prevalence for each is likely to increase. Although exact numbers are not known, based on chance alone, many people will be affected by both lower airways obstruction and concomitant upper airway obstruction during sleep. Some recent studies suggest that there is a reciprocal interaction, with chronic lung disease predisposing to OSA, and OSA worsening control and outcomes from chronic lung disease. Thus, the combination of wake and sleep respiratory disorders can create an overlap syndrome with unique pathophysiological, diagnostic and therapeutic concerns. Although much work needs to be done, given the above, Respirologists, Sleep Medicine and Primary Care providers must be vigilant for overlap syndromes. Accurate diagnosis of, for example, OSA as a cause of nocturnal symptoms in a patient with asthma is likely to limit further ineffective titration of medications for asthma. Moreover, prompt treatment of OSA in the overlap syndromes will not only offer symptomatic benefit of OSA, but also improve symptoms and healthcare resource utilization attributable to obstructive lung disease, and in COPD, it may reduce mortality.
RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs

This paper studies learning logic rules for reasoning on knowledge graphs. Logic rules provide interpretable explanations when used for prediction as well as being able to generalize to other tasks, and hence are critical to learn. Existing methods either suffer from the problem of searching in a large search space (e.g., neural logic programming) or ineffective optimization due to sparse rewards (e.g., techniques based on reinforcement learning). To address these limitations, this paper proposes a probabilistic model called RNNLogic. RNNLogic treats logic rules as a latent variable, and simultaneously trains a rule generator as well as a reasoning predictor with logic rules. We develop an EM-based algorithm for optimization. In each iteration, the reasoning predictor is first updated to explore some generated logic rules for reasoning. Then in the E-step, we select a set of high-quality rules from all generated rules with both the rule generator and reasoning predictor via posterior inference; and in the M-step, the rule generator is updated with the rules selected in the E-step. Experiments on four datasets prove the effectiveness of RNNLogic.
Improve Adversarial Robustness via Weight Penalization on Classification Layer

It is well-known that deep neural networks are vulnerable to adversarial attacks. Recent studies show that well-designed classification parts can lead to better robustness. However, there is still much space for improvement along this line. In this paper, we first prove that, from a geometric point of view, the robustness of a neural network is equivalent to some angular margin condition of the classifier weights. We then explain why ReLU type function is not a good choice for activation under this framework. These findings reveal the limitations of the existing approaches and lead us to develop a novel light-weight-penalized defensive method, which is simple and has a good scalability. Empirical results on multiple benchmark datasets demonstrate that our method can effectively improve the robustness of the network without requiring too much additional computation, while maintaining a high classification precision for clean data.
Generating Instructions at Different Levels of Abstraction

When generating technical instructions, it is often convenient to describe complex objects in the world at different levels of abstraction. A novice user might need an object explained piece by piece, while for an expert, talking about the complex object (e.g. a wall or railing) directly may be more succinct and efficient. We show how to generate building instructions at different levels of abstraction in Minecraft. We introduce the use of hierarchical planning to this end, a method from AI planning which can capture the structure of complex objects neatly. A crowdsourcing evaluation shows that the choice of abstraction level matters to users, and that an abstraction strategy which balances low-level and high-level object descriptions compares favorably to ones which don’t.