Miracle-dz/UniFAR-FTUs huggingface dataset:
Controlled molecule generator for optimizing multiple chemical properties

Generating a novel and optimized molecule with desired chemical properties is an essential part of the drug discovery process. Failure to meet one of the required properties can frequently lead to failure in a clinical test which is costly. In addition, optimizing these multiple properties is a challenging task because the optimization of one property is prone to changing other properties. In this paper, we pose this multi-property optimization problem as a sequence translation process and propose a new optimized molecule generator model based on the Transformer with two constraint networks: property prediction and similarity prediction. We further improve the model by incorporating score predictions from these constraint networks in a modified beam search algorithm. The experiments demonstrate that our proposed model, Controlled Molecule Generator (CMG), outperforms state-of-the-art models by a significant margin for optimizing multiple properties simultaneously.
Improved description of atomic environments using low-cost polynomial functions with compact support

The prediction of chemical properties using machine learning techniques calls for a set of appropriate descriptors that accurately describe atomic and, on a larger scale, molecular environments. A mapping of conformational information on a space spanned by atom-centred symmetry functions (SF) has become a standard technique for energy and force predictions using high-dimensional neural network potentials (HDNNP). An appropriate choice of SFs is particularly crucial for accurate force predictions. Established atom-centred SFs, however, are limited in their flexibility, since their functional form restricts the angular domain that can be sampled without introducing problematic derivative discontinuities. Here, we introduce a class of atom-centred SFs based on polynomials with compact support called polynomial symmetry functions (PSF), which enable a free choice of both, the angular and the radial domain covered. We demonstrate that the accuracy of PSFs is either on par or considerably better than that of conventional, atom-centred SFs. In particular, a generic set of PSFs with an intuitive choice of the angular domain inspired by organic chemistry considerably improves prediction accuracy for organic molecules in the gaseous and liquid phase, with reductions in force prediction errors over a test set approaching 50% for certain systems. Contrary to established atom-centred SFs, computation of PSF does not involve any exponentials, and their intrinsic compact support supersedes use of separate cutoff functions, facilitating the choice of their free parameters. Most importantly, the number of floating point operations required to compute polynomial SFs introduced here is considerably lower than that of other state-of-the-art SFs, enabling their efficient implementation without the need of highly optimised code structures or caching, with speedups with respect to other state-of-the-art SFs reaching a factor of 4.5 to 5. This low-effort performance benefit substantially simplifies their use in new programs and emerging platforms such as graphical processing units. Overall, polynomial SFs with compact support improve accuracy of both, energy and force predictions with HDNNPs while enabling significant speedups compared to their well-established counterparts.
VisualHints: A Visual-Lingual Environment for Multimodal Reinforcement Learning

We present VisualHints, a novel environment for multimodal reinforcement learning (RL) involving text-based interactions along with visual hints (obtained from the environment). Real-life problems often demand that agents interact with the environment using both natural language information and visual perception towards solving a goal. However, most traditional RL environments either solve pure vision-based tasks like Atari games or video-based robotic manipulation; or entirely use natural language as a mode of interaction, like Text-based games and dialog systems. In this work, we aim to bridge this gap and unify these two approaches in a single environment for multimodal RL. We introduce an extension of the TextWorld cooking environment with the addition of visual clues interspersed throughout the environment. The goal is to force an RL agent to use both text and visual features to predict natural language action commands for solving the final task of cooking a meal. We enable variations and difficulties in our environment to emulate various interactive real-world scenarios. We present a baseline multimodal agent for solving such problems using CNN-based feature extraction from visual hints and LSTMs for textual feature extraction. We believe that our proposed visual-lingual environment will facilitate novel problem settings for the RL community.
A Homotopic Method to Solve the Lasso Problems with an Improved Upper Bound of Convergence Rate

In optimization, it is known that when the objective functions are strictly convex and well-conditioned, gradient based approaches can be extremely effective, e.g., achieving the exponential rate in convergence. On the other hand, the existing Lasso-type of estimator in general cannot achieve the optimal rate due to the undesirable behavior of the absolute function at the origin. A homotopic method is to use a sequence of surrogate functions to approximate the $\ell_1$ penalty that is used in the Lasso-type of estimators. The surrogate functions will converge to the $\ell_1$ penalty in the Lasso estimator. At the same time, each surrogate function is strictly convex, which enables provable faster numerical rate of convergence. In this paper, we demonstrate that by meticulously defining the surrogate functions, one can prove faster numerical convergence rate than any existing methods in computing for the Lasso-type of estimators. Namely, the state-of-the-art algorithms can only guarantee $O(1/\epsilon)$ or $O(1/\sqrt{\epsilon})$ convergence rates, while we can prove an $O([\log(1/\epsilon)]^2)$ for the newly proposed algorithm. Our numerical simulations show that the new algorithm also performs better empirically.
Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models

The bias-variance trade-off is a central concept in supervised learning. In classical statistics, increasing the complexity of a model (e.g., number of parameters) reduces bias but also increases variance. Until recently, it was commonly believed that optimal performance is achieved at intermediate model complexities which strike a balance between bias and variance. Modern Deep Learning methods flout this dogma, achieving state-of-the-art performance using "over-parameterized models" where the number of fit parameters is large enough to perfectly fit the training data. As a result, understanding bias and variance in over-parameterized models has emerged as a fundamental problem in machine learning. Here, we use methods from statistical physics to derive analytic expressions for bias and variance in two minimal models of over-parameterization (linear regression and two-layer neural networks with nonlinear data distributions), allowing us to disentangle properties stemming from the model architecture and random sampling of data. In both models, increasing the number of fit parameters leads to a phase transition where the training error goes to zero and the test error diverges as a result of the variance (while the bias remains finite). Beyond this threshold, the test error of the two-layer neural network decreases due to a monotonic decrease in both the bias and variance in contrast with the classical bias-variance trade-off. We also show that in contrast with classical intuition, over-parameterized models can overfit even in the absence of noise and exhibit bias even if the student and teacher models match. We synthesize these results to construct a holistic understanding of generalization error and the bias-variance trade-off in over-parameterized models and relate our results to random matrix theory.
Deep generative factorization for speech signal

Various information factors are blended in speech signals, which forms the primary difficulty for most speech information processing tasks. An intuitive idea is to factorize speech signal into individual information factors (e.g., phonetic content and speaker trait), though it turns out to be highly challenging. This paper presents a speech factorization approach based on a novel factorial discriminative normalization flow model (factorial DNF). Experiments conducted on a two-factor case that involves phonetic content and speaker trait demonstrates that the proposed factorial DNF has powerful capability to factorize speech signals and outperforms several comparative models in terms of information representation and manipulation.
MarbleNet: Deep 1D Time-Channel Separable Convolutional Neural Network for Voice Activity Detection

We present MarbleNet, an end-to-end neural network for Voice Activity Detection (VAD). MarbleNet is a deep residual network composed from blocks of 1D time-channel separable convolution, batch-normalization, ReLU and dropout layers. When compared to a state-of-the-art VAD model, MarbleNet is able to achieve similar performance with roughly 1/10-th the parameter cost. We further conduct extensive ablation studies on different training methods and choices of parameters in order to study the robustness of MarbleNet in real-world VAD tasks.
Triple-view Convolutional Neural Networks for COVID-19 Diagnosis with Chest X-ray

The Coronavirus Disease 2019 (COVID-19) is affecting increasingly large number of people worldwide, posing significant stress to the health care systems. Early and accurate diagnosis of COVID-19 is critical in screening of infected patients and breaking the person-to-person transmission. Chest X-ray (CXR) based computer-aided diagnosis of COVID-19 using deep learning becomes a promising solution to this end. However, the diverse and various radiographic features of COVID-19 make it challenging, especially when considering each CXR scan typically only generates one single image. Data scarcity is another issue since collecting large-scale medical CXR data set could be difficult at present. Therefore, how to extract more informative and relevant features from the limited samples available becomes essential. To address these issues, unlike traditional methods processing each CXR image from a single view, this paper proposes triple-view convolutional neural networks for COVID-19 diagnosis with CXR images. Specifically, the proposed networks extract individual features from three views of each CXR image, i.e., the left lung view, the right lung view and the overall view, in three streams and then integrate them for joint diagnosis. The proposed network structure respects the anatomical structure of human lungs and is well aligned with clinical diagnosis of COVID-19 in practice. In addition, the labeling of the views does not require experts' domain knowledge, which is needed by many existing methods. The experimental results show that the proposed method achieves state-of-the-art performance, especially in the more challenging three class classification task, and admits wide generality and high flexibility.
A Members First Approach to Enabling LinkedIn's Labor Market Insights at Scale

We describe the privatization method used in reporting labor market insights from LinkedIn's Economic Graph, including the differentially private algorithms used to protect member's privacy. The reports show who are the top employers, as well as what are the top jobs and skills in a given country/region and industry. We hope this data will help governments and citizens track labor market trends during the COVID-19 pandemic while also protecting the privacy of our members.
Implications of the lowest frequency detection of the persistent counterpart of FRB121102

Context. The repeating FRB121102 is so far the only extra-galactic Fast Radio Burst found to be associated with a counterpart, a steady radio source with a nearly flat spectral energy distribution (SED) in centimeter wavelengths. Aims. Previous observations of the persistent source down to $1.6$~GHz has shown no sign of a spectral turn-over. Absorption is expected to eventually cause a turn-over at lower frequencies. Better constraints on the physical parameters of the emitting medium can be derived by detecting the self-absorption frequency. Methods. We used the Giant Metre-Wave Radio Telescope (GMRT) during the period of July to December 2017 to observe the source at low radio frequencies down to $400$~MHz. Results. The spectral energy distribution of the source remains optically thin even at $400$~MHz, with a spectral index of $\nu^{-(0.07 \pm 0.03)}$ similar to what is seen in Galactic plerions. Using a generic synchrotron radiation model, we obtain constraints on properties of the non-thermal plasma and the central engine powering it. Conclusions. We present low-frequency detections of the persistent source associated with FRB121102. Its characteristic flat SED extends down to $400$~MHz. Like Galactic plerions, the energy in the persistent source is carried predominantly by leptons. The emitting plasma has a $B 524 \left(\frac{B}{0.01 {\rm G}} \right)^{-3/2}$. We show that the energetics of the persistent source requires an initial spin period shorter than 36~ms, and the magnetic field of the neutron star must exceed $4.5\times 10^{12}$~G. This implies that the persistent source does not necessarily require energetic input from a magnetar.
Dynamics of Zeroes Under Repeated Differentiation

ABSTRACT Consider a random polynomial Pn of degree n whose roots are independent random variables sampled according to some probability distribution μ 0 on the complex plane . It is natural to conjecture that, for a fixed and as , the zeroes of the -th derivative of Pn are distributed according to some measure μt on . Assuming either that μ 0 is concentrated on the real line or that it is rotationally invariant, Steinerberger [Proc. AMS, 2019] and O’Rourke and Steinerberger [Proc. AMS, 2021] derived nonlocal transport equations for the density of roots. We introduce a different method to treat such problems. In the rotationally invariant case, we obtain a closed formula for , the asymptotic density of the radial parts of the roots of the -th derivative of Pn . Although its derivation is non-rigorous, we provide numerical evidence for its correctness and prove that it solves the PDE of O’Rourke and Steinerberger. Moreover, we present several examples in which the solution is fully explicit (including the special case in which the initial condition is an arbitrary convex combination of delta functions) and analyze some properties of the solutions such as the behavior of void annuli and circles of zeroes. As an additional support for the correctness of the method, we show that a similar method, applied to the case when μ 0 is concentrated on the real line, gives a correct result which is known to have an interpretation in terms of free probability.
Discovery of runs-of-homozygosity diplotype clusters and their associations with diseases in UK Biobank

Runs of homozygosity (ROH) segments, contiguous homozygous regions in a genome were traditionally linked to families and inbred populations. However, a growing literature suggests that ROHs are ubiquitous in outbred populations. Still, most existing genetic studies of ROH in populations are limited to aggregated ROH content across the genome, which does not offer the resolution for mapping causal loci. This limitation is mainly due to a lack of methods for efficient identification of shared ROH diplotypes. Here, we present a new method, ROH-DICE, to find large ROH diplotype clusters, sufficiently long ROHs shared by a sufficient number of individuals, in large cohorts. ROH-DICE identified over 1 million ROH diplotypes that span over 100 SNPs and shared by more than 100 UK Biobank participants. Moreover, we found significant associations of clustered ROH diplotypes across the genome with various self-reported diseases, with the strongest associations found between the extended HLA region and autoimmune disorders. We found an association between a diplotype covering the HFE gene and haemochromatosis, even though the well-known causal SNP was not directly genotyped nor imputed. Using genome-wide scan, we identified a putative association between carriers of an ROH diplotype in chromosome 4 and an increase of mortality among COVID-19 patients. In summary, our ROH-DICE method, by calling out large ROH diplotypes in a large outbred population, enables further population genetics into the demographic history of large populations. More importantly, our method enables a new genome-wide mapping approach for finding disease-causing loci with multi-marker recessive effects at population scale.
Synthetic superfluid chemistry with vortex-trapped quantum impurities

We explore the effect of using two-dimensional matter-wave vortices to confine an ensemble of bosonic quantum impurities. This is modelled theoretically using a mass-imbalanced homogeneous two component Gross-Pitaevskii equation. By changing the mass imbalance of our system we find the shape of the vortices are deformed even at modest imbalances, leading to `barrel' shaped vortices; which we quantify using a multi-component variational approach. The energy of impurity carrying vortex pairs are computed, revealing a mass-dependent energy splitting. We then compute the excited states of the impurity, which we in turn use to construct `covalent bonds' for vortex pairs, leading to the prediction of impurity mediated bound states. Our work opens a new route to simulating synthetic chemical reactions with superfluid systems.
A Probabilistic Representation of Deep Learning for Improving The Information Theoretic Interpretability

In this paper, we propose a probabilistic representation of MultiLayer Perceptrons (MLPs) to improve the information-theoretic interpretability. Above all, we demonstrate that the activations being i.i. d. is not valid for all the hidden layers of MLPs, thus the existing mutual information estimators based on non-parametric inference methods, e.g., empirical distributions and Kernel Density Estimate (KDE), are invalid for measuring the information flow in MLPs. Moreover, we introduce explicit probabilistic explanations for MLPs: (i) we define the probability space (Omega_F, t, P_F) for a fully connected layer f and demonstrate the great effect of an activation function on the probability measure P_F ; (ii) we prove the entire architecture of MLPs as a Gibbs distribution P; and (iii) the back-propagation aims to optimize the sample space Omega_F of all the fully connected layers of MLPs for learning an optimal Gibbs distribution P* to express the statistical connection between the input and the label. Based on the probabilistic explanations for MLPs, we improve the information-theoretic interpretability of MLPs in three aspects: (i) the random variable of f is discrete and the corresponding entropy is finite; (ii) the information bottleneck theory cannot correctly explain the information flow in MLPs if we take into account the back-propagation; and (iii) we propose novel information-theoretic explanations for the generalization of MLPs. Finally, we demonstrate the proposed probabilistic representation and information-theoretic explanations for MLPs in a synthetic dataset and benchmark datasets.
Behavior Priors for Efficient Reinforcement Learning

As we deploy reinforcement learning agents to solve increasingly challenging problems, methods that allow us to inject prior knowledge about the structure of the world and effective solution strategies becomes increasingly important. In this work we consider how information and architectural constraints can be combined with ideas from the probabilistic modeling literature to learn behavior priors that capture the common movement and interaction patterns that are shared across a set of related tasks or contexts. For example the day-to day behavior of humans comprises distinctive locomotion and manipulation patterns that recur across many different situations and goals. We discuss how such behavior patterns can be captured using probabilistic trajectory models and how these can be integrated effectively into reinforcement learning schemes, e.g.\ to facilitate multi-task and transfer learning. We then extend these ideas to latent variable models and consider a formulation to learn hierarchical priors that capture different aspects of the behavior in reusable modules. We discuss how such latent variable formulations connect to related work on hierarchical reinforcement learning (HRL) and mutual information and curiosity based objectives, thereby offering an alternative perspective on existing ideas. We demonstrate the effectiveness of our framework by applying it to a range of simulated continuous control domains.
A Multi-task Two-stream Spatiotemporal Convolutional Neural Network for Convective Storm Nowcasting

The goal of convective storm nowcasting is local prediction of severe and imminent convective storms. Here, we consider the convective storm nowcasting problem from the perspective of machine learning. First, we use a pixel-wise sampling method to construct spatiotemporal features for nowcasting, and flexibly adjust the proportions of positive and negative samples in the training set to mitigate class-imbalance issues. Second, we employ a concise two-stream convolutional neural network to extract spatial and temporal cues for nowcasting. This simplifies the network structure, reduces the training time requirement, and improves classification accuracy. The two-stream network used both radar and satellite data. In the resulting two-stream, fused convolutional neural network, some of the parameters are entered into a single-stream convolutional neural network, but it can learn the features of many data. Further, considering the relevance of classification and regression tasks, we develop a multi-task learning strategy that predicts the labels used in such tasks. We integrate two-stream multi-task learning into a single convolutional neural network. Given the compact architecture, this network is more efficient and easier to optimize than existing recurrent neural networks.
Confirming the Explosive Outflow in G5.89 with ALMA

The explosive molecular outflow detected decades ago in the Orion BN/KL region of massive star formation was considered to be a bizarre event. This belief was strengthened by the nondetection of similar cases over the years with the only exception of the marginal case of DR21. Here, we confirm a similar explosive outflow associated with the UCHII region G5.89−0.39 that indicates that this phenomenon is not unique to Orion or DR21. Sensitive and high angular resolution (∼0.″1) Atacama Large Millimeter/submillimeter Array (ALMA) CO(2−1) and SiO(5−4) observations show that the molecular outflow in the massive star-forming region G5.89−0.39 is indeed an explosive outflow with an age of about 1000 yr and a liberated kinetic energy of 1046–49 erg. Our new CO(2−1) ALMA observations revealed over 30 molecular filaments, with Hubble-like expansion motions, pointing to the center of UCHII region. In addition, the SiO(5−4) observations reveal warmer and strong shocks very close to the origin of the explosion, confirming the true nature of the flow. A simple estimation for the occurrence of these explosive events during the formation of the massive stars indicates an event rate of once every ∼100 yr, which is close to the supernovae rate.