Miracle-dz/UniFAR-FTUs huggingface dataset:
Robust Semi-Supervised Learning with Out of Distribution Data

Semi-supervised learning (SSL) based on deep neural networks (DNNs) has recently been proven effective. However, recent work [Oliver et al., 2018] shows that the performance of SSL could degrade substantially when the unlabeled set has out-of-distribution examples (OODs). In this work, we first study the key causes about the negative impact of OOD on SSL. We found that (1) OODs close to the decision boundary have a larger effect on the performance of existing SSL algorithms than the OODs far away from the decision boundary and (2) Batch Normalization (BN), a popular module in deep networks, could degrade the performance of a DNN for SSL substantially when the unlabeled set contains OODs. To address these causes, we proposed a novel unified robust SSL approach for many existing SSL algorithms in order to improve their robustness against OODs. In particular, we proposed a simple modification to batch normalization, called weighted batch normalization, capable of improving the robustness of BN against OODs. We developed two efficient hyperparameter optimization algorithms that have different tradeoffs in computational efficiency and accuracy. The first is meta-approximation and the second is implicit-differentiation based approximation. Both algorithms learn to reweight the unlabeled samples in order to improve the robustness of SSL against OODs. Extensive experiments on both synthetic and real-world datasets demonstrate that our proposed approach significantly improves the robustness of four representative SSL algorithms against OODs, in comparison with four state-of-the-art robust SSL approaches. We performed an ablation study to demonstrate which components of our approach are most important for its success.
More insights into bar quenching

The underlying nature of the process of star formation quenching in the central regions of barred disc galaxies that is due to the action of stellar bar is not fully understood. We present a multi-wavelength study of four barred galaxies using the archival data from optical, ultraviolet, infrared, CO, and HI imaging data on star formation progression and stellar and gas distribution to better understand the process of bar quenching. We found that for three galaxies, the region between the nuclear or central sub-kiloparsec region and the end of the bar (bar region) is devoid of neutral and molecular hydrogen. While the detected neutral hydrogen is very negligible, we note that molecular hydrogen is present abundantly in the nuclear or central sub-kiloparsec regions of all four galaxies. The bar co-rotation radius is also devoid of recent star formation for three out of four galaxies. One galaxy shows significant molecular hydrogen along the bar, which might mean that the gas is still being funnelled to the centre by the action of the stellar bar. Significant star formation is also present along the bar co-rotation radius of this galaxy. The study presented here supports a scenario in which gas redistribution as a result of the action of stellar bar clears the bar region of fuel for further star formation and eventually leads to star formation quenching in the bar region.
Density-body size relationships in local aquatic communities

Studies of density-body size relationships report generalities in the structure of communities and suggest patterns of energy use among species of different sizes. Most density-body size relationships, however, have been developed from data on populations throughout the world and are expected, therefore, to provide only an approximate description of any specific community. In this study, we test 1) whether log-linear density-body size relationships can also be used to describe local communities and 2) whether these relationships vary systematically with environmental conditions. Our analysis is based on population densities of phytoplankton, zooplankton, zoobenthos and fish species measured in 18 lakes worldwide. Mean annual population density (D, individuals m -2 ) decreases log-linearly with increasing species body size (M, μg fresh mass; log D = a + b x log M), with slopes and intercepts which vary among lakes (b = -1.10 to -0.74, a = 4.5 to 6.9). In contrast with studies which focus on only one group of organisms (e.g. birds, insects, mammals), found that local communities are well described by density-body size relationships (r 2 = 0.78-0.98) when more complete communities are considered. The slopes of density-body size relationships measure the relative density of small to large species, and tend to be shallower (lower relative density of small species) in lakes near cities, but more steeply negative (greater relative density of small species) in deep, highly productive lakes which flush rapidly. The intercepts measure average population density and are positively related to primary productivity. Such differences in size structure among aquatic communities are large enough to suggest shifts, for example, in the relative energy use of small and large species.
Detecting Anomalous Graphs in Labeled Multi-Graph Databases

Within a large database 𝒢 containing graphs with labeled nodes and directed, multi-edges; how can we detect the anomalous graphs? Most existing work are designed for plain (unlabeled) and/or simple (unweighted) graphs. We introduce CODEtect, the first approach that addresses the anomaly detection task for graph databases with such complex nature. To this end, it identifies a small representative set 𝒮 of structural patterns (i.e., node-labeled network motifs) that losslessly compress database 𝒢 as concisely as possible. Graphs that do not compress well are flagged as anomalous. CODEtect exhibits two novel building blocks: (i) a motif-based lossless graph encoding scheme, and (ii) fast memory-efficient search algorithms for 𝒮. We show the effectiveness of CODEtect on transaction graph databases from three different corporations and statistically similar synthetic datasets, where existing baselines adjusted for the task fall behind significantly, across different types of anomalies and performance metrics.
Two Are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders

Named entity recognition and relation extraction are two important fundamental problems. Joint learning algorithms have been proposed to solve both tasks simultaneously, and many of them cast the joint task as a table-filling problem. However, they typically focused on learning a single encoder (usually learning representation in the form of a table) to capture information required for both tasks within the same space. We argue that it can be beneficial to design two distinct encoders to capture such two different types of information in the learning process. In this work, we propose the novel {\em table-sequence encoders} where two different encoders -- a table encoder and a sequence encoder are designed to help each other in the representation learning process. Our experiments confirm the advantages of having {\em two} encoders over {\em one} encoder. On several standard datasets, our model shows significant improvements over existing approaches.
Two-epoch Orbit Estimation for Wide Binaries Resolved in Hipparcos and Gaia

The Hipparcos catalog and its Double and Multiple System Annex (DMSA) lists 4099 components with individual proper motions and coordinates on the epoch 1991.25. Many of these long-period binary stars are also present in Gaia Data Release 2 (DR2). Using the available relative positions and proper motions separated by 24.25 yr, the equations of relative orbital motion can be solved for the two epoch eccentric anomalies, orbital period, and eccentricity. This method employs elimination of the linear Thiele–Innes unknowns and nonlinear optimization of the remaining condition equations. The quality of these solutions is compromised by the insufficient condition and modest precision of the Hipparcos astrometric data, as revealed by Monte Carlo simulations with artificially perturbed data points. The presence of multiple systems and optical pairs can also perturb the results. Limited experiments with artificial data indicate that useful estimates can be obtained with a 25 yr epoch difference for wide binaries with orbital periods up to ∼500 yr. The prospects of this method dramatically improve with the proposed next-generation space astrometry missions such as Gaia-NIR and Theia, especially when additional conditions are included from astrometric or spectroscopic measurements. An ancillary catalog of cross-identification and astrometric information for 1295 double-star pairs cross-matched in Gaia DR2 and Hipparcos is also published.
Adaptive Subcarrier, Parameter, and Power Allocation for Partitioned Edge Learning Over Broadband Channels

In this paper, we consider partitioned edge learning (PARTEL), which implements parameter-server training, a well known distributed learning method, in a wireless network. Thereby, PARTEL leverages distributed computation resources at edge devices to train a large-scale artificial intelligence (AI) model by dynamically partitioning the model into parametric blocks for separated updating at devices. Targeting broadband channels, we consider the joint control of parameter allocation, sub-channel allocation, and transmission power to improve the performance of PARTEL. Specifically, the policies for joint SUbcarrier, Parameter, and POweR allocaTion (SUPPORT) are optimized under the criterion of minimum learning latency. Two cases are considered. First, for the case of decomposable models (e.g., logistic regression), the latency-minimization problem is a mixed-integer program and non-convex. Due to its intractability, we develop a practical solution by integer relaxation and transforming it into an equivalent convex problem of model size maximization under a latency constraint. Thereby, a low-complexity algorithm is designed to compute the SUPPORT policy. Second, consider the case of deep neural network (DNN) models which can be trained using PARTEL by introducing some auxiliary variables. This, however, introduces constraints on model partitioning reducing the granularity of parameter allocation. The preceding policy is extended to DNN models by applying the proposed techniques of load rounding and proportional adjustment to rein in latency expansion caused by the load granularity constraints.
Tatum-Level Drum Transcription Based on a Convolutional Recurrent Neural Network with Language Model-Based Regularized Training

This paper describes a neural drum transcription method that detects from music signals the onset times of drums at the tatum level, where tatum times are assumed to be estimated in advance. In conventional studies on drum transcription, deep neural networks (DNNs) have often been used to take a music spectrogram as input and estimate the onset times of drums at the frame level. The major problem with such frame-to-frame DNNs, however, is that the estimated onset times do not often conform with the typical tatum-level patterns appearing in symbolic drum scores because the long-term musically meaningful structures of those patterns are difficult to learn at the frame level. To solve this problem, we propose a regularized training method for a frame-to-tatum DNN. In the proposed method, a tatum-level probabilistic language model (gated recurrent unit (GRU) network or repetition-aware bi-gram model) is trained from an extensive collection of drum scores. Given that the musical naturalness of tatum-level onset times can be evaluated by the language model, the frame-to-tatum DNN is trained with a regularizer based on the pretrained language model. The experimental results demonstrate the effectiveness of the proposed regularized training method.
Krylov-subspace approach for the efficient control of quantum many-body dynamics

The Gradient Ascent Pulse Engineering (GRAPE) is a celebrated control algorithm with excellent converging rates, owing to a piece-wise-constant ansatz for the control function that allows for cheap objective gradients. However, the computational effort involved in the exact simulation of quantum dynamics quickly becomes a bottleneck limiting the control of large systems. In this paper, we propose a modified version of GRAPE that uses Krylov approximations to deal efficiently with high-dimensional state spaces. Even though the number of parameters required by an arbitrary control task scales linearly with the dimension of the system, we find a constant elementary computational effort (the effort per parameter). Since the elementary effort of GRAPE is super-quadratic, this speed up allows us to reach dimensions far beyond. The performance of the K-GRAPE algorithm is benchmarked in the paradigmatic XXZ spin-chain model.
Test-Cost Sensitive Methods for Identifying Nearby Points

Real-world applications that involve missing values are often constrained by the cost to obtain data. Test-cost sensitive, or costly feature, methods additionally consider the cost of acquiring features. Such methods have been extensively studied in the problem of classification. In this paper, we study a related problem of test-cost sensitive methods to identify nearby points from a large set, given a new point with some unknown feature values. We present two models, one based on a tree and another based on Deep Reinforcement Learning. In our simulations, we show that the models outperform random agents on a set of five real-world data sets.
Quantitative predictions of photo-emission dynamics in metal halide perovskites via machine learning

Metal halide perovskite (MHP) optoelectronics may become a viable alternative to standard Si-based technologies, but the current lack of long-term stability precludes their commercial adoption. Exposure to standard operational stressors (light, temperature, bias, oxygen, and water) often instigate optical and electronic dynamics, calling for a systematic investigation into MHP photophysical processes and the development of quantitative models for their prediction. We resolve the moisture-driven light emission dynamics for both methylammonium lead tribromide and triiodide thin films as a function of relative humidity (rH). With the humidity and photoluminescence time series, we train recurrent neural networks and establish their ability to quantitatively predict the path of future light emission with <11% error over 12 hours. Together, our in situ rH-PL measurements and machine learning forecasting models provide a framework for the rational design of future stable perovskite devices and, thus, a faster transition towards commercial applications.
Theoretical and Experimental Comparison of Off-Policy Evaluation from Dependent Samples

We theoretically and experimentally compare estimators for off-policy evaluation (OPE) using dependent samples obtained via multi-armed bandit (MAB) algorithms. The goal of OPE is to evaluate a new policy using historical data. Because the MAB algorithms sequentially update the policy based on past observations, the generated samples are not independent and identically distributed. To conduct OPE from dependent samples, we need to use some techniques for constructing the estimator with asymptotic normality. In particular, we focus on a doubly robust (DR) estimator, which consists of an inverse probability weighting (IPW) component and an estimator of the conditionally expected outcome. We first summarize existing and new theoretical results for such OPE estimators. Then, we compare their empirical properties using benchmark datasets with other estimators, such as an estimator with cross-fitting.
Magnetostriction in elastomers with mixtures of magnetically hard and soft microparticles: effects of nonlinear magnetization and matrix rigidity

Abstract In this contribution, a magnetoactive elastomer (MAE) of mixed content, i.e., a polymer matrix filled with a mixture of magnetically soft and magnetically hard spherical particles, is considered. The object we focus on is an elementary unit of this composite, for which we take a set consisting of a permanent spherical micromagnet surrounded by an elastomer layer filled with magnetically soft microparticles. We present a comparative treatment of this unit from two essentially different viewpoints. The first one is a coarse-grained molecular dynamics simulation model, which presents the composite as a bead-spring assembly and is able to deliver information of all the microstructural changes of the assembly. The second approach is entirely based on the continuum magnetomechanical description of the system, whose direct yield is the macroscopic field-induced response of the MAE to external field, as this model ignores all the microstructural details of the magnetization process. We find that, differing in certain details, both frameworks are coherent in predicting that a unit comprising magnetically soft and hard particles may display a nontrivial reentrant (prolate/oblate/prolate) axial deformation under variation of the applied field strength. The flexibility of the proposed combination of the two complementary frameworks enables us to look deeper into the manifestation of the magnetic response: with respect to the magnetically soft particles, we compare the linear regime of magnetization to that with saturation, which we describe by the Fröhlich–Kennelly approximation; with respect to the polymer matrix, we analyze the dependence of the reentrant deformation on its rigidity.
High Definition image classification in Geoscience using Machine Learning

High Definition (HD) digital photos taken with drones are widely used in the study of Geoscience. However, blurry images are often taken in collected data, and it takes a lot of time and effort to distinguish clear images from blurry ones. In this work, we apply Machine learning techniques, such as Support Vector Machine (SVM) and Neural Network (NN) to classify HD images in Geoscience as clear and blurry, and therefore automate data cleaning in Geoscience. We compare the results of classification based on features abstracted from several mathematical models. Some of the implementation of our machine learning tool is freely available at: this https URL.
Frames of group-sets and their application in bundle theory.

We study fiber bundles where the fibers are not a group $G$, but a free $G$-space with disjoint orbits. These bundles closely resemble principal bundles, hence we call them semi-principal bundles. The study of such bundles is facilitated by defining the notion of a basis of a $G$-set, in analogy with a basis of a vector space. The symmetry group of these bases is a wreath product. Similar to vector bundles, using the notion of a basis induces a frame bundle construction, which in this case results in a principal bundle with the wreath product as structure group. This construction can be formalized in the language of a functor, which retracts the semi-principal bundles to the principal bundles. In addition, semi-principal bundles support parallel transport just like principal bundles, and this carries over to the frame bundle. This theory could provide a mathematical framework for particular gauge theories.
Scattering from rough thin films: DDA-simulations

We investigate the wave-optical light scattering properties of deformed thin circular films of constant thickness using the discrete-dipole approximation. Effects on the intensity distribution of the scattered light due to different statistical roughness models, model dependant roughness parameters, and uncorrelated random small-scale porosity of the inhomogeneous medium are studied. The usability of discrete-dipole approximation to rough-surface scattering problems is evaluated by considering thin films as computationally feasible rough-surface analogs. The effects due to small-scale inhomogeneity of the scattering medium are compared with the analytic approximation by Maxwell Garnett and the results are found to agree with the approximation. © 2020 Optical Society of America
A Variational Auto-Encoder Approach for Image Transmission in Wireless Channel

Recent advancements in information technology and the widespread use of the Internet have led to easier access to data worldwide. As a result, transmitting data through noisy channels is inevitable. Reducing the size of data and protecting it during transmission from corruption due to channel noises are two classical problems in communication and information theory. Recently, inspired by deep neural networks' success in different tasks, many works have been done to address these two problems using deep learning techniques. In this paper, we investigate the performance of variational auto-encoders and compare the results with standard auto-encoders. Our findings suggest that variational auto-encoders are more robust to channel degradation than auto-encoders. Furthermore, we have tried to excel in the human perceptual quality of reconstructed images by using perception-based error metrics as our network's loss function. To this end, we use the structural similarity index (SSIM) as a perception-based metric to optimize the proposed neural network. Our experiments demonstrate that the SSIM metric visually improves the quality of the reconstructed images at the receiver.
Studying the UK job market during the COVID-19 crisis with online job ads

The COVID-19 global pandemic and the lockdown policies enacted to mitigate it have had profound effects on the labour market. Understanding these effects requires us to obtain and analyse data in as close to real time as possible, especially as rules change rapidly and local lockdowns are enacted. This work studies the UK labour market by analysing data from the online job board Reed.co.uk, using topic modelling and geo-inference methods to break down the data by sector and geography. I also study how the salary, contract type, and mode of work have changed since the COVID-19 crisis hit the UK in March. Overall, vacancies were down by 60 to 70% in the first weeks of lockdown. By the end of the year numbers had recovered somewhat, but the total job ad deficit is measured to be over 40%. Broken down by sector, vacancies for hospitality and graduate jobs are greatly reduced, while there were more care work and nursing vacancies during lockdown. Differences by geography are less significant than between sectors, though there is some indication that local lockdowns stall recovery and less badly hit areas may have experienced a smaller reduction in vacancies. There are also small but significant changes in the salary distribution and number of full time and permanent jobs. As well as the analysis, this work presents an open methodology that enables a rapid and detailed survey of the job market in unsettled conditions and describes a web application jobtrender.com that allows others to query this data set.
Towards blood flow in the virtual human: efficient self-coupling of HemeLB

Many scientific and medical researchers are working towards the creation of a virtual human—a personalized digital copy of an individual—that will assist in a patient’s diagnosis, treatment and recovery. The complex nature of living systems means that the development of this remains a major challenge. We describe progress in enabling the HemeLB lattice Boltzmann code to simulate 3D macroscopic blood flow on a full human scale. Significant developments in memory management and load balancing allow near linear scaling performance of the code on hundreds of thousands of computer cores. Integral to the construction of a virtual human, we also outline the implementation of a self-coupling strategy for HemeLB. This allows simultaneous simulation of arterial and venous vascular trees based on human-specific geometries.