Miracle-dz/UniFAR-FTUs huggingface dataset:
Soft-Root-Sign Activation Function

The choice of activation function in deep networks has a significant effect on the training dynamics and task performance. At present, the most effective and widely-used activation function is ReLU. However, because of the non-zero mean, negative missing and unbounded output, ReLU is at a potential disadvantage during optimization. To this end, we introduce a novel activation function to manage to overcome the above three challenges. The proposed nonlinearity, namely "Soft-Root-Sign" (SRS), is smooth, non-monotonic, and bounded. Notably, the bounded property of SRS distinguishes itself from most state-of-the-art activation functions. In contrast to ReLU, SRS can adaptively adjust the output by a pair of independent trainable parameters to capture negative information and provide zero-mean property, which leading not only to better generalization performance, but also to faster learning speed. It also avoids and rectifies the output distribution to be scattered in the non-negative real number space, making it more compatible with batch normalization (BN) and less sensitive to initialization. In experiments, we evaluated SRS on deep networks applied to a variety of tasks, including image classification, machine translation and generative modelling. Our SRS matches or exceeds models with ReLU and other state-of-the-art nonlinearities, showing that the proposed activation function is generalized and can achieve high performance across tasks. Ablation study further verified the compatibility with BN and self-adaptability for different initialization.
Estimating the impact of treatment compliance over time on smoking cessation using data from ecological momentary assessments (EMA)

The Wisconsin Smoker's Health Study (WSHS2) was a longitudinal trial conducted to compare the effectiveness of two commonly used smoking cessation treatments, varenicline and combination nicotine replacement therapy (cNRT) with the less intense standard of care, nicotine patch. The main outcome of the WSHS2 study was that all three treatments had equivalent treatment effects. However, in-depth analysis of the compliance data collected via ecological momentary assessment (EMA) were not analyzed. Compliance to the treatment regimens may represent a confounder as varenicline and cNRT are more intense treatments and would likely have larger treatment effects if all subjects complied. In order to estimate the causal compliance effect, we view the counterfactual, the outcome that would have been observed if the subject was allocated to the treatment counter to the fact, as a missing data problem and proceed to impute the counterfactual. Our contribution to the methodological literature lies in the extension of this idea to a more general analytic approach that includes mediators and confounders of the mediator-outcome relationship. Simulation results suggest that our method works well and application to the WSHS2 data suggest that the treatment effect of nicotine patch, varenicline, and cNRT are equivalent after accounting for differences in treatment compliance.
Stability of Kerr black holes in generalized hybrid metric-Palatini gravity

It is shown that the Kerr solution exists in the generalized hybrid metric-Palatini gravity theory and that for certain choices of the function $f(R,\mathcal R)$ that characterizes the theory, the Kerr solution can be stable against perturbations on the scalar degree of freedom of the theory. We start by verifying which are the most general conditions on the function $f(R,\mathcal R)$ that allow for the general relativistic Kerr solution to also be a solution of this theory. We perform a scalar perturbation in the trace of the metric tensor, which in turn imposes a perturbation in both the Ricci and Palatini scalar curvatures. To first order in the perturbation, the equations of motion, namely the field equations and the equation that relates the Ricci and the Palatini curvature scalars, can be rewritten in terms of a fourth-order wave equation for the perturbation $\delta R$ which can be factorized into two second-order massive wave equations for the same variable. The usual ansatz and separation methods are applied and stability bounds on the effective mass of the Ricci scalar perturbation are obtained. These stability regimes are studied case by case and specific forms of the function $f(R,\mathcal R)$ that allow for a stable Kerr solution to exist within the perturbation regime studied are obtained.
Out-of-Distribution Generalization via Risk Extrapolation (REx)

Generalizing outside of the training distribution is an open challenge for current machine learning systems. A weak form of out-of-distribution (OoD) generalization is the ability to successfully interpolate between multiple observed distributions. One way to achieve this is through robust optimization, which seeks to minimize the worst-case risk over convex combinations of the training distributions. However, a much stronger form of OoD generalization is the ability of models to extrapolate beyond the distributions observed during training. In pursuit of strong OoD generalization, we introduce the principle of Risk Extrapolation (REx). REx can be viewed as encouraging robustness over affine combinations of training risks, by encouraging strict equality between training risks. We show conceptually how this principle enables extrapolation, and demonstrate the effectiveness and scalability of instantiations of REx on various OoD generalization tasks. Our code can be found at this https URL.
Unsupervised machine learning of quenched gauge symmetries: A proof-of-concept demonstration

In condensed matter physics, one of the goals of machine learning is the classification of phases of matter. The consideration of a system's symmetries can significantly assist the machine in this goal. We demonstrate the ability of an unsupervised machine learning protocol, the Principal Component Analysis method, to detect hidden quenched gauge symmetries introduced via the so-called Mattis gauge transformation. Our work reveals that unsupervised machine learning can identify hidden properties of a model and may therefore provide new insights into the models themselves.
Fixed points for group actions on 2-dimensional affine buildings

We prove a local-to-global result for fixed points of groups acting on affine buildings (possibly non-discrete) of types $\tilde{A}_1\times \tilde{A}_1, \tilde{A}_2$ or $\tilde{C}_2$. In the discrete case, our theorem establishes a conjecture by Marquis.
Uniqueness of the measure of maximal entropy for the standard map

In this paper we prove that for sufficiently large parameters the standard map has a unique measure of maximal entropy (m.m.e.). Moreover, we prove: the m.m. e. is Bernoulli, and the periodic points with Lyapunov exponents bounded away from zero equidistribute with respect to the m.m. e. We prove some estimates regarding the Hausdorff dimension of the m.m. e. and about the density of the support of the measure on the manifold. For a generic large parameter, we prove that the support of the m.m. e. has Hausdorff dimension $2$. We also obtain the $C^2$-robustness of several of these properties.
Possible lattice and charge order in CuxBi2Te2Se

Metal intercalation into layered topological insulator materials such as the binary chalcogenide Bi2X3 (X = Te or Se) has yielded novel two-dimensional (2D) electron-gas physics, phase transitions to superconductivity, as well as interesting magnetic ground states. Of recent interest is the intercalation-driven interplay between lattice distortions, density wave ordering, and the emergence of new phenomena in the vicinity of instabilities induced by intercalation. Here, we examine the effects of Cu-intercalation on the ternary chalcogenide Bi2Te2Se. We report the discovery, in Cu0.3Bi2Te2Se, of a periodic lattice distortion (PLD) at room temperature, together with a charge density wave (CDW) transition around Td = 220 K. We also report, for the first time, a complete study of the CuxBi2Te2Se system, and the effect of Cu-intercalation on crystal structure, phonon structure, and electronic properties for 0.0 ≤ x ≤ 0.5. Our electron diffraction studies reveal strong Bragg spots at reciprocal lattice positions forbidden by ABC stacking, possibly resulting from stacking faults, or a superlattice. The c-axis lattice parameter varies monotonically with x for 0 < x < 0.2, but drops precipitously for higher x. Similarly, Raman phonon modes A 1 g 2 and E g 2 soften monotonically for 0 < x < 0.2 but harden sharply for x > 0.2. This indicates that Cu likely intercalates up to x ∼ 0.2, followed by partial site-substitutions at higher values. The resulting strain makes the 0.2 ≤ x ≤ 0.3 region susceptible to instabilities and distortions. Our results point toward the presence of an incommensurate CDW above Td = 220 K. This work strengthens prevalent thought that intercalation contributes significantly to instabilities in the lattice and charge degrees of freedom in layered chalcogenides. Further work is required to uncover additional density wave transitions, and possible ground states such as superconductivity.
Online Binary Space Partitioning Forests

The Binary Space Partitioning-Tree~(BSP-Tree) process was recently proposed as an efficient strategy for space partitioning tasks. Because it uses more than one dimension to partition the space, the BSP-Tree Process is more efficient and flexible than conventional axis-aligned cutting strategies. However, due to its batch learning setting, it is not well suited to large-scale classification and regression problems. In this paper, we develop an online BSP-Forest framework to address this limitation. With the arrival of new data, the resulting online algorithm can simultaneously expand the space coverage and refine the partition structure, with guaranteed universal consistency for both classification and regression problems. The effectiveness and competitive performance of the online BSP-Forest is verified via simulations on real-world datasets.
ALMA Observations of Quasar Host Galaxies at z ≃ 4.8

We present ALMA Band 7 data of the [C ii] emission line and underlying far-IR (FIR) continuum for 12 luminous quasars at powered by fast-growing supermassive black holes (SMBHs). Our total sample consists of 18 quasars, 12 of which are presented here for the first time. The new sources consist of six Herschel/SPIRE-detected systems, which we define as “FIR-bright” sources, and six Herschel/SPIRE-undetected systems, which we define as “FIR-faint” sources. We determine dust masses for the quasars hosts of , implying interstellar medium gas masses comparable to the dynamical masses derived from the [C ii] kinematics. It is found that, on average, the Mg ii line is blueshifted by with respect to the [C ii] emission line, which is also observed when complementing our observations with data from the literature. We find that all of our FIR-bright subsample and most of the FIR-faint objects lie above the main sequence of star-forming galaxies at . We detect companion submillimeter galaxies for two sources, both FIR-faint, with a range of projected distances of kpc and typical velocity shifts of from the quasar hosts. Of our total sample of 18 quasars, 5/18 are found to have dust-obscured star-forming companions.
Learned Threshold Pruning

This paper presents a novel differentiable method for unstructured weight pruning of deep neural networks. Our learned-threshold pruning (LTP) method enjoys a number of important advantages. First, it learns per-layer thresholds via gradient descent, unlike conventional methods where they are set as input. Making thresholds trainable also makes LTP computationally efficient, hence scalable to deeper networks. For example, it takes less than $30$ epochs for LTP to prune most networks on ImageNet. This is in contrast to other methods that search for per-layer thresholds via a computationally intensive iterative pruning and fine-tuning process. Additionally, with a novel differentiable $L_0$ regularization, LTP is able to operate effectively on architectures with batch-normalization. This is important since $L_1$ and $L_2$ penalties lose their regularizing effect in networks with batch-normalization. Finally, LTP generates a trail of progressively sparser networks from which the desired pruned network can be picked based on sparsity and performance requirements. These features allow LTP to achieve state-of-the-art compression rates on ImageNet networks such as AlexNet ($26.4\times$ compression with $79.1\%$ Top-5 accuracy) and ResNet50 ($9.1\times$ compression with $92.0\%$ Top-5 accuracy). We also show that LTP effectively prunes newer architectures, such as EfficientNet, MobileNetV2 and MixNet.
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime

Deep neural networks can achieve remarkable generalization performances while interpolating the training data perfectly. Rather than the U-curve emblematic of the bias-variance trade-off, their test error often follows a "double descent" - a mark of the beneficial role of overparametrization. In this work, we develop a quantitative theory for this phenomenon in the so-called lazy learning regime of neural networks, by considering the problem of learning a high-dimensional function with random features regression. We obtain a precise asymptotic expression for the bias-variance decomposition of the test error, and show that the bias displays a phase transition at the interpolation threshold, beyond which it remains constant. We disentangle the variances stemming from the sampling of the dataset, from the additive noise corrupting the labels, and from the initialization of the weights. Following up on Geiger et al. 2019, we first show that the latter two contributions are the crux of the double descent: they lead to the overfitting peak at the interpolation threshold and to the decay of the test error upon overparametrization. We then quantify how they are suppressed by ensemble averaging the outputs of K independently initialized estimators. When K is sent to infinity, the test error remains constant beyond the interpolation threshold. We further compare the effects of overparametrizing, ensembling and regularizing. Finally, we present numerical experiments on classic deep learning setups to show that our results hold qualitatively in realistic lazy learning scenarios.
Simultaneous confidence bands for nonparametric regression with partially missing covariates

In this paper, we consider a weighted local linear estimator based on the inverse selection probability for nonparametric regression with missing covariates at random. The asymptotic distribution of the maximal deviation between the estimator and the true regression function is derived and an asymptotically accurate simultaneous confidence band is constructed. The estimator for the regression function is shown to be oracally efficient in the sense that it is uniformly indistinguishable from that when the selection probabilities are known. Finite sample performance is examined via simulation studies which support our asymptotic theory. The proposed method is demonstrated via an analysis of a data set from the Canada 2010/2011 Youth Student Survey.
Everything About You: A Multimodal Approach towards Friendship Inference in Online Social Networks

Most previous works in privacy of Online Social Networks (OSN) focus on a restricted scenario of using one type of information to infer another type of information or using only static profile data such as username, profile picture or home location. However the multimedia footprints of users has become extremely diverse nowadays. In reality, an adversary would exploit all types of information obtainable over time, to achieve its goal. In this paper, we analyse OSN privacy by jointly exploiting longterm multimodal information. We focus in particular on inference of social relationships. We consider five popular components of posts shared by users, namely images, hashtags, captions, geo-locations and published friendships. Large scale evaluation on a real-world OSN dataset shows that while our monomodal attacks achieve strong predictions, our multimodal attack leads to a stronger performance with AUC (area under the ROC curve) above 0.9. Our results highlight the need for multimodal obfuscation approaches towards protecting privacy in an era where multimedia footprints of users get increasingly diverse.
Nonreciprocal Terahertz Second-Harmonic Generation in Superconducting NbN under Supercurrent Injection.

Giant second-harmonic generation in the terahertz (THz) frequency range is observed in a thin film of an s-wave superconductor NbN, where the time-reversal (T) and space-inversion (P) symmetries are simultaneously broken by supercurrent injection. We demonstrate that the phase of the second-harmonic signal flips when the direction of supercurrent is inverted; i.e., the signal is ascribed to the nonreciprocal response that occurs under broken P and T symmetries. The temperature dependence of the SH signal exhibits a sharp resonance, which is accounted for by the vortex motion driven by the THz electric field in an anharmonic pinning potential. The maximum conversion ratio η_{SHG} reaches ≈10^{-2} in a thin film NbN with the thickness of 25 nm after the field cooling with a very small magnetic field of ≈1 Oe, for a relatively weak incident THz electric field of 2.8 kV/cm at 0.48 THz.
Differentially Private Deep Learning with Smooth Sensitivity

Ensuring the privacy of sensitive data used to train modern machine learning models is of paramount importance in many areas of practice. One approach to study these concerns is through the lens of differential privacy. In this framework, privacy guarantees are generally obtained by perturbing models in such a way that specifics of data used to train the model are made ambiguous. A particular instance of this approach is through a "teacher-student" framework, wherein the teacher, who owns the sensitive data, provides the student with useful, but noisy, information, hopefully allowing the student model to perform well on a given task without access to particular features of the sensitive data. Because stronger privacy guarantees generally involve more significant perturbation on the part of the teacher, deploying existing frameworks fundamentally involves a trade-off between student's performance and privacy guarantee. One of the most important techniques used in previous works involves an ensemble of teacher models, which return information to a student based on a noisy voting procedure. In this work, we propose a novel voting mechanism with smooth sensitivity, which we call Immutable Noisy ArgMax, that, under certain conditions, can bear very large random noising from the teacher without affecting the useful information transferred to the student. Compared with previous work, our approach improves over the state-of-the-art methods on all measures, and scale to larger tasks with both better performance and stronger privacy ($\epsilon \approx 0$). This new proposed framework can be applied with any machine learning models, and provides an appealing solution for tasks that requires training on a large amount of data.
When Does the Onset of Multiple Stellar Populations in Star Clusters Occur? III. No Evidence of Significant Chemical Variations in Main-sequence Stars of NGC 419

Recent studies have revealed that the onset age for the presence of multiple stellar populations (MPs) in star clusters seems to correspond to the disappearance of the extended main-sequence turnoff in young clusters, a pattern associated with stellar rotations. A speculative suggestion is that MPs might be caused by the magnetic brake, a stellar evolutionary effect linked to rotation. In this work, we use the young massive cluster NGC 419 as a testbed. We examined if its magnetically baked MS stars would exhibit MPs. Using the deep ultraviolet and visible images observed through the Hubble Space Telescope, combined with a specific color index that is sensitive to the nitrogen (N) abundance, we examined if its late G- and K-type MS stars are affected by N variation. Our analysis reports that the morphology of its GK-type MS is most likely a simple stellar population, and only a negligible probability, which indicates a N variation up to 0.4 dex is present. We therefore conclude that there is no significant N variation among its GK-type MS stars. The absence of a significant chemical variation among the late-type MS stars indicates that MPs might not be a specific pattern of magnetically braked stars.
Successive dielectric anomalies and magnetoelectric coupling in honeycomb Fe$_4$Nb$_2$O$_9$

By combining single crystal x-ray and neutron diffraction, and the magnetodielectric measurements on single crystal Fe4Nb2O9, we present the magnetic structure and the symmetry-allowed magnetoelectric coupling in Fe4Nb2O9. It undergoes an antiferromagnetic transition at TN=93 K, followed by a displacive transition at TS=70 K. The temperature-dependent dielectric constant of Fe4Nb2O9 is strongly anisotropic with the first anomaly at 93 K due to the exchange striction as a result of the long range spin order, and the second one at 70 K emanating from the structural phase transition primarily driven by the O atomic displacements. Magneticfield induced magnetoelectric coupling was observed in single crystal Fe4Nb2O9 and is compatible with the solved magnetic structure that is characteristic of antiferromagnetically arranged ferromagnetic chains in the honeycomb plane. We propose that such magnetic symmetry should be immune to external magnetic fields to some extent favored by the freedom of rotation of moments in the honeycomb plane, laying out a promising system to control the magnetoelectric properties by magnetic fields.
Unsupervised Learning of Depth, Optical Flow and Pose With Occlusion From 3D Geometry

In autonomous driving, monocular sequences contain lots of information. Monocular depth estimation, camera ego-motion estimation and optical flow estimation in consecutive frames are high-profile concerns recently. By analyzing tasks above, pixels in the middle frame are modeled into three parts: the rigid region, the non-rigid region, and the occluded region. In joint unsupervised training of depth and pose, we can segment the occluded region explicitly. The occlusion information is used in unsupervised learning of depth, pose and optical flow, as the image reconstructed by depth-pose and optical flow will be invalid in occluded regions. A less-than-mean mask is designed to further exclude the mismatched pixels interfered with by motion or illumination change in the training of depth and pose networks. This method is also used to exclude some trivial mismatched pixels in the training of the optical flow network. Maximum normalization is proposed for depth smoothness term to restrain depth degradation in textureless regions. In the occluded region, as depth and camera motion can provide more reliable motion estimation, they can be used to instruct unsupervised learning of optical flow. Our experiments in KITTI dataset demonstrate that the model based on three regions, full and explicit segmentation of the occlusion region, the rigid region, and the non-rigid region with corresponding unsupervised losses can improve performance on three tasks significantly. The source code is available at: https://github.com/guangmingw/DOPlearning.