Miracle-dz/UniFAR-FTUs huggingface dataset:
Explicit Regularization of Stochastic Gradient Methods through Duality

We consider stochastic gradient methods under the interpolation regime where a perfect fit can be obtained (minimum loss at each observation). While previous work highlighted the implicit regularization of such algorithms, we consider an explicit regularization framework as a minimum Bregman divergence convex feasibility problem. Using convex duality, we propose randomized Dykstra-style algorithms based on randomized dual coordinate ascent. For non-accelerated coordinate descent, we obtain an algorithm which bears strong similarities with (non-averaged) stochastic mirror descent on specific functions, as it is is equivalent for quadratic objectives, and equivalent in the early iterations for more general objectives. It comes with the benefit of an explicit convergence theorem to a minimum norm solution. For accelerated coordinate descent, we obtain a new algorithm that has better convergence properties than existing stochastic gradient methods in the interpolating regime. This leads to accelerated versions of the perceptron for generic $\ell_p$-norm regularizers, which we illustrate in experiments.
Sample Efficient Ensemble Learning with Catalyst.RL

We present Catalyst.RL, an open-source PyTorch framework for reproducible and sample efficient reinforcement learning (RL) research. Main features of Catalyst.RL include large-scale asynchronous distributed training, efficient implementations of various RL algorithms and auxiliary tricks, such as n-step returns, value distributions, hyperbolic reinforcement learning, etc. To demonstrate the effectiveness of Catalyst.RL, we applied it to a physics-based reinforcement learning challenge "NeurIPS 2019: Learn to Move -- Walk Around" with the objective to build a locomotion controller for a human musculoskeletal model. The environment is computationally expensive, has a high-dimensional continuous action space and is stochastic. Our team took the 2nd place, capitalizing on the ability of Catalyst.RL to train high-quality and sample-efficient RL agents in only a few hours of training time. The implementation along with experiments is open-sourced so results can be reproduced and novel ideas tried out.
Correlation functions of Landau gauge Yang-Mills theory

Correlation functions of Yang-Mills theory in the Landau gauge are calculated from their equations of motion. The employed setup is completely parameter-free and leads, within errors, to good quantitative agreement with corresponding lattice results for the ghost and gluon propagators as well as the ghost-gluon and three-gluon vertices. Also the four-gluon vertex is calculated. The present setup allows for the first time for a unique subtraction of quadratic divergences in the gluon propagator Dyson-Schwinger equation. Thus, there are no ambiguities which can arise due to the use of models or auxiliary workarounds. In addition, several self-tests of the results are described that allow assessing the truncation error in a self-consistent way. This enables a new perspective on how to identify limitations of the present setup and develop future improvements.
Differentially Private Naïve Bayes Classifier Using Smooth Sensitivity

Abstract There is increasing awareness of the need to protect individual privacy in the training data used to develop machine learning models. Differential Privacy is a strong concept of protecting individuals. Naïve Bayes is a popular machine learning algorithm, used as a baseline for many tasks. In this work, we have provided a differentially private Naïve Bayes classifier that adds noise proportional to the smooth sensitivity of its parameters. We compare our results to Vaidya, Shafiq, Basu, and Hong [1] which scales noise to the global sensitivity of the parameters. Our experimental results on real-world datasets show that smooth sensitivity significantly improves accuracy while still guaranteeing ɛ-differential privacy.
Distance-based Formation Tracking with Unknown Bounded Reference Velocity

This paper studies a distance-based formation tracking problem in the d-dimensional space. The formation has a directed acyclic leader-following structure with d leaders moving at a same unknown bounded velocity. Distributed control laws are proposed for follower agents to maintain the desired formation shape and move at the leaders’ velocity in finite time. Simulation results are also provided to support the theoretical analysis.
A Simple Class Decision Balancing for Incremental Learning

Class incremental learning (CIL) problem, in which a learning agent continuously learns new classes from incrementally arriving training data batches, has gained much attention recently in AI and computer vision community due to both fundamental and practical perspectives of the problem. For mitigating the main difficulty of deep neural network(DNN)-based CIL, the catastrophic forgetting, recent work showed that a simple fine-tuning (FT) based schemes can outperform the earlier attempts of using knowledge distillation, particularly when a small-sized exemplar-memory for storing samples from the previously learned classes is allowed. The core limitation of the vanilla FT, however, is the severe classification score bias between the new and previously learned classes, and several state-of-the-art methods proposed to rectify the bias via additional post-processing of the scores. In this paper, we propose two simple modifications for the vanilla FT, separated softmax (SS) layer and ratio-preserving (RP) mini-batches for SGD updates. Our scheme, dubbed as SS-IL, is shown to give much more balanced class decisions, have much less biased scores, and outperform strong state-of-the-art baselines on several large-scale benchmark datasets, without any sophisticated post-processing of the scores. We also give several novel analyses our and baseline methods, confirming the effectiveness of our approach in CIL.
Fairness in ad auctions through inverse proportionality

We study the tradeoff between social welfare maximization and fairness in the context of ad auctions. We study an ad auction setting where users arrive one at a time, $k$ advertisers submit values for each user, and the auction assigns a distribution over ads to each user. Following the works of Dwork and Ilvento (2019) and Chawla et al. (2020), our goal is to design a truthful auction that satisfies "individual fairness" in its outcomes: informally speaking, users that are similar to each other should obtain similar allocations of ads. We express the fairness constraint as a kind of stability condition: any two users that are assigned multiplicatively similar values by all the advertisers must receive additively similar allocations for each advertiser. This value stability constraint is expressed as a function that maps the multiplicative distance between value vectors to the maximum allowable $\ell_\infty$ distance between the corresponding allocations. Standard auctions do not satisfy this kind of value stability. Our main contribution is a new class of allocation algorithms called Inverse Proportional Allocation that achieve value stability with respect to an expressive class of stability conditions. These allocation algorithms are truthful and prior-free, and achieve a constant factor approximation to the optimal (unconstrained) social welfare. In particular, the approximation ratio is independent of the number of advertisers in the system. In this respect, these allocation algorithms greatly surpass the guarantees achieved in previous work. In fact, our algorithms achieve a near optimal tradeoff between fairness and social welfare under a mild assumption on the value stability constraint. We also extend our results to broader notions of fairness that we call subset fairness.
The Erratic Path to Coalescence of LISA Massive Black Hole Binaries in Subparsec-resolution Simulations of Smooth Circumnuclear Gas Disks

We perform high-resolution simulations to follow the orbital decay of 5 × 105 and 107 M⊙ massive black hole (MBH) pairs embedded in a circumnuclear gas disk (CND), from the CND scale (100 pc) down to 0.1–0.01 pc, the scale at which a circumbinary disk (CBD) could form. The MBHs’ erratic orbital evolution is characterized by three stages: (i) a slow initial decay that leads to some circularization; (ii) a fast-migration phase, analogous to type III migration for massive planets in protoplanetary disks, in which angular momentum is efficiently drained by disk-driven torques arising from the co-orbital region of the secondary MBH, at a distance of 1–3 Hill radii; and (iii) a very slow decay phase, in which orbital angular momentum can even increase. In this last stage, a CBD forms when the parsec-scale decay rate becomes small enough to allow sufficient time for a cavity to be carved. When this happens, the MBH separation nearly stalls in our higher-resolution run. We suggest an empirically modified gap-opening criterion that takes into account such timescale effects, as well as other deviations from standard assumptions. Interestingly, a CBD does not form in the lower-resolution runs, resulting in a faster subparsec decay. Our findings show that the subparsec MBH pairing in gaseous disks is a stochastic and fragile process. Additional mechanisms, such as the stellar-driven hardening, may be necessary to guarantee that the gravitational wave emission phase is entered and the MBHs become accessible to future detectors such as the Laser Interferometer Space Antenna.
Consistent Scaling Exponents at the Deconfined Quantum-Critical Point

We report a quantum Monte Carlo study of the phase transition between antiferromagnetic and valence-bond solid ground states in the square-lattice S = 1/2 J–Q model. The critical correlation function of the Q terms gives a scaling dimension corresponding to the value ν = 0.455 ± 0.002 of the correlation-length exponent. This value agrees with previous (less precise) results from conventional methods, e.g., finite-size scaling of the near-critical order parameters. We also study the Q-derivatives of the Binder cumulants of the order parameters for L2 lattices with L up to 448. The slope grows as L1/ν with a value of ν consistent with the scaling dimension of the Q term. There are no indications of runaway flow to a first-order phase transition. The mutually consistent estimates of ν provide compelling support for a continuous deconfined quantum-critical point.
Higher topological charge and the QCD vacuum

It is shown that gauge field configurations with higher topological charge modify the structure of the QCD vacuum, which is reflected in its dependence on the CP-violating topological phase $\theta$. To explore this, topological susceptibilities and the production of axion dark matter are studied here. The former characterize the topological charge distribution and are therefore sensitive probes of the topological structure of QCD. The latter depends on the effective potential of axions, which is determined by the $\theta$-dependence of QCD. The production of cold dark matter through the vacuum realignement mechanism of axions can therefore be affected by higher topological charge effects. This is discussed qualitatively in the deconfined phase at high temperatures, where a description based on a dilute gas of instantons with arbitrary topological charge is valid. As a result, topological susceptibilities exhibit a characteristic temperature dependence due to anharmonic modifications of the $\theta$-dependence. Furthermore, multi-instanton effects give rise to a topological mechanism to increase the amount of axion dark matter.
Fundamental Limits of Online Network-Caching

Optimal caching of files in a content distribution network (CDN) is a problem of fundamental and growing commercial interest. Although many different caching algorithms are in use today, the fundamental performance limits of network caching algorithms from an online learning point-of-view remain poorly understood to date. In this paper, we resolve this question in the following two settings: (1) a single user connected to a single cache, and (2) a set of users and a set of caches interconnected through a bipartite network. Recently, an online gradient-based coded caching policy was shown to enjoy sub-linear regret. However, due to the lack of known regret lower bounds, the question of the optimality of the proposed policy was left open. In this paper, we settle this question by deriving tight non-asymptotic regret lower bounds in both of the above settings. In addition to that, we propose a new Follow-the-Perturbed-Leader-based uncoded caching policy with near-optimal regret. Technically, the lower-bounds are obtained by relating the online caching problem to the classic probabilistic paradigm of balls-into-bins. Our proofs make extensive use of a new result on the expected load in the most populated half of the bins, which might also be of independent interest. We evaluate the performance of the caching policies by experimenting with the popular MovieLens dataset and conclude the paper with design recommendations and a list of open problems.
Distributed Bayesian clustering

In many modern applications, there is interest in analyzing enormous data sets that cannot be easily moved across computers or loaded into memory on a single computer. In such settings, it is very common to be interested in clustering. Existing distributed clustering algorithms are mostly distance or density based without a likelihood specification, precluding the possibility of formal statistical inference. We introduce a nearly embarrassingly parallel algorithm using a Bayesian finite mixture of mixtures model for distributed clustering, which we term distributed Bayesian clustering (DIB-C). DIB-C can flexibly accommodate data sets with various shapes (e.g. skewed or multi-modal). With data randomly partitioned and distributed, we first run Markov chain Monte Carlo in an embarrassingly parallel manner to obtain local clustering draws and then refine across nodes for a final cluster estimate based on any loss function on the space of partitions. DIB-C can also provide a posterior predictive distribution, estimate cluster densities, and quickly classify new subjects. Both simulation studies and real data applications show superior performance of DIB-C in terms of robustness and computational efficiency.
Neutrino Phenomenology from Democratic Approach

In this article, we derive the tiny neutrino masses and mixings from the democratic and diagonal texture approach, which consistent with the recent experimental oscillation data. The unitary rotation matrices, which diagonalize the neutrino mass matrices are obtained by a specific parametrization of the Pontecorvo-Maki-Nakagawa-Sakata (PMNS) mixing matrix. From which, we tried to calculate all three mixing angles as well as the Dirac CP-violating phase interms of model mixing parameters. In particular, the deviation from the tribimaximal mixing is explained in this model. Along with the Jarlskog parameter in terms of model parameter and neutrino-less double beta decay (NDBD) has been discussed briefly.
Regularization in High-Dimensional Regression and Classification via Random Matrix Theory

We study general singular value shrinkage estimators in high-dimensional regression and classification, when the number of features and the sample size both grow proportionally to infinity. We allow models with general covariance matrices that include a large class of data generating distributions. As far as the implications of our results are concerned, we find exact asymptotic formulas for both the training and test errors in regression models fitted by gradient descent, which provides theoretical insights for early stopping as a regularization method. In addition, we propose a numerical method based on the empirical spectra of covariance matrices for the optimal eigenvalue shrinkage classifier in linear discriminant analysis. Finally, we derive optimal estimators for the dense mean vectors of high-dimensional distributions. Throughout our analysis we rely on recent advances in random matrix theory and develop further results of independent mathematical interest.
Sample-Path Large Deviations for Unbounded Additive Functionals of the Reflected Random Walk

We prove a sample-path large deviation principle (LDP) with sublinear speed for unbounded functionals of certain Markov chains induced by the Lindley recursion. The LDP holds in the Skorokhod space [Formula: see text] equipped with the [Formula: see text] topology. Our technique hinges on a suitable decomposition of the Markov chain in terms of regeneration cycles. Each regeneration cycle denotes the area accumulated during the busy period of the reflected random walk. We prove a large deviation principle for the area under the busy period of the Markov random walk, and we show that it exhibits a heavy-tailed behavior. Funding: The research of B. Zwart and M. Bazhba is supported by the Nederlandse Organisatie voor Wetenschappelijk Onderzoek [Grant 639.033. 413]. The research of J. Blanchet is supported by the National Science Foundation (NSF) [Grants 1915967, 1820942, and 1838576] as well as the Defense Advanced Research Projects Agency [Grant N660011824028]. The research of C. -H. Rhee is supported by the NSF [Grant CMMI-2146530].
Quasi-experimental Designs for Assessing Response on Social Media to Policy Changes

Regulation of tobacco products is rapidly evolving. Understanding public sentiment in response to changes is very important as authorities assess how to effectively protect population health. Social media systems are widely recognized to be useful for collecting data about human preferences and perceptions. However, how social media data may be used, in rapid policy change settings, given challenges of narrow time periods and specific locations and non-representative the population using social media is an open question. In this paper we apply quasi-experimental designs, which have been used previously in observational data such as social media, to control for time and location confounders on social media, and then use content analysis of Twitter and Reddit posts to illustrate the content of reactions to tobacco flavor bans and the effect of taxation on e-cigarettes. Conclusions distill the potential role of social media in settings of rapidly changing regulation, in complement to what is learned by traditional denominator-based representative surveys.
When the Guard failed the Droid: A case study of Android malware

Android malware is a persistent threat to billions of users around the world. As a countermeasure, Android malware detection systems are occasionally implemented. However, these systems are often vulnerable to \emph{evasion attacks}, in which an adversary manipulates malicious instances so that they are misidentified as benign. In this paper, we launch various innovative evasion attacks against several Android malware detection systems. The vulnerability inherent to all of these systems is that they are part of Androguard~\cite{desnos2011androguard}, a popular open source library used in Android malware detection systems. Some of the detection systems decrease to a 0\% detection rate after the attack. Therefore, the use of open source libraries in malware detection systems calls for caution. In addition, we present a novel evaluation scheme for evasion attack generation that exploits the weak spots of known Android malware detection systems. In so doing, we evaluate the functionality and maliciousness of the manipulated instances created by our evasion attacks. We found variations in both the maliciousness and functionality tests of our manipulated apps. We show that non-functional apps, while considered malicious, do not threaten users and are thus useless from an attacker's point of view. We conclude that evasion attacks must be assessed for both functionality and maliciousness to evaluate their impact, a step which is far from commonplace today.
On the UV completion of the $O(N)$ model in $6-\epsilon$ dimensions: a stable large-charge sector

We study large charge sectors in the $O(N)$ model in $6-\epsilon $ dimensions. For $4<d<6$, in perturbation theory, the quartic $O(N)$ theory has a UV stable fixed point at large $N$. It was recently argued that this fixed point can be described in terms of an IR fixed point of a cubic $O(N)$ model. By considering a double scaling limit of large charge and weak couplings, we compute two-point and all "extremal" higher-point correlation functions for large charge operators and find a precise equivalence between both pictures. Instanton instabilities are found to be exponentially suppressed at large charge. We also consider correlation function of $U(1)$-invariant meson operators in the $O(2N)\supset U(1)\times SU(N)$ theory, as a first step towards tests of (higher spin) $AdS/CFT$.
Epidemic Potential for Human Infection with Influenza A (H7N9) Virus in China through Web Search Behaviors: A Data-Driven Study

Since the beginning of September 2016, a steep upsurge of the human cases of avian influenza A (H7N9) virus has been reported in China, which are alarming public concern for the pandemic potential of the H7N9 virus. In this study, we collected the data from H7N9 epidemics and H7N9-related Baidu Search Index (BSI) in China between September 2013 and June 2017. And we observed a strong correlation between the numbers of Influenza A (H7N9) cases and H7N9-related BSI in Guangdong province and Shanghai municipality (p<0.001). Autoregressive integrated moving average (ARIMA) models were constructed for the dynamic estimation of seasonal H7N9 outbreaks in 2016-2017 and the online search data acted as an external regressor with the historical H7N9 epidemic data in the forecasting model to improve the quality of predictions. Predictions by the models closely matched the actual numbers of reported cases during current H7N9 epidemic season. Especially, the estimated numbers of reported cases sharply increased to reach 49.88 (95% CI: 0-194.05) in Guangdong and 9.05 (95% CI: 0-37.43) in Shanghai from December 2016 to June 2017. Moreover, this accessible and flexible dynamic forecast model could be used in the monitoring of H7N9 virus to provide advanced warning of future emerging infection diseases. Author summary As the availability and popularity of the internet has greatly increased in recent years, an increasing number of cyber users, including patients and their family members, search online for health information on personal computers (PCs) and mobile phones (MPs) before seeking medical attention, making it possible to investigate the influenza prevalence by monitoring changes in frequencies of uses of particular search terms. In this study, we collected the data from H7N9 epidemics and H7N9-related Baidu Search Index (BSI) in China between September 2013 and June 2017. And then, we showed a strong correlation between the numbers of Influenza A (H7N9) cases and H7N9-related BSI in Guangdong province and Shanghai municipality (p<0.001). Furthermore, we reconstructed an improved dynamic forecasting method for outbreaks of H7N9 influenza using Autoregressive integrated moving average (ARIMA) models to predict future patterns of H7N9 transmission and the online search data acted as an external regressor with the historical H7N9 epidemic data in the forecasting model to improve the quality of predictions. Our results suggest that data from the Baidu search engine, combed with data from a traditional disease surveillance system, may be considered for early detection of H7N9 influenza outbreaks in mainland China.