Miracle-dz/UniFAR-FTUs huggingface dataset:
A Comprehensive Survey on Local Differential Privacy toward Data Statistics and Analysis

Collecting and analyzing massive data generated from smart devices have become increasingly pervasive in crowdsensing, which are the building blocks for data-driven decision-making. However, extensive statistics and analysis of such data will seriously threaten the privacy of participating users. Local differential privacy (LDP) was proposed as an excellent and prevalent privacy model with distributed architecture, which can provide strong privacy guarantees for each user while collecting and analyzing data. LDP ensures that each user’s data is locally perturbed first in the client-side and then sent to the server-side, thereby protecting data from privacy leaks on both the client-side and server-side. This survey presents a comprehensive and systematic overview of LDP with respect to privacy models, research tasks, enabling mechanisms, and various applications. Specifically, we first provide a theoretical summarization of LDP, including the LDP model, the variants of LDP, and the basic framework of LDP algorithms. Then, we investigate and compare the diverse LDP mechanisms for various data statistics and analysis tasks from the perspectives of frequency estimation, mean estimation, and machine learning. Furthermore, we also summarize practical LDP-based application scenarios. Finally, we outline several future research directions under LDP.
The Risks of Invariant Risk Minimization

Invariant Causal Prediction (Peters et al., 2016) is a technique for out-of-distribution generalization which assumes that some aspects of the data distribution vary across the training set but that the underlying causal mechanisms remain constant. Recently, Arjovsky et al. (2019) proposed Invariant Risk Minimization (IRM), an objective based on this idea for learning deep, invariant features of data which are a complex function of latent variables; many alternatives have subsequently been suggested. However, formal guarantees for all of these works are severely lacking. In this paper, we present the first analysis of classification under the IRM objective$-$as well as these recently proposed alternatives$-$under a fairly natural and general model. In the linear case, we show simple conditions under which the optimal solution succeeds or, more often, fails to recover the optimal invariant predictor. We furthermore present the very first results in the non-linear regime: we demonstrate that IRM can fail catastrophically unless the test data are sufficiently similar to the training distribution$-$this is precisely the issue that it was intended to solve. Thus, in this setting we find that IRM and its alternatives fundamentally do not improve over standard Empirical Risk Minimization.
SMYRF: Efficient Attention using Asymmetric Clustering

We propose a novel type of balanced clustering algorithm to approximate attention. Attention complexity is reduced from $O(N^2)$ to $O(N \log N)$, where $N$ is the sequence length. Our algorithm, SMYRF, uses Locality Sensitive Hashing (LSH) in a novel way by defining new Asymmetric transformations and an adaptive scheme that produces balanced clusters. The biggest advantage of SMYRF is that it can be used as a drop-in replacement for dense attention layers without any retraining. On the contrary, prior fast attention methods impose constraints (e.g. queries and keys share the same vector representations) and require re-training from scratch. We apply our method to pre-trained state-of-the-art Natural Language Processing and Computer Vision models and we report significant memory and speed benefits. Notably, SMYRF-BERT outperforms (slightly) BERT on GLUE, while using $50\%$ less memory. We also show that SMYRF can be used interchangeably with dense attention before and after training. Finally, we use SMYRF to train GANs with attention in high resolutions. Using a single TPU, we were able to scale attention to 128x128=16k and 256x256=65k tokens on BigGAN on CelebA-HQ.
Outbursting Quasi-Hilda Asteroid P/2010 H2 (Vales)

Quasi-Hilda asteroid P/2010 H2 (Vales) underwent a spectacular photometric outburst by ≥7.5 mag (factor of ≥103) in 2010. Here, we present our optical observations of this event in the four month period from April 20 to August 10. The outburst, starting UT 2010 April 15.70, released dust particles of total cross-section 17,600 km2 (albedo 0.1 assumed) and mass ∼1.2 × 109 kg, this being about 10−4 of the mass of the nucleus, taken as a sphere of radius 1.5 km and density 500 kg m−3. While the rising phase of the outburst was very steep (brightness doubling time of hours), subsequent fading occurred slowly (fading timescales increasing from weeks to months), as large, low velocity particles drifted away from the nucleus. A simple model of the fading lightcurve indicates that the ejected particles occupied a broad range of sizes, from ∼1 μm to 6 cm, and followed a differential power-law distribution with index 3.6 ± 0.1 (similar to that in other comets). The fastest particles had speeds ≥210 m s−1, indicating gas-drag acceleration of small grains well coupled to the flow. Low-energy processes known to drive mass loss in active asteroids, including rotational disruption; thermal and desiccation stress cracking; and electrostatic repulsion, cannot generate the high particles speeds measured in P/Vales, and are discounted. Impact origin is unlikely given the short dynamical lifetimes of the quasi-Hildas and the low collision probabilities of these objects. The specific energy of the ejecta is estimated at 220 J kg−1. The outburst follows a series of encounters with Jupiter in the previous century, consistent with the delayed activation of buried supervolatiles (and/or the crystallization of subsurface amorphous ice) by conducted heat following an inward displacement of the perihelion. A potential origin in the debris cloud produced by avalanche is also considered.
Studies on Formation of Liquid and Gaseous Formaldehyde-Induced DNA-Protein Crosslinks in Rat Marrow Cells

Formaldehyde (HCHO) is a genotoxicant which is suspected to have the ability to induce leukemia in epidemiology field. However, little is known about the possible mechanism how this agent induces leukemia. In this study, rat marrow cells are chosen to be the exposure targets to both liquid and gaseous formaldehyde. DNA-protein crosslinks are evaluated via KCl-SDS assay while single cell gel electrophoresis technique (comet assay) is used to test the DNA strand breaks of bone marrow cells. Our results show that low concentrations of formaldehyde (liquid: 5μmol·L-1 and 25μmol·L-1, gaseous: 0.5mg·m-3 and 1.0mg·m-3) can cause significant DNA breakage (p 0.01); the higher concentrations of liquid formaldehyde (liquid: 125μmol·L-1 and 625μmol·L-1, gaseous: 3.0mg·m-3) can cause significant DPC (p0.01; p0.05) compared with the control. The results suggest that formaldehyde can induce DNA damage in rat marrow cells which imply formaldehyde is highly possible to induce leukemia.
How Does Mixup Help With Robustness and Generalization?

Mixup is a popular data augmentation technique based on taking convex combinations of pairs of examples and their labels. This simple technique has been shown to substantially improve both the robustness and the generalization of the trained model. However, it is not well-understood why such improvement occurs. In this paper, we provide theoretical analysis to demonstrate how using Mixup in training helps model robustness and generalization. For robustness, we show that minimizing the Mixup loss corresponds to approximately minimizing an upper bound of the adversarial loss. This explains why models obtained by Mixup training exhibits robustness to several kinds of adversarial attacks such as Fast Gradient Sign Method (FGSM). For generalization, we prove that Mixup augmentation corresponds to a specific type of data-adaptive regularization which reduces overfitting. Our analysis provides new insights and a framework to understand Mixup.
Human-centric Dialog Training via Offline Reinforcement Learning

How can we train a dialog model to produce better conversations by learning from human feedback, without the risk of humans teaching it harmful chat behaviors? We start by hosting models online, and gather human feedback from real-time, open-ended conversations, which we then use to train and improve the models using offline reinforcement learning (RL). We identify implicit conversational cues including language similarity, elicitation of laughter, sentiment, and more, which indicate positive human feedback, and embed these in multiple reward functions. A well-known challenge is that learning an RL policy in an offline setting usually fails due to the lack of ability to explore and the tendency to make over-optimistic estimates of future reward. These problems become even harder when using RL for language models, which can easily have a 20,000 action vocabulary and many possible reward functions. We solve the challenge by developing a novel class of offline RL algorithms. These algorithms use KL-control to penalize divergence from a pre-trained prior language model, and use a new strategy to make the algorithm pessimistic, instead of optimistic, in the face of uncertainty. We test the resulting dialog model with ratings from 80 users in an open-domain setting and find it achieves significant improvements over existing deep offline RL approaches. The novel offline RL method is viable for improving any existing generative dialog model using a static dataset of human feedback.
Scaling in Internally Heated Convection: A Unifying Theory

We offer a unifying theory for turbulent, purely internally heated convection, generalizing the unifying theories of Grossmann and Lohse (2000, https://doi.org/10.1017/S0022112099007545; 2001, https://doi.org/10.1103/PhysRevLett.86.3316) for Rayleigh‐Bénard turbulence and of Shishkina et al. (2016, https://doi.org/10.1002/2015GL067003) for turbulent horizontal convection, which are both based on the splitting of the kinetic and thermal dissipation rates in respective boundary and bulk contributions. We obtain the mean temperature of the system and the Reynolds number (which are the response parameters) as function of the control parameters, namely the internal thermal driving strength (called, when nondimensionalized, the Rayleigh‐Roberts number) and the Prandtl number. The results of the theory are consistent with our direct numerical simulations of the Boussinesq equations.
Voting-based Approaches For Differentially Private Federated Learning

While federated learning (FL) enables distributed agents to collaboratively train a centralized model without sharing data with each other, it fails to protect users against inference attacks that mine private information from the centralized model. Thus, facilitating federated learning methods with differential privacy (DPFL) becomes attractive. Existing algorithms based on privately aggregating clipped gradients require many rounds of communication, which may not converge, and cannot scale up to large-capacity models due to explicit dimension-dependence in its added noise. In this paper, we adopt the knowledge transfer model of private learning pioneered by Papernot et al. (2017; 2018) and extend their algorithm PATE, as well as the recent alternative PrivateKNN (Zhu et al., 2020) to the federated learning setting. The key difference is that our method privately aggregates the labels from the agents in a voting scheme, instead of aggregating the gradients, hence avoiding the dimension dependence and achieving significant savings in communication cost. Theoretically, we show that when the margins of the voting scores are large, the agents enjoy exponentially higher accuracy and stronger (data-dependent) differential privacy guarantees on both agent-level and instance-level. Extensive experiments show that our approach significantly improves the privacy-utility trade-off over the current state-of-the-art in DPFL.
DESCNet: Developing Efficient Scratchpad Memories for Capsule Network Hardware

Deep neural networks (DNNs) have been established as the state-of-the-art method for advanced machine learning applications. Recently proposed by the Google Brain’s team, the capsule networks (CapsNets) have improved the generalization ability, as compared to DNNs, due to their multidimensional capsules and preserving the spatial relationship between different objects. However, they pose significantly high computational and memory requirements, making their energy-efficient inference a challenging task. This article provides, for the first time, an in-depth analysis to highlight the design and runtime challenges for the (on-chip scratchpad) memories deployed in hardware accelerators executing fast CapsNets inference. To enable an efficient design, we propose an application-specific memory architecture, called DESCNet, which minimizes the off-chip memory accesses, while efficiently feeding the data to the hardware accelerator executing CapsNets inference. We analyze the corresponding on-chip memory requirement and leverage it to propose a methodology for exploring different scratchpad memory (SPM) designs and their energy/area tradeoffs. Afterward, an application-specific power-gating technique for the on-chip SPM is employed to further reduce its energy consumption, depending upon the mapped dataflow of the CapsNet and the utilization across different operations of its processing. We integrated our DESCNet memory design, as well as another state-of-the-art memory design Marchisio et al. [2018] for comparison studies, with an opensource DNN accelerator executing Google’s CapsNet model Sabour et al. [2017] for the MNIST dataset. We also enhanced the design to execute the recent deep CapsNet model Rajasegaran et al. [2019] for the CIFAR10 dataset. Note: we use the same benchmarks and test conditions for which these CapsNets have been proposed and evaluated by their respective teams. The complete hardware is synthesized for a 32-nm CMOS technology using the ASIC-design flow with Synopsys tools and CACTI-P, and detailed area, performance, and power/energy estimation is performed using different configurations. Our results for a selected Pareto-optimal solution demonstrate no performance loss and an energy reduction of 79% for the complete accelerator, including computational units and memories, when compared to the state-of-the-art design.
Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domains

GANs can generate photo-realistic images from the domain of their training data. However, those wanting to use them for creative purposes often want to generate imagery from a truly novel domain, a task which GANs are inherently unable to do. It is also desirable to have a level of control so that there is a degree of artistic direction rather than purely curation of random results. Here we present a method for interpolating between generative models of the StyleGAN architecture in a resolution dependent manner. This allows us to generate images from an entirely novel domain and do this with a degree of control over the nature of the output.
Effect of the crust on neutron star empirical relations

We analyze how the crust equation of state affects several neutron star properties and how it impacts on possible constraints inferred from astrophysical observations. Using three distinct crusts, we generate three sets of model-independent equations of state describing stellar matter from a Taylor expansion around saturation density. The equations of state are thermodynamically consistent, causal, and compatible with astrophysical observations. The relations between the tidal deformability $\Lambda$ and compactness $C$, Love number $k_2$ and radius of neutron star with mass $M$ are studied, and the effect of the crust equation of state on these relations analyzed. In most of the relations, the impact of the crust equation of state is not larger that 2\%. If, however, a fixed neutron star mass is considered, the relation between the tidal deformability and the radius depends on the crust. We have found that the relation $\Lambda_{M_i} = \alpha R_{M_i}^{\beta}$ becomes almost exact and crust independent for massive neutron stars. It is shown that it is possible to determine the tidal deformability of an 1.4$M_\odot$ star from the GW179817 effective tidal deformability $\tilde\Lambda$ with an accuracy of at least $\approx 10\%$. A high correlation between $\tilde\Lambda$ and the radius of the most massive star of the neutron star binary was confirmed, however, it was demonstrated that the crust has an effect of $\approx 14\%$ on this relation. We have found that the relation $\Lambda_1/\Lambda_2=q^a$ depends on $M_{\text{chirp}}$ as $a\sim \sqrt{M_{\text{chirp}}}$.
The Color of Habitability

From life on other planets to virtual classrooms this thesis spans a wide array of research topics all based on how we see other worlds. Our understanding of everything from moon phases, the planets in our Solar System, and exoplanet atmospheres come from our interpretation of light and one day, our knowledge of light will be used as evidence for the discovery of life on another planet. In the time before we scattered rovers, landers, and brave souls across the Solar System we only knew of the planets and moons from the light they reflected from the Sun back to us. We are in much the same situation with exoplanets today. Our telescopes can gather the light from distant worlds but they are too far out of reach to confirm our observations with in-situ measurements. Soon we will be able to gather light from even smaller exoplanets and eventually Earth-sized exoplanets orbiting in their star's habitable zone. As a reference guide to these upcoming observations what better place to compare to than our own Solar System. What we've done is take measurements of planets in our own Solar System and treat them as exoplanets to determine how different surface types can be differentiated. The result is a database of the spectra, geometric albedos, and color of 19 Solar System objects for use as an exoplanetary field guide. A step beyond the field guide is a way to explore worlds only physics and our imaginations are limited by. By using computer models, we can create thousands of planets to determine the physical and chemical stability of any environment. One parameter domain of interest is the role of surface color on a planet's habitability. Different materials have unique thermal properties that either cool or heat a surface depending on their color and the light that hits them. Dark oceans absorb light well and heat up while white sand is highly reflective and keeps cool.
Identifying Melanoma Images using EfficientNet Ensemble: Winning Solution to the SIIM-ISIC Melanoma Classification Challenge

We present our winning solution to the SIIM-ISIC Melanoma Classification Challenge. It is an ensemble of convolutions neural network (CNN) models with different backbones and input sizes, most of which are image-only models while a few of them used image-level and patient-level metadata. The keys to our winning are: (1) stable validation scheme (2) good choice of model target (3) carefully tuned pipeline and (4) ensembling with very diverse models. The winning submission scored 0.9600 AUC on cross validation and 0.9490 AUC on private leaderboard.
Ni–Sn intermetallics as an efficient buffering matrix of Si anodes in Li-ion batteries

For a successful integration of silicon in high-capacity anodes of Li-ion batteries, its intrinsic capacity decay on cycling due to severe volume swelling should be minimized. In this work, Ni–Sn intermetallics are studied as a buffering matrix during reversible lithiation of Si-based anodes. Si/Ni–Sn composites have been synthesized by mechanical milling using C and Al as process control agents. Ni3Sn4 and Ni3Sn2 intermetallics and their bi-phasic mixture were used as constituents of the buffering matrix. The structure, composition and morphology of the composites have been analyzed by X-ray diffraction (XRD), 119Sn Transmission Mossbauer Spectroscopy (TMS) and scanning electron microscopy (SEM). They consist of ∼150 nm Si nanoparticles embedded in a multi-phase matrix, the nanostructuration of which improves on increasing the Ni3Sn4 amount. The electrochemical properties of the composites were analyzed by galvanostatic cycling in half-cells. Best results for practical applications are found for the bi-phasic matrix Ni3Sn4–Ni3Sn2 in which Ni3Sn4 is electrochemically active while Ni3Sn2 is inactive. Low capacity loss, 0.04% per cycle, and high coulombic efficiency, 99.6%, were obtained over 200 cycles while maintaining a high reversible capacity above 500 mA h g−1 at moderate regime C/5.
Efficient Wasserstein Natural Gradients for Reinforcement Learning

A novel optimization approach is proposed for application to policy gradient methods and evolution strategies for reinforcement learning (RL). The procedure uses a computationally efficient Wasserstein natural gradient (WNG) descent that takes advantage of the geometry induced by a Wasserstein penalty to speed optimization. This method follows the recent theme in RL of including a divergence penalty in the objective to establish a trust region. Experiments on challenging tasks demonstrate improvements in both computational cost and performance over advanced baselines.
Forbidden Subgraphs of Power Graphs

The undirected power graph (or simply power graph) of a group $G$, denoted by $P(G)$, is a graph whose vertices are the elements of the group $G$, in which two vertices $u$ and $v$ are connected by an edge between if and only if either $u=v^i$ or $v=u^j$ for some $i$, $j$. A number of important graph classes, including perfect graphs, cographs, chordal graphs, split graphs, and threshold graphs, can be defined either structurally or in terms of forbidden induced subgraphs. We examine each of these five classes and attempt to determine for which groups $G$ the power graph $P(G)$ lies in the class under consideration. We give complete results in the case of nilpotent groups, and partial results in greater generality. In particular, the power graph is always perfect; and we determine completely the groups whose power graph is a threshold or split graph (the answer is the same for both classes). We give a number of open problems.
Nearly Minimax Optimal Reward-free Reinforcement Learning

We study the reward-free reinforcement learning framework, which is particularly suitable for batch reinforcement learning and scenarios where one needs policies for multiple reward functions. This framework has two phases. In the exploration phase, the agent collects trajectories by interacting with the environment without using any reward signal. In the planning phase, the agent needs to return a near-optimal policy for arbitrary reward functions. We give a new efficient algorithm, \textbf{S}taged \textbf{S}ampling + \textbf{T}runcated \textbf{P}lanning (\algoname), which interacts with the environment at most $O\left( \frac{S^2A}{\epsilon^2}\text{poly}\log\left(\frac{SAH}{\epsilon}\right) \right)$ episodes in the exploration phase, and guarantees to output a near-optimal policy for arbitrary reward functions in the planning phase. Here, $S$ is the size of state space, $A$ is the size of action space, $H$ is the planning horizon, and $\epsilon$ is the target accuracy relative to the total reward. Notably, our sample complexity scales only \emph{logarithmically} with $H$, in contrast to all existing results which scale \emph{polynomially} with $H$. Furthermore, this bound matches the minimax lower bound $\Omega\left(\frac{S^2A}{\epsilon^2}\right)$ up to logarithmic factors. Our results rely on three new techniques : 1) A new sufficient condition for the dataset to plan for an $\epsilon$-suboptimal policy; 2) A new way to plan efficiently under the proposed condition using soft-truncated planning; 3) Constructing extended MDP to maximize the truncated accumulative rewards efficiently.