Miracle-dz/UniFAR-FTUs huggingface dataset:
Transferring Adversarial Robustness Through Robust Representation Matching

With the widespread use of machine learning, concerns over its security and reliability have become prevalent. As such, many have developed defenses to harden neural networks against adversarial examples, imperceptibly perturbed inputs that are reliably misclassified. Adversarial training in which adversarial examples are generated and used during training is one of the few known defenses able to reliably withstand such attacks against neural networks. However, adversarial training imposes a significant training overhead and scales poorly with model complexity and input dimension. In this paper, we propose Robust Representation Matching (RRM), a low-cost method to transfer the robustness of an adversarially trained model to a new model being trained for the same task irrespective of architectural differences. Inspired by student-teacher learning, our method introduces a novel training loss that encourages the student to learn the teacher's robust representations. Compared to prior works, RRM is superior with respect to both model performance and adversarial training time. On CIFAR-10, RRM trains a robust model $\sim 1.8\times$ faster than the state-of-the-art. Furthermore, RRM remains effective on higher-dimensional datasets. On Restricted-ImageNet, RRM trains a ResNet50 model $\sim 18\times$ faster than standard adversarial training.
Short-range Atomic Topology of Ab initio Generated Pd-rich Amorphous PdSi Alloys

Since the pioneering efforts of Duwez and coworkers in 1965, when a solid amorphous phase of Pd-Si (a-PdSi) was obtained in the vicinity of the eutectic concentration, much work has been done. However, some points related to the atomic structures remain to be systematized. In this work, 8 amorphous Pd100-cSic alloys (c = 2.5, 5, 10, 13.34, 15, 17.5, 20, and 22 at %) were generated by molecular dynamics ab initio simulations; the shortrange structure is analyzed using several correlation functions, like Pair Distribution Functions, reduced Pair Distribution Functions, Plane Angle Distribution Functions. Other related properties, like nearest-neighbors and some Frank-Kasper polyhedra are reported. The generated samples correctly reproduce the scarce experimental pair correlation functions that are reported in the literature. An unexpected outcome is the appearance of structural changes in the neighborhood of Pd86.66Si13.34. The explicit amorphous topologies are reported. Introduction As part of a study to investigate the possible magnetic properties of the palladium-rich region of the amorphous palladium-silicon system, a-PdSi [1], a detailed analysis of the amorphous structures generated in the process was undertaken, structures that were systematically developed to make possible a study of the evolution of their topology using consistent simulational parameters. Evidently, it is necessary to create amorphous structures throughout the whole concentration range to produce a complete characterization of the PdSi system; however, since the predicted magnetism for the solid alloys disappeared for concentrations of the order of 20 atomic percent, at %, silicon [1], the study of higher concentrations of silicon were not carried out, but a complementary, more extensive study, is a must. Palladium is a peculiar elemental solid since its location in the periodic table places it in a transitional state from partially filled d-band solids to completely full d-band atomic structures. A metal with paramagnetism fully corroborated in the crystalline state, experiences the promotion of electrons from the full d-band to the energetically near s-state due to the closeness of the energy levels involved. Palladium has also manifested magnetic properties of the ferrotype when expanded [2], when amorphized [3] and when contaminated with hydrogen, deuterium or tritium [4] or with Si [1].
An Oracle Gradient Regularized Newton Method for Quadratic Measurements Regression

Recently, recovering an unknown signal from quadratic measurements has gained popularity because it includes many interesting applications as special cases such as phase retrieval, fusion frame phase retrieval, and positive operator-valued measure. In this paper, by employing the least squares approach to reconstruct the signal, we establish the non-asymptotic statistical property showing that the gap between the estimator and the true signal is vanished in the noiseless case and is bounded in the noisy case by an error rate of $O(\sqrt{p\log(1+2n)/n})$, where $n$ and $p$ are the number of measurements and the dimension of the signal, respectively. We develop a gradient regularized Newton method (GRNM) to solve the least squares problem and prove that it converges to a unique local minimum at a superlinear rate under certain mild conditions. In addition to the deterministic results, GRNM can reconstruct the true signal exactly for the noiseless case and achieve the above error rate with a high probability for the noisy case. Numerical experiments demonstrate the GRNM performs nicely in terms of high order of recovery accuracy, faster computational speed, and strong recovery capability.
Conformal field theory at the critical point of monolayer-bilayer phases of subvalent cations in honeycomb layered materials

Godwill Mbiti Kanyolo ∗ and Titus Masese 3, † The University of Electro-Communications, Department of Engineering Science, 1-5-1 Chofugaoka, Chofu, Tokyo 182-8585 Research Institute of Electrochemical Energy (RIECEN), National Institute of Advanced Industrial Science and Technology (AIST), 1-8-31 Midorigaoka, Ikeda, Osaka 563-8577, Japan AIST-Kyoto University Chemical Energy Materials Open Innovation Laboratory (ChEM-OIL), Yoshidahonmachi, Sakyo-ku, Kyoto-shi 606-8501, Japan
Efficient Continual Learning Ensembles in Neural Network Subspaces

A growing body of research in continual learning focuses on the catastrophic forgetting problem. While many attempts have been made to alleviate this problem, the majority of the methods assume a single model in the continual learning setup. In this work, we question this assumption and show that employing ensemble models can be a simple yet eﬀective method to improve continual performance. However, the training and inference cost of ensembles can increase linearly with the number of models. Motivated by this limitation, we leverage the recent advances in the deep learning optimization literature, such as mode connectivity and neural network subspaces, to derive a new method that is both computationally advantageous and can outperform the state-of-the-art continual learning algorithms
Interpreting Language Models with Contrastive Explanations

Model interpretability methods are often used to explain NLP model decisions on tasks such as text classification, where the output space is relatively small. However, when applied to language generation, where the output space often consists of tens of thousands of tokens, these methods are unable to provide informative explanations. Language models must consider various features to predict a token, such as its part of speech, number, tense, or semantics.Existing explanation methods conflate evidence for all these features into a single explanation, which is less interpretable for human understanding.To disentangle the different decisions in language modeling, we focus on explaining language models contrastively: we look for salient input tokens that explain why the model predicted one token instead of another. We demonstrate that contrastive explanations are quantifiably better than non-contrastive explanations in verifying major grammatical phenomena, and that they significantly improve contrastive model simulatability for human observers. We also identify groups of contrastive decisions where the model uses similar evidence, and we are able to characterize what input tokens models use during various language generation decisions.
Non-Clairvoyant Scheduling with Predictions Revisited

In non-clairvoyant scheduling, the task is to nd an online strategy for scheduling jobs with a priori unknown processing requirements with the objective to minimize the total (weighted) completion time. We revisit this well-studied problem in a recently popular learning-augmented setting that integrates (untrusted) predictions in online algorithm design. While previous works used predictions on processing requirements, we propose a new prediction model, which provides a relative order of jobs which could be seen as predicting algorithmic actions rather than parts of the unknown input. We show that these predictions have desired properties, admit a natural error measure as well as algorithms with strong performance guarantees and that they are learnable in both, theory and practice. We generalize the algorithmic framework proposed in the seminal paper by Kumar et al. (NeurIPS’18) and present the rst learning-augmented scheduling results for weighted jobs and unrelated machines. We demonstrate in empirical experiments the practicability and superior performance compared to the previously suggested single-machine algorithms.
SN 2020jfo: A Short-plateau Type II Supernova from a Low-mass Progenitor

We present spectroscopic and photometric observations of the Type IIP supernova, SN 2020jfo, in ultraviolet and optical wavelengths. SN 2020jfo occurred in the spiral galaxy M61 (NGC 4303), with eight observed supernovae in the past 100 yr. SN 2020jfo exhibited a short plateau lasting < 65 days, and achieved a maximum brightness in Vband of M V = −17.4 ± 0.4 mag at about 8.0 ± 0.5 days since explosion. From the bolometric light curve, we have estimated the mass of 56Ni synthesized in the explosion to be 0.033 ± 0.006 M ⊙. The observed spectral features are typical for a Type IIP supernova except for shallow Hα absorption throughout the evolution and the presence of stable 58Ni feature at 7378 Å, in the nebular phase. Using hydrodynamical modeling in the MESA + STELLA framework, an ejecta mass of ∼5 M ⊙ is estimated. Models also indicate SN 2020jfo could be the result of a red supergiant progenitor with M ZAMS ∼ 12 M ⊙. Bolometric light-curve modeling revealed the presence of a secondary radiation source for initial ∼20 days, which has been attributed to interaction with a circumstellar material of mass ∼ 0.2 M ⊙, which most likely was ejected due to enhanced mass loss about 20 yr prior to the supernova explosion.
Mid- and Far-infrared Color–Color Relations within Local Galaxies

We present an extensive archival analysis of a sample of local galaxies, combining multiwavelength data from GALEX, Spitzer, and Herschel to investigate “blue-side” mid-infrared (MIR) and “red-side” far-infrared (FIR) color–color correlations within the observed infrared spectral energy distributions. Our sample largely consists of the KINGFISH galaxies, with the important addition of a select few including NGC 5236 (M83) and NGC 4449. With data from the far-ultraviolet (∼0.15 μm) through 500 μm convolved to common angular resolution, we measure the photometry of kiloparsec-scale star-forming regions 36″ × 36″ in size. Star formation rates (SFRs), stellar masses, and metallicity distributions are derived throughout our sample. Focusing on the f 70/f 500 “FIR” and f 8/f 24 “MIR” flux density ratios (colors), we find that a subsample of galaxies demonstrate a strong IR color–color correlation within their star-forming regions, while others demonstrate uncorrelated colors. This division is driven by two main effects: (1) the local strength of star formation (SF) and (2) the metal content of the interstellar medium (ISM). Galaxies uniformly dominated by high surface densities of SF (e.g., NGC 5236) demonstrate strong IR color–color correlations, while galaxies that exhibit lower levels of SF and mixed environments (e.g., NGC 5457) demonstrate weaker or no correlation—explained by the increasing effect of varying ISM heating and metal content on the IR colors, specifically in the MIR. We find large dispersion in the SFR–L 8 (8 μm luminosity) relation that is traced by the metallicity distributions, consistent with extant studies, highlighting its problematic use as an SFR indicator across diverse systems/samples.
Fast and Scalable Memristive In-Memory Sorting with Column-Skipping Algorithm

Memristive in-memory sorting has been proposed recently to improve hardware sorting efficiency. Using iterative in-memory min computations, data movements between memory and external processing units can be eliminated for improved latency and energy efficiency. However, the bit-traversal algorithm to search the min requires a large number of column reads on memristive memory. In this work, we propose a column-skipping algorithm with help of a near-memory circuit. Redundant column reads can be skipped based on recorded states for improved latency and hardware efficiency. To enhance the scalability, we develop a multi-bank management that enables column-skipping for dataset stored in different memristive memory banks. Prototype column-skipping sorters are implemented with a 1T1R memristive memory in 40nm CMOS technology. Experimented on a variety of sorting datasets, the length-1024 32-bit column-skipping sorter with state recording of 2 demonstrates up to 4.08× speedup, 3.14× area efficiency and 3.39× energy efficiency, respectively, over the latest memristive in-memory sorting.
Sample Efficient Grasp Learning Using Equivariant Models

In planar grasp detection, the goal is to learn a function from an image of a scene onto a set of feasible grasp poses in $\mathrm{SE}(2)$. In this paper, we recognize that the optimal grasp function is $\mathrm{SE}(2)$-equivariant and can be modeled using an equivariant convolutional neural network. As a result, we are able to significantly improve the sample efficiency of grasp learning, obtaining a good approximation of the grasp function after only 600 grasp attempts. This is few enough that we can learn to grasp completely on a physical robot in about 1.5 hours.
Mixed Precision Iterative Refinement with Sparse Approximate Inverse Preconditioning

With the commercial availability of mixed precision hardware, mixed precision GMRES-based iterative refinement schemes have emerged as popular approaches for solving sparse linear systems. Existing analyses of these approaches, however, are based on using full LU factorizations to construct preconditioners for use within GMRES in each refinement step. In practical applications, inexact preconditioning techniques, such as incomplete LU or sparse approximate inverses, are often used for performance reasons. In this work, we investigate the use of sparse approximate inverse preconditioners based on Frobenius norm minimization within GMRES-based iterative refinement. We analyze the computation of sparse approximate inverses in finite precision and derive constraints under which user-specified stopping criteria will be satisfied. We then analyze the behavior of and convergence constraints for a five-precision GMRES-based iterative refinement scheme that uses sparse approximate inverse preconditioning, which we call SPAI-GMRES-IR. Our numerical experiments confirm the theoretical analysis and illustrate the resulting tradeoffs between preconditioner sparsity and GMRES-IR convergence rate.
I/O-Optimal Algorithms for Symmetric Linear Algebra Kernels

In this paper, we consider two fundamental symmetric kernels in linear algebra: the Cholesky factorization and the symmetric rank-k update (SYRK), with the classical three nested loops algorithms for these kernels. In addition, we consider a machine model with a fast memory of size S and an unbounded slow memory. In this model, all computations must be performed on operands in fast memory, and the goal is to minimize the amount of communication between slow and fast memories. As the set of computations is fixed by the choice of the algorithm, only the ordering of the computations (the schedule) directly influences the volume of communications.
SIDIS Measurement with A=3 Nuclei

We introduce a new experimental effort at Jefferson Lab (JLab) to precisely measure the ratios of charged pion electroproduction in Semi-Inclusive Deep Inelastic Scattering (SIDIS) from $^2$D, $^3$He, and $^3$H targets \cite{c12-21-004}. This conditionally approved experiment (C12-21-004) aims to run in Hall-B using the standard CLAS12 configuration and a new target system developed for the approved quasi-elastic experiment (E12-20-005). In this data-driven study, we will measure the cross-sections as a function of ($x$, $Q^2$, $z$, $P_T$) to allow the extraction of the unpolarized parton distribution functions (PDFs), transverse momentum distributions (TMDs) and fragmentation functions (FFs) in A = 3 nuclei. By using super-ratios of pion yields of SIDIS off light nuclei over a wide $x_B$ range, we search for evidence of a flavor dependence in the EMC effect, giving us new insights into the effect of the nuclear environment on valance quarks. Double-ratios between A = 3 mirror nuclei can provide a direct measurement of the d/u ratios at large x due to their similar and well-understood nuclear corrections. With the utilization of mirror nuclei and the large kinematic range, and high precision of CLAS12, we will be able to maintain the sensitivity to the underlying physics but dramatically decrease the nuclear uncertainties due to attenuation and hadronization in heavy nuclei targets.
Berman Codes: A Generalization of Reed-Muller Codes that Achieve BEC Capacity

We identify a family of binary codes whose structure is similar to Reed-Muller (RM) codes and which include RM codes as a strict subclass. The codes in this family are denoted as ${\mathcal{C}_n}(r,m)$, and their duals are denoted as ${{\mathcal{B}}_n}(r,m)$. The length of these codes is nm, where n ≥ 2, and r is their ‘order’. When n = 2, ${\mathcal{C}_n}(r,m)$ is the RM code of order r and length 2m. The special case of these codes corresponding to n being an odd prime was studied by Berman (1967) and Blackmore and Norton (2001). Following the terminology introduced by Blackmore and Norton, we refer to ${{\mathcal{B}}_n}(r,m)$ as the Berman code and ${\mathcal{C}_n}(r,m)$ as the dual Berman code. We identify these codes using a recursive Plotkin-like construction, and we show that these codes have a rich automorphism group. Applying a result of Kumar et al. (2016) to this set of automorphisms, we show that these codes achieve the capacity of the binary erasure channel (BEC) under bit-MAP decoding.
VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learning

We propose VRL3, a powerful data-driven framework with a simple design for solving challenging visual deep reinforcement learning (DRL) tasks. We analyze a number of major obstacles in taking a data-driven approach, and present a suite of design principles, novel findings, and critical insights about data-driven visual DRL. Our framework has three stages: in stage 1, we leverage non-RL datasets (e.g. ImageNet) to learn task-agnostic visual representations; in stage 2, we use offline RL data (e.g. a limited number of expert demonstrations) to convert the task-agnostic representations into more powerful task-specific representations; in stage 3, we fine-tune the agent with online RL. On a set of challenging hand manipulation tasks with sparse reward and realistic visual inputs, compared to the previous SOTA, VRL3 achieves an average of 780% better sample efficiency. And on the hardest task, VRL3 is 1220% more sample efficient (2440% when using a wider encoder) and solves the task with only 10% of the computation. These significant results clearly demonstrate the great potential of data-driven deep reinforcement learning.
Holistic Attention-Fusion Adversarial Network for Single Image Defogging

—Adversarial learning-based image defogging meth-ods have been extensively studied in computer vision due to their remarkable performance. However, most existing methods have limited defogging capabilities for real cases because they are trained on the paired clear and synthesized foggy images of the same scenes. In addition, they have limitations in preserving vivid color and rich textual details in defogging. To address these issues, we develop a novel generative adversarial network, called holistic attention-fusion adversarial network (HAAN), for single image defogging. HAAN consists of a Fog2Fogfree block and a Fogfree2Fog block. In each block, there are three learning-based modules, namely, fog removal, color-texture recovery, and fog synthetic, that are constrained each other to generate high quality images. HAAN is designed to exploit the self-similarity of texture and structure information by learning the holistic channel-spatial feature correlations between the foggy image with its several derived images. Moreover, in the fog synthetic module, we utilize the atmospheric scattering model to guide it to improve the generative quality by focusing on an atmospheric light optimization with a novel sky segmentation network. Extensive experiments on both synthetic and real-world datasets show that HAAN outperforms state-of-the-art defogging methods in terms of quantitative accuracy and subjective visual quality.
Bayes-Optimal Classifiers under Group Fairness

Machine learning algorithms are becoming integrated into more and more high-stakes decision-making processes, such as in social welfare issues. Due to the need of mitigating the potentially disparate impacts from algorithmic predictions, many approaches have been proposed in the emerging area of fair machine learning. However, the fundamental problem of characterizing Bayes-optimal classifiers under various group fairness constraints has only been investigated in some special cases. Based on the classical Neyman-Pearson argument (Neyman and Pearson, 1933; Shao, 2003) for optimal hypothesis testing, this paper provides a unified framework for deriving Bayes-optimal classifiers under group fairness. This enables us to propose a group-based thresholding method we call FairBayes, that can directly control disparity, and achieve an essentially optimal fairness-accuracy tradeoff. These advantages are supported by thorough experiments.
CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing

The text-based speech editor allows the editing of speech through intuitive cutting, copying, and pasting operations to speed up the process of editing speech. However, the major drawback of current systems is that edited speech often sounds unnatural due to cut-copy-paste operation. In addition, it is not obvious how to synthesize records according to a new word not appearing in the transcript. This paper first proposes a novel end-to-end text-based speech editing method called context-aware mask prediction network (CampNet), which can solve unnatural prosody in the edited region and synthesize the speech corresponding to the unseen words in the transcript. Secondly, to cover various situations of text-based speech editing, we design three text-based operations based on CampNet: deletion, insertion, and replacement. Thirdly, to synthesize the speech corresponding to long text, a word-level autoregressive generation method is proposed. Fourthly, we propose a speaker adaptation method using only one sentence for CampNet and explore the ability of few-shot learning based on CampNet, which provides a new idea for speech forgery tasks. The subjective and objective experiments on VCTK and LibriTTS datasets11Examples of generated speech can be found at https://hairuo55.github. io/CampNet show that the speech editing results based on CampNet are better than TTS technology, manual editing, and VoCo method. We also conduct detailed ablation experiments to explore the effect of the CampNet structure on its performance. Finally, the experiment shows that speaker adaptation with only one sentence can further improve the naturalness of speech editing for one-shot learning.