Miracle-dz/UniFAR-FTUs huggingface dataset:
Local Explanation of Dialogue Response Generation

In comparison to the interpretation of classification models, the explanation of sequence generation models is also an important problem, however it has seen little attention. In this work, we study model-agnostic explanations of a representative text generation task -- dialogue response generation. Dialog response generation is challenging with its open-ended sentences and multiple acceptable responses. To gain insights into the reasoning process of a generation model, we propose a new method, local explanation of response generation (LERG) that regards the explanations as the mutual interaction of segments in input and output sentences. LERG views the sequence prediction as uncertainty estimation of a human response and then creates explanations by perturbing the input and calculating the certainty change over the human response. We show that LERG adheres to desired properties of explanations for text generation including unbiased approximation, consistency and cause identification. Empirically, our results show that our method consistently improves other widely used methods on proposed automatic- and human- evaluation metrics for this new task by 4.4-12.8%. Our analysis demonstrates that LERG can extract both explicit and implicit relations between input and output segments.
Neural Networks for Partially Linear Quantile Regression

Deep learning has enjoyed tremendous success in a variety of applications but its application to quantile regressions remains scarce. A major advantage of the deep learning approach is its flexibility to model complex data in a more parsimonious way than nonparametric smoothing methods. However, while deep learning brought breakthroughs in prediction, it often lacks interpretability due to the black-box nature of multilayer structure with millions of parameters, hence it is not well suited for statistical inference. In this paper, we leverage the advantages of deep learning to apply it to quantile regression where the goal to produce interpretable results and perform statistical inference. We achieve this by adopting a semiparametric approach based on the partially linear quantile regression model, where covariates of primary interest for statistical inference are modelled linearly and all other covariates are modelled nonparametrically by means of a deep neural network. In addition to the new methodology, we provide theoretical justification for the proposed model by establishing the root-$n$ consistency and asymptotically normality of the parametric coefficient estimator and the minimax optimal convergence rate of the neural nonparametric function estimator. Across several simulated and real data examples, our proposed model empirically produces superior estimates and more accurate predictions than various alternative approaches.
Observational prospects for phase transitions at LISA: Fisher matrix analysis

A first order phase transition at the electroweak scale would lead to the production of gravitational waves that may be observable at upcoming space-based gravitational wave (GW) detectors such as LISA (Laser Interferometer Space Antenna). As the Standard Model has no phase transition, LISA can be used to search for new physics by searching for a stochastic gravitational wave background. In this work we investigate LISA's sensitivity to the thermodynamic parameters encoded in the stochastic background produced by a phase transition, using the sound shell model to characterise the gravitational wave power spectrum, and the Fisher matrix to estimate uncertainties. We explore a parameter space with transition strengths 0.01 <α< 0.5 and phase boundary speeds 0.4 <v w< 0.9, for transitions nucleating at T n = 100 GeV, with mean bubble spacings 0.1 and 0.01 of the Hubble length, and sound speed c/√(3). We show that the power spectrum in the sound shell model can be well approximated by a four-parameter double broken power law, and find that the peak power and frequency can be measured to approximately 10% accuracy for signal-to-noise ratios (SNRs) above 20. Determinations of the underlying thermodynamic parameters are complicated by degeneracies, but in all cases the phase boundary speed will be the best constrained parameter. Turning to the principal components of the Fisher matrix, a signal-to-noise ratio above 20 produces a relative uncertainty less than 3% in the two highest-order principal components, indicating good prospects for combinations of parameters. The highest-order principal component is dominated by the wall speed. These estimates of parameter sensitivity provide a preliminary accuracy target for theoretical calculations of thermodynamic parameters.
LoTSS jellyfish galaxies

Ram pressure stripping is a frequently cited mechanism for quenching galaxy star formation in dense environments. Numerous examples of ram pressure stripping in galaxy clusters are present in literature; however, substantially less work has been focused on ram pressure stripping in lower-mass groups, the most common galaxy environment in the local Universe. In this work we use the LOFAR Two-metre Sky Survey (LoTSS) to search for jellyfish galaxies (i.e., galaxies with ram pressure stripped tails extending beyond the optical disk) in ∼500 SDSS groups (z < 0.05), making this the most comprehensive search for ram pressure stripping in groups to date. We identify 60 jellyfish galaxies in groups with extended, asymmetric radio continuum tails, which are found across the entire range of group mass from 1012.5 < Mgroup < 1014 h−1 M⊙. We compare the group jellyfish galaxies identified in this work with the LoTSS jellyfish galaxies in clusters presented in a recent work, allowing us to compare the effects of ram pressure stripping across three decades in group and cluster mass. We find that jellyfish galaxies are most commonly found in clusters, with the frequency decreasing towards the lowest-mass groups. Both the orientation of observed radio continuum tails and the positions of group jellyfish galaxies in phase space suggest that galaxies are stripped more slowly in groups relative to clusters. Finally, we find that the star formation rates of jellyfish galaxies in groups are consistent with ‘normal’ star-forming group galaxies, which is in contrast to cluster jellyfish galaxies that have clearly enhanced star formation rates. On the whole, there is clear evidence for ongoing ram pressure stripping in galaxy groups (down to very low group masses), though the frequency of jellyfish galaxies and the strength of ram pressure stripping appear lower in groups than in clusters. Differences in the efficiency of ram pressure stripping in groups versus clusters likely contribute to the positive trend between the quenched fraction and host halo mass observed in the local Universe.
K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernets

In one-shot weight sharing for NAS, the weights of each operation (at each layer) are supposed to be identical for all architectures (paths) in the supernet. However, this rules out the possibility of adjusting operation weights to cater for different paths, which limits the reliability of the evaluation results. In this paper, instead of counting on a single supernet, we introduce $K$-shot supernets and take their weights for each operation as a dictionary. The operation weight for each path is represented as a convex combination of items in a dictionary with a simplex code. This enables a matrix approximation of the stand-alone weight matrix with a higher rank ($K>1$). A \textit{simplex-net} is introduced to produce architecture-customized code for each path. As a result, all paths can adaptively learn how to share weights in the $K$-shot supernets and acquire corresponding weights for better evaluation. $K$-shot supernets and simplex-net can be iteratively trained, and we further extend the search to the channel dimension. Extensive experiments on benchmark datasets validate that K-shot NAS significantly improves the evaluation accuracy of paths and thus brings in impressive performance improvements.
Soft pinning: Experimental validation of static correlations in supercooled molecular glass-forming liquids

Abstract Enormous enhancement in the viscosity of a liquid near its glass transition is a hallmark of glass transition. Within a class of theoretical frameworks, it is connected to growing many-body static correlations near the transition, often called “amorphous ordering. ” At the same time, some theories do not invoke the existence of such a static length scale in the problem. Thus, proving the existence and possible estimation of the static length scales of amorphous order in different glass-forming liquids is very important to validate or falsify the predictions of these theories and unravel the true physics of glass formation. Experiments on molecular glass-forming liquids become pivotal in this scenario as the viscosity grows several folds (∼1014), and simulations or colloidal glass experiments fail to access these required long-time scales. Here we design an experiment to extract the static length scales in molecular liquids using dilute amounts of another large molecule as a pinning site. Results from dielectric relaxation experiments on supercooled Glycerol with different pinning concentrations of Sorbitol and Glucose, as well as the simulations on a few model glass-forming liquids with pinning sites, indicate the versatility of the proposed method, opening possible new avenues to study the physics of glass transition in other molecular liquids.
Label Noise SGD Provably Prefers Flat Global Minimizers

In overparametrized models, the noise in stochastic gradient descent (SGD) implicitly regularizes the optimization trajectory and determines which local minimum SGD converges to. Motivated by empirical studies that demonstrate that training with noisy labels improves generalization, we study the implicit regularization effect of SGD with label noise. We show that SGD with label noise converges to a stationary point of a regularized loss L ( θ )+ λR ( θ ) , where L ( θ ) is the training loss, λ is an effective regularization parameter depending on the step size, strength of the label noise, and the batch size, and R ( θ ) is an explicit regularizer that penalizes sharp minimizers. Our analysis uncovers an additional regularization effect of large learning rates beyond the linear scaling rule that penalizes large eigenvalues of the Hessian more than small ones. We also prove extensions to classiﬁcation with general loss functions, signiﬁcantly strengthening the prior work of Blanc et al. [3] to global convergence and large learning rates and of HaoChen et al. [12] to general models.
Diagnostics of homologous solar-surge plasma as observed by IRIS and SDO

Surges have regularly been observed mostly in Hα 6563 Å and Ca ii 8542 Å. However, surges response to other prominent lines of the interface-region (Mg ii k 2796.35 Å & h 2803.52 Å, O iv 1401.15 Å, Si iv 1402.77 Å) is not well studied. Here, the evolution and kinematics of six homologous surges are analysed using IRIS and AIA observations. These surges were observed on 7 July 2014, located very close to the limb. DEM analysis is performed on these surges where the co-existence of cool (log T/K = 6.35) and relatively hot (log T/K = 6.95) components have been found at the base. This demonstrates that the bases of surges undergo substantial heating. During the emission of these surges in the above mentioned interface-region lines, being reported here for the first time, two peaks have been observed in the initial phase of emission, where one peak is found to be constant while other one as varying, i.e., non-constant (observed red to blueshifts across the surge evolution) in nature. This suggests the rotational motion of surge plasma. The heated base and rotating plasma suggests the occurrence of magnetic reconnection, most likely, as the trigger for homologous surges. During the emission of these surges, it is found that despite being optically thick (i.e., Rkh < 2.0), central reversal was not observed for Mg ii k & h lines. Further, Rkh increases with surge emission in time and it is found to have positive correlation with Doppler Velocity while negative with Gaussian width.
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models

Targeted syntactic evaluations have demonstrated the ability of language models to perform subject-verb agreement given difficult contexts. To elucidate the mechanisms by which the models accomplish this behavior, this study applies causal mediation analysis to pre-trained neural language models. We investigate the magnitude of models’ preferences for grammatical inflections, as well as whether neurons process subject-verb agreement similarly across sentences with different syntactic structures. We uncover similarities and differences across architectures and model sizes—notably, that larger models do not necessarily learn stronger preferences. We also observe two distinct mechanisms for producing subject-verb agreement depending on the syntactic structure of the input sentence. Finally, we find that language models rely on similar sets of neurons when given sentences with similar syntactic structure.
Degenerations of k-positive surface group representations

We introduce k–positive representations, a large class of {1, . , k}– Anosov surface group representations into PGL(E) that share many features with Hitchin representations, and we study their degenerations: unless they are Hitchin, they can be deformed to non-discrete representations, but any limit is at least (k − 3)–positive and irreducible limits are (k − 1)–positive. A major ingredient, of independent interest, is a general limit theorem for positively ratioed representations.
To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs

Due to the discrete nature of words, language GANs require to be optimized from rewards provided by discriminator networks, via reinforcement learning methods. This is a much harder setting than for continuous tasks, which enjoy gradient flows from discriminators to generators, usually leading to dramatic learning instabilities. However, we claim that this can be solved by making discriminator and generator networks cooperate to produce output sequences during training. These cooperative outputs, inherently built to obtain higher discrimination scores, not only provide denser rewards for training, but also form a more compact artificial set for discriminator training, hence improving its accuracy and stability. In this paper, we show that our SelfGAN framework, built on this cooperative principle, outperforms Teacher Forcing and obtains state-of-the-art results on two challenging tasks, Summarization and Question Generation.
Approximation error method for imaging the human head by electrical impedance tomography

This work considers electrical impedance tomography imaging of the human head, with the ultimate goal of locating and classifying a stroke in emergency care. One of the main difficulties in the envisioned application is that the electrode locations and the shape of the head are not precisely known, leading to significant imaging artifacts due to impedance tomography being sensitive to modeling errors. In this study, the natural variations in the geometry of the head and skull are modeled based on a library of head anatomies. The effect of these variations, as well as that of misplaced electrodes, on (absolute) impedance tomography measurements is in turn modeled by the approximation error method. This enables reliably reconstructing the conductivity perturbation caused by the stroke in an average head model, instead of the actual head, relative to its average conductivity levels. The functionality of a certain edge-preferring reconstruction algorithm for locating the stroke is demonstrated via numerical experiments based on simulated three-dimensional data.
Generate, Annotate, and Learn: Generative Models Advance Self-Training and Knowledge Distillation

Semi-Supervised Learning (SSL) has seen success in many application domains, but this success often hinges on the availability of task-speciﬁc unlabeled data. Knowledge distillation (KD) has enabled compressing deep networks and ensembles, achieving the best results when distilling knowledge on fresh task-speciﬁc unlabeled examples. However, task-speciﬁc unlabeled data can be challenging to ﬁnd. We present a general framework called “generate, annotate, and learn (GAL)” that uses unconditional generative models to synthesize in-domain unlabeled data, helping advance SSL and KD on different tasks. To obtain strong task-speciﬁc generative models, we adopt generic generative models, pretrained on open-domain data, and ﬁne-tune them on inputs from speciﬁc tasks. Then, we use existing clas-siﬁers to annotate generated unlabeled examples with soft pseudo labels, which are used for additional training. When self-training is combined with samples generated from GPT-2-large, ﬁne-tuned on the inputs of each GLUE task, we outperform a strong RoBERTa-large baseline on the GLUE benchmark. Moreover, KD on GPT-2 samples yields a new state-of-the-art for 6-layer transformers on the GLUE leaderboard. Finally, self-training with GAL offers signiﬁcant gains on image classiﬁcation on CIFAR-10 and four tabular tasks from the UCI repository.
Courteous Behavior of Automated Vehicles at Unsignalized Intersections Via Reinforcement Learning

The transition from today's mostly human-driven traffic to a purely automated one will be a gradual evolution, with the effect that we will likely experience mixed traffic in the near future. Connected and automated vehicles can benefit human-driven ones and the whole traffic system in different ways, for example by improving collision avoidance and reducing traffic waves. Many studies have been carried out to improve intersection management, a significant bottleneck in traffic, with intelligent traffic signals or exclusively automated vehicles. However, the problem of how to improve mixed traffic at unsignalized intersections has received less attention. In this letter, we propose a novel approach to optimizing traffic flow at intersections in mixed traffic situations using deep reinforcement learning. Our reinforcement learning agent learns a policy for a centralized controller to let connected autonomous vehicles at unsignalized intersections give up their right of way and yield to other vehicles to optimize traffic flow. We implemented our approach and tested it in the traffic simulator SUMO based on simulated and real traffic data. The experimental evaluation demonstrates that our method significantly improves traffic flow through unsignalized intersections in mixed traffic settings and also provides better performance in a wide range of traffic situations compared to the state of the art.
Semi-Supervised and Unsupervised Sense Annotation via Translations

Acquisition of multilingual training data continues to be a challenge in word sense disambiguation (WSD). To address this problem, unsupervised approaches have been proposed to automatically generate sense annotations for training supervised WSD systems. We present three new methods for creating sense-annotated corpora which leverage translations, parallel bitexts, lexical resources, as well as contextual and synset embeddings. Our semi-supervised method applies machine translation to transfer existing sense annotations to other languages. Our two unsupervised methods refine sense annotations produced by a knowledge-based WSD system via lexical translations in a parallel corpus. We obtain state-of-the-art results on standard WSD benchmarks.
Adversarial purification with Score-based generative models

While adversarial training is considered as a standard defense method against adversarial attacks for image classifiers, adversarial purification, which purifies attacked images into clean images with a standalone purification model, has shown promises as an alternative defense method. Recently, an Energy-Based Model (EBM) trained with Markov-Chain Monte-Carlo (MCMC) has been highlighted as a purification model, where an attacked image is purified by running a long Markov-chain using the gradients of the EBM. Yet, the practicality of the adversarial purification using an EBM remains questionable because the number of MCMC steps required for such purification is too large. In this paper, we propose a novel adversarial purification method based on an EBM trained with Denoising Score-Matching (DSM). We show that an EBM trained with DSM can quickly purify attacked images within a few steps. We further introduce a simple yet effective randomized purification scheme that injects random noises into images before purification. This process screens the adversarial perturbations imposed on images by the random noises and brings the images to the regime where the EBM can denoise well. We show that our purification method is robust against various attacks and demonstrate its state-of-the-art performances.
Gradual Domain Adaptation in the Wild: When Intermediate Distributions are Absent

We focus on the problem of domain adaptation when the goal is shifting the model towards the target distribution, rather than learning domain invariant representations. It has been shown that under the following two assumptions: (a) access to samples from intermediate distributions, and (b) samples being annotated with the amount of change from the source distribution, self-training can be successfully applied on gradually shifted samples to adapt the model toward the target distribution. We hypothesize having (a) is enough to enable iterative self-training to slowly adapt the model to the target distribution, by making use of an implicit curriculum. In the case where (a) does not hold, we observe that iterative self-training falls short. We propose GIFT, a method that creates virtual samples from intermediate distributions by interpolating representations of examples from source and target domains. We evaluate an iterative-self-training method on datasets with natural distribution shifts, and show that when applied on top of other domain adaptation methods, it improves the performance of the model on the target dataset. We run an analysis on a synthetic dataset to show that in the presence of (a) iterative-self-training naturally forms a curriculum of samples. Furthermore, we show that when (a) does not hold, GIFT performs better than iterative self-training.
Within-layer Diversity Reduces Generalization Gap

Neural networks are composed of multiple layers arranged in a hierarchical structure jointly trained with a gradient-based optimization. At each optimization step, neurons at a given layer receive feedback from neurons belonging to higher layers of the hierarchy. In this paper, we propose to complement this traditional ’between-layer’ feed-back with additional ’within-layer’ feedback to encourage diversity of the activations within the same layer. To this end, we measure the pairwise similarity between the outputs of the neurons and use it to model the layer’s overall diversity. By pe-nalizing similarities and promoting diversity, we encourage each neuron to learn a distinctive representation and, thus, to enrich the data representation learned within the layer and to increase the total capacity of the model. We theoretically and empirically study how the within-layer activation diversity affects the generalization performance of a neural network and prove that increasing the diversity of hidden activations reduces the generalization gap.