Miracle-dz/UniFAR-FTUs huggingface dataset:
Expression Acceleration: Seamless Parallelization of Typed High-Level Languages

Efficient parallelization of algorithms on general-purpose GPUs is today essential in many areas. However, it is a non-trivial task for software engineers to utilize GPUs to im-prove the performance of high-level programs in general. Although many domain-specific approaches are available for GPU acceleration, it is difficult to accelerate existing high-level programs without rewriting parts of the programs using low-level GPU code. In this paper, we propose a different approach, where expressions are marked for acceleration, and the compiler automatically infers which code needs to be accelerated. We call this approach expression acceleration . We design a compiler pipeline for the approach and show how to handle several challenges, including expression extraction, well-formedness, and compiling using multiple backends. The approach is designed and implemented within a statically-typed functional intermediate language and eval-uated using three distinct non-trivial case studies.
DOLPH: Diffusion Models for Phase Retrieval

Phase retrieval refers to the problem of recovering an image from the magnitudes of its complex-valued linear measurements. Since the problem is ill-posed, the recovery requires prior knowledge on the unknown image. We present DOLPH as a new deep model-based architecture for phase retrieval that integrates an image prior speciﬁed using a diffusion model with a nonconvex data-ﬁdelity term for phase retrieval. Diffusion models are a recent class of deep generative models that are relatively easy to train due to their implementation as image denoisers. DOLPH reconstructs high-quality solutions by alternating data-consistency updates with the sampling step of a diffusion model. Our numerical results show the robustness of DOLPH to noise and its ability to generate several candidate solutions given a set of measurements.
Adaptive Compression for Communication-Efficient Distributed Training

We propose Adaptive Compressed Gradient Descent (AdaCGD) - a novel optimization algorithm for communication-efficient training of supervised machine learning models with adaptive compression level. Our approach is inspired by the recently proposed three point compressor (3PC) framework of Richtarik et al. (2022), which includes error feedback (EF21), lazily aggregated gradient (LAG), and their combination as special cases, and offers the current state-of-the-art rates for these methods under weak assumptions. While the above mechanisms offer a fixed compression level, or adapt between two extremes only, our proposal is to perform a much finer adaptation. In particular, we allow the user to choose any number of arbitrarily chosen contractive compression mechanisms, such as Top-K sparsification with a user-defined selection of sparsification levels K, or quantization with a user-defined selection of quantization levels, or their combination. AdaCGD chooses the appropriate compressor and compression level adaptively during the optimization process. Besides i) proposing a theoretically-grounded multi-adaptive communication compression mechanism, we further ii) extend the 3PC framework to bidirectional compression, i.e., we allow the server to compress as well, and iii) provide sharp convergence bounds in the strongly convex, convex and nonconvex settings. The convex regime results are new even for several key special cases of our general mechanism, including 3PC and EF21. In all regimes, our rates are superior compared to all existing adaptive compression methods.
SADT: Combining Sharpness-Aware Minimization with Self-Distillation for Improved Model Generalization

Methods for improving deep neural network training times and model generalizability consist of various data augmentation, regularization, and optimization approaches, which tend to be sensitive to hyperparameter settings and make reproducibility more challenging. This work jointly considers two recent training strategies that address model generalizability: sharpness-aware minimization, and self-distillation, and proposes the novel training strategy of Sharpness-Aware Distilled Teachers (SADT). The experimental section of this work shows that SADT consistently outperforms previously published training strategies in model convergence time, test-time performance, and model generalizability over various neural architectures, datasets, and hyperparameter settings. process is a progressive regularization method that needs to be studied further [8]. This work presents a study on the fundamental mechanisms of both sharpness-aware minimization and self-distillation methods, and consequently proposes a structured family of training strategies to improve training performance. The proposed Sharpness-Aware Distilled Teachers (SADT) approach creates an improved variant of the teacher model from the original teacher model within a single distillation round. Consequently, we show that SADT achieves considerable improvement in convergence speed and generalizability over other works [1, 9, 10] that operate in a single training round. The contributions of this paper are:
On the Analysis and Optimization of Fast Conditional Handover With Hand Blockage for Mobility

Although frequency range 2 (FR2) systems are an essential part of 5G-Advanced and future 3GPP releases, the mobility performance of multi-panel user equipment (MPUE) with hand blockage is still an area open for research and standardization. In this article, a comprehensive study on the mobility performance of MPUE with hand blockage is performed for conditional handover (CHO) and its potential enhancement denoted by fast conditional handover (FCHO). In contrast to CHO, in FCHO the MPUE can reuse earlier target cell preparations after each handover to autonomously execute subsequent handovers. This saves both the signaling overhead associated with the reconfiguration and re-preparation of target cells after each handover and reduces mobility failures. Results have shown that FCHO offers considerable mobility performance gains as compared to CHO for different hand blockage cases that are dependent on the hand position around the MPUE. For the worst-case hand blockage scenario, it is seen that mobility failures reduce by 10.5% and 19.3% for the 60 km/h and 120 km/h mobility scenarios, respectively. This gain comes at the expense of reserving the handover resources of an MPUE for a longer time given that the target cell configurations are not necessarily released after each handover. In this article, the longer resource reservation problem in FCHO is analysed and three different resource reservation optimization techniques are introduced. Results have shown that these optimization techniques not only reduce the resource reservation time but also significantly reduce the signaling overhead at the possible expense of a tolerable degradation in mobility performance.
Entity Matching by Pool-based Active Learning

The goal of entity matching is to find the corresponding records representing the same entity from different data sources. At present, in the mainstream methods, rule-based entity matching methods need tremendous domain knowledge. Machine-learning-based or deep-learning-based entity matching methods need a large number of labeled samples to build the model, which is difficult to achieve in some applications. In addition, learning-based methods are more likely to overfit, so the quality requirements of training samples are very high. In this paper, we present an active learning method for entity matching tasks. This method needs to manually label only a small number of valuable samples, and use these labeled samples to build a model with high quality. This paper proposes hybrid uncertainty as a query strategy to find those valuable samples for labeling, which can minimize the number of labeled training samples and at the same time meet the requirements of entity matching tasks. The proposed method is validated on seven data sets in different fields. The experiments show that the proposed method uses only a small number of labeled samples and achieves better effects compared to current existing approaches.
Birth–death dynamics for sampling: global convergence, approximations and their asymptotics

Motivated by the challenge of sampling Gibbs measures with nonconvex potentials, we study a continuum birth–death dynamics. We improve results in previous works (Liu et al 2023 Appl. Math. Optim. 87 48; Lu et al 2019 arXiv:1905.09863) and provide weaker hypotheses under which the probability density of the birth–death governed by Kullback–Leibler divergence or by χ 2 divergence converge exponentially fast to the Gibbs equilibrium measure, with a universal rate that is independent of the potential barrier. To build a practical numerical sampler based on the pure birth–death dynamics, we consider an interacting particle system, which is inspired by the gradient flow structure and the classical Fokker–Planck equation and relies on kernel-based approximations of the measure. Using the technique of Γ-convergence of gradient flows, we show that on the torus, smooth and bounded positive solutions of the kernelised dynamics converge on finite time intervals, to the pure birth–death dynamics as the kernel bandwidth shrinks to zero. Moreover we provide quantitative estimates on the bias of minimisers of the energy corresponding to the kernelised dynamics. Finally we prove the long-time asymptotic results on the convergence of the asymptotic states of the kernelised dynamics towards the Gibbs measure.
The future is different: Large pre-trained language models fail in prediction tasks

Large pre-trained language models (LPLM) have shown spectacular success when fine-tuned on downstream supervised tasks. Yet, it is known that their performance can drastically drop when there is a distribution shift between the data used during training and that used at inference time. In this paper we focus on data distributions that naturally change over time and introduce four new REDDIT datasets, namely the WALLSTREETBETS, ASKSCIENCE, THE DONALD, and POLITICS sub-reddits. First, we empirically demonstrate that LPLM can display average performance drops of about 88% (in the best case! ) when predicting the popularity of future posts from sub-reddits whose topic distribution changes with time. We then introduce a simple methodology that leverages neural variational dynamic topic models and attention mechanisms to infer temporal language model representations for regression tasks. Our models display performance drops of only about 40% in the worst cases (2% in the best ones) when predicting the popularity of future posts, while using only about 7% of the total number of parameters of LPLM and providing interpretable representations that offer insight into real-world events, like the GameStop short squeeze of 2021
Extragalactic Magnetism with SOFIA (SALSA Legacy Program): The Magnetic Fields in the Multiphase Interstellar Medium of the Antennae Galaxies

Mergers are thought to be a fundamental channel for galaxy growth, perturbing the gas dynamics and the magnetic fields (B-fields) in the interstellar medium (ISM). However, the mechanisms that amplify and dissipate B-fields during a merger remain unclear. We characterize the morphology of the ordered B-fields in the multiphase ISM of the closest merger of two spiral galaxies, the Antennae galaxies. We compare the inferred B-fields using 154 μm thermal dust and 11 cm radio synchrotron emission polarimetric observations. We find that the 154 μm B-fields are more ordered across the Antennae galaxies than the 11 cm B-fields. The turbulent-to-ordered 154 μm B-field increases at the galaxy cores and star-forming regions. The relic spiral arm has an ordered spiral 154 μm B-field, while the 11 cm B-field is radial. The 154 μm B-field may be dominated by turbulent dynamos with high 12CO(1–0) velocity dispersion driven by star-forming regions, while the 11 cm B-field is cospatial with high H i velocity dispersion driven by galaxy interaction. This result shows the dissociation between the warm gas mainly disturbed by the merger, and the dense gas still following the dynamics of the relic spiral arm. We find a ∼8.9 kpc scale ordered B-field connecting the two galaxies. The base of the tidal tail is cospatial with the H i and 12CO(1–0) emission and has compressed and/or sheared 154 μm and 11 cm B-fields driven by the merger. We suggest that amplified B-fields, with respect to the rest of the system and other spiral galaxies, may be supporting the gas flow between both galaxies and the tidal tail.
Room temperature giant magnetoresistance in La1-xPbxMnO3

Oxides of the La1-xPbxMnO3 system show giant magnetoresistance around 300 K. In La1-xPbxMnO3 the magnitude of the magnetoresistance (MR) at 320 K is 85% at 6 T and it is nearly temperature independent over a temperature range 40-320 K. All the compositions (0.1<or=x<or=0.5) show a peak in resistivity around 300-340 K and the magnitude of the MR also shows a peak around the same temperatures. This investigation suggests how to hole dope LaMnO3 in order to achieve maximum MR around 300 K so that these materials can have practical applications.
Machine learning-based sampling of virtual experiments within the full stress state to identify parameters of anisotropic yield models

This paper presents a new machine learning-based approach to investigate anisotropic yield surfaces of sheet metals by virtual experiments. The new sampling approach is based upon the machine learning technique active learning, which has been adapted to efficiently sample virtual experiments with respect to the full stress state in order to identify parameters of anisotropic yield models. The approach was employed to sample virtual experiments based on the crystal plasticity finite element method (CPFEM) for a DX56D deep drawing steel and compared with two state-of-the-art sampling methods taken from literature. The resulting points on the initial yield surface for all three sampling methods were then used to identify parameters of the anisotropic yield models Hill48, Yld91, Yld2004-18p, and Yld2004-27p. The results show that the new machine learning-based sampling approach has a higher sampling efficiency than the two state-of-the-art sampling methods. It was also discovered that the effect of the sampling method on the resulting yield surfaces for Hill48, Yld91, Yld2004-18p, and Yld2004-27p is less significant than the choice of the anisotropic yield model for a sufficiently large number of sampled yield points. In this respect, Yld2004-27p was identified as being sufficiently flexible to represent the anisotropic yield surface of DX56D within full stress state with reasonable accuracy.
Disentangled representation learning for multilingual speaker recognition

The goal of this paper is to learn robust speaker representation for bilingual speaking scenario. The majority of the world's population speak at least two languages; however, most speaker recognition systems fail to recognise the same speaker when speaking in different languages. Popular speaker recognition evaluation sets do not consider the bilingual scenario, making it difficult to analyse the effect of bilingual speakers on speaker recognition performance. In this paper, we publish a large-scale evaluation set named VoxCeleb1-B derived from VoxCeleb that considers bilingual scenarios. We introduce an effective disentanglement learning strategy that combines adversarial and metric learning-based methods. This approach addresses the bilingual situation by disentangling language-related information from speaker representation while ensuring stable speaker representation learning. Our language-disentangled learning method only uses language pseudo-labels without manual information.
Composite Feature Selection using Deep Ensembles

In many real world problems, features do not act alone but in combination with each other. For example, in genomics, diseases might not be caused by any single mutation but require the presence of multiple mutations. Prior work on feature selection either seeks to identify individual features or can only determine relevant groups from a predefined set. We investigate the problem of discovering groups of predictive features without predefined grouping. To do so, we define predictive groups in terms of linear and non-linear interactions between features. We introduce a novel deep learning architecture that uses an ensemble of feature selection models to find predictive groups, without requiring candidate groups to be provided. The selected groups are sparse and exhibit minimum overlap. Furthermore, we propose a new metric to measure similarity between discovered groups and the ground truth. We demonstrate the utility of our model on multiple synthetic tasks and semi-synthetic chemistry datasets, where the ground truth structure is known, as well as an image dataset and a real-world cancer dataset.
Adversarial Training with Complementary Labels: On the Benefit of Gradually Informative Attacks

Adversarial training (AT) with imperfect supervision is significant but receives limited attention. To push AT towards more practical scenarios, we explore a brand new yet challenging setting, i.e., AT with complementary labels (CLs), which specify a class that a data sample does not belong to. However, the direct combination of AT with existing methods for CLs results in consistent failure, but not on a simple baseline of two-stage training. In this paper, we further explore the phenomenon and identify the underlying challenges of AT with CLs as intractable adversarial optimization and low-quality adversarial examples. To address the above problems, we propose a new learning strategy using gradually informative attacks, which consists of two critical components: 1) Warm-up Attack (Warm-up) gently raises the adversarial perturbation budgets to ease the adversarial optimization with CLs; 2) Pseudo-Label Attack (PLA) incorporates the progressively informative model predictions into a corrected complementary loss. Extensive experiments are conducted to demonstrate the effectiveness of our method on a range of benchmarked datasets. The code is publicly available at: https://github.com/RoyalSkye/ATCL.
Asymptotic symmetry and group invariance for randomization

Symmetry is a cornerstone of much of mathematics, and many probability distributions possess symmetries characterized by their invariance to a collection of group actions. Thus, many mathematical and statistical methods rely on such symmetry holding and ostensibly fail if symmetry is broken. This work considers under what conditions a sequence of probability measures asymptotically gains such symmetry or invariance to a collection of group actions. Considering the many symmetries of the Gaussian distribution, this work effectively proposes a non-parametric type of central limit theorem. That is, a Lipschitz function of a high dimensional random vector will be asymptotically invariant to the actions of certain compact topological groups. Applications of this include a partial law of the iterated logarithm for uniformly random points in an $\ell_p^n$-ball and an asymptotic equivalence between classical parametric statistical tests and their randomization counterparts even when invariance assumptions are violated.
Recognizing Nested Entities from Flat Supervision: A New NER Subtask, Feasibility and Challenges

Many recent named entity recognition (NER) studies criticize flat NER for its non-overlapping assumption, and switch to investigating nested NER. However, existing nested NER models heavily rely on training data annotated with nested entities, while labeling such data is costly. This study proposes a new subtask, nested-from-flat NER, which corresponds to a realistic application scenario: given data annotated with flat entities only, one may still desire the trained model capable of recognizing nested entities. To address this task, we train span-based models and deliberately ignore the spans nested inside labeled entities, since these spans are possibly unlabeled entities. With nested entities removed from the training data, our model achieves 54.8%, 54.2% and 41.1% F1 scores on the subset of spans within entities on ACE 2004, ACE 2005 and GENIA, respectively. This suggests the effectiveness of our approach and the feasibility of the task. In addition, the model's performance on flat entities is entirely unaffected. We further manually annotate the nested entities in the test set of CoNLL 2003, creating a nested-from-flat NER benchmark. Analysis results show that the main challenges stem from the data and annotation inconsistencies between the flat and nested entities.
THE CLUSTER LENSING AND SUPERNOVA SURVEY WITH HUBBLE (CLASH): STRONG-LENSING ANALYSIS OF A383 FROM 16-BAND HST/WFC3/ACS IMAGING

We examine the inner mass distribution of the relaxed galaxy cluster A383 (z = 0.189), in deep 16 band Hubble Space Telescope/ACS+WFC3 imaging taken as part of the Cluster Lensing And Supernova survey with Hubble (CLASH) multi-cycle treasury program. Our program is designed to study the dark matter distribution in 25 massive clusters, and balances depth with a wide wavelength coverage, 2000–16000 Å, to better identify lensed systems and generate precise photometric redshifts. This photometric information together with the predictive strength of our strong-lensing analysis method identifies 13 new multiply lensed images and candidates, so that a total of 27 multiple images of nine systems are used to tightly constrain the inner mass profile gradient, dlog Σ/dlog r ≃ −0.6 ± 0.1 (r < 160 kpc). We find consistency with the standard distance–redshift relation for the full range spanned by the lensed images, 1.01 < z < 6.03, with the higher-redshift sources deflected through larger angles as expected. The inner mass profile derived here is consistent with the results of our independent weak-lensing analysis of wide-field Subaru images, with good agreement in the region of overlap (∼0.7–1 arcmin). Combining weak and strong lensing, the overall mass profile is well fitted by a Navarro–Frenk–White profile with Mvir = (5.37+0.70− 0.63 ± 0.26) × 1014 M☉ h−1 and a relatively high concentration, cvir = 8.77+0.44− 0.42 ± 0.23, which lies above the standard c–M relation similar to other well-studied clusters. The critical radius of A383 is modest by the standards of other lensing clusters, rE ≃ 16 ± 2″ (for zs = 2.55), so the relatively large number of lensed images uncovered here with precise photometric redshifts validates our imaging strategy for the CLASH survey. In total we aim to provide similarly high-quality lensing data for 25 clusters, 20 of which are X-ray-selected relaxed clusters, enabling a precise determination of the representative mass profile free from lensing bias.
Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning

Goal-conditioned reinforcement learning (RL) is a promising direction for training agents that are capable of solving multiple tasks and reach a diverse set of objectives. How to \textit{specify} and \textit{ground} these goals in such a way that we can both reliably reach goals during training as well as generalize to new goals during evaluation remains an open area of research. Defining goals in the space of noisy and high-dimensional sensory inputs poses a challenge for training goal-conditioned agents, or even for generalization to novel goals. We propose to address this by learning factorial representations of goals and processing the resulting representation via a discretization bottleneck, for coarser goal specification, through an approach we call DGRL. We show that applying a discretizing bottleneck can improve performance in goal-conditioned RL setups, by experimentally evaluating this method on tasks ranging from maze environments to complex robotic navigation and manipulation. Additionally, we prove a theorem lower-bounding the expected return on out-of-distribution goals, while still allowing for specifying goals with expressive combinatorial structure.