Miracle-dz/UniFAR-FTUs huggingface dataset:
SIRe-Networks: Skip Connections over Interlaced Multi-Task Learning and Residual Connections for Structure Preserving Object Classification

Improving existing neural network architectures can involve several design choices such as manipulating the loss functions, employing a diverse learning strategy, exploiting gradient evolution at training time, optimizing the network hyper-parameters, or increasing the architecture depth. The latter approach is a straightforward solution, since it directly enhances the representation capabilities of a network; however, the increased depth generally incurs in the well-known vanishing gradient problem. In this paper, borrowing from different methods addressing this issue, we introduce an interlaced multi-task learning strategy, deﬁned SIRe, to reduce the vanishing gradient in relation to the object classiﬁcation task. The presented methodology directly improves a convolutional neural network (CNN) by enforcing the input image structure preservation through interlaced auto-encoders, and further reﬁnes the base network architecture by means of skip and residual connections. To validate the presented methodology, a simple CNN and various implementations of famous networks are extended via the SIRe strategy and extensively tested on the CIFAR100 dataset; where the SIRe-extended architectures achieve signiﬁcantly increased performances across all models, thus conﬁrming the presented approach effectiveness.
Contrastive Learning for Unsupervised Radar Place Recognition

We present a method for unsupervised learning of single-scan radar embeddings from sequences of radar images that is suitable for solving the place recognition problem with complex radar data. Our method is based on invariant instance feature learning but is tailored for the task of re-localisation by exploiting for data augmentation the temporal successivity of data as collected by a mobile platform moving through the scene smoothly. We experiment across two prominent urban radar datasets totalling over 400 km of driving and show that we achieve a new radar place recognition state-of-the-art. Specifically, the proposed system proves correct for 98.38 % of the queries that it is presented with over a challenging re-localisation sequence, using only the single nearest neighbour in the learned metric space. We also find that our learned model shows better understanding of out-of-lane loop closures at arbitrary orientation than non-learned radar scan descriptors.
VC dimension of partially quantized neural networks in the overparametrized regime

Vapnik-Chervonenkis (VC) theory has so far been unable to explain the small generalization error of overparametrized neural networks. Indeed, existing applications of VC theory to large networks obtain upper bounds on VC dimension that are proportional to the number of weights, and for a large class of networks, these upper bound are known to be tight. In this work, we focus on a class of partially quantized networks that we refer to as hyperplane arrangement neural networks (HANNs). Using a sample compression analysis, we show that HANNs can have VC dimension significantly smaller than the number of weights, while being highly expressive. In particular, empirical risk minimization over HANNs in the overparametrized regime achieves the minimax rate for classification with Lipschitz posterior class probability. We further demonstrate the expressivity of HANNs empirically. On a panel of 121 UCI datasets, overparametrized HANNs match the performance of state-of-the-art full-precision models.
Local integrals of motion in quasiperiodic many-body localized systems

Local integrals of motion play a central role in the understanding of many-body localization in many-body quantum systems in one dimension subject to a random external potential, but the question of how these local integrals of motion change in a deterministic quasiperiodic potential is one that has received significantly less attention. Here we develop a powerful new implementation of the continuous unitary transform formalism and use this method to directly compute both the effective Hamiltonian and the local integrals of motion for many-body quantum systems subject to a quasiperiodic potential. We show that the effective interactions between local integrals of motion retain a strong fingerprint of the underlying quasiperiodic potential, exhibiting sharp features at distances associated with the incommensurate wavelength used to generate the potential. Furthermore, the local integrals of motion themselves may be expressed in terms of an operator expansion which allows us to estimate the critical strength of quasiperiodic potential required to lead to a localization/delocalization transition, by means of a finite size scaling analysis.
The quadrupole moment of compact binaries to the fourth post-Newtonian order: II. Dimensional regularization and renormalization

The regularization and renormalization of the radiative mass-type quadrupole moment of inspiralling compact binaries (without spins) is investigated at the fourth post-Newtonian (4PN) approximation of general relativity. As clear from the conservative 4PN equations of motion, a dimensional regularization has to be implemented in order to properly treat the non-linear interactions experienced by gravitational waves during their propagation towards future null infinity. By implementing such procedure, we show that the poles coming from the source moment (computed in a companion paper) are exactly cancelled in the radiative moment, as expected for a physical quantity. We thus define and obtain a ‘renormalized’ source quadrupole, three-dimensional by nature, which is an important step towards the computation of the gravitational-wave flux with 4PN accuracy. Furthermore, we explicitly prove the equivalence between the dimensional regularization and the previously used Hadamard partie finie scheme up to the 3PN order.
Capturing Structural Locality in Non-parametric Language Models

Structural locality is a ubiquitous feature of real-world datasets, wherein data points are organized into local hierarchies. Some examples include topical clusters in text or project hierarchies in source code repositories. In this paper, we explore utilizing this structural locality within non-parametric language models, which generate sequences that reference retrieved examples from an external source. We propose a simple yet effective approach for adding locality information into such models by adding learned parameters that improve the likelihood of retrieving examples from local neighborhoods. Experiments on two different domains, Java source code and Wikipedia text, demonstrate that locality features improve model efficacy over models without access to these features, with interesting differences. We also perform an analysis of how and where locality features contribute to improved performance and why the traditionally used contextual similarity metrics alone are not enough to grasp the locality structure.
Language Modeling using LMUs: 10x Better Data Efficiency or Improved Scaling Compared to Transformers

Recent studies have demonstrated that the performance of transformers on the task of language modeling obeys a power-law relationship with model size over six orders of magnitude. While transformers exhibit impressive scaling, their performance hinges on processing large amounts of data, and their computational and memory requirements grow quadratically with sequence length. Motivated by these considerations, we construct a Legendre Memory Unit based model that introduces a general prior for sequence processing and exhibits an $O(n)$ and $O(n \ln n)$ (or better) dependency for memory and computation respectively. Over three orders of magnitude, we show that our new architecture attains the same accuracy as transformers with 10x fewer tokens. We also show that for the same amount of training our model improves the loss over transformers about as much as transformers improve over LSTMs. Additionally, we demonstrate that adding global self-attention complements our architecture and the augmented model improves performance even further.
Contextual Combinatorial Volatile Bandits via Gaussian Processes

We consider a contextual bandit problem with a combinatorial action set and time-varying base arm availability. At the beginning of each round, the agent observes the set of available base arms and their contexts and then selects an action that is a feasible subset of the set of available base arms to maximize its cumulative reward in the long run. We assume that the mean outcomes of base arms are samples from a Gaussian Process indexed by the context set X , and the expected reward is Lipschitz continuous in expected base arm outcomes. For this setup, we propose an algorithm called Optimistic Combinatorial Learning and Optimization with Kernel Upper Conﬁdence Bounds (O’CLOK-UCB) and prove that it incurs ˜ O ( K (cid:112) Tγ T ) regret with high probability, where γ T is the maximum information gain associated with the set of base arm contexts that appeared in the ﬁrst T rounds and K is the maximum cardinality of any feasible action over all rounds. To dramatically speed up the algorithm, we also propose a variant of O’CLOK-UCB that uses sparse GPs. Finally, we experimentally show that both algorithms exploit inter-base arm outcome correlation and vastly outperform the previous state-of-the-art UCB-based algorithms in realistic setups.
Transient flows and migration in granular suspensions: key role of Reynolds-like dilatancy

Abstract We investigate the transient dynamics of a sheared suspension of neutrally buoyant particles in pressure-imposed rheology configuration, subject to a sudden change in shear rate or external pressure. Discrete element method simulations show that, depending on the flow parameters (particle and system size, initial volume fraction), the early stress response of the suspension may differ strongly from the prediction of the suspension balance model based on the steady-state rheology. We show that a two-phase model incorporating the Reynolds-like dilatancy law of Pailha & Pouliquen (J. Fluid Mech., vol. 633, 2009, pp. 115–135), which prescribes the dilation rate of the suspension over a strain scale $\gamma _0$, captures quantitatively the suspension dilation/compaction over the whole range of parameters investigated. Together with the Darcy flow induced by the pore pressure gradient during dilation or compaction, this Reynolds-like dilatancy implies that the early stress response of the suspension is non-local, with a non-local length scale $\ell$ that scales with the particle size and diverges algebraically at jamming. In regions affected by $\ell$, the stress level is fixed, not by the steady-state rheology, but by the Darcy fluid pressure gradient resulting from the dilation/compaction rate. Our results extend the validity of the Reynolds-like dilatancy flow rule, initially proposed for jammed suspensions, to flowing suspension below the critical volume fraction at which the suspension jams, thereby providing a unified framework to describe dilation and shear-induced migration. They pave the way for understanding more complex unsteady flows of dense suspensions, such as impacts, transient avalanches or the impulsive response of shear-thickening suspensions.
Secure Byzantine-Robust Distributed Learning via Clustering

Federated learning systems that jointly preserve Byzantine robustness and privacy have remained an open problem. Robust aggregation, the standard defense for Byzantine attacks, generally requires server access to individual updates or nonlinear computation -- thus is incompatible with privacy-preserving methods such as secure aggregation via multiparty computation. To this end, we propose SHARE (Secure Hierarchical Robust Aggregation), a distributed learning framework designed to cryptographically preserve client update privacy and robustness to Byzantine adversaries simultaneously. The key idea is to incorporate secure averaging among randomly clustered clients before filtering malicious updates through robust aggregation. Experiments show that SHARE has similar robustness guarantees as existing techniques while enhancing privacy.
Jets in Common Envelopes: A low mass main sequence star in a red giant

We present small-scale three-dimensional hydrodynamical simulations of the evolution of a 0.3 M⊙ main sequence star which launches two perpendicular jets within the envelope of a 0.88 M⊙ red giant. Based on previous large-scale simulations, we study the dynamics of the jets either when the secondary star is grazing, when it has plunged-in, or when it is well-within the envelope of the red giant (in each stage for ∼11 days). The dynamics of the jets through the common envelope (CE) depend on the conditions of the environment as well as on their powering. In the grazing stage and the commencement of the plunge self-regulated jets need higher efficiencies to break out of the envelope of the RG. Deep inside the CE, on the timescales simulated, jets are choked independently of whether they are self-regulated or constantly powered. Jets able to break out of the envelope of the RG in large-scale simulations, are choked in our small-scale simulations. The accreted angular momentum on to the secondary star is not large enough to form a disk. The mass accretion on to the MS star is 1-10 per cent of the Bondi-Hoyle-Littleton rate (∼10−3–10−1 M⊙ yr−1). High luminosity emission, from X-rays to UV and optical, is expected if the jets break out of the CE. Our simulations illustrate the need for inclusion of more realistic accretion and jet models in the dynamical evolution of the CEs.
Hardy spaces associated with ball quasi‐Banach function spaces on spaces of homogeneous type: Characterizations of maximal functions, decompositions, and dual spaces

Let (X,ρ,μ)$({\mathcal {X}},\rho ,\mu )$ be a space of homogeneous type in the sense of Coifman and Weiss, and let Y(X)$Y({\mathcal {X}})$ be a ball quasi‐Banach function space on X${\mathcal {X}}$ , which supports both a Fefferman–Stein vector‐valued maximal inequality and the boundedness of the powered Hardy–Littlewood maximal operator on its associate space. The authors first introduce the Hardy space HY∗(X)$H_{Y}^*({\mathcal {X}})$ , associated with Y(X)$Y({\mathcal {X}})$ , via the grand maximal function and then establish its various real‐variable characterizations, respectively, in terms of radial or nontangential maximal functions, atoms or finite atoms, and molecules. As an application, the authors give the dual space of HY∗(X)$H_{Y}^*({\mathcal {X}})$ , which proves to be a ball Campanato‐type function space associated with Y(X)$Y({\mathcal {X}})$ . All these results have a wide range of generality and, particularly, even when they are applied to variable Hardy spaces, the obtained results are also new. The major novelties of this paper exist in that, to escape both the reverse doubling condition of μ and the triangle inequality of ρ, the authors cleverly construct admissible sequences of balls and fully use the geometrical properties of X${\mathcal {X}}$ expressed by dyadic reference points or dyadic cubes and, to overcome the difficulty caused by the lack of the good dense subset of HY∗(X)$H_{Y}^*({\mathcal {X}})$ , the authors further prove that Y(X)$Y({\mathcal {X}})$ can be embedded into the weighted Lebesgue space with certain special weight and then can fully use the known results of the weighted Lebesgue space.
The Dark Energy Survey Bright Arcs Survey: Candidate Strongly Lensed Galaxy Systems from the Dark Energy Survey 5000 Square Degree Footprint

We report the combined results of eight searches for strong gravitational lens systems in the full 5000 square degrees of Dark Energy Survey (DES) observations. The observations accumulated by the end of the third observing season fully covered the DES footprint in five filters (grizY), with an i-band limiting magnitude (at 10σ) of 23.44. In four searches, a list of potential candidates was identified using a color and magnitude selection from the object catalogs created from the first three observing seasons. Three other searches were conducted at the locations of previously identified galaxy clusters. Cutout images of potential candidates were then visually scanned using an object viewer. An additional set of candidates came from a data-quality check of a subset of the color–coadd tiles created from the full DES six-season data set. A short list of the most promising strong-lens candidates was then numerically ranked according to whether or not we judged them to be bona fide strong gravitational lens systems. These searches discovered a diverse set of 247 strong-lens candidate systems, of which 81 are identified for the first time. We provide the coordinates, magnitudes, and photometric properties of the lens and source objects, and an estimate of the Einstein radius for 81 new systems and 166 previously reported systems. This catalog will be of use for selecting interesting systems for detailed follow up, studies of galaxy cluster and group mass profiles, as well as a training/validation set for automated strong-lens searches.
Interactively Generating Explanations for Transformer Language Models

Transformer language models are state-of-the-art in a multitude of NLP tasks. Despite these successes, their opaqueness remains problematic. Recent methods aiming to provide interpretability and explainability to black-box models primarily focus on post-hoc explanations of (sometimes spurious) input-output correlations. Instead, we emphasize using prototype networks directly incorporated into the model architecture and hence explain the reasoning process behind the network’s decisions. Moreover, while our architecture performs on par with several language models, it enables one to learn from user interactions. This not only offers a better understanding of language models, but uses human capabilities to incorporate knowledge outside of the rigid range of purely data-driven approaches.
MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning

Inspired by human visual attention, we propose a novel inverse reinforcement learning formulation using Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) for predicting the visual attention of drivers in accident-prone situations. MEDIRL predicts fixation locations that lead to maximal rewards by learning a task-sensitive reward function from eye fixation patterns recorded from attentive drivers. Additionally, we introduce EyeCar, a new driver attention dataset in accident-prone situations. We conduct comprehensive experiments to evaluate our proposed model on three common benchmarks: (DR(eye)VE, BDD-A, DADA-2000), and our EyeCar dataset. Results indicate that MEDIRL outperforms existing models for predicting attention and achieves state-of-the-art performance. We present extensive ablation studies to provide more insights into different features of our proposed model.1
Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection

In industrial manufacturing processes, errors frequently occur at unpredictable times and in unknown manifestations. We tackle the problem of automatic defect detection without requiring any image samples of defective parts. Recent works model the distribution of defect-free image data, using either strong statistical priors or overly simplified data representations. In contrast, our approach handles fine-grained representations incorporating the global and local image context while flexibly estimating the density. To this end, we propose a novel fully convolutional cross-scale normalizing flow (CS-Flow) that jointly processes multiple feature maps of different scales. Using normalizing flows to assign meaningful likelihoods to input samples allows for efficient defect detection on image-level. Moreover, due to the preserved spatial arrangement the latent space of the normalizing flow is interpretable which enables to localize defective regions in the image. Our work sets a new state-of-the-art in image-level defect detection on the benchmark datasets Magnetic Tile Defects and MVTec AD showing a 100% AUROC on 4 out of 15 classes.
ParaDiS: Parallelly Distributable Slimmable Neural Networks

When several limited power devices are available, one of the most efficient ways to make profit of these resources, while reducing the processing latency and communication load, is to run in parallel several neural sub-networks and to fuse the result at the end of processing. However, such a combination of sub-networks must be trained specifically for each particular configuration of devices (characterized by number of devices and their capacities) which may vary over different model deployments and even within the same deployment. In this work we introduce parallelly distributable slimmable (ParaDiS) neural networks that are splittable in parallel among various device configurations without retraining. While inspired by slimmable networks allowing instant adaptation to resources on just one device, ParaDiS networks consist of several multi-device distributable configurations or switches that strongly share the parameters between them. We evaluate ParaDiS framework on MobileNet v1 and ResNet-50 architectures on ImageNet classification task and WDSR architecture for image super-resolution task. We show that ParaDiS switches achieve similar or better accuracy than the individual models, i.e., distributed models of the same structure trained individually. Moreover, we show that, as compared to universally slimmable networks that are not distributable, the accuracy of distributable ParaDiS switches either does not drop at all or drops by a maximum of 1 % only in the worst cases. Finally, once distributed over several devices, ParaDiS outperforms greatly slimmable models.
Disambiguation-BERT for N-best Rescoring in Low-Resource Conversational ASR

We study the inclusion of past conversational context through BERT language models into a CTC-based Automatic Speech Recognition (ASR) system via N-best rescoring. We introduce a data-eﬃcient strategy to ﬁne-tune BERT on transcript disambiguation without external data. Our results show word error rate recoveries up to 37.2% with context-augmented BERT rescoring. We do this in low-resource data domains, both in language (Norwegian), tone (spontaneous, conversational), and topics (parliament proceedings and customer service phone calls). We show how the nature of the data greatly aﬀects the performance of context-augmented N-best rescoring.
Discovery, TESS Characterization, and Modeling of Pulsations in the Extremely Low-mass White Dwarf GD 278

We report the discovery of pulsations in the extremely low-mass (ELM), likely helium-core white dwarf GD 278 via ground- and space-based photometry. GD 278 was observed by the Transiting Exoplanet Survey Satellite (TESS) in Sector 18 at a 2 minute cadence for roughly 24 days. The TESS data reveal at least 19 significant periodicities between 2447 and 6729 s, one of which is the longest pulsation period ever detected in a white dwarf. Previous spectroscopy found that this white dwarf is in a 4.61 hr orbit with an unseen >0.4 M ⊙ companion and has T eff = 9230 ± 100 K and logg=6.627±0.056 , which corresponds to a mass of 0.191 ± 0.013 M ⊙. Patterns in the TESS pulsation frequencies from rotational splittings appear to reveal a stellar rotation period of roughly 10 hr, making GD 278 the first ELM white dwarf with a measured rotation rate. The patterns inform our mode identification for asteroseismic fits, which, unfortunately, do not reveal a global best-fit solution. Asteroseismology reveals two main solutions roughly consistent with the spectroscopic parameters of this ELM white dwarf, but with vastly different hydrogen-layer masses; future seismic fits could be further improved by using the stellar parallax. GD 278 is now the tenth known pulsating ELM white dwarf; it is only the fifth known to be in a short-period binary, but is the first with extended, space-based photometry.