Miracle-dz/UniFAR-FTUs huggingface dataset:
Less is More: Parameter-Free Text Classification with Gzip

Deep neural networks (DNNs) are often used for text classification tasks as they usually achieve high levels of accuracy. However, DNNs can be computationally intensive with billions of parameters and large amounts of labeled data, which can make them expensive to use, to optimize and to transfer to out-of-distribution (OOD) cases in practice. In this paper, we propose a non-parametric alternative to DNNs that's easy, light-weight and universal in text classification: a combination of a simple compressor like gzip with a $k$-nearest-neighbor classifier. Without any training, pre-training or fine-tuning, our method achieves results that are competitive with non-pretrained deep learning methods on six in-distributed datasets. It even outperforms BERT on all five OOD datasets, including four low-resource languages. Our method also performs particularly well in few-shot settings where labeled data are too scarce for DNNs to achieve a satisfying accuracy.
Study on Domain Name System (DNS) Abuse: Technical Report

A safe and secure Domain Name System (DNS) is of paramount importance for the digital economy and society. Malicious activities on the DNS, generally referred to as"DNS abuse"are frequent and severe problems affecting online security and undermining users' trust in the Internet. The proposed definition of DNS abuse is as follows: Domain Name System (DNS) abuse is any activity that makes use of domain names or the DNS protocol to carry out harmful or illegal activity. DNS abuse exploits the domain name registration process, the domain name resolution process, or other services associated with the domain name (e.g., shared web hosting service). Notably, we distinguish between: maliciously registered domain names: domain name registered with the malicious intent to carry out harmful or illegal activity compromised domain names: domain name registered by bona fide third-party for legitimate purposes, compromised by malicious actors to carry out harmful and illegal activity. DNS abuse disrupts, damages, or otherwise adversely impacts the DNS and the Internet infrastructure, their users or other persons.
A Natural Bias for Language Generation Models

After just a few hundred training updates, a standard probabilistic model for language generation has likely not yet learnt many semantic or syntactic rules of natural language, making it difficult to estimate the probability distribution over next tokens. Yet around this point, these models have identified a simple, loss-minimising behaviour: to output the unigram distribution of the target training corpus. The use of such a heuristic raises the question: Can we initialise our models with this behaviour and save precious compute resources and model capacity? Here we show that we can effectively endow standard neural language generation models with a separate module that reflects unigram frequency statistics as prior knowledge, simply by initialising the bias term in a model’s final linear layer with the log-unigram distribution. We use neural machine translation as a test bed for this simple technique and observe that it: (i) improves learning efficiency; (ii) achieves better overall performance; and perhaps most importantly (iii) appears to disentangle strong frequency effects by encouraging the model to specialise in non-frequency-related aspects of language.
A Fundamental Derivative in Gasdynamics

The quantity which is here called the fundamental derivative has been defined as the nondimensional form Γ≡12ρ3c4(∂2Υ/∂P2)s. The relation of Γ to other thermodynamic variables is discussed. It is already known that the existence of conventional compression shocks requires Γ>0. It is shown that other dynamic behavior of compressible fluids is fixed by the sign of Γ. Particular emphasis is given to phenomena corresponding to negative Γ. These phenomena include the area variation of a transonic passage, the form of a Prandtl‐Meyer wave, the behavior of adiabatic flow with friction, and nonlinear wave propagation. Formulas and numerical values are given for Γ in various substances.
Machine Learning Containers are Bloated and Vulnerable

Today’s software is bloated leading to signiﬁcant resource wastage. This bloat is prevalent across the entire software stack, from the operating system, all the way to software backends, frontends, and web-pages. In this paper, we study how prevalent bloat is in machine learning containers. We develop MMLB, a framework to analyze bloat in machine learning containers, measuring the amount of bloat that exists on the container and package levels. Our tool quantiﬁes the sources of bloat and removes them. We integrate our tool with vulnerability analysis tools to measure how bloat affects container vulnerabilities. We experimentally study 15 machine learning containers from the ofﬁcial Tensorﬂow, Pytorch, and NVIDIA container registries under different tasks, (i.e., training, tuning, and serving). Our ﬁndings show that machine learning containers contain bloat encompassing up to 80% of the container size. We ﬁnd that debloating machine learning containers speeds provisioning times by up to 3 . 7 × and removes up to 98% of all vulnerabilities detected by vulnerability analysis tools such as Grype. Finally, we relate our results to the larger discussion about technical debt in machine learning systems.
Rate-Privacy-Storage Tradeoff in Federated Learning with Top $r$ Sparsification

We investigate the trade-off between rate, privacy and storage in federated learning (FL) with top $r$ sparsification, where the users and the servers in the FL system only share the most significant $r$ and $r^{\prime}$ fractions, respectively, of updates and parameters in the FL process, to reduce the communication cost. We present schemes that guarantee information theoretic privacy of the values and indices of the sparse updates sent by the users at the expense of a larger storage cost. To this end, we generalize the scheme to reduce the storage cost by allowing a certain amount of information leakage. Thus, we provide the general trade-off between the communication cost, storage cost, and information leakage in private FL with top $r$ sparsification, along the lines of two proposed schemes.
Active learning strategies for atomic cluster expansion models

The atomic cluster expansion (ACE) was proposed recently as a new class of data-driven interatomic potentials with a formally complete basis set. Since the development of any interatomic potential requires a careful selection of training data and thorough validation, an automation of the construction of the training dataset as well as an indication of a model's uncertainty are highly desirable. In this work, we compare the performance of two approaches for uncertainty indication of ACE models based on the D-optimality criterion and ensemble learning. While both approaches show comparable predictions, the extrapolation grade based on the D-optimality (MaxVol algorithm) is more computationally efficient. In addition, the extrapolation grade indicator enables an active exploration of new structures, opening the way to the automated discovery of rare-event configurations. We demonstrate that active learning is also applicable to explore local atomic environments from large-scale MD simulations.
Implicit Actions and Non-blocking Failure Recovery with MPI

Scientific applications have long embraced the MPI as the environment of choice to execute on large distributed systems. The User-Level Failure Mitigation (ULFM) specification extends the MPI standard to address resilience and enable MPI applications to restore their communication capability after a failure. This works builds upon the wide body of experience gained in the field to eliminate a gap between current practice and the ideal, more asynchronous, recovery model in which the fault tolerance activities of multiple components can be carried out simultaneously and overlap. This work proposes to: (1) provide the required consistency in fault reporting to applications (i.e., enable an application to assess the success of a computational phase without incurring an unacceptable performance hit); (2) bring forward the building blocks that permit the effective scoping of fault recovery in an application, so that independent components in an application can recover without interfering with each other, and separate groups of processes in the application can recover independently or in unison; and (3) overlap recovery activities necessary to restore the consistency of the system (e.g., eviction of faulty processes from the communication group) with application recovery activities (e.g., dataset restoration from checkpoints).
Dynamics and instabilities in exoplanetary systems

Abstract Extrasolar planet surveys have discovered over two dozen multiple planet systems. As radial velocity searches push towards higher precisions and longer survey durations, they can be expected to discover an even higher fraction of multiple planet systems. Combined with radial velocity data, dynamical studies of these systems can constrain planet masses and inclinations, measure the significance of resonant and secular interactions, and provide insights into the formation and evolution of these systems. Here, we review the dynamical properties of known extrasolar multiple planet systems and their implications for planet formation theory. We conclude by outlining pressing questions to be addressed by a combination of future observations and theoretical research.
Accurate Open-Set Recognition for Memory Workload

How can we accurately identify new memory workloads while classifying known memory workloads? Verifying DRAM (Dynamic Random Access Memory) using various workloads is an important task to guarantee the quality of DRAM. A crucial component in the process is open-set recognition which aims to detect new workloads not seen in the training phase. Despite its importance, however, existing open-set recognition methods are unsatisfactory in terms of accuracy since they fail to exploit the characteristics of workload sequences. In this article, we propose Acorn, an accurate open-set recognition method capturing the characteristics of workload sequences. Acorn extracts two types of feature vectors to capture sequential patterns and spatial locality patterns in memory access. Acorn then uses the feature vectors to accurately classify a subsequence into one of the known classes or identify it as the unknown class. Experiments show that Acorn achieves state-of-the-art accuracy, giving up to 37% points higher unknown class detection accuracy while achieving comparable known class classification accuracy than existing methods.
DuNST: Dual Noisy Self Training for Semi-Supervised Controllable Text Generation

Self-training (ST) has prospered again in language understanding by augmenting the fine-tuning of big pre-trained models when labeled data is insufficient. However, it remains challenging to incorporate ST into attribute-controllable language generation. Augmented only by self-generated pseudo text, generation models over-exploit the previously learned text space and fail to explore a larger one, suffering from a restricted generalization boundary and limited controllability. In this work, we propose DuNST, a novel ST framework to tackle these problems. DuNST jointly models text generation and classification as a dual process and further perturbs and escapes from the collapsed space by adding two kinds of flexible noise. In this way, our model could construct and utilize both pseudo text generated from given labels and pseudo labels predicted from available unlabeled text, which are gradually refined during the ST phase. We theoretically demonstrate that DuNST can be regarded as enhancing the exploration of the potentially larger real text space while maintaining exploitation, guaranteeing improved performance. Experiments on three controllable generation tasks show that DuNST significantly boosts control accuracy with comparable generation fluency and diversity against several strong baselines.
Unitarity Estimation for Quantum Channels

Estimating the unitarity of an unknown quantum channel <inline-formula> <tex-math notation="LaTeX">$\mathcal {E}$ </tex-math></inline-formula> provides information on how much it is unitary, which is a basic and important problem in quantum device certification and benchmarking. Unitarity estimation can be performed with either coherent or incoherent access, where the former in general leads to better query complexity while the latter allows more practical implementations. In this paper, we provide a unified framework for unitarity estimation, which induces ancilla-efficient algorithms that use <inline-formula> <tex-math notation="LaTeX">$O(\epsilon ^{-2})$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$O(\sqrt {d}\cdot \epsilon ^{-2})$ </tex-math></inline-formula> calls to <inline-formula> <tex-math notation="LaTeX">$\mathcal {E}$ </tex-math></inline-formula> with coherent and incoherent accesses, respectively, where <inline-formula> <tex-math notation="LaTeX">$d$ </tex-math></inline-formula> is the dimension of the system that <inline-formula> <tex-math notation="LaTeX">$\mathcal {E}$ </tex-math></inline-formula> acts on and <inline-formula> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> is the required precision. We further show that both the <inline-formula> <tex-math notation="LaTeX">$d$ </tex-math></inline-formula>-dependence and <inline-formula> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula>-dependence of our algorithms are optimal. As part of our results, we settle the query complexity of the distinguishing problem for depolarizing and unitary channels with incoherent access by giving a matching lower bound <inline-formula> <tex-math notation="LaTeX">$\Omega (\sqrt {d})$ </tex-math></inline-formula>, improving the prior best lower bound <inline-formula> <tex-math notation="LaTeX">$\Omega (\sqrt [{3}]{d})$ </tex-math></inline-formula> by (Aharonov et al., 2022) and (Chen et al., FOCS 2021).
Cascaded Compositional Residual Learning for Complex Interactive Behaviors

Real-world autonomous missions often require rich interaction with nearby objects, such as doors or switches, along with effective navigation. However, such complex behaviors are difficult to learn because they involve both high-level planning and low-level motor control. We present a novel framework, Cascaded Compositional Residual Learning (CCRL), which learns composite skills by recursively leveraging a library of previously learned control policies. Our framework combines multiple levels of pre-learned skills by using multiplicative skill composition and residual action learning. We also introduce a goal synthesis network and an observation selector to support combination of heterogeneous skills, each with its unique goals and observation space. Finally, we develop residual regularization for learning policies that solve a new task, while preserving the style of the motion enforced by the skill library. We show that our framework learns joint-level control policies for a diverse set of motor skills ranging from basic locomotion to complex interactive navigation, including navigating around obstacles, pushing objects, crawling under a table, pushing a door open with its leg, and holding it open while walking through it. The proposed CCRL framework leads to policies with consistent styles and lower joint torques, and successfully transfer to a real Unitree A1 robot without any additional fine-tuning.
Experimental observation of one-dimensional motion of interstitial skyrmion in FeGe

nano-second current pulse through the Au wires in the Si 3 N 4 /Si film. A notch is carved at the upper left edge of the nanostripe. The nanostripe's width, length, and thickness are 1 μm, 6 μm, and 150 nm, respectively. The electron beam is along z direction. b, Representative snapshots of the nucleation of the interstitial skyrmion under the current pulse. The wavevector q of spin helix is indicated by the white arrow. The current density is ~ 11 M A ∙ cm 1) and the external magnetic field is B = 0 mT. The black arrows indicate the positions of interstitial skyrmion in spin helix. c, Representative snapshots of the motion of the interstitial skyrmion. The current density is ~ 9.7 M A ∙ cm 1) . The black arrows indicate the positions of the interstitial skyrmion. The summary of the skyrmion trajectories under successive current pulses is schematically indicated with the yellow dots.
Temporal moments for transport with mass transfer described by an arbitrary memory function in heterogeneous media

Temporal moment equations are generalized for transport under linear mass transfer, which has been used to model a broad range of small‐scale processes: kinetic sorption, diffusion into immobile regions, and transport through heterogeneous aquifers. Solving the moment equations, which are formally identical to steady state transport equations, is computationally more efficient than evaluating the temporal moments by integrating the transient flux concentrations. We derive recursive relations for the moments of the flux concentration, which involve the moments of the memory function but do not dependent on its shape. It turns out that two mass transfer models have the same kth temporal moment if the moments of order lower than k are equal. Particularly, the mean retention time, i.e., the first moment of the retention probability density function (pdf) in the immobile domain, decides the second temporal moment of concentration. A mass transfer model with two first‐order rate coefficients can match up to the fourth temporal moment described by a multirate model with a predescribed pdf of the mass transfer rate coefficient. The kth temporal moment is finite when the (k‐1)th moment of the memory function exists.
Digital cognitive–behavioural therapy to reduce suicidal ideation and behaviours: a systematic review and meta-analysis of individual participant data

Question Digital interventions based on cognitive–behavioural therapy (iCBT) is associated with reductions in suicidal ideation. However, fine-grained analyses of effects and potential effect-moderating variables are missing. This study aimed to investigate the effectiveness of iCBT on suicidal ideation, effect moderators, effects on suicide attempts and predictors of adherence. Study selection and analysis We systematically searched CENTRAL, PsycINFO, Embase and PubMed for randomised controlled trials that investigated iCBT for suicidal ideation or behaviours. Participants reporting baseline suicidal ideation were eligible. We conducted a one-stage individual participant data (IPD) meta-analysis. Suicidal ideation was the primary outcome, analysed as three indices: severity of suicidal ideation, reliable changes and treatment response. Findings We included IPD from nine out of ten eligible trials (2037 participants). iCBT showed significant reductions of suicidal ideation compared with control conditions across all indices (severity: b=−0.247, 95% CI −0.322 to −0.173; reliable changes: b=0.633, 95% CI 0.408 to 0.859; treatment response: b=0.606, 95% CI 0.410 to 0.801). In iCBT, the rate of reliable improvement was 40.5% (controls: 27.3%); the deterioration rate was 2.8% (controls: 5.1%). No participant-level moderator effects were identified. The effects on treatment response were higher for trials with waitlist-controls compared with active controls. There were insufficient data on suicide attempts. Human support and female gender predicted treatment adherence. The main source of potential bias was missing outcome data. Conclusions The current evidence indicates that iCBT is effective in reducing suicidal ideation irrespective of age, gender and previous suicide attempts. Future studies should rigorously assess suicidal behaviour and drop-out reasons.
Rapid Phenotypic Convergence towards Collateral Sensitivity in Clinical Isolates of Pseudomonas aeruginosa Presenting Different Genomic Backgrounds

Collateral sensitivity (CS) is a trade-off of antibiotic resistance evolution that could be exploited to design strategies for treating bacterial infections. Clinical application of CS requires it to robustly emerge in different genomic backgrounds. ABSTRACT Collateral sensitivity (CS) is an evolutionary trade-off by which acquisition of resistance to an antibiotic leads to increased susceptibility to another. This Achilles’ heel of antibiotic resistance could be exploited to design evolution-based strategies for treating bacterial infections. To date, most studies in the field have focused on the identification of CS patterns in model strains. However, one of the main requirements for the clinical application of this trade-off is that it must be robust and has to emerge in different genomic backgrounds, including preexisting drug-resistant isolates, since infections are frequently caused by pathogens already resistant to antibiotics. Here, we report the first analysis of CS robustness in clinical strains of Pseudomonas aeruginosa presenting different ab initio mutational resistomes. We identified a robust CS pattern associated with short-term evolution in the presence of ciprofloxacin of clinical P. aeruginosa isolates, including representatives of high-risk epidemic clones belonging to sequence type (ST) 111, ST175, and ST244. We observed the acquisition of different ciprofloxacin resistance mutations in strains presenting varied STs and different preexisting mutational resistomes. Importantly, despite these genetic differences, the use of ciprofloxacin led to a robust CS to aztreonam and tobramycin. In addition, we describe the possible application of this evolutionary trade-off to drive P. aeruginosa infections to extinction by using the combination of ciprofloxacin-tobramycin or ciprofloxacin-aztreonam. Our results support the notion that the identification of robust patterns of CS may establish the basis for developing evolution-informed treatment strategies to tackle bacterial infections, including those due to antibiotic-resistant pathogens. IMPORTANCE Collateral sensitivity (CS) is a trade-off of antibiotic resistance evolution that could be exploited to design strategies for treating bacterial infections. Clinical application of CS requires it to robustly emerge in different genomic backgrounds. In this study, we performed an analysis to identify robust patterns of CS associated with the use of ciprofloxacin in clinical isolates of P. aeruginosa presenting different mutational resistomes and including high-risk epidemic clones (ST111, ST175, and ST244). We demonstrate the robustness of CS to tobramycin and aztreonam and the potential application of this evolutionary observation to drive P. aeruginosa infections to extinction. Our results support the notion that the identification of robust CS patterns may establish the basis for developing evolutionary strategies to tackle bacterial infections, including those due to antibiotic-resistant pathogens.
Theory of Equilibrium Temperatures in Radiative-Turbulent Atmospheres

Abstract We have developed a thermodynamic model for the determination of the temperature profile based on the balance of radiative and turbulent fluxes. In the context of a one-dimensional case, we show that the temperature field is governed by a first-order differential-integro equation involving temperature to the fourth power. In conjunction with the temperature profile determination, parameterization programs for the transfer of broadband thermal infrared and solar fluxes in inhomogeneous atmospheres are developed. In addition, the vertical transport is parameterized using a first-order closure scheme with the eddy thermal diffusion coefficients derived either from known theories or from available data. By virtue of an efficient perturbation technique devised for the solution of the thermal equilibrium temperature, we show that the simulated temperatures compare well with those of a standard atmosphere employing climatological water vapor, ozone and cloud profiles. Simulations using the steady-state .