Miracle-dz/UniFAR-FTUs huggingface dataset:
Beyond prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations

Recent work has demonstrated that pre-trained language models (PLMs) are zero-shot learners. However, most existing zero-shot methods involve heavy human engineering or complicated self-training pipelines, hindering their application to new situations. In this work, we show that zero-shot text classification can be improved simply by clustering texts in the embedding spaces of PLMs. Specifically, we fit the unlabeled texts with a Bayesian Gaussian Mixture Model after initializing cluster positions and shapes using class names. Despite its simplicity, this approach achieves superior or comparable performance on both topic and sentiment classification datasets and outperforms prior works significantly on unbalanced datasets. We further explore the applicability of our clustering approach by evaluating it on 14 datasets with more diverse topics, text lengths, and numbers of classes. Our approach achieves an average of 20% absolute improvement over prompt-based zero-shot learning. Finally, we compare different PLM embedding spaces and find that texts are well-clustered by topics even if the PLM is not explicitly pre-trained to generate meaningful sentence embeddings. This work indicates that PLM embeddings can categorize texts without task-specific fine-tuning, thus providing a new way to analyze and utilize their knowledge and zero-shot learning ability.
Searching for axion forces with precision precession in storage rings

We consider different types of storage rings as precision probes of axion-mediated monopole-dipole forces. We show that current and planned experiments aiming to measure magnetic and electric dipole moments of protons, muons and electrons very precisely may explore new parts of the parameter space beyond existing laboratory bounds and, in some cases, beyond astrophysical constraints. Remarkably, a light axion coupled to muons may explain the FNAL/BNL $(g-2)_\mu$ anomaly as an environmental effect -- the coherent axion field generated by the earth nucleons induces an extra contribution to the anomalous precession frequency of the muon explaining the discrepancy with respect to the SM prediction.
Study on Poisson distribution and geometric distribution motivated by Chv´atal’s conjecture

Let B ( n, p ) denote a binomial random variable with parameters n and p . Chv´atal’s conjecture says that for any ﬁxed n ≥ 2, as m ranges over { 0 , . , n } , the probability q m := P ( B ( n, m/n ) ≤ m ) is the smallest when m is closest to 2 n 3 . Motivated by this conjecture, in this note, we consider the corresponding minimum value problem on the probability that a random variable is not more than its expectation, when its distribution is Poisson distribution or geometric distribution. We give a complete answer for each case. Some related questions will be introduced. MSC: 60C05, 60E15
Improving Bilingual Lexicon Induction with Cross-Encoder Reranking

Bilingual lexicon induction (BLI) with limited bilingual supervision is a crucial yet challenging task in multilingual NLP. Current state-of-the-art BLI methods rely on the induction of cross-lingual word embeddings (CLWEs) to capture cross-lingual word similarities; such CLWEs are obtained 1) via traditional static models (e.g., VecMap), or 2) by extracting type-level CLWEs from multilingual pretrained language models (mPLMs), or 3) through combining the former two options. In this work, we propose a novel semi-supervised post-hoc reranking method termed BLICEr (BLI with Cross-Encoder Reranking), applicable to any precalculated CLWE space, which improves their BLI capability. The key idea is to 'extract' cross-lingual lexical knowledge from mPLMs, and then combine it with the original CLWEs. This crucial step is done via 1) creating a word similarity dataset, comprising positive word pairs (i.e., true translations) and hard negative pairs induced from the original CLWE space, and then 2) fine-tuning an mPLM (e.g., mBERT or XLM-R) in a cross-encoder manner to predict the similarity scores. At inference, we 3) combine the similarity score from the original CLWE space with the score from the BLI-tuned cross-encoder. BLICEr establishes new state-of-the-art results on two standard BLI benchmarks spanning a wide spectrum of diverse languages: it substantially outperforms a series of strong baselines across the board. We also validate the robustness of BLICEr with different CLWEs.
Flatter, faster: scaling momentum for optimal speedup of SGD

Commonly used optimization algorithms often show a trade-off between good generalization and fast training times. For instance, stochastic gradient descent (SGD) tends to have good generalization; however, adaptive gradient methods have superior training times. Momentum can help accelerate training with SGD, but so far there has been no principled way to select the momentum hyperparameter. Here we study training dynamics arising from the interplay between SGD with label noise and momentum in the training of overparametrized neural networks. We find that scaling the momentum hyperparameter $1-\beta$ with the learning rate to the power of $2/3$ maximally accelerates training, without sacrificing generalization. To analytically derive this result we develop an architecture-independent framework, where the main assumption is the existence of a degenerate manifold of global minimizers, as is natural in overparametrized models. Training dynamics display the emergence of two characteristic timescales that are well-separated for generic values of the hyperparameters. The maximum acceleration of training is reached when these two timescales meet, which in turn determines the scaling limit we propose. We confirm our scaling rule for synthetic regression problems (matrix sensing and teacher-student paradigm) and classification for realistic datasets (ResNet-18 on CIFAR10, 6-layer MLP on FashionMNIST), suggesting the robustness of our scaling rule to variations in architectures and datasets.
Nested Sequents for Intuitionistic Grammar Logics via Structural Refinement

Intuitionistic grammar logics fuse constructive and multi-modal reasoning while permitting the use of converse modalities, serving as a generalization of standard intuitionistic modal logics. In this paper, we provide deﬁnitions of these logics as well as establish a suitable proof theory thereof. In particular, we show how to apply the structural reﬁnement methodology to extract cut-free nested sequent calculi for intuitionistic grammar logics from their semantics. This method pro-ceeds by ﬁrst transforming the semantics of these logics into sound and complete labeled sequent systems, which we prove have favorable proof-theoretic properties such as syntactic cut-elimination. We then transform these labeled systems into nested sequent systems via the introduction of propagation rules and the elimination of structural rules. Our derived proof systems are then put to use, whereby we prove the conservativity of intuitionistic grammar logics over their modal counterparts, establish the general undecidability of these logics, and recognize a decidable subclass, referred to as simple intuitionistic grammar logics.
Towards Attribute-Entangled Controllable Text Generation: A Pilot Study of Blessing Generation

Controllable Text Generation (CTG) has obtained great success due to its fine-grained generation ability obtained by focusing on multiple attributes. However, most existing CTG researches overlook how to utilize the attribute entanglement to enhance the diversity of the controlled generated texts. Facing this dilemma, we focus on a novel CTG scenario, i.e., blessing generation which is challenging because high-quality blessing texts require CTG models to comprehensively consider the entanglement between multiple attributes (e.g., objects and occasions). To promote the research on blessing generation, we present EBleT, a large-scale Entangled Blessing Text dataset containing 293K English sentences annotated with multiple attributes. Furthermore, we propose novel evaluation metrics to measure the quality of the blessing texts generated by the baseline models we designed. Our study opens a new research direction for controllable text generation and enables the development of attribute-entangled CTG models.
Downlink Massive MIMO Channel Estimation via Deep Unrolling : Sparsity Exploitations in Angular Domain

—In frequency division duplex (FDD) massive MIMO systems, reliable downlink channel estimation is essential for the subsequent data transmission but is realized at the cost of massive pilot overhead due to hundreds of antennas at base station (BS). In order to reduce the pilot overhead without compromising the estimation, compressive sensing (CS) based methods have been widely applied for channel estimation by exploiting the inherent sparse structure of massive MIMO channel in angular domain. However, they still suffer from high complexity during optimization process and the requirement of prior knowledge on sparsity information. To overcome these challenges, this paper develops a novel hybrid channel estimation framework by integrating the model-driven CS and data-driven deep unrolling techniques. The proposed framework is composed of a coarse estimation part and a ﬁne correction part, which is implemented in a two-stage manner to exploit both inter- and intra-frame sparsities of channels in angular domain. Then, two estimation schemes are designed depending on whether priori sparsity information is required, where the second scheme designs a new thresholding function to eliminate such requirement. Numerical results are provided to verify that our schemes can achieve high accuracy with low pilot overhead and low complexity.
Learning Probabilistic Models from Generator Latent Spaces with Hat EBM

This work proposes a method for using any generator network as the foundation of an Energy-Based Model (EBM). Our formulation posits that observed images are the sum of unobserved latent variables passed through the generator network and a residual random variable that spans the gap between the generator output and the image manifold. One can then define an EBM that includes the generator as part of its forward pass, which we call the Hat EBM. The model can be trained without inferring the latent variables of the observed data or calculating the generator Jacobian determinant. This enables explicit probabilistic modeling of the output distribution of any type of generator network. Experiments show strong performance of the proposed method on (1) unconditional ImageNet synthesis at 128x128 resolution, (2) refining the output of existing generators, and (3) learning EBMs that incorporate non-probabilistic generators. Code and pretrained models to reproduce our results are available at https://github.com/point0bar1/hat-ebm.
SEVGGNet-LSTM: a fused deep learning model for ECG classification

This paper presents a fused deep learning algorithm for ECG classiﬁcation. It takes advantages of the combined convolutional and recurrent neural network for ECG classiﬁcation, and the weight allocation capability of attention mechanism. The input ECG signals are ﬁrstly segmented and normalized, and then fed into the combined VGG and LSTM network for feature extraction and classiﬁcation. An attention mechanism (SE block) is embedded into the core network for increasing the weight of important features. Two databases from different sources and devices are employed for performance validation, and the results well demonstrate the effectiveness and robustness of the proposed algorithm.
Circuit Solutions towards Broadband Piezoelectric Energy Harvesting: An Impedance Analysis

In the studies of piezoelectric energy harvesting (PEH) systems, literature has shown that circuit advancement has a signiﬁcant eﬀect on the enhancement of energy harvesting capability in resonance. On the other hand, some recent studies using the phase-variable (PV) synchronized switch technologies have found that the advanced circuit solutions can also broaden the harvesting bandwidth, i.e., improving the oﬀ-resonance energy harvesting capability, to some extent. However, the available span of the electrically induced dynamics by the existing energy harvesting circuits was not properly deﬁned and demonstrated. Performance comparison among diﬀerent circuits cannot be fairly achieved without using a common theoretical language. Given these, this paper provides an impedance-based analysis and comparison on the electromechanical joint dynamics of the PEH systems using diﬀerent interface circuits, in particular, with a speciﬁc focus on their contributions towards the improvement of energy harvesting bandwidth. Given that the resonance tunability by circuit solutions has received no attention in the conventional ideal model of kinetic energy harvester, we ﬁrstly propose a more inclusive ideal model for better generalization. In practice, it was proven that the attainable dynamic ranges of the practical energy harvesting circuits are only some subsets of the ideal realm. A detailed quantitative study on the attainable ranges of the PV synchronized switch circuit solutions is provided after the introduction of the ideal target. Simulation and experimental results of diﬀerent interface circuits show good agreement with the theoretical analysis. It can be concluded that the resonance tunability strongly depends on the achievable extent in the reactive (imaginary) direction of the equivalent impedance plane. In practice, the electromechanical coupling conditions and dielectric loss might also inﬂuence the resonance tunability. The general ideal model and quantitative impedance analysis provided in this paper help guide the future design eﬀort towards high-capability and broadband PEH systems.
Hydrodynamical Simulations of Corotating Interaction Regions and Discrete Absorption Components in Rotating O-Star Winds

We present two-dimensional hydrodynamical simulations of corotating stream structure in the wind from a rotating O star, together with resulting synthetic line profiles showing discrete absorption components (DACs). An azimuthal variation is induced by a local increase or decrease in the radiative driving force, as would arise from a bright or dark ``star spot'' in the equatorial plane. Since much of the emergent wind structure seems independent of the exact method of perturbation, we expect similar morphology in winds perturbed by localized magnetic fields or nonradial pulsations, as well as by either rotationally-modulated structure or transient mass ejections. We find that bright spots with enhanced driving generate high-density, low-speed streams, while dark spots generate low-density, high-speed streams. Corotating interaction regions (CIRs) form where fast material collides with slow material -- e.g. at the leading (trailing) edge of a stream from a dark (bright) spot, often steepening into shocks. The unperturbed supersonic wind obliquely impacts the high-density CIR and sends back a nonlinear signal which takes the form of a sharp propagating discontinuity (``kink'' or ``plateau'') in the radial velocity gradient. These features travel inward in the co-moving frame at the radiative-acoustic characteristic speed, and thus slowly outward in the star's frame. We find that these slow kinks, rather than the CIRs themselves, are more likely to result in high-opacity DACs in the absorption troughs of unsaturated P Cygni line profiles.
MatKG: The Largest Knowledge Graph in Materials Science -- Entities, Relations, and Link Prediction through Graph Representation Learning

This paper introduces MatKG, a novel graph database of key concepts in material science spanning the traditional material-structure-property-processing paradigm. MatKG is autonomously generated through transformer-based, large language models and generates pseudo ontological schema through statistical co-occurrence mapping. At present, MatKG contains over 2 million unique relationship triples derived from 80,000 entities. This allows the curated analysis, querying, and visualization of materials knowledge at unique resolution and scale. Further, Knowledge Graph Embedding models are used to learn embedding representations of nodes in the graph which are used for downstream tasks such as link prediction and entity disambiguation. MatKG allows the rapid dissemination and assimilation of data when used as a knowledge base, while enabling the discovery of new relations when trained as an embedding model.
ERL-Re2: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representation

Deep Reinforcement Learning (Deep RL) and Evolutionary Algorithms (EA) are two major paradigms of policy optimization with distinct learning principles, i.e., gradient-based v.s. gradient-free. An appealing research direction is integrating Deep RL and EA to devise new methods by fusing their complementary advantages. However, existing works on combining Deep RL and EA have two common drawbacks: 1) the RL agent and EA agents learn their policies individually, neglecting efficient sharing of useful common knowledge; 2) parameter-level policy optimization guarantees no semantic level of behavior evolution for the EA side. In this paper, we propose Evolutionary Reinforcement Learning with Two-scale State Representation and Policy Representation (ERL-Re$^2$), a novel solution to the aforementioned two drawbacks. The key idea of ERL-Re$^2$ is two-scale representation: all EA and RL policies share the same nonlinear state representation while maintaining individual} linear policy representations. The state representation conveys expressive common features of the environment learned by all the agents collectively; the linear policy representation provides a favorable space for efficient policy optimization, where novel behavior-level crossover and mutation operations can be performed. Moreover, the linear policy representation allows convenient generalization of policy fitness with the help of the Policy-extended Value Function Approximator (PeVFA), further improving the sample efficiency of fitness estimation. The experiments on a range of continuous control tasks show that ERL-Re$^2$ consistently outperforms advanced baselines and achieves the State Of The Art (SOTA). Our code is available on https://github.com/yeshenpy/ERL-Re2.
Mutation testing optimisations using the Clang front‐end

Mutation testing is the state‐of‐the‐art technique for assessing the fault detection capacity of a test suite. Unfortunately, a full mutation analysis is often prohibitively expensive. The CppCheck project for instance, demands a build time of 5.8 min and a test execution time of 17 s on our desktop computer. An unoptimised mutation analysis, for 55,000 generated mutants took 11.8 days in total, of which 4.3 days is spent on (re)compiling the project. In this paper, we present a feasibility study, investigating how a number of optimisation strategies can be implemented based on the Clang front‐end. These optimisation strategies allow to eliminate the compilation and execution overhead in order to support efficient mutation testing for the C language family. We provide a proof‐of‐concept tool that achieves a speedup of between 2 × and 30 × . We make a detailed analysis of the speedup induced by the optimisations, elaborate on the lessons learned and point out avenues for further improvements.
Filter and evolve: progressive pseudo label refining for semi-supervised automatic speech recognition

Fine-tuning self-supervised pre-trained models using pseudo-labels can effectively improve speech recognition performance. But, low-quality pseudo-labels can misguide decision boundaries and degrade performance. We propose a simple yet effective strategy to ﬁlter low-quality pseudo-labels to al-leviate this problem. Speciﬁcally, pseudo-labels are produced over the entire training set and ﬁltered via average probability scores calculated from the model’s output. Subsequently, an optimal percentage of utterances with high probability scores are considered reliable training data with trustworthy labels. The model is iteratively updated to correct the unreliable pseudo-labels to minimize the effect of noisy labels. The process above is repeated until unreliable pseudo-labels have been adequately corrected. Extensive experiments on LibriSpeech show that these ﬁltered samples enable the reﬁned model to yield more correct predictions, leading to better ASR performances under various experimental settings.
Enabling Atomic Durability for Persistent Memory with Transiently Persistent CPU Cache

Persistent memory (pmem) products bring the persistence domain up to the memory level. Intel recently introduced the eADR feature that guarantees to flush data buffered in CPU cache to pmem on a power outage, thereby making the CPU cache a transient persistence domain. Researchers have explored how to enable the atomic durability for applications' in-pmem data. In this paper, we exploit the eADR-supported CPU cache to do so. A modified cache line, until written back to pmem, is a natural redo log copy of the in-pmem data. However, a write-back due to cache replacement or eADR on a crash overwrites the original copy. We accordingly develop Hercules, a hardware logging design for the transaction-level atomic durability, with supportive components installed in CPU cache, memory controller (MC), and pmem. When a transaction commits, Hercules commits on-chip its data staying in cache lines. For cache lines evicted before the commit, Hercules asks the MC to redirect and persist them into in-pmem log entries and commits them off-chip upon committing the transaction. Hercules lazily conducts pmem writes only for cache replacements at runtime. On a crash, Hercules saves metadata and data for active transactions into pmem for recovery. Experiments show that, by using CPU cache for both buffering and logging, Hercules yields much higher throughput and incurs significantly fewer pmem writes than state-of-the-art designs.
Explainable Deep Learning to Profile Mitochondrial Disease Using High Dimensional Protein Expression Data

Mitochondrial diseases are currently untreatable due to our limited understanding of their pathology. We study the expression of various mitochondrial proteins in skeletal myofibres (SM) taken from biopsies in order to discover processes involved in mitochondrial pathology using Imaging Mass Cytometry (IMC). IMC produces high dimensional multichannel (in our case 10) pseudo-images representing spatial variation in the expression of a panel of proteins within a tissue, including subcellular variation. Statistical analysis of these images requires semi-automated annotation of thousands of SM in IMC images of patient muscle biopsies. In this paper we investigate the use of deep learning (DL) on raw IMC data to analyse it without any manual pre-processing steps, statistical summaries or statistical models. For this we first t rain s tate-of-art c omputer vision DL models on all available image channels, both combined and individually. We observed better than expected accuracy for many of these models. We then apply state-of-the-art explainable techniques relevant to computer vision DL to find the basis of the predictions of these models. Some of the resulting visual explainable maps highlight features in the images that appear consistent with the latest hypotheses about mitochondrial disease progression within myofibres.