OpenRaiser/Intern-Atlas huggingface dataset:
Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning
Venue: arXiv cs.CL 2025
Year: 2025

Instruction-fine-tuned large language models (LLMs) under 14B parameters continue to underperform on natural language understanding (NLU) tasks, often trailing smaller models like BERT-base on benchmarks such as GLUE and SuperGLUE. Motivated by the success of reinforcement learning in reasoning tasks (e.g., DeepSeek), we explore Proximal Policy Optimization (PPO) as a framework to improve the NLU capabilities of LLMs. We frame NLU as a reinforcement learning environment, treating token generation as a sequence of actions and optimizing for reward signals based on alignment with ground-truth labels. PPO consistently outperforms supervised fine-tuning, yielding an average improvement of 6.3 points on GLUE, and surpasses zero-shot and few-shot prompting by 38.7 and 26.1 points, respectively. Notably, PPO-tuned models outperform GPT-4o by over 4\% on average across sentiment and natural language inference tasks, including gains of 7.3\% on the Mental Health dataset and 10.9\% on SIGA-nli. This work highlights a promising direction for adapting LLMs to new tasks by reframing them as reinforcement learning problems, enabling learning through simple end-task rewards rather than extensive data curation.
An Annotated Dataset of Errors in Premodern Greek and Baselines for Detecting Them
Venue: arXiv cs.CL 2025
Year: 2025

As premodern texts are passed down over centuries, errors inevitably accrue. These errors can be challenging to identify, as some have survived undetected for so long precisely because they are so elusive. While prior work has evaluated error detection methods on artificially-generated errors, we introduce the first dataset of real errors in premodern Greek, enabling the evaluation of error detection methods on errors that genuinely accumulated at some stage in the centuries-long copying process. To create this dataset, we use metrics derived from BERT conditionals to sample 1,000 words more likely to contain errors, which are then annotated and labeled by a domain expert as errors or not. We then propose and evaluate new error detection methods and find that our discriminator-based detector outperforms all other methods, improving the true positive rate for classifying real errors by 5%. We additionally observe that scribal errors are more difficult to detect than print or digitization errors. Our dataset enables the evaluation of error detection methods on real errors in premodern texts for the first time, providing a benchmark for developing more effective error detection algorithms to assist scholars in restoring premodern works.
SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI
Venue: arXiv cs.CR 2025
Year: 2025

Existing benchmarks for evaluating the security risks and capabilities (e.g., vulnerability detection) of code-generating large language models (LLMs) face several key limitations: (1) limited coverage of risk and capabilities; (2) reliance on static evaluation metrics such as LLM judgments or rule-based detection, which lack the precision of dynamic analysis; and (3) a trade-off between data quality and benchmark scale. To address these challenges, we introduce a general and scalable benchmark construction framework that begins with manually validated, high-quality seed examples and expands them via targeted mutations. Our approach provides a comprehensive suite of artifacts so the benchmark can support comprehensive risk assessment and security capability evaluation using dynamic metrics. By combining expert insights with automated generation, we strike a balance between manual effort, data quality, and benchmark scale. Applying this framework to Python, C/C++, and Java, we build SeCodePLT, a dataset of more than 5.9k samples spanning 44 CWE-based risk categories and three security capabilities. Compared with state-of-the-art benchmarks, SeCodePLT offers broader coverage, higher data fidelity, and substantially greater scale. We use SeCodePLT to evaluate leading code LLMs and agents, revealing their strengths and weaknesses in both generating secure code and identifying or fixing vulnerabilities.
Differentiable Weightless Neural Networks
Venue: arXiv cs.LG 2025
Year: 2025

We introduce the Differentiable Weightless Neural Network (DWN), a model based on interconnected lookup tables. Training of DWNs is enabled by a novel Extended Finite Difference technique for approximate differentiation of binary values. We propose Learnable Mapping, Learnable Reduction, and Spectral Regularization to further improve the accuracy and efficiency of these models. We evaluate DWNs in three edge computing contexts: (1) an FPGA-based hardware accelerator, where they demonstrate superior latency, throughput, energy efficiency, and model area compared to state-of-the-art solutions, (2) a low-power microcontroller, where they achieve preferable accuracy to XGBoost while subject to stringent memory constraints, and (3) ultra-low-cost chips, where they consistently outperform small models in both accuracy and projected hardware area. DWNs also compare favorably against leading approaches for tabular datasets, with higher average rank. Overall, our work positions DWNs as a pioneering solution for edge-compatible high-throughput neural networks.
Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent
Venue: arXiv cs.LG 2025
Year: 2025

In-context learning has been recognized as a key factor in the success of Large Language Models (LLMs). It refers to the model's ability to learn patterns on the fly from provided in-context examples in the prompt during inference. Previous studies have demonstrated that the Transformer architecture used in LLMs can implement a single-step gradient descent update by processing in-context examples in a single forward pass. Recent work has further shown that, during in-context learning, a looped Transformer can implement multi-step gradient descent updates in forward passes. However, their theoretical results require an exponential number of in-context examples, $n = \exp(Ω(T))$, where $T$ is the number of loops or passes, to achieve a reasonably low error. In this paper, we study linear looped Transformers in-context learning on linear vector generation tasks. We show that linear looped Transformers can implement multi-step gradient descent efficiently for in-context learning. Our results demonstrate that as long as the input data has a constant condition number, e.g., $n = O(d)$, the linear looped Transformers can achieve a small error by multi-step gradient descent during in-context learning. Furthermore, our preliminary experiments validate our theoretical analysis. Our findings reveal that the Transformer architecture possesses a stronger in-context learning capability than previously understood, offering new insights into the mechanisms behind LLMs and potentially guiding the better design of efficient inference algorithms for LLMs.
Energy Efficient Transmission Parameters Selection Method Using Reinforcement Learning in Distributed LoRa Networks
Venue: arXiv cs.NI 2025
Year: 2025

With the increase in demand for Internet of Things (IoT) applications, the number of IoT devices has drastically grown, making spectrum resources seriously insufficient. Transmission collisions and retransmissions increase power consumption. Therefore, even in long-range (LoRa) networks, selecting appropriate transmission parameters, such as channel and transmission power, is essential to improve energy efficiency. However, due to the limited computational ability and memory, traditional transmission parameter selection methods for LoRa networks are challenging to implement on LoRa devices. To solve this problem, a distributed reinforcement learning-based channel and transmission power selection method is proposed, which can be implemented on the LoRa devices to improve energy efficiency in this paper. Specifically, the channel and transmission power selection problem in LoRa networks is first mapped to the multi-armed-bandit (MAB) problem. Then, an MAB-based method is introduced to solve the formulated transmission parameter selection problem based on the acknowledgment (ACK) packet and the power consumption for data transmission of the LoRa device. The performance of the proposed method is evaluated by the constructed actual LoRa network. Experimental results show that the proposed method performs better than fixed assignment, adaptive data rate low-complexity (ADR-Lite), and $ε$-greedy-based methods in terms of both transmission success rate and energy efficiency.
DynaCLR: Contrastive Learning of Cellular Dynamics with Temporal Regularization
Venue: arXiv cs.CV 2025
Year: 2025

We report DynaCLR, a self-supervised method for embedding cell and organelle Dynamics via Contrastive Learning of Representations of time-lapse images. DynaCLR integrates single-cell tracking and time-aware contrastive sampling to learn robust, temporally regularized representations of cell dynamics. DynaCLR embeddings generalize effectively to in-distribution and out-of-distribution datasets, and can be used for several downstream tasks with sparse human annotations. We demonstrate efficient annotations of cell states with a human-in-the-loop using fluorescence and label-free imaging channels. DynaCLR method enables diverse downstream biological analyses: classification of cell division and infection, clustering heterogeneous cell migration patterns, cross-modal distillation of cell states from fluorescence to label-free channel, alignment of asynchronous cellular responses and broken cell tracks, and discovering organelle response due to infection. DynaCLR is a flexible method for comparative analyses of dynamic cellular responses to pharmacological, microbial, and genetic perturbations. We provide PyTorch-based implementations of the model training and inference pipeline (https://github.com/mehta-lab/viscy) and a GUI (https://github.com/czbiohub-sf/napari-iohub) for the visualization and annotation of trajectories of cells in the real space and the embedding space.
AdvBDGen: Adversarially Fortified Prompt-Specific Fuzzy Backdoor Generator Against LLM Alignment
Venue: arXiv cs.LG 2025
Year: 2025

With the growing adoption of reinforcement learning with human feedback (RLHF) for aligning large language models (LLMs), the risk of backdoor installation during alignment has increased, leading to unintended and harmful behaviors. Existing backdoor triggers are typically limited to fixed word patterns, making them detectable during data cleaning and easily removable post-poisoning. In this work, we explore the use of prompt-specific paraphrases as backdoor triggers, enhancing their stealth and resistance to removal during LLM alignment. We propose AdvBDGen, an adversarially fortified generative fine-tuning framework that automatically generates prompt-specific backdoors that are effective, stealthy, and transferable across models. AdvBDGen employs a generator-discriminator pair, fortified by an adversary, to ensure the installability and stealthiness of backdoors. It enables the crafting and successful installation of complex triggers using as little as 3% of the fine-tuning data. Once installed, these backdoors can jailbreak LLMs during inference, demonstrate improved stability against perturbations compared to traditional constant triggers, and are more challenging to remove. These findings underscore an urgent need for the research community to develop more robust defenses against adversarial backdoor threats in LLM alignment.
BRC20 Pinning Attack
Venue: arXiv cs.CR 2025
Year: 2025

BRC20 tokens are a type of non-fungible asset on the Bitcoin network. They allow users to embed customised content within Bitcoin's satoshis. The token frenzy reached a market size of US\$2.811\,b (2023Q3--2025Q1). However, this intuitive design has not undergone serious security scrutiny. We present the first analysis of BRC20's \emph{transfer} mechanism and identify a new attack vector. A typical BRC20 transfer involves two "bundled" on-chain transactions with different fee levels: the first (i.e., \textbf{Tx1}) with a lower fee inscribes the \textsf{transfer} request, while the second (i.e., \textbf{Tx2}) with a higher fee finalizes the actual transfer. An adversary can send a manipulated fee transaction (falling between the two fee levels), which causes \textbf{Tx1} to be processed while \textbf{Tx2} is pinned in the mempool. This locks BRC20 liquidity and disrupts normal withdrawal requests from users. We term this the \emph{BRC20 pinning attack}. We validated the attack in real-world settings in collaboration with Binance researchers. With their knowledge and permission, we conducted a controlled test against Binance's ORDI hot wallet, resulting in a temporary suspension of ORDI withdrawals for 3.5 hours. Recovery was performed shortly after. Further analysis confirms that the attack can be applied to over \textbf{90\%} of inscription-based tokens within the Bitcoin ecosystem.
Before & After: The Effect of EU's 2022 Code of Practice on Disinformation
Venue: arXiv cs.CY 2025
Year: 2025

Over the past few years, the European Commission has made significant steps to reduce disinformation in cyberspace. One of those steps has been the introduction of the 2022 "Strengthened Code of Practice on Disinformation". Signed by leading online platforms, this Strengthened Code of Practice on Disinformation is an attempt to combat disinformation on the Web. The Code of Practice includes a variety of measures including the demonetization of disinformation, urging, for example, advertisers "to avoid the placement of advertising next to Disinformation content". In this work, we set out to explore what was the impact of the Code of Practice and especially to explore to what extent ad networks continue to advertise on dis-/mis-information sites. We perform a historical analysis and find that, although at a hasty glance things may seem to be improving, there is really no significant reduction in the amount of advertising relationships among popular misinformation websites and major ad networks. In fact, we show that ad networks have withdrawn mostly from unpopular misinformation websites with very few visitors, but still form relationships with highly unreliable websites that account for the majority of misinformation traffic. To make matters worse, we show that ad networks continue to place advertisements of legitimate companies next to misinformation content. We show that major ad networks place ads in almost 400 misinformation websites in our dataset.
Can sparse autoencoders make sense of gene expression latent variable models?
Venue: arXiv cs.LG 2025
Year: 2025

Sparse autoencoders (SAEs) have lately been used to uncover interpretable latent features in large language models. By projecting dense embeddings into a much higher-dimensional and sparse space, learned features become disentangled and easier to interpret. This work explores the potential of SAEs for decomposing embeddings in complex and high-dimensional biological data. Using simulated data, it outlines the efficacy, hyperparameter landscape, and limitations of SAEs when it comes to extracting ground truth generative variables from latent space. The application to embeddings from pretrained single-cell models shows that SAEs can find and steer key biological processes and even uncover subtle biological signals that might otherwise be missed. This work further introduces scFeatureLens, an automated interpretability approach for linking SAE features and biological concepts from gene sets to enable large-scale analysis and hypothesis generation in single-cell gene expression models.
Parameter estimation of structural dynamics with neural operators enabled surrogate modeling
Venue: arXiv cs.CE 2025
Year: 2025

Parameter estimation in structural dynamics generally involves inferring the values of physical, geometric, or even customized parameters based on first principles or expert knowledge, which is challenging for complex structural systems. In this work, we present a unified deep learning-based framework for parameterization, forward modeling, and inverse modeling of structural dynamics. The parameterization is flexible and can be user-defined, including physical and/or non-physical (customized) parameters. In the forward modeling, we train a neural operator for response prediction -- forming a surrogate model, which leverages the defined system parameters and excitation forces as inputs to the model. The inverse modeling focuses on estimating system parameters. In particular, the learned forward surrogate model (which is differentiable) is utilized for preliminary parameter estimation via gradient-based optimization; to further boost the parameter estimation, we introduce a neural refinement method to mitigate ill-posed problems, which often occur in the former. The framework's effectiveness is verified numerically and experimentally, in both interpolation and extrapolation cases, indicating its capability to capture intrinsic dynamics of structural systems from both forward and inverse perspectives. Moreover, the framework's flexibility is expected to support a wide range of applications, including surrogate modeling, structural identification, damage detection, and inverse design of structural systems.
Fuzzy Aristotelian Diagrams
Venue: arXiv cs.LO 2025
Year: 2025

After a concise introduction to the square of opposition, in particular, and, Aristotelian Diagrams, in general, I describe how one can create a mathematical universe to host these objects. Since these objects assume that the underlying logic is the bivalent logic, I have used these objects as a starting point to introduce fuzzy Aristotelian Diagrams and describe a mathematical formulation of them. In addition, I outline the characteristrics of a mathematical universe that hosts them.
Random expansions of trees with bounded height
Venue: arXiv cs.LO 2025
Year: 2025

We consider a sequence $\mathbf{T} = (\mathcal{T}_n : n \in \mathbb{N}^+)$ of trees $\mathcal{T}_n$ where, for some $Δ\in \mathbb{N}^+$ every $\mathcal{T}_n$ has height at most $Δ$ and as $n \to \infty$ the minimal number of children of a nonleaf tends to infinity. We can view every tree as a (first-order) $τ$-structure where $τ$ is a signature with one binary relation symbol. For a fixed (arbitrary) finite and relational signature $σ\supseteq τ$ we consider the set $\mathbf{W}_n$ of expansions of $\mathcal{T}_n$ to $σ$ and a probability distribution $\mathbb{P}_n$ on $\mathbf{W}_n$ which is determined by a (parametrized/lifted) Probabilistic Graphical Model (PGM) $\mathbb{G}$ which can use the information given by $\mathcal{T}_n$. The kind of PGM that we consider uses formulas of a many-valued logic that we call $PLA^*$ with truth values in the unit interval $[0, 1]$. We also use $PLA^*$ to express queries, or events, on $\mathbf{W}_n$. With this setup we prove that, under some assumptions on $\mathbf{T}$, $\mathbb{G}$, and a (possibly quite complex) formula $φ(x_1, \ldots, x_k)$ of $PLA^*$, as $n \to \infty$, if $a_1, \ldots, a_k$ are vertices of the tree $\mathcal{T}_n$ then the value of $φ(a_1, \ldots, a_k)$ will, with high probability, be almost the same as the value of $ψ(a_1, \ldots, a_k)$, where $ψ(x_1, \ldots, x_k)$ is a ``simple'' formula the value of which can always be computed quickly (without reference to $n$), and $ψ$ itself can be found by using only the information that defines $\mathbf{T}$, $\mathbb{G}$ and $φ$. A corollary of this, subject to the same conditions, is a probabilistic convergence law for $PLA^*$-formulas.
KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entities
Venue: arXiv cs.CV 2025
Year: 2025

Recent advances in text-to-image generation have improved the quality of synthesized images, but evaluations mainly focus on aesthetics or alignment with text prompts. Thus, it remains unclear whether these models can accurately represent a wide variety of realistic visual entities. To bridge this gap, we propose KITTEN, a benchmark for Knowledge-InTensive image generaTion on real-world ENtities. Using KITTEN, we conduct a systematic study of the latest text-to-image models and retrieval-augmented models, focusing on their ability to generate real-world visual entities, such as landmarks and animals. Analysis using carefully designed human evaluations, automatic metrics, and MLLM evaluations show that even advanced text-to-image models fail to generate accurate visual details of entities. While retrieval-augmented models improve entity fidelity by incorporating reference images, they tend to over-rely on them and struggle to create novel configurations of the entity in creative text prompts.
Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functions
Venue: arXiv cs.LG 2025
Year: 2025

In reinforcement learning, off-policy actor-critic methods like DDPG and TD3 use deterministic policy gradients: the Q-function is learned from environment data, while the actor maximizes it via gradient ascent. We observe that in complex tasks such as dexterous manipulation and restricted locomotion with mobility constraints, the Q-function exhibits many local optima, making gradient ascent prone to getting stuck. To address this, we introduce SAVO, an actor architecture that (i) generates multiple action proposals and selects the one with the highest Q-value, and (ii) approximates the Q-function repeatedly by truncating poor local optima to guide gradient ascent more effectively. We evaluate tasks such as restricted locomotion, dexterous manipulation, and large discrete-action space recommender systems and show that our actor finds optimal actions more frequently and outperforms alternate actor architectures.
GaVaMoE: Gaussian-Variational Gated Mixture of Experts for Explainable Recommendation
Venue: arXiv cs.IR 2025
Year: 2025

Large language model-based explainable recommendation (LLM-based ER) systems show promise in generating human-like explanations for recommendations. However, they face challenges in modeling user-item collaborative preferences, personalizing explanations, and handling sparse user-item interactions. To address these issues, we propose GaVaMoE, a novel Gaussian-Variational Gated Mixture of Experts framework for explainable recommendation. GaVaMoE introduces two key components: (1) a rating reconstruction module that employs Variational Autoencoder (VAE) with a Gaussian Mixture Model (GMM) to capture complex user-item collaborative preferences, serving as a pre-trained multi-gating mechanism; and (2) a set of fine-grained expert models coupled with the multi-gating mechanism for generating highly personalized explanations. The VAE component models latent factors in user-item interactions, while the GMM clusters users with similar behaviors. Each cluster corresponds to a gate in the multi-gating mechanism, routing user-item pairs to appropriate expert models. This architecture enables GaVaMoE to generate tailored explanations for specific user types and preferences, mitigating data sparsity by leveraging user similarities. Extensive experiments on three real-world datasets demonstrate that GaVaMoE significantly outperforms existing methods in explanation quality, personalization, and consistency. Notably, GaVaMoE exhibits robust performance in scenarios with sparse user-item interactions, maintaining high-quality explanations even for users with limited historical data.