OpenRaiser/Intern-Atlas huggingface dataset:
On A Class of Greedy Sparse Recovery Algorithms
Venue: arXiv cs.IT 2025
Year: 2025

Sparse signal recovery deals with finding the sparest solution of an under-determined linear system $x = Q s$. In this paper, we propose a novel greedy approach to addressing the challenges from such a problem. Such an approach is based on a characterization of solutions to the system, which allows us to work on the sparse recovery in the $s$-space directly with a given measure. With $l_2$-based measure, an orthogonal matching pursuit (OMP)-type algorithm is proposed, which significantly outperforms the classical OMP algorithm in terms of recovery accuracy while maintaining comparable computational complexity. An $l_1$-based algorithm, denoted as $\text{Alg}_{GL1}$, is derived. Such an algorithm significantly outperforms the classical basis pursuit (BP) algorithm. Combining with the CoSaMP-strategy for selecting atoms, a class of high performance greedy algorithms is also derived. Extensive numerical simulations on both synthetic and image data are carried out, with which the superior performance of our proposed algorithms is demonstrated in terms of sparse recovery accuracy and robustness against numerical instability of the system matrix $Q$ and disturbance in the measurement $x$.
TMT: Tri-Modal Translation between Speech, Image, and Text by Processing Different Modalities as Different Languages
Venue: arXiv cs.CL 2025
Year: 2025

The capability to jointly process multi-modal information is becoming an essential task. However, the limited number of paired multi-modal data and the large computational requirements in multi-modal learning hinder the development. We propose a novel Tri-Modal Translation (TMT) model that translates between arbitrary modalities spanning speech, image, and text. We introduce a novel viewpoint, where we interpret different modalities as different languages, and treat multi-modal translation as a well-established machine translation problem. To this end, we tokenize speech and image data into discrete tokens, which provide a unified interface across modalities and significantly decrease the computational cost. In the proposed TMT, a multi-modal encoder-decoder conducts the core translation, whereas modality-specific processing is conducted only within the tokenization and detokenization stages. We evaluate the proposed TMT on all six modality translation tasks. TMT outperforms single model counterparts consistently, demonstrating that unifying tasks is beneficial not only for practicality but also for performance.
Honeybee: Byzantine Tolerant Decentralized Peer Sampling with Verifiable Random Walks
Venue: arXiv cs.NI 2025
Year: 2025

Popular blockchains today have hundreds of thousands of nodes and need to be able to support sophisticated scaling solutions$\unicode{x2013}$such as sharding, data availability sampling, and layer-2 methods. Designing secure and efficient peer-to-peer (p2p) networking protocols at these scales to support the tight demands of the upper layer crypto-economic primitives is a highly non-trivial endeavor. We identify decentralized, uniform random sampling of nodes as a fundamental capability necessary for building robust p2p networks in emerging blockchain networks. Sampling algorithms used in practice today (primarily for address discovery) rely on either distributed hash tables (e.g., Kademlia) or sharing addresses with neighbors (e.g., GossipSub), and are not secure in a Sybil setting. We present Honeybee, a decentralized algorithm for sampling nodes that uses verifiable random walks and table consistency checks. Honeybee is secure against attacks even in the presence of an overwhelming number of Byzantine nodes (e.g., $\geq50\%$ of the network). We evaluate Honeybee through experiments and show that the quality of sampling achieved by Honeybee is significantly better compared to the state-of-the-art. Our proposed algorithm has implications for network design in both full nodes and light nodes.
Scaling Robust Optimization for Multi-Agent Robotic Systems: A Distributed Perspective
Venue: arXiv cs.RO 2025
Year: 2025

This paper presents a novel distributed robust optimization scheme for steering distributions of multi-agent systems under stochastic and deterministic uncertainty. Robust optimization is a subfield of optimization which aims to discover an optimal solution that remains robustly feasible for all possible realizations of the problem parameters within a given uncertainty set. Such approaches would naturally constitute an ideal candidate for multi-robot control, where in addition to stochastic noise, there might be exogenous deterministic disturbances. Nevertheless, as these methods are usually associated with significantly high computational demands, their application to multi-agent robotics has remained limited. The scope of this work is to propose a scalable robust optimization framework that effectively addresses both types of uncertainties, while retaining computational efficiency and scalability. In this direction, we provide tractable approximations for robust constraints that are relevant in multi-robot settings. Subsequently, we demonstrate how computations can be distributed through an Alternating Direction Method of Multipliers (ADMM) approach towards achieving scalability and communication efficiency. All improvements are also theoretically justified by establishing and comparing the resulting computational complexities. Simulation results highlight the performance of the proposed algorithm in effectively handling both stochastic and deterministic uncertainty in multi-robot systems. The scalability of the method is also emphasized by showcasing tasks with up to hundreds of agents. The results of this work indicate the promise of blending robust optimization, distribution steering and distributed optimization towards achieving scalable, safe and robust multi-robot control.
RobKiNet: Robotic Kinematics Informed Neural Network for Optimal Robot Configuration Prediction
Venue: arXiv cs.RO 2025
Year: 2025

Task and Motion Planning (TAMP) is essential for robots to interact with the world and accomplish complex tasks. The TAMP problem involves a critical gap: exploring the robot's configuration parameters (such as chassis position and robotic arm joint angles) within continuous space to ensure that task-level global constraints are met while also enhancing the efficiency of subsequent motion planning. Existing methods still have significant room for improvement in terms of efficiency. Recognizing that robot kinematics is a key factor in motion planning, we propose a framework called the Robotic Kinematics Informed Neural Network (RobKiNet) as a bridge between task and motion layers. RobKiNet integrates kinematic knowledge into neural networks to train models capable of efficient configuration prediction. We designed a Chassis Motion Predictor(CMP) and a Full Motion Predictor(FMP) using RobKiNet, which employed two entirely different sets of forward and inverse kinematics constraints to achieve loosely coupled control and whole-body control, respectively. Experiments demonstrate that CMP and FMP can predict configuration parameters with 96.67% and 98% accuracy, respectively. That means that the corresponding motion planning can achieve a speedup of 24.24x and 153x compared to random sampling. Furthermore, RobKiNet demonstrates remarkable data efficiency. CMP only requires 1/71 and FMP only requires 1/15052 of the training data for the same prediction accuracy compared to other deep learning methods. These results demonstrate the great potential of RoboKiNet in robot applications.
Performance of Double-Stacked Intelligent Metasurface-Assisted Multiuser Massive MIMO Communications in the Wave Domain
Venue: arXiv cs.IT 2025
Year: 2025

Although reconfigurable intelligent surface (RIS) is a promising technology for shaping the propagation environment, it consists of a single-layer structure within inherent limitations regarding the number of beam steering patterns. Based on the recently revolutionary technology, denoted as stacked intelligent metasurface (SIM), we propose its implementation not only on the base station (BS) side in a massive multiple-input multiple-output (mMIMO) setup but also in the intermediate space between the base station and the users to adjust the environment further as needed. For the sake of convenience, we call the former BS SIM (BSIM), and the latter channel SIM (CSIM). Hence, we achieve wave-based combining at the BS and wave-based configuration at the intermediate space. Specifically, we propose a channel estimation method with reduced overhead, being crucial for SIMassisted communications. Next, we derive the uplink sum spectral efficiency (SE) in closed form in terms of statistical channel state information (CSI). Notably, we optimize the phase shifts of both BSIM and CSIM simultaneously by using the projected gradient ascent method (PGAM). Compared to previous works on SIMs, we study the uplink transmission, a mMIMO setup, channel estimation in a single phase, a second SIM at the intermediate space, and simultaneous optimization of the two SIMs. Simulation results show the impact of various parameters on the sum SE, and demonstrate the superiority of our optimization approach compared to the alternating optimization (AO) method.
From RAGs to riches: Utilizing large language models to write documents for clinical trials
Venue: arXiv cs.CL 2025
Year: 2025

This manuscript has now been published: - Link to article on journal website: https://journals.sagepub.com/doi/10.1177/17407745251320806 - Pubmed link: https://pubmed.ncbi.nlm.nih.gov/40013826/
ProLLaMA: A Protein Large Language Model for Multi-Task Protein Language Processing
Venue: arXiv cs.CE 2025
Year: 2025

Recent advances in Protein Language Models (PLMs) have transformed protein engineering, yet unlike their counterparts in Natural Language Processing (NLP), current PLMs exhibit a fundamental limitation: they excel in either Protein Language Understanding (PLU) or Protein Language Generation (PLG), but rarely both. This fragmentation hinders progress in protein engineering. To bridge this gap, we introduce ProLLaMA, a multitask protein language model enhanced by the Evolutionary Protein Generation Framework (EPGF). We construct a comprehensive instruction dataset containing approximately 13 million samples with over 11,000 superfamily annotations to facilitate better modeling of sequence-function landscapes. We leverage a two-stage training approach to develop ProLLaMA, a multitask LLM with protein domain expertise. Our EPGF addresses the mismatch between statistic language modeling and biological constraints through three innovations: a multi-dimensional interpretable scorer, hierarchical efficient decoding, and a probabilistic-biophysical joint selection mechanism. Extensive experiments demonstrate that ProLLaMA excels in both unconditional and controllable protein generation tasks, achieving superior structural quality metrics compared to existing PLMs. Additionally, ProLLaMA demonstrates strong understanding capabilities with a 67.1% exact match rate in superfamily prediction. EPGF significantly enhances the biological viability of generated sequences, as evidenced by improved biophysical scores (+4.3%) and structural metrics (+14.5%). The project is available at https://github.com/PKU-YuanGroup/ProLLaMA.
QF-tuner: Breaking Tradition in Reinforcement Learning
Venue: arXiv cs.LG 2025
Year: 2025

In reinforcement learning algorithms, the hyperparameters tuning method refers to choosing the optimal parameters that may increase the overall performance. Manual or random hyperparameter tuning methods can lead to different results in the reinforcement learning algorithms. In this paper, we propose a new method called QF-tuner for automatic hyperparameter tuning in the Q learning algorithm using the FOX optimization algorithm (FOX). Furthermore, a new objective function has been employed within FOX that prioritizes reward over learning error and time. QF tuner starts by running the FOX and tries to minimize the fitness value derived from observations at each iteration by executing the Q-learning algorithm. The proposed method has been evaluated using two control tasks from the OpenAI Gym: CartPole and FrozenLake. The empirical results indicate that the QF-tuner outperforms other optimization algorithms, such as particle swarm optimization (PSO), bees algorithm (BA), genetic algorithms (GA), and the random method. However, on the FrozenLake task, the QF-tuner increased rewards by 36% and reduced learning time by 26%, while on the CartPole task, it increased rewards by 57% and reduced learning time by 20%. Thus, the QF-tuner is an essential method for hyperparameter tuning in Q-learning algorithms, enabling more effective solutions to control task problems.
The Complexity of Diameter on H-free graphs
Venue: arXiv cs.DS 2025
Year: 2025

The intensively studied Diameter problem is to find the diameter of a given connected graph. We investigate, for the first time in a structured manner, the complexity of Diameter for H-free graphs, that is, graphs that do not contain a fixed graph H as an induced subgraph. We first show that if H is not a linear forest with small components, then Diameter cannot be solved in subquadratic time for H-free graphs under SETH. For some small linear forests, we do show linear-time algorithms for solving Diameter. For other linear forests H, we make progress towards linear-time algorithms by considering specific diameter values. If H is a linear forest, the maximum value of the diameter of any graph in a connected H-free graph class is some constant dmax dependent only on H. We give linear-time algorithms for deciding if a connected H-free graph has diameter dmax, for several linear forests H. In contrast, for one such linear forest H, Diameter cannot be solved in subquadratic time for H-free graphs under SETH. Moreover, we even show that, for several other linear forests H, one cannot decide in subquadratic time if a connected H-free graph has diameter dmax under SETH.
SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection
Venue: arXiv cs.CL 2025
Year: 2025

Instruction tuning (IT) is crucial to tailoring large language models (LLMs) towards human-centric interactions. Recent advancements have shown that the careful selection of a small, high-quality subset of IT data can significantly enhance the performance of LLMs. Despite this, common approaches often rely on additional models or data, which increases costs and limits widespread adoption. In this work, we propose a novel approach, termed SelectIT, that capitalizes on the foundational capabilities of the LLM itself. Specifically, we exploit the intrinsic uncertainty present in LLMs to more effectively select high-quality IT data, without the need for extra resources. Furthermore, we introduce a curated IT dataset, the Selective Alpaca, created by applying SelectIT to the Alpaca-GPT4 dataset. Empirical results demonstrate that IT using Selective Alpaca leads to substantial model ability enhancement. The robustness of SelectIT has also been corroborated in various foundation models and domain-specific tasks. Our findings suggest that longer and more computationally intensive IT data may serve as superior sources of IT, offering valuable insights for future research in this area. Data, code, and scripts are freely available at https://github.com/Blue-Raincoat/SelectIT.
Auto-Tuning for OpenMP Dynamic Scheduling applied to Full Waveform Inversion
Venue: arXiv cs.DC 2025
Year: 2025

Full Waveform Inversion (FWI) is a widely used method in seismic data processing, capable of estimating models that represent the characteristics of the geological layers of the subsurface. Because it works with a massive amount of data, the execution of this method requires much time and computational resources. Techniques such as FWI adapt well to parallel computing and can be parallelized in shared memory systems using the application programming interface (API) OpenMP. The management of parallel tasks can be performed through loop schedulers contained in OpenMP. The dynamic scheduler stands out for distributing predefined fixed-size chunk sizes to idle processing cores at runtime. It can better adapt to FWI, where data processing can be irregular. However, the relationship between the size of the chunk and the runtime is unknown. Optimization techniques can employ meta-heuristics to explore the parameter search space, avoiding testing all possible solutions. Here, we propose a strategy to use the Parameter Auto-Tuning for Shared Memory Algorithms (PATSMA), with Coupled Simulated Annealing (CSA) as its optimization method, to automatically adjust the chunk for the dynamic scheduling of wave propagation, one of the most expensive steps in FWI. Since testing each candidate chunk in the complete FWI is unpractical, our approach consists of running a PATSMA where the objective function is the runtime of the first time iteration of the first seismic shot of the first FWI iteration. The resulting chunk is then employed in all wave propagations involved in an FWI. We conducted tests to measure the runtime of an FWI using the proposed auto-tuning, varying the problem size and running on different computational environments. The results show that applying the proposed auto-tuning in an FWI reduces its runtime by up to 70.46% compared to standard OpenMP schedulers.
Multi-Parameter Mechanisms for Consumer Surplus Maximization
Venue: arXiv cs.GT 2025
Year: 2025

We consider the problem of designing auctions which maximize consumer surplus (i.e., the social welfare minus the payments charged to the buyers). In the consumer surplus maximization problem, a seller with a set of goods faces a set of strategic buyers with private values, each of whom aims to maximize their own individual utility. The seller, in contrast, aims to allocate the goods in a way which maximizes the total buyer utility. The seller must then elicit the values of the buyers in order to decide what goods to award each buyer. The canonical approach in mechanism design to ensure truthful reporting of the private information is to find appropriate prices to charge each buyer in order to align their objective with the objective of the seller. Indeed, there are many celebrated results to this end when the seller's objective is welfare maximization [Clarke, 1971, Groves, 1973, Vickrey, 1961] or revenue maximization [Myerson, 1981]. However, in the case of consumer surplus maximization the picture is less clear -- using high payments to ensure the highest value bidders are served necessarily decreases their surplus utility, but using low payments may lead the seller into serving lower value bidders. Our main result in this paper is a framework for designing mechanisms which maximize consumer surplus. We instantiate our framework in a variety of canonical multi-parameter auction settings (i.e., unit-demand bidders with heterogeneous items, multi-unit auctions, and auctions with divisible goods) and use it to design auctions achieving consumer surplus with optimal approximation guarantees against the total social welfare. Along the way, we answer an open question posed by Hartline and Roughgarden [2008] for the two bidders single item setting.
LCEN: A Nonlinear, Interpretable Feature Selection and Machine Learning Algorithm
Venue: arXiv cs.LG 2025
Year: 2025

Interpretable models can have advantages over black-box models, and interpretability is essential for the application of machine learning in critical settings, such as aviation or medicine. This article introduces the LASSO-Clip-EN (LCEN) algorithm for nonlinear, interpretable feature selection and machine learning modeling. In a wide variety of artificial and empirical datasets, LCEN constructed sparse and frequently more accurate models than other methods, including sparse, nonlinear methods, on tested datasets. LCEN was empirically observed to be robust against many issues typically present in datasets and modeling, including noise, multicollinearity, and data scarcity. As a feature selection algorithm, LCEN matched or surpassed the thresholded elastic net but was, on average, 10.3-fold faster based on our experiments. LCEN for feature selection can also rediscover multiple physical laws from empirical data. As a machine learning algorithm, when tested on processes with no known physical laws, LCEN achieved better results than many other dense and sparse methods -- including being comparable to or better than ANNs on multiple datasets.
MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning
Venue: arXiv cs.CL 2025
Year: 2025

Parameter-efficient fine-tuning (PEFT) is a popular method for tailoring pre-trained large language models (LLMs), especially as the models' scale and the diversity of tasks increase. Low-rank adaptation (LoRA) is based on the idea that the adaptation process is intrinsically low-dimensional, i.e., significant model changes can be represented with relatively few parameters. However, decreasing the rank encounters challenges with generalization errors for specific tasks when compared to full-parameter fine-tuning. We present MELoRA, a mini-ensemble low-rank adapters that uses fewer trainable parameters while maintaining a higher rank, thereby offering improved performance potential. The core idea is to freeze original pretrained weights and train a group of mini LoRAs with only a small number of parameters. This can capture a significant degree of diversity among mini LoRAs, thus promoting better generalization ability. We conduct a theoretical analysis and empirical studies on various NLP tasks. Our experimental results show that, compared to LoRA, MELoRA achieves better performance with 8 times fewer trainable parameters on natural language understanding tasks and 36 times fewer trainable parameters on instruction following tasks, which demonstrates the effectiveness of MELoRA.
Image space formalism of convolutional neural networks for k-space interpolation
Venue: arXiv cs.CV 2025
Year: 2025

Purpose: Noise resilience in image reconstructions by scan-specific robust artificial neural networks for k-space interpolation (RAKI) is linked to nonlinear activations in k-space. To gain a deeper understanding of this relationship, an image space formalism of RAKI is introduced for analyzing noise propagation analytically, identifying and characterizing image reconstruction features and to describe the role of nonlinear activations in a human readable manner. Methods: The image space formalism for RAKI inference is employed by expressing nonlinear activations in k-space as element-wise multiplications with activation masks, which transform into convolutions in image space. Jacobians of the de-aliased, coil-combined image relative to the aliased coil images can be expressed algebraically, and thus, the noise amplification is quantified analytically (g-factor maps). We analyze the role of nonlinearity for noise resilience by controlling the degree of nonlinearity in the reconstruction model via the negative slope parameter in leaky ReLU. Results: The analytical g-factor maps correspond with those obtained from Monte Carlo simulations and from an auto differentiation approach for in vivo brain images. Apparent blurring and contrast loss artifacts are identified as implications of enhanced noise resilience. These residual artifacts can be traded against noise resilience by adjusting the degree of nonlinearity in the model (Tikhonov-like regularization) in case of limited training data. The inspection of image space activations reveals an autocorrelation pattern leading to a potential center artifact. Conclusion: The image space formalism of RAKI provides the means for analytical quantitative noisepropagation analysis and human-readable visualization of the effects of the nonlinear activation functions in k-space.