OpenRaiser/Intern-Atlas huggingface dataset:
Towards Pareto Optimal Throughput in Small Language Model Serving
Venue: arXiv cs.CL 2025
Year: 2025

Large language models (LLMs) have revolutionized the state-of-the-art of many different natural language processing tasks. Although serving LLMs is computationally and memory demanding, the rise of Small Language Models (SLMs) offers new opportunities for resource-constrained users, who now are able to serve small models with cutting-edge performance. In this paper, we present a set of experiments designed to benchmark SLM inference at performance and energy levels. Our analysis provides a new perspective in serving, highlighting that the small memory footprint of SLMs allows for reaching the Pareto-optimal throughput within the resource capacity of a single accelerator. In this regard, we present an initial set of findings demonstrating how model replication can effectively improve resource utilization for serving SLMs.
Space Physiology and Technology: Musculoskeletal Adaptations, Countermeasures, and Opportunities for Wearable Systems
Venue: arXiv cs.RO 2025
Year: 2025

Space poses significant challenges for humans, leading to physiological adaptations in response to an environment vastly different from Earth. A comprehensive understanding of these physiological adaptations is needed to devise effective countermeasures to support human life in space. This narrative review first focuses on the impact of the environment in space on the musculoskeletal system. It highlights the complex interplay between bone and muscle adaptations and their implications on astronaut health. Despite advances in current countermeasures, such as resistive exercise and pharmacological interventions, they remain partially effective, bulky, and resource-intensive, posing challenges for future missions aboard compact spacecraft. This review proposes wearable sensing and robotic technology as a promising alternative to overcome these limitations. Wearable systems, such as sensor-integrated suits and (soft) exoskeletons, can provide real-time monitoring, dynamic loading, and exercise protocols tailored to individual needs. These systems are lightweight, modular, and capable of operating in confined environments, making them ideal for long-duration missions. In addition to space applications, wearable technologies hold significant promise for terrestrial uses, supporting rehabilitation and assistance for the ageing population, individuals with musculoskeletal disorders, and enhance physical performance in healthy users. By integrating advanced materials, sensors and actuators, and intelligent and energy-efficient control, these technologies can bridge gaps in current countermeasures while offering broader applications on Earth.
Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single View
Venue: arXiv cs.CV 2025
Year: 2025

Single-view 3D reconstruction is currently approached from two dominant perspectives: reconstruction of scenes with limited diversity using 3D data supervision or reconstruction of diverse singular objects using large image priors. However, real-world scenarios are far more complex and exceed the capabilities of these methods. We therefore propose a hybrid method following a divide-and-conquer strategy. We first process the scene holistically, extracting depth and semantic information, and then leverage an object-level method for the detailed reconstruction of individual components. By splitting the problem into simpler tasks, our system is able to generalize to various types of scenes without retraining or fine-tuning. We purposely design our pipeline to be highly modular with independent, self-contained modules, to avoid the need for end-to-end training of the whole system. This enables the pipeline to naturally improve as future methods can replace the individual modules. We demonstrate the reconstruction performance of our approach on both synthetic and real-world scenes, comparing favorable against prior works. Project page: https://andreeadogaru.github.io/Gen3DSR
Personalized LLM Response Generation with Parameterized Memory Injection
Venue: arXiv cs.CL 2025
Year: 2025

Large Language Models (LLMs) have exhibited remarkable proficiency in comprehending and generating natural language. On the other hand, personalized LLM response generation holds the potential to offer substantial benefits for individuals in critical areas such as medical. Existing research has explored memory-augmented methods to prompt the LLM with pre-stored user-specific knowledge for personalized response generation in terms of new queries. We contend that such paradigm is unable to perceive fine-granularity information. In this study, we propose a novel \textbf{M}emory-\textbf{i}njected approach using parameter-efficient fine-tuning (PEFT) and along with a Bayesian Optimisation searching strategy to achieve \textbf{L}LM \textbf{P}ersonalization(\textbf{MiLP}).
Robust Concept Erasure Using Task Vectors
Venue: arXiv cs.CV 2025
Year: 2025

With the rapid growth of text-to-image models, a variety of techniques have been suggested to prevent undesirable image generations. Yet, these methods often only protect against specific user prompts and have been shown to allow unsafe generations with other inputs. Here we focus on unconditionally erasing a concept from a text-to-image model rather than conditioning the erasure on the user's prompt. We first show that compared to input-dependent erasure methods, concept erasure that uses Task Vectors (TV) is more robust to unexpected user inputs, not seen during training. However, TV-based erasure can also affect the core performance of the edited model, particularly when the required edit strength is unknown. To this end, we propose a method called Diverse Inversion, which we use to estimate the required strength of the TV edit. Diverse Inversion finds within the model input space a large set of word embeddings, each of which induces the generation of the target concept. We find that encouraging diversity in the set makes our estimation more robust to unexpected prompts. Finally, we show that Diverse Inversion enables us to apply a TV edit only to a subset of the model weights, enhancing the erasure capabilities while better maintaining the core functionality of the model.
Does Biomedical Training Lead to Better Medical Performance?
Venue: arXiv cs.CL 2025
Year: 2025

Large Language Models (LLMs) are expected to significantly contribute to patient care, diagnostics, and administrative processes. Emerging biomedical LLMs aim to address healthcare-specific challenges, including privacy demands and computational constraints. Assessing the models' suitability for this sensitive application area is of the utmost importance. However, biomedical training has not been systematically evaluated on medical tasks. This study investigates the effect of biomedical training in the context of six practical medical tasks evaluating $25$ models. In contrast to previous evaluations, our results reveal a performance decline in nine out of twelve biomedical models after fine-tuning, particularly on tasks involving hallucinations, ICD10 coding, and instruction adherence. General-domain models like Meta-Llama-3.1-70B-Instruct outperformed their biomedical counterparts, indicating a trade-off between domain-specific fine-tuning and general medical task performance. We open-source all evaluation scripts and datasets at https://github.com/TIO-IKIM/CLUE to support further research in this critical area.
Designing Robots to Help Women
Venue: arXiv cs.RO 2025
Year: 2025

Robots are being designed to help people in an increasing variety of settings--but seemingly little attention has been given so far to the specific needs of women, who represent roughly half of the world's population but are highly underrepresented in robotics. Here we used a speculative prototyping approach to explore this expansive design space: First, we identified some potential challenges of interest, including crimes and illnesses that disproportionately affect women, as well as potential opportunities for designers, which were visualized in five sketches. Then, one of the sketched scenarios was further explored by developing a prototype, of a robotic helper drone equipped with computer vision to detect hidden cameras that could be used to spy on women. While object detection introduced some errors, hidden cameras were identified with a reasonable accuracy of 80% (Intersection over Union (IoU) score: 0.40). Our aim is that the identified challenges and opportunities could help spark discussion and inspire designers, toward realizing a safer, more inclusive future through responsible use of technology.
Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
Venue: arXiv cs.LG 2025
Year: 2025

LLM-based auto-annotators have become a key component of the LLM development process due to their cost-effectiveness and scalability compared to human-based evaluation. However, these auto-annotators can introduce biases that are hard to remove. Even simple, known confounders such as preference for longer outputs remain in existing automated evaluation metrics. We propose a simple regression analysis approach for controlling biases in auto-evaluations. As a real case study, we focus on reducing the length bias of AlpacaEval, a fast and affordable benchmark for instruction-tuned LLMs that uses LLMs to estimate response quality. Despite being highly correlated with human preferences, AlpacaEval is known to favor models that generate longer outputs. We introduce a length-controlled AlpacaEval that aims to answer the counterfactual question: "What would the preference be if the model's and baseline's output had the same length?" To achieve this, we first fit a generalized linear model to predict the biased auto-annotator's preferences based on the mediators we want to control for (length difference) and other relevant features. We then obtain length-controlled preferences by predicting preferences while conditioning the GLM with a zero difference in lengths. Length-controlling not only improves the robustness of the metric to manipulations in model verbosity, but we also find that it increases the Spearman correlation with LMSYS Chatbot Arena from 0.94 to 0.98.
TCAN: Text-oriented Cross Attention Network for Multimodal Sentiment Analysis
Venue: arXiv cs.MM 2025
Year: 2025

Multimodal Sentiment Analysis (MSA) endeavors to understand human sentiment by leveraging language, visual, and acoustic modalities. Despite the remarkable performance exhibited by previous MSA approaches, the presence of inherent multimodal heterogeneities poses a challenge, with the contribution of different modalities varying considerably. Past research predominantly focused on improving representation learning techniques and feature fusion strategies. However, many of these efforts overlooked the variation in semantic richness among different modalities, treating each modality uniformly. This approach may lead to underestimating the significance of strong modalities while overemphasizing the importance of weak ones. Motivated by these insights, we introduce a Text-oriented Cross-Attention Network (TCAN), emphasizing the predominant role of the text modality in MSA. Specifically, for each multimodal sample, by taking unaligned sequences of the three modalities as inputs, we initially allocate the extracted unimodal features into a visual-text and an acoustic-text pair. Subsequently, we implement self-attention on the text modality and apply text-queried cross-attention to the visual and acoustic modalities. To mitigate the influence of noise signals and redundant features, we incorporate a gated control mechanism into the framework. Additionally, we introduce unimodal joint learning to gain a deeper understanding of homogeneous emotional tendencies across diverse modalities through backpropagation. Experimental results demonstrate that TCAN consistently outperforms state-of-the-art MSA methods on two datasets (CMU-MOSI and CMU-MOSEI).
On the Limitations of Large Language Models (LLMs): False Attribution
Venue: arXiv cs.CL 2025
Year: 2025

In this work, we introduce a new hallucination metric - Simple Hallucination Index (SHI) and provide insight into one important limitation of the parametric knowledge of large language models (LLMs), i.e. false attribution. The task of automatic author attribution for relatively small chunks of text is an important NLP task but can be challenging. We empirically evaluate the power of 3 open SotA LLMs in zero-shot setting (Gemma-7B, Mixtral 8x7B, and LLaMA-2-13B). We acquired the top 10 most popular books of a month, according to Project Gutenberg, divided each one into equal chunks of 400 words, and prompted each LLM to predict the author. We then randomly sampled 162 chunks per book for human evaluation, based on the error margin of 7% and a confidence level of 95%. The average results show that Mixtral 8x7B has the highest prediction accuracy, the lowest SHI, and a Pearson's correlation (r) of 0.724, 0.263, and -0.9996, respectively, followed by LLaMA-2-13B and Gemma-7B. However, Mixtral 8x7B suffers from high hallucinations for 3 books, rising as high as a SHI of 0.87 (in the range 0-1, where 1 is the worst). The strong negative correlation of accuracy and SHI, given by r, demonstrates the fidelity of the new hallucination metric, which may generalize to other tasks. We also show that prediction accuracies correlate positively with the frequencies of Wikipedia instances of the book titles instead of the downloads and we perform error analyses of predictions. We publicly release the annotated chunks of data and our codes to aid the reproducibility and evaluation of other models.
Range Longest Increasing Subsequence and its Relatives
Venue: arXiv cs.DS 2025
Year: 2025

In this work, we present a plethora of results for the range longest increasing subsequence problem (Range-LIS) and its variants. The input to RLIS is a sequence $S$ of $n$ real numbers and a collection $Q$ of $m$ query ranges, and for each query in $Q$, the goal is to report the LIS of the sequence $S$ restricted to that query. Our two main results are for the following generalizations of the RLIS problem. 2D range queries: In this variant of the RLIS problem, each query is a pair of ranges, one of indices and the other of values, and we provide a randomized algorithm with running time $\tilde{O}(m n^{1/2} + n^{3/2}) + O(k)$, where $k$ is the cumulative length of the $m$ output subsequences. This improves on the elementary $O(mn)$-time algorithm when $m$ is at least $n^{1/2}$. Previously, the only known result breaking the quadratic barrier was due to Tiskin [SODA'10], which could only handle 1D range queries (i.e., each query was a range of indices) and also just outputted the length of the LIS (instead of reporting the subsequence achieving that length). Colored sequences: In this variant of the RLIS problem, each element in $S$ is colored, and for each query in $Q$, the goal is to report a monochromatic LIS contained in the sequence $S$ restricted to that query. For 2D queries, we provide a randomized algorithm for this colored version with running time $\tilde{O}(m n^{2/3} + n^{5/3}) + O(k)$. Moreover, for 1D queries, we provide an improved algorithm with running time $\tilde{O}(m n^{1/2} + n^{3/2}) + O(k)$. Thus, we again improve on the elementary $O(mn)$-time algorithm. Additionally, assuming the well-known Combinatorial Boolean Matrix Multiplication Hypothesis, we prove that the running time for 1D queries is essentially tight for combinatorial algorithms.
MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators
Venue: arXiv cs.CV 2025
Year: 2025

Recent advances in Text-to-Video generation (T2V) have achieved remarkable success in synthesizing high-quality general videos from textual descriptions. A largely overlooked problem in T2V is that existing models have not adequately encoded physical knowledge of the real world, thus generated videos tend to have limited motion and poor variations. In this paper, we propose \textbf{MagicTime}, a metamorphic time-lapse video generation model, which learns real-world physics knowledge from time-lapse videos and implements metamorphic generation. First, we design a MagicAdapter scheme to decouple spatial and temporal training, encode more physical knowledge from metamorphic videos, and transform pre-trained T2V models to generate metamorphic videos. Second, we introduce a Dynamic Frames Extraction strategy to adapt to metamorphic time-lapse videos, which have a wider variation range and cover dramatic object metamorphic processes, thus embodying more physical knowledge than general videos. Finally, we introduce a Magic Text-Encoder to improve the understanding of metamorphic video prompts. Furthermore, we create a time-lapse video-text dataset called \textbf{ChronoMagic}, specifically curated to unlock the metamorphic video generation ability. Extensive experiments demonstrate the superiority and effectiveness of MagicTime for generating high-quality and dynamic metamorphic videos, suggesting time-lapse video generation is a promising path toward building metamorphic simulators of the physical world. Code: https://github.com/PKU-YuanGroup/MagicTime
SoK: On Gradient Leakage in Federated Learning
Venue: arXiv cs.CR 2025
Year: 2025

Federated learning (FL) facilitates collaborative model training among multiple clients without raw data exposure. However, recent studies have shown that clients' private training data can be reconstructed from shared gradients in FL, a vulnerability known as gradient inversion attacks (GIAs). While GIAs have demonstrated effectiveness under \emph{ideal settings and auxiliary assumptions}, their actual efficacy against \emph{practical FL systems} remains under-explored. To address this gap, we conduct a comprehensive study on GIAs in this work. We start with a survey of GIAs that establishes a timeline to trace their evolution and develops a systematization to uncover their inherent threats. By rethinking GIA in practical FL systems, three fundamental aspects influencing GIA's effectiveness are identified: \textit{training setup}, \textit{model}, and \textit{post-processing}. Guided by these aspects, we perform extensive theoretical and empirical evaluations of SOTA GIAs across diverse settings. Our findings highlight that GIA is notably \textit{constrained}, \textit{fragile}, and \textit{easily defensible}. Specifically, GIAs exhibit inherent limitations against practical local training settings. Additionally, their effectiveness is highly sensitive to the trained model, and even simple post-processing techniques applied to gradients can serve as effective defenses. Our work provides crucial insights into the limited threats of GIAs in practical FL systems. By rectifying prior misconceptions, we hope to inspire more accurate and realistic investigations on this topic.
Predefined Software Environment Runtimes As A Measure For Reproducibility
Venue: arXiv cs.MS 2025
Year: 2025

As part of Mathematical Research Data Initiative (MaRDI), we have developed a way to preserve a software package into an easy to deploy and use sandbox environment we call a "runtime", via a program we developed called MaPS : MaRDI Packaging System. The program relies on Linux user namespaces to isolate a library environment from the host system, making the sandboxed software reproducible on other systems, with minimal effort. Moreover an overlay filesystem makes local edits persistent. This project will aid reproducibility efforts of research papers: both mathematical and from other disciplines. As a proof of concept, we provide runtimes for the OSCAR Computer Algebra System, polymake software for research in polyhedral geometry, and VIBRANT Virus Identification By iteRative ANnoTation. The software is in a prerelease state: the interface for creating, deploying, and executing runtimes is final, and an interface for easily publishing runtimes is under active development. We thus propose publishing predefined, distributable software environment runtimes along with research papers in an effort to make research with software based results reproducible.
Hiden Topics in Robotic Process Automation -- an Approach based on AI
Venue: arXiv cs.CY 2025
Year: 2025

Robotic Process Automation (RPA) has rapidly evolved into a widely recognized and influential software technology. Its growing relevance has sparked diverse research efforts across various disciplines. This study aims to map the scientific landscape of RPA by identifying key thematic areas, tracking their development over time, and assessing their academic impact. To achieve this, we apply an unsupervised machine learning technique Latent Dirichlet Allocation (LDA) to analyze the abstracts of over 2,000 scholarly articles. Our analysis reveals 100 distinct research topics, with 15 of the most prominent themes featured in a science map designed to support future exploration and understanding of RPA's expanding research frontier.
THOUGHTSCULPT: Reasoning with Intermediate Revision and Search
Venue: arXiv cs.CL 2025
Year: 2025

We present THOUGHTSCULPT, a general reasoning and search method for tasks with outputs that can be decomposed into components. THOUGHTSCULPT explores a search tree of potential solutions using Monte Carlo Tree Search (MCTS), building solutions one action at a time and evaluating according to any domain-specific heuristic, which in practice is often simply an LLM evaluator. Critically, our action space includes revision actions: THOUGHTSCULPT may choose to revise part of its previous output rather than continuing to build the rest of its output. Empirically, THOUGHTSCULPT outperforms state-of-the-art reasoning methods across three challenging tasks: Story Outline Improvement (up to +30% interestingness), Mini-Crosswords Solving (up to +16% word success rate), and Constrained Generation (up to +10% concept coverage).
A Quantitative Security Analysis of S-boxes in the NIST Lightweight Cryptography Finalists
Venue: arXiv cs.CR 2025
Year: 2025

Lightweight cryptography was primarily inspired by the design criteria of symmetric cryptography. It plays a vital role in ensuring the security, privacy, and reliability of microelectronic devices without compromising the overall functionality and efficiency. However, the increasingly platform specific design requirements prompted the development of a standard lightweight algorithm. In 2017, NIST put forward security requirements for a standard lightweight scheme - security strength of at least 112 bits against known cryptanalysis attacks, mitigation against side channel and fault injection attacks, and implementation efficiency. After three rounds of review, ASCON was crowned as the winner of the competition. Evaluating the individual components used in any cryptographic algorithm is an important step in the verification of security claims. A fundamental component used to ensure Shannon's property of confusion in cryptographic primitives is an S-box. Hence, the quality of an S-box is a significant contributing factor in the security strength of a cipher. In this paper, we evaluate the S-boxes of 6 NIST LWC competition finalists based on well-known cryptographic properties, and comment on how the results reflect upon NIST security requirements. Our findings have revealed that these S-boxes do not comply with the basic notions of avalanche, making it vulnerable to high-order sophisticated cryptanalysis.