OpenRaiser/Intern-Atlas huggingface dataset:
Total cost of ownership and evaluation of Google cloud resources for the ATLAS experiment at the LHC
Venue: arXiv cs.DC 2025
Year: 2025

The ATLAS Google Project was established as part of an ongoing evaluation of the use of commercial clouds by the ATLAS Collaboration, in anticipation of the potential future adoption of such resources by WLCG grid sites to fulfil or complement their computing pledges. Seamless integration of Google cloud resources into the worldwide ATLAS distributed computing infrastructure was achieved at large scale and for an extended period of time, and hence cloud resources are shown to be an effective mechanism to provide additional, flexible computing capacity to ATLAS. For the first time a total cost of ownership analysis has been performed, to identify the dominant cost drivers and explore effective mechanisms for cost control. Network usage significantly impacts the costs of certain ATLAS workflows, underscoring the importance of implementing such mechanisms. Resource bursting has been successfully demonstrated, whilst exposing the true cost of this type of activity. A follow-up to the project is underway to investigate methods for improving the integration of cloud resources in data-intensive distributed computing environments and reducing costs related to network connectivity, which represents the primary expense when extensively utilising cloud resources.
Metabook: A Mobile-to-Headset Pipeline for 3D Story Book Creation in Augmented Reality
Venue: arXiv cs.HC 2025
Year: 2025

The AR 3D book has shown significant potential in enhancing students' learning outcomes. However, the creation process of 3D books requires a significant investment of time, effort, and specialized skills. Thus, in this paper, we first conduct a three-day workshop investigating how AI can support the automated creation of 3D books. Informed by the design insights derived from the workshop, we developed Metabook, a system that enables even novice users to create 3D books from text automatically. To our knowledge, Metabook is the first system to offer end-to-end 3D book generation. A follow-up study with adult users indicates that Metabook enables inexperienced users to create 3D books, achieving reduced efforts and shortened preparation time. We subsequently recruited 22 children to examine the effects of AR 3D books on children's learning compared with paper-based books. The findings indicate that 3D books significantly enhance children's interest, improve memory retention, and reduce cognitive load, though no significant improvement was observed in comprehension. We conclude by discussing strategies for more effectively leveraging 3D books to support children's learning and offer practical recommendations for educators.
CG-FedLLM: How to Compress Gradients in Federated Fune-tuning for Large Language Models
Venue: arXiv cs.LG 2025
Year: 2025

The success of current Large-Language Models (LLMs) hinges on extensive training data that is collected and stored centrally, called Centralized Learning (CL). However, such a collection manner poses a privacy threat, and one potential solution is Federated Learning (FL), which transfers gradients, not raw data, among clients. Unlike traditional networks, FL for LLMs incurs significant communication costs due to their tremendous parameters. This study introduces an innovative approach to compress gradients to improve communication efficiency during LLM FL, formulating the new FL pipeline named CG-FedLLM. This approach integrates an encoder on the client side to acquire the compressed gradient features and a decoder on the server side to reconstruct the gradients. We also developed a novel training strategy that comprises Temporal-ensemble Gradient-Aware Pre-training (TGAP) to identify characteristic gradients of the target model and Federated AutoEncoder-Involved Fine-tuning (FAF) to compress gradients adaptively. Extensive experiments confirm that our approach reduces communication costs and improves performance (e.g., average 3 points increment compared with traditional CL- and FL-based fine-tuning with LlaMA on a well-recognized benchmark, C-Eval). This improvement is because our encoder-decoder, trained via TGAP and FAF, can filter gradients while selectively preserving critical features. Furthermore, we present a series of experimental analyses focusing on the signal-to-noise ratio, compression rate, and robustness within this privacy-centric framework, providing insight into developing more efficient and secure LLMs.
Rank Reduction Autoencoders
Venue: arXiv cs.LG 2025
Year: 2025

The choice of an appropriate bottleneck dimension and the application of effective regularization are both essential for Autoencoders to learn meaningful representations from unlabeled data. In this paper, we introduce a new class of deterministic autoencoders, Rank Reduction Autoencoders (RRAEs), which regularize their latent spaces by employing a truncated singular value decomposition (SVD) during training. In RRAEs, the bottleneck is defined by the rank of the latent matrix, thereby alleviating the dependence of the encoder/decoder architecture on the bottleneck size. This approach enabled us to propose an adaptive algorithm (aRRAEs) that efficiently determines the optimal bottleneck size during training. We empirically demonstrate that both RRAEs and aRRAEs are stable, scalable, and reliable, as they do not introduce any additional training hyperparameters. We evaluate our proposed architecture on a synthetic data set, as well as on MNIST, Fashion MNIST, and CelebA. Our results show that RRAEs offer several advantages over Vanilla AEs with both large and small latent spaces, and outperform other regularizing AE architectures.
MagicPose4D: Crafting Articulated Models with Appearance and Motion Control
Venue: arXiv cs.CV 2025
Year: 2025

With the success of 2D and 3D visual generative models, there is growing interest in generating 4D content. Existing methods primarily rely on text prompts to produce 4D content, but they often fall short of accurately defining complex or rare motions. To address this limitation, we propose MagicPose4D, a novel framework for refined control over both appearance and motion in 4D generation. Unlike current 4D generation methods, MagicPose4D accepts monocular videos or mesh sequences as motion prompts, enabling precise and customizable motion control. MagicPose4D comprises two key modules: (i) Dual-Phase 4D Reconstruction Module, which operates in two phases. The first phase focuses on capturing the model's shape using accurate 2D supervision and less accurate but geometrically informative 3D pseudo-supervision without imposing skeleton constraints. The second phase extracts the 3D motion (skeleton poses) using more accurate pseudo-3D supervision, obtained in the first phase and introduces kinematic chain-based skeleton constraints to ensure physical plausibility. Additionally, we propose a Global-local Chamfer loss that aligns the overall distribution of predicted mesh vertices with the supervision while maintaining part-level alignment without extra annotations. (ii) Cross-category Motion Transfer Module, which leverages the extracted motion from the 4D reconstruction module and uses a kinematic-chain-based skeleton to achieve cross-category motion transfer. It ensures smooth transitions between frames through dynamic rigidity, facilitating robust generalization without additional training. Through extensive experiments, we demonstrate that MagicPose4D significantly improves the accuracy and consistency of 4D content generation, outperforming existing methods in various benchmarks.
Refining Skewed Perceptions in Vision-Language Contrastive Models through Visual Representations
Venue: arXiv cs.CV 2025
Year: 2025

Large vision-language contrastive models (VLCMs), such as CLIP, have become foundational, demonstrating remarkable success across a variety of downstream tasks. Despite their advantages, these models, akin to other foundational systems, inherit biases from the disproportionate distribution of real-world data, leading to misconceptions about the actual environment. Prevalent datasets like ImageNet are often riddled with non-causal, spurious correlations that can diminish VLCM performance in scenarios where these contextual elements are absent. This study presents an investigation into how a simple linear probe can effectively distill task-specific core features from CLIP's embedding for downstream applications. Our analysis reveals that the CLIP text representations are often tainted by spurious correlations, inherited in the biased pre-training dataset. Empirical evidence suggests that relying on visual representations from CLIP, as opposed to text embedding, is more effective to refine the skewed perceptions in VLCMs, emphasizing the superior utility of visual representations in overcoming embedded biases. Our code can be found here.
$T^2$ of Thoughts: Temperature Tree Elicits Reasoning in Large Language Models
Venue: arXiv cs.CL 2025
Year: 2025

Large Language Models (LLMs) have emerged as powerful tools in artificial intelligence, especially in complex decision-making scenarios, but their static problem-solving strategies often limit their adaptability to dynamic environments. We explore the enhancement of reasoning capabilities in LLMs through Temperature Tree ($T^2$) prompting via a heuristic algorithm, termed as $T^2$ of Thoughts ($T^2oT$). The primary focus is on enhancing decision-making processes by dynamically adjusting search parameters, especially temperature, to improve accuracy without increasing computational demands. We empirically validate that our hybrid $T^2oT$ approach yields enhancements in, single-solution accuracy, multi-solution generation and text generation quality. Our findings suggest that while dynamic search depth adjustments based on temperature can yield mixed results, a fixed search depth, when coupled with adaptive capabilities of $T^2oT$, provides a more reliable and versatile problem-solving strategy. This work highlights the potential for future explorations in optimizing algorithmic interactions with foundational language models, particularly illustrated by our development for the Game of 24 and Creative Writing tasks.
Mixture of Experts Meets Prompt-Based Continual Learning
Venue: arXiv cs.LG 2025
Year: 2025

Exploiting the power of pre-trained models, prompt-based approaches stand out compared to other continual learning solutions in effectively preventing catastrophic forgetting, even with very few learnable parameters and without the need for a memory buffer. While existing prompt-based continual learning methods excel in leveraging prompts for state-of-the-art performance, they often lack a theoretical explanation for the effectiveness of prompting. This paper conducts a theoretical analysis to unravel how prompts bestow such advantages in continual learning, thus offering a new perspective on prompt design. We first show that the attention block of pre-trained models like Vision Transformers inherently encodes a special mixture of experts architecture, characterized by linear experts and quadratic gating score functions. This realization drives us to provide a novel view on prefix tuning, reframing it as the addition of new task-specific experts, thereby inspiring the design of a novel gating mechanism termed Non-linear Residual Gates (NoRGa). Through the incorporation of non-linear activation and residual connection, NoRGa enhances continual learning performance while preserving parameter efficiency. The effectiveness of NoRGa is substantiated both theoretically and empirically across diverse benchmarks and pretraining paradigms. Our code is publicly available at https://github.com/Minhchuyentoancbn/MoE_PromptCL
Transient Nonlinear Electrothermal Adjoint Sensitivity Analysis for HVDC Cable Joints
Venue: arXiv cs.CE 2025
Year: 2025

Efficient computation of sensitivities is a promising approach for efficiently of designing and optimizing high voltage direct current cable joints. This paper presents the adjoint variable method for coupled nonlinear transient electrothermal problems as an efficient approach to compute sensitivities with respect to a large number of design parameters. The method is used to compute material sensitivities of a 320kV high voltage direct current cable joint specimen. The results are validated against sensitivities obtained via the direct sensitivity method.
Push and Pull: A Framework for Measuring Attentional Agency on Digital Platforms
Venue: arXiv cs.CY 2025
Year: 2025

We propose a framework for measuring attentional agency, which we define as a user's ability to allocate attention according to their own desires, goals, and intentions on digital platforms that use statistical learning to prioritize informational content. Such platforms extend people's limited powers of attention by extrapolating their preferences to large collections of previously unconsidered informational objects. However, platforms typically also allow users to influence the attention of other users in various ways. We introduce a formal framework for measuring how much a given platform empowers each user to both pull information into their own attention and push information into the attention of others. We also use these definitions to clarify the implications of generative foundation models and other recent advances in AI for the structure and efficiency of digital platforms. We conclude with a set of possible strategies for better understanding and reshaping attentional agency online.
(Worst-Case) Optimal Adaptive Dynamic Bitvectors
Venue: arXiv cs.DS 2025
Year: 2025

While operations {\em rank} and {\em select} on static bitvectors can be supported in constant time, lower bounds show that supporting updates raises the cost per operation to $Θ(\log n/ \log\log n)$ on bitvectors holding $n$ bits. This is a shame in scenarios where updates are possible but uncommon. We develop a representation of bitvectors that we call adaptive dynamic bitvector, which uses the asymptotically optimal $n+o(n)$ bits of space and, if there are $q$ queries per update, supports all the operations in $O(\log(n/q)/\log\log n)$ amortized time. Further, we prove that this time is \new{worst-case} optimal in the cell probe model. We describe a large number of applications of our representation to other compact dynamic data structures.
Targeted Nakamoto: A Bitcoin Protocol to Balance Network Security and Carbon Emissions
Venue: arXiv cs.CR 2025
Year: 2025

In a Proof-of-Work blockchain such as Bitcoin mining hashrate is increasing in the block reward. An increase in hashrate reduces network vulnerability to attack (a reduction in security cost) while increasing carbon emissions and electricity cost (an increase in externalities cost). This implies a tradeoff in total cost at different levels of hashrate and the existence of a hashrate interval where total cost is minimized. Targeted Nakamoto is a Proof-of-Work protocol augmentation that incentivizes miners to hone in on a target hashrate interval. When hashrate is above target a ceiling is placed on the block reward a miner can receive. When hashrate is below target a floor is placed underneath the miner's block reward. Monetary neutrality is maintained by a proportional increase in spending potential among addresses holding UTXO's to match a deduction from total block reward when the ceiling is operative and a proportional reduction in spending potential among addresses holding UTXO's to match an increase over the total block reward when the floor is binding.
Output-Constrained Decision Trees
Venue: arXiv cs.LG 2025
Year: 2025

Incorporating domain-specific constraints into machine learning models is essential for generating predictions that are both accurate and feasible in real-world applications. This paper introduces new methods for training Output-Constrained Regression Trees (OCRT), addressing the limitations of traditional decision trees in constrained multi-target regression tasks. We propose three approaches: M-OCRT, which uses split-based mixed integer programming to enforce constraints; E-OCRT, which employs an exhaustive search for optimal splits and solves constrained prediction problems at each decision node; and EP-OCRT, which applies post-hoc constrained optimization to tree predictions. To illustrate their potential uses in ensemble learning, we also introduce a random forest framework working under convex feasible sets. We validate the proposed methods through a computational study both on synthetic and industry-driven hierarchical time series datasets. Our results demonstrate that imposing constraints on decision tree training results in accurate and feasible predictions.
Learning accurate and interpretable tree-based models
Venue: arXiv cs.LG 2025
Year: 2025

Decision trees and their ensembles are popular in machine learning as easy-to-understand models. Several techniques have been proposed in the literature for learning tree-based classifiers, with different techniques working well for data from different domains. In this work, we develop approaches to design tree-based learning algorithms given repeated access to data from the same domain. We study multiple formulations covering different aspects and popular techniques for learning decision tree based approaches. We propose novel parameterized classes of node splitting criteria in top-down algorithms, which interpolate between popularly used entropy and Gini impurity based criteria, and provide theoretical bounds on the number of samples needed to learn the splitting function appropriate for the data at hand. We also study the sample complexity of tuning prior parameters in Bayesian decision tree learning, and extend our results to decision tree regression. We further consider the problem of tuning hyperparameters in pruning the decision tree for classical pruning algorithms including min-cost complexity pruning. In addition, our techniques can be used to optimize the explainability versus accuracy trade-off when using decision trees. We extend our results to tuning popular tree-based ensembles, including random forests and gradient-boosted trees. We demonstrate the significance of our approach on real world datasets by learning data-specific decision trees which are simultaneously more accurate and interpretable.
Rényi Neural Processes
Venue: arXiv cs.LG 2025
Year: 2025

Neural Processes (NPs) are deep probabilistic models that represent stochastic processes by conditioning their prior distributions on a set of context points. Despite their advantages in uncertainty estimation for complex distributions, NPs enforce parameterization coupling between the conditional prior model and the posterior model. We show that this coupling amounts to prior misspecification and revisit the NP objective to address this issue. More specifically, we propose Rényi Neural Processes (RNP), a method that replaces the standard KL divergence with the Rényi divergence, dampening the effects of the misspecified prior during posterior updates. We validate our approach across multiple benchmarks including regression and image inpainting tasks, and show significant performance improvements of RNPs in real-world problems. Our extensive experiments show consistently better log-likelihoods over state-of-the-art NP models.
ComFace: Facial Representation Learning with Synthetic Data for Comparing Faces
Venue: arXiv cs.CV 2025
Year: 2025

Daily monitoring of intra-personal facial changes associated with health and emotional conditions has great potential to be useful for medical, healthcare, and emotion recognition fields. However, the approach for capturing intra-personal facial changes is relatively unexplored due to the difficulty of collecting temporally changing face images. In this paper, we propose a facial representation learning method using synthetic images for comparing faces, called ComFace, which is designed to capture intra-personal facial changes. For effective representation learning, ComFace aims to acquire two feature representations, i.e., inter-personal facial differences and intra-personal facial changes. The key point of our method is the use of synthetic face images to overcome the limitations of collecting real intra-personal face images. Facial representations learned by ComFace are transferred to three extensive downstream tasks for comparing faces: estimating facial expression changes, weight changes, and age changes from two face images of the same individual. Our ComFace, trained using only synthetic data, achieves comparable to or better transfer performance than general pre-training and state-of-the-art representation learning methods trained using real images.
Quantifying Multipolar Polarization
Venue: arXiv cs.SI 2025
Year: 2025

Studying and understanding social networks is crucial for accurately defining ideological polarization, since they enable precise modeling of social structures. One of the limitations of many methods for quantifying polarization on networks is the assumption of a two-dimensional opinion space. This prevents accurate study of multipolar systems like multi-party political systems, where modeling more than two opinion poles is beneficial. Here, I experimentally compare methods for quantifying multipolar polarization on a network and find that the average pairwise distance extension of generalized Euclidean distance conforms to several desired properties, showing its advantages over other methods. This allows the study of multipolar polarized systems based on an empirically and intuitively good metric.
LDPKiT: Superimposing Remote Queries for Privacy-Preserving Local Model Training
Venue: arXiv cs.LG 2025
Year: 2025

Users of modern Machine Learning (ML) cloud services face a privacy conundrum -- on one hand, they may have concerns about sending private data to the service for inference, but on the other hand, for specialized models, there may be no alternative but to use the proprietary model of the ML service. In this work, we present LDPKiT, a framework for non-adversarial, privacy-preserving model extraction that leverages a user's private in-distribution data while bounding privacy leakage. LDPKiT introduces a novel superimposition technique that generates approximately in-distribution samples, enabling effective knowledge transfer under local differential privacy (LDP). Experiments on Fashion-MNIST, SVHN, and PathMNIST demonstrate that LDPKiT consistently improves utility while maintaining privacy, with benefits that become more pronounced at stronger noise levels. For example, on SVHN, LDPKiT achieves nearly the same inference accuracy at $ε=1.25$ as at $ε=2.0$, yielding stronger privacy guarantees with less than a 2% accuracy reduction. We further conduct sensitivity analyses to examine the effect of dataset size on performance and provide a systematic analysis of latent space representations, offering theoretical insights into the accuracy gains of LDPKiT.