OpenRaiser/Intern-Atlas huggingface dataset:
Context Copying Modulation: The Role of Entropy Neurons in Managing Parametric and Contextual Knowledge Conflicts
Venue: arXiv cs.CL 2025
Year: 2025

The behavior of Large Language Models (LLMs) when facing contextual information that conflicts with their internal parametric knowledge is inconsistent, with no generally accepted explanation for the expected outcome distribution. Recent work has identified in autoregressive transformer models a class of neurons -- called entropy neurons -- that produce a significant effect on the model output entropy while having an overall moderate impact on the ranking of the predicted tokens. In this paper, we investigate the preliminary claim that these neurons are involved in inhibiting context copying behavior in transformers by looking at their role in resolving conflicts between contextual and parametric information. We show that entropy neurons are responsible for suppressing context copying across a range of LLMs, and that ablating them leads to a significant change in the generation process. These results enhance our understanding of the internal dynamics of LLMs when handling conflicting information.
Privacy-Preserving Decentralized Federated Learning via Explainable Adaptive Differential Privacy
Venue: arXiv cs.CR 2025
Year: 2025

Decentralized Federated Learning (DFL) enables collaborative model training without a central server, but it remains vulnerable to privacy leakage because shared model updates can expose sensitive information through inversion, reconstruction, and membership inference attacks. Differential Privacy (DP) provides formal safeguards, yet existing DP-enabled DFL methods operate as black-boxes that cannot track cumulative noise added across clients and rounds, forcing each participant to inject worst-case perturbations that severely degrade accuracy. We propose PrivateDFL, a new explainable and privacy-preserving framework that addresses this gap by combining a HyperDimensional Computing (HD) model with a transparent DP noise accountant tailored to decentralized learning. HD offers structured, noise-tolerant high-dimensional representations, while the accountant explicitly tracks cumulative perturbations so each client adds only the minimal incremental noise required to satisfy its (epsilon, delta) budget. This yields significantly tighter and more interpretable privacy-utility tradeoffs than prior DP-DFL approaches. Experiments on MNIST (image), ISOLET (speech), and UCI-HAR (wearable sensor) show that PrivateDFL consistently surpasses centralized DP-SGD and Renyi-DP Transformer and deep learning baselines under both IID and non-IID partitions, improving accuracy by up to 24.4% on MNIST, over 80% on ISOLET, and 14.7% on UCI-HAR, while reducing inference latency by up to 76 times and energy consumption by up to 36 times. These results position PrivateDFL as an efficient and trustworthy solution for privacy-sensitive pattern recognition applications such as healthcare, finance, human-activity monitoring, and industrial sensing. Future work will extend the accountant to adversarial participation, heterogeneous privacy budgets, and dynamic topologies.
Kalman Bayesian Transformer
Venue: arXiv cs.LG 2025
Year: 2025

Sequential fine-tuning of transformers is useful when new data arrive sequentially, especially with shifting distributions. Unlike batch learning, sequential learning demands that training be stabilized despite a small amount of data by balancing new information and previously learned knowledge in the pre-trained models. This challenge is further complicated when training is to be completed in latency-critical environments and learning must additionally quantify and be mediated by uncertainty. Motivated by these challenges, we propose a novel method that frames sequential fine-tuning as a posterior inference problem within a Bayesian framework. Our approach integrates closed-form moment propagation of random variables, Kalman Bayesian Neural Networks, and Taylor approximations of the moments of softmax functions. By explicitly accounting for pre-trained models as priors and adaptively balancing them against new information based on quantified uncertainty, our method achieves robust and data-efficient sequential learning. The effectiveness of our method is demonstrated through numerical simulations involving sequential adaptation of a decision transformer to tasks characterized by distribution shifts and limited memory resources.
Struct-Bench: A Benchmark for Differentially Private Structured Text Generation
Venue: arXiv cs.CL 2025
Year: 2025

Differentially private (DP) synthetic data generation is a promising technique for utilizing private datasets that otherwise cannot be exposed for model training or other analytics. While much research literature has focused on generating private unstructured text and image data, in enterprise settings, structured data (e.g., tabular) is more common, often including natural language fields or components. Existing synthetic data evaluation techniques (e.g., FID) struggle to capture the structural properties and correlations of such datasets. In this work, we propose Struct-Bench, a framework and benchmark for evaluating synthetic datasets derived from structured datasets that contain natural language data. The Struct-Bench framework requires users to provide a representation of their dataset structure as a Context-Free Grammar (CFG). Our benchmark comprises 5 real-world and 2 synthetically generated datasets, each annotated with CFGs. We show that these datasets demonstrably present a great challenge even for state-of-the-art DP synthetic data generation methods. Struct-Bench also includes reference implementations of different metrics and a leaderboard, thereby providing researchers a standardized evaluation platform to benchmark and investigate privacy-preserving synthetic data generation methods. Further, we also present a case study showing how to use Struct-Bench to improve the synthetic data quality of Private Evolution (PE) on structured data. The benchmark and the leaderboard have been publicly made available at https://struct-bench.github.io.
Maestro: Self-Improving Text-to-Image Generation via Agent Orchestration
Venue: arXiv cs.AI 2025
Year: 2025

Text-to-image (T2I) models, while offering immense creative potential, are highly reliant on human intervention, posing significant usability challenges that often necessitate manual, iterative prompt engineering over often underspecified prompts. This paper introduces Maestro, a novel self-evolving image generation system that enables T2I models to autonomously self-improve generated images through iterative evolution of prompts, using only an initial prompt. Maestro incorporates two key innovations: 1) self-critique, where specialized multimodal LLM (MLLM) agents act as 'critics' to identify weaknesses in generated images, correct for under-specification, and provide interpretable edit signals, which are then integrated by a 'verifier' agent while preserving user intent; and 2) self-evolution, utilizing MLLM-as-a-judge for head-to-head comparisons between iteratively generated images, eschewing problematic images, and evolving creative prompt candidates that align with user intents. Extensive experiments on complex T2I tasks using black-box models demonstrate that Maestro significantly improves image quality over initial prompts and state-of-the-art automated methods, with effectiveness scaling with more advanced MLLM components. This work presents a robust, interpretable, and effective pathway towards self-improving T2I generation.
MinatoLoader: Accelerating Machine Learning Training Through Efficient Data Preprocessing
Venue: arXiv cs.DC 2025
Year: 2025

Data loaders are used by Machine Learning (ML) frameworks like PyTorch and TensorFlow to apply transformations to data before feeding it into the accelerator. This operation is called data preprocessing. Data preprocessing plays an important role in the ML training workflow because if it is inefficiently pipelined with the training, it can yield high GPU idleness, resulting in important training delays. Unfortunately, existing data loaders turn out to waste GPU resources, with $76\%$ GPU idleness when using the PyTorch data loader, for example. One key source of inefficiency is the variability in preprocessing time across samples within the same dataset. Existing data loaders are oblivious to this variability, and they construct batches without any consideration of slow or fast samples. In this case, the entire batch is delayed by a single slow sample, stalling the training pipeline and resulting in head-of-line blocking. To address these inefficiencies, we present MinatoLoader, a general-purpose data loader for PyTorch that accelerates training and improves GPU utilization. MinatoLoader is designed for a single-server setup, containing multiple GPUs. It continuously prepares data in the background and actively constructs batches by prioritizing fast-to-preprocess samples, while slower samples are processed in parallel. We evaluate MinatoLoader on servers with V100 and A100 GPUs. On a machine with four A100 GPUs, MinatoLoader improves the training time of a wide range of workloads by up to $7.5\times$ ($3.6\times$ on average) over PyTorch DataLoader and Pecan, and up to $3\times$ ($2.2\times$ on average) over DALI. It also increases average GPU utilization from 46.4\% with PyTorch to 90.45\%, while preserving model accuracy and enabling faster convergence.
Understanding Computer Science Students' Career Fair Experiences: Goals, Preparation, and Outcomes
Venue: arXiv cs.CY 2025
Year: 2025

The technology industry offers exciting and diverse career opportunities, ranging from traditional software development to emerging fields such as artificial intelligence, cybersecurity, and data science. Career fairs play a crucial role in helping Computer Science (CS) students understand the various career pathways available to them in the industry. However, limited research exists on how CS students experience and benefit from these events. Through a survey of 86 students, we investigate their motivations for attending, preparation strategies, and learning outcomes, including exposure to new career paths and technologies. We envision our findings providing valuable insights for career services professionals, educators, and industry leaders in improving the career development processes of CS students.
On Closure Properties of Read-Once Oblivious Algebraic Branching Programs
Venue: arXiv cs.CC 2025
Year: 2025

We investigate the closure properties of read-once oblivious Algebraic Branching Programs (roABPs) under various natural algebraic operations and prove the following. - Non-closure under factoring: There is a sequence of explicit polynomials $(f_n(x_1,\ldots, x_n))_n$ that have $\mathsf{poly}(n)$-sized roABPs such that some irreducible factor of $f_n$ does not have roABPs of superpolynomial size in any order. - Non-closure under powering: There is a sequence of polynomials $(f_n(x_1,\ldots, x_n))_n$ with $\mathsf{poly}(n)$-sized roABPs such that any super-constant power of $f_n$ does not have roABPs of polynomial size in any order (and $f_n^n$ requires exponential size in any order). - Non-closure under symmetric compositions: There are symmetric polynomials $(f_n(e_1,\ldots, e_n))_n$ that have roABPs of polynomial size such that $f_n(x_1,\ldots, x_n)$ do not have roABPs of subexponential size. (Here, $e_1,\ldots, e_n$ denote the elementary symmetric polynomials in $n$ variables.) These results should be viewed in light of known results on models such as algebraic circuits, (general) algebraic branching programs, formulas and constant-depth circuits, all of which are known to be closed under these operations. To prove non-closure under factoring, we construct hard polynomials based on expander graphs using gadgets that lift their hardness from sparse polynomials to roABPs. For symmetric compositions, we show that the circulant polynomial requires roABPs of exponential size in every variable order.
PolyTruth: Multilingual Disinformation Detection using Transformer-Based Language Models
Venue: arXiv cs.CL 2025
Year: 2025

Disinformation spreads rapidly across linguistic boundaries, yet most AI models are still benchmarked only on English. We address this gap with a systematic comparison of five multilingual transformer models: mBERT, XLM, XLM-RoBERTa, RemBERT, and mT5 on a common fake-vs-true machine learning classification task. While transformer-based language models have demonstrated notable success in detecting disinformation in English, their effectiveness in multilingual contexts still remains up for debate. To facilitate evaluation, we introduce PolyTruth Disinfo Corpus, a novel corpus of 60,486 statement pairs (false claim vs. factual correction) spanning over twenty five languages that collectively cover five language families and a broad topical range from politics, health, climate, finance, and conspiracy, half of which are fact-checked disinformation claims verified by an augmented MindBugs Discovery dataset. Our experiments revealed performance variations. Models such as RemBERT achieved better overall accuracy, particularly excelling in low-resource languages, whereas models like mBERT and XLM exhibit considerable limitations when training data is scarce. We provide a discussion of these performance patterns and implications for real-world deployment. The dataset is publicly available on our GitHub repository to encourage further experimentation and advancement. Our findings illuminate both the potential and the current limitations of AI systems for multilingual disinformation detection.
Every Camera Effect, Every Time, All at Once: 4D Gaussian Ray Tracing for Physics-based Camera Effect Data Generation
Venue: arXiv cs.CV 2025
Year: 2025

Common computer vision systems typically assume ideal pinhole cameras but fail when facing real-world camera effects such as fisheye distortion and rolling shutter, mainly due to the lack of learning from training data with camera effects. Existing data generation approaches suffer from either high costs, sim-to-real gaps or fail to accurately model camera effects. To address this bottleneck, we propose 4D Gaussian Ray Tracing (4D-GRT), a novel two-stage pipeline that combines 4D Gaussian Splatting with physically-based ray tracing for camera effect simulation. Given multi-view videos, 4D-GRT first reconstructs dynamic scenes, then applies ray tracing to generate videos with controllable, physically accurate camera effects. 4D-GRT achieves the fastest rendering speed while performing better or comparable rendering quality compared to existing baselines. Additionally, we construct eight synthetic dynamic scenes in indoor environments across four camera effects as a benchmark to evaluate generated videos with camera effects.
Uniquely-Decodable Coding for Zero-Error Network Function Computation
Venue: arXiv cs.IT 2025
Year: 2025

We consider uniquely-decodable coding for zero-error network function computation, where in a directed acyclic graph, the single sink node is required to compute with zero error a target function multiple times, whose arguments are the information sources generated at a set of source nodes. We are interested in the computing capacity from the information theoretic point of view, which is defined as the infimum of the maximum expected number of bits transmitted on all the edges for computing the target function once on average. We first prove some new results on clique entropy, in particular, the substitution lemma of clique entropy for probabilistic graphs with a certain condition. With them, we prove a lower bound on the computing capacity associated with clique entropies of the induced characteristic graphs, where the obtained lower bound is applicable to arbitrary network topologies, arbitrary information sources, and arbitrary target functions. By refining the probability distribution of information sources, we further strictly improve the obtained lower bound. In addition, we compare uniquely-decodable network function-computing coding and fixed-length network function-computing coding, and show that the former indeed outperforms the latter in terms of the computing capacity. Therein, we provide a novel graph-theoretic explanation of the key parameter in the best known bound on the computing capacity for fixed-length network function-computing codes, which would be helpful to improve the existing results.
Emoanti: audio anti-deepfake with refined emotion-guided representations
Venue: arXiv cs.SD 2025
Year: 2025

Audio deepfake is so sophisticated that the lack of effective detection methods is fatal. While most detection systems primarily rely on low-level acoustic features or pretrained speech representations, they frequently neglect high-level emotional cues, which can offer complementary and potentially anti-deepfake information to enhance generalization. In this work, we propose a novel audio anti-deepfake system that utilizes emotional features (EmoAnti) by exploiting a pretrained Wav2Vec2 (W2V2) model fine-tuned on emotion recognition tasks, which derives emotion-guided representations, then designing a dedicated feature extractor based on convolutional layers with residual connections to effectively capture and refine emotional characteristics from the transformer layers outputs. Experimental results show that our proposed architecture achieves state-of-the-art performance on both the ASVspoof2019LA and ASVspoof2021LA benchmarks, and demonstrates strong generalization on the ASVspoof2021DF dataset. Our proposed approach's code is available at Anonymous GitHub1.
Automatic Generation of a Cryptography Misuse Taxonomy Using Large Language Models
Venue: arXiv cs.CR 2025
Year: 2025

The prevalence of cryptographic API misuse (CAM) is compromising the effectiveness of cryptography and in turn the security of modern systems and applications. Despite extensive efforts to develop CAM detection tools, these tools typically rely on a limited set of predefined rules from human-curated knowledge. This rigid, rule-based approach hinders adaptation to evolving CAM patterns in real practices. We propose leveraging large language models (LLMs), trained on publicly available cryptography-related data, to automatically detect and classify CAMs in real-world code to address this limitation. Our method enables the development and continuous expansion of a CAM taxonomy, supporting developers and detection tools in tracking and understanding emerging CAM patterns. Specifically, we develop an LLM-agnostic prompt engineering method to guide LLMs in detecting CAM instances from C/C++, Java, Python, and Go code, and then classifying them into a hierarchical taxonomy. Using a data set of 3,492 real-world software programs, we demonstrate the effectiveness of our approach with mainstream LLMs, including GPT, Llama, Gemini, and Claude. It also allows us to quantitatively measure and compare the performance of these LLMs in analyzing CAM in realistic code. Our evaluation produced a taxonomy with 279 base CAM categories, 36 of which are not addressed by existing taxonomies. To validate its practical value, we encode 11 newly identified CAM types into detection rules and integrate them into existing tools. Experiments show that such integration expands the tools' detection capabilities.
OpenUrban3D: Annotation-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds
Venue: arXiv cs.CV 2025
Year: 2025

Open-vocabulary semantic segmentation enables models to recognize and segment objects from arbitrary natural language descriptions, offering the flexibility to handle novel, fine-grained, or functionally defined categories beyond fixed label sets. While this capability is crucial for large-scale urban point clouds that support applications such as digital twins, smart city management, and urban analytics, it remains largely unexplored in this domain. The main obstacles are the frequent absence of high-quality, well-aligned multi-view imagery in large-scale urban point cloud datasets and the poor generalization of existing three-dimensional (3D) segmentation pipelines across diverse urban environments with substantial variation in geometry, scale, and appearance. To address these challenges, we present OpenUrban3D, the first 3D open-vocabulary semantic segmentation framework for large-scale urban scenes that operates without aligned multi-view images, pre-trained point cloud segmentation networks, or manual annotations. Our approach generates robust semantic features directly from raw point clouds through multi-view, multi-granularity rendering, mask-level vision-language feature extraction, and sample-balanced fusion, followed by distillation into a 3D backbone model. This design enables zero-shot segmentation for arbitrary text queries while capturing both semantic richness and geometric priors. Extensive experiments on large-scale urban benchmarks, including SensatUrban and SUM, show that OpenUrban3D achieves significant improvements in both segmentation accuracy and cross-scene generalization over existing methods, demonstrating its potential as a flexible and scalable solution for 3D urban scene understanding.
Tracer: A Forensic Framework for Detecting Fraudulent Speedruns from Game Replays
Venue: arXiv cs.HC 2025
Year: 2025

Speedrun, a practice of completing a game as quickly as possible, has fostered vibrant communities driven by creativity, competition, and mastery of game mechanics and motor skills. However, this contest also attracts malicious actors as financial incentives come into play. As media and software manipulation techniques advance - such as spliced footage, modified game software and live stream with staged setups - forged speedruns have become increasingly difficult to detect. Volunteer-driven communities invest significant effort to verify submissions, yet the process remains slow, inconsistent, and reliant on informal expertise. In high-profile cases, fraudulent runs have gone undetected for years, allowing perpetrators to gain fame and financial benefits through monetised viewership, sponsorships, donations, and community bounties. To address this gap, we propose Tracer, Tamper Recognition via Analysis of Continuity and Events in game Runs, a modular framework for identifying artefacts of manipulation in speedrun submissions. Tracer provides structured guidelines across audiovisual, physical, and cyberspace dimensions, systematically documenting dispersed in-game knowledge and previously reported fraudulent cases to enhance verification efficiency.
A Broadcast Channel Framework for MIMO-OFDM Integrated Sensing and Communication
Venue: arXiv cs.IT 2025
Year: 2025

Integrated sensing and communication (ISAC) is expected to be one of the major features of 6G wireless networks. In an ISAC system, communications and sensing functionalities are jointly performed using the same waveform, frequency band and hardware, thereby enabling various use cases such as in cyber physical systems, digital twin and smart cities. A major challenge to the design and analysis of ISAC is a unified framework that incorporates the two distinct functions. By viewing ISAC as a type of broadcast channel, in this paper, we propose a unified ISAC framework in which communication and sensing signals are broadcast to the actual communication users and virtual sensing users. This framework allows the application of existing multiplexing schemes, such as dirty paper coding (DPC) and frequency division multiplexing (FDM) that have been intensively studied in data communications and information theory. Within this framework, we propose different superposition coding schemes, for cases when the sensing waveform is known or unknown to the communication receiver. We propose the waveform optimization algorithms in a multiple-input multiple-output (MIMO) setting accounting for the effects of clutter and Doppler shift. The proposed framework is numerically evaluated for different schemes under various sensing and communications performance metrics.