OpenRaiser/Intern-Atlas huggingface dataset:
DLEN: Dual Branch of Transformer for Low-Light Image Enhancement in Dual Domains
Venue: arXiv cs.CV 2025
Year: 2025

Low-light image enhancement (LLE) aims to improve the visual quality of images captured in poorly lit conditions, which often suffer from low brightness, low contrast, noise, and color distortions. These issues hinder the performance of computer vision tasks such as object detection, facial recognition, and autonomous driving.Traditional enhancement techniques, such as multi-scale fusion and histogram equalization, fail to preserve fine details and often struggle with maintaining the natural appearance of enhanced images under complex lighting conditions. Although the Retinex theory provides a foundation for image decomposition, it often amplifies noise, leading to suboptimal image quality. In this paper, we propose the Dual Light Enhance Network (DLEN), a novel architecture that incorporates two distinct attention mechanisms, considering both spatial and frequency domains. Our model introduces a learnable wavelet transform module in the illumination estimation phase, preserving high- and low-frequency components to enhance edge and texture details. Additionally, we design a dual-branch structure that leverages the power of the Transformer architecture to enhance both the illumination and structural components of the image.Through extensive experiments, our model outperforms state-of-the-art methods on standard benchmarks.Code is available here: https://github.com/LaLaLoXX/DLEN
FOCUS: First Order Concentrated Updating Scheme
Venue: arXiv cs.LG 2025
Year: 2025

Large language models (LLMs) demonstrate remarkable performance, and improving their pre-training process appears to be key to enhancing their capabilities further. Based on the documented success of Adam, learning rate decay, and weight decay, we hypothesize that the pre-training loss landscape features a narrowing valley structure. Through experiments with synthetic loss functions, we discover that when gradient query noise is high relative to the valley's sharpness, Adam's performance falls behind that of Signum because Adam reduces the effective step size too drastically. This observation led us to develop FOCUS, an optimizer that enhances Signum by incorporating attraction toward moving averaged parameters, allowing it to handle noise better while maintaining larger step sizes. In training GPT-2, FOCUS proves to be more stable than Signum and faster than Adam. These results suggest that gradient noise may be an underappreciated limiting factor in LLM training, and FOCUS offers promising solutions.
Bounds and Codes for General Phased Burst Errors
Venue: arXiv cs.IT 2025
Year: 2025

Phased burst errors (PBEs) are bursts of errors occurring at one or more known locations. The correction of PBEs is a classical topic in coding theory, with prominent applications such as the design of array codes for memory systems or distributed storage. We propose a general yet fine-grained approach to this problem, accounting not only for the number of bursts but also the error structure in each burst. By modeling PBEs as an error set in an adversarial channel, we investigate bounds on the maximal size of codes that can correct them. The PBE-correction capability of generalized concatenated codes is analyzed, and asymptotically good PBE-correcting codes are constructed, recovering a classical construction in a specific problem instance.
Improved Decoding of Tanner Codes
Venue: arXiv cs.IT 2025
Year: 2025

In this paper, we present improved decoding algorithms for expander-based Tanner codes. We begin by developing a randomized linear-time decoding algorithm that, under the condition that $ δd_0 > 2 $, corrects up to $ αn $ errors for a Tanner code $ T(G, C_0) $, where $ G $ is a $ (c, d, α, δ) $-bipartite expander with $n$ left vertices, and $ C_0 \subseteq \mathbb{F}_2^d $ is a linear inner code with minimum distance $ d_0 $. This result improves upon the previous work of Cheng, Ouyang, Shangguan, and Shen (RANDOM 2024), which required $ δd_0 > 3 $. We further derandomize the algorithm to obtain a deterministic linear-time decoding algorithm with the same decoding radius. Our algorithm improves upon the previous deterministic algorithm of Cheng et al. by achieving a decoding radius of $ αn $, compared with the previous radius of $ \frac{2α}{d_0(1 + 0.5cδ) }n$. Additionally, we investigate the size-expansion trade-off introduced by the recent work of Chen, Cheng, Li, and Ouyang (IEEE TIT 2023), and use it to provide new bounds on the minimum distance of Tanner codes. Specifically, we prove that the minimum distance of a Tanner code $T(G,C_0)$ is approximately $f_δ^{-1} \left( \frac{1}{d_0} \right) αn $, where $ f_δ(\cdot) $ is the Size-Expansion Function. As another application, we improve the decoding radius of our decoding algorithms from $αn$ to approximately $f_δ^{-1}\left(\frac{2}{d_0}\right)αn$.
Wrap-Decoding in Asynchronous Unsourced Multiple Access With and Without Delay Information
Venue: arXiv cs.IT 2025
Year: 2025

An asynchronous $\ka$-active-user unsourced multiple access channel (AUMAC) is a key model for uncoordinated massive access in future networks. We focus on a scenario where each transmission is subject to the maximal delay constraint ($\dm$), and the precise delay of each user is unknown at the receiver. The combined effects of asynchronicity and uncertain delays require analysis over all possible delay-codeword combinations, making the complexity of the analysis grow with $\dm$ and $\ka$ exponentially. To overcome the complexity, we employ a wrap-decoder for the AUMAC and derive a uniform upper bound on the per-user probability of error (PUPE). The numerical result shows the trade-off between energy per bit and the number of active users under various delay constraints. Furthermore, in our considered AUMAC, decoding without explicit delay information is shown to achieve nearly the same energy efficiency as decoding with perfect delay knowledge.
History-Deterministic Parity Automata: Games, Complexity, and the 2-Token Theorem
Venue: arXiv cs.FL 2025
Year: 2025

History-deterministic automata are a restricted class of nondeterministic automata where the nondeterminism while reading an input can be resolved successfully based on the prefix read so far. History-deterministic automata are exponentially more succinct than deterministic automata, while still retaining some of the algorithmic properties of deterministic automata, especially in the context of reactive synthesis. This thesis focuses on the problem of checking history-determinism for parity automata. Our main result is the 2-token theorem, due to which we obtain that checking history-determinism for parity automata with a fixed parity index can be checked in PTIME. This improves the naive EXPTIME upper bound of Henzinger and Piterman that has stood since 2006. More precisely, we show that the so-called 2-token game, which can be solved in PTIME for parity automata with a fixed parity index, characterises history-determinism for parity automata. This game was introduced by Bagnol and Kuperberg in 2018, who showed that to decide if a Büchi automaton is history-deterministic, it suffices to find the winner of the 2-token game on it. They conjectured that this 2-token game based characterisation of history-determinism extends to parity automata. We prove Bagnol and Kuperberg's conjecture that the winner of the 2-token game characterises history-determinism on parity automata. We also give a polynomial time determinisation procedure for history-deterministic Büchi automata, thus solving an open problem of Kuperberg and Skrzypczak from 2015. This result is a consequence of our proof of the 2-token theorem. Finally, we also show NP-hardness for the problem of checking history-determinism for parity automata when the parity index is not fixed. This is an improvement from the lower bound of solving parity games shown by Kuperberg and Skrzypczak in 2015.
A Hybrid Supervised and Self-Supervised Graph Neural Network for Edge-Centric Applications
Venue: arXiv cs.LG 2025
Year: 2025

This paper presents a novel graph-based deep learning model for tasks involving relations between two nodes (edge-centric tasks), where the focus lies on predicting relationships and interactions between pairs of nodes rather than node properties themselves. This model combines supervised and self-supervised learning, taking into account for the loss function the embeddings learned and patterns with and without ground truth. Additionally it incorporates an attention mechanism that leverages both node and edge features. The architecture, trained end-to-end, comprises two primary components: embedding generation and prediction. First, a graph neural network (GNN) transform raw node features into dense, low-dimensional embeddings, incorporating edge attributes. Then, a feedforward neural model processes the node embeddings to produce the final output. Experiments demonstrate that our model matches or exceeds existing methods for protein-protein interactions prediction and Gene Ontology (GO) terms prediction. The model also performs effectively with one-hot encoding for node features, providing a solution for the previously unsolved problem of predicting similarity between compounds with unknown structures.
Light commodity devices for building vehicular ad hoc networks: An experimental study
Venue: arXiv cs.NI 2025
Year: 2025

Vehicular communication networks represent both an opportunity and a challenge for providing smart mobility services by using a hybrid solution that relies on cellular connectivity and short range communications. The evaluation of this kind of network is overwhelmingly carried out in the present literature with simulations. However, the degree of realism of the results obtained is limited because simulations simplify real world interactions too much in many cases. In this article, we define an outdoor testbed to evaluate the performance of short range vehicular communications by using real world personal portable devices (smartphones, tablets, and laptops), two different PHY standards (IEEE 802.11g and IEEE 802.11a), and vehicles. Our test results on the 2.4 GHz band show that smartphones can be used to communicate vehicles within a range up to 75 m, while tablets can attain up to 125 m in mobility conditions. Moreover, we observe that vehicles equipped with laptops exchange multimedia information with nodes located further than 150 m. The communications on the 5 GHz band achieved an effective transmission range of up to 100 m. This, together with the optimization of the protocols used, could take our commodity lightweight devices to a new realm of use in the next generation of ad hoc mobility communications for moving through the city.
Metric for Evaluating Performance of Reference-Free Demorphing Methods
Venue: arXiv cs.CV 2025
Year: 2025

A facial morph is an image created by combining two (or more) face images pertaining to two (or more) distinct identities. Reference-free face demorphing inverts the process and tries to recover the face images constituting a facial morph without using any other information. However, there is no consensus on the evaluation metrics to be used to evaluate and compare such demorphing techniques. In this paper, we first analyze the shortcomings of the demorphing metrics currently used in the literature. We then propose a new metric called biometrically cross-weighted IQA that overcomes these issues and extensively benchmark current methods on the proposed metric to show its efficacy. Experiments on three existing demorphing methods and six datasets on two commonly used face matchers validate the efficacy of our proposed metric.
Vision-Language Models for Automated Chest X-ray Interpretation: Leveraging ViT and GPT-2
Venue: arXiv cs.CV 2025
Year: 2025

Radiology plays a pivotal role in modern medicine due to its non-invasive diagnostic capabilities. However, the manual generation of unstructured medical reports is time consuming and prone to errors. It creates a significant bottleneck in clinical workflows. Despite advancements in AI-generated radiology reports, challenges remain in achieving detailed and accurate report generation. In this study we have evaluated different combinations of multimodal models that integrate Computer Vision and Natural Language Processing to generate comprehensive radiology reports. We employed a pretrained Vision Transformer (ViT-B16) and a SWIN Transformer as the image encoders. The BART and GPT-2 models serve as the textual decoders. We used Chest X-ray images and reports from the IU-Xray dataset to evaluate the usability of the SWIN Transformer-BART, SWIN Transformer-GPT-2, ViT-B16-BART and ViT-B16-GPT-2 models for report generation. We aimed at finding the best combination among the models. The SWIN-BART model performs as the best-performing model among the four models achieving remarkable results in almost all the evaluation metrics like ROUGE, BLEU and BERTScore.
Efficient Algorithm for Sparse Fourier Transform of Generalized $q$-ary Functions
Venue: arXiv cs.CC 2025
Year: 2025

Computing the Fourier transform of a $q$-ary function $f:\mathbb{Z}_{q}^n\rightarrow \mathbb{R}$, which maps $q$-ary sequences to real numbers, is an important problem in mathematics with wide-ranging applications in biology, signal processing, and machine learning. Previous studies have shown that, under the sparsity assumption, the Fourier transform can be computed efficiently using fast and sample-efficient algorithms. However, in most practical settings, the function is defined over a more general space -- the space of generalized $q$-ary sequences $\mathbb{Z}_{q_1} \times \mathbb{Z}_{q_2} \times \cdots \times \mathbb{Z}_{q_n}$ -- where each $\mathbb{Z}_{q_i}$ corresponds to integers modulo $q_i$. Herein, we develop GFast, a coding theoretic algorithm that computes the $S$-sparse Fourier transform of $f$ with a sample complexity of $O(Sn)$, computational complexity of $O(Sn \log N)$, and a failure probability that approaches zero as $N=\prod_{i=1}^n q_i \rightarrow \infty$ with $S = N^δ$ for some $0 \leq δ< 1$. We show that a noise-robust version of GFast computes the transform with a sample complexity of $O(Sn^2)$ and computational complexity of $O(Sn^2 \log N)$ under the same high probability guarantees. Additionally, we demonstrate that GFast computes the sparse Fourier transform of generalized $q$-ary functions $8\times$ faster using $16\times$ fewer samples on synthetic experiments, and enables explaining real-world heart disease diagnosis and protein fitness models using up to $13\times$ fewer samples compared to existing Fourier algorithms applied to the most efficient parameterization of the models as $q$-ary functions.
CAT and DOG: Improved Codes for Private Distributed Matrix Multiplication
Venue: arXiv cs.IT 2025
Year: 2025

We present novel constructions of polynomial codes for private distributed matrix multiplication (PDMM/SDMM) using outer product partitioning (OPP). We extend the degree table framework from the literature to cyclic-addition degree tables (CATs). By using roots of unity as evaluation points, we enable modulo-addition in the table. Based on CATs, we present an explicit construction, called CATx, that requires fewer workers than existing schemes in the low-privacy regime. Additionally, we present new families of schemes based on conventional degree tables, called GASPrs and DOGrs, that outperform the state-of-the-art for a wide range of parameters.
Is Long Context All You Need? Leveraging LLM's Extended Context for NL2SQL
Venue: arXiv cs.DB 2025
Year: 2025

Large Language Models (LLMs) have demonstrated impressive capabilities across a range of natural language processing tasks. In particular, improvements in reasoning abilities and the expansion of context windows have opened new avenues for leveraging these powerful models. NL2SQL is challenging in that the natural language question is inherently ambiguous, while the SQL generation requires a precise understanding of complex data schema and semantics. One approach to this semantic ambiguous problem is to provide more and sufficient contextual information. In this work, we explore the performance and the latency trade-offs of the extended context window (a.k.a., long context) offered by Google's state-of-the-art LLM (\textit{gemini-1.5-pro}). We study the impact of various contextual information, including column example values, question and SQL query pairs, user-provided hints, SQL documentation, and schema. To the best of our knowledge, this is the first work to study how the extended context window and extra contextual information can help NL2SQL generation with respect to both accuracy and latency cost. We show that long context LLMs are robust and do not get lost in the extended contextual information. Additionally, our long-context NL2SQL pipeline based on Google's \textit{gemini-pro-1.5} achieve strong performances on various benchmark datasets without finetuning and expensive self-consistency based techniques.
InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling
Venue: arXiv cs.CV 2025
Year: 2025

This paper aims to improve the performance of video multimodal large language models (MLLM) via long and rich context (LRC) modeling. As a result, we develop a new version of InternVideo2.5 with a focus on enhancing the original MLLMs' ability to perceive fine-grained details and capture long-form temporal structure in videos. Specifically, our approach incorporates dense vision task annotations into MLLMs using direct preference optimization and develops compact spatiotemporal representations through adaptive hierarchical token compression. Experimental results demonstrate this unique design of LRC greatly improves the results of video MLLM in mainstream video understanding benchmarks (short & long), enabling the MLLM to memorize significantly longer video inputs (at least 6x longer than the original), and master specialized vision capabilities like object tracking and segmentation. Our work highlights the importance of multimodal context richness (length and fineness) in empowering MLLM's innate abilites (focus and memory), providing new insights for future research on video MLLM. Code and models are available at https://github.com/OpenGVLab/InternVideo/tree/main/InternVideo2.5
Physics of Skill Learning
Venue: arXiv cs.LG 2025
Year: 2025

We aim to understand physics of skill learning, i.e., how skills are learned in neural networks during training. We start by observing the Domino effect, i.e., skills are learned sequentially, and notably, some skills kick off learning right after others complete learning, similar to the sequential fall of domino cards. To understand the Domino effect and relevant behaviors of skill learning, we take physicists' approach of abstraction and simplification. We propose three models with varying complexities -- the Geometry model, the Resource model, and the Domino model, trading between reality and simplicity. The Domino effect can be reproduced in the Geometry model, whose resource interpretation inspires the Resource model, which can be further simplified to the Domino model. These models present different levels of abstraction and simplification; each is useful to study some aspects of skill learning. The Geometry model provides interesting insights into neural scaling laws and optimizers; the Resource model sheds light on the learning dynamics of compositional tasks; the Domino model reveals the benefits of modularity. These models are not only conceptually interesting -- e.g., we show how Chinchilla scaling laws can emerge from the Geometry model, but also are useful in practice by inspiring algorithmic development -- e.g., we show how simple algorithmic changes, motivated by these toy models, can speed up the training of deep learning models.
SplitQuant: Layer Splitting for Low-Bit Neural Network Quantization
Venue: arXiv cs.LG 2025
Year: 2025

Quantization for deep neural networks (DNNs) is the process of mapping the parameter values of DNNs from original data types to other data types of lower precision to reduce model sizes and make inference faster. Quantization often maps different original values to a single quantized value because the range of the original values is larger than the range of the quantized values. This leads to the degradation of the accuracy of the quantized DNNs. Outliers are a main cause of the degradation of quantization resolution because they enlarge the range of original values. To solve the problem, the percentile method is often used to clip outliers. However, clipping the outliers has another problem of removing the important and strong signals in the DNNs. This paper proposes SplitQuant to keep the outliers and improve the quantization resolution at the same time. SplitQuant narrows down the range of the original values and mitigates the effect of outliers by splitting each quantizable layer into three mathematically equivalent layers and applies different scaling factors. Especially, weights and biases are clustered into lower, middle and upper clusters for optimized split. By preprocessing DNNs with SplitQuant, quantization algorithms can achieve better results. SplitQuant was applied on two BERT-Tiny models and improved the accuracy of INT2 quantization by 3.3%p and 2.1%p, achieving accuracies comparable to those of the original FP32 models.
Deploying Privacy Guardrails for LLMs: A Comparative Analysis of Real-World Applications
Venue: arXiv cs.CR 2025
Year: 2025

The adoption of Large Language Models (LLMs) has revolutionized AI applications but poses significant challenges in safeguarding user privacy. Ensuring compliance with privacy regulations such as GDPR and CCPA while addressing nuanced privacy risks requires robust and scalable frameworks. This paper presents a detailed study of OneShield Privacy Guard, a framework designed to mitigate privacy risks in user inputs and LLM outputs across enterprise and open-source settings. We analyze two real-world deployments:(1) a multilingual privacy-preserving system integrated with Data and Model Factory, focusing on enterprise-scale data governance; and (2) PR Insights, an open-source repository emphasizing automated triaging and community-driven refinements. In Deployment 1, OneShield achieved a 0.95 F1 score in detecting sensitive entities like dates, names, and phone numbers across 26 languages, outperforming state-of-the-art tool such as StarPII and Presidio by up to 12\%. Deployment 2, with an average F1 score of 0.86, reduced manual effort by over 300 hours in three months, accurately flagging 8.25\% of 1,256 pull requests for privacy risks with enhanced context sensitivity. These results demonstrate OneShield's adaptability and efficacy in diverse environments, offering actionable insights for context-aware entity recognition, automated compliance, and ethical AI adoption. This work advances privacy-preserving frameworks, supporting user trust and compliance across operational contexts.