OpenRaiser/Intern-Atlas huggingface dataset:
Qualitative Mechanism Independence
Venue: arXiv cs.IT 2025
Year: 2025

We define what it means for a joint probability distribution to be compatible with a set of independent causal mechanisms, at a qualitative level -- or, more precisely, with a directed hypergraph ${\mathcal{A}}$, which is the qualitative structure of a probabilistic dependency graph (PDG). When ${\mathcal{A}}$ represents a qualitative Bayesian network, QIM-compatibility with ${\mathcal{A}}$ reduces to satisfying the appropriate conditional independencies. But giving semantics to hypergraphs using QIM-compatibility lets us do much more. For one thing, we can capture functional dependencies. For another, we can capture important aspects of causality using compatibility: we can use compatibility to understand cyclic causal graphs, and to demonstrate structural compatibility, we must essentially produce a causal model. Finally, QIM-compatibility has deep connections to information theory. Applying our notion to cyclic structures helps to clarify a longstanding conceptual issue in information theory.
TinyLLaVA-Video: Towards Smaller LMMs for Video Understanding with Group Resampler
Venue: arXiv cs.CV 2025
Year: 2025

Video behavior recognition and scene understanding are fundamental tasks in multimodal intelligence, serving as critical building blocks for numerous real-world applications. Through large multimodal models (LMMs) have achieved remarkable progress in video understanding, most existing open-source models rely on over 7B parameters and require large-scale datasets for training, making them resource-intensive and inaccessible to many researchers. Furthermore, lightweight models face persistent challenges in effectively processing long visual sequences and temporal understanding. In this work, we introduce TinyLLaVA-Video, a lightweight yet powerful video understanding model with approximately 3.6B parameters. The cornerstone of our design is the video-level group resampler, a novel mechanism that significantly reduces and controls the number of visual tokens at the video level. Unlike traditional image-level resampler, our approach effectively mitigates redundancy while enhancing temporal comprehension, leading to improved performance on video-based tasks. In addition, TinyLLaVA-Video demonstrates exceptional efficiency, requiring only one day of training on 8 A100-40G GPUs. It surpasses several existing 7B-parameter models on multiple benchmarks. We believe this work provides a valuable foundation for future research on lightweight video understanding models. The code and weights is available at https://github.com/ZhangXJ199/TinyLLaVA-Video.
Studying Behavioral Addiction by Combining Surveys and Digital Traces: A Case Study of TikTok
Venue: arXiv cs.SI 2025
Year: 2025

Opaque algorithms disseminate and mediate the content that users consume on online social media platforms. This algorithmic mediation serves users with contents of their liking, on the other hand, it may cause several inadvertent risks to society at scale. While some of these risks, e.g., filter bubbles or dissemination of hateful content, are well studied in the community, behavioral addiction, designated by the Digital Services Act (DSA) as a potential systemic risk, has been understudied. In this work, we aim to study if one can effectively diagnose behavioral addiction using digital data traces from social media platforms. Focusing on the TikTok short-format video platform as a case study, we employ a novel mixed methodology of combining survey responses with data donations of behavioral traces. We survey 1590 TikTok users and stratify them into three addiction groups (i.e., less/moderately/highly likely addicted). Then, we obtain data donations from 107 surveyed participants. By analyzing users' data we find that, among others, highly likely addicted users spend more time watching TikTok videos and keep coming back to TikTok throughout the day, indicating a compulsion to use the platform. Finally, by using basic user engagement features, we train classifier models to identify highly likely addicted users with $F_1 \geq 0.55$. The performance of the classifier models suggests predicting addictive users solely based on their usage is rather difficult.
Estimating the Optimal Number of Clusters in Categorical Data Clustering by Silhouette Coefficient
Venue: arXiv cs.LG 2025
Year: 2025

The problem of estimating the number of clusters (say k) is one of the major challenges for the partitional clustering. This paper proposes an algorithm named k-SCC to estimate the optimal k in categorical data clustering. For the clustering step, the algorithm uses the kernel density estimation approach to define cluster centers. In addition, it uses an information-theoretic based dissimilarity to measure the distance between centers and objects in each cluster. The silhouette analysis based approach is then used to evaluate the quality of different clustering obtained in the former step to choose the best k. Comparative experiments were conducted on both synthetic and real datasets to compare the performance of k-SCC with three other algorithms. Experimental results show that k-SCC outperforms the compared algorithms in determining the number of clusters for each dataset.
Commute Your Domains: Trajectory Optimality Criterion for Multi-Domain Learning
Venue: arXiv cs.LG 2025
Year: 2025

In multi-domain learning, a single model is trained on diverse data domains to leverage shared knowledge and improve generalization. The order in which the data from these domains is used for training can significantly affect the model's performance on each domain. However, this dependence is under-studied. In this paper, we investigate the influence of training order (or data mixing) in multi-domain learning using the concept of Lie bracket of gradient vector fields. By analyzing the infinitesimal effects of changing the training order, we identify regions in the parameter space where altering the order between two training domains can benefit the target loss. We validate the predictions of our theoretical framework on the influence of training order (or data mixing) both on a toy example and bilingual LLM pre-training.
PCAP-Backdoor: Backdoor Poisoning Generator for Network Traffic in CPS/IoT Environments
Venue: arXiv cs.LG 2025
Year: 2025

The rapid expansion of connected devices has made them prime targets for cyberattacks. To address these threats, deep learning-based, data-driven intrusion detection systems (IDS) have emerged as powerful tools for detecting and mitigating such attacks. These IDSs analyze network traffic to identify unusual patterns and anomalies that may indicate potential security breaches. However, prior research has shown that deep learning models are vulnerable to backdoor attacks, where attackers inject triggers into the model to manipulate its behavior and cause misclassifications of network traffic. In this paper, we explore the susceptibility of deep learning-based IDS systems to backdoor attacks in the context of network traffic analysis. We introduce \texttt{PCAP-Backdoor}, a novel technique that facilitates backdoor poisoning attacks on PCAP datasets. Our experiments on real-world Cyber-Physical Systems (CPS) and Internet of Things (IoT) network traffic datasets demonstrate that attackers can effectively backdoor a model by poisoning as little as 1\% or less of the entire training dataset. Moreover, we show that an attacker can introduce a trigger into benign traffic during model training yet cause the backdoored model to misclassify malicious traffic when the trigger is present. Finally, we highlight the difficulty of detecting this trigger-based backdoor, even when using existing backdoor defense techniques.
ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer
Venue: arXiv cs.CL 2025
Year: 2025

As is known, hybrid quadratic and subquadratic attention models in multi-head architectures have surpassed both Transformer and Linear RNN models , with these works primarily focusing on reducing KV complexity and improving efficiency. For further research on expressiveness, we introduce our series of models distilled from Qwen 2.5, based on pure native RWKV-7 attention, which aims to make RNN more expressive and demonstrates state tracking ability beyond transformers. We work with QRWK 32B based on RWKV-6 architecture, another approach that reduces the entire knowledge processing time to just 8 hours using 16 AMD MI300X GPUs while maintaining Qwen 2.5's performance. In fact, the distillation process can utilize any LLM, not just Qwen, and enables knowledge transfer from larger LLMs to smaller ones with more fewer tokens. We will explain the detailed process and share our insights on building more powerful foundation models. Please note that this is an ongoing work that will be updated continuously. The model checkpoints and source code are available at \href{https://github.com/yynil/RWKVInside}{https://github.com/yynil/RWKVInside}, \href{https://huggingface.co/RWKV-Red-Team/ARWKV-7B-Preview-0.1}{https://huggingface.co/RWKV-Red-Team/ARWKV-7B-Preview-0.1}.
Error Classification of Large Language Models on Math Word Problems: A Dynamically Adaptive Framework
Venue: arXiv cs.CL 2025
Year: 2025

Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains. Math Word Problems (MWPs) serve as a crucial benchmark for evaluating LLMs' reasoning abilities. While most research primarily focuses on improving accuracy, it often neglects understanding and addressing the underlying patterns of errors. Current error classification methods rely on static and predefined categories, which limit their ability to capture the full spectrum of error patterns in mathematical reasoning. To enable systematic error analysis, we collect error samples from 15 different LLMs of varying sizes across four distinct MWP datasets using multiple sampling strategies. Based on this extensive collection, we introduce MWPES-300K, a comprehensive dataset containing 304,865 error samples that cover diverse error patterns and reasoning paths. To reduce human bias and enable fine-grained analysis of error patterns, we propose a novel framework for automated dynamic error classification in mathematical reasoning. Experimental results demonstrate that dataset characteristics significantly shape error patterns, which evolve from basic to complex manifestations as model capabilities increase. With deeper insights into error patterns, we propose Error-Aware Prompting (EAP) that incorporates common error patterns as explicit guidance, leading to significant improvements in mathematical reasoning performance.
Advancing TDFN: Precise Fixation Point Generation Using Reconstruction Differences
Venue: arXiv cs.CV 2025
Year: 2025

Wang and Wang (2025) proposed the Task-Driven Fixation Network (TDFN) based on the fixation mechanism, which leverages low-resolution information along with high-resolution details near fixation points to accomplish specific visual tasks. The model employs reinforcement learning to generate fixation points. However, training reinforcement learning models is challenging, particularly when aiming to generate pixel-level accurate fixation points on high-resolution images. This paper introduces an improved fixation point generation method by leveraging the difference between the reconstructed image and the input image to train the fixation point generator. This approach directs fixation points to areas with significant differences between the reconstructed and input images. Experimental results demonstrate that this method achieves highly accurate fixation points, significantly enhances the network's classification accuracy, and reduces the average number of required fixations to achieve a predefined accuracy level.
Engage and Mobilize! Understanding Evolving Patterns of Social Media Usage in Emergency Management
Venue: arXiv cs.HC 2025
Year: 2025

The work of Emergency Management (EM) agencies requires timely collection of relevant data to inform decision-making for operations and public communication before, during, and after a disaster. However, the limited human resources available to deploy for field data collection is a persistent problem for EM agencies. Thus, many of these agencies have started leveraging social media as a supplemental data source and a new venue to engage with the public. While prior research has analyzed the potential benefits and attitudes of practitioners and the public when leveraging social media during disasters, a gap exists in the critical analysis of the actual practices and uses of social media among EM agencies, across both geographical regions and phases of the EM lifecycle - typically mitigation, preparedness, response, and recovery. In this paper, we conduct a mixed-method analysis to update and fill this gap on how EM practitioners in the U.S. and Europe use social media, building on a survey study of about 150 professionals and a follow-up interview study with 11 participants. The results indicate that using social media is no longer a non-traditional practice in operational and informational processes for the decision-making of EM agencies working at both the local level (e.g., county or town) and non-local level (e.g., state/province, federal/national) for emergency management. Especially, the practitioners affiliated with agencies working at the local level have a very high perceived value of social media for situational awareness (e.g., analyzing disaster extent and impact) and public communication (e.g., disseminating timely information and correcting errors in crisis coverage). We conclude with the policy, technological, and socio-technical needs to design future social media analytics systems to support the work of EM agencies in such communication including the applications of AI.
Stepback: Enhanced Disentanglement for Voice Conversion via Multi-Task Learning
Venue: arXiv cs.SD 2025
Year: 2025

Voice conversion (VC) modifies voice characteristics while preserving linguistic content. This paper presents the Stepback network, a novel model for converting speaker identity using non-parallel data. Unlike traditional VC methods that rely on parallel data, our approach leverages deep learning techniques to enhance disentanglement completion and linguistic content preservation. The Stepback network incorporates a dual flow of different domain data inputs and uses constraints with self-destructive amendments to optimize the content encoder. Extensive experiments show that our model significantly improves VC performance, reducing training costs while achieving high-quality voice conversion. The Stepback network's design offers a promising solution for advanced voice conversion tasks.
Deterministic Reservoir Computing for Chaotic Time Series Prediction
Venue: arXiv cs.LG 2025
Year: 2025

Reservoir Computing was shown in recent years to be useful as efficient to learn networks in the field of time series tasks. Their randomized initialization, a computational benefit, results in drawbacks in theoretical analysis of large random graphs, because of which deterministic variations are an still open field of research. Building upon Next-Gen Reservoir Computing and the Temporal Convolution Derived Reservoir Computing, we propose a deterministic alternative to the higher-dimensional mapping therein, TCRC-LM and TCRC-CM, utilizing the parametrized but deterministic Logistic mapping and Chebyshev maps. To further enhance the predictive capabilities in the task of time series forecasting, we propose the novel utilization of the Lobachevsky function as non-linear activation function. As a result, we observe a new, fully deterministic network being able to outperform TCRCs and classical Reservoir Computing in the form of the prominent Echo State Networks by up to $99.99\%$ for the non-chaotic time series and $87.13\%$ for the chaotic ones.
Understanding Attitudes and Trust of Generative AI Chatbots for Social Anxiety Support
Venue: arXiv cs.HC 2025
Year: 2025

Social anxiety (SA) has become increasingly prevalent. Traditional coping strategies often face accessibility challenges. Generative AI (GenAI), known for their knowledgeable and conversational capabilities, are emerging as alternative tools for mental well-being. With the increased integration of GenAI, it is important to examine individuals' attitudes and trust in GenAI chatbots' support for SA. Through a mixed-method approach that involved surveys (n = 159) and interviews (n = 17), we found that individuals with severe symptoms tended to trust and embrace GenAI chatbots more readily, valuing their non-judgmental support and perceived emotional comprehension. However, those with milder symptoms prioritized technical reliability. We identified factors influencing trust, such as GenAI chatbots' ability to generate empathetic responses and its context-sensitive limitations, which were particularly important among individuals with SA. We also discuss the design implications and use of GenAI chatbots in fostering cognitive and emotional trust, with practical and design considerations.
A Comprehensive Survey on Self-Interpretable Neural Networks
Venue: arXiv cs.LG 2025
Year: 2025

Neural networks have achieved remarkable success across various fields. However, the lack of interpretability limits their practical use, particularly in critical decision-making scenarios. Post-hoc interpretability, which provides explanations for pre-trained models, is often at risk of robustness and fidelity. This has inspired a rising interest in self-interpretable neural networks, which inherently reveal the prediction rationale through the model structures. Although there exist surveys on post-hoc interpretability, a comprehensive and systematic survey of self-interpretable neural networks is still missing. To address this gap, we first collect and review existing works on self-interpretable neural networks and provide a structured summary of their methodologies from five key perspectives: attribution-based, function-based, concept-based, prototype-based, and rule-based self-interpretation. We also present concrete, visualized examples of model explanations and discuss their applicability across diverse scenarios, including image, text, graph data, and deep reinforcement learning. Additionally, we summarize existing evaluation metrics for self-interpretability and identify open challenges in this field, offering insights for future research. To support ongoing developments, we present a publicly accessible resource to track advancements in this domain: https://github.com/yangji721/Awesome-Self-Interpretable-Neural-Network.
Rate Distortion Approach to Joint Communication and Sensing With Markov States: Open Loop Case
Venue: arXiv cs.IT 2025
Year: 2025

We investigate a joint communication and sensing (JCAS) framework in which a transmitter concurrently transmits information to a receiver and estimates a state of interest based on noisy observations. The state is assumed to evolve according to a known dynamical model. Past state estimates may then be used to inform current state estimates. We show that Bayesian filtering constitutes the optimal sensing strategy. We analyze JCAS performance under an open loop encoding strategy with results presented in terms of the tradeoff between asymptotic communication rate and expected per-block distortion of the state. We illustrate the general result by specializing the analysis to a beam-pointing model with mobile state tracking. Our results shed light on the relative performance of two beam control strategies, beam-switching and multi-beam.
A Machine Learning Approach to Automatic Fall Detection of Soldiers
Venue: arXiv cs.LG 2025
Year: 2025

Military personnel and security agents often face significant physical risks during conflict and engagement situations, particularly in urban operations. Ensuring the rapid and accurate communication of incidents involving injuries is crucial for the timely execution of rescue operations. This article presents research conducted under the scope of the Brazilian Navy's ``Soldier of the Future'' project, focusing on the development of a Casualty Detection System to identify injuries that could incapacitate a soldier and lead to severe blood loss. The study specifically addresses the detection of soldier falls, which may indicate critical injuries such as hypovolemic hemorrhagic shock. To generate the publicly available dataset, we used smartwatches and smartphones as wearable devices to collect inertial data from soldiers during various activities, including simulated falls. The data were used to train 1D Convolutional Neural Networks (CNN1D) with the objective of accurately classifying falls that could result from life-threatening injuries. We explored different sensor placements (on the wrists and near the center of mass) and various approaches to using inertial variables, including linear and angular accelerations. The neural network models were optimized using Bayesian techniques to enhance their performance. The best-performing model and its results, discussed in this article, contribute to the advancement of automated systems for monitoring soldier safety and improving response times in engagement scenarios.
Classifying Deepfakes Using Swin Transformers
Venue: arXiv cs.CV 2025
Year: 2025

The proliferation of deepfake technology poses significant challenges to the authenticity and trustworthiness of digital media, necessitating the development of robust detection methods. This study explores the application of Swin Transformers, a state-of-the-art architecture leveraging shifted windows for self-attention, in detecting and classifying deepfake images. Using the Real and Fake Face Detection dataset by Yonsei University's Computational Intelligence Photography Lab, we evaluate the Swin Transformer and hybrid models such as Swin-ResNet and Swin-KNN, focusing on their ability to identify subtle manipulation artifacts. Our results demonstrate that the Swin Transformer outperforms conventional CNN-based architectures, including VGG16, ResNet18, and AlexNet, achieving a test accuracy of 71.29%. Additionally, we present insights into hybrid model design, highlighting the complementary strengths of transformer and CNN-based approaches in deepfake detection. This study underscores the potential of transformer-based architectures for improving accuracy and generalizability in image-based manipulation detection, paving the way for more effective countermeasures against deepfake threats.