OpenRaiser/Intern-Atlas huggingface dataset:
Generative Sequential Notification Optimization via Multi-Objective Decision Transformers
Venue: arXiv cs.LG 2025
Year: 2025

Notifications are an important communication channel for delivering timely and relevant information. Optimizing their delivery involves addressing complex sequential decision-making challenges under constraints such as message utility and user fatigue. Offline reinforcement learning (RL) methods, such as Conservative Q-Learning (CQL), have been applied to this problem but face practical challenges at scale, including instability, sensitivity to distribution shifts, limited reproducibility, and difficulties with explainability in high-dimensional recommendation settings. We present a Decision Transformer (DT) based framework that reframes policy learning as return-conditioned supervised learning, improving robustness, scalability, and modeling flexibility. Our contributions include a real-world comparison with CQL, a multi-reward design suitable for non-episodic tasks, a quantile regression approach to return-to-go conditioning, and a production-ready system with circular buffer-based sequence processing for near-real-time inference. Extensive offline and online experiments in a deployed notification system show that our approach improves notification utility and overall session activity while minimizing user fatigue. Compared to a multi-objective CQL-based agent, the DT-based approach achieved a +0.72% increase in sessions for notification decision-making at LinkedIn by making notification recommendation more relevant.
Can we cite Wikipedia? What if Wikipedia was more reliable than its detractors ?
Venue: arXiv cs.CY 2025
Year: 2025

Wikipedia, a widely successful encyclopedia recognized in academic circles and used by both students and professors alike, has led educators to question whether it can be cited as an information source, given its widespread use for this very purpose. The dilemma quickly emerged: if Wikipedia has become the go-to information source for so many, why can't it be cited? If consulting and using Wikipedia as a source of information is permitted, why does it become controversial the moment one attempts to cite it? This manuscript examines the systematic rejection of Wikipedia in academic settings, not to argue for its legitimacy as a source, but to demonstrate that its reliability is often underestimated while traditional academic sources enjoy disproportionate credibility, despite their increasingly apparent shortcomings. The central thesis posits that Wikipedia's rejection stems from an outdated epistemological bias that overlooks both the project's verification mechanisms and the structural crises affecting scientific publishing.
Classification of Vision-Based Tactile Sensors: A Review
Venue: arXiv cs.RO 2025
Year: 2025

Vision-based tactile sensors (VBTS) have gained widespread application in robotic hands, grippers and prosthetics due to their high spatial resolution, low manufacturing costs, and ease of customization. While VBTSs have common design features, such as a camera module, they can differ in a rich diversity of sensing principles, material compositions, multimodal approaches, and data interpretation methods. Here, we propose a novel classification of VBTS that categorizes the technology into two primary sensing principles based on the underlying transduction of contact into a tactile image: the Marker-Based Transduction Principle and the Intensity-Based Transduction Principle. Marker-Based Transduction interprets tactile information by detecting marker displacement and changes in marker density. In contrast, Intensity-Based Transduction maps external disturbances with variations in pixel values. Depending on the design of the contact module, Marker-Based Transduction can be further divided into two subtypes: Simple Marker-Based (SMB) and Morphological Marker-Based (MMB) mechanisms. Similarly, the Intensity-Based Transduction Principle encompasses the Reflective Layer-based (RLB) and Transparent Layer-Based (TLB) mechanisms. This paper provides a comparative study of the hardware characteristics of these four types of sensors including various combination types, and discusses the commonly used methods for interpreting tactile information. This~comparison reveals some current challenges faced by VBTS technology and directions for future research.
Federated learning over physical channels: adaptive algorithms with near-optimal guarantees
Venue: arXiv cs.LG 2025
Year: 2025

In federated learning, communication cost can be significantly reduced by transmitting the information over the air through physical channels. In this paper, we propose a new class of adaptive federated stochastic gradient descent (SGD) algorithms that can be implemented over physical channels, taking into account both channel noise and hardware constraints. We establish theoretical guarantees for the proposed algorithms, demonstrating convergence rates that are adaptive to the stochastic gradient noise level. We also demonstrate the practical effectiveness of our algorithms through simulation studies with deep learning models.
A Keyframe-Based Approach for Auditing Bias in YouTube Shorts Recommendations
Venue: arXiv cs.SI 2025
Year: 2025

YouTube Shorts and other short-form video platforms now influence how billions engage with content, yet their recommendation systems remain largely opaque. Small shifts in promoted content can significantly impact user exposure, especially for politically sensitive topics. In this work, we propose a keyframe-based method to audit bias and drift in short-form video recommendations. Rather than analyzing full videos or relying on metadata, we extract perceptually salient keyframes, generate captions, and embed both into a shared content space. Using visual mapping across recommendation chains, we observe consistent shifts and clustering patterns that indicate topic drift and potential filtering. Comparing politically sensitive topics with general YouTube categories, we find notable differences in recommendation behavior. Our findings show that keyframes provide an efficient and interpretable lens for understanding bias in short-form video algorithms.
Energy-Efficient Split Learning for Resource-Constrained Environments: A Smart Farming Solution
Venue: arXiv cs.DC 2025
Year: 2025

Smart farming systems encounter significant challenges, including limited resources, the need for data privacy, and poor connectivity in rural areas. To address these issues, we present eEnergy-Split, an energy-efficient framework that utilizes split learning (SL) to enable collaborative model training without direct data sharing or heavy computation on edge devices. By distributing the model between edge devices and a central server, eEnergy-Split reduces on-device energy usage by up to 86 percent compared to federated learning (FL) while safeguarding data privacy. Moreover, SL improves classification accuracy by up to 6.2 percent over FL on ResNet-18 and by more modest amounts on GoogleNet and MobileNetV2. We propose an optimal edge deployment algorithm and a UAV trajectory planning strategy that solves the Traveling Salesman Problem (TSP) exactly to minimize flight cost and extend and maximize communication rounds. Comprehensive evaluations on agricultural pest datasets reveal that eEnergy-Split lowers UAV energy consumption compared to baseline methods and boosts overall accuracy by up to 17 percent. Notably, the energy efficiency of SL is shown to be model-dependent-yielding substantial savings in lightweight models like MobileNet, while communication and memory overheads may reduce efficiency gains in deeper networks. These results highlight the potential of combining SL with energy-aware design to deliver a scalable, privacy-preserving solution for resource-constrained smart farming environments.
Use of Physicochemical Modification Methods for Producing Traditional and Nanomodified Polymeric Composites with Improved Operational Properties
Venue: arXiv cs.CE 2025
Year: 2025

Various aspects of the methods of physical and physicochemical modification of components of filled thermoplastic composite materials are analyzed, aimed at improving the surface properties of the fillers and the technological properties of the polymer matrix during their interaction. It is noted that the improvement of the interfacial interaction of the components of polymer reactoplastic composites, including adhesive strength, is a key factor for improving the reliability of the cured filled composite. As a promising area of research, a modification of the surface of the reinforcing fibrous filler and the technological characteristics of the liquid polymer binder, aimed at increasing their contact properties in the composite, was chosen. The effectiveness of the physical method of modifying the components of composites in the form of low-frequency ultrasonic processing is described. The peculiarities of cluster formation and physicochemical modification of epoxy polymers filled with dispersed fillers are analyzed. Attention is focused on the effectiveness of ultrasonic processing in the cavitation mode for deagglomeration and uniform distribution of nanoparticles in a liquid medium during the creation of nanocomposites. Experimentally confirmed is the improvement of the technological properties of liquid epoxy polymers, modified by ultrasound, used for the impregnation of oriented fibrous fillers, as well as the improvement of the physicomechanical properties of the sonicated epoxy matrices. Some issues of biological modifications of components of polymers for functional application are briefly reviewed.
On the Optimization of Methods for Establishing Well-Connected Communities
Venue: arXiv cs.SI 2025
Year: 2025

Community detection plays a central role in uncovering meso scale structures in networks. However, existing methods often suffer from disconnected or weakly connected clusters, undermining interpretability and robustness. Well-Connected Clusters (WCC) and Connectivity Modifier (CM) algorithms are post-processing techniques that improve the accuracy of many clustering methods. However, they are computationally prohibitive on massive graphs. In this work, we present optimized parallel implementations of WCC and CM using the HPE Chapel programming language. First, we design fast and efficient parallel algorithms that leverage Chapel's parallel constructs to achieve substantial performance improvements and scalability on modern multicore architectures. Second, we integrate this software into Arkouda/Arachne, an open-source, high-performance framework for large-scale graph analytics. Our implementations uniquely enable well-connected community detection on massive graphs with more than 2 billion edges, providing a practical solution for connectivity-preserving clustering at web scale. For example, our implementations of WCC and CM enable community detection of the over 2-billion edge Open-Alex dataset in minutes using 128 cores, a result infeasible to compute previously.
Exploring the interplay between Planetary Boundaries and Sustainable Development Goals using Large Language Models
Venue: arXiv cs.CY 2025
Year: 2025

By analyzing 40,037 climate articles using Large Language Models (LLMs), we identified interactions between Planetary Boundaries (PBs) and Sustainable Development Goals (SDGs). An automated reasoner distinguished true trade-offs (SDG progress harming PBs) and synergies (mutual reinforcement) from double positives and negatives (shared drivers). Results show 21.1% true trade-offs, 28.3% synergies, and 19.5% neutral interactions, with the remainder being double positive or negative. Key findings include conflicts between land-use goals (SDG2/SDG6) and land system boundaries (PB6), together with the underrepresentation of social SDGs in the climate literature. Our study highlights the need for integrated policies that align development goals with planetary limits to reduce systemic conflicts. We propose three steps: (1) integrated socio-ecological metrics, (2) governance ensuring that SDG progress respects Earth system limits, and (3) equity measures protecting marginalized groups from boundary compliance costs.
Preference Robustness for DPO with Applications to Public Health
Venue: arXiv cs.LG 2025
Year: 2025

We study an LLM fine-tuning task for designing reward functions for sequential resource allocation problems in public health, guided by human preferences expressed in natural language. This setting presents a challenging testbed for alignment due to complex and ambiguous objectives and limited data availability. We propose DPO-PRO, a robust fine-tuning algorithm based on Direct Preference Optimization (DPO), which accounts for uncertainty in the preference distribution using a lightweight Distributionally Robust Optimization (DRO) formulation. Unlike prior DRO-based DPO methods, DPO-PRO is significantly less conservative. We evaluate DPO-PRO on a real-world maternal mobile health program operated by the non-profit organization ARMMAN, as well as on standard alignment benchmarks. Experimental results demonstrate that our method consistently improves robustness to noisy preference signals compared to existing DPO variants. Moreover, DPO-PRO achieves comparable performance to prior self-reflection-based baseline for reward function design, while requiring significantly lower inference-time cost.
Deep Research is the New Analytics System: Towards Building the Runtime for AI-Driven Analytics
Venue: arXiv cs.AI 2025
Year: 2025

With advances in large language models (LLMs), researchers are creating new systems that can perform AI-driven analytics over large unstructured datasets. Recent work has explored executing such analytics queries using semantic operators -- a declarative set of AI-powered data transformations with natural language specifications. However, even when optimized, these operators can be expensive to execute on millions of records and their iterator execution semantics make them ill-suited for interactive data analytics tasks. In another line of work, Deep Research systems have demonstrated an ability to answer natural language question(s) over large datasets. These systems use one or more LLM agent(s) to plan their execution, process the dataset(s), and iteratively refine their answer. However, these systems do not explicitly optimize their query plans which can lead to poor plan execution. In order for AI-driven analytics to excel, we need a runtime which combines the optimized execution of semantic operators with the flexibility and more dynamic execution of Deep Research systems. As a first step towards this vision, we build a prototype which enables Deep Research agents to write and execute optimized semantic operator programs. We evaluate our prototype and demonstrate that it can outperform a handcrafted semantic operator program and open Deep Research systems on two basic queries. Compared to a standard open Deep Research agent, our prototype achieves up to 1.95x better F1-score. Furthermore, even if we give the agent access to semantic operators as tools, our prototype still achieves cost and runtime savings of up to 76.8% and 72.7% thanks to its optimized execution.
Analysis of Speaker Verification Performance Trade-offs with Neural Audio Codec Transmission
Venue: arXiv cs.SD 2025
Year: 2025

Neural audio codecs (NACs) have made significant advancements in recent years and are rapidly being adopted in many audio processing pipelines. However, they can introduce audio distortions which degrade speaker verification (SV) performance. This study investigates the impact of both traditional and neural audio codecs at varying bitrates on three state of-the-art SV models evaluated on the VoxCeleb1 dataset. Our findings reveal a consistent degradation in SV performance across all models and codecs as bitrates decrease. Notably, NACs do not fundamentally break SV performance when compared to traditional codecs. They outperform Opus by 6-8% at low-bitrates (< 12 kbps) and remain marginally behind at higher bitrates ($\approx$ 24 kbps), with an EER increase of only 0.4-0.7%. The disparity at higher bitrates is likely due to the primary optimization of NACs for perceptual quality, which can inadvertently discard critical speaker-discriminative features, unlike Opus which was designed to preserve vocal characteristics. Our investigation suggests that NACs are a feasible alternative to traditional codecs, especially under bandwidth limitations. To bridge the gap at higher bitrates, future work should focus on developing speaker-aware NACs or retraining and adapting SV models.
Computational Social Science and Critical Studies of Education and Technology: An Improbable Combination?
Venue: arXiv cs.CY 2025
Year: 2025

As belief around the potential of computational social science grows, fuelled by recent advances in machine learning, data scientists are ostensibly becoming the new experts in education. Scholars engaged in critical studies of education and technology have sought to interrogate the growing datafication of education yet tend not to use computational methods as part of this response. In this paper, we discuss the feasibility and desirability of the use of computational approaches as part of a critical research agenda. Presenting and reflecting upon two examples of projects that use computational methods in education to explore questions of equity and justice, we suggest that such approaches might help expand the capacity of critical researchers to highlight existing inequalities, make visible possible approaches for beginning to address such inequalities, and engage marginalised communities in designing and ultimately deploying these possibilities. Drawing upon work within the fields of Critical Data Studies and Science and Technology Studies, we further reflect on the two cases to discuss the possibilities and challenges of reimagining computational methods for critical research in education and technology, focusing on six areas of consideration: criticality, philosophy, inclusivity, context, classification, and responsibility.
HF-RAG: Hierarchical Fusion-based RAG with Multiple Sources and Rankers
Venue: arXiv cs.IR 2025
Year: 2025

Leveraging both labeled (input-output associations) and unlabeled data (wider contextual grounding) may provide complementary benefits in retrieval augmented generation (RAG). However, effectively combining evidence from these heterogeneous sources is challenging as the respective similarity scores are not inter-comparable. Additionally, aggregating beliefs from the outputs of multiple rankers can improve the effectiveness of RAG. Our proposed method first aggregates the top-documents from a number of IR models using a standard rank fusion technique for each source (labeled and unlabeled). Next, we standardize the retrieval score distributions within each source by applying z-score transformation before merging the top-retrieved documents from the two sources. We evaluate our approach on the fact verification task, demonstrating that it consistently improves over the best-performing individual ranker or source and also shows better out-of-domain generalization.
Generalizable Skill Learning for Construction Robots with Crowdsourced Natural Language Instructions, Composable Skills Standardization, and Large Language Model
Venue: arXiv cs.RO 2025
Year: 2025

The quasi-repetitive nature of construction work and the resulting lack of generalizability in programming construction robots presents persistent challenges to the broad adoption of robots in the construction industry. Robots cannot achieve generalist capabilities as skills learnt from one domain cannot readily transfer to another work domain or be directly used to perform a different set of tasks. Human workers have to arduously reprogram their scene-understanding, path-planning, and manipulation components to enable the robots to perform alternate work tasks. The methods presented in this paper resolve a significant proportion of such reprogramming workload by proposing a generalizable learning architecture that directly teaches robots versatile task-performance skills through crowdsourced online natural language instructions. A Large Language Model (LLM), a standardized and modularized hierarchical modeling approach, and Building Information Modeling-Robot sematic data pipeline are developed to address the multi-task skill transfer problem. The proposed skill standardization scheme and LLM-based hierarchical skill learning framework were tested with a long-horizon drywall installation experiment using a full-scale industrial robotic manipulator. The resulting robot task learning scheme achieves multi-task reprogramming with minimal effort and high quality.
UrbanTwin: Building High-Fidelity Digital Twins for Sim2Real LiDAR Perception and Evaluation
Venue: arXiv cs.CV 2025
Year: 2025

LiDAR-based perception in intelligent transportation systems (ITS) relies on deep neural networks trained with large-scale labeled datasets. However, creating such datasets is expensive, time-consuming, and labor-intensive, limiting the scalability of perception systems. Sim2Real learning offers a scalable alternative, but its success depends on the simulation's fidelity to real-world environments, dynamics, and sensors. This tutorial introduces a reproducible workflow for building high-fidelity digital twins (HiFi DTs) to generate realistic synthetic datasets. We outline practical steps for modeling static geometry, road infrastructure, and dynamic traffic using open-source resources such as satellite imagery, OpenStreetMap, and sensor specifications. The resulting environments support scalable and cost-effective data generation for robust Sim2Real learning. Using this workflow, we have released three synthetic LiDAR datasets, namely UT-LUMPI, UT-V2X-Real, and UT-TUMTraf-I, which closely replicate real locations and outperform real-data-trained baselines in perception tasks. This guide enables broader adoption of HiFi DTs in ITS research and deployment.
High-Fidelity Digital Twins for Bridging the Sim2Real Gap in LiDAR-Based ITS Perception
Venue: arXiv cs.CV 2025
Year: 2025

Sim2Real domain transfer offers a cost-effective and scalable approach for developing LiDAR-based perception (e.g., object detection, tracking, segmentation) in Intelligent Transportation Systems (ITS). However, perception models trained in simulation often under perform on real-world data due to distributional shifts. To address this Sim2Real gap, this paper proposes a high-fidelity digital twin (HiFi DT) framework that incorporates real-world background geometry, lane-level road topology, and sensor-specific specifications and placement. We formalize the domain adaptation challenge underlying Sim2Real learning and present a systematic method for constructing simulation environments that yield in-domain synthetic data. An off-the-shelf 3D object detector is trained on HiFi DT-generated synthetic data and evaluated on real data. Our experiments show that the DT-trained model outperforms the equivalent model trained on real data by 4.8%. To understand this gain, we quantify distributional alignment between synthetic and real data using multiple metrics, including Chamfer Distance (CD), Maximum Mean Discrepancy (MMD), Earth Mover's Distance (EMD), and Fr'echet Distance (FD), at both raw-input and latent-feature levels. Results demonstrate that HiFi DTs substantially reduce domain shift and improve generalization across diverse evaluation scenarios. These findings underscore the significant role of digital twins in enabling reliable, simulation-based LiDAR perception for real-world ITS applications.