OpenRaiser/Intern-Atlas huggingface dataset:
Effectively Leveraging CLIP for Generating Situational Summaries of Images and Videos
Venue: arXiv cs.CV 2025
Year: 2025

Situation recognition refers to the ability of an agent to identify and understand various situations or contexts based on available information and sensory inputs. It involves the cognitive process of interpreting data from the environment to determine what is happening, what factors are involved, and what actions caused those situations. This interpretation of situations is formulated as a semantic role labeling problem in computer vision-based situation recognition. Situations depicted in images and videos hold pivotal information, essential for various applications like image and video captioning, multimedia retrieval, autonomous systems and event monitoring. However, existing methods often struggle with ambiguity and lack of context in generating meaningful and accurate predictions. Leveraging multimodal models such as CLIP, we propose ClipSitu, which sidesteps the need for full fine-tuning and achieves state-of-the-art results in situation recognition and localization tasks. ClipSitu harnesses CLIP-based image, verb, and role embeddings to predict nouns fulfilling all the roles associated with a verb, providing a comprehensive understanding of depicted scenarios. Through a cross-attention Transformer, ClipSitu XTF enhances the connection between semantic role queries and visual token representations, leading to superior performance in situation recognition. We also propose a verb-wise role prediction model with near-perfect accuracy to create an end-to-end framework for producing situational summaries for out-of-domain images. We show that situational summaries empower our ClipSitu models to produce structured descriptions with reduced ambiguity compared to generic captions. Finally, we extend ClipSitu to video situation recognition to showcase its versatility and produce comparable performance to state-of-the-art methods.
Who Should Run Advanced AI Evaluations -- AISIs?
Venue: arXiv cs.CY 2025
Year: 2025

Artificial Intelligence (AI) Safety Institutes and governments worldwide are deciding whether they evaluate advanced AI themselves, support a private evaluation ecosystem or do both. Evaluation regimes have been established in a wide range of industry contexts to monitor and evaluate firms' compliance with regulation. Evaluation is a necessary governance tool to understand and manage the risks of a technology. This paper draws from nine such regimes to inform (i) who should evaluate which parts of advanced AI; and (ii) how much capacity public bodies may need to evaluate advanced AI effectively. First, the effective responsibility distribution between public and private evaluators depends heavily on specific industry and evaluation conditions. On the basis of advanced AI's risk profile, the sensitivity of information involved in the evaluation process, and the high costs of verifying safety and benefit claims of AI Labs, we recommend that public bodies become directly involved in safety critical, especially gray- and white-box, AI model evaluations. Governance and security audits, which are well-established in other industry contexts, as well as black-box model evaluations, may be more efficiently provided by a private market of evaluators and auditors under public oversight. Secondly, to effectively fulfil their role in advanced AI audits, public bodies need extensive access to models and facilities. AISI's capacity should scale with the industry's risk level, size and market concentration, potentially requiring 100s of employees for evaluations in large jurisdictions like the EU or US, like in nuclear safety and life sciences.
A Comprehensive Survey on Retrieval Methods in Recommender Systems
Venue: arXiv cs.IR 2025
Year: 2025

In an era dominated by information overload, effective recommender systems are essential for managing the deluge of data across digital platforms. Multi-stage cascade ranking systems are widely used in the industry, with retrieval and ranking being two typical stages. Retrieval methods sift through vast candidates to filter out irrelevant items, while ranking methods prioritize these candidates to present the most relevant items to users. Unlike studies focusing on the ranking stage, this survey explores the critical yet often overlooked retrieval stage of recommender systems. To achieve precise and efficient personalized retrieval, we summarize existing work in three key areas: improving similarity computation between user and item, enhancing indexing mechanisms for efficient retrieval, and optimizing training methods of retrieval. We also provide a comprehensive set of benchmarking experiments on three public datasets. Furthermore, we highlight current industrial applications through a case study on retrieval practices at a specific company, covering the entire retrieval process and online serving, along with practical implications and challenges. By detailing the retrieval stage, which is fundamental for effective recommendation, this survey aims to bridge the existing knowledge gap and serve as a cornerstone for researchers interested in optimizing this critical component of cascade recommender systems.
Socio-cognitive Networks between Researchers: Investigating Scientific Dualities with the Group-Oriented Relational Hyperevent Model
Venue: arXiv cs.SI 2025
Year: 2025

Understanding why researchers cite certain works remains a key question in the study of scientific networks. Prior research has identified factors such as relevance, group cohesion, and source crediting. However, the interplay between cognitive and social dimensions in citation behavior - often conceptualized as a socio-cognitive network - is frequently overlooked, particularly regarding the intermediary steps that lead to a citation. Since a citation first requires a work to be published by a set of authors, we examine how the structure of coauthorship networks influences citation patterns. To investigate this relationship, we analyze the citation and collaboration behavior of Chilean astronomers from 2013 to 2015 using the Group-Oriented Relational Hyperevent Model, which allows us to study coauthorship and citation networks in a joint framework. Our findings suggest that when selecting which works to cite, authors favor recent research and maintain cognitive continuity across cited works. At the same time, we observe that coherent groups - closely connected coauthors - tend to be co-cited more frequently in subsequent publications, reinforcing the interdependence of collaboration and citation networks.
Optimal Sampling under Cost for Remote Estimation of the Wiener Process over a Channel with Delay
Venue: arXiv cs.NI 2025
Year: 2025

We address the optimal sampling of a Wiener process under sampling and transmission costs, with the samples being forwarded to a remote estimator over a channel with IID delay. The goal of the estimator is to reconstruct the real-time signal by minimizing a long-term average cost that includes both the mean squared estimation error (MSE) and the costs associated with sampling and transmission from causally received samples. Rather than pursuing the conventional MMSE estimate, our objective is to derive a policy that optimally balances estimation accuracy and resource expenditure, yielding an MSE-optimal solution under explicit cost constraints. We look for optimal online strategies for both sampling and transmission. By employing Lagrange relaxation and iterative backward induction, we derive an optimal policy that balances the trade-offs between estimation accuracy and costs. We validate our approach through comprehensive simulations, evaluating various scenarios including balanced costs, high sampling costs, high transmission costs, and different transmission delay statistics. Our results demonstrate the effectiveness and robustness of the proposed joint sampling and transmission policy in maintaining lower MSE compared to conventional periodic sampling methods. The differences are particularly striking under high delay variability. We also analyze the convergence behavior of the cost function. We believe our formulation and results provide insights into the design and implementation of efficient remote estimation systems in stochastic networks.
VITAL: Interactive Few-Shot Imitation Learning via Visual Human-in-the-Loop Corrections
Venue: arXiv cs.RO 2025
Year: 2025

Imitation Learning (IL) has emerged as a powerful approach in robotics, allowing robots to acquire new skills by mimicking human actions. Despite its potential, the data collection process for IL remains a significant challenge due to the logistical difficulties and high costs associated with obtaining high-quality demonstrations. To address these issues, we propose a large-scale data generation from a handful of demonstrations through data augmentation in simulation. Our approach leverages affordable hardware and visual processing techniques to collect demonstrations, which are then augmented to create extensive training datasets for imitation learning. By utilizing both real and simulated environments, along with human-in-the-loop corrections, we enhance the generalizability and robustness of the learned policies. We evaluated our method through several rounds of experiments in both simulated and real-robot settings, focusing on tasks of varying complexity, including bottle collecting, stacking objects, and hammering. Our experimental results validate the effectiveness of our approach in learning robust robot policies from simulated data, significantly improved by human-in-the-loop corrections and real-world data integration. Additionally, we demonstrate the framework's capability to generalize to new tasks, such as setting a drink tray, showcasing its adaptability and potential for handling a wide range of real-world manipulation tasks. A video of the experiments can be found at: https://youtu.be/YeVAMRqRe64?si=R179xDlEGc7nPu8i
DDU-Net: A Domain Decomposition-Based CNN for High-Resolution Image Segmentation on Multiple GPUs
Venue: arXiv cs.CV 2025
Year: 2025

The segmentation of ultra-high resolution images poses challenges such as loss of spatial information or computational inefficiency. In this work, a novel approach that combines encoder-decoder architectures with domain decomposition strategies to address these challenges is proposed. Specifically, a domain decomposition-based U-Net (DDU-Net) architecture is introduced, which partitions input images into non-overlapping patches that can be processed independently on separate devices. A communication network is added to facilitate inter-patch information exchange to enhance the understanding of spatial context. Experimental validation is performed on a synthetic dataset that is designed to measure the effectiveness of the communication network. Then, the performance is tested on the DeepGlobe land cover classification dataset as a real-world benchmark data set. The results demonstrate that the approach, which includes inter-patch communication for images divided into $16\times16$ non-overlapping subimages, achieves a $2-3\,\%$ higher intersection over union (IoU) score compared to the same network without inter-patch communication. The performance of the network which includes communication is equivalent to that of a baseline U-Net trained on the full image, showing that our model provides an effective solution for segmenting ultra-high-resolution images while preserving spatial context. The code is available at https://github.com/corne00/DDU-Net.
DD-rPPGNet: De-interfering and Descriptive Feature Learning for Unsupervised rPPG Estimation
Venue: arXiv cs.CV 2025
Year: 2025

Remote Photoplethysmography (rPPG) aims to measure physiological signals and Heart Rate (HR) from facial videos. Recent unsupervised rPPG estimation methods have shown promising potential in estimating rPPG signals from facial regions without relying on ground truth rPPG signals. However, these methods seem oblivious to interference existing in rPPG signals and still result in unsatisfactory performance. In this paper, we propose a novel De-interfered and Descriptive rPPG Estimation Network (DD-rPPGNet) to eliminate the interference within rPPG features for learning genuine rPPG signals. First, we investigate the characteristics of local spatial-temporal similarities of interference and design a novel unsupervised model to estimate the interference. Next, we propose an unsupervised de-interfered method to learn genuine rPPG signals with two stages. In the first stage, we estimate the initial rPPG signals by contrastive learning from both the training data and their augmented counterparts. In the second stage, we use the estimated interference features to derive de-interfered rPPG features and encourage the rPPG signals to be distinct from the interference. In addition, we propose an effective descriptive rPPG feature learning by developing a strong 3D Learnable Descriptive Convolution (3DLDC) to capture the subtle chrominance changes for enhancing rPPG estimation. Extensive experiments conducted on five rPPG benchmark datasets demonstrate that the proposed DD-rPPGNet outperforms previous unsupervised rPPG estimation methods and achieves competitive performances with state-of-the-art supervised rPPG methods. The code is available at: https://github.com/Pei-KaiHuang/TIFS2025-DD-rPPGNet
Learning Effective Representations for Retrieval Using Self-Distillation with Adaptive Relevance Margins
Venue: arXiv cs.IR 2025
Year: 2025

Representation-based retrieval models, so-called bi-encoders, estimate the relevance of a document to a query by calculating the similarity of their respective embeddings. Current state-of-the-art bi-encoders are trained using an expensive training regime involving knowledge distillation from a teacher model and batch-sampling. Instead of relying on a teacher model, we contribute a novel parameter-free loss function for self-supervision that exploits the pre-trained language modeling capabilities of the encoder model as a training signal, eliminating the need for batch sampling by performing implicit hard negative mining. We investigate the capabilities of our proposed approach through extensive experiments, demonstrating that self-distillation can match the effectiveness of teacher distillation using only 13.5% of the data, while offering a speedup in training time between 3x and 15x compared to parametrized losses. All code and data is made openly available.
Accelerating Transfer Function Update for Distance Map based Volume Rendering
Venue: arXiv cs.GR 2025
Year: 2025

Direct volume rendering using ray-casting is widely used in practice. By using GPUs and applying acceleration techniques as empty space skipping, high frame rates are possible on modern hardware. This enables performance-critical use-cases such as virtual reality volume rendering. The currently fastest known technique uses volumetric distance maps to skip empty sections of the volume during ray-casting but requires the distance map to be updated per transfer function change. In this paper, we demonstrate a technique for subdividing the volume intensity range into partitions and deriving what we call partitioned distance maps. These can be used to accelerate the distance map computation for a newly changed transfer function by a factor up to 30. This allows the currently fastest known empty space skipping approach to be used while maintaining high frame rates even when the transfer function is changed frequently.
Multiway Alignment of Political Attitudes
Venue: arXiv cs.SI 2025
Year: 2025

The related concepts of partisan belief systems, issue alignment, and partisan sorting are central to our understanding of politics. These phenomena have been studied using measures of alignment between pairs of topics, or how much individuals' attitudes toward a topic reveal about their attitudes toward another topic. We introduce a higher-order measure that extends the assessment of alignment beyond pairs of topics by quantifying the amount of information individuals' opinions on one topic reveal about a set of topics simultaneously. Applying this approach to legislative voting behavior shows that parliamentary systems typically exhibit similar multiway alignment characteristics, but can change in response to shifting intergroup dynamics. In American National Election Studies surveys, our approach reveals a growing significance of party identification together with a consistent rise in multiway alignment over time.
Review of Explainable Graph-Based Recommender Systems
Venue: arXiv cs.IR 2025
Year: 2025

Explainability of recommender systems has become essential to ensure users' trust and satisfaction. Various types of explainable recommender systems have been proposed including explainable graph-based recommender systems. This review paper discusses state-of-the-art approaches of these systems and categorizes them based on three aspects: learning methods, explaining methods, and explanation types. It also explores the commonly used datasets, explainability evaluation methods, and future directions of this research area. Compared with the existing review papers, this paper focuses on explainability based on graphs and covers the topics required for developing novel explainable graph-based recommender systems.
Multiple Greedy Quasi-Newton Methods for Saddle Point Problems
Venue: arXiv cs.AI 2025
Year: 2025

This paper introduces the Multiple Greedy Quasi-Newton (MGSR1-SP) method, a novel approach to solving strongly-convex-strongly-concave (SCSC) saddle point problems. Our method enhances the approximation of the squared indefinite Hessian matrix inherent in these problems, significantly improving both stability and efficiency through iterative greedy updates. We provide a thorough theoretical analysis of MGSR1-SP, demonstrating its linear-quadratic convergence rate. Numerical experiments conducted on AUC maximization and adversarial debiasing problems, compared with state-of-the-art algorithms, underscore our method's enhanced convergence rate. These results affirm the potential of MGSR1-SP to improve performance across a broad spectrum of machine learning applications where efficient and accurate Hessian approximations are crucial.
Infrequent Resolving Algorithm for Online Linear Programming
Venue: arXiv cs.DS 2025
Year: 2025

Online linear programming (OLP) has gained significant attention from both researchers and practitioners due to its extensive applications, such as online auction, network revenue management, order fulfillment and advertising. Existing OLP algorithms fall into two categories: LP-based algorithms and LP-free algorithms. The former one typically guarantees better performance but requires solving a large number of LPs, which could be computationally expensive. In contrast, LP-free algorithm only requires first-order computations but induces a worse performance. In this work, we bridge the gap between these two extremes by proposing a well-performing algorithm, that solves LPs at a few selected time points and conducts first-order computations at other time points. Specifically, for the case where the inputs are drawn from an unknown finite-support distribution, the proposed algorithm achieves a constant regret (even for the hard "degenerate" case) while solving LPs only O(log log T) times over the time horizon T. Moreover, when we are allowed to solve LPs only M times, we design the corresponding schedule such that the proposed algorithm can guarantee a nearly O(T^((1/2)^(M-1)) regret. Our work highlights the value of resolving both at the beginning and the end of the selling horizon, and provides a novel framework to prove the performance guarantee of the proposed policy under different infrequent resolving schedules. Numerical experiments are conducted to demonstrate the efficiency of the proposed algorithms.
Contact-conditioned learning of multi-gait locomotion policies
Venue: arXiv cs.RO 2025
Year: 2025

In this paper, we examine the effects of goal representation on the performance and generalization in multi-gait policy learning settings for legged robots. To study this problem in isolation, we cast the policy learning problem as imitating model predictive controllers that can generate multiple gaits. We hypothesize that conditioning a learned policy on future contact switches is a suitable goal representation for learning a single policy that can generate a variety of gaits. Our rationale is that policies conditioned on contact information can leverage the shared structure between different gaits. Our extensive simulation results demonstrate the validity of our hypothesis for learning multiple gaits on a bipedal and a quadrupedal robot. Most interestingly, our results show that contact-conditioned policies generalize much better than other common goal representations in the literature, when the robot is tested outside the distribution of the training data.
Design and Implementation of ARA Wireless Living Lab for Rural Broadband and Applications
Venue: arXiv cs.NI 2025
Year: 2025

Addressing the broadband gap between rural and urban regions requires rural-focused wireless research and innovation. In the meantime, rural regions provide rich, diverse use cases of advanced wireless, and they offer unique real-world settings for piloting applications that advance the frontiers of wireless systems (e.g., teleoperation of ground and aerial vehicles). To fill the broadband gap and to leverage the unique opportunities that rural regions provide for piloting advanced wireless applications, we design and implement the ARA wireless living lab for research and innovation in rural wireless systems and their applications in precision agriculture, community services, and so on. ARA focuses on the unique community, application, and economic context of rural regions, and it features the first-of-its-kind, real-world deployment of long-distance, high-capacity terrestrial wireless x-haul and access platforms as well as low-earth-orbit (LEO) satellite communications platforms across a rural area of diameter over 30 km. With both software-defined radios and programmable COTS systems, and through effective orchestration of these wireless resources with fiber as well as compute resources embedded end-to-end across user equipment (UE), base stations (BS), edge, and cloud, including support for Bring Your Own Device (BYOD), ARA offers programmability, performance, robustness, and heterogeneity at the same time, thus enabling rural-focused co-evolution of wireless and applications while helping advance the frontiers of wireless systems in domains such as Open RAN, NextG, and agriculture applications.
Exploiting the Semantic Knowledge of Pre-trained Text-Encoders for Continual Learning
Venue: arXiv cs.CV 2025
Year: 2025

Deep neural networks (DNNs) excel on fixed datasets but struggle with incremental and shifting data in real-world scenarios. Continual learning addresses this challenge by allowing models to learn from new data while retaining previously learned knowledge. Existing methods mainly rely on visual features, often neglecting the rich semantic information encoded in text. The semantic knowledge available in the label information of the images, offers important semantic information that can be related with previously acquired knowledge of semantic classes. Consequently, effectively leveraging this information throughout continual learning is expected to be beneficial. To address this, we propose integrating semantic guidance within and across tasks by capturing semantic similarity using text embeddings. We start from a pre-trained CLIP model, employ the \emph{Semantically-guided Representation Learning (SG-RL)} module for a soft-assignment towards all current task classes, and use the Semantically-guided Knowledge Distillation (SG-KD) module for enhanced knowledge transfer. Experimental results demonstrate the superiority of our method on general and fine-grained datasets. Our code can be found in https://github.com/aprilsveryown/semantically-guided-continual-learning.