OpenRaiser/Intern-Atlas huggingface dataset:
Ddiprompt: Drug-drug interaction event prediction based on graph prompt learning
Authors: Yingying Wang, Yun Xiong, Xixi Wu, Xiangguo Sun, Jiawei Zhang, Guangyong Zheng
Venue: Conference CIKM 2024
Year: 2024

Drug combinations can cause adverse drug-drug interactions(DDIs). Identifying specific effects is crucial for developing safer therapies. Previous works on DDI event prediction have typically been limited to using labels of specific events as supervision, which renders them insufficient to address two significant challenges: (1) the bias caused by highly imbalanced event distribution where certain interaction types are vastly underrepresented. (2) the scarcity of labeled data for rare events, a pervasive issue where rare yet potentially critical interactions are often overlooked or under-explored due to limited available data. In response, we offer "DDIPrompt", an innovative solution inspired by the recent advancements in graph prompt learning. Our framework aims to address these issues by leveraging the intrinsic knowledge from pre-trained models, which can be efficiently deployed with minimal downstream data. Specifically, to solve the first challenge, DDIPrompt features a hierarchical pre-training strategy to foster a generalized and comprehensive understanding of drug properties. It captures intra-molecular structures through augmented links based on structural proximity between drugs, further learns inter-molecular interactions emphasizing edge connections rather than concrete catagories. For the second challenge, we implement a prototype-enhanced prompting mechanism during inference. This mechanism, refined by few-shot examples from each category, effectively harnesses the rich pre-training knowledge to enhance prediction accuracy, particularly for these rare but crucial interactions. Comprehensive evaluations on two benchmark datasets demonstrate DDIPrompt's SOTA performance, especially for those rare DDI events.
Metamatbench: Integrating heterogeneous data, computational tools, and visual interface for metamaterial discovery
Authors: Jianpeng Chen, Wangzhi Zhan, Haohui Wang, Zian Jia, Jingru Gan, Junkai Zhang, Jingyuan Qi, Tingwei Chen, Lifu Huang, Muhao Chen, Ling Li, Wei Wang, Dawei Zhou
Venue: Conference KDD 2025
Year: 2025

Metamaterials, engineered materials with architected structures across multiple length scales, offer unprecedented and tunable mechanical properties that surpass those of conventional materials. However, leveraging advanced machine learning (ML) for metamaterial discovery is hindered by three fundamental challenges: (C1) Data Heterogeneity Challenge arises from heterogeneous data sources, heterogeneous composition scales, and heterogeneous structure categories; (C2) Model Complexity Challenge stems from the intricate geometric constraints of ML models, which complicate their adaptation to metamaterial structures; and (C3) Human-AI Collaboration Challenge comes from the “dual black-box” nature of sophisticated ML models and the need for intuitive user interfaces. To tackle these challenges, we introduce a unified framework, named MetamatBench, that operates on three levels. (1) At the data level, we integrate and standardize 5 heterogeneous, multi-modal metamaterial datasets. (2) The ML level provides a comprehensive toolkit that adapts 17 state-of-the-art ML methods for metamaterial discovery. It also includes a comprehensive evaluation suite with 12 novel performance metrics with finite element-based assessments to ensure accurate and reliable model validation. (3) The user level features a visual-interactive interface that bridges the gap between complex ML techniques and non-ML researchers, advancing property prediction and inverse design of metamaterials for research and applications. MetamatBench offers a unified platform that enables machine learning researchers and practitioners to develop and evaluate new methodologies in metamaterial discovery. For accessibility and reproducibility, we open-source our benchmark and the code-base at https://github.com/cjpcool/Metamaterial-Benchmark.
Novelty-aware graph traversal and expansion for hierarchical reinforcement learning
Authors: Jongchan Park, Seungjun Oh, Yusung Kim
Venue: Conference CIKM 2024
Year: 2024

Hierarchical Reinforcement Learning (HRL) is specially designed for environments characterized by long-term goals and sparse rewards. High-level policies in HRL learn to generate appropriate subgoals aimed at accomplishing the final goal, while low-level policies focus on achieving these designated subgoals. Recently, graph-based HRL algorithms have demonstrated enhanced learning capabilities through the structural representation of state spaces as graphs. However, existing graph-based HRL methods still often generate inefficient subgoals. This paper introduces a new method, Novelty-aware Graph Traversal and Expansion (NGTE), which selects an optimal node at the graph boundary, termed an Outpost Subgoal, as a direct path toward the final goal. Once the Outpost Subgoal is reached, NGTE transitions into an exploration phase, offering exploration subgoals within a reachable distance to efficiently expand the graph. Demonstrated in complex environments such as quadruped robot navigation and robotic arm manipulation, NGTE consistently outperforms existing graph and non-graph HRL methods, showing outstanding performance, especially in the most challenging scenarios with fixed start and fixed goal conditions.
Pre-training for recommendation unlearning
Authors: Guoxuan Chen, Lianghao Xia, Chao Huang
Venue: Conference SIGIR 2025
Year: 2025

Modern recommender systems powered by Graph Neural Networks (GNNs) excel at modeling complex user-item interactions, yet increasingly face scenarios requiring selective forgetting of training data. Beyond user requests to remove specific interactions due to privacy concerns or preference changes, regulatory frameworks mandate recommender systems' ability to eliminate the influence of certain user data from models. This recommendation unlearning challenge presents unique difficulties as removing connections within interaction graphs creates ripple effects throughout the model, potentially impacting recommendations for numerous users. Traditional approaches suffer from significant drawbacks: fragmentation methods damage graph structure and diminish performance, while influence function techniques make assumptions that may not hold in complex GNNs, particularly with self-supervised or random architectures. To address these limitations, we propose a novel model-agnostic pre-training paradigm UnlearnRec that prepares systems for efficient unlearning operations. Our Influence Encoder takes unlearning requests together with existing model parameters and directly produces updated parameters of unlearned model with little fine-tuning, avoiding complete retraining while preserving model performance characteristics. Extensive evaluation on public benchmarks demonstrates that our method delivers exceptional unlearning effectiveness while providing more than 10x speedup compared to retraining approaches. We release our method implementation at: https://github.com/HKUDS/UnlearnRec.
Calibration of time-series forecasting: Detecting and adapting context-driven distribution shift
Authors: Mouxiang Chen, Lefei Shen, Han Fu, Zhuo Li 0014, Jianling Sun, Chenghao Liu
Venue: Conference KDD 2024
Year: 2024

Recent years have witnessed the success of introducing deep learning models to time series forecasting. From a data generation perspective, we illustrate that existing models are susceptible to distribution shifts driven by temporal contexts, whether observed or unobserved. Such context-driven distribution shift (CDS) introduces biases in predictions within specific contexts and poses challenges for conventional training paradigms. In this paper, we introduce a universal calibration methodology for the detection and adaptation of CDS with a trained model. To this end, we propose a novel CDS detector, termed the "residual-based CDS detector" or "Reconditionor", which quantifies the model's vulnerability to CDS by evaluating the mutual information between prediction residuals and their corresponding contexts. A high Reconditionor score indicates a severe susceptibility, thereby necessitating model adaptation. In this circumstance, we put forth a straightforward yet potent adapter framework for model calibration, termed the "sample-level contextualized adapter" or "SOLID". This framework involves the curation of a contextually similar dataset to the provided test sample and the subsequent fine-tuning of the model's prediction layer with a limited number of steps. Our theoretical analysis demonstrates that this adaptation strategy can achieve an optimal bias-variance trade-off. Notably, our proposed Reconditionor and SOLID are model-agnostic and readily adaptable to a wide range of models. Extensive experiments show that SOLID consistently enhances the performance of current forecasting models on real-world datasets, especially on cases with substantial CDS detected by the proposed Reconditionor, thus validating the effectiveness of the calibration approach.
Multi-modal data processing for foundation models: Practical guidances and use cases
Authors: Daoyuan Chen, Yaliang Li, Bolin Ding
Venue: Conference KDD 2024
Year: 2024

In the foundation models era, efficiently processing multi-modal data is crucial. This tutorial covers key techniques for multi-modal data processing and introduces the open-source Data-Juicer system, designed to tackle the complexities of data variety, quality, and scale. Participants will learn how to use Data-Juicer's operators and tools for formatting, mapping, filtering, deduplicating, and selecting multi-modal data efficiently and effectively. They will also be familiar with the Data-Juicer Sandbox Lab, where users can easily experiment with diverse data recipes that represent methodical sequences of operators and streamline the creation of scalable data processing pipelines. This experience solidifies the concepts discussed, as well as provides a space for innovation and exploration, highlighting how data recipes can be optimized and deployed in high-performance distributed environments.
Egomotion and the stabilized world
Authors: David J. Heeger, Greg Hager
Venue: Conference ICCV 1988
Year: 1988

This paper formulates the tasks of moving-object detection and motion-based depth recovery as a problem of sensor fusion in the presence of uncertainty. We utilize two sensor systems, one providing information about the local image velocity (spsecifying a point in image-velocity space), and the othter providing information about the camera motion (specifying a line segment in image-velocity space). 1Ne utilize Mahalanobis distance as a threshold rule for determining consistency between the measurements from the two sensor systems. We also suggest a framework for using the resulting segmented flow field to update estimates of the egomotion parameters.
Geometric context from a single image
Authors: Derek Hoiem, Alexei A. Efros, Martial Hebert
Venue: Conference ICCV 2005
Year: 2005

Many computer vision algorithms limit their performance by ignoring the underlying 3D geometric structure in the image. We show that we can estimate the coarse geometric properties of a scene by learning appearance-based models of geometric classes, even in cluttered natural scenes. Geometric classes describe the 3D orientation of an image region with respect to the camera. We provide a multiple-hypothesis framework for robustly estimating scene structure from a single image and obtaining confidences for each geometric label. These confidences can then be used to improve the performance of many other applications. We provide a thorough quantitative evaluation of our algorithm on a set of outdoor images and demonstrate its usefulness in two applications: object detection and automatic single-view reconstruction.
Detection and matching of rectilinear structures
Authors: Branislav Micusík, Horst Wildenauer, Jana Kosecka
Venue: Conference CVPR 2008
Year: 2008

Indoor and outdoor urban environments posses many regularities which can be efficiently exploited and used for general image parsing tasks. We present a novel approach for detecting rectilinear structures and demonstrate their use for wide baseline stereo matching, planar 3D reconstruction, and computation of geometric context. Assuming a presence of dominant orthogonal vanishing directions, we proceed by formulating the detection of the rectilinear structures as a labeling problem on detected line segments. The line segment labels, respecting the proposed grammar rules, are established as the MAP assignment of the corresponding MRF. The proposed framework allows to detect both full as well as partial rectangles, rectangle-in-rectangle structures, and rectangles sharing edges. The use of detected rectangles is demonstrated in the context of difficult wide baseline matching tasks in the presence of repetitive structures and large appearance changes.
Unsupervised detection and segmentation of identical objects
Authors: Minsu Cho, Young Min Shin, Kyoung Mu Lee
Venue: Conference CVPR 2010
Year: 2010

We address an unsupervised object detection and segmentation problem that goes beyond the conventional assumptions of one-to-one object correspondences or modeltest settings between images. Our method can detect and segment identical objects directly from a single image or a handful of images without any supervision. To detect and segment all the object-level correspondences from the given images, a novel multi-layer match-growing method is proposed that starts from initial local feature matches and explores the images by intra-layer expansion and inter-layer merge. It estimates geometric relations between object entities and establishes ‘object correspondence networks’ that connect matching objects. Experiments demonstrate robust performance of our method on challenging datasets.
Level-set person segmentation and tracking with multi-region appearance models and top-down shape information
Authors: Esther Horbert, Konstantinos Rematas, Bastian Leibe
Venue: Conference ICCV 2011
Year: 2011

In this paper, we address the problem of segmentation-based tracking of multiple articulated persons. We propose two improvements to current level-set tracking formulations. The first is a localized appearance model that uses additional level-sets in order to enforce a hierarchical subdivision of the object shape into multiple connected regions with distinct appearance models. The second is a novel mechanism to include detailed object shape information in the form of a per-pixel figure/ground probability map obtained from an object detection process. Both contributions are seamlessly integrated into the level-set framework. Together, they considerably improve the accuracy of the tracked segmentations. We experimentally evaluate our proposed approach on two challenging sequences and demonstrate its good performance in practice.
Image based detection of geometric changes in urban environments
Authors: Aparna Taneja, Luca Ballan, Marc Pollefeys
Venue: Conference ICCV 2011
Year: 2011

In this paper, we propose an efficient technique to detect changes in the geometry of an urban environment using some images observing its current state. The proposed method can be used to significantly optimize the process of updating the 3D model of a city changing over time, by restricting this process to only those areas where changes are detected. With this application in mind, we designed our algorithm to specifically detect only structural changes in the environment, ignoring any changes in its appearance, and ignoring also all the changes which are not relevant for update purposes, such as cars, people etc. As a by-product, the algorithm also provides a coarse geometry of the detected changes. The performance of the proposed method was tested on four different kinds of urban environments and compared with two alternative techniques.
A hierarchical field framework for unified context-based classification
Authors: Sanjiv Kumar, Martial Hebert
Venue: Conference ICCV 2005
Year: 2005

We present a two-layer hierarchical formulation to exploit different levels of contextual information in images for robust classification. Each layer is modeled as a conditional field that allows one to capture arbitrary observation-dependent label interactions. The proposed framework has two main advantages. First, it encodes both the short-range interactions (e.g., pixelwise label smoothing) as well as the long-range interactions (e.g., relative configurations of objects or regions) in a tractable manner. Second, the formulation is general enough to be applied to different domains ranging from pixelwise image labeling to contextual object detection. The parameters of the model are learned using a sequential maximum-likelihood approximation. The benefits of the proposed framework are demonstrated on four different datasets and comparison results are presented
3d layoutcrf for multi-view object class recognition and segmentation
Authors: Derek Hoiem, Carsten Rother, John M. Winn
Venue: Conference CVPR 2007
Year: 2007

We introduce an approach to accurately detect and segment partially occluded objects in various viewpoints and scales. Our main contribution is a novel framework for combining object-level descriptions (such as position, shape, and color) with pixel-level appearance, boundary, and occlusion reasoning. In training, we exploit a rough 3D object model to learn physically localized part appearances. To find and segment objects in an image, we generate proposals based on the appearance and layout of local parts. The proposals are then refined after incorporating object-level information, and overlapping objects compete for pixels to produce a final description and segmentation of objects in the scene. A further contribution is a novel instance penalty, which is handled very efficiently during inference. We experimentally validate our approach on the challenging PASCAL'06 car database.
D-Clutter: Building object model library from unsupervised segmentation of cluttered scenes
Authors: Gowri Somanath, M. V. Rohith, Dimitris N. Metaxas, Chandra Kambhamettu
Venue: Conference CVPR 2009
Year: 2009

Autonomous systems which learn and utilize a limited visual vocabulary have wide spread applications. Enabling such systems to segment a set of cluttered scenes into objects is a challenging vision problem owing to the non-homogeneous texture of objects and the random configurations of multiple objects in each scene. We present a solution to the following question: given a collection of images where each object appears in one or more images and multiple objects occur in each image, how best can we extract the boundaries of the different objects? The algorithm is presented with a set of stereo images, with one stereo pair per scene. The novelty of our work is the use of both color/texture and structure to refine previously determined object boundaries to achieve segmentation consistent with each of the input scenes presented. The algorithm populates an object library, which consists of a 3D model per object. Since an object is characterized both by texture and structure, for most purposes this representation is both complete and concise.
Camera calibration with known rotation
Authors: Jan-Michael Frahm, Reinhard Koch
Venue: Conference ICCV 2003
Year: 2003

We address the problem of using external rotation information with uncalibrated video sequences. The main problem addressed is, what is the benefit of the orientation information for camera calibration? It is shown that in case of a rotating camera the camera calibration problem is linear even in the case that all intrinsic parameters vary. For arbitrarily moving cameras the calibration problem is also linear but underdetermined for the general case of varying all intrinsic parameters. However, if certain constraints are applied to the intrinsic parameters the camera calibration can be computed linearly. It is analyzed which constraints are needed for camera calibration of freely moving cameras. Furthermore we address the problem of aligning the camera data with the rotation sensor data in time. We give an approach to align these data in case of a rotating camera.
Growing semantically meaningful models for visual slam
Authors: Alex Flint, Christopher Mei, Ian D. Reid 0001, David William Murray 0001
Venue: Conference CVPR 2010
Year: 2010

Though modern Visual Simultaneous Localisation and Mapping (vSLAM) systems are capable of localising robustly and efficiently even in the case of a monocular camera, the maps produced are typically sparse point-clouds that are difficult to interpret and of little use for higher-level reasoning tasks such as scene understanding or human- machine interaction. In this paper we begin to address this deficiency, presenting progress on expanding the competency of visual SLAM systems to build richer maps. Specifically, we concentrate on modelling indoor scenes using semantically meaningful surfaces and accompanying labels, such as “floor”, “wall”, and “ceiling” - an important step towards a representation that can support higher-level reasoning and planning. We leverage the Manhattan world assumption and show how to extract vanishing directions jointly across a video stream. We then propose a guided line detector that utilises known vanishing points to extract extremely subtle axis- aligned edges. We utilise recent advances in single view structure recovery to building geometric scene models and demonstrate our system operating on-line.
Comparison of local plane fitting methods for range data
Authors: Caihua Wang, Hideki Tanahashi, Hidekazu Hirayu, Yoshinori Niwa, Kazuhiko Yamamoto
Venue: Conference CVPR 2001
Year: 2001

In this research, we introduce a reasonable noise model for range data which is obtained by a laser radar range finder, and derive two simple approximate solutions of optimal local plane fitting the range data under the noise model. We compare our methods with general least-squares based methods, such as Z-function fitting, the eigenvalue method, the maximum likelihood estimation method, and the renormalization method, an iterative method to obtain the optimal fitting of planes of range data under the noise model. All the methods are compared and evaluated using both synthetic range data and real range data with ground truth. From the experimental evaluation results, the proposed methods are shown to be effective, and the general least-squares-based methods are shown to be unsuitable for the assumed noise model.
Set Oriented Retrieval
Authors: Abraham Bookstein
Venue: Conference SIGIR 1988
Year: 1988

The broad way in which we look at how an IRS functions influences the types of questions we ask about it and the ways we try to improve performance. In the recent past, retrieval methodologies have been based on retrieving documents one at a time. In this paper we are introducing a set oriented view. We observe that this view is quite consistent with the single-document or sequential methods, and define a precise model to capture the set-oriented approach. We then examine a number consequences of the model, such as the limitations implied by a finite index vocabulary. Finally, we discuss various ways in which the set orientation can influence our thinking about IR.
A Statistical Similarity Measure
Authors: S. K. Michael Wong, Yiyu Yao
Venue: Conference SIGIR 1987
Year: 1987

Within the framework of the vector space models, a statistical similarity measure between document and query is proposed. In this approach the assumption that term (or atomic) vectors are pairwise orthogonal is not required. In addition, it provides a natural and consistent interpretation of term occurrence frequencies obtained from autoindexing.