OpenRaiser/Intern-Atlas huggingface dataset:
Weakly Supervised Video Moment Retrieval via Location-irrelevant Proposal Learning
Authors: Wei Ji, Ruiqi Shi, Yinwei Wei, Shanshan Zhao, Roger Zimmermann
Venue: Conference WWW 2024
Year: 2024

This paper deals with Video Moment Retrieval (VMR) in a weakly-supervised fashion, which aims to retrieve local video clips with only global video-level descriptions. Scrutinizing the recent advances in VMR, we find that the fully-supervised models achieve strong performance, but they are heavily relied on the precise temporal annotations. Weakly-supervised methods do not rely on temporal annotations, however, their performance is much weaker than the fully-supervised ones. To fill such gap, we propose to take advantage of a pretrained video-text model as hitchhiker to generate pseudo temporal labels. The pseudo temporal labels, together with the descriptive labels, are then utilized to guide the training of the proposed VMR model. The proposed Location-irrelevant Proposal Learning (LPL) model is based on a pretrained video-text model with cross-modal prompt learning, together with different strategies to generate reasonable proposals with various lengths. Despite the simplicity, we find that our method performs much better than the previous state-of-the-art methods on standard benchmarks, eg., +4.4% and +1.4% in mIoU on the Charades and ActivityNet-Caption datasets respectively, which benefits from training with fine-grained video-text pairs. Further experiments on two synthetic datasets with shuffled temporal location and longer video length demonstrate our model's robustness towards temporal localization bias as well as its strength in handling long video sequences.
Group-Aware Long- and Short-Term Graph Representation Learning for Sequential Group Recommendation
Authors: Wen Wang 0016, Wei Zhang 0056, Jun Rao, Zhijie Qiu, Bo Zhang 0056, Leyu Lin, Hongyuan Zha
Venue: Conference SIGIR 2020
Year: 2020

Sequential recommendation and group recommendation are two important branches in the field of recommender system. While considerable efforts have been devoted to these two branches in an independent way, we combine them by proposing the novel sequential group recommendation problem which enables modeling group dynamic representations and is crucial for achieving better group recommendation performance. The major challenge of the problem is how to effectively learn dynamic group representations based on the sequential user-item interactions of group members in the past time frames. To address this, we devise a Group-aware Long- and Short-term Graph Representation Learning approach, namely GLS-GRL, for sequential group recommendation. Specifically, for a target group, we construct a group-aware long-term graph to capture user-item interactions and item-item co-occurrence in the whole history, and a group-aware short-term graph to contain the same information regarding only the current time frame. Based on the graphs, GLS-GRL performs graph representation learning to obtain long-term and short-term user representations, and further adaptively fuse them to gain integrated user representations. Finally, group representations are obtained by a constrained user-interacted attention mechanism which encodes the correlations between group members. Comprehensive experiments demonstrate that GLS-GRL achieves better performance than several strong alternatives coming from sequential recommendation and group recommendation methods, validating the effectiveness of the core components in GLS-GRL.
GraphERT– Transformers-based Temporal Dynamic Graph Embedding
Authors: Moran Beladev, Gilad Katz, Lior Rokach, Uriel Singer, Kira Radinsky
Venue: Conference CIKM 2023
Year: 2023

Dynamic temporal graphs evolve over time, adding and removing nodes and edges between time snapshots. The tasks performed on such graphs are diverse and include detecting temporal trends, finding graph-to-graph similarities, and graph visualization and clustering. For all these tasks, it is necessary to embed the entire graph in a low-dimensional space by using graph-level representations instead of the more common node-level representations. This embedding requires handling the appearance of new nodes over time as well as capturing temporal patterns of the entire graph. Most existing methods perform temporal node embeddings and focus on different methods of aggregating them for a graph-based representation. In this work, we propose an end-to-end architecture that captures both the node embeddings and their influence in a structural context during a specific time period of the graph. We present GraphERT (Graph Embedding Representation using Transformers), a novel approach to temporal graph-level embeddings. Our method pioneers the use of Transformers to seamlessly integrate graph structure learning with temporal analysis. By employing a masked language model on sequences of graph random walks, together with a novel temporal classification task, our model not only comprehends the intricate graph dynamics but also unravels the temporal significance of each node and path. This novel training paradigm empowers GraphERT to capture the essence of both the structural and temporal aspects of graphs, surpassing state-of-the-art approaches across multiple tasks on real-world datasets.
Explainable neural rule learning
Authors: Shaoyun Shi, Yuexiang Xie, Zhen Wang, Bolin Ding, Yaliang Li, Min Zhang
Venue: Conference WWW 2022
Year: 2022

Although neural networks have achieved great successes in various machine learning tasks, people can hardly know what neural networks learn from data due to their black-box nature. The lack of such explainability is one of the limitations of neural networks when applied in domains, e.g., healthcare and finance, that demand transparency and accountability. Moreover, explainability is beneficial for guiding a neural network to learn the causal patterns that can extrapolate out-of-distribution (OOD) data, which is critical in real-world applications and has surged as a hot research topic. In order to improve the explainability of neural networks, we propose a novel method—Explainable Neural Rule Learning (denoted as ENRL), with the aim to integrate the expressiveness of neural networks and the explainability of rule-based systems. Specifically, we first design several operator modules and guide them to behave as certain relational operators via self-supervised learning. With input feature fields and learnable context values serving as arguments, these operator modules are used as predicates to constitute the atomic propositions. Then we employ neural logical operations to combine atomic propositions into a collection of rules. Finally, we design a voting mechanism for these rules so that they collaboratively make up our predictive model. Thus, rule learning is transformed to neural architecture search, that is, to choose the appropriate arrangements of feature fields and operator modules. After searching for a specific architecture and learning the involved modules, the resulting neural network explicitly expresses some rules and thus possesses explainability. Therefore, we can predict for each input instance according to rules it satisfies, which at the same time explains how the neural network makes that decision. We conduct a series of experiments on both synthetic and real-world datasets to evaluate ENRL. Compared with conventional neural networks, ENRL achieves competitive in-distribution performance while providing the extra benefits of explainability. Meanwhile, ENRL significantly alleviates performance drop on OOD test data, implying the effectiveness of rule learning. Codes are provided at https://github.com/Shuriken13/ENRL.
Maca: Memory-aided coarse-to-fine alignment for text-based person search
Authors: Liangxu Su, Rong Quan, Zhiyuan Qi, Jie Qin
Venue: Conference SIGIR 2024
Year: 2024

Text-based person search (TBPS) aims to search for the target person in the full image through textual descriptions. The key to addressing this task is to effectively perform cross-modality alignment between text and images. In this paper, we propose a novel TBPS framework, named Memory-Aided Coarse-to-fine Alignment (MACA), to learn an accurate and reliable alignment between the two modalities. Firstly, we introduce a proposal-based alignment module, which performs contrastive learning to accurately align the textual modality with different pedestrian proposals at a coarse-grained level. Secondly, for the fine-grained alignment, we propose an attribute-based alignment module to mitigate unreliable features by aligning text-wise details with image-wise global features. Moreover, we introduce an intuitive memory bank strategy to supplement useful negative samples for more effective contrastive learning, improving the convergence and generalization ability of the model based on the learned discriminative features. Extensive experiments on CUHK-SYSU-TBPS and PRW-TBPS demonstrate the superiority of MACA over state-of-the-art approaches. The code is available at https://github.com/suliangxu/MACA.
Alleviating Matching Bias in Marketing Recommendations
Authors: Junpeng Fang, Qing Cui, Gongduo Zhang, Caizhi Tang, Lihong Gu, Longfei Li, Jinjie Gu, Jun Zhou, Fei Wu
Venue: Conference SIGIR 2023
Year: 2023

In marketing recommendations, the campaign organizers will distribute coupons to users to encourage consumption. In general, a series of strategies are employed to interfere with the coupon distribution process, leading to a growing imbalance between user-coupon interactions, resulting in a bias in the estimation of conversion probabilities. We refer to the estimation bias as the matching bias. In this paper, we explore how to alleviate the matching bias from the causal-effect perspective. We regard the historical distributions of users and coupons over each other as confounders and characterize the matching bias as a confounding effect to reveal and eliminate the spurious correlations between user-coupon representations and conversion probabilities. Then we propose a new training paradigm named De-Matching Bias Recommendation (DMBR) to remove the confounding effects during model training via the backdoor adjustment. We instantiate DMBR on two representative models: DNN and MMOE, and conduct extensive offline and online experiments to demonstrate the effectiveness of our proposed paradigm.
Counterfactual Data Augmentation for Debiased Coupon Recommendations Based on Potential Knowledge
Authors: Junpeng Fang, Gongduo Zhang, Qing Cui, Lihong Gu, Longfei Li, Jinjie Gu, Jun Zhou
Venue: Conference WWW 2024
Year: 2024

In real-world coupon recommendations, the coupon allocation process is influenced by both the recommendation model trained with historical interaction data and marketing tactics aimed at specific commercial goals. These tactics can cause an imbalance in user-coupon interactions, leading to a deviation from users' natural preferences. We refer to this deviation as the matching bias. Theoretically, unbiased data which is assumed to be collected via a randomized allocating policy (i.e., without model or tactics intervention) is ideal training data because it reflects the user's natural preferences. However, obtaining unbiased data in real-world scenarios is costly and sometimes unfeasible. To address this problem, we propose a novel model-agnostic training paradigm named Counterfactual Data Augmentation for debiased coupon recommendations based on Potential Knowledge (CDAPK) for the marketing scenario that allocates coupons with discounts. We leverage the counterfactual data augmentation technique to answer the following key question: If a user is offered a coupon that he has never seen before in his history, will he use this coupon? By creating the counterfactual interaction data and assigning labels based on the potential knowledge of the given scenario, CDAPK shifts the original data distribution into an unbiased distribution, facilitating model optimization and debiasing. The advantage of CDAPK lies in its ability to approximate the ideal states of the training data without depleting the real-world traffic flow. We implement CDAPK on five representative models: FM, DNN, NCF, MASKNET, and DEEPFM, and conduct extensive offline and online experiments against SOTA debiasing methods to validate the superiority of CDAPK.
Think rationally about what you see: Continuous rationale extraction for relation extraction
Authors: Xuming Hu, Zhaochen Hong, Chenwei Zhang, Irwin King, Philip S. Yu
Venue: Conference SIGIR 2023
Year: 2023

Relation extraction (RE) aims to extract potential relations according to the context of two entities, thus, deriving rational contexts from sentences plays an important role. Previous works either focus on how to leverage the entity information (e.g., entity types, entity verbalization) to inference relations, but ignore context-focused content, or use counterfactual thinking to remove the model's bias of potential relations in entities, but the relation reasoning process will still be hindered by irrelevant content. Therefore, how to preserve relevant content and remove noisy segments from sentences is a crucial task. In addition, retained content needs to be fluent enough to maintain semantic coherence and interpretability. In this work, we propose a novel rationale extraction framework named RE2, which leverages two continuity and sparsity factors to obtain relevant and coherent rationales from sentences. To solve the problem that the gold rationales are not labeled, RE2 applies an optimizable binary mask to each token in the sentence, and adjust the rationales that need to be selected according to the relation label. Experiments on four datasets show that RE2 surpasses baselines.
Object correspondence as a machine learning problem
Authors: Bernhard Schölkopf, Florian Steinke, Volker Blanz
Venue: Conference ICML 2005
Year: 2005

We propose machine learning methods for the estimation of deformation fields that transform two given objects into each other, thereby establishing a dense point to point correspondence. The fields are computed using a modified support vector machine containing a penalty enforcing that points of one object will be mapped to "similar" points on the other one. Our system, which contains little engineering or domain knowledge, delivers state of the art performance. We present application results including close to photorealistic morphs of 3D head models.
Induction of Semantic Classes from Natural Language Text
Authors: Dekang Lin, Patrick Pantel
Venue: Conference KDD 2001
Year: 2001

Many applications dealing with textual information require classification of words into semantic classes (or concepts). However, manually constructing semantic classes is a tedious task. In this paper, we present an algorithm, UNICON, for UNsupervised Induction of CONcepts. Some advantages of UNICON over previous approaches include the ability to classify words with low frequency counts, the ability to cluster a large number of elements in a high-dimensional space, and the ability to classify previously unknown words into existing clusters. Furthermore, since the algorithm is unsupervised, a set of concepts may be constructed for any corpus.
Sources of evidence for vertical selection
Authors: Jaime Arguello, Fernando Diaz, Jamie Callan, Jean-François Crespo
Venue: Conference SIGIR 2009
Year: 2009

Web search providers often include search services for domain-specific subcollections, called verticals, such as news, images, videos, job postings, company summaries, and artist profiles. We address the problem of vertical selection, predicting relevant verticals (if any) for queries issued to the search engine's main web search page. In contrast to prior query classification and resource selection tasks, vertical selection is associated with unique resources that can inform the classification decision. We focus on three sources of evidence: (1) the query string, from which features are derived independent of external resources, (2) logs of queries previously issued directly to the vertical, and (3) corpora representative of vertical content. We focus on 18 different verticals, which differ in terms of semantics, media type, size, and level of query traffic. We compare our method to prior work in federated search and retrieval effectiveness prediction. An in-depth error analysis reveals unique challenges across different verticals and provides insight into vertical selection for future work.
Structure compilation: trading structure for features
Authors: Percy Liang, Hal Daumé III, Dan Klein 0001
Venue: Conference ICML 2008
Year: 2008

Structured models often achieve excellent performance but can be slow at test time. We investigate structure compilation, where we replace structure with features, which are often computationally simpler but unfortunately statistically more complex. We analyze this tradeoff theoretically and empirically on three natural language processing tasks. We also introduce a simple method to transfer predictive power from structure to features via unlabeled data, while incurring a minimal statistical penalty.
Query clustering using click-through graph
Authors: Jeonghee Yi, Farzin Maghoul
Venue: Conference WWW 2009
Year: 2009

In this paper we describe a problem of discovering query clusters from a click-through graph of web search logs. The graph consists of a set of web search queries, a set of pages selected for the queries, and a set of directed edges that connects a query node and a page node clicked by a user for the query. The proposed method extracts all maximal bipartite cliques (bicliques) from a click-through graph and compute an equivalence set of queries (i.e., a query cluster) from the maximal bicliques. A cluster of queries is formed from the queries in a biclique. We present a scalable algorithm that enumerates all maximal bicliques from the click-through graph. We have conducted experiments on Yahoo web search queries and the result is promising.
Query dependent ranking using k-nearest neighbor
Authors: Xiubo Geng, Tie-Yan Liu, Tao Qin, Andrew Arnold, Hang Li, Heung-Yeung Shum
Venue: Conference SIGIR 2008
Year: 2008

Many ranking models have been proposed in information retrieval, and recently machine learning techniques have also been applied to ranking model construction. Most of the existing methods do not take into consideration the fact that significant differences exist between queries, and only resort to a single function in ranking of documents. In this paper, we argue that it is necessary to employ different ranking models for different queries and onduct what we call query-dependent ranking. As the first such attempt, we propose a K-Nearest Neighbor (KNN) method for query-dependent ranking. We first consider an online method which creates a ranking model for a given query by using the labeled neighbors of the query in the query feature space and then rank the documents with respect to the query using the created model. Next, we give two offline approximations of the method, which create the ranking models in advance to enhance the efficiency of ranking. And we prove a theory which indicates that the approximations are accurate in terms of difference in loss of prediction, if the learning algorithm used is stable with respect to minor changes in training examples. Our experimental results show that the proposed online and offline methods both outperform the baseline method of using a single ranking function.
A time-dependent topic model for multiple text streams
Authors: Liangjie Hong, Byron Dom, Siva Gurumurthy, Kostas Tsioutsiouliklis
Venue: Conference KDD 2011
Year: 2011

In recent years social media have become indispensable tools for information dissemination, operating in tandem with traditional media outlets such as newspapers, and it has become critical to understand the interaction between the new and old sources of news. Although social media as well as traditional media have attracted attention from several research communities, most of the prior work has been limited to a single medium. In addition temporal analysis of these sources can provide an understanding of how information spreads and evolves. Modeling temporal dynamics while considering multiple sources is a challenging research problem. In this paper we address the problem of modeling text streams from two news sources - Twitter and Yahoo! News. Our analysis addresses both their individual properties (including temporal dynamics) and their inter-relationships. This work extends standard topic models by allowing each text stream to have both local topics and shared topics. For temporal modeling we associate each topic with a time-dependent function that characterizes its popularity over time. By integrating the two models, we effectively model the temporal dynamics of multiple correlated text streams in a unified framework. We evaluate our model on a large-scale dataset, consisting of text streams from both Twitter and news feeds from Yahoo! News. Besides overcoming the limitations of existing models, we show that our work achieves better perplexity on unseen data and identifies more coherent topics. We also provide analysis of finding real-world events from the topics obtained by our model.
Adaptive Event Detection with Time-varying Poisson Processes
Authors: Alexander Ihler, Jon Hutchins, Padhraic Smyth
Venue: Conference KDD 2006
Year: 2006

Time-series of count data are generated in many different contexts, such as web access logging, freeway traffic monitoring, and security logs associated with buildings. Since this data measures the aggregated behavior of individual human beings, it typically exhibits a periodicity in time on a number of scales (daily, weekly,etc.) that reflects the rhythms of the underlying human activity and makes the data appear non-homogeneous. At the same time, the data is often corrupted by a number of bursty periods of unusual behavior such as building events, traffic accidents, and so forth. The data mining problem of finding and extracting these anomalous events is made difficult by both of these elements. In this paper we describe a framework for unsupervised learning in this context, based on a time-varying Poisson process model that can also account for anomalous events. We show how the parameters of this model can be learned from count time series using statistical estimation techniques. We demonstrate the utility of this model on two datasets for which we have partial ground truth in the form of known events, one from freeway traffic data and another from building access data, and show that the model performs significantly better than a non-probabilistic, threshold-based technique. We also describe how the model can be used to investigate different degrees of periodicity in the data, including systematic day-of-week and time-of-day effects, and make inferences about the detected events (e.g., popularity or level of attendance). Our experimental results indicate that the proposed time-varying Poisson model provides a robust and accurate framework for adaptively and autonomously learning how to separate unusual bursty events from traces of normal human activity.
A Syntactic Tree Matching Approach to Finding Similar Questions in Community-based QA Services
Authors: Kai Wang, Zhaoyan Ming, Tat-Seng Chua
Venue: Conference SIGIR 2009
Year: 2009

While traditional question answering (QA) systems tailored to the TREC QA task work relatively well for simple questions, they do not suffice to answer real world questions. The community-based QA systems offer this service well, as they contain large archives of such questions where manually crafted answers are directly available. However, finding similar questions in the QA archive is not trivial. In this paper, we propose a new retrieval framework based on syntactic tree structure to tackle the similar question matching problem. We build a ground-truth set from Yahoo! Answers, and experimental results show that our method outperforms traditional bag-of-word or tree kernel based methods by 8.3% in mean average precision. It further achieves up to 50% improvement by incorporating semantic features as well as matching of potential answers. Our model does not rely on training, and it is demonstrated to be robust against grammatical errors as well.