OpenRaiser/Intern-Atlas huggingface dataset:
User attention-guided multimodal dialog systems
Authors: Chen Cui, Wenjie Wang, Xuemeng Song, Minlie Huang, Xin-Shun Xu, Liqiang Nie
Venue: Conference SIGIR 2019
Year: 2019

As an intelligent way to interact with computers, the dialog system has been catching more and more attention. However, most research efforts only focus on text-based dialog systems, completely ignoring the rich semantics conveyed by the visual cues. Indeed, the desire for multimodal task-oriented dialog systems is growing with the rapid expansion of many domains, such as the online retailing and travel. Besides, few work considers the hierarchical product taxonomy and the users' attention to products explicitly. The fact is that users tend to express their attention to the semantic attributes of products such as color and style as the dialog goes on. Towards this end, in this work, we present a hierarchical User attention-guided Multimodal Dialog system, named UMD for short. UMD leverages a bidirectional Recurrent Neural Network to model the ongoing dialog between users and chatbots at a high level; As to the low level, the multimodal encoder and decoder are capable of encoding multimodal utterances and generating multimodal responses, respectively. The multimodal encoder learns the visual presentation of images with the help of a taxonomy-attribute combined tree, and then the visual features interact with textual features through an attention mechanism; whereas the multimodal decoder selects the required visual images and generates textual responses according to the dialog history. To evaluate our proposed model, we conduct extensive experiments on a public multimodal dialog dataset in the retailing domain. Experimental results demonstrate that our model outperforms the existing state-of-the-art methods by integrating the multimodal utterances and encoding the visual features based on the users' attribute-level attention.
Hybrid Fusion with Intra- and Cross-Modality Attention for Image-Recipe Retrieval
Authors: Jiao Li, Xing Xu 0001, Wei Yu, Fumin Shen, Zuo Cao, Kai Zuo, Heng Tao Shen
Venue: Conference SIGIR 2021
Year: 2021

Image-recipe retrieval, which aims at retrieving the relevant recipe from a food image and vice versa, is now attracting widespread attention, since sharing food-related images and recipes on the Internet has become a popular trend. Existing methods have formulated this problem as a typical cross-modal retrieval task by learning the image-recipe similarity. Though these methods have made inspiring achievements for image-recipe retrieval, they may still be less effective to jointly incorporate the three crucial points: (1) the association between ingredients and instructions, (2) fine-grained image information, and (3) the latent alignment between recipes and images. To this end, we propose a novel framework namedHybrid Fusion with Intra- and Cross-Modality Attention (HF-ICMA) to learn accurate image-recipe similarity. Our HF-ICMA model adopts an intra-recipe fusion module to focus on the interaction between ingredients and instructions within a recipe, and further enriches the expressions of the two separate embeddings. Meanwhile, an image-recipe fusion module is devised to explore the potential relationship between fine-grained image regions and ingredients from the recipe, which jointly forms the final image-recipe similarity from both the local and global aspects. Extensive experiments on the large-scale benchmark dataset Recipe1M show that our model significantly outperforms the state-of-the-art approaches on various image-recipe retrieval scenarios.
Graph-structured context understanding for knowledge-grounded response generation
Authors: Yanran Li, Wenjie Li 0002, Zhitao Wang
Venue: Conference SIGIR 2021
Year: 2021

In this work, we establish a context graph from both conversation utterances and external knowledge, and develop a novel graph-based encoder to better understand the conversation context. Specifically, the encoder fuses the information in the context graph stage-by-stage and provides global context-graph-aware representations of each node in the graph to facilitate knowledge-grounded response generation. On a large-scale conversation corpus, we validate the effectiveness of the proposed approach and demonstrate the benefit of knowledge in conversation understanding.
A Low-Cost, Controllable and Interpretable Task- Oriented Chatbot: With Real-World After-Sale Services as Example
Authors: Xiangyu Xi, Chenxu Lv, Yuncheng Hua, Wei Ye, Chaobo Sun, Shuaipeng Liu, Fan Yang, Guanglu Wan
Venue: Conference SIGIR 2022
Year: 2022

Though widely used in industry, traditional task-oriented dialogue systems suffer from three bottlenecks: (i) difficult ontology construction (e.g., intents and slots); (ii) poor controllability and interpretability; (iii) annotation-hungry. In this paper, we propose to represent utterance with a simpler concept named Dialogue Action, upon which we construct a tree-structured TaskFlow and further build task-oriented chatbot with TaskFlow as core component. A framework is presented to automatically construct TaskFlow from large-scale dialogues and deploy online. Our experiments on real-world after-sale customer services show TaskFlow can satisfy the major needs, as well as reduce the developer burden effectively.
On Extractive Summarization for Profile-centric Neural Expert Search in Academia
Authors: Rennan C. Lima, Rodrygo L. T. Santos
Venue: Conference SIGIR 2022
Year: 2022

Identifying academic experts is crucial for the progress of science, enabling researchers to connect, form networks, and collaborate on the most pressing research problems. A key challenge for ranking experts in response to a query is how to infer their expertise from the publications they coauthored. Profile-centric approaches represent candidate experts by concatenating all their publications into a text-based profile. Despite offering a complete picture of each candidate's scientific output, such lengthy profiles make it inefficient to leverage state-of-the-art neural architectures for inferring expertise. To overcome this limitation, we investigate the suitability of extractive summarization as a mechanism to reduce candidate profiles for semantic encoding using Transformers. Our thorough experiments with a representative academic search test collection demonstrate the benefits of encoding summarized profiles for an improved expertise inference.
Contrastive meta learning with behavior multiplicity for recommendation
Authors: Wei Wei, Chao Huang, Lianghao Xia, Yong Xu, Jiashu Zhao, Dawei Yin
Venue: Conference WSDM 2022
Year: 2022

A well-informed recommendation framework could not only help users identify their interested items, but also benefit the revenue of various online platforms (e.g., e-commerce, social media). Traditional recommendation models usually assume that only a single type of interaction exists between user and item, and fail to model the multiplex user-item relationships from multi-typed user behavior data, such as page view, add-to-favourite and purchase. While some recent studies propose to capture the dependencies across different types of behaviors, two important challenges have been less explored: i) Dealing with the sparse supervision signal under target behaviors (e.g., purchase). ii) Capturing the personalized multi-behavior patterns with customized dependency modeling. To tackle the above challenges, we devise a new model CML, Contrastive Meta Learning (CML), to maintain dedicated cross-type behavior dependency for different users. In particular, we propose a multi-behavior contrastive learning framework to distill transferable knowledge across different types of behaviors via the constructed contrastive loss. In addition, to capture the diverse multi-behavior patterns, we design a contrastive meta network to encode the customized behavior heterogeneity for different users. Extensive experiments on three real-world datasets indicate that our method consistently outperforms various state-of-the-art recommendation methods. Our empirical studies further suggest that the contrastive meta learning paradigm offers great potential for capturing the behavior multiplicity in recommendation. We release our model implementation at: https://github.com/weiwei1206/CML.git.
Generative adversarial framework for cold-start item recommendation
Authors: Hao Chen, Zefan Wang, Feiran Huang, Xiao Huang, Yue Xu, Yishi Lin, Peng He, Zhoujun Li
Venue: Conference SIGIR 2022
Year: 2022

The cold-start problem has been a long-standing issue in recommendation. Embedding-based recommendation models provide recommendations by learning embeddings for each user and item from historical interactions. Therefore, such embedding-based models perform badly for cold items which haven't emerged in the training set. The most common solutions are to generate the cold embedding for the cold item from its content features. However, the cold embeddings generated from contents have different distribution as the warm embeddings are learned from historical interactions. In this case, current cold-start methods are facing an interesting seesaw phenomenon, which improves the recommendation of either the cold items or the warm items but hurts the opposite ones. To this end, we propose a general framework named Generative Adversarial Recommendation (GAR). By training the generator and the recommender adversarially, the generated cold item embeddings can have similar distribution as the warm embeddings that can even fool the recommender. Simultaneously, the recommender is fine-tuned to correctly rank the "fake'' warm embeddings and the real warm embeddings. Consequently, the recommendation of the warms and the colds will not influence each other, thus avoiding the seesaw phenomenon. Additionally, GAR could be applied to any off-the-shelf recommendation model. Experiments on two datasets present that GAR has strong overall recommendation performance in cold-starting both the CF-based model (improved by over 30.18%) and the GNN-based model (improved by over 17.78%).
Alleviating cold-start problems in recommendation through pseudo-labelling over knowledge graph
Authors: Riku Togashi, Mayu Otani, Shin&apos, ichi Satoh
Venue: Conference WSDM 2021
Year: 2021

Solving cold-start problems is indispensable to provide meaningful recommendation results for new users and items. Under sparsely observed data, unobserved user-item pairs are also a vital source for distilling latent users' information needs. Most present works leverage unobserved samples for extracting negative signals. However, such an optimisation strategy can lead to biased results toward already popular items by frequently handling new items as negative instances. In this study, we tackle the cold-start problems for new users/items by appropriately leveraging unobserved samples. We propose a knowledge graph (KG)-aware recommender based on graph neural networks, which augments labelled samples through pseudo-labelling. Our approach aggressively employs unobserved samples as positive instances and brings new items into the spotlight. To avoid exhaustive label assignments to all possible pairs of users and items, we exploit a KG for selecting probably positive items for each user. We also utilise an improved negative sampling strategy and thereby suppress the exacerbation of popularity biases. Through experiments, we demonstrate that our approach achieves improvements over the state-of-the-art KG-aware recommenders in a variety of scenarios; in particular, our methodology successfully improves recommendation performance for cold-start users/items.
Investigating accuracy-novelty performance for graph-based collaborative filtering
Authors: Minghao Zhao, Le Wu, Yile Liang, Lei Chen, Jian Zhang, Qilin Deng, Kai Wang, Xudong Shen, Tangjie Lv, Runze Wu
Venue: Conference SIGIR 2022
Year: 2022

Recent years have witnessed the great accuracy performance of graph-based Collaborative Filtering (CF) models for recommender systems. By taking the user-item interaction behavior as a graph, these graph-based CF models borrow the success of Graph Neural Networks (GNN), and iteratively perform neighborhood aggregation to propagate the collaborative signals. While conventional CF models are known for facing the challenges of the popularity bias that favors popular items, one may wonder "Whether the existing graph-based CF models alleviate or exacerbate the popularity bias of recommender systems?" To answer this question, we first investigate the two-fold performances w.r.t. accuracy and novelty for existing graph-based CF methods. The empirical results show that symmetric neighborhood aggregation adopted by most existing graph-based CF models exacerbates the popularity bias and this phenomenon becomes more serious as the depth of graph propagation increases. Further, we theoretically analyze the cause of popularity bias for graph-based CF. Then, we propose a simple yet effective plugin, namely r-AdjNorm, to achieve an accuracy-novelty trade-off by controlling the normalization strength in the neighborhood aggregation process. Meanwhile, r-AdjNorm can be smoothly applied to the existing graph-based CF backbones without additional computation. Finally, experimental results on three benchmark datasets show that our proposed method can improve novelty without sacrificing accuracy under various graph-based CF backbones.
Discrimination between Tasks with User Activity Patterns during Information Search
Authors: Michael J. Cole, Chathra Hendahewa, Nicholas J. Belkin, Chirag Shah
Venue: Conference SIGIR 2014
Year: 2014

Can the activity patterns of page use during information search sessions discriminate between different types of information seeking tasks? We model sequences of interactions with search result and content pages during information search sessions. Two representations are created: the sequences of page use and a cognitive representation of page interactions. The cognitive representation is based on logged eye movement patterns of textual information acquisition via the reading process. Page sequence actions from task sessions (n=109) in a user study are analyzed. The study tasks differed from one another in basic dimensions of complexity, specificity,level, and the type of information product (intellectual or factual). The results show that differences in task types can be measured at both the level of observations of page type sequences and at the level of cognitive activity on the pages. We discuss the implications for personalization of search systems, measurement of task similarity and the development of user-centered information systems that can support the user's current and expected search intentions.
An Effective Implicit Relevance Feedback Technique Using Affective, Physiological and Behavioural Features
Authors: Yashar Moshfeghi, Joemon M. Jose
Venue: Conference SIGIR 2013
Year: 2013

The effectiveness of various behavioural signals for implicit relevance feedback models has been exhaustively studied. Despite the advantages of such techniques for a real time information retrieval system, most of the behavioural signals are noisy and therefore not reliable enough to be employed. Among many, a combination of dwell time and task information has been shown to be effective for relevance judgement prediction. However, the task information might not be available to the system at all times. Thus, there is a need for other sources of information which can be used as a substitute for task information. Recently, affective and physiological signals have shown promise as a potential source of information for relevance judgement prediction. However, their accuracy is not high enough to be applicable on their own.
Understanding information need: An fMRI study
Authors: Yashar Moshfeghi, Peter Triantafillou, Frank E. Pollick
Venue: Conference SIGIR 2016
Year: 2016

The raison d'etre of IR is to satisfy human information need. But, do we really understand information need? Despite advances in the past few decades in both the IR and relevant scientific communities, this question is largely unanswered. We do not really understand how an information need emerges and how it is physically manifested. Information need refers to a complex concept: at the very initial state of the phenomenon (i.e. at a visceral level), even the searcher may not be aware of its existence. This renders the measuring of this concept (using traditional behaviour studies) nearly impossible. In this paper, we investigate the connection between an information need and brain activity. Using functional Magnetic Resonance Imaging (fMRI), we measured the brain activity of twenty four participants while they performed a Question Answering (Q/A) Task, where the questions were carefully selected and developed from TREC-8 and TREC 2001 Q/A Track. The results of this experiment revealed a distributed network of brain regions commonly associated with activities related to information need and retrieval and differing brain activity in processing scenarios when participants knew the answer to a given question and when they did not and needed to search. We believe our study and conclusions constitute an important step in unravelling the nature of information need and therefore better satisfying it.
Predicting Online Shopping Search Satisfaction and User Behaviors with Electrodermal Activity
Authors: Yingying Wu, Yiqun Liu, Ning Su, Shaoping Ma, Wenwu Ou
Venue: Conference WWW 2017
Year: 2017

Electrodermal activity (EDA) delineates positive and negative emotions, especially in the lower arousal range, which reflects small variations. This study derived formulas to extract features from the EDA graph and found they are effective to predict search satisfaction and explain mobile shopping behaviors including add to cart, purchase, and abandonment (added to cart without purchase). The best features found in the generalized linear mixed model with the binomial family agree with known physiological results. To the best of our knowledge, this study is the first to use physiological methods to study satisfaction and user behavior.
Axiomatically regularized pre-training for ad hoc search
Authors: Jia Chen, Yiqun Liu, Yan Fang, Jiaxin Mao, Hui Fang, Shenghao Yang, Xiaohui Xie, Min Zhang, Shaoping Ma
Venue: Conference SIGIR 2022
Year: 2022

Recently, pre-training methods tailored for IR tasks have achieved great success. However, as the mechanisms behind the performance improvement remain under-investigated, the interpretability and robustness of these pre-trained models still need to be improved. Axiomatic IR aims to identify a set of desirable properties expressed mathematically as formal constraints to guide the design of ranking models. Existing studies have already shown that considering certain axioms may help improve the effectiveness and interpretability of IR models. However, there still lack efforts of incorporating these IR axioms into pre-training methodologies. To shed light on this research question, we propose a novel pre-training method with \underlineA xiomatic \underlineRe gularization for ad hoc \underlineS earch (ARES). In the ARES framework, a number of existing IR axioms are re-organized to generate training samples to be fitted in the pre-training process. These training samples then guide neural rankers to learn the desirable ranking properties. Compared to existing pre-training approaches, ARES is more intuitive and explainable. Experimental results on multiple publicly available benchmark datasets have shown the effectiveness of ARES in both full-resource and low-resource (e.g., zero-shot and few-shot) settings. An intuitive case study also indicates that ARES has learned useful knowledge that existing pre-trained models (e.g., BERT and PROP) fail to possess. This work provides insights into improving the interpretability of pre-trained models and the guidance of incorporating IR axioms or human heuristics into pre-training methods.
The Datasets Dilemma: How Much Do We Really Know About Recommendation Datasets?
Authors: Jin Yao Chin, Yile Chen, Gao Cong
Venue: Conference WSDM 2022
Year: 2022

There has been sustained interest from both academia and industry throughout the years due to the importance and practicability of recommendation systems. However, several recent papers have pointed out critical issues with the evaluation process in recommender systems. Likewise, this paper takes an in-depth look at a fundamental but often neglected aspect of the evaluation procedure, i.e. the datasets themselves. To do so, we adopt a systematic and comprehensive approach to understand the datasets used for implicit feedback based top-K recommendation. We start by examining recent papers from top-tier conferences to find out how different datasets have been utilised thus far. Next, we look at the characteristics of these datasets to understand their similarities and differences. Finally, we conduct an empirical study to determine whether the choice of datasets used for evaluation can influence the observations and/or conclusions obtained. Our findings suggest that greater attention needs to be paid to the selection process of datasets used for evaluating recommender systems in order to improve the robustness of the obtained results.
Random walks in recommender systems: exact computation and simulations
Authors: Colin Cooper, Sang-Hyuk Lee, Tomasz Radzik, Yiannis Siantos
Venue: Conference WWW 2014
Year: 2014

A recommender system uses information about known associations between users and items to compute for a given user an ordered recommendation list of items which this user might be interested in acquiring. We consider ordering rules based on various parameters of random walks on the graph representing associations between users and items. We experimentally compare the quality of recommendations and the required computational resources of two approaches: (i) calculate the exact values of the relevant random walk parameters using matrix algebra; (ii) estimate these values by simulating random walks. In our experiments we include methods proposed by Fouss et al. and Gori and Pucci, method P3, which is based on the distribution of the random walk after three steps, and method P3a, which generalises P3. We show that the simple method P3 can outperform previous methods and method P3a can offer further improvements. We show that the time- and memory-efficiency of direct simulation of random walks allows application of these methods to large datasets. We use in our experiments the three MovieLens datasets.
Admm slim: Sparse recommendations for many users
Authors: Harald Steck, Maria Dimakopoulou, Nickolai Riabov, Tony Jebara
Venue: Conference WSDM 2020
Year: 2020

The Sparse Linear Method (SLIM) is a well-established approach for top-N recommendations. This article proposes several improvements that are enabled by the Alternating Directions Method of Multipliers (ADMM), a well-known optimization method with many application areas. First, we show that optimizing the original SLIM-objective by ADMM results in an approach where the training time is independent of the number of users in the training data, and hence trivially scales to large numbers of users. Second, the flexibility of ADMM allows us to switch on and off the various constraints and regularization terms in the original SLIM-objective, in order to empirically assess their contributions to ranking accuracy on given data. Third, we also propose two extensions to the original SLIM training-objective in order to improve recommendation accuracy further without increasing the computational cost. In our experiments on three well-known data-sets, we first compare to the original SLIM-implementation and find that not only ADMM reduces training time considerably, but also achieves an improvement in recommendation accuracy due to better optimization. We then compare to various state-of-the-art approaches and observe up to 25% improvement in recommendation accuracy in our experiments. Finally, we evaluate the importance of sparsity and the non-negativity constraint in the original SLIM-objective with sub-sampling experiments that simulate scenarios of cold-starting and large catalog sizes compared to relatively small user base, which often occur in practice.