OpenRaiser/Intern-Atlas huggingface dataset:
The JOKER Corpus: English-French Parallel Data for Multilingual Wordplay Recognition
Authors: Liana Ermakova, Anne-Gwenn Bosser, Adam Jatowt, Tristan Miller
Venue: Conference SIGIR 2023
Year: 2023

Despite recent advances in information retrieval and natural language processing, rhetorical devices that exploit ambiguity or subvert linguistic rules remain a challenge for such systems. However, corpus-based analysis of wordplay has been a perennial topic of scholarship in the humanities, including literary criticism, language education, and translation studies. The immense data-gathering effort required for these studies points to the need for specialized text retrieval and classification technology, and consequently for appropriate test collections. In this paper, we introduce and analyze a new dataset for research and applications in the retrieval and processing of wordplay. Developed for the JOKER track at CLEF 2023, our annotated corpus extends and improves upon past English wordplay detection datasets in several ways. First, we introduce hundreds of additional positive examples of wordplay; second, we provide French translations for the examples; and third, we provide negative examples of non-wordplay with characteristics closely matching those of the positive examples. This last feature helps ensure that AI models learn to effectively distinguish wordplay from non-wordplay, and not simply texts differing in length, style, or vocabulary. Our test collection represents then a step towards wordplay-aware multilingual information retrieval.
Knowledge Tracing Model with Learning and Forgetting Behavior
Authors: Mingzhi Chen, Quanlong Guan, Yizhou He, Zhenyu He, Liangda Fang, Weiqi Luo
Venue: Conference CIKM 2022
Year: 2022

The Knowledge Tracing (KT) task aims to trace the changes of students' knowledge state in real time according to students' historical learning behavior, and predict students' future learning performance. The modern KT models have two problems. One is that these KT models can't reflect students' actual knowledge level. Most KT models only judge students' knowledge state based on their performance in exercises, and poor performance will lead to a decline in knowledge state. However, the essence of students' learning process is the process of acquiring knowledge, which is also a manifestation of learning behavior. Even if they answer the exercises incorrectly, they will still gain knowledge. The other problem is that many KT models don't pay enough attention to the impact of students' forgetting behavior on the knowledge state in the learning process. In fact, learning and forgetting behavior run through students' learning process, and their effects on students' knowledge state shouldn't be ignored. In this paper, based on educational psychology theory, we propose a knowledge tracing model with learning and forgetting behavior (LFBKT). LFBKT comprehensively considers the factors that affect learning and forgetting behavior to build the knowledge acquisition layer, knowledge absorption layer and knowledge forgetting layer. In addition, LFBKT introduces difficulty information to enrich the information of the exercise itself, while taking into account other answering performances besides the answer. Experimental results on two public datasets show that LFBKT can better trace students' knowledge state and outperforms existing models in terms of ACC and AUC.
Cognitive diagnosis focusing on knowledge concepts
Authors: Sheng Li, Quanlong Guan, Liangda Fang, Fang Xiao, Zhenyu He, Yizhou He, Weiqi Luo
Venue: Conference CIKM 2022
Year: 2022

Cognitive diagnosis is a crucial task in the field of educational measurement and psychology, which aims to diagnose the strengths and weaknesses of participants. Existing cognitive diagnosis methods only consider which of knowledge concepts are involved in the knowledge components of exercises, but ignore the fact that different knowledge concepts have different effects on practice scores in actual learning situations. Therefore, researchers need to reshape the learning scene by combining the multi-factor relationships between knowledge components. In this paper, in order to more comprehensively simulate the interaction between students and exercises, we developed a neural network-based CDMFKC model for cognitive diagnosis. Our method not only captures the nonlinear interaction between exercise characteristics, student performance, and their mastery of each knowledge concept, but also further considers the impact of knowledge concepts by designing the difficulty and discrimination of knowledge concepts, and uses multiple neural layers to model their interaction so as to obtain accurate and interpretable diagnostic results. In addition, we propose an improved CDMFKC model with guessing parameter and slipping parameter designed by knowledge concept proficiency and student proficiency vectors. We validate the performance of these two diagnostic models on six real datasets. The experimental results show that the two models have better effects in the aspects of accuracy, rationality and interpretability.
Question difficulty consistent knowledge tracing
Authors: Guimei Liu, Huijing Zhan, Jung-Jae Kim
Venue: Conference WWW 2024
Year: 2024

Knowledge tracing aims to estimate knowledge states of students over a set of skills based on students' past learning activities. Deep learning based knowledge tracing models show superior performance to traditional knowledge tracing approaches. Early works like DKT use skill IDs and student responses only. Recent works also incorporate questions IDs into their models and achieve much improved performance in the next question correctness prediction task. However, predictions made by these models are thus on specific questions, and it is not straightforward to translate them to estimation of students' knowledge states over skills. In this paper, we propose to replace question IDs with question difficulty levels in deep knowledge tracing models. The predictions made by our model can be more readily translated to students' knowledge states over skills. Furthermore, by using question difficulty levels to replace question IDs, we can also alleviate the cold-start problem in knowledge tracing as online learning platforms are updated frequently with new questions. We further use two techniques to smooth the predicted scores. One is to combine embeddings of nearby difficulty levels using the Hann function. The other is to constrain the predicted probabilities to be consistent with question difficulties by imposing a penalty if they are not consistent. We conduct extensive experiments to study the performance of the proposed model. Our experimental results show that our model outperforms the state-of-the-art knowledge tracing models in terms of both accuracy and consistency with question difficulty levels.
Real-time emotion pre-recognition in conversations with contrastive multi-modal dialogue pre-training
Authors: Xincheng Ju, Dong Zhang, Suyang Zhu, Junhui Li, Shoushan Li, Guodong Zhou
Venue: Conference CIKM 2023
Year: 2023

This paper presents our pioneering effort in addressing a new and realistic scenario in multi-modal dialogue systems called Multi-modal Real-time Emotion Pre-recognition in Conversations (MREPC). The objective is to predict the emotion of a forthcoming target utterance that is highly likely to occur. We believe that this task can enhance the dialogue system's understanding of the interlocutor's state of mind, enabling it to prepare an appropriate response in advance. However, addressing MREPC poses the following challenges:1) Previous studies on emotion elicitation typically focus on textual modality and perform sentiment forecasting within a fixed contextual scenario. 2) Previous studies on multi-modal emotion recognition aim to predict the emotion of existing utterances, making it difficult to extend these approaches to MREPC due to the absence of the target utterance. To tackle these challenges, we construct two benchmark multi-modal datasets for MREPC and propose a task-specific multi-modal contrastive pre-training approach. This approach leverages large-scale unlabeled multi-modal dialogues to facilitate emotion pre-recognition for potential utterances of specific target speakers. Through detailed experiments and extensive analysis, we demonstrate that our proposed multi-modal contrastive pre-training architecture effectively enhances the performance of multi-modal real-time emotion pre-recognition in conversations.
Fake news detection via multi-scale semantic alignment and cross-modal attention
Authors: Jiandong Wang, Hongguang Zhang, Chun Liu, Xiongjun Yang
Venue: Conference SIGIR 2024
Year: 2024

The effective utilization of multimodal information is crucial for the task of fake news detection. The primary challenge lies in reducing the semantic distance between the news evidences of different modalities and accurately aligning them in the latent space. It has been observed that the relevance of news image content varies across coarse-to-fine scales, depending on the object and its class label. For instance, the visual evidence to justify a news item sometimes may be a very local detail such as the tiny changes of facial expression and limb position while sometimes may be the global composition. Consequently, how to align news text with image at the most discriminative scale significantly impacts detection performance. However, very few studies have addressed this issue in fake news detection. In this paper, we delve deeper into this issue and propose a simple yet effective Multi-scale Semantic Alignment and Cross-modal Attention (MSACA) network. Specifically, we construct hierarchical multi-scale images for each news data, enhance the semantic consistency between text and images in the latent space, and employ an attention module to select the deterministic embeddings in an end-to-end manner. Extensive experiments on two real-world benchmarks demonstrate the superior performance of our proposed MSACA network.
Dynamic gaussian embedding of authors
Authors: Antoine Gourru, Julien Velcin, Christophe Gravier, Julien Jacques
Venue: Conference WWW 2022
Year: 2022

Authors publish documents in a dynamic manner. Their topic of interest and writing style might shift over time. Tasks such as author classification, author identification or link prediction are difficult to solve in such complex data settings. We propose a new representation learning model, DGEA (for Dynamic Gaussian Embedding of Authors), that is more suited to solve these tasks by capturing this temporal evolution. We formulate a general embedding framework: author representation at time t is a Gaussian distribution that leverages pre-trained document vectors, and that depends on the publications observed until t. The representations should retain some form of multi-topic information and temporal smoothness. We propose two models that fit into this framework. The first one, K-DGEA, uses a first order Markov model optimized with an Expectation Maximization Algorithm with Kalman Equations. The second, R-DGEA, makes use of a Recurrent Neural Network to model the time dependence. We evaluate our method on several quantitative tasks: author identification, classification, and co-authorship prediction, on two datasets written in English. In addition, our model is language agnostic since it only requires pre-trained document embeddings. It outperforms existing baselines by up to 18% on an author classification task on a news articles dataset.
Knowledge enhanced search result diversification
Authors: Zhan Su, Zhicheng Dou, Yutao Zhu 0001, Ji-Rong Wen
Venue: Conference KDD 2022
Year: 2022

Search result diversification focuses on reducing redundancy and improving subtopic richness in the results for a given query. Most existing approaches measure document diversity mainly based on text or pre-trained representations. However, some underlying relationships between the query and documents are difficult for the model to capture only from the content. Given that the knowledge base can offer well-defined entities and explicit relationships between entities, we exploit knowledge to model the relationship between documents and the query and propose a knowledge-enhanced search result diversification approach KEDIV. Concretely, we build a query-specific relation graph to model the complicated query-document relationship from an entity view. Then a graph neural network and node weight adjust algorithm are applied to the relation graph to obtain context-aware entity representations and document representations at each selection step. The diversity features are derived from the updated node representations of the relation graph. In this way, we can take advantage of entities' abundant information to model document's diversity in search result diversification. Experimental results on commonly used datasets show that our proposed approach can outperform the state-of-the-art methods.
Exploring high-order user preference with knowledge graph for recommendation
Authors: Caijun Xu, Fuwei Zhang, Zhao Zhang, Fuzhen Zhuang, Rui Liu
Venue: Conference CIKM 2024
Year: 2024

Knowledge Graph (KG) has proven its effectiveness in recommendation systems. Recent knowledge-aware recommendation methods, which utilize graph neural networks and contrastive learning, underestimate two issues: 1) The neglect of modeling the latent relationships between users and entities; 2) The insufficiency of traditional cross-view contrastive learning whose domain is incapable of covering all nodes in a graph. To address these issues, we propose a novel model named Knowledge-aware User Preference Network (KUPN). Specifically, KUPN first constructs the relational preference view containing a new graph named User Preference Graph (UPG) to model the potential relationships between users and entities. Then, we adopt a novel attentive information aggregation to learn the UPG. In addition, we obtain semantic information of users and entities from collaborative knowledge view which consists of KG and Interaction Graph (IG) as supplementary. Finally, we apply a cross-view contrastive learning for complete domains between dynamic relational preference view and collaborative knowledge view. Extensive experiments on three real-world datasets demonstrate the superiority of KUPN against the state-of-the-art methods.
Can Generative LLMs Create Query Variants for Test Collections? An Exploratory Study
Authors: Marwah Alaofi, Luke Gallagher, Mark Sanderson, Falk Scholer, Paul Thomas
Venue: Conference SIGIR 2023
Year: 2023

This paper explores the utility of a Large Language Model (LLM) to automatically generate queries and query variants from a description of an information need. Given a set of information needs described as backstories, we explore how similar the queries generated by the LLM are to those generated by humans. We quantify the similarity using different metrics and examine how the use of each set would contribute to document pooling when building test collections. Our results show potential in using LLMs to generate query variants. While they may not fully capture the wide variety of human-generated variants, they generate similar sets of relevant documents, reaching up to 71.1% overlap at a pool depth of 100.
Scale and object aware image retargeting for thumbnail browsing
Authors: Jin Sun 0011, Haibin Ling
Venue: Conference ICCV 2011
Year: 2011

Many image retargeting algorithms, despite aesthetically carving images smaller, pay limited attention to image browsing tasks where tiny thumbnails are presented. When applying traditional retargeting methods for generating thumbnails, several important issues frequently arise, including thumbnail scales, object completeness and local structure smoothness. To address these issues, we propose a novel image retargeting algorithm, Scale and Object Aware Retargeting (SOAR), which has four components: (1) a scale dependent saliency map to integrate size information of thumbnails, (2) objectness (Alexe et al. 2010) for preserving object completeness, (3) a cyclic seam carving algorithm to guide continuous retarget warping, and (4) a thin-plate-spline (TPS) retarget warping algorithm that champions local structure smoothness. The effectiveness of the proposed algorithm is evaluated both quantitatively and qualitatively. The quantitative evaluation is conducted through an image browsing user study to measure the effectiveness of different thumbnail generating algorithms, followed by the ANOVA analysis. The qualitative study is performed on the RetargetMe benchmark dataset. In both studies, SOAR generates very promising performance, in comparison with state-of-the-art retargeting algorithms.
Automatic salient object extraction with contextual cue
Authors: Le Wang 0003, Jianru Xue, Nanning Zheng 0001, Gang Hua 0001
Venue: Conference ICCV 2011
Year: 2011

We present a method for automatically extracting salient object from a single image, which is cast in an energy minimization framework. Unlike most previous methods that only leverage appearance cues, we employ an auto-context cue as a complementary data term. Benefitting from a generic saliency model for bootstrapping, the segmentation of the salient object and the learning of the auto-context model are iteratively performed without any user intervention. Upon convergence, we obtain not only a clear separation of the salient object, but also an auto-context classifier which can be used to recognize the same type of object in other images. Our experiments on four benchmarks demonstrated the efficacy of the added contextual cue. It is shown that our method compares favorably with the state-of-the-art, some of which even embraced user interactions.
Cookpad Image Dataset: An Image Collection As Infrastructure for Food Research
Authors: Jun Harashima, Yuichiro Someya, Yohei Kikuta
Venue: Conference SIGIR 2017
Year: 2017

In food-related services, image information is as important as text information for users. For example, in recipe search services, users find recipes based not only on text but also images. To promote studies on food images, many datasets have recently been published. However, they have the following three limitations: most of the datasets include only thousands of images, they only take account of images after cooking not during the cooking process, and the images are not linked to any recipes. In this study, we construct the Cookpad Image Dataset, a novel collection of food images taken from Cookpad, the largest recipe search service in the world. The dataset includes more than 1.64 million images after cooking, and it is the largest among existing datasets. Additionally, it includes more than 3.10 million images taken during the cooking process. To the best of our knowledge, there are no datasets that include such images. Furthermore, the dataset is designed to link to an existing recipe corpus and thus, a variety of recipe texts, such as the title, description, ingredients, and process, is available for each image. In this paper, we described our dataset's features in detail and compared it with existing datasets.
Estimating camera response functions using probabilistic intensity similarity
Authors: Jun Takamatsu, Yasuyuki Matsushita, Katsushi Ikeuchi
Venue: Conference CVPR 2008
Year: 2008

We propose a method for estimating camera response functions using a probabilistic intensity similarity measure. The similarity measure represents the likelihood of two intensity observations corresponding to the same scene radiance in the presence of noise. We show that the response function and the intensity similarity measure are strongly related. Our method requires several input images of a static scene taken from the same viewing position with fixed camera parameters. Noise causes pixel values at the same pixel coordinate to vary in these images, even though they measure the same scene radiance. We use these fluctuations to estimate the response function by maximizing the intensity similarity function for all pixels. Unlike prior noise-based estimation methods, our method requires only a small number of images, so it works with digital cameras as well as video cameras. Moreover, our method does not rely on any special image processing or statistical prior models. Real-world experiments using different cameras demonstrate the effectiveness of the technique.
Radiometric calibration using temporal irradiance mixtures
Authors: Bennett Wilburn, Hui Xu, Yasuyuki Matsushita
Venue: Conference CVPR 2008
Year: 2008

We propose a new method for sampling camera response functions: temporally mixing two uncalibrated irradiances within a single camera exposure. Calibration methods rely on some known relationship between irradiance at the camera image plane and measured pixel intensities. Prior approaches use a color checker chart with known reflectances, registered images with different exposure ratios, or even the irradiance distribution along edges in images. We show that temporally blending irradiances allows us to densely sample the camera response function with known relative irradiances. Our first method computes the camera response curve using temporal mixtures of two pixel intensities on an uncalibrated computer display. The second approach makes use of temporal irradiance mixtures caused by motion blur. Both methods require only one input image, although more images can be used for improved robustness to noise or to cover more of the response curve. We show that our methods compute accurate response functions for a variety of cameras.
Natural image denoising: Optimality and inherent bounds
Authors: Anat Levin, Boaz Nadler
Venue: Conference CVPR 2011
Year: 2011

The goal of natural image denoising is to estimate a clean version of a given noisy image, utilizing prior knowledge on the statistics of natural images. The problem has been studied intensively with considerable progress made in recent years. However, it seems that image denoising algorithms are starting to converge and recent algorithms improve over previous ones by only fractional dB values. It is thus important to understand how much more can we still improve natural image denoising algorithms and what are the inherent limits imposed by the actual statistics of the data. The challenge in evaluating such limits is that constructing proper models of natural image statistics is a long standing and yet unsolved problem. To overcome the absence of accurate image priors, this paper takes a non parametric approach and represents the distribution of natural images using a huge set of 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">10</sup> patches. We then derive a simple statistical measure which provides a lower bound on the optimal Bayesian minimum mean square error (MMSE). This imposes a limit on the best possible results of denoising algorithms which utilize a fixed support around a denoised pixel and a generic natural image prior. Our findings suggest that for small windows, state of the art denoising algorithms are approaching optimality and cannot be further improved beyond ~ 0.1dB values.
Fast pixel/part selection with sparse eigenvectors
Authors: Bernard Moghaddam, Yair Weiss, Shai Avidan
Venue: Conference ICCV 2007
Year: 2007

We extend the "Sparse LDA" algorithm of [7] with new sparsity bounds on 2-class separability and efficient partitioned matrix inverse techniques leading to 1000-fold speed-ups. This mitigates the 0(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> ) scaling that has limited this algorithm's applicability to vision problems and also prioritizes the less-myopic backward elimination stage by making it faster than forward selection. Experiments include "sparse eigenfaces" and gender classification on FERET data as well as pixel/part selection for OCR on MNIST data using Bayesian (GP) classification. Sparse- LDA is an attractive alternative to the more demanding Automatic Relevance Determination. State-of-the-art recognition is obtained while discarding the majority of pixels in all experiments. Our sparse models also show a better fit to data in terms of the "evidence" or marginal likelihood.