OpenRaiser/Intern-Atlas huggingface dataset:
Training data attribution: Was your model secretly trained on data created by mine?
Authors: Likun Zhang, Hao Wu, Lingcui Zhang, Fengyuan Xu, Jin Cao, Fenghua Li, Ben Niu
Venue: Conference KDD 2025
Year: 2025

The emergence of text-to-image models has recently sparked significant interest, but the attendant is a looming shadow of potential infringement by violating user terms. Specifically, an adversary may exploit data created by a commercial model to train their own without proper authorization. To address such risk, it is crucial to investigate the attribution of a suspicious model's training data by determining whether its training data originates, wholly or partially, from a specific source model. To trace the generated data, existing methods need to apply additional watermarks during either the training or inference phases of the source model. However, these methods are impractical for pre-trained models that have been released, especially when model owners lack security expertise. To tackle this challenge, we propose an injection-free training data attribution method for text-to-image models. It can identify whether a model's training data stems from a certain source model without adding additional watermarks on the source model. The rationale of our method lies in the inherent memorization characteristic of text-to-image models. The memorization of training data is inherited through the data generated by the source model to the model trained on that data, making the source model and the infringing model exhibit consistent behaviors on specific samples. Therefore, from instance-level, we develop detection-based and generation-based strategies to uncover these distinct samples and using them as inherent watermarks to verify if a suspicious model originates from the source model. Besides, we also propose a statistical-level attribution method, utilizing the shadow model technique to train an attribution discriminator. Experiments demonstrate that the attribution accuracy and AUC scores of our methods are over 80% even when the infringing model only uses a small proportion of generated data.
Model-agnostic random weighting for out-of-distribution generalization
Authors: Yue He 0001, Pengfei Tian, Renzhe Xu, Xinwei Shen 0002, Xingxuan Zhang, Peng Cui 0001
Venue: Conference KDD 2024
Year: 2024

Despite the encouraging successes in numerous applications, machine learning methods grounded on the i.i.d. assumption often experience performance deterioration when confronted with the distribution shift between training and test data. This challenge has instigated recent research endeavors focusing on out-of-distribution (OOD) generalization. A particularly pervasive and intricate OOD problem is to enhance the model's generalization ability by training it on samples drawn from a single environment. In response to the problem, we propose a simple model-agnostic method tailored for a practical OOD scenario in this paper. Our approach centers on pursuing robust weighted empirical risks, utilizing randomly shifted training distributions derived through a specific sample-based weighting strategy. Furthermore, we theoretically establish that the expected risk of the shifted training distribution can bound the expected risk of the test distribution. This theoretical foundation ensures the improved prediction performance of our method when employed in uncertain test distributions. Extensive experiments conducted on diverse real-world datasets affirm the effectiveness of our method, highlighting its potential to address the distribution shifts in machine learning applications.
Novelty detection from an ego-centric perspective
Authors: Omid Aghazadeh, Josephine Sullivan, Stefan Carlsson
Venue: Conference CVPR 2011
Year: 2011

This paper demonstrates a system for the automatic extraction of novelty in images captured from a small video camera attached to a subject's chest, replicating his visual perspective, while performing activities which are repeated daily. Novelty is detected when a (sub)sequence cannot be registered to previously stored sequences captured while performing the same daily activity. Sequence registration is performed by measuring appearance and geometric similarity of individual frames and exploiting the invariant temporal order of the activity. Experimental results demonstrate that this is a robust way to detect novelties induced by variations in the wearer's ego-motion such as stopping and talking to a person. This is an essentially new and generic way of automatically extracting information of interest to the camera wearer and can be used as input to a system for life logging or memory support.
QUILT: Implementing a Large Scale Cross-language Text Retrieval System
Authors: Mark W. Davis, William C. Ogden
Venue: Conference SIGIR 1997
Year: 1997

AhtractQ~T (QWXY Uaa Interfacewith Light Tmnaladon8)k a prototype implementadon of a completecmas-langusge textretrieval 8ystemtbat takeaFkgli8hqude3snd pmduceaw811glo8a translations of Spani8hdocumenla.The ayatcmindexesthe Spm
Text clustering with extended user feedback
Authors: Yifen Huang, Tom M. Mitchell
Venue: Conference SIGIR 2006
Year: 2006

Text clustering is most commonly treated as a fully automated task without user feedback. However, a variety of researchers have explored mixed-initiative clustering methods which allow a user to interact with and advise the clustering algorithm. This mixed-initiative approach is especially attractive for text clustering tasks where the user is trying to organize a corpus of documents into clusters for some particular purpose (e.g., clustering their email into folders that reflect various activities in which they are involved). This paper introduces a new approach to mixed-initiative clustering that handles several natural types of user feedback. We first introduce a new probabilistic generative model for text clustering (the SpeClustering model) and show that it outperforms the commonly used mixture of multinomials clustering model, even when used in fully autonomous mode with no user input. We then describe how to incorporate four distinct types of user feedback into the clustering algorithm, and provide experimental evidence showing substantial improvements in text clustering when this user feedback is incorporated.
Deconstructing nuggets: the stability and reliability of complex question answering evaluation
Authors: Jimmy Lin, Pengyi Zhang
Venue: Conference SIGIR 2007
Year: 2007

A methodology based on "information nuggets" has recently emerged as the de facto standard by which answers to complex questions are evaluated. After several implementations in the TREC question answering tracks, the community has gained a better understanding of its many characteristics. This paper focuses on one particular aspect of the evaluation: the human assignment of nuggets to answer strings, which serves as the basis of the F-score computation. As a byproduct of the TREC 2006 ciQA task, identical answer strings were independently evaluated twice, which allowed us to assess the consistency of human judgments. Based on these results, we explored simulations of assessor behavior that provide a method to quantify scoring variations. Understanding these variations in turn lets researchers be more confident in their comparisons of systems.
Improved algorithms for topic distillation in a hyperlinked environment
Authors: Krishna Bharat, Monika Rauch Henzinger
Venue: Conference SIGIR 1998
Year: 1998

This paper addresses the problem of topic distillation on the World Wide Web, namely, given a typical user query to find quality documents related to the query topic. Connectivity analysis has been shown to be useful in identifying high quality pages within a topic specific graph of hyperlinked documents. The essence of our approach is to augment a previous connectivity analysis based algorithm with content analysis. We identify three problems with the existing approach and devise algorithms to tackle them. The results of a user evaluation are reported that show an improvement of precision at 10 documents by at least 45% over pure connectivity analysis.
To tag or not to tag - harvesting adjacent metadata in large-scale tagging systems
Authors: Adriana Budura, Sebastian Michel, Philippe Cudré-Mauroux, Karl Aberer
Venue: Conference SIGIR 2008
Year: 2008

We present HAMLET, a suite of principles, scoring models and algorithms to automatically propagate metadata along edges in a document neighborhood. As a showcase scenario we consider tag prediction in community-based Web 2.0 tagging applications. Experiments using real-world data demonstrate the viability of our approach in large-scale environments where tags are scarce. To the best of our knowledge, HAMLET is the first system to promote an efficient and precise reuse of shared metadata in highly dynamic, large-scale Web 2.0 tagging systems.
P-tag: large scale automatic generation of personalized annotation tags for the web
Authors: Paul-Alexandru Chirita, Stefania Costache, Wolfgang Nejdl, Siegfried Handschuh
Venue: Conference WWW 2007
Year: 2007

The success of the Semantic Web depends on the availability of Web pages annotated with metadata. Free form metadata or tags, as used in social bookmarking and folksonomies, have become more and more popular and successful. Such tags are relevant keywords associated with or assigned to a piece of information (e.g., a Web page), describing the item and enabling keyword-based classification. In this paper we propose P-TAG, a method which automatically generates personalized tags for Web pages. Upon browsing a Web page, P-TAG produces keywords relevant both to its textual content, but also to the data residing on the surfer's Desktop, thus expressing a personalized viewpoint. Empirical evaluations with several algorithms pursuing this approach showed very promising results. We are therefore very confident that such a user oriented automatic tagging approach can provide large scale personalized metadata annotations as an important step towards realizing the Semantic Web.
Predicting the ratings of multimedia items for making personalized recommendations
Authors: Rani Qumsiyeh, Yiu-Kai Ng
Venue: Conference SIGIR 2012
Year: 2012

Existing multimedia recommenders suggest a specific type of multimedia items rather than items of different types personalized for a user based on his/her preference. Assume that a user is interested in a particular family movie, it is appealing if a multimedia recommendation system can suggest other movies, music, books, and paintings closely related to the movie. We propose a comprehensive, personalized multimedia recommendation system, denoted MudRecS, which makes recommendations on movies, music, books, and paintings similar in content to other movies, music, books, and/or paintings that a MudRecS user is interested in. MudRecS does not rely on users' access patterns/histories, connection information extracted from social networking sites, collaborated filtering methods, or user personal attributes (such as gender and age) to perform the recommendation task. It simply considers the users' ratings, genres, role players (authors or artists), and reviews of different multimedia items, which are abundant and easy to find on the Web. MudRecS predicts the ratings of multimedia items that match the interests of a user to make recommendations. The performance ofMudRecS has been compared with current state-of-the-art multimedia recommenders using various multimedia datasets, and the experimental results show that MudRecS significantly outperforms other systems in accurately predicting the ratings of multimedia items to be recommended.
A comparison of sentence retrieval techniques
Authors: Niranjan Balasubramanian, James Allan, W. Bruce Croft
Venue: Conference SIGIR 2007
Year: 2007

Identifying redundant information in sentences is useful for several applications such as summarization, document provenance, detecting text reuse and novelty detection. The task of identifying redundant information in sentences is defined as follows: Given a query sentence the task is to retrieve sentences from a given collection that express all or some subset of the information present in the query sentence. Sentence retrieval techniques rank sentences based on some measure of their similarity to a query. The effectiveness of such techniques depends on the similarity measure used to rank sentences. An effective retrieval model should be able to handle low word overlap between query and candidate sentences and go beyond just word overlap. Simple language modeling techniques like query likelihood retrieval have outperformed TF-IDF and word overlap based methods for ranking sentences. In this paper, we compare the performance of sentence retrieval using different language modeling techniques for the problem of identifying redundant information.
Transforming patents into prior-art queries
Authors: Xiaobing Xue, W. Bruce Croft
Venue: Conference SIGIR 2009
Year: 2009

Searching for prior-art patents is an essential step for the patent examiner to validate or invalidate a patent application. In this paper, we consider the whole patent as the query, which reduces the burden on the user, and also makes many more potential search features available. We explore how to automatically transform the query patent into an effective search query, especially focusing on the effect of different patent fields. Experiments show that the background summary of a patent is the most useful source of terms for generating a query, even though most previous work used the patent claims.
Mining the web to create minority language corpora
Authors: Rayid Ghani, Rosie Jones, Dunja Mladenic
Venue: Conference CIKM 2001
Year: 2001

The Web is a valuable source of language specific resources but the process of collecting, organizing and utilizing these resources is difficult. We describe CorpusBuilder, an approach for automatically generating Web-search queries for collecting documents in a minority language. It differs from pseudo-relevance feedback in that retrieved documents are labeled by an automatic language classifier as relevant or irrelevant, and this feedback is used to generate new queries. We experiment with various query-generation methods and query-lengths to find inclusion/exclusion terms that are helpful for retrieving documents in the target language and find that using odds-ratio scores calculated over the documents acquired so far was one of the most consistently accurate query-generation methods. We also describe experiments using a handful of words elicited from a user instead of initial documents and show that the methods perform similarly. Experiments applying the same approach to multiple languages are also presented showing that our approach generalizes to a variety of languages.
Who watches (and shares) what on youtube? and when?: using twitter to understand youtube viewership
Authors: Adiya Abisheva, Venkata Rama Kiran Garimella, David García, Ingmar Weber
Venue: Conference WSDM 2014
Year: 2014

By combining multiple social media datasets, it is possible to gain insight into each dataset that goes beyond what could be obtained with either individually. In this paper we combine user-centric data from Twitter with video-centric data from YouTube to build a rich picture of who watches and shares what on YouTube. We study 87K Twitter users, 5.6 million YouTube videos and 15 million video sharing events from user-, video- and sharing-event-centric perspectives. We show that features of Twitter users correlate with YouTube features and sharing-related features. For example, urban users are quicker to share than rural users. We find a superlinear relationship between initial Twitter shares and the final amounts of views. We discover that Twitter activity metrics play more role in video popularity than mere amount of followers. We also reveal the existence of correlated behavior concerning the time between video creation and sharing within certain timescales, showing the time onset for a coherent response, and the time limit after which collective responses are extremely unlikely. Response times depend on the category of the video, suggesting Twitter video sharing is highly dependent on the video content. To the best of our knowledge, this is the first large-scale study combining YouTube and Twitter data, and it reveals novel, detailed insights into who watches (and shares) what on YouTube, and when.
Sopra: a new social personalized ranking function for improving web search
Authors: Mohamed Reda Bouadjenek, Hakim Hacid, Mokrane Bouzeghoub
Venue: Conference SIGIR 2013
Year: 2013

We present in this paper a contribution to IR modeling by proposing a new ranking function called SoPRa that considers the social dimension of the Web. This social dimension is any social information that surrounds documents along with the social context of users. Currently, our approach relies on folksonomies for extracting these social contexts, but it can be extended to use any social meta-data, e.g. comments, ratings, tweets, etc. The evaluation performed on our approach shows its benefits for personalized search.
Personalized social query expansion using social bookmarking systems
Authors: Mohamed Reda Bouadjenek, Hakim Hacid, Mokrane Bouzeghoub, Johann Daigremont
Venue: Conference SIGIR 2011
Year: 2011

We propose a new approach for social and personalized query expansion using social structures in the Web 2.0. While focusing on social tagging systems, the proposed approach considers (i) the semantic similarity between tags composing a query, (ii) a social proximity between the query and the user profile, and (iii) on the fly, a strategy for expanding user queries. The proposed approach has been evaluated using a large dataset crawled from del.icio.us.
Truth discovery in data streams: A single-pass probabilistic approach
Authors: Zhou Zhao, James Cheng, Wilfred Ng
Venue: Conference CIKM 2014
Year: 2014

Truth discovery is a long-standing problem for assessing the validity of information from various data sources that may provide different and conflicting information. With the increasing prominence of data streams arising in a wide range of applications such as weather forecast and stock price prediction, effective techniques for truth discovery in data streams are demanded. However, existing work mainly focuses on truth discovery in the context of static databases, which is not applicable in applications involving streaming data. This motivates us to develop new techniques to tackle the problem of truth discovery in data streams.
Sentiment strength prediction using auxiliary features
Authors: Huijun Chen, Xin Li, Yanghui Rao, Haoran Xie, Fu Lee Wang, Tak-Lam Wong
Venue: Conference WWW 2017
Year: 2017

With an increasingly large amount of sentimental information embedded in online documents, sentiment analysis is quite valuable to product recommendation, opinion summarization, and so forth. Different from most works on identifying documents' qualitative affective information, this research focuses on the measurement of users' intensity over each sentimental category. Affect indicates positive or negative sentiment, while cognition includes certainty and tentative. Thus, our research can help bridge the cognitive and affective gaps between users and documents. The contributions of this study are twofold: (i) we proposed a neural network-based framework to sentiment strength prediction by convolving hybrid vectors, and (ii) we considered words jointly with a set of linguistic features for enhancing model robustness and adaptiveness. By exploiting the auxiliary features of sentiments from the corpus, the proposed model did not rely on well-established lexicons, and showed its robustness over sparse words. Experiments on six corpora validated the effectiveness of our sentiment strength prediction method.
Web document encoding for structure-aware keyphrase extraction
Authors: Jihyuk Kim, Young-In Song, Seung-won Hwang
Venue: Conference SIGIR 2021
Year: 2021

We study keyphrase extraction (KPE) from Web documents. Our key contribution is encoding Web documents to leverage structure, such as title or anchors, by building a graph of words representing both (a) position-based proximity and (b) structural relations. We evaluate KPE performance on real-world search engine NAVER and human-annotated KPE benchmarks, and ours outperforms state-of-the-arts in both tasks.
Visual question rewriting for increasing response rate
Authors: Jiayi Wei, Xilian Li, Yi Zhang 0001, Xin Eric Wang
Venue: Conference SIGIR 2021
Year: 2021

When a human asks questions online, or when a conversational virtual agent asks human questions, questions triggering emotions or with details might more likely to get responses or answers. we explore how to automatically rewrite natural language questions to improve the response rate from people. In particular, a new task of Visual Question Rewriting(VQR) task is introduced to explore how visual information can be used to improve the new questions. A data set containing around 4K bland questions, attractive questions and images triples is collected. We developed some baseline sequence to sequence models and more advanced transformer based models, which take a bland question and a related image as input and output a rewritten question that is expected to be more attractive. Offline experiments and mechanical Turk based evaluations show that it is possible to rewrite bland questions in a more detailed and attractive way to increase the response rate, and images can be helpful.
Learning early exit strategies for additive ranking ensembles
Authors: Francesco Busolin, Claudio Lucchese, Franco Maria Nardini, Salvatore Orlando 0001, Raffaele Perego 0001, Salvatore Trani
Venue: Conference SIGIR 2021
Year: 2021

Modern search engine ranking pipelines are commonly based on large machine-learned ensembles of regression trees. We propose LEAR, a novel - learned - technique aimed to reduce the average number of trees traversed by documents to accumulate the scores, thus reducing the overall query response time. LEAR exploits a classifier that predicts whether a document can early exit the ensemble because it is unlikely to be ranked among the final top-k results. The early exit decision occurs at a sentinel point, i.e., after having evaluated a limited number of trees, and the partial scores are exploited to filter out non-promising documents. We evaluate LEAR by deploying it in a production-like setting, adopting a state-of-the-art algorithm for ensembles traversal. We provide a comprehensive experimental evaluation on two public datasets. The experiments show that LEAR has a significant impact on the efficiency of the query processing without hindering its ranking quality. In detail, on a first dataset, LEAR is able to achieve a speedup of 3x without any loss in [email protected], while on a second dataset the speedup is larger than 5x with a negligible [email protected] loss (< 0.05%).
NTCIR Lifelog: The first test collection for lifelog research
Authors: Cathal Gurrin, Hideo Joho, Frank Hopfgartner, Liting Zhou, Rami Albatal
Venue: Conference SIGIR 2016
Year: 2016

Test collections have a long history of supporting repeatable and comparable evaluation in Information Retrieval (IR). However, thus far, no shared test collection exists for IR systems that are designed to index and retrieve multimodal lifelog data. In this paper we introduce the first test collection for personal lifelog data, which has been employed for the NTCIR12-Lifelog task. In this paper, the requirements for the test collection are motivated, the process of creating the test collection is described, along with an overview of the test collection. Finally suggestions are given for possible applications of the test collection.