OpenRaiser/Intern-Atlas huggingface dataset:
Linguistically-enriched and context-awarezero-shot slot filling
Authors: A. B. Siddique, Fuad T. Jamour, Vagelis Hristidis
Venue: Conference WWW 2021
Year: 2021

Slot filling is identifying contiguous spans of words in an utterance that correspond to certain parameters (i.e., slots) of a user request/query. Slot filling is one of the most important challenges in modern task-oriented dialog systems. Supervised approaches have proven effective at tackling this challenge, but they need a significant amount of labeled training data in a given domain. However, new domains (i.e., unseen in training) may emerge after deployment. Thus, it is imperative that these models seamlessly adapt and fill slots from both seen and unseen domains – unseen domains contain unseen slot types with no training data, and even seen slots in unseen domains are typically presented in different contexts. This setting is commonly referred to as zero-shot slot filling. Little work has focused on this setting, with limited experimental evaluation. Existing models that mainly rely on context-independent embedding-based similarity measures fail to detect slot values in unseen domains or do so only partially. We propose a new zero-shot slot filling neural model, , which works in three steps. Step one acquires domain-oblivious, context-aware representations of utterance words by exploiting (a) linguistic features such as part-of-speech tags; (b) named entity recognition cues; and (c) contextual embeddings from pre-trained language models. Step two fine-tunes these rich representations and produces slot-independent tags for each word. Step three exploits generalizable context-aware utterance-slot similarity features at the word level, uses slot-independent tags, and contextualizes them to produce slot-specific predictions for each word. Our thorough evaluation on four diverse public datasets demonstrates that our approach consistently outperforms state-of-the-art models by 17.52%, 22.15%, 17.42%, and 17.95% on average for unseen domains on SNIPS, ATIS, MultiWOZ, and SGD datasets, respectively.
Mining Persistent Activity in Continually Evolving Networks
Authors: Caleb Belth, Xinyi Zheng, Danai Koutra
Venue: Conference KDD 2020
Year: 2020

Frequent pattern mining is a key area of study that gives insights into the structure and dynamics of evolving networks, such as social or road networks. However, not only does a network evolve, but often the way that it evolves, itself evolves. Thus, knowing, in addition to patterns' frequencies, for how long and how regularly they have occurred---i.e., their persistence---can add to our understanding of evolving networks. In this work, we propose the problem of mining activity that persists through time in continually evolving networks---i.e., activity that repeatedly and consistently occurs. We extend the notion of temporal motifs to capture activity among specific nodes, in what we call activity snippets, which are small sequences of edge-updates that reoccur. We propose axioms and properties that a measure of persistence should satisfy, and develop such a persistence measure. We also propose PENminer, an efficient framework for mining activity snippets' Persistence in Evolving Networks, and design both offline and streaming algorithms. We apply PENminer to numerous real, large-scale evolving networks and edge streams, and find activity that is surprisingly regular over a long period of time, but too infrequent to be discovered by aggregate count alone, and bursts of activity exposed by their lack of persistence. Our findings with PENminer include neighborhoods in NYC where taxi traffic persisted through Hurricane Sandy, the opening of new bike-stations, characteristics of social network users, and more. Moreover, we use PENminer towards identifying anomalies in multiple networks, outperforming baselines at identifying subtle anomalies by 9.8-48% in AUC.
Cascade-based echo chamber detection
Authors: Marco Minici, Federico Cinus, Corrado Monti, Francesco Bonchi, Giuseppe Manco
Venue: Conference CIKM 2022
Year: 2022

Despite echo chambers in social media have been under considerable scrutiny, general models for their detection and analysis are missing. In this work, we aim to fill this gap by proposing a probabilistic generative model that explains social media footprints -- i.e., social network structure and propagations of information -- through a set of latent communities, characterized by a degree of echo-chamber behavior and by an opinion polarity. Specifically, echo chambers are modeled as communities that are permeable to pieces of information with similar ideological polarity, and impermeable to information of opposed leaning: this allows discriminating echo chambers from communities that lack a clear ideological alignment. To learn the model parameters we propose a scalable, stochastic adaptation of the Generalized Expectation Maximization algorithm, that optimizes the joint likelihood of observing social connections and information propagation. Experiments on synthetic data show that our algorithm is able to correctly reconstruct ground-truth latent communities with their degree of echo-chamber behavior and opinion polarity. Experiments on real-world data about polarized social and political debates, such as the Brexit referendum or the COVID-19 vaccine campaign, confirm the effectiveness of our proposal in detecting echo chambers. Finally, we show how our model can improve accuracy in auxiliary predictive tasks, such as stance detection and prediction of future propagations.
A joint optimization of incrementality and revenue to satisfy both advertiser and publisher
Authors: Dmitry Pechyony, Rosie Jones, Xiaojing Li
Venue: Conference WWW 2013
Year: 2013

A long-standing goal in advertising is to reduce wasted costs due to advertising to people who are unlikely to buy, as well as to those who would make a purchase whether they saw an ad or not. The ideal audience for the advertiser are those incremental users who would buy if shown an ad, and would not buy, if not shown the ad. On the other hand, for publishers who are paid when the user clicks or buys, revenue may be maximized by showing ads to those users who are most likely to click or purchase. We show analytically and empirically that an optimization towards one metric might result in an inferior performance in the other one. We present a novel algorithm, called SLC, that performs a joint optimization towards both advertisers' and publishers' goals and provides superior results in both.
Real-time on-device troubleshooting recommendation for smartphones
Authors: Keiichi Ochiai, Kohei Senkawa, Naoki Yamamoto, Yuya Tanaka, Yusuke Fukazawa
Venue: Conference KDD 2019
Year: 2019

Billions of people are using smartphones everyday and they often face problems and troubles with both the hardware as well as the software. Such problems lead to frustrated users and low customer satisfaction. Developing an automatic machine learning-based solution that would detect that the user has a problem and would engage in troubleshooting has the potential to significantly improve customer satisfaction and retention. Here, we design and implement a system that based on the user's smartphone activity detects that the user has a problem and requires help. Our system automatically detects a user has a problem and then helps with the troubleshooting by recommending possible solutions to the identified problem. We train our system based on large-scale customer support center data and show that it can both detect that a user has a problem as well as predict the category of the problem (89.7% accuracy) and quickly provide a solution (in 10.4ms). Our system has been deployed in commercial service since January, 2019. Online evaluation result showed that machine learning based approach outperforms the existing method by approximately 30% regarding the user problem solving rate.
Bid Shading in The Brave New World of First-Price Auctions
Authors: Djordje Gligorijevic, Tian Zhou, Bharatbhushan Shetty, Brendan Kitts, Shengjun Pan, Junwei Pan, Aaron Flores
Venue: Conference CIKM 2020
Year: 2020

Online auctions play a central role in online advertising, and are one of the main reasons for the industry's scalability and growth. With great changes in how auctions are being organized, such as changing the second- to first-price auction type, advertisers and demand platforms are compelled to adapt to a new volatile environment. Bid shading is a known technique for preventing overpaying in auction systems that can help maintain the strategy equilibrium in first-price auctions, tackling one of its greatest drawbacks. In this study, we propose a machine learning approach of modeling optimal bid shading for non-censored online first-price ad auctions. We clearly motivate the approach and extensively evaluate it in both offline and online settings on a major demand side platform. The results demonstrate the superiority and robustness of the new approach as compared to the existing approaches across a range of performance metrics.
Analysing trends in computer science research: A preliminary study using the microsoft academic graph
Authors: Suhendry Effendy, Roland H. C. Yap
Venue: Conference WWW 2017
Year: 2017

Research in Computer Science (CS) evolves rapidly in a dynamic fashion. New research area may emerge and attract researchers, while older areas may have lesser interest from researchers. Studying how trends evolve in CS can be interesting from several dimensions. Furthermore, it can be used to craft research agendas. In this paper, we present trend analysis on research area in CS. We also look at citation trend analysis. Our analysis is performed using the Microsoft Academic Graph dataset. We propose the FoS score to measure the level of interest in any particular research area or topic. We apply the FoS score to investigate general publication trends, citation trends, evolution of research areas, and relation between research areas in CS.
Neural Unified Review Recommendation with Cross Attention
Authors: Hongtao Liu 0008, Wenjun Wang 0002, Hongyan Xu 0001, Qiyao Peng 0001, Pengfei Jiao
Venue: Conference SIGIR 2020
Year: 2020

There are two main paradigms to exploit review information for recommendation. One is to concatenate all reviews of a user/item into a long document, which may neglect the different usefulness of reviews. The other paradigm is review-level i.e., analyzing each review separately to learn user/item features. In fact, the two paradigms are complementary, and fusing them together has the potential to learn more comprehensive features of users/items. Hence, we propose a unified framework to jointly learn document- and review-level representations of users/items. We design a document encoder to learn document-level features of users/items. Then, we use a review encoder to learn representations of reviews from words, and a user/item encoder to learn review-level features of users/items. Besides, different reviews from the same user may have different importance for different target items due to different item characteristics. We propose a cross attention model for user representation learning whose query vector is the embedding of target item ID, and apply it to the above three encoders to select different informative words and reviews for different target items. Extensive experiments validate the effectiveness of our method.
Efficient long-term degradation profiling in time series for complex physical systems
Authors: Liudmila Ulanova, Tan Yan, Haifeng Chen, Guofei Jiang, Eamonn J. Keogh, Kai Zhang
Venue: Conference KDD 2015
Year: 2015

The long term operation of physical systems inevitably leads to their wearing out, and may cause degradations in performance or the unexpected failure of the entire system. To reduce the possibility of such unanticipated failures, the system must be monitored for tell-tale symptoms of degradation that are suggestive of imminent failure. In this work, we introduce a novel time series analysis technique that allows the decomposition of the time series into trend and fluctuation components, providing the monitoring software with actionable information about the changes of the system's behavior over time. We analyze the underlying problem and formulate it to a Quadratic Programming (QP) problem that can be solved with existing QP-solvers. However, when the profiling resolution is high, as generally required by real-world applications, such a decomposition becomes intractable to general QP-solvers. To speed up the problem solving, we further transform the problem and present a novel QP formulation, Non-negative QP, for the problem and demonstrate a tractable solution that bypasses the use of slow general QP-solvers. We demonstrate our ideas on both synthetic and real datasets, showing that our method allows us to accurately extract the degradation phenomenon of time series. We further demonstrate the generality of our ideas by applying them beyond classic machine prognostics to problems in identifying the influence of news events on currency exchange rates and stock prices. We fully implement our profiling system and deploy it into several physical systems, such as chemical plants and nuclear power plants, and it greatly helps detect the degradation phenomenon, and diagnose the corresponding components.
Samba: Identifying Inappropriate Videos for Young Children on YouTube
Authors: Le Binh, Rajat Tandon, Chingis Oinar, Jeffrey Liu, Uma Durairaj, Jiani Guo, Spencer Zahabizadeh, Sanjana Ilango, Jeremy Tang, Fred Morstatter, Simon S. Woo, Jelena Mirkovic
Venue: Conference CIKM 2022
Year: 2022

YouTube videos are one of the most effective platforms for disseminating creative material and ideas, and they appeal to a diverse audience. Along with adults and older children, young children are avid consumers of YouTube materials. Children often lack means to evaluate if a given content is appropriate for their age, and parents have very limited options to enforce content restrictions on YouTube. Young children can thus become exposed to inappropriate content, such as violent, scary or disturbing videos on YouTube. Previous studies demonstrated that YouTube videos can be classified into appropriate or inappropriate for young viewers using video metadata, such as video thumbnails, title, comments, etc. Metadata-based approaches achieve high accuracy, but still have significant misclassifications, due to the reliability of input features. In this paper, we propose a fusion model, called Samba, which uses both metadata and video subtitles for content classification. Using subtitles in the model helps better infer the true nature of a video improving classification accuracy. On a large-scale, comprehensive dataset of 70K videos, we show that Samba achieves 95% accuracy, outperforming other state-of-the-art classifiers by at least 7%. We also publicly release our dataset.
SIGIR 2018 Tutorial on Health Search (HS2018): A Full-day from Consumers to Clinicians
Authors: Guido Zuccon, Bevan Koopman
Venue: Conference SIGIR 2018
Year: 2018

The HS2018 tutorial will cover topics from an area of information retrieval (IR) with significant societal impact --- health search. Whether it is searching patient records, helping medical professionals find best-practice evidence, or helping the public locate reliable and readable health information online, health search is a challenging area for IR research with an actively growing community and many open problems. This tutorial will provide attendees with a full stack of knowledge on health search, from understanding users and their problems to practical, hands-on sessions on current tools and techniques, current campaigns and evaluation resources, as well as important open questions and future directions.
Fielded sequential dependence model for ad-hoc entity retrieval in the web of data
Authors: Nikita Zhiltsov, Alexander Kotov, Fedor Nikolaev
Venue: Conference SIGIR 2015
Year: 2015

Previously proposed approaches to ad-hoc entity retrieval in the Web of Data (ERWD) used multi-fielded representation of entities and relied on standard unigram bag-of-words retrieval models. Although retrieval models incorporating term dependencies have been shown to be significantly more effective than the unigram bag-of-words ones for ad hoc document retrieval, it is not known whether accounting for term dependencies can improve retrieval from the Web of Data. In this work, we propose a novel retrieval model that incorporates term dependencies into structured document retrieval and apply it to the task of ERWD. In the proposed model, the document field weights and the relative importance of unigrams and bigrams are optimized with respect to the target retrieval metric using a learning-to-rank method. Experiments on a publicly available benchmark indicate significant improvement of the accuracy of retrieval results by the proposed model over state-of-the-art retrieval models for ERWD.
Learning search engine specific query transformations for question answering
Authors: Eugene Agichtein, Steve Lawrence, Luis Gravano
Venue: Conference WWW 2001
Year: 2001

We introduce a method for learning query transformations that improves the ability to retrieve answers to questions from an information retrieval system. During the training stage the method involves automatically learning phrase features for classifying questions into different types, automatically generating candidate query transformations from a training set of question/answer pairs, and automatically evaluating the candidate transforms on target information retrieval systems such as real-world general purpose search engines. At run time, questions are transformed into a set of queries, and re-ranking is performed on the documents retrieved. We present a prototype search engine, Tritus, that applies the method to web search engines. Blind evaluation on a set of real queries from a web search engine log shows that the method significantly outperforms the underlying web search engines as well as a commercial search engine specializing in question answering. Keywords Web search, quer...
Probabilistic score estimation with piecewise logistic regression
Authors: Jian Zhang 0003, Yiming Yang 0002
Venue: Conference ICML 2004
Year: 2004

Well-calibrated probabilities are necessary in many applications like probabilistic frameworks or cost-sensitive tasks. Based on previous success of asymmetric Laplace method in calibrating text classifiers' scores, we propose to use piecewise logistic regression, which is a simple extension of standard logistic regression, as an alternative method in the discriminative family. We show that both methods have the flexibility to be piecewise linear functions in log-odds, but they are based on quite different assumptions. We evaluated asymmetric Laplace method, piecewise logistic regression and standard logistic regression over standard text categorization collections (Reuters-21578 and TRECAP) with three classifiers (SVM, Naive Bayes and Logistic Regression Classifier), and observed that piecewise logistic regression performs significantly better than the other two methods in the log-loss metric.
TSCAN: a novel method for topic summarization and content anatomy
Authors: Chien Chin Chen, Meng Chang Chen
Venue: Conference SIGIR 2008
Year: 2008

A topic is defined as a seminal event or activity along with all directly related events and activities. It is represented as a chronological sequence of documents by different authors published on the Internet. In this paper, we define a task called topic anatomy, which summarizes and associates core parts of a topic graphically so that readers can understand the content easily. The proposed topic anatomy model, called TSCAN, derives the major themes of a topic from the eigenvectors of a temporal block association matrix. Then, the significant events of the themes and their summaries are extracted by examining the constitution of the eigenvectors. Finally, the extracted events are associated through their temporal closeness and context similarity to form the evolution graph of the topic. Experiments based on the official TDT4 corpus demonstrate that the generated evolution graphs comprehensibly describe the storylines of topics. Moreover, in terms of content coverage and consistency, the produced summaries are superior to those of other summarization methods based on human composed reference summaries.
Highlighting disputed claims on the web
Authors: Rob Ennals, Beth Trushkowsky, John Mark Agosta
Venue: Conference WWW 2010
Year: 2010

We describe Dispute Finder, a browser extension that alerts a user when information they read online is disputed by a source that they might trust. Dispute Finder examines the text on the page that the user is browsing and highlights any phrases that resemble known disputed claims. If a user clicks on a highlighted phrase then Dispute Finder shows them a list of articles that support other points of view.
Probabilistic models of ranking novel documents for faceted topic retrieval
Authors: Ben Carterette, Praveen Chandar
Venue: Conference CIKM 2009
Year: 2009

Traditional models of information retrieval assume documents are independently relevant. But when the goal is retrieving diverse or novel information about a topic, retrieval models need to capture dependencies between documents. Such tasks require alternative evaluation and optimization methods that operate on different types of relevance judgments. We define faceted topic retrieval as a particular novelty-driven task with the goal of finding a set of documents that cover the different facets of an information need. A faceted topic retrieval system must be able to cover as many facets as possible with the smallest number of documents. We introduce two novel models for faceted topic retrieval, one based on pruning a set of retrieved documents and one based on retrieving sets of documents through direct optimization of evaluation measures. We compare the performance of our models to MMR and the probabilistic model due to Zhai et al. on a set of 60 topics annotated with facets, showing that our models are competitive.
Building Directories for Social Tagging Systems
Authors: Denis Helic, Markus Strohmaier
Venue: Conference CIKM 2011
Year: 2011

Today, a number of algorithms exist for constructing tag hierarchies from social tagging data. While these algorithms were designed with ontological goals in mind, we know very little about their properties from an information retrieval perspective, such as whether these tag hierarchies support efficient navigation in social tagging systems. The aim of this paper is to investigate the usefulness of such tag hierarchies (sometimes also called folksonomies - from folk-generated taxonomy) as directories that aid navigation in social tagging systems. To this end, we simulate navigation of directories as decentralized search on a network of tags using Kleinberg's model. In this model, a tag hierarchy can be applied as background knowledge for decentralized search. By constraining the visibility of nodes in the directories we aim to mimic typical constraints imposed by a practical user interface (UI), such as limiting the number of displayed subcategories or related categories. Our experiments on five different social tagging datasets show that existing tag hierarchy algorithms can support navigation in theory, but our results also demonstrate that they face tremendous challenges when user interface (UI) restrictions are taken into account. Based on this observation, we introduce a new algorithm that constructs efficiently navigable directories on our datasets. The results are relevant for engineers and scientists aiming to improve navigability of social tagging systems.