OpenRaiser/Intern-Atlas huggingface dataset:
Cfir: Fast and effective long-text to image retrieval for large corpora
Authors: Zijun Long, Xuri Ge, Richard McCreadie, Joemon M. Jose
Venue: Conference SIGIR 2024
Year: 2024

Text-to-image retrieval aims to find the relevant images based on a text query, which is important in various use-cases, such as digital libraries, e-commerce, and multimedia databases. Although Multimodal Large Language Models (MLLMs) demonstrate state-of-the-art performance, they exhibit limitations in handling large-scale, diverse, and ambiguous real-world needs of retrieval, due to the computation cost and the injective embeddings they produce. This paper presents a two-stage Coarse-to-Fine Index-shared Retrieval (CFIR) framework, designed for fast and effective large-scale long-text to image retrieval. The first stage, Entity-based Ranking (ER), adapts to long-text query ambiguity by employing a multiple-queries-to-multiple-targets paradigm, facilitating candidate filtering for the next stage. The second stage, Summary-based Re-ranking (SR), refines these rankings using summarized queries. We also propose a specialized Decoupling-BEiT-3 encoder, optimized for handling ambiguous user needs and both stages, which also enhances computational efficiency through vector-based similarity inference. Evaluation on the AToMiC dataset reveals that CFIR surpasses existing MLLMs by up to 11.06% in Recall@1000, while reducing training and retrieval times by 68.75% and 99.79%, respectively. We will release our code to facilitate future research at https://github.com/longkukuhi/CFIR.
Nearest neighbors in high-dimensional data: The emergence and influence of hubs
Authors: Milos Radovanovic 0001, Alexandros Nanopoulos, Mirjana Ivanovic
Venue: Conference ICML 2009
Year: 2009

High dimensionality can pose severe difficulties, widely recognized as different aspects of the curse of dimensionality. In this paper we study a new aspect of the curse pertaining to the distribution of k-occurrences, i.e., the number of times a point appears among the k nearest neighbors of other points in a data set. We show that, as dimensionality increases, this distribution becomes considerably skewed and hub points emerge (points with very high k-occurrences). We examine the origin of this phenomenon, showing that it is an inherent property of high-dimensional vector space, and explore its influence on applications based on measuring distances in vector spaces, notably classification, clustering, and information retrieval.
Dynamic Flow Distribution Prediction for Urban Dockless E-Scooter Sharing Reconfiguration
Authors: Suining He, Kang G. Shin
Venue: Conference WWW 2020
Year: 2020

Thanks to recent progresses in mobile payment, IoT, electric motors, batteries and location-based services, Dockless E-scooter Sharing (DES) has become a popular means of last-mile commute for a growing number of (smart) cities. As e-scooters are getting deployed dynamically and flexibly across city regions that expand and/or shrink, with subsequent social, commercial and environmental evaluation, accurate prediction of the distribution of e-scooters given reconfigured regions becomes essential for the city planners and service providers. To meet this need, we propose GCScoot, a novel dynamic flow distribution prediction for reconfiguring urban DES systems. Based on the real-world datasets with reconfiguration, we analyze the mobility features of the e-scooter distribution and flow dynamics for the data-driven designs. To adapt to dynamic reconfiguration of DES deployment, we propose a novel spatio-temporal graph capsule neural network within GCScoot to predict the future dockless e-scooter flows given the reconfigured regions. GCScoot preprocesses the historical spatial e-scooter distributions into flow graph structures, where discretized city regions are considered as nodes and their mutual flows as edges. Given data-driven designs regarding distance, ride flows and region connectivity, the dynamic region-to-region correlations embedded within the temporal flow graphs are captured through the graph capsule neural network which accurately predicts the DES flows. We have conducted extensive empirical studies upon three different e-scooter datasets (>2.8 million rides in total) in populous US cities including Austin TX, Louisville KY and Minneapolis MN. The evaluation results have corroborated the accuracy and effectiveness of GCScoot in predicting dynamic distribution of dockless e-scooters’ mobility.
Not all features are equal: Discovering essential features for preserving prediction privacy
Authors: Fatemehsadat Mireshghallah, Mohammadkazem Taram, Ali Jalali, Ahmed Taha Elthakeb, Dean M. Tullsen, Hadi Esmaeilzadeh
Venue: Conference WWW 2021
Year: 2021

When receiving machine learning services from the cloud, the provider does not need to receive all features; in fact, only a subset of the features are necessary for the target prediction task. Discerning this subset is the key problem of this work. We formulate this problem as a gradient-based perturbation maximization method that discovers this subset in the input feature space with respect to the functionality of the prediction model used by the provider. After identifying the subset, our framework, Cloak, suppresses the rest of the features using utility-preserving constant values that are discovered through a separate gradient-based optimization process. We show that Cloak does not necessarily require collaboration from the service provider beyond its normal service, and can be applied in scenarios where we only have black-box access to the service provider's model. We theoretically guarantee that Cloak's optimizations reduce the upper bound of the Mutual Information (MI) between the data and the sifted representations that are sent out. Experimental results show that Cloak reduces the mutual information between the input and the sifted representations by 85.01% with only a negligible reduction in utility (1.42%). In addition, we show that Cloak greatly diminishes adversaries' ability to learn and infer non-conducive features.
Video scene understanding using multi-scale analysis
Authors: Yang Yang, Jingen Liu, Mubarak Shah
Venue: Conference ICCV 2009
Year: 2009

We propose a novel method for automatically discovering key motion patterns happening in a scene by observing the scene for an extended period. Our method does not rely on object detection and tracking, and uses low level features, the direction of pixel wise optical flow. We first divide the video into clips and estimate a sequence of flow-fields. Each moving pixel is quantized based on its location and motion direction. This is essentially a bag of words representation of clips. Once a bag of words representation is obtained, we proceed to the screening stage, using a measure called the `conditional entropy'. After obtaining useful words we apply Diffusion maps. Diffusion maps framework embeds the manifold points into a lower dimensional space while preserving the intrinsic local geometric structure. Finally, these useful words in lower dimensional space are clustered to discover key motion patterns. Diffusion map embedding involves diffusion time parameter which gives us ability to detect key motion patterns at different scales using multi-scale analysis. In addition, clips which are represented in terms of frequency of motion patterns can also be clustered to determine multiple dominant motion patterns which occur simultaneously, providing us further understanding of the scene. We have tested our approach on two challenging datasets and obtained interesting and promising results.
Goal: A challenging knowledge-grounded video captioning benchmark for real-time soccer commentary generation
Authors: Ji Qi, Jifan Yu, Teng Tu, Kunyu Gao, Yifan Xu, Xinyu Guan, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li, Jie Tang
Venue: Conference CIKM 2023
Year: 2023

Despite the recent emergence of video captioning models, how to generate vivid, fine-grained video descriptions based on the background knowledge (i.e., long and informative commentary about the domain-specific scenes with appropriate reasoning) is still far from being solved, which however has great applications such as automatic sports narrative. Based on soccer game videos and synchronized commentary data, we present GOAL, a benchmark of over 8.9k soccer video clips, 22k sentences, and 42k knowledge triples for proposing a challenging new task setting as Knowledge-grounded Video Captioning (KGVC). We experimentally test existing state-of-the-art (SOTA) methods on this resource to demonstrate the future directions for improvement in this challenging task. We hope that our data resource (now available at https://github.com/THU-KEG/goal) can serve researchers and developers interested in knowledge-grounded cross-modal applications.
Hoeffding and Bernstein races for selecting policies in evolutionary direct policy search
Authors: Verena Heidrich-Meisner, Christian Igel
Venue: Conference ICML 2009
Year: 2009

Uncertainty arises in reinforcement learning from various sources, and therefore it is necessary to consider statistics based on several roll-outs for evaluating behavioral policies. We add an adaptive uncertainty handling based on Hoeffding and empirical Bernstein races to the CMA-ES, a variable metric evolution strategy proposed for direct policy search. The uncertainty handling adjusts individually the number of episodes considered for the evaluation of a policy. The performance estimation is kept just accurate enough for a sufficiently good ranking of candidate policies, which is in turn sufficient for the CMA-ES to find better solutions. This increases the learning speed as well as the robustness of the algorithm.
The motion coherence theory
Authors: Alan L. Yuille, Norberto M. Grzywacz
Venue: Conference ICCV 1988
Year: 1988

Tliere are a number of important phenom- ena in motion perception involving colicrcnce. Examples include motion capture and motion cooperativity. We propose a theoretical model, called the motion coherence tlieory, that gives a possible explanation for these effects (Yuille and Grzywacz, 1988a,b). In this framework, the aperture problem can also be thought of as a problem of coherence and given a similar explanation. We propose the concept of a velocity field dcfined everywhere in the image, even where there is no explicit motion information available. Through a cost function, tlie model imposes smoothness on the velocity field in a more general way than previous theories. In this paper, we provide a de- tailed theoretical analysis of the motion coherence theory. We discuss its relations with previous theories and show that some of t1ic.m arc approximations to it. A sccorid pa- per (Grzywacz, Smith, and Yuillc, 1088) provides exten- sions and cletnilcd comparisons to psychophysical plienom- cna. Tlic theory applies to both short-range and long- range motion. It places them in the same computational framework aiid provides a way to define interactions be- twcr:11 the two 1)roccsses.
Properties of energy edge detectors
Authors: Paul Kube
Venue: Conference CVPR 1992
Year: 1992

The author introduces a framework for investigating the properties of energy edge detectors and uses it to derive some results of interest. He shows a necessary condition on the form of constituent linear filters in quadratic detectors, subject to some conditions, and demonstrates some limitations of such detectors. It is shown that no quadratic detector can detect an edge at 0 for both a sinewave and a cosine wave, which has implications for detecting narrowband edges with spatially local filters. It is also shown that the scale-space behavior of energy detectors is not well-behaved, in that it contains bifurcations as scale increases, i.e. new edges can be created as the image is smoothed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Topical semantics of twitter links
Authors: Michael J. Welch, Uri Schonfeld, Dan He, Junghoo Cho
Venue: Conference WSDM 2011
Year: 2011

Twitter, a micro-blogging platform with an estimated 20 million unique monthly visitors and over 100 million registered users, offers an abundance of rich, structured data at a rate exceeding 600 tweets per second. Recent efforts to leverage this social data to rank users by quality and topical relevance have largely focused on the "follow" relationship. Twitter's data offers additional implicit relationships between users, however, such as "retweets" and "mentions". In this paper we investigate the semantics of the follow and retweet relationships. Specifically, we show that the transitivity of topical relevance is better preserved over retweet links, and that retweeting a user is a significantly stronger indicator of topical interest than following him. We demonstrate these properties by ranking users with two variants of the PageRank algorithm; one based on the follows sub-graph and one based on the implicit retweet sub-graph. We perform a user study to assess the topical relevance of the resulting top-ranked users.
Compact explanatory opinion summarization
Authors: Hyun Duk Kim, Malú Castellanos, Meichun Hsu, ChengXiang Zhai, Umeshwar Dayal, Riddhiman Ghosh
Venue: Conference CIKM 2013
Year: 2013

In this paper, we propose a novel opinion summarization problem called compact explanatory opinion summarization (CEOS) which aims to extract within-sentence explanatory text segments from input opinionated texts to help users better understand the detailed reasons of sentiments. We propose and study general methods for identifying candidate boundaries and scoring the explanatoriness of text segments using Hidden Markov Models. We create new data sets and use a new evaluation measure to evaluate CEOS. Experimental results show that the proposed methods are effective for generating an explanatory opinion summary, outperforming a standard text summarization method.
Power dynamics in spoken interactions: a case study on 2012 Republican primary debates
Authors: Vinodkumar Prabhakaran, Ajita John, Dorée D. Seligmann
Venue: Conference WWW 2013
Year: 2013

In this paper, we explore how the power differential between participants of an interaction affects the way they interact in the context of political debates. We analyze the 2012 Republican presidential primary debates where we model the power index of each candidate in terms of their poll standings. We find that the candidates' power indices affected the way they interacted with others in the debates as well as how others interacted with them.
Explanatory semantic relatedness and explicit spatialization for exploratory search
Authors: Brent J. Hecht, Samuel Carton, Mahmood Quaderi, Johannes Schöning, Martin Raubal, Darren Gergle, Doug Downey
Venue: Conference SIGIR 2012
Year: 2012

Exploratory search, in which a user investigates complex concepts, is cumbersome with today's search engines. We present a new exploratory search approach that generates interactive visualizations of query concepts using thematic cartography (e.g. choropleth maps, heat maps). We show how the approach can be applied broadly across both geographic and non-geographic contexts through explicit spatialization, a novel method that leverages any figure or diagram -- from a periodic table, to a parliamentary seating chart, to a world map -- as a spatial search environment. We enable this capability by introducing explanatory semantic relatedness measures. These measures extend frequently-used semantic relatedness measures to not only estimate the degree of relatedness between two concepts, but also generate human-readable explanations for their estimates by mining Wikipedia's text, hyperlinks, and category structure. We implement our approach in a system called Atlasify, evaluate its key components, and present several use cases.
Mining advices from weblogs
Authors: Alfan Farizki Wicaksono, Sung-Hyon Myaeng
Venue: Conference CIKM 2012
Year: 2012

Weblog, one of the fastest growing user generated contents, often contains key learnings gleaned from people's past experiences which are really worthy to be well presented to other people. One of the key learnings contained in weblogs is often vented in the form of advice. In this paper, we aim to provide a methodology to extract sentences that reveal advices on weblogs. We observed our data to discover the characteristics of advices contained in weblogs. Based on this observation, we define our task as a classification problem using various linguistic features. We show that our proposed method significantly outperforms the baseline. The presence or absence of imperative mood expression appears to be the most important feature in this task. It is also worth noting that the work presented in this paper is the first attempt on mining advices from English data.
Entity-driven type hierarchy construction for freebase
Authors: Jyun-Yu Jiang, Pu-Jen Cheng, Chin-Yew Lin
Venue: Conference WWW 2015
Year: 2015

The hierarchical structure of a knowledge base system can lead to various valuable applications; however, many knowledge base systems do not have such property. In this paper, we propose an entity-driven approach to automatically construct the hierarchical structure of entities for knowledge base systems. By deriving type dependencies from entity information, the initial graph of types will be constructed, and then modified to become a hierarchical structure by several graph algorithms. Experimental results show the effectiveness of our method in terms of constructing reasonable type hierarchy for knowledge base systems.
Improving the precision of RDF question/answering systems: A why not approach
Authors: Xinbo Zhang, Lei Zou
Venue: Conference WWW 2017
Year: 2017

Given a natural language question qNL over an RDF dataset D, an RDF Question/Answering (Q/A) system first translatesqNL into a SPARQL query graph Q and then evaluates Q over the underlying knowledge graph to figure out the answers Q(D). However, due to the challenge of understanding natural language questions and the complexity of linking phrases with specific RDF items (e.g., entities and predicates), the translated query graph Q may be incorrect, leading to some wrong or missing answers. In order to improve the system's precision, we propose a self-learning solution based on the users' feedback over Q(D. Specifically, our method automatically refines the SPARQL query Q into a new query graph Q′ with minimum modifications (over the original query Q). The new query will fix the errors and omissions of the query results. Furthermore, each amendment will also be used to improve the precision in answering subsequent natural language questions.
Event registry: Learning about world events from news
Authors: Gregor Leban, Blaz Fortuna, Janez Brank, Marko Grobelnik
Venue: Conference WWW 2014
Year: 2014

Event Registry is a system that can analyze news articles and identify in them mentioned world events. The system is able to identify groups of articles that describe the same event. It can identify groups of articles in different languages that describe the same event and represent them as a single event. From articles in each event it can then extract event's core information, such as event location, date, who is involved and what is it about. Extracted information is stored in a database. A user interface is available that allows users to search for events using extensive search options, to visualize and aggregate the search results, to inspect individual events and to identify related events.
Predicting emerging social conventions in online social networks
Authors: Farshad Kooti, Winter A. Mason, P. Krishna Gummadi, Meeyoung Cha
Venue: Conference CIKM 2012
Year: 2012

The way in which social conventions emerge in communities has been of interest to social scientists for decades. Here we report on the emergence of a particular social convention on Twitter---the way to indicate a tweet is being reposted and attributing the content to its source. Despite being invented at different times and having different adoption rates, only two variations became widely adopted. In this paper we describe this process in detail, highlighting the factors that come into play in deciding which variation individuals will adopt. Our classification analysis demonstrates that the date of adoption and the number of exposures are particularly important in the adoption process, while personal features (such as the number of followers and join date) and the number of adopter friends have less discriminative power in predicting adoptions. We discuss implications of these findings in the design of future Web applications and services.
Towards a benchmark for fact checking with knowledge bases
Authors: Viet-Phi Huynh, Paolo Papotti
Venue: Conference WWW 2018
Year: 2018

Fact checking is the task of determining if a given claim holds. Several algorithms have been developed to check facts with reference information in the form of knowledge bases. While individual algorithms have been experimentally evaluated, we provide a first publicly available benchmark evaluating fact checking implementations across a range of assumptions about the properties of the facts and the reference data. We used our benchmark to compare algorithms designed on different principles and assumptions, as well as algorithms that can solve similar tasks developed in closely related communities. Our evaluation provided us with a number of new insights concerning the factors that impact the performance of the different methods.
Detecting Duplicate Posts in Programming QA Communities via Latent Semantics and Association Rules
Authors: Wei Emma Zhang, Quan Z. Sheng, Jey Han Lau, Ermyas Abebe
Venue: Conference WWW 2017
Year: 2017

Programming community-based question-answering (PCQA) websites such as Stack Overflow enable programmers to find working solutions to their questions. Despite detailed posting guidelines, duplicate questions that have been answered are frequently created. To tackle this problem, Stack Overflow provides a mechanism for reputable users to manually mark duplicate questions. This is a laborious effort, and leads to many duplicate questions remain undetected. Existing duplicate detection methodologies from traditional community based question-answering (CQA) websites are difficult to be adopted directly to PCQA, as PCQA posts often contain source code which is linguistically very different from natural languages. In this paper, we propose a methodology designed for the PCQA domain to detect duplicate questions. We model the detection as a classification problem over question pairs. To extract features for question pairs, our methodology leverages continuous word vectors from the deep learning literature, topic model features and phrases pairs that co-occur frequently in duplicate questions mined using machine translation systems. These features capture semantic similarities between questions and produce a strong performance for duplicate detection. Experiments on a range of real-world datasets demonstrate that our method works very well; in some cases over 30% improvement compared to state-of-the-art benchmarks. As a product of one of the proposed features, the association score feature, we have mined a set of associated phrases from duplicate questions on Stack Overflow and open the dataset to the public.