OpenRaiser/Intern-Atlas huggingface dataset:
Article Clipper: A System for Web Article Extraction
Authors: Jian Fan, Ping Luo, Suk Hwan Lim, Sam Liu, Joshi Parag, Jerry Liu
Venue: Conference KDD 2011
Year: 2011

Many people use the Web as the main source of information in their daily lives. However, most web pages contain non-informative components such as side bars, footers, headers, and advertisements, which are undesirable for certain applications like printing. We demonstrate a system that automatically extracts the informative contents from news- and blog-like web pages. In contrast to many existing methods that are limited to identifying only the text or the bounding rectangular region, our system not only identifies the content but also the structural roles of various content components such as title, paragraphs, images and captions. The structural information enables re-layout of the content in a pleasing way. Besides the article text extraction, our system includes the following components: 1) print-link detection to identify the URL link for printing, and to use it for more reliable analysis and recognition; 2) title detection incorporating both visual cues and HTML tags; 3) image and caption detection utilizing extensive visual cues; 4) multiple-page and next page URL detection. The performance of our system has been thoroughly evaluated using a human labeled ground truth dataset consisting of 2000 web pages from 100 major web sites. We show accurate results using such a dataset.
Security Challenges in an Increasingly Tangled Web
Authors: Deepak Kumar, Zane Ma, Zakir Durumeric, Ariana Mirian, Joshua Mason, J. Alex Halderman, Michael D. Bailey
Venue: Conference WWW 2017
Year: 2017

Over the past 20 years, websites have grown increasingly complex and interconnected. In 2016, only a negligible number of sites are dependency free, and over 90% of sites rely on external content. In this paper, we investigate the current state of web dependencies and explore two security challenges associated with the increasing reliance on external services: (1) the expanded attack surface associated with serving unknown, implicitly trusted third-party content, and (2) how the increased set of external dependencies impacts HTTPS adoption. We hope that by shedding light on these issues, we can encourage developers to consider the security risks associated with serving third-party content and prompt service providers to more widely deploy HTTPS.
A Fast and Simple Method for Extracting Relevant Content from News Webpages
Authors: Eduardo Sany Laber, Críston Pereira de Souza, Iam Vita Jabour, Evelin Carvalho Freire de Amorim, Eduardo Teixeira Cardoso, Raúl P. Rentería, Lúcio Cunha Tinoco, Caio Dias Valentim
Venue: Conference CIKM 2009
Year: 2009

We propose NCE, an efficient algorithm to identify and extract relevant content from news webpages. We define relevant as the textual sections that more objectively describe the main event in the article. This includes the title and the main body section, and excludes comments about the story and presentation elements.
Test-driven evaluation of linked data quality
Authors: Dimitris Kontokostas, Patrick Westphal, Sören Auer, Sebastian Hellmann, Jens Lehmann, Roland Cornelissen, Amrapali Zaveri
Venue: Conference WWW 2014
Year: 2014

Linked Open Data (LOD) comprises an unprecedented volume of structured data on the Web. However, these datasets are of varying quality ranging from extensively curated datasets to crowdsourced or extracted data of often relatively low quality. We present a methodology for test-driven quality assessment of Linked Data, which is inspired by test-driven software development. We argue that vocabularies, ontologies and knowledge bases should be accompanied by a number of test cases, which help to ensure a basic level of quality. We present a methodology for assessing the quality of linked data resources, based on a formalization of bad smells and data quality problems. Our formalization employs SPARQL query templates, which are instantiated into concrete quality test case queries. Based on an extensive survey, we compile a comprehensive library of data quality test case patterns. We perform automatic test case instantiation based on schema constraints or semi-automatically enriched schemata and allow the user to generate specific test case instantiations that are applicable to a schema or dataset. We provide an extensive evaluation of five LOD datasets, manual test case instantiation for five schemas and automatic test case instantiations for all available schemata registered with Linked Open Vocabularies (LOV). One of the main advantages of our approach is that domain specific semantics can be encoded in the data quality test cases, thus being able to discover data quality problems beyond conventional quality heuristics.
Inferring movement trajectories from GPS snippets
Authors: Mu Li, Amr Ahmed, Alexander J. Smola
Venue: Conference WSDM 2015
Year: 2015

Inferring movement trajectories can be a challenging task, in particular when detailed tracking information is not available due to privacy and data collection constraints. In this paper we present a complete and computationally tractable model for estimating and predicting trajectories based on sparsely sampled, anonymous GPS land-marks that we call GPS snippets. To combat data sparsity we use mapping data as side information to constrain the inference process. We show the efficacy of our approach on a set of prediction tasks over data collected from different cities in the US.
Efficient Hidden Trajectory Reconstruction from Sparse Data
Authors: Ning Yang, Philip S. Yu
Venue: Conference CIKM 2016
Year: 2016

In this paper, we investigate the problem of reconstructing hidden trajectories from a collective of separate spatial-temporal points without ID information, given the number of hidden trajectories. The challenge is three-fold: lack of meaningful features, data sparsity, and missing trajectory links. We propose a novel approach called Hidden Trajectory Reconstruction (HTR). From an information-theoretic perspective, we devise five novel temporal features and combine them into an Latent Spatial-Temporal Feature Vector (LSTFV) to characterize the dynamics of a single spatial-temporal point. The proposed features have the potential of distinguishing spatial-temporal points between trajectories. To overcome the data sparsity, we assemble the LSTFVs to a sparse Temporal Feature Tensor (TF-Tensor) and propose an algorithm called Parallel Iterative Collaborative Approximation of Sparse Tensor (PICAST). PICAST approximates the TF-Tensor by decomposing it into a tensor product of a low-rank core identity tensor and three dense factor matrices with a divide-and-conquer strategy. To achieve a dense approximate tensor with good accuracy and efficiency, PICAST minimizes a sparsity-measure and fuses an additional matrix of static geographical region features. To recover the missing trajectory links, we propose a mapping, Cross-Temporal Connectivity Preserving Transformation (CTCPT), to map the LSTFVs of the separate spatial-temporal points to an intrinsic space called Cross-Temporal Connectivity Preserving Space (CTCPS). CTCPT uses Cross-Temporal Connectivity (CTC) to evaluate whether two spatial-temporal points belong to the same trajectory and if they do, how strong the connectivity between them is. Due to the CTCPT, the hidden trajectories can be reconstructed from clusters generated in CTCPS by a clustering algorithm. At last, the extensive experiments conducted on synthetic datasets and real datasets verify the effectiveness and efficiency of our algorithms.
Concept-based interactive query expansion
Authors: Bruno M. Fonseca, Paulo Braz Golgher, Bruno Pôssas, Berthier A. Ribeiro-Neto, Nivio Ziviani
Venue: Conference CIKM 2005
Year: 2005

Despite the recent advances in search quality, the fast increase in the size of the Web collection has introduced new challenges for Web ranking algorithms. In fact, there are still many situations in which the users are presented with imprecise or very poor results. One of the key difficulties is the fact that users usually submit very short and ambiguous queries, and they do not fully specify their information needs. That is, it is necessary to improve the query formation process if better answers are to be provided. In this work we propose a novel concept-based query expansion technique, which allows disambiguating queries submitted to search engines. The concepts are extracted by analyzing and locating cycles in a special type of query relations graph. This is a directed graph built from query relations mined using association rules. The concepts related to the current query are then shown to the user who selects the one concept that he interprets is most related to his query. This concept is used to expand the original query and the expanded query is processed instead. Using a Web test collection, we show that our approach leads to gains in average precision figures of roughly 32%. Further, if the user also provides information on the type of relation between his query and the selected concept, the gains in average precision go up to roughly 52%.
Translation of Natural Language Query Into Keyword Query Using a RNN Encoder-Decoder
Authors: Hyun-Je Song, A.-Yeong Kim, Seong-Bae Park
Venue: Conference SIGIR 2017
Year: 2017

The number of natural language queries submitted to search engines is increasing as search environments get diversified. However, legacy search engines are still optimized for short keyword queries. Thus, the use of natural language queries at legacy search engines degrades the retrieval performance of the engines. This paper proposes a novel method to translate a natural language query into a keyword query relevant to the natural language query for retrieving better search results without change of the engines. The proposed method formulates the translation as a generation task. That is, the method generates a keyword query from a natural language query by preserving the semantics of the natural language query. A recurrent neural network encoder-decoder architecture is adopted as a generator of keyword queries from natural language queries. In addition, an attention mechanism is also used to cope with long natural language queries.
Relevant Document Discovery for Fact-Checking Articles
Authors: Xuezhi Wang, Cong Yu, Simon Baumgartner, Flip Korn
Venue: Conference WWW 2018
Year: 2018

With the support of major search platforms such as Google and Bing, fact-checking articles, which can be identified by their adoption of the schema.org ClaimReview structured markup, have gained widespread recognition for their role in the fight against digital misinformation. A claim-relevant document is an online document that addresses, and potentially expresses a stance towards, some claim. The claim-relevance discovery problem, then, is to find claim-relevant documents. Depending on the verdict from the fact check, claim-relevance discovery can help identify online misinformation. In this paper, we provide an initial approach to the claim-relevance discovery problem by leveraging various information retrieval and machine learning techniques. The system consists of three phases. First, we retrieve candidate documents based on various features in the fact-checking article. Second, we apply a relevance classifier to filter away documents that do not address the claim. Third, we apply a language feature based classifier to distinguish documents with different stances towards the claim. We experimentally demonstrate that our solution achieves solid results on a large-scale dataset and beats state-of-the-art baselines. Finally, we highlight a rich set of case studies to demonstrate the myriad of remaining challenges and that this problem is far from being solved.
Pruned query evaluation using pre-computed impacts
Authors: Vo Ngoc Anh, Alistair Moffat
Venue: Conference SIGIR 2006
Year: 2006

Exhaustive evaluation of ranked queries can be expensive, particularly when only a small subset of the overall ranking is required, or when queries contain common terms. This concern gives rise to techniques for dynamic query pruning, that is, methods for eliminating redundant parts of the usual exhaustive evaluation, yet still generating a demonstrably "good enough" set of answers to the query. In this work we propose new pruning methods that make use of impact-sorted indexes. Compared to exhaustive evaluation, the new methods reduce the amount of computation performed, reduce the amount of memory required for accumulators, reduce the amount of data transferred from disk, and at the same time allow performance guarantees in terms of precision and mean average precision. These strong claims are backed by experiments using the TREC Terabyte collection and queries.
Pruning policies for two-tiered inverted index with correctness guarantee
Authors: Alexandros Ntoulas, Junghoo Cho
Venue: Conference SIGIR 2007
Year: 2007

The Web search engines maintain large-scale inverted indexes which are queried thousands of times per second by users eager for information. In order to cope with the vast amounts of query loads, search engines prune their index to keep documents that are likely to be returned as top results, and use this pruned index to compute the first batches of results. While this approach can improve performance by reducing the size of the index, if we compute the top results only from the pruned index we may notice a significant degradation in the result quality: if a document should be in the top results but was not included in the pruned index, it will be placed behind the results computed from the pruned index. Given the fierce competition in the online search market, this phenomenon is clearly undesirable.
Ranking specialization for web search: a divide-and-conquer approach by using topical ranksvm
Authors: Jiang Bian 0002, Xin Li, Fan Li, Zhaohui Zheng 0001, Hongyuan Zha
Venue: Conference WWW 2010
Year: 2010

Many ranking algorithms applying machine learning techniques have been proposed in informational retrieval and Web search. However, most of existing approaches do not explicitly take into account the fact that queries vary significantly in terms of ranking and entail different treatments regarding the ranking models. In this paper, we apply a divide-and-conquer framework for ranking specialization, i.e. learning multiple ranking models by addressing query difference. We first generate query representation by aggregating ranking features through pseudo feedbacks, and employ unsupervised clustering methods to identify a set of ranking-sensitive query topics based on training queries. To learn multiple ranking models for respective ranking-sensitive query topics, we define a global loss function by combining the ranking risks of all query topics, and we propose a unified SVM-based learning process to minimize the global loss. Moreover, we employ an ensemble approach to generate the ranking result for each test query by applying a set of ranking models of the most appropriate query topics. We conduct experiments using a benchmark dataset for learning ranking functions as well as a dataset from a commercial search engine. Experimental results show that our proposed approach can significantly improve the ranking performance over existing single-model approaches as well as straightforward local ranking approaches, and the automatically identified ranking-sensitive topics are more useful for enhancing ranking performance than pre-defined query categorization.
Fast evaluation of structured queries for information retrieval
Authors: Eric W. Brown
Venue: Conference SIGIR 1995
Year: 1995

Information retrieval systems are being challenged to manage larger and larger document collections. In an effort to provide better retrieval performance on large collections, more sophisticated retrieval techniques have been developed that support rich, structured queries. Structured queries are not amenable to previously proposed optimization techniques. Optimizing execution, however, is even more important in the context of large document collections. We present a new structured query optimization technique which we have implemented in an inference network-based information retrieval system. Experimental results show that query evaluation time can be reduced by more than half with little impact on retrieval effectiveness.
Before and After GDPR: The Changes in Third Party Presence at Public and Private European Websites
Authors: Jannick Kirk Sørensen, Sokol Kosta
Venue: Conference WWW 2019
Year: 2019

The commencement of EU's General Data Protection Regulation (GDPR) has led to massive compliance and consent activities on websites. But did the new regulation result in fewer third party server appearances? Based on an eight months longitudinal study from February to September 2018 of 1250 popular websites in Europe and US, we present a mapping of the subtle shifts in the third party topology before and after May 25, 2018. The 1250 websites cover 39 European countries from EU, EEA, and outside EU, belonging to categories that cover both public-oriented citizen services, as well as commercially-oriented sites. The developments in the numbers and types of third party vary for categories of websites and countries. Analyzing the number of third parties over time, even though we notice a decline in the number of third parties in websites belonging to certain categories, we are cautious about attributing these effects to the general assumption that GDPR would lead to less third party activity. We believe that it is quite difficult to draw conclusions on cause-effect relationships in such a complex environment with many impacting factors.
Embedding identity and interest for social networks
Authors: Linchuan Xu, Xiaokai Wei, Jiannong Cao, Philip S. Yu
Venue: Conference WWW 2017
Year: 2017

Network embedding fills the gap of applying tuple-based data mining models to networked datasets through learning latent representations or embeddings. However, it may not be likely to associate latent embeddings with physical meanings just as the name, latent embedding, literally suggests. Hence, models built on embeddings may not be interpretable. In this paper, we thus propose to learn identity embeddings and interest embeddings, where user identity includes demographic and affiliation information, and interest is demonstrated by activities or topics users are interested in. With identity and interest information, we can make data mining models not only more interpretable, but also more accurate, which is demonstrated on three real-world social networks in link prediction and multi-task classification.
Learning Graph-based Embedding For Time-Aware Product Recommendation
Authors: Yuqi Li, Weizheng Chen, Hongfei Yan
Venue: Conference CIKM 2017
Year: 2017

In this paper, we propose a novel Product Graph Embedding (PGE) model to investigate time-aware product recommendation by leveraging the network representation learning technique. Our model captures the sequential influences of products by transforming the historical purchase records into a product graph. Then the product can be transformed into a low dimensional vector by the network embedding model. Once products are projected into the latent space, we present a novel method to compute user's latest preferences, which projects users into the same latent space as products. This method is based on time-decay functions and the embedding of sequential products that the user purchased. Thus, relatedness between a product and a user can be measured by the similarity between the embedding vectors which represent the product and the user's preferences. The experimental results on purchase records crawled from JINGDONG, show the superiority of our proposed framework for personalized product recommendation.
Negative confidence-aware weakly supervised binary classification for effective review helpfulness classification
Authors: Xi Wang, Iadh Ounis, Craig MacDonald
Venue: Conference CIKM 2020
Year: 2020

The incompleteness of positive labels and the presence of many unlabelled instances are common problems in binary classification applications such as in review helpfulness classification. Various studies from the classification literature consider all unlabelled instances as negative examples. However, a classification model that learns to classify binary instances with incomplete positive labels while assuming all unlabelled data to be negative examples will often generate a biased classifier. In this work, we propose a novel Negative Confidence-aware Weakly Supervised approach (NCWS), which customises a binary classification loss function by discriminating the unlabelled examples with different negative confidences during the classifier's training. NCWS allows to effectively, unbiasedly identify and separate positive and negative instances after its integration into various binary classifiers from the literature, including SVM, CNN and BERT-based classifiers. We use the review helpfulness classification as a test case for examining the effectiveness of our NCWS approach. We thoroughly evaluate NCWS by using three different datasets, namely one from Yelp (venue reviews), and two from Amazon (Kindle and Electronics reviews). Our results show that NCWS outperforms strong baselines from the literature including an existing SVM-based approach (i.e. SVM-P), the positive and unlabelled learning-based approach (i.e. C-PU) and the positive confidence-based approach (i.e. P-conf) in addressing the classifier's bias problem. Moreover, we further examine the effectiveness of NCWS by using its classified helpful reviews in a state-of-the-art review-based venue recommendation model (i.e. DeepCoNN) and demonstrate the benefits of using NCWS in enhancing venue recommendation effectiveness in comparison to the baselines.
DéJà vu: A Contextualized Temporal Attention Mechanism for Sequential Recommendation
Authors: Jibang Wu, Renqin Cai, Hongning Wang
Venue: Conference WWW 2020
Year: 2020

Predicting users’ preferences based on their sequential behaviors in history is challenging and crucial for modern recommender systems. Most existing sequential recommendation algorithms focus on transitional structure among the sequential actions, but largely ignore the temporal and context information, when modeling the influence of a historical event to current prediction. In this paper, we argue that the influence from the past events on a user’s current action should vary over the course of time and under different context. Thus, we propose a Contextualized Temporal Attention Mechanism that learns to weigh historical actions’ influence on not only what action it is, but also when and how the action took place. More specifically, to dynamically calibrate the relative input dependence from the self-attention mechanism, we deploy multiple parameterized kernel functions to learn various temporal dynamics, and then use the context information to determine which of these reweighing kernels to follow for each input. In empirical evaluations on two large public recommendation datasets, our model consistently outperformed an extensive set of state-of-the-art sequential recommendation methods.