OpenRaiser/Intern-Atlas huggingface dataset:
Matching Cross Network for Learning to Rank in Personal Search
Authors: Zhen Qin, Zhongliang Li, Michael Bendersky, Donald Metzler
Venue: WWW
Year: 2020

Recent neural ranking algorithms focus on learning semantic matching between query and document terms. However, practical learning to rank systems typically rely on a wide range of side information beyond query and document textual features, like location, user context, etc. It is common practice to concatenate all of these features and rely on deep models to learn a complex representation. We study how to effectively and efficiently combine textual information from queries and documents with other useful but less prominent side information for learning to rank. We conduct synthetic experiments to show that: 1) neural networks are inefficient at learning the interaction between two prominent features (e.g., query and document embedding features) in the presence of other less prominent features; 2) direct application of a state-of-art method for higher-order feature generation is also inefficient. Based on the above observations, we propose a simple but effective matching cross network (MCN) method for learning to rank with side information. MCN conducts an element-wise multiplication matching of query and document embeddings and leverages a technique called latent cross to effectively learn the interaction between matching output and all side information. The approach is easy to implement, and adds minimal parameters and latency overhead to standard neural ranking architectures. We conduct extensive experiments using two of the world’s largest personal search engines, Gmail and Google Drive search, and show that each proposed component adds meaningful gains against a strong production baseline with minimal latency overhead, thereby demonstrating the practical effectiveness and efficiency of the proposed approach.
Optimizing Ontology Materialization with Equivalent Role and Inverse Role Rewriting
Authors: Xiaoyu Qin, Xiaowang Zhang, Zhiyong Feng
Venue: WWW
Year: 2020

Materialization, as a basic approach to ontology-mediated query answering, extends an ontology by exploiting the semantic knowledge expressed in the ontology to improve query answering over data. However, those existing materialization approaches are often inefficient, especially when ontologies consist of complex roles. In this paper, we propose a new rewriting algorithm for equivalent roles and inverse roles to both improve materialization efficiency and reduce memory consumption. Moreover, we develop a role scoring algorithm to optimize role rewriting. The preliminary experiments show that our approach can significantly reduce materialization times and memory consumption.
VirtualCrowd: A Simulation Platform for Microtask Crowdsourcing Campaigns
Authors: Sihang Qiu, Alessandro Bozzon, Geert-Jan Houben
Venue: WWW
Year: 2020

This demo presents VirtualCrowd, a simulation platform for crowdsourcing campaigns. The platform allows the design, configuration, step-by-step execution, and analysis of customized tasks, worker profiles, and crowdsourcing strategies. The platform will be demonstrated through a crowd-mapping example in two cities, which will highlight the utility of VirtualCrowd for complex crowdsourcing tasks in real world settings.
Continuous-Time Link Prediction via Temporal Dependent Graph Neural Network
Authors: Liang Qu, Huaisheng Zhu, Qiqi Duan, Yuhui Shi
Venue: WWW
Year: 2020

Recently, graph neural networks (GNNs) have been shown to be an effective tool for learning the node representations of the networks and have achieved good performance on the semi-supervised node classification task. However, most existing GNNs methods fail to take networks’ temporal information into account, therefore, cannot be well applied to dynamic network applications such as the continuous-time link prediction task. To address this problem, we propose a Temporal Dependent Graph Neural Network (TDGNN), a simple yet effective dynamic network representation learning framework which incorporates the network temporal information into GNNs. TDGNN introduces a novel Temporal Aggregator (TDAgg) to aggregate the neighbor nodes’ features and edges’ temporal information to obtain the target node representations. Specifically, it assigns the neighbor nodes aggregation weights using an exponential distribution to bias different edges’ temporal information. The performance of the proposed method has been validated on six real-world dynamic network datasets for the continuous-time link prediction task. The experimental results show that the proposed method outperforms several state-of-the-art baselines.
Facebook Ads as a Demographic Tool to Measure the Urban-Rural Divide
Authors: Daniele Rama, Yelena Mejova, Michele Tizzoni, Kyriaki Kalimeri, Ingmar Weber
Venue: WWW
Year: 2020

In the global move toward urbanization, making sure the people remaining in rural areas are not left behind in terms of development and policy considerations is a priority for governments worldwide. However, it is increasingly challenging to track important statistics concerning this sparse, geographically dispersed population, resulting in a lack of reliable, up-to-date data. In this study, we examine the usefulness of the Facebook Advertising platform, which offers a digital “census” of over two billions of its users, in measuring potential rural-urban inequalities. We focus on Italy, a country where about 30% of the population lives in rural areas. First, we show that the population statistics that Facebook produces suffer from instability across time and incomplete coverage of sparsely populated municipalities. To overcome such limitation, we propose an alternative methodology for estimating Facebook Ads audiences that nearly triples the coverage of the rural municipalities from 19% to 55% and makes feasible fine-grained sub-population analysis. Using official national census data, we evaluate our approach and confirm known significant urban-rural divides in terms of educational attainment and income. Extending the analysis to Facebook-specific user “interests” and behaviors, we provide further insights on the divide, for instance, finding that rural areas show a higher interest in gambling. Notably, we find that the most predictive features of income in rural areas differ from those for urban centres, suggesting researchers need to consider a broader range of attributes when examining rural wellbeing. The findings of this study illustrate the necessity of improving existing tools and methodologies to include under-represented populations in digital demographic studies – the failure to do so could result in misleading observations, conclusions, and most importantly, policies.
Towards Complete Decentralised Verification of Data with Confidentiality: Different ways to connect Solid Pods and Blockchain
Authors: Manoharan Ramachandran, Niaz Chowdhury, Allan Third, John Domingue, Kevin Quick, Michelle Bachler
Venue: WWW
Year: 2020

Over-centralisation of data leads to tampering and sharing user information without the consent of the owners. This problem has been studied extensively in recent times providing separate solutions involving distributed storage, Blockchain technology and Solid Pods. Individually these solutions are not sufficient to build realistic applications in a decentralised environment; however, a combination of them can effectively provide more powerful and useful use-cases. In this paper, we propose the methods of combining Solid Pods and distributed ledgers in introducing complete decentralisation of data with total user-control, keeping the integrity of the stored information intact through Blockchain-based verification. We demonstrated multiple configurations of our solutions, offering several new use-cases in various sectors. These configurations introduce new dimensions on the Web and mobile applications’ data storage that developers can benefit from building Distributed Applications (DApps) in a complete decentralised environment.
Neural Zero-Shot Fine-Grained Entity Typing
Authors: Yankun Ren, Jianbin Lin, Jun Zhou
Venue: WWW
Year: 2020

Fine-grained entity typing is a task to assign types to entity mentions dependent on mentions’ context. Due to the heavy work of human annotation, high quality training data is always not enough, zero-shot fine-grained entity typing becomes important. Previous zero-shot works rely on hand-crafted features and suffers from noisy distant supervision induced training data. In this paper, we propose a Neural Zero-Shot Fine-Grained Entity Typing (NZFET) model. NZFET is an end-to-end neural model free from hand-crafted features. The entity type attention in NZFET makes model focus on information relevent to the entity type. In our experiments, NZFET obtains better results on popular datasets than previous works. And we show experimentally that our method is robust against noisy training data.
ResQueue: A Smarter Datacenter Flow Scheduler
Authors: Hamed Rezaei, Balajee Vamanan
Venue: WWW
Year: 2020

Datacenters host a mix of applications: foreground applications perform distributed lookups in order to service user queries and background applications perform batch processing tasks such as data reorganization, backup, and replication. While background flows produce the most load, foreground applications produce the most number of flows. Because packets from both types of applications compete at switches for network bandwidth, the performance of applications is sensitive to scheduling mechanisms. Existing schedulers use flow size to distinguish critical flows from non-critical flows. However, recent studies on datacenter workloads reveal that most flows are small (e.g., most flows consist of only a handful number of packets). In light of recent findings, we make the key observation that because most flows are small, flow size is not sufficient to distinguish critical flows from non-critical flows and therefore existing flow schedulers do not achieve the desired prioritization. In this paper, we introduce ResQueue, which uses a combination of flow size and packet history to calculate the priority of each flow. Our evaluation shows that ResQueue improves tail flow completion times of short flows by up to 60% over the state-of-the-art flow scheduling mechanisms.
Unsupervised Dual-Cascade Learning with Pseudo-Feedback Distillation for Query-Focused Extractive Summarization
Authors: Haggai Roitman, Guy Feigenblat, Doron Cohen, Odellia Boni, David Konopnicki
Venue: WWW
Year: 2020

We propose Dual-CES – a novel unsupervised, query-focused, multi-document extractive summarizer. Dual-CES builds on top of the Cross Entropy Summarizer (CES) and is designed to better handle the tradeoff between saliency and focus in summarization. To this end, Dual-CES employs a two-step dual-cascade optimization approach with saliency-based pseudo-feedback distillation. Overall, Dual-CES significantly outperforms all other state-of-the-art unsupervised alternatives. Dual-CES is even shown to be able to outperform strong supervised summarizers.
Fast Hierarchical Graph Clustering in Linear-Time
Authors: Ryan A. Rossi, Nesreen K. Ahmed, Eunyee Koh, Sungchul Kim
Venue: WWW
Year: 2020

While there has been a lot of research on graph clustering (community detection), most work (i) does not address the hierarchical community detection problem or are (ii) inefficient for large networks. In this work, we describe an approach called hLP that addresses both these limitations. Notably, hLP is fast and efficient for discovering a hierarchy of communities in large networks with a worst-case time and space complexity that is linear in the number of edges and nodes, respectively. The experiments demonstrate the effectiveness of hLP. Finally, we show an application for visualizing large networks with hundreds of thousands of nodes and edges.
Fast and Accurate Estimation of Typed Graphlets
Authors: Ryan A. Rossi, Anup Rao, Tung Mai, Nesreen K. Ahmed
Venue: WWW
Year: 2020

Typed graphlets are small typed (labeled, colored) induced subgraphs and were recently shown to be the fundamental building blocks of rich complex heterogeneous networks. In many applications, speed is more important than accuracy, and it is sufficient to trade-off a tiny amount of accuracy for a significantly faster method. In this work, we propose fast and accurate estimators for typed graphlets. The typed graphlet estimation techniques naturally support general heterogeneous graphs with any arbitrary number of types, which include bipartite, k-partite, k-star, labeled graphs, and attributed networks as special cases. The experiments demonstrate the effectiveness of the typed graphlet estimation techniques.
Wikimedia Public (Research) Resources
Authors: Diego Sáez-Trumper, Miriam Redi
Venue: WWW
Year: 2020

The Wikimedia Foundation’s mission is to disseminate open knowledge effectively and globally. In keeping with this mission, we support research in areas that benefit the free knowledge community. We aim to make any work with our support openly available to the public. At the same time that we do a minimalist user data collection, all the material (text and multimedia) available in our projects is public and reusable by everybody under a creative commons license. Moreover, all the article’s history and interactions among users are also public, and we offer a set of tools for accessing such data. In this tutorial we are going to give an overview on all the data sources, and a detailed explanation of how to interact with this content including data and tools such as the Wikipedia Dumps, Quarry (SQL Replicas), Pageviews, PAWS (Wikimedia hosted Jupyter Notebooks), WikiData and put special attention to the multimedia content hosted by Wikimedia Commons project.
Dirty Clicks: A Study of the Usability and Security Implications of Click-related Behaviors on the Web
Authors: Iskander Sánchez-Rola, Davide Balzarotti, Christopher Kruegel, Giovanni Vigna, Igor Santos
Venue: WWW
Year: 2020

Web pages have evolved into very complex dynamic applications, which are often very opaque and difficult for non-experts to understand. At the same time, security researchers push for more transparent web applications, which can help users in taking important security-related decisions about which information to disclose, which link to visit, and which online service to trust. In this paper, we look at one of the simplest but also most representative aspect that captures the struggle between these opposite demands: a mouse click. In particular, we present the first comprehensive study of the possible security and privacy implications that clicks can have from a user perspective, analyzing the disconnect that exists between what is shown to users and what actually happens after. We started by identifying and classifying possible problems. We then implemented a crawler that performed nearly 2.5M clicks looking for signs of misbehavior. We analyzed all the interactions created as a result of those clicks, and discovered that the vast majority of domains are putting users at risk by either obscuring the real target of links or by not providing sufficient information for users to make an informed decision. We conclude the paper by proposing a set of countermeasures.
Investigating Cash Bail Patterns in Pennsylvania
Authors: Roshan Santhosh, Shweta Chopra, Nupur Baghel, Yi-Nung Huang, Marc Rauckhorst, Shubham Annadate
Venue: WWW
Year: 2020

A key component of pretrial justice reform has been to overhaul the cash bail system, which has been known to unfairly and disproportionately affect poor communities of color. The lack of clear standards and the variability in decision making by magistrate judges has resulted in a system that is inconsistent in determining who must pay cash bail as well as the amount of bail. Together with the lack of any metrics or transparency associated with this process, the problem has escalated. The American Civil Liberties Union (ACLU) has been at the forefront of this fight to reform the bail determination system. To support their efforts, we utilized a dataset containing all criminal offenses charged within the state of Pennsylvania in 2016-17 in order to derive analytical insights, and perform a series of statistical tests to evaluate and quantify biases in the bail determination system in Pennsylvania. In addition to this, we also developed a predictive model and utilised model interpretability tools to evaluate the factors affecting bail type assignment. One of the key focus areas of our work was the determination of racial bias, particularly between the defendants identified as Black and White in the court records. Through our work, we were able to develop a pipeline for identifying racial bias and conclusively establish that bail assignment patterns do vary across judges.
Citizens as More Than Sensors, Citizens as Agents for Change
Authors: Saiph Savage
Venue: WWW
Year: 2020

How might we tap into citizens’ knowledge, context, and passion on social media to have them be much more than just sensors, but rather, agents of change? What would it take to be able to orchestrate citizens on social media to execute ambitious goals, e.g., creating new infrastructure to support the indoor navigation of the visually impaired? The main challenge that is blocking these type of interactions at scale is that citizens have heterogeneous knowledge and usually lack the skills needed to complete the work needed to reach the collective goals. To address this problem, I introduce a novel architecture conformed of Intelligent Civic Blocks that use machine learning to: 1) help citizens on social media to build at execution time the skills they need to complete work associated with the large-scale collective effort; 2) empower citizens to develop their creativity and entrepreneurship to propose creative solutions to build improved societies. In this talk, I will present case-studies showcasing how this proposed architecture can lead to collective action organized on social media in a range of areas (see Fig. 1 for an example). In difference to prior work that focused primarily on using citizens as simple sensors, this research pushes towards new intelligent systems that enable the orchestration of collective action on social media to create improved societies.
Becoming the Super Turker: Increasing Wages via a Strategy from High Earning Workers
Authors: Saiph Savage, Chun-Wei Chiang, Susumu Saito, Carlos Toxtli, Jeffrey P. Bigham
Venue: WWW
Year: 2020

Crowd markets have traditionally limited workers by not providing transparency information concerning which tasks pay fairly or which requesters are unreliable. Researchers believe that a key reason why crowd workers earn low wages is due to this lack of transparency. As a result, tools have been developed to provide more transparency within crowd markets to help workers. However, while most workers use these tools, they still earn less than minimum wage. We argue that the missing element is guidance on how to use transparency information. In this paper, we explore how novice workers can improve their earnings by following the transparency criteria of Super Turkers, i.e., crowd workers who earn higher salaries on Amazon Mechanical Turk (MTurk). We believe that Super Turkers have developed effective processes for using transparency information. Therefore, by having novices follow a Super Turker criteria (one that is simple and popular among Super Turkers), we can help novices increase their wages. For this purpose, we: (i) conducted a survey and data analysis to computationally identify a simple yet common criteria that Super Turkers use for handling transparency tools; (ii) deployed a two-week field experiment with novices who followed this Super Turker criteria to find better work on MTurk. Novices in our study viewed over 25,000 tasks by 1,394 requesters. We found that novices who utilized this Super Turkers’ criteria earned better wages than other novices. Our results highlight that tool development to support crowd workers should be paired with educational opportunities that teach workers how to effectively use the tools and their related metrics (e.g., transparency values). We finish with design recommendations for empowering crowd workers to earn higher salaries.
Mitigating Misinformation in Online Social Network with Top-k Debunkers and Evolving User Opinions
Authors: Akrati Saxena, Wynne Hsu, Mong-Li Lee, Hai Leong Chieu, Lynette Ng, Loo-Nin Teow
Venue: WWW
Year: 2020

Online social networks provide an easy platform to share the information, and the spread of fake news and rumors has become prevalent, with severe consequences on major events including the US and Jakarta elections. Existing works have designed methods to find a set of top-k users to launch truth-campaigns and mitigate the negative influence of misinformation. The assumption is that these top-k users are open and willing to disseminate fact-checked content. Further, these methods assume that as misinformation and counter messages propagate in the network, user opinions once formed do not change. In this work, we address a more realistic scenario where users’ opinion can fluctuate before some deadline, and the goal is to find a good seed set of users from a set of debunkers to minimize the impact of misinformation. We propose a new opinion model that takes into account users’ biases and their social neighbours’ opinions. Based on this model, we design a mitigation solution to identify a subset of debunkers that maximizes the number of users who have been exposed to the misinformation but choose to believe the counter message. Experiments on Facebook and Twitter datasets demonstrate that our proposed solution is effective in mitigating the negative influences of misinformation.
Sampling Query Variations for Learning to Rank to Improve Automatic Boolean Query Generation in Systematic Reviews
Authors: Harrisen Scells, Guido Zuccon, Mohamed A. Sharaf, Bevan Koopman
Venue: WWW
Year: 2020

Searching medical literature for synthesis in a systematic review is a complex and labour intensive task. In this context, expert searchers construct lengthy Boolean queries. The universe of possible query variations can be massive: a single query can be composed of hundreds of field-restricted search terms/phrases or ontological concepts, each grouped by a logical operator nested to depths of sometimes five or more levels deep. With the many choices about how to construct a query, it is difficult to both formulate and recognise effective queries. To address this challenge, automatic methods have recently been explored for generating and selecting effective Boolean query variations for systematic reviews. The limiting factor of these methods is that it is computationally infeasible to process all query variations for training the methods. To overcome this, we propose novel query variation sampling methods for training Learning to Rank models to rank queries. Our results show that query sampling methods do directly impact the ability of a Learning to Rank model to effectively identify good query variations. Thus, selecting appropriate query sampling methods is a key problem for the automatic reformulation of effective Boolean queries for systematic review literature search. We find that the best sampling strategies are those which balance the diversity of queries with the quantity of queries.