OpenRaiser/Intern-Atlas huggingface dataset:
"Answer ka type kya he?": Learning to Classify Questions in Code-Mixed Language
Authors: Khyathi Chandu Raghavi, Manoj Kumar Chinnakotla, Manish Shrivastava
Venue: WWW
Year: 2015

Code-Mixing (CM) is defined as the embedding of linguistic units such as phrases, words, and morphemes of one language into an utterance of another language. CM is a natural phenomenon observed in many multilingual societies. It helps in speeding-up communication and allows wider variety of expression due to which it has become a popular mode of communication in social media forums like Facebook and Twitter. However, current Question Answering (QA) research and systems only support expressing a question in a single language which is an unrealistic and hard proposition especially for certain domains like health and technology. In this paper, we take the first step towards the development of a full-fledged QA system in CM language which is building a Question Classification (QC) system. The QC system analyzes the user question and infers the expected Answer Type (AType). The AType helps in locating and verifying the answer as it imposes certain type-specific constraints. In this paper, we present our initial efforts towards building a full-fledged QA system for CM language. We learn a basic Support Vector Machine (SVM) based QC system for English-Hindi CM questions. Due to the inherent complexities involved in processing CM language and also the unavailability of language processing resources such POS taggers, Chunkers, Parsers, we design our current system using only word-level resources such as language identification, transliteration and lexical translation. To reduce data sparsity and leverage resources available in a resource-rich language, in stead of extracting features directly from the original CM words, we translate them commonly into English and then perform featurization. We created an evaluation dataset for this task and our system achieves an accuracy of 63% and 45% in coarse-grained and fine-grained categories of the question taxanomy. The idea of translating features into English indeed helps in improving accuracy over the unigram baseline.
Linking a Community Platform to the Linked Open Data Cloud
Authors: Enayat Rajabi, Ivana Marenzi
Venue: WWW
Year: 2015

Linked Data promises access to a vast amount of resources for learners and teachers. Various research projects have focused on providing educational resources as Linked Data. In many of these projects the focus has been on interoperability of metadata and on linking them into the linked data cloud. In this paper we focus on the community aspect. We start from the observation that sharing data is most valuable within communities of practice with common interests and goals, and community members are interested in suitable resources to be used in specific learning scenarios. The community of practice we are focusing on is an English language teaching and learning community, which we have been supporting through the LearnWeb2.0 platform for the last two years. We analyse the requirements of this specific community as a basis to enrich the current collected materials with open educational resources taken from the Linked Data Cloud. To this aim, we performed an interlinking approach in order to enrich the learning resources exposed as RDF (Resource Description Framework) in the LearnWeb2.0 platform with additional information taken from the Web.
Using Reference Groups to Assess Academic Productivity in Computer Science
Authors: Sabir Ribas, Berthier A. Ribeiro-Neto, Edmundo de Souza e Silva, Alberto Hideki Ueda, Nivio Ziviani
Venue: WWW
Year: 2015

In this paper we discuss the problem of how to assess academic productivity based on publication outputs. We are interested in knowing how well a research group in an area of knowledge is doing relatively to a pre-selected set of reference groups, where each group is composed by academics or researchers. To assess academic productivity we adopt a new metric we propose, which we call P-score. We use P-score, citation counts and H-Index to obtain rankings of researchers in Brazil. Experimental results using data from the area of Computer Science show that P-score outperforms citation counts and H-Index when assessed against the official ranking produced by the Brazilian National Research Council (CNPq). This is of our interest for two reasons. First, it suggests that citation-based metrics, despite wide adoption, can be improved upon. Second, contrary to citation-based metrics, the P-score metric does not require access to the content of publications to be computed.
Challenges of Forecasting and Measuring a Complex Networked World
Authors: Bruno Ribeiro
Venue: WWW
Year: 2015

A new era of data analytics of online social networks promises tremendous high-impact societal, business, and healthcare applications. As more users join online social networks, the data available for analysis and forecast of human social and collective behavior grows at an incredible pace. The first part of this talk introduces an apparent paradox, where larger online social networks entail more user data but also less analytic and forecasting capabilities [7]. More specifically, the paradox applies to forecasting properties of network processes such as network cascades, showing that in some scenarios unbiased long term forecasting becomes increasingly inaccurate as the network grows but, paradoxically, short term forecasting -- such as the predictions in Cheng et al. [2] and Ribeiro et al. [7] -- improves with network size. We discuss the theoretic foundations of this paradox and its connections with known information theoretic measures such as Shannon capacity. We also discuss the implications of this paradox on the scalability of big data applications and show how information theory tools -- such as Fisher information [3,8] -- can be used to design more accurate and scalable methods for network analytics [6,8,10]. The second part of the talk focuses on how these results impact our ability to perform network analytics when network data is only available through crawlers and the complete network topology is unknown [1,4,5,9].
Beyond Models: Forecasting Complex Network Processes Directly from Data
Authors: Bruno Ribeiro, Minh X. Hoang, Ambuj K. Singh
Venue: WWW
Year: 2015

Complex network phenomena -- such as information cascades in online social networks -- are hard to fully observe, model, and forecast. In forecasting, a recent trend has been to forgo the use of parsimonious models in favor of models with increasingly large degrees of freedom that are trained to learn the behavior of a process from historical data. Extrapolating this trend into the future, eventually we would renounce models all together. But is it possible to forecast the evolution of a complex stochastic process directly from the data without a model? In this work we show that model-free forecasting is possible. We present SED, an algorithm that forecasts process statistics based on relationships of statistical equivalence using two general axioms and historical data. To the best of our knowledge, SED is the first method that can perform axiomatic, model-free forecasts of complex stochastic processes. Our simulations using simple and complex evolving processes and tests performed on a large real-world dataset show promising results.
An Urban Data Profiler
Authors: Daniel Castellani Ribeiro, Huy T. Vo, Juliana Freire, Cláudio T. Silva
Venue: WWW
Year: 2015

Large volumes of urban data are being made available through a variety of open portals. Besides promoting transparency, these data can bring benefits to government, science, citizens and industry. It is no longer a fantasy to ask "if you could know anything about a city, what do you want to know" and to ponder what could be done with that information. However, the great number and variety of datasets creates a new challenge: how to find relevant datasets. While existing portals provide search interfaces, these are often limited to keyword searches over the limited metadata associated each dataset, for example, attribute names and textual description. In this paper, we present a new tool, UrbanProfiler, that automatically extracts detailed information from datasets. This information includes attribute types, value distributions, and geographical information, which can be used to support complex search queries as well as visualizations that help users explore and obtain insight into the contents of a data collection. Besides describing the tool and its implementation, we present case studies that illustrate how the tool was used to explore a large open urban data repository.
Groupsourcing: Team Competition Designs for Crowdsourcing
Authors: Markus Rokicki, Sergej Zerr, Stefan Siersdorfer
Venue: WWW
Year: 2015

Many data processing tasks such as semantic annotation of images, translation of texts in foreign languages, and labeling of training data for machine learning models require human input, and, on a large scale, can only be accurately solved using crowd based online work. Recent work shows that frameworks where crowd workers compete against each other can drastically reduce crowdsourcing costs, and outperform conventional reward schemes where the payment of online workers is proportional to the number of accomplished tasks ("pay-per-task"). In this paper, we investigate how team mechanisms can be leveraged to further improve the cost efficiency of crowdsourcing competitions. To this end, we introduce strategies for team based crowdsourcing, ranging from team formation processes where workers are randomly assigned to competing teams, over strategies involving self-organization where workers actively participate in team building, to combinations of team and individual competitions. Our large-scale experimental evaluation with more than 1,100 participants and overall 5,400 hours of work spent by crowd workers demonstrates that our team based crowdsourcing mechanisms are well accepted by online workers and lead to substantial performance boosts.
#Microposts2015 - 5th Workshop on 'Making Sense of Microposts': Big things come in small packages
Authors: Matthew Rowe, Milan Stankovic, Aba-Sah Dadzie
Venue: WWW
Year: 2015

#Microposts2015, the 5th workshop on 'Making Sense of Microposts', is summarised by the sub-theme: 'big things come in small packages'. The workshop was borne out of research we were each carrying out as microblogging platforms became increasingly popular, and their value as a publishing platform and the data generated as a result began to be recognised. This phenomenon continues to grow, as microblogs provide a low-effort means of publishing information within private, but moreso public, fora, giving a voice to all in all arenas.
Probabilistic Deduplication of Anonymous Web Traffic
Authors: Rishiraj Saha Roy, Ritwik Sinha, Niyati Chhaya, Shiv Kumar Saini
Venue: WWW
Year: 2015

Cookies and log in-based authentication often provide incomplete data for stitching website visitors across multiple sources, necessitating probabilistic deduplication. We address this challenge by formulating the problem as a binary classification task for pairs of anonymous visitors. We compute visitor proximity vectors by converting categorical variables like IP addresses, product search keywords and URLs with very high cardinalities to continuous numeric variables using the Jaccard coefficient for each attribute. Our method achieves about 90% AUC and F-scores in identifying whether two cookies map to the same visitor, while providing insights on the relative importance of available features in Web analytics towards the deduplication process.
Online Searching in English as a Foreign Language
Authors: Gyöngyi Rózsa, Anita Komlodi, Peng Chu
Venue: WWW
Year: 2015

Online searching is a central element of internet users' information behaviors. Searching is usually executed in a user's native language, but searching in English as a foreign language is often necessitated by the lack of content in languages that are underrepresented in Web content. This paper reports results from a study of searching in English as a foreign language and aims at understanding this particular group of users' behaviors. Searchers whose native language is not English may have to resort to queries in English in support of their information needs due to the lack or low quality of the web content in their own language. However, when searching for information in a foreign language, users face a unique set of challenges that are not present for native language searching. We studied this problem through qualitative research methods and report results from focus groups in this paper. The results reported in this paper describe typical problems foreign language searchers face, the differences in information-seeking behavior in English and in the participants' native language, and advice and ideas shared by the focus group participants about how to search effectively and efficiently in English.
Exploring Supervised Methods for Temporal Link Prediction in Heterogeneous Social Networks
Authors: Nataliia Rümmele, Ryutaro Ichise, Hannes Werthner
Venue: WWW
Year: 2015

In the link prediction problem, formulated as a binary classification problem, we want to classify each pair of disconnected nodes in the network whether they will be connected by a link in the future. We study link formation in social networks with two types of links over several time periods. To solve the link prediction problem, we follow the approach of counting 3-node graphlets and suggest three extensions to the original method. By performing experiments on two real-world social networks we show that the new methods have a predictive power, however, network evolution cannot be explained by one specific feature at all time points. We also observe that some network properties can point at features which are more effective for temporal link prediction.
Authentication Melee: A Usability Analysis of Seven Web Authentication Systems
Authors: Scott Ruoti, Brent Roberts, Kent E. Seamons
Venue: WWW
Year: 2015

Passwords continue to dominate the authentication landscape in spite of numerous proposals to replace them. Even though usability is a key factor in replacing passwords, very few alternatives have been subjected to formal usability studies, and even fewer have been analyzed using a standard metric. We report the results of four within-subjects usability studies for seven web authentication systems. These systems span federated, smartphone, paper tokens, and email-based approaches. Our results indicate that participants prefer single sign-on systems. We report several insightful findings based on participants' qualitative responses: (1) transparency increases usability but also leads to confusion and a lack of trust, (2) participants prefer single sign-on but wish to augment it with site-specific low-entropy passwords, and (3) participants are intrigued by biometrics and phone-based authentication. We utilize the Systems Usability Scale (SUS) as a standard metric for empirical analysis and find that it produces reliable, replicable results. SUS proves to be an accurate measure of baseline usability. We recommend that new authentication systems be formally evaluated for usability using SUS, and should meet a minimum acceptable SUS score before receiving serious consideration.
A Hybrid Framework for Online Execution of Linked Data Queries
Authors: Mohamed M. Sabri
Venue: WWW
Year: 2015

Linked Data has been widely adopted over the last few years, with the size of the Linked Data cloud almost doubling every year. However, there is still no well-defined, efficient mechanism for querying such a Web of Data. We propose a framework that incorporates a set of optimizations to tackle various limitations in the state-of-the-art. The framework aims at combining the centralized query optimization capabilities of the data warehouse-based approaches with the result freshness and explorative data source discovery capabilities of link-traversal approaches. This is achieved by augmenting base-line link-traversal query execution with a set of optimization techniques. The proposed optimizations fall under two categories: metadata-based optimizations and semantics-based optimizations.
From Script Idea to TV Rerun: The Idea of Linked Production Data in the Media Value Chain
Authors: Harald Sack
Venue: WWW
Year: 2015

Within the process of the production of a film or tv program a significant amount of metadata is created and - most times - lost again. As a consequence most of this valuable information has to be costly recreated in subsequent steps of media production, distribution, and archival. On the other hand, there is no commonly used metadata exchange format throughout all steps of the media value chain. Furthermore, technical systems and software applications used in the media production process often have proprietary interfaces for data exchange. In the course of the D-Werft project funded by the German government, metadata exchange through all steps of the media value chain is to be fostered by the application of Linked Data principles. Starting with the idea for a script, metadata from existing systems and applications will be mapped to ontologies to be reused in subsequent production steps. Also for distribution and archival, metadata collected during the production process is a valuable asset to be reused for semantic and exploratory search as well as for intelligent movie recommendation and customized advertising.
Twitter Floods when it Rains: A Case Study of the UK Floods in early 2014
Authors: Antonia Saravanou, George Valkanas, Dimitrios Gunopulos, Gennady L. Andrienko
Venue: WWW
Year: 2015

Twitter is one of the most prominent social media platforms nowadays. A primary reason that has brought the medium at the spotlight of academic attention is its real-time nature, with people constantly uploading information regarding their surroundings. This trait, coupled with the service's data access policy for researchers and developers, has allowed the community to explore Twitter's potential as a news reporting tool. Finding out promptly about newsworthy events can prove extremely useful in crisis management situations. In this paper, we explore the use of Twitter as a mechanism used in disaster relief, and consequently in public safety. In particular, we perform a case study on the floods that occurred in the United Kingdom during January 2014, and how these were reflected on Twitter, according to tweets (i.e., posts) submitted by the users. We present a systematic algorithmic analysis of tweets collected with respect to our use case scenario, supplemented by visual analytic tools. Our objective is to identify meaningful and effective ways to take advantage of the wealth of Twitter data in crisis management, and we report on the findings of our analysis.
Predicting Pinterest: Organising the World's Images with Human-machine Collaboration
Authors: Nishanth Sastry
Venue: WWW
Year: 2015

The user generated content revolution has created a glut of multimedia content online -- from Flickr to Facebook, new images are being made available for public consumption everyday. In this talk, we will first explore how, on sites such as Pinterest, users are bringing order to this burgeoning collection by manually curating collections of images in ways that are highly personalised and relevant to their own use. We will then discuss the phenomenon of social bootstrapping, whereby existing mature social networks such as Facebook are helping bootstrap engaged communities of content curators on external sites such as Pinterest. Finally, we will demonstrate how the manual effort involved in curation can be amplified using a unique human-machine collaboration: By treating the curation efforts of a subset of users on Pinterest as a distributed human computation over a low-dimensional approximation of the content corpus, we derive simple yet powerful signals, which, when combined with image-related features drawn from state-of-the-art deep learning techniques, allow us to automatically and accurately populate the personalised curated collections of all other users.
What's in this paper?: Combining Rhetorical Entities with Linked Open Data for Semantic Literature Querying
Authors: Bahar Sateli, René Witte
Venue: WWW
Year: 2015

Finding research literature pertaining to a task at hand is one of the essential tasks that scientists face on daily basis. Standard information retrieval techniques allow to quickly obtain a vast number of potentially relevant documents. Unfortunately, the search results then require significant effort for manual inspection, where we would rather select relevant publications based on more fine-grained, semantically rich queries involving a publication's contributions, methods, or application domains. We argue that a novel combination of three distinct methods can significantly advance this vision: (i) Natural Language Processing (NLP) for Rhetorical Entity (RE) detection; (ii) Named Entity (NE) recognition based on the Linked Open Data (LOD) cloud; and (iii) automatic generation of RDF triples for both NEs and REs using semantic web ontologies to interconnect them. Combined in a single workflow, these techniques allow us to automatically construct a knowledge base that facilitates numerous advanced use cases for managing scientific documents.
Pushing the Limits of Instance Matching Systems: A Semantics-Aware Benchmark for Linked Data
Authors: Tzanina Saveta, Evangelia Daskalaki, Giorgos Flouris, Irini Fundulaki, Melanie Herschel, Axel-Cyrille Ngonga Ngomo
Venue: WWW
Year: 2015

The architectural choices behind the Data Web have led to the publication of large interrelated data sets that contain different descriptions for the same real-world objects. Due to the mere size of current online datasets, such duplicate instances are most commonly detected (semi-)automatically using instance matching frameworks. Choosing the right framework for this purpose remains tedious, as current instance matching benchmarks fail to provide end users and developers with the necessary insights pertaining to how current frameworks behave when dealing with real data. In this poster, we present the Semantic Publishing Instance Matching Benchmark (SPIMBENCH) which allows the benchmarking of instance matching systems against not only structure-based and value-based test cases, but also against semantics-aware test cases based on OWL axioms. SPIMBENCH features a scalable data generator and a weighted gold standard that can be used for debugging instance matching systems and for reporting how well they perform in various matching tasks.
Cloud WorkBench: Benchmarking IaaS Providers based on Infrastructure-as-Code
Authors: Joel Scheuner, Jürgen Cito, Philipp Leitner, Harald C. Gall
Venue: WWW
Year: 2015

Optimizing the deployment of applications in Infrastructure-as-a-Service clouds requires to evaluate the costs and performance of different combinations of cloud configurations which is unfortunately a cumbersome and error-prone process. In this paper, we present Cloud WorkBench (CWB), a concrete implementation of a cloud benchmarking Web service, which fosters the definition of reusable and representative benchmarks. We demonstrate the complete cycle of benchmarking an IaaS service with the sample benchmark SysBench. In distinction to existing work, our system is based on the notion of Infrastructure-as-Code, which is a state of the art concept to define IT infrastructure in a reproducible, well-defined, and testable way.
Unbiased Ranking Evaluation on a Budget
Authors: Tobias Schnabel, Adith Swaminathan, Thorsten Joachims
Venue: WWW
Year: 2015

We address the problem of assessing the quality of a ranking system (e.g., search engine, recommender system, review ranker) given a fixed budget for collecting expert judgments. In particular, we propose a method that selects which items to judge in order to optimize the accuracy of the quality estimate. Our method is not only efficient, but also provides estimates that are unbiased --- unlike common approaches that tend to underestimate performance or that have a bias against new systems that are evaluated re-using previous relevance scores.