pwc-archive/papers-with-abstracts huggingface dataset:
Exploiting Sentence-Level Representations for Passage Ranking
Authors: Jurek Leonhardt, Fabian Beringer, Avishek Anand
Date: 2021-06-14 00:00:00
Tasks: Open-Domain Question Answering, Passage Ranking, Passage Re-Ranking, Question Answering, Re-Ranking, Sentence

Recently, pre-trained contextual models, such as BERT, have shown to perform well in language related tasks. We revisit the design decisions that govern the applicability of these models for the passage re-ranking task in open-domain question answering. We find that common approaches in the literature rely on fine-tuning a pre-trained BERT model and using a single, global representation of the input, discarding useful fine-grained relevance signals in token- or sentence-level representations. We argue that these discarded tokens hold useful information that can be leveraged. In this paper, we explicitly model the sentence-level representations by using Dynamic Memory Networks (DMNs) and conduct empirical evaluation to show improvements in passage re-ranking over fine-tuned vanilla BERT models by memory-enhanced explicit sentence modelling on a diverse set of open-domain QA datasets. We further show that freezing the BERT model and only training the DMN layer still comes close to the original performance, while improving training efficiency drastically. This indicates that the usual fine-tuning step mostly helps to aggregate the inherent information in a single output token, as opposed to adapting the whole model to the new task, and only achieves rather small gains.
Deep Reinforcement Learning based Group Recommender System
Authors: Zefang Liu, Shuran Wen, Yinzhu Quan
Date: 2021-06-13 00:00:00
Tasks: Deep Reinforcement Learning, Recommendation Systems, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Group recommender systems are widely used in current web applications. In this paper, we propose a novel group recommender system based on the deep reinforcement learning. We introduce the MovieLens data at first and generate one random group dataset, MovieLens-Rand, from it. This randomly generated dataset is described and analyzed. We also present experimental settings and two state-of-art baselines, AGREE and GroupIM. The framework of our novel model, the Deep Reinforcement learning based Group Recommender system (DRGR), is proposed. Actor-critic networks are implemented with the deep deterministic policy gradient algorithm. The DRGR model is applied on the MovieLens-Rand dataset with two baselines. Compared with baselines, we conclude that DRGR performs better than GroupIM due to long interaction histories but worse than AGREE because of the self-attention mechanism. We express advantages and shortcomings of DRGR and also give future improvement directions at the end.
Curriculum Pre-Training Heterogeneous Subgraph Transformer for Top-$N$ Recommendation
Authors: Hui Wang, Kun Zhou, Wayne Xin Zhao, Jingyuan Wang, Ji-Rong Wen
Date: 2021-06-12 00:00:00
Tasks: Recommendation Systems

Due to the flexibility in modelling data heterogeneity, heterogeneous information network (HIN) has been adopted to characterize complex and heterogeneous auxiliary data in top-$N$ recommender systems, called \emph{HIN-based recommendation}. HIN characterizes complex, heterogeneous data relations, containing a variety of information that may not be related to the recommendation task. Therefore, it is challenging to effectively leverage useful information from HINs for improving the recommendation performance. To address the above issue, we propose a Curriculum pre-training based HEterogeneous Subgraph Transformer (called \emph{CHEST}) with new \emph{data characterization}, \emph{representation model} and \emph{learning algorithm}. Specifically, we consider extracting useful information from HIN to compose the interaction-specific heterogeneous subgraph, containing both sufficient and relevant context information for recommendation. Then we capture the rich semantics (\eg graph structure and path semantics) within the subgraph via a heterogeneous subgraph Transformer, where we encode the subgraph with multi-slot sequence representations. Besides, we design a curriculum pre-training strategy to provide an elementary-to-advanced learning process, by which we smoothly transfer basic semantics in HIN for modeling user-item interaction relation. Extensive experiments conducted on three real-world datasets demonstrate the superiority of our proposed method over a number of competitive baselines, especially when only limited training data is available.
MoParkeR : Multi-objective Parking Recommendation
Authors: Mohammad Saiedur Rahaman, Wei Shao, Flora D. Salim, Ayad Turky, Andy Song, Jeffrey Chan, Junliang Jiang, Doug Bradbrook
Date: 2021-06-10 00:00:00
Tasks: Recommendation Systems

Existing parking recommendation solutions mainly focus on finding and suggesting parking spaces based on the unoccupied options only. However, there are other factors associated with parking spaces that can influence someone's choice of parking such as fare, parking rule, walking distance to destination, travel time, likelihood to be unoccupied at a given time. More importantly, these factors may change over time and conflict with each other which makes the recommendations produced by current parking recommender systems ineffective. In this paper, we propose a novel problem called multi-objective parking recommendation. We present a solution by designing a multi-objective parking recommendation engine called MoParkeR that considers various conflicting factors together. Specifically, we utilise a non-dominated sorting technique to calculate a set of Pareto-optimal solutions, consisting of recommended trade-off parking spots. We conduct extensive experiments using two real-world datasets to show the applicability of our multi-objective recommendation methodology.
Predicting the Popularity of Reddit Posts with AI
Authors: Juno Kim
Date: 2021-06-08 00:00:00
Tasks: BIG-bench Machine Learning, regression

Social media creates crucial mass changes, as popular posts and opinions cast a significant influence on users' decisions and thought processes. For example, the recent Reddit uprising inspired by r/wallstreetbets which had remarkable economic impact was started with a series of posts on the thread. The prediction of posts that may have a notable impact will allow for the preparation of possible following trends. This study aims to develop a machine learning model capable of accurately predicting the popularity of a Reddit post. Specifically, the model is predicting the number of upvotes a post will receive based on its textual content. I experimented with three different models: a baseline linear regression model, a random forest regression model, and a neural network. I collected Reddit post data from an online data set and analyzed the model's performance when trained on a single subreddit and a collection of subreddits. The results showed that the neural network model performed the best when the loss of the models were compared. With the use of a machine learning model to predict social trends through the reaction users have to post, a better picture of the near future can be envisioned.
Network-based Topic Interaction Map for Big Data Mining of COVID-19 Biomedical Literature
Authors: Yeseul Jeon, Dongjun Chung, Jina Park, Ick Hoon Jin
Date: 2021-06-07 00:00:00

Since the emergence of the worldwide pandemic of COVID-19, relevant research has been published at a dazzling pace, which yields an abundant amount of big data in biomedical literature. Due to the high volum of relevant literature, it is practically impossible to follow up the research manually. Topic modeling is a well-known unsupervised learning that aims to reveal latent topics from text data. In this paper, we propose a novel analytical framework for estimating topic interactions and effective visualization to improve topics' relationships. We first estimate topic-word distributions using the biterm topic model and estimate the topics' interaction based on the word distribution using the latent space item response model. We mapped these latent topics onto networks to visualize relationships among the topics. Moreover, in the proposed approach, we developed a score that is helpful in selecting meaningful words that characterize the topic. We figure out how topics are related by looking at how their relationships change. We do this with a "trajectory plot" that is made with different levels of word richness. These findings provide a thoroughly mined and intuitive representation of relationships between topics related to a specific research area. The application of this proposed framework to the PubMed literature demonstrates utility of our approach in understanding of the topic composition related to COVID-19 studies in the stage of its emergence.
Cognitive-aware Short-text Understanding for Inferring Professions
Authors: Sayna Esmailzadeh, Saeid Hosseini, Mohammad Reza Kangavari, Wen Hua
Date: 2021-06-04 00:00:00

Leveraging short-text contents to estimate the occupation of microblog authors has significant gains in many applications. Yet challenges abound. Firstly brief textual contents come with excessive lexical noise that makes the inference problem challenging. Secondly, cognitive-semantics are not evident, and important linguistic features are latent in short-text contents. Thirdly, it is hard to measure the correlation between the cognitive short-text semantics and the features pertaining various occupations. We argue that the multi-aspect cognitive features are needed to correctly associate short-text contents to a particular job and discover suitable people for the careers. To this end, we devise a novel framework that on the one hand, can infer short-text contents and exploit cognitive features, and on the other hand, fuses various adopted novel algorithms, such as curve fitting, support vector, and boosting modules to better predict the occupation of the authors. The final estimation module manufactures the $R^w$-tree via coherence weight to tune the best outcome in the inferring process. We conduct comprehensive experiments on real-life Twitter data. The experimental results show that compared to other rivals, our cognitive multi-aspect model can achieve a higher performance in the career estimation procedure, where it is inevitable to neglect the contextual semantics of users.
MexPub: Deep Transfer Learning for Metadata Extraction from German Publications
Authors: Zeyd Boukhers, Nada Beili, Timo Hartmann, Prantik Goswami, Muhammad Arslan Zafar
Date: 2021-06-04 00:00:00
Tasks: Transfer Learning

Extracting metadata from scientific papers can be considered a solved problem in NLP due to the high accuracy of state-of-the-art methods. However, this does not apply to German scientific publications, which have a variety of styles and layouts. In contrast to most of the English scientific publications that follow standard and simple layouts, the order, content, position and size of metadata in German publications vary greatly among publications. This variety makes traditional NLP methods fail to accurately extract metadata from these publications. In this paper, we present a method that extracts metadata from PDF documents with different layouts and styles by viewing the document as an image. We used Mask R-CNN that is trained on COCO dataset and finetuned with PubLayNet dataset that consists of ~200K PDF snapshots with five basic classes (e.g. text, figure, etc). We refine-tuned the model on our proposed synthetic dataset consisting of ~30K article snapshots to extract nine patterns (i.e. author, title, etc). Our synthetic dataset is generated using contents in both languages German and English and a finite set of challenging templates obtained from German publications. Our method achieved an average accuracy of around $90\%$ which validates its capability to accurately extract metadata from a variety of PDF documents with challenging templates.
Mixture of Virtual-Kernel Experts for Multi-Objective User Profile Modeling
Authors: Zhenhui Xu, Meng Zhao, Liqun Liu, Lei Xiao, Xiaopeng Zhang, Bifeng Zhang
Date: 2021-06-04 00:00:00
Tasks: Recommendation Systems, TAG

In industrial applications like online advertising and recommendation systems, diverse and accurate user profiles can greatly help improve personalization. Deep learning is widely applied to mine expressive tags to users from their historical interactions in the system, e.g., click, conversion action in the advertising chain. The usual approach is to take a certain action as the objective, and introduce multiple independent Two-Tower models to predict the possibility of users' action on tags (known as CTR or CVR prediction). The predicted users' high probably attractive tags are to represent their preferences. However, the single-action models cannot learn complementarily and support effective training on data-sparse actions. Besides, limited by the lack of information fusion between the two towers, the model learns insufficiently to represent users' preferences on various tag \textbf{topics} well. This paper introduces a novel multi-task model called Mixture of Virtual-Kernel Experts (MVKE) to learn user preferences on various actions and topics unitedly. In MVKE, we propose a concept of Virtual-Kernel Expert, which focuses on modeling one particular facet of the user's preferences, and all of them learn coordinately. Besides, the gate-based structure used in MVKE builds an information fusion bridge between two towers, improving the model's capability and maintaining high efficiency. We apply the model in Tencent Advertising System, where both online and offline evaluations show that our method has a significant improvement compared with the existing ones and brings about an obvious lift to actual advertising revenue.
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network
Authors: Nicholas FitzGerald, Jan A. Botha, Daniel Gillick, Daniel M. Bikel, Tom Kwiatkowski, Andrew McCallum
Date: 2021-06-02 00:00:00
Proceeding: ACL 2021 5
Tasks: Entity Linking, Entity Retrieval, Retrieval

We present an instance-based nearest neighbor approach to entity linking. In contrast to most prior entity retrieval systems which represent each entity with a single vector, we build a contextualized mention-encoder that learns to place similar mentions of the same entity closer in vector space than mentions of different entities. This approach allows all mentions of an entity to serve as "class prototypes" as inference involves retrieving from the full set of labeled entity mentions in the training set and applying the nearest mention neighbor's entity label. Our model is trained on a large multilingual corpus of mention pairs derived from Wikipedia hyperlinks, and performs nearest neighbor inference on an index of 700 million mentions. It is simpler to train, gives more interpretable predictions, and outperforms all other systems on two multilingual entity linking benchmarks.
Is it a click bait? Let's predict using Machine Learning
Authors: Sohom Ghosh
Date: 2021-06-01 00:00:00
Tasks: Articles, BIG-bench Machine Learning

In this era of digitisation, news reader tend to read news online. This is because, online media instantly provides access to a wide variety of content. Thus, people don't have to wait for tomorrow's newspaper to know what's happening today. Along with these virtues, online news have some vices as well. One such vice is presence of social media posts (tweets) relating to news articles whose sole purpose is to draw attention of the users rather than directing them to read the actual content. Such posts are referred to as clickbaits. The objective of this project is to develop a system which would be capable of predicting how likely are the social media posts (tweets) relating to new articles tend to be clickbait.
Zipf Matrix Factorization : Matrix Factorization with Matthew Effect Reduction
Authors: Hao Wang
Date: 2021-06-01 00:00:00
Tasks: Fairness, Learning-To-Rank, Recommendation Systems

Recommender system recommends interesting items to users based on users' past information history. Researchers have been paying attention to improvement of algorithmic performance such as MAE and precision@K. Major techniques such as matrix factorization and learning to rank are optimized based on such evaluation metrics. However, the intrinsic Matthew Effect problem poses great threat to the fairness of the recommender system, and the unfairness problem cannot be resolved by optimization of traditional metrics. In this paper, we propose a novel algorithm that incorporates Matthew Effect reduction with the matrix factorization framework. We demonstrate that our approach can boost the fairness of the algorithm and enhances performance evaluated by traditional metrics.
A Query-Driven Topic Model
Authors: Zheng Fang, Yulan He, Rob Procter
Date: 2021-05-28 00:00:00
Proceeding: Findings (ACL) 2021 8
Tasks: model, Topic Models

Topic modeling is an unsupervised method for revealing the hidden semantic structure of a corpus. It has been increasingly widely adopted as a tool in the social sciences, including political science, digital humanities and sociological research in general. One desirable property of topic models is to allow users to find topics describing a specific aspect of the corpus. A possible solution is to incorporate domain-specific knowledge into topic modeling, but this requires a specification from domain experts. We propose a novel query-driven topic model that allows users to specify a simple query in words or phrases and return query-related topics, thus avoiding tedious work from domain experts. Our proposed approach is particularly attractive when the user-specified query has a low occurrence in a text corpus, making it difficult for traditional topic models built on word cooccurrence patterns to identify relevant topics. Experimental results demonstrate the effectiveness of our model in comparison with both classical topic models and neural topic models.
Internal Wiring of Cartesian Verbs and Prepositions
Authors: Bob Coecke, Martha Lewis, Dan Marsden
Date: 2018-11-08 00:00:00

Categorical compositional distributional semantics (CCDS) allows one to compute the meaning of phrases and sentences from the meaning of their constituent words. A type-structure carried over from the traditional categorial model of grammar a la Lambek becomes a 'wire-structure' that mediates the interaction of word meanings. However, CCDS has a much richer logical structure than plain categorical semantics in that certain words can also be given an 'internal wiring' that either provides their entire meaning or reduces the size their meaning space. Previous examples of internal wiring include relative pronouns and intersective adjectives. Here we establish the same for a large class of well-behaved transitive verbs to which we refer as Cartesian verbs, and reduce the meaning space from a ternary tensor to a unary one. Some experimental evidence is also provided.
Intelligent Vision Based Wear Forecasting on Surfaces of Machine Tool Elements
Authors: Tobias Schlagenhauf, Niklas Burghardt
Date: 2021-06-12 00:00:00
Tasks: Defect Detection, Prognosis

This paper addresses the ability to enable machines to automatically detect failures on machine tool components as well as estimating the severity of the failures, which is a critical step towards autonomous production machines. Extracting information about the severity of failures has been a substantial part of classical, as well as Machine Learning based machine vision systems. Efforts have been undertaken to automatically predict the severity of failures on machine tool components for predictive maintenance purposes. Though, most approaches only partly cover a completely automatic system from detecting failures to the prognosis of their future severity. To the best of the authors knowledge, this is the first time a vision-based system for defect detection and prognosis of failures on metallic surfaces in general and on Ball Screw Drives in specific has been proposed. The authors show that they can do both, detect and prognose the evolution of a failure on the surface of a Ball Screw Drive.
Topological data analysis identifies emerging adaptive mutations in SARS-CoV-2
Authors: Michael Bleher, Lukas Hahn, Maximilian Neumann, Juan Angel Patino-Galindo, Mathieu Carriere, Ulrich Bauer, Raul Rabadan, Andreas Ott
Date: 2021-06-14 00:00:00
Tasks: Topological Data Analysis

The COVID-19 pandemic has initiated an unprecedented worldwide effort to characterize its evolution through the mapping of mutations of the coronavirus SARS-CoV-2. The early identification of mutations that could confer adaptive advantages to the virus, such as higher infectivity or immune evasion, is of paramount importance. However, the large number of currently available genomes precludes the efficient use of phylogeny-based methods. Here we present CoVtRec, a fast and scalable Topological Data Analysis approach for the surveillance of emerging adaptive mutations in large genomic datasets. Our method overcomes limitations of state-of-the-art phylogeny-based approaches by quantifying the potential adaptiveness of mutations merely by their topological footprint in the genome alignment, without resorting to the reconstruction of a single optimal phylogenetic tree. Analyzing millions of SARS-CoV-2 genomes from GISAID, we find a correlation between topological signals and adaptation to the human host. By leveraging the stratification by time in sequence data, our method enables the high-resolution longitudinal analysis of topological signals of adaptation. We characterize the convergent evolution of the coronavirus throughout the whole pandemic to date, report on emerging potentially adaptive mutations, and pinpoint mutations in Variants of Concern that are likely associated with positive selection. Our approach can improve the surveillance of mutations of concern and guide experimental studies.
A Cognitive Science perspective for learning how to design meaningful user experiences and human-centered technology
Authors: Sara Kingsley
Date: 2021-06-02 00:00:00

This paper reviews literature in cognitive science, human-computer interaction (HCI) and natural-language processing (NLP) to consider how analogical reasoning (AR) could help inform the design of communication and learning technologies, as well as online communities and digital platforms. First, analogical reasoning (AR) is defined, and use-cases of AR in the computing sciences are presented. The concept of schema is introduced, along with use-cases in computing. Finally, recommendations are offered for future work on using analogical reasoning and schema methods in the computing sciences.
Design and Experimental Assessment of Detection Schemes for Air Interface Attacks in Adverse Scenarios
Authors: Danilo Orlando, Ivan Palamà, Stefania Bartoletti, Giuseppe Bianchi, Nicola Blefari Melazzi
Date: 2021-06-14 00:00:00

In this letter, we propose three schemes designed to detect attacks over the air interface in cellular networks. These decision rules rely on the generalized likelihood ratio test, and are fed by data that can be acquired using common off-the-shelf receivers. In addition to more classical (barrage/smart) noise jamming attacks, we further assess the capability of the proposed schemes to detect the stealthy activation of a rogue base station. The evaluation is carried out through an experimentation of a LTE system concretely reproduced using Software-Defined Radios. Illustrative examples confirm that the proposed schemes can effectively detect air interface threats with high probability.