pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Spoken Utterance Classification
Authors: Shahab Jalalvand, Srinivas Bangalore
Date: 2021-07-02 00:00:00
Tasks: Classification, Sentence, Sentence Embedding, Sentence-Embedding, Spoken Language Understanding

An intelligent virtual assistant (IVA) enables effortless conversations in call routing through spoken utterance classification (SUC) which is a special form of spoken language understanding (SLU). Building a SUC system requires a large amount of supervised in-domain data that is not always available. In this paper, we introduce an unsupervised spoken utterance classification approach (USUC) that does not require any in-domain data except for the intent labels and a few para-phrases per intent. USUC is consisting of a KNN classifier (K=1) and a complex embedding model trained on a large amount of unsupervised customer service corpus. Among all embedding models, we demonstrate that Elmo works best for USUC. However, an Elmo model is too slow to be used at run-time for call routing. To resolve this issue, first, we compute the uni- and bi-gram embedding vectors offline and we build a lookup table of n-grams and their corresponding embedding vector. Then we use this table to compute sentence embedding vectors at run-time, along with back-off techniques for unseen n-grams. Experiments show that USUC outperforms the traditional utterance classification methods by reducing the classification error rate from 32.9% to 27.0% without requiring supervised data. Moreover, our lookup and back-off technique increases the processing speed from 16 utterances per second to 118 utterances per second.
Concept Identification of Directly and Indirectly Related Mentions Referring to Groups of Persons
Authors: Anastasia Zhukova, Felix Hamborg, Karsten Donnay, Bela Gipp
Date: 2021-07-02 00:00:00
Tasks: Articles, Clustering, Dimensionality Reduction, Entity Resolution

Unsupervised concept identification through clustering, i.e., identification of semantically related words and phrases, is a common approach to identify contextual primitives employed in various use cases, e.g., text dimension reduction, i.e., replace words with the concepts to reduce the vocabulary size, summarization, and named entity resolution. We demonstrate the first results of an unsupervised approach for the identification of groups of persons as actors extracted from a set of related articles. Specifically, the approach clusters mentions of groups of persons that act as non-named entity actors in the texts, e.g., "migrant families" = "asylum-seekers." Compared to our baseline, the approach keeps the mentions of the geopolitical entities separated, e.g., "Iran leaders" != "European leaders," and clusters (in)directly related mentions with diverse wording, e.g., "American officials" = "Trump Administration."
Learned Token Pruning for Transformers
Authors: Sehoon Kim, Sheng Shen, David Thorsley, Amir Gholami, Woosuk Kwon, Joseph Hassoun, Kurt Keutzer
Date: 2021-07-02 00:00:00
Tasks: Sentence

Deploying transformer models in practice is challenging due to their inference cost, which scales quadratically with input sequence length. To address this, we present a novel Learned Token Pruning (LTP) method which adaptively removes unimportant tokens as an input sequence passes through transformer layers. In particular, LTP prunes tokens with an attention score below a threshold value which is learned for each layer during training. Our threshold-based method allows the length of the pruned sequence to vary adaptively based on the input sequence, and avoids algorithmically expensive operations such as top-k token selection. We extensively test the performance of LTP on GLUE tasks and show that our method outperforms the prior state-of-the-art token pruning methods by up to ~2.5% higher accuracy with the same amount of FLOPs. In particular, LTP achieves up to 2.1x FLOPs reduction with less than 1% accuracy drop, which results in up to 1.9x and 2.0x throughput improvement on Intel Haswell CPUs and NVIDIA V100 GPUs, respectively. Furthermore, we demonstrate that LTP is more robust than prior methods to variations on input sentence lengths. Our code has been developed in PyTorch and has been open-sourced.
He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics
Authors: Nanjiang Jiang, Marie-Catherine de Marneffe
Date: 2021-07-02 00:00:00

We investigate how well BERT performs on predicting factuality in several existing English datasets, encompassing various linguistic constructions. Although BERT obtains a strong performance on most datasets, it does so by exploiting common surface patterns that correlate with certain factuality labels, and it fails on instances where pragmatic reasoning is necessary. Contrary to what the high performance suggests, we are still far from having a robust system for factuality prediction.
Interactive decoding of words from visual speech recognition models
Authors: Brendan Shillingford, Yannis Assael, Misha Denil
Date: 2021-07-01 00:00:00
Tasks: Position, speech-recognition, Speech Recognition, Visual Speech Recognition

This work describes an interactive decoding method to improve the performance of visual speech recognition systems using user input to compensate for the inherent ambiguity of the task. Unlike most phoneme-to-word decoding pipelines, which produce phonemes and feed these through a finite state transducer, our method instead expands words in lockstep, facilitating the insertion of interaction points at each word position. Interaction points enable us to solicit input during decoding, allowing users to interactively direct the decoding process. We simulate the behavior of user input using an oracle to give an automated evaluation, and show promise for the use of this method for text input.
A Primer on Pretrained Multilingual Language Models
Authors: Sumanth Doddapaneni, Gowtham Ramesh, Mitesh M. Khapra, Anoop Kunchukuttan, Pratyush Kumar
Date: 2021-07-01 00:00:00
Tasks: Joint Multilingual Sentence Representations, Multilingual text classification, Multilingual Word Embeddings, Pretrained Multilingual Language Models, Survey, Transfer Learning, XLM-R

Multilingual Language Models (\MLLMs) such as mBERT, XLM, XLM-R, \textit{etc.} have emerged as a viable option for bringing the power of pretraining to a large number of languages. Given their success in zero-shot transfer learning, there has emerged a large body of work in (i) building bigger \MLLMs~covering a large number of languages (ii) creating exhaustive benchmarks covering a wider variety of tasks and languages for evaluating \MLLMs~ (iii) analysing the performance of \MLLMs~on monolingual, zero-shot cross-lingual and bilingual tasks (iv) understanding the universal language patterns (if any) learnt by \MLLMs~ and (v) augmenting the (often) limited capacity of \MLLMs~ to improve their performance on seen or even unseen languages. In this survey, we review the existing literature covering the above broad areas of research pertaining to \MLLMs. Based on our survey, we recommend some promising directions of future research.
AC-SUM-GAN: Connecting Actor-Critic and Generative Adversarial Networks for Unsupervised Video Summarization
Authors: Evlampios Apostolidis, Eleni Adamantidou, Alexandros I. Metsai, Vasileios Mezaris, Ioannis Patras
Date: 2020-11-16 00:00:00
Proceeding: IEEE Transactions on Circuits and Systems for Video Technology 2020 11
Tasks: Generative Adversarial Network, Unsupervised Video Summarization, Video Summarization

This paper presents a new method for unsupervised video summarization. The proposed architecture embeds an Actor-Critic model into a Generative Adversarial Network and formulates the selection of important video fragments (that will be used to form the summary) as a sequence generation task. The Actor and the Critic take part in a game that incrementally leads to the selection of the video key-fragments, and their choices at each step of the game result in a set of rewards from the Discriminator. The designed training workflow allows the Actor and Critic to discover a space of actions and automatically learn a policy for key-fragment selection. Moreover, the introduced criterion for choosing the best model after the training ends, enables the automatic selection of proper values for parameters of the training process that are not learned from the data (such as the regularization factor σ). Experimental evaluation on two benchmark datasets (SumMe and TVSum) demonstrates that the proposed AC-SUM-GAN model performs consistently well and gives SoA results in comparison to unsupervised methods, that are also competitive with respect to supervised methods.
Structural biology in the clouds: The WeNMR-EOSC Ecosystem
Authors: Rodrigo Vargas Honorato, Panagiotis I. Koukos, Brian Jiménez-García, Andrei Tsaregorodtsev, Marco Verlato, Andrea Giachetti, Antonio Rosato, Alexandre M. J. J. Bonvin
Date: 2021-07-02 00:00:00
Tasks: CPU, Distributed Computing

Structural biology aims at characterizing the structural and dynamic properties of biological macromolecules at atomic details. Gaining insight into three dimensional structures of biomolecules and their interactions is critical for understanding the vast majority of cellular processes, with direct applications in health and food sciences. Since 2010, the WeNMR project (www.wenmr.eu) has implemented numerous web-based services to facilitate the use of advanced computational tools by researchers in the field, using the high throughput computing infrastructure provided by EGI. These services have been further developed in subsequent initiatives under H2020 projects and are now operating as Thematic Services in the European Open Science Cloud (EOSC) portal (www.eosc-portal.eu), sending >12 millions of jobs and using around 4000 CPU-years per year. Here we review 10 years of successful e-infrastructure solutions serving a large worldwide community of over 23,000 users to date, providing them with user-friendly, web-based solutions that run complex workflows in structural biology. The current set of active WeNMR portals are described, together with the complex backend machinery that allows distributed computing resources to be harvested efficiently.
From Epidemic to Pandemic Modelling
Authors: Shannon Connolly, David Gilbert, Monika Heiner
Date: 2021-07-01 00:00:00

We present a methodology for systematically extending epidemic models to multilevel and multiscale spatio-temporal pandemic ones. Our approach builds on the use of coloured stochastic and continuous Petri nets facilitating the sound component-based extension of basic SIR models to include population stratification and also spatio-geographic information and travel connections, represented as graphs, resulting in robust stratified pandemic metapopulation models. This method is inherently easy to use, producing scalable and reusable models with a high degree of clarity and accessibility which can be read either in a deterministic or stochastic paradigm. Our method is supported by a publicly available platform PetriNuts; it enables the visual construction and editing of models; deterministic, stochastic and hybrid simulation as well as structural and behavioural analysis. All the models are available as supplementary material, ensuring reproducibility.
MegazordNet: combining statistical and machine learning standpoints for time series forecasting
Authors: Angelo Garangau Menezes, Saulo Martiello Mastelini
Date: 2021-06-23 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Time Series, Time Series Analysis, Time Series Forecasting

Forecasting financial time series is considered to be a difficult task due to the chaotic feature of the series. Statistical approaches have shown solid results in some specific problems such as predicting market direction and single-price of stocks; however, with the recent advances in deep learning and big data techniques, new promising options have arises to tackle financial time series forecasting. Moreover, recent literature has shown that employing a combination of statistics and machine learning may improve accuracy in the forecasts in comparison to single solutions. Taking into consideration the mentioned aspects, in this work, we proposed the MegazordNet, a framework that explores statistical features within a financial series combined with a structured deep learning model for time series forecasting. We evaluated our approach predicting the closing price of stocks in the S&P 500 using different metrics, and we were able to beat single statistical and machine learning methods.
Artificial Neural Network for Cybersecurity: A Comprehensive Review
Authors: Prajoy Podder, Subrato Bharati, M. Rubaiyat Hossain Mondal, Pinto Kumar Paul, Utku Kose
Date: 2021-06-20 00:00:00
Tasks: Intrusion Detection, Malware Detection

Cybersecurity is a very emerging field that protects systems, networks, and data from digital attacks. With the increase in the scale of the Internet and the evolution of cyber attacks, developing novel cybersecurity tools has become important, particularly for Internet of things (IoT) networks. This paper provides a systematic review of the application of deep learning (DL) approaches for cybersecurity. This paper provides a short description of DL methods which is used in cybersecurity, including deep belief networks, generative adversarial networks, recurrent neural networks, and others. Next, we illustrate the differences between shallow learning and DL. Moreover, a discussion is provided on the currently prevailing cyber-attacks in IoT and other networks, and the effectiveness of DL methods to manage these attacks. Besides, this paper describes studies that highlight the DL technique, cybersecurity applications, and the source of datasets. Next, a discussion is provided on the feasibility of DL systems for malware detection and classification, intrusion detection, and other frequent cyber-attacks, including identifying file type, spam, and network traffic. Our review indicates that high classification accuracy of 99.72% is obtained by restricted Boltzmann machine (RBM) when applied to a custom dataset, while long short-term memory (LSTM) achieves an accuracy of 99.80% for KDD Cup 99 dataset. Finally, this article discusses the importance of cybersecurity for reliable and practicable IoT-driven healthcare systems.
Structure-aware reinforcement learning for node-overload protection in mobile edge computing
Authors: Anirudha Jitani, Aditya Mahajan, Zhongwen Zhu, Hatem Abou-zeid, Emmanuel T. Fapi, Hakimeh Purmehdi
Date: 2021-06-29 00:00:00
Tasks: Edge-computing, reinforcement-learning, Reinforcement Learning (RL)

Mobile Edge Computing (MEC) refers to the concept of placing computational capability and applications at the edge of the network, providing benefits such as reduced latency in handling client requests, reduced network congestion, and improved performance of applications. The performance and reliability of MEC are degraded significantly when one or several edge servers in the cluster are overloaded. Especially when a server crashes due to the overload, it causes service failures in MEC. In this work, an adaptive admission control policy to prevent edge node from getting overloaded is presented. This approach is based on a recently-proposed low complexity RL (Reinforcement Learning) algorithm called SALMUT (Structure-Aware Learning for Multiple Thresholds), which exploits the structure of the optimal admission control policy in multi-class queues for an average-cost setting. We extend the framework to work for node overload-protection problem in a discounted-cost setting. The proposed solution is validated using several scenarios mimicking real-world deployments in two different settings - computer simulations and a docker testbed. Our empirical evaluations show that the total discounted cost incurred by SALMUT is similar to state-of-the-art deep RL algorithms such as PPO (Proximal Policy Optimization) and A2C (Advantage Actor Critic) but requires an order of magnitude less time to train, outputs easily interpretable policy, and can be deployed in an online manner.
Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning
Authors: Peng Yang, Tony Q. S. Quek, Jingxuan Chen, Chaoqun You, Xianbin Cao
Date: 2021-06-03 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

This paper investigates the problem of providing ultra-reliable and energy-efficient virtual reality (VR) experiences for wireless mobile users. To ensure reliable ultra-high-definition (UHD) video frame delivery to mobile users and enhance their immersive visual experiences, a coordinated multipoint (CoMP) transmission technique and millimeter wave (mmWave) communications are exploited. Owing to user movement and time-varying wireless channels, the wireless VR experience enhancement problem is formulated as a sequence-dependent and mixed-integer problem with a goal of maximizing users' feeling of presence (FoP) in the virtual world, subject to power consumption constraints on access points (APs) and users' head-mounted displays (HMDs). The problem, however, is hard to be directly solved due to the lack of users' accurate tracking information and the sequence-dependent and mixed-integer characteristics. To overcome this challenge, we develop a parallel echo state network (ESN) learning method to predict users' tracking information by training fresh and historical tracking samples separately collected by APs. With the learnt results, we propose a deep reinforcement learning (DRL) based optimization algorithm to solve the formulated problem. In this algorithm, we implement deep neural networks (DNNs) as a scalable solution to produce integer decision variables and solving a continuous power control problem to criticize the integer decision variables. Finally, the performance of the proposed algorithm is compared with various benchmark algorithms, and the impact of different design parameters is also discussed. Simulation results demonstrate that the proposed algorithm is more 4.14% energy-efficient than the benchmark algorithms.
WiCluster: Passive Indoor 2D/3D Positioning using WiFi without Precise Labels
Authors: Ilia Karmanov, Farhad G. Zanjani, Simone Merlin, Ishaque Kadampot, Daniel Dijkman
Date: 2021-05-31 00:00:00
Tasks: Dimensionality Reduction, Position, Triplet

We introduce WiCluster, a new machine learning (ML) approach for passive indoor positioning using radio frequency (RF) channel state information (CSI). WiCluster can predict both a zone-level position and a precise 2D or 3D position, without using any precise position labels during training. Prior CSI-based indoor positioning work has relied on non-parametric approaches using digital signal-processing (DSP) and, more recently, parametric approaches (e.g., fully supervised ML methods). However these do not handle the complexity of real-world environments well and do not meet requirements for large-scale commercial deployments: the accuracy of DSP-based method deteriorates significantly in non-line-of-sight conditions, while supervised ML methods need large amounts of hard-to-acquire centimeter accuracy position labels. In contrast, WiCluster is precise, requires weaker label-information that can be easily collected, and works well in non-line-of-sight conditions. Our first contribution is a novel dimensionality reduction method for charting. It combines a triplet-loss with a multi-scale clustering-loss to map the high-dimensional CSI representation to a 2D/3D latent space. Our second contribution is two weakly supervised losses that map this latent space into a Cartesian map, resulting in meter-accuracy position results. These losses only require simple to acquire priors: a sketch of the floorplan, approximate access-point locations and a few CSI packets that are labelled with the corresponding zone in the floorplan. Thirdly, we report results and a robustness study for 2D positioning in two single-floor office buildings and 3D positioning in a two-story home.
Gamers Private Network Performance Forecasting. From Raw Data to the Data Warehouse with Machine Learning and Neural Nets
Authors: Albert Wong, Chun Yin Chiu, Gaétan Hains, Jack Humphrey, Hans Fuhrmann, Youry Khmelevsky, Chris Mazur
Date: 2021-05-26 00:00:00

Gamers Private Network (GPN) is a client/server technology that guarantees a connection for online video games that is more reliable and lower latency than a standard internet connection. Users of the GPN technology benefit from a stable and high-quality gaming experience for online games, which are hosted and played across the world. After transforming a massive volume of raw networking data collected by WTFast, we have structured the cleaned data into a special-purpose data warehouse and completed the extensive analysis using machine learning and neural nets technologies, and business intelligence tools. These analyses demonstrate the ability to predict and quantify changes in the network and demonstrate the benefits gained from the use of a GPN for users when connected to an online game session.
The Customer is Always Right: Customer-Centered Pooling for Ride-Hailing Systems
Authors: Paul Karaenke, Maximilian Schiffer, Stefan Waldherr
Date: 2021-07-02 00:00:00

Today's ride-hailing systems experienced significant growth and ride-pooling promises to allow for efficient and sustainable on-demand transportation. However, efficient ride-pooling requires a large pool of participating customers. To increase the customers' willingness for participation, we study a novel customer-centered pooling (CCP) mechanism, accounting for individual customers' pooling benefits. We study the benefit of this mechanism from a customer, fleet operator, and system perspective, and compare it to existing provider-centered pooling (PCP) mechanisms. We prove that it is individually rational and weakly dominant for a customer to participate in CCP, but not for PCP. We substantiate this analysis with complementary numerical studies, implementing a simulation environment based on real-world data that allows us to assess both mechanisms' benefit. To this end, we present results for both pooling mechanisms and show that pooling can benefit all stakeholders in on-demand transportation. Moreover, we analyze in which cases a CCP mechanism Pareto dominates a PCP mechanism and show that a mobility service operator would prefer CCP mechanisms over PCP mechanisms for all price segments. Simultaneously, CCP mechanisms reduce the overall distance driven in the system up to 32% compared to not pooling customers. Our results provide decision support for mobility service operators that want to implement and improve pooling mechanisms as they allow us to analyze the impact of a CCP and a PCP mechanism from a holistic perspective. Among others, we show that CCP mechanisms can lead to a win-win situation for operators and customers while simultaneously improving system performance and reducing emissions.