pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Cross-Lingual Representation Learning
Authors: Sebastian Ruder, Anders S{\o}gaard, Ivan Vuli{\'c}
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Representation Learning, Structured Prediction

In this tutorial, we provide a comprehensive survey of the exciting recent work on cutting-edge weakly-supervised and unsupervised cross-lingual word representations. After providing a brief history of supervised cross-lingual word representations, we focus on: 1) how to induce weakly-supervised and unsupervised cross-lingual word representations in truly resource-poor settings where bilingual supervision cannot be guaranteed; 2) critical examinations of different training conditions and requirements under which unsupervised algorithms can and cannot work effectively; 3) more robust methods for distant language pairs that can mitigate instability issues and low performance for distant language pairs; 4) how to comprehensively evaluate such representations; and 5) diverse applications that benefit from cross-lingual word representations (e.g., MT, dialogue, cross-lingual sequence labeling and structured prediction applications, cross-lingual IR).
Advances in Argument Mining
Authors: Katarzyna Budzynska, Chris Reed
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Argument Mining, Opinion Mining, Sentiment Analysis

This course aims to introduce students to an exciting and dynamic area that has witnessed remarkable growth over the past 36 months. Argument mining builds on opinion mining, sentiment analysis and related to tasks to automatically extract not just *what* people think, but *why* they hold the opinions they do. From being largely beyond the state of the art barely five years ago, there are now many hundreds of papers on the topic, millions of dollars of commercial and research investment, and the 6th ACL workshop on the topic will be in Florence in 2019. The tutors have delivered tutorials on argument mining at ACL 2016, at IJCAI 2016 and at ESSLLI 2017; for ACL 2019, we have developed a tutorial that provides a synthesis of the major advances in the area over the past three years.
Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
Authors: Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain, Saravanan Krishnan
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Knowledge Graphs, Translation

In this tutorial, we wish to cover the foundational, methodological, and system development aspects of translating structured data (such as data in tabular form) and knowledge bases (such as knowledge graphs) into natural language. The attendees of the tutorial will be able to take away from this tutorial, (1) the basic ideas around how modern NLP and NLG techniques could be applied to describe and summarize textual data in format that is non-linguistic in nature or has some structure, and (2) a few interesting open-ended questions, which could lead to significant research contributions in future. The tutorial aims to convey challenges and nuances in structured data translation, data representation techniques, and domain adaptable solutions for translation of the data into natural language form. Various solutions, starting from traditional rule based/heuristic driven and modern data-driven and ultra-modern deep-neural style architectures will be discussed, followed by a brief discussion on evaluation and quality estimation. A significant portion of the tutorial will be dedicated towards unsupervised, scalable, and adaptable solutions, given that systems for such an important task will never naturally enjoy sustainable large scale domain independent labeled (parallel) data.
Near-Field Joint Localization and Synchronization
Authors: Henk Wymeersch
Date: 2019-07-17 00:00:00

In 5G communication, linear and planar arrays are used for both positioning and communication. As the arrays grow in size, the conventional far-field assumption is increasingly being violated and curvature of the wavefront should be taken into account. We explicitly contrast near-field and far-field uplink localization performance from a Fisher information perspective and show how a simple algorithm can provide a rough initial estimate for maximum likelihood estimation. Our results show that invoking the more general near-field model allows for joint estimation of a user's location and clock bias in the uplink, while the far-field model may lead to overly pessimistic performance assessment.
Finding a "Kneedle" in a Haystack: Detecting Knee Points in System Behavior
Authors: Ville Satopaa, Jeannie Albrecht, David Irwin, Barath Raghavan
Date: 2011-07-25 00:00:00
Proceeding: 2011 31st International Conference on Distributed Computing Systems Workshops 2011 7

Computer systems often reach a point at which the relative cost to increase some tunable parameter is no longer worth the corresponding performance benefit. These "knees'' typically represent beneficial points that system designers have long selected to best balance inherent trade-offs. While prior work largely uses ad hoc, system-specific approaches to detect knees, we present Kneedle, a general approach to on line and off line knee detection that is applicable to a wide range of systems. We define a knee formally for continuous functions using the mathematical concept of curvature and compare our definition against alternatives. We then evaluate Kneedle's accuracy against existing algorithms on both synthetic and real data sets, and evaluate its performance in two different applications.
Actions Speak Louder than Goals:Valuing Player Actions in Soccer
Authors: Tom Decroos, Lotte Bransen, Jan Van Haaren, Jesse Davis
Date: 2019-07-10 00:00:00
Proceeding: Published in KDD 2018 2019 7
Tasks: Football Action Valuation

Assessing the impact of the individual actions performed by soccerplayers during games is a crucial aspect of the player recruitmentprocess. Unfortunately, most traditional metrics fall short in ad-dressing this task as they either focus on rare actions like shotsand goals alone or fail to account for the context in which theactions occurred. This paper introduces (1) a new language for de-scribing individual player actions on the pitch and (2) a frameworkfor valuing any type of player action based on its impact on thegame outcome while accounting for the context in which the actionhappened. By aggregating soccer players’ action values, their totaloffensive and defensive contributions to their team can be quan-tified. We show how our approach considers relevant contextualinformation that traditional player evaluation metrics ignore andpresent a number of use cases related to scouting and playing stylecharacterization in the 2016/2017 and 2017/2018 seasons in Europe’stop competitions.
Language Modelling for Sound Event Detection with Teacher Forcing and Scheduled Sampling
Authors: Konstantinos Drossos, Shayan Gharib, Paul Magron, Tuomas Virtanen
Date: 2019-07-19 00:00:00
Proceeding: Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE) 2019 2019 10
Tasks: Event Detection, Language Modelling, Machine Translation, Sound Event Detection

A sound event detection (SED) method typically takes as an input a sequence of audio frames and predicts the activities of sound events in each frame. In real-life recordings, the sound events exhibit some temporal structure: for instance, a "car horn" will likely be followed by a "car passing by". While this temporal structure is widely exploited in sequence prediction tasks (e.g., in machine translation), where language models (LM) are exploited, it is not satisfactorily modeled in SED. In this work we propose a method which allows a recurrent neural network (RNN) to learn an LM for the SED task. The method conditions the input of the RNN with the activities of classes at the previous time step. We evaluate our method using F1 score and error rate (ER) over three different and publicly available datasets; the TUT-SED Synthetic 2016 and the TUT Sound Events 2016 and 2017 datasets. The obtained results show an increase of 9% and 2% at the F1 (higher is better) and a decrease of 7% and 2% at ER (lower is better) for the TUT Sound Events 2016 and 2017 datasets, respectively, when using our method. On the contrary, with our method there is a decrease of 4% at F1 score and an increase of 7% at ER for the TUT-SED Synthetic 2016 dataset.
Lookahead Optimizer: k steps forward, 1 step back
Authors: Michael R. Zhang, James Lucas, Geoffrey Hinton, Jimmy Ba
Date: 2019-07-19 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Image Classification, Machine Translation, Stochastic Optimization, Translation

The vast majority of successful deep neural networks are trained using variants of stochastic gradient descent (SGD) algorithms. Recent attempts to improve SGD can be broadly categorized into two approaches: (1) adaptive learning rate schemes, such as AdaGrad and Adam, and (2) accelerated schemes, such as heavy-ball and Nesterov momentum. In this paper, we propose a new optimization algorithm, Lookahead, that is orthogonal to these previous approaches and iteratively updates two sets of weights. Intuitively, the algorithm chooses a search direction by looking ahead at the sequence of fast weights generated by another optimizer. We show that Lookahead improves the learning stability and lowers the variance of its inner optimizer with negligible computation and memory cost. We empirically demonstrate Lookahead can significantly improve the performance of SGD and Adam, even with their default hyperparameter settings on ImageNet, CIFAR-10/100, neural machine translation, and Penn Treebank.
Fast and robust detection of solar modules in electroluminescence images
Authors: Mathis Hoffmann, Bernd Doll, Florian Talkenberg, Christoph J. Brabec, Andreas K. Maier, Vincent Christlein
Date: 2019-07-19 00:00:00

Fast, non-destructive and on-site quality control tools, mainly high sensitive imaging techniques, are important to assess the reliability of photovoltaic plants. To minimize the risk of further damages and electrical yield losses, electroluminescence (EL) imaging is used to detect local defects in an early stage, which might cause future electric losses. For an automated defect recognition on EL measurements, a robust detection and rectification of modules, as well as an optional segmentation into cells is required. This paper introduces a method to detect solar modules and crossing points between solar cells in EL images. We only require 1-D image statistics for the detection, resulting in an approach that is computationally efficient. In addition, the method is able to detect the modules under perspective distortion and in scenarios, where multiple modules are visible in the image. We compare our method to the state of the art and show that it is superior in presence of perspective distortion while the performance on images, where the module is roughly coplanar to the detector, is similar to the reference method. Finally, we show that we greatly improve in terms of computational time in comparison to the reference method.
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Authors: Xiaofei Wang, Yiwen Han, Victor C. M. Leung, Dusit Niyato, Xueqiang Yan, Xu Chen
Date: 2019-07-19 00:00:00
Tasks: Cloud Computing, Deep Learning, Edge-computing, Face Recognition, Management, Survey

Ubiquitous sensors and smart devices from factories and communities are generating massive amounts of data, and ever-increasing computing power is driving the core of computation and services from the cloud to the edge of the network. As an important enabler broadly changing people's lives, from face recognition to ambitious smart factories and cities, developments of artificial intelligence (especially deep learning, DL) based applications and services are thriving. However, due to efficiency and latency issues, the current cloud computing service architecture hinders the vision of "providing artificial intelligence for every person and every organization at everywhere". Thus, unleashing DL services using resources at the network edge near the data sources has emerged as a desirable solution. Therefore, edge intelligence, aiming to facilitate the deployment of DL services by edge computing, has received significant attention. In addition, DL, as the representative technique of artificial intelligence, can be integrated into edge computing frameworks to build intelligent edge for dynamic, adaptive edge maintenance and management. With regard to mutually beneficial edge intelligence and intelligent edge, this paper introduces and discusses: 1) the application scenarios of both; 2) the practical implementation methods and enabling technologies, namely DL training and inference in the customized edge computing framework; 3) challenges and future trends of more pervasive and fine-grained intelligence. We believe that by consolidating information scattered across the communication, networking, and DL areas, this survey can help readers to understand the connections between enabling technologies while promoting further discussions on the fusion of edge intelligence and intelligent edge, i.e., Edge DL.
A Graph Neural Network Approach for Scalable Wireless Power Control
Authors: Yifei Shen, Yuanming Shi, Jun Zhang, Khaled B. Letaief
Date: 2019-07-19 00:00:00
Tasks: Graph Neural Network

Deep neural networks have recently emerged as a disruptive technology to solve NP-hard wireless resource allocation problems in a real-time manner. However, the adopted neural network structures, e.g., multi-layer perceptron (MLP) and convolutional neural network (CNN), are inherited from deep learning for image processing tasks, and thus are not tailored to problems in wireless networks. In particular, the performance of these methods deteriorates dramatically when the wireless network size becomes large. In this paper, we propose to utilize graph neural networks (GNNs) to develop scalable methods for solving the power control problem in $K$-user interference channels. Specifically, a $K$-user interference channel is first modeled as a complete graph, where the quantitative information of wireless channels is incorporated as the features of the graph. We then propose an interference graph convolutional neural network (IGCNet) to learn the optimal power control in an unsupervised manner. It is shown that one-layer IGCNet is a universal approximator to continuous set functions, which well matches the permutation invariance property of interference channels and it is robust to imperfect channel state information (CSI). Extensive simulations will show that the proposed IGCNet outperforms existing methods and achieves significant speedup over the classic algorithm for power control, namely, WMMSE. The code is available on https://github.com/yshenaw/Globecom2019.
Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets
Authors: Frederik Kratzert, Daniel Klotz, Guy Shalev, Günter Klambauer, Sepp Hochreiter, Grey Nearing
Date: 2019-07-19 00:00:00
Tasks: Benchmarking, BIG-bench Machine Learning, Time Series, Time Series Analysis

Regional rainfall-runoff modeling is an old but still mostly out-standing problem in Hydrological Sciences. The problem currently is that traditional hydrological models degrade significantly in performance when calibrated for multiple basins together instead of for a single basin alone. In this paper, we propose a novel, data-driven approach using Long Short-Term Memory networks (LSTMs), and demonstrate that under a 'big data' paradigm, this is not necessarily the case. By training a single LSTM model on 531 basins from the CAMELS data set using meteorological time series data and static catchment attributes, we were able to significantly improve performance compared to a set of several different hydrological benchmark models. Our proposed approach not only significantly outperforms hydrological models that were calibrated regionally but also achieves better performance than hydrological models that were calibrated for each basin individually. Furthermore, we propose an adaption to the standard LSTM architecture, which we call an Entity-Aware-LSTM (EA-LSTM), that allows for learning, and embedding as a feature layer in a deep learning model, catchment similarities. We show that this learned catchment similarity corresponds well with what we would expect from prior hydrological understanding.
Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications
Authors: Shusen Liu, Di Wang, Dan Maljovec, Rushil Anirudh, Jayaraman J. Thiagarajan, Sam Ade Jacobs, Brian C. Van Essen, David Hysom, Jae-Seung Yeom, Jim Gaffney, Luc Peterson, Peter B. Robinson, Harsh Bhatia, Valerio Pascucci, Brian K. Spears, Peer-Timo Bremer
Date: 2019-07-19 00:00:00
Tasks: graph construction, Topological Data Analysis

With the rapid adoption of machine learning techniques for large-scale applications in science and engineering comes the convergence of two grand challenges in visualization. First, the utilization of black box models (e.g., deep neural networks) calls for advanced techniques in exploring and interpreting model behaviors. Second, the rapid growth in computing has produced enormous datasets that require techniques that can handle millions or more samples. Although some solutions to these interpretability challenges have been proposed, they typically do not scale beyond thousands of samples, nor do they provide the high-level intuition scientists are looking for. Here, we present the first scalable solution to explore and analyze high-dimensional functions often encountered in the scientific data analysis pipeline. By combining a new streaming neighborhood graph construction, the corresponding topology computation, and a novel data aggregation scheme, namely topology aware datacubes, we enable interactive exploration of both the topological and the geometric aspect of high-dimensional data. Following two use cases from high-energy-density (HED) physics and computational biology, we demonstrate how these capabilities have led to crucial new insights in both applications.
Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model
Authors: Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Lenka Zdeborová
Date: 2019-07-18 00:00:00

Gradient-based algorithms are effective for many machine learning tasks, but despite ample recent effort and some progress, it often remains unclear why they work in practice in optimising high-dimensional non-convex functions and why they find good minima instead of being trapped in spurious ones. Here we present a quantitative theory explaining this behaviour in a spiked matrix-tensor model. Our framework is based on the Kac-Rice analysis of stationary points and a closed-form analysis of gradient-flow originating from statistical physics. We show that there is a well defined region of parameters where the gradient-flow algorithm finds a good global minimum despite the presence of exponentially many spurious local minima. We show that this is achieved by surfing on saddles that have strong negative direction towards the global minima, a phenomenon that is connected to a BBP-type threshold in the Hessian describing the critical points of the landscapes.
VRSTC: Occlusion-Free Video Person Re-Identification
Authors: Ruibing Hou, Bingpeng Ma, Hong Chang, Xinqian Gu, Shiguang Shan, Xilin Chen
Date: 2019-07-19 00:00:00
Proceeding: CVPR 2019 6
Tasks: Person Re-Identification, Video-Based Person Re-Identification

Video person re-identification (re-ID) plays an important role in surveillance video analysis. However, the performance of video re-ID degenerates severely under partial occlusion. In this paper, we propose a novel network, called Spatio-Temporal Completion network (STCnet), to explicitly handle partial occlusion problem. Different from most previous works that discard the occluded frames, STCnet can recover the appearance of the occluded parts. For one thing, the spatial structure of a pedestrian frame can be used to predict the occluded body parts from the unoccluded body parts of this frame. For another, the temporal patterns of pedestrian sequence provide important clues to generate the contents of occluded parts. With the Spatio-temporal information, STCnet can recover the appearance for the occluded parts, which could be leveraged with those unoccluded parts for more accurate video re-ID. By combining a re-ID network with STCnet, a video re-ID framework robust to partial occlusion (VRSTC) is proposed. Experiments on three challenging video re-ID databases demonstrate that the proposed approach outperforms the state-of-the-art.
Interaction-and-Aggregation Network for Person Re-identification
Authors: Ruibing Hou, Bingpeng Ma, Hong Chang, Xinqian Gu, Shiguang Shan, Xilin Chen
Date: 2019-07-19 00:00:00
Proceeding: CVPR 2019 6
Tasks: Person Re-Identification

Person re-identification (reID) benefits greatly from deep convolutional neural networks (CNNs) which learn robust feature embeddings. However, CNNs are inherently limited in modeling the large variations in person pose and scale due to their fixed geometric structures. In this paper, we propose a novel network structure, Interaction-and-Aggregation (IA), to enhance the feature representation capability of CNNs. Firstly, Spatial IA (SIA) module is introduced. It models the interdependencies between spatial features and then aggregates the correlated features corresponding to the same body parts. Unlike CNNs which extract features from fixed rectangle regions, SIA can adaptively determine the receptive fields according to the input person pose and scale. Secondly, we introduce Channel IA (CIA) module which selectively aggregates channel features to enhance the feature representation, especially for smallscale visual cues. Further, IA network can be constructed by inserting IA blocks into CNNs at any depth. We validate the effectiveness of our model for person reID by demonstrating its superiority over state-of-the-art methods on three benchmark datasets.
Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery
Authors: Kristian Hartikainen, Xinyang Geng, Tuomas Haarnoja, Sergey Levine
Date: 2019-07-18 00:00:00
Proceeding: ICLR 2020 1
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning requires manual specification of a reward function to learn a task. While in principle this reward function only needs to specify the task goal, in practice reinforcement learning can be very time-consuming or even infeasible unless the reward function is shaped so as to provide a smooth gradient towards a successful outcome. This shaping is difficult to specify by hand, particularly when the task is learned from raw observations, such as images. In this paper, we study how we can automatically learn dynamical distances: a measure of the expected number of time steps to reach a given goal state from any other state. These dynamical distances can be used to provide well-shaped reward functions for reaching new goals, making it possible to learn complex tasks efficiently. We show that dynamical distances can be used in a semi-supervised regime, where unsupervised interaction with the environment is used to learn the dynamical distances, while a small amount of preference supervision is used to determine the task goal, without any manually engineered reward function or goal examples. We evaluate our method both on a real-world robot and in simulation. We show that our method can learn to turn a valve with a real-world 9-DoF hand, using raw image observations and just ten preference labels, without any other supervision. Videos of the learned skills can be found on the project website: https://sites.google.com/view/dynamical-distance-learning.