pwc-archive/papers-with-abstracts huggingface dataset:
The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT
Authors: Jörg Tiedemann
Date: 2020-10-13 00:00:00
Tasks: Few-Shot Learning, Machine Translation, Translation

This paper describes the development of a new benchmark for machine translation that provides training and test data for thousands of language pairs covering over 500 languages and tools for creating state-of-the-art translation models from that collection. The main goal is to trigger the development of open translation tools and models with a much broader coverage of the World's languages. Using the package it is possible to work on realistic low-resource scenarios avoiding artificially reduced setups that are common when demonstrating zero-shot or few-shot learning. For the first time, this package provides a comprehensive collection of diverse data sets in hundreds of languages with systematic language and script annotation and data splits to extend the narrow coverage of existing benchmarks. Together with the data release, we also provide a growing number of pre-trained baseline models for individual language pairs and selected language groups.
Broadly-Exploring, Local-Policy Trees for Long-Horizon Task Planning
Authors: Brian Ichter, Pierre Sermanet, Corey Lynch
Date: 2020-10-13 00:00:00
Tasks: Motion Planning, Task Planning

Long-horizon planning in realistic environments requires the ability to reason over sequential tasks in high-dimensional state spaces with complex dynamics. Classical motion planning algorithms, such as rapidly-exploring random trees, are capable of efficiently exploring large state spaces and computing long-horizon, sequential plans. However, these algorithms are generally challenged with complex, stochastic, and high-dimensional state spaces as well as in the presence of narrow passages, which naturally emerge in tasks that interact with the environment. Machine learning offers a promising solution for its ability to learn general policies that can handle complex interactions and high-dimensional observations. However, these policies are generally limited in horizon length. Our approach, Broadly-Exploring, Local-policy Trees (BELT), merges these two approaches to leverage the strengths of both through a task-conditioned, model-based tree search. BELT uses an RRT-inspired tree search to efficiently explore the state space. Locally, the exploration is guided by a task-conditioned, learned policy capable of performing general short-horizon tasks. This task space can be quite general and abstract; its only requirements are to be sampleable and to well-cover the space of useful tasks. This search is aided by a task-conditioned model that temporally extends dynamics propagation to allow long-horizon search and sequential reasoning over tasks. BELT is demonstrated experimentally to be able to plan long-horizon, sequential trajectories with a goal conditioned policy and generate plans that are robust.
RANDGAN: Randomized Generative Adversarial Network for Detection of COVID-19 in Chest X-ray
Authors: Saman Motamed, Patrik Rogalla, Farzad Khalvati
Date: 2020-10-06 00:00:00
Tasks: Anomaly Detection, COVID-19 Diagnosis, COVID-19 Image Segmentation, Generative Adversarial Network, Transfer Learning

COVID-19 spread across the globe at an immense rate has left healthcare systems incapacitated to diagnose and test patients at the needed rate. Studies have shown promising results for detection of COVID-19 from viral bacterial pneumonia in chest X-rays. Automation of COVID-19 testing using medical images can speed up the testing process of patients where health care systems lack sufficient numbers of the reverse-transcription polymerase chain reaction (RT-PCR) tests. Supervised deep learning models such as convolutional neural networks (CNN) need enough labeled data for all classes to correctly learn the task of detection. Gathering labeled data is a cumbersome task and requires time and resources which could further strain health care systems and radiologists at the early stages of a pandemic such as COVID-19. In this study, we propose a randomized generative adversarial network (RANDGAN) that detects images of an unknown class (COVID-19) from known and labelled classes (Normal and Viral Pneumonia) without the need for labels and training data from the unknown class of images (COVID-19). We used the largest publicly available COVID-19 chest X-ray dataset, COVIDx, which is comprised of Normal, Pneumonia, and COVID-19 images from multiple public databases. In this work, we use transfer learning to segment the lungs in the COVIDx dataset. Next, we show why segmentation of the region of interest (lungs) is vital to correctly learn the task of classification, specifically in datasets that contain images from different resources as it is the case for the COVIDx dataset. Finally, we show improved results in detection of COVID-19 cases using our generative model (RANDGAN) compared to conventional generative adversarial networks (GANs) for anomaly detection in medical images, improving the area under the ROC curve from 0.71 to 0.77.
A Lightweight Music Texture Transfer System
Authors: Xutan Peng, Chen Li, Zhi Cai, Faqiang Shi, Yidan Liu, JianXin Li
Date: 2018-09-27 00:00:00
Proceeding: arXiv preprint 2018 9
Tasks: Music Texture Transfer

Deep learning researches on the transformation problems for image and text have raised great attention. However, present methods for music feature transfer using neural networks are far from practical application. In this paper, we initiate a novel system for transferring the texture of music, and release it as an open source project. Its core algorithm is composed of a converter which represents sounds as texture spectra, a corresponding reconstructor and a feed-forward transfer network. We evaluate this system from multiple perspectives, and experimental results reveal that it achieves convincing results in both sound effects and computational performance.
Advanced Machine Learning Techniques for Self-Interference Cancellation in Full-Duplex Radios
Authors: Andreas Toftegaard Kristensen, Andreas Burg, Alexios Balatsoukas-Stimming
Date: 2019-12-14 00:00:00
Tasks: BIG-bench Machine Learning

In-band full-duplex systems allow for more efficient use of temporal and spectral resources by transmitting and receiving information at the same time and on the same frequency. However, this creates a strong self-interference signal at the receiver, making the use of self-interference cancellation critical. Recently, neural networks have been used to perform digital self-interference with lower computational complexity compared to a traditional polynomial model. In this paper, we examine the use of advanced neural networks, such as recurrent and complex-valued neural networks, and we perform an in-depth network architecture exploration. Our neural network architecture exploration reveals that complex-valued neural networks can significantly reduce both the number of floating-point operations and parameters compared to a polynomial model, whereas the real-valued networks only reduce the number of floating-point operations. For example, at a digital self-interference cancellation of 44.51 dB, a complex-valued neural network requires 33.7 % fewer floating-point operations and 26.9 % fewer parameters compared to the polynomial model.
Non-Linear Digital Self-Interference Cancellation for In-Band Full-Duplex Radios Using Neural Networks
Authors: Alexios Balatsoukas-Stimming
Date: 2017-11-01 00:00:00

Full-duplex systems require very strong self-interference cancellation in order to operate correctly and a significant part of the self-interference signal is due to non-linear effects created by various transceiver impairments. As such, linear cancellation alone is usually not sufficient and sophisticated non-linear cancellation algorithms have been proposed in the literature. In this work, we investigate the use of a neural network as an alternative to the traditional non-linear cancellation method that is based on polynomial basis functions. Measurement results from a full-duplex testbed demonstrate that a small and simple feed-forward neural network canceler works exceptionally well, as it can match the performance of the polynomial non-linear canceler with significantly lower computational complexity.
Scenic: A Language for Scenario Specification and Data Generation
Authors: Daniel J. Fremont, Edward Kim, Tommaso Dreossi, Shromona Ghosh, Xiangyu Yue, Alberto L. Sangiovanni-Vincentelli, Sanjit A. Seshia
Date: 2020-10-13 00:00:00
Tasks: Probabilistic Programming, Synthetic Data Generation

We propose a new probabilistic programming language for the design and analysis of cyber-physical systems, especially those based on machine learning. Specifically, we consider the problems of training a system to be robust to rare events, testing its performance under different conditions, and debugging failures. We show how a probabilistic programming language can help address these problems by specifying distributions encoding interesting types of inputs, then sampling these to generate specialized training and test data. More generally, such languages can be used to write environment models, an essential prerequisite to any formal analysis. In this paper, we focus on systems like autonomous cars and robots, whose environment at any point in time is a 'scene', a configuration of physical objects and agents. We design a domain-specific language, Scenic, for describing scenarios that are distributions over scenes and the behaviors of their agents over time. As a probabilistic programming language, Scenic allows assigning distributions to features of the scene, as well as declaratively imposing hard and soft constraints over the scene. We develop specialized techniques for sampling from the resulting distribution, taking advantage of the structure provided by Scenic's domain-specific syntax. Finally, we apply Scenic in a case study on a convolutional neural network designed to detect cars in road images, improving its performance beyond that achieved by state-of-the-art synthetic data generation methods.
Fantastic Features and Where to Find Them: Detecting Cognitive Impairment with a Subsequence Classification Guided Approach
Authors: Benjamin Eyre, Aparna Balagopalan, Jekaterina Novikova
Date: 2020-10-13 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: BIG-bench Machine Learning, Feature Engineering, General Classification

Despite the widely reported success of embedding-based machine learning methods on natural language processing tasks, the use of more easily interpreted engineered features remains common in fields such as cognitive impairment (CI) detection. Manually engineering features from noisy text is time and resource consuming, and can potentially result in features that do not enhance model performance. To combat this, we describe a new approach to feature engineering that leverages sequential machine learning models and domain knowledge to predict which features help enhance performance. We provide a concrete example of this method on a standard data set of CI speech and demonstrate that CI classification accuracy improves by 2.3% over a strong baseline when using features produced by this method. This demonstration provides an ex-ample of how this method can be used to assist classification in fields where interpretability is important, such as health care.
Does my multimodal model learn cross-modal interactions? It's harder to tell than you might think!
Authors: Jack Hessel, Lillian Lee
Date: 2020-10-13 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Diagnostic, Image-text Classification, Question Answering, text-classification, Text Classification, Visual Question Answering, Visual Question Answering (VQA)

Modeling expressive cross-modal interactions seems crucial in multimodal tasks, such as visual question answering. However, sometimes high-performing black-box algorithms turn out to be mostly exploiting unimodal signals in the data. We propose a new diagnostic tool, empirical multimodally-additive function projection (EMAP), for isolating whether or not cross-modal interactions improve performance for a given model on a given task. This function projection modifies model predictions so that cross-modal interactions are eliminated, isolating the additive, unimodal structure. For seven image+text classification tasks (on each of which we set new state-of-the-art benchmarks), we find that, in many cases, removing cross-modal interactions results in little to no performance degradation. Surprisingly, this holds even when expressive models, with capacity to consider interactions, otherwise outperform less expressive models; thus, performance improvements, even when present, often cannot be attributed to consideration of cross-modal feature interactions. We hence recommend that researchers in multimodal machine learning report the performance not only of unimodal baselines, but also the EMAP of their best-performing model.
Optimal Low-Degree Hardness of Maximum Independent Set
Authors: Alexander S. Wein
Date: 2020-10-13 00:00:00

We study the algorithmic task of finding a large independent set in a sparse Erd\H{o}s-R\'{e}nyi random graph with $n$ vertices and average degree $d$. The maximum independent set is known to have size $(2 \log d / d)n$ in the double limit $n \to \infty$ followed by $d \to \infty$, but the best known polynomial-time algorithms can only find an independent set of half-optimal size $(\log d / d)n$. We show that the class of low-degree polynomial algorithms can find independent sets of half-optimal size but no larger, improving upon a result of Gamarnik, Jagannath, and the author. This generalizes earlier work by Rahman and Vir\'ag, which proved the analogous result for the weaker class of local algorithms.
Unified lower bounds for interactive high-dimensional estimation under information constraints
Date: 2020-10-13 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: parameter estimation

We consider distributed parameter estimation using interactive protocols subject to local information constraints such as bandwidth limitations, local differential privacy, and restricted measurements. We provide a unified framework enabling us to derive a variety of (tight) minimax lower bounds for different parametric families of distributions, both continuous and discrete, under any $\ell_p$ loss. Our lower bound framework is versatile and yields "plug-and-play" bounds that are widely applicable to a large range of estimation problems, and, for the prototypical case of the Gaussian family, circumvents limitations of previous techniques. In particular, our approach recovers bounds obtained using data processing inequalities and Cram\'er--Rao bounds, two other alternative approaches for proving lower bounds in our setting of interest. Further, for the families considered, we complement our lower bounds with matching upper bounds.
Accurate Calibration of Agent-based Epidemiological Models with Neural Network Surrogates
Authors: Rushil Anirudh, Jayaraman J. Thiagarajan, Peer-Timo Bremer, Timothy C. Germann, Sara Y. Del Valle, Frederick H. Streitz
Date: 2020-10-13 00:00:00

Calibrating complex epidemiological models to observed data is a crucial step to provide both insights into the current disease dynamics, i.e.\ by estimating a reproductive number, as well as to provide reliable forecasts and scenario explorations. Here we present a new approach to calibrate an agent-based model -- EpiCast -- using a large set of simulation ensembles for different major metropolitan areas of the United States. In particular, we propose: a new neural network based surrogate model able to simultaneously emulate all different locations; and a novel posterior estimation that provides not only more accurate posterior estimates of all parameters but enables the joint fitting of global parameters across regions.
Toward Few-step Adversarial Training from a Frequency Perspective
Authors: Hans Shih-Han Wang, Cory Cornelius, Brandon Edwards, Jason Martin
Date: 2020-10-13 00:00:00

We investigate adversarial-sample generation methods from a frequency domain perspective and extend standard $l_{\infty}$ Projected Gradient Descent (PGD) to the frequency domain. The resulting method, which we call Spectral Projected Gradient Descent (SPGD), has better success rate compared to PGD during early steps of the method. Adversarially training models using SPGD achieves greater adversarial accuracy compared to PGD when holding the number of attack steps constant. The use of SPGD can, therefore, reduce the overhead of adversarial training when utilizing adversarial generation with a smaller number of steps. However, we also prove that SPGD is equivalent to a variant of the PGD ordinarily used for the $l_{\infty}$ threat model. This PGD variant omits the sign function which is ordinarily applied to the gradient. SPGD can, therefore, be performed without explicitly transforming into the frequency domain. Finally, we visualize the perturbations SPGD generates and find they use both high and low-frequency components, which suggests that removing either high-frequency components or low-frequency components is not an effective defense.
Modeling Atmospheric Data and Identifying Dynamics: Temporal Data-Driven Modeling of Air Pollutants
Authors: Javier Rubio-Herrero, Carlos Ortiz Marrero, Wai-Tong Louis Fan
Date: 2020-10-13 00:00:00
Tasks: Time Series, Time Series Analysis

Atmospheric modeling has recently experienced a surge with the advent of deep learning. Most of these models, however, predict concentrations of pollutants following a data-driven approach in which the physical laws that govern their behaviors and relationships remain hidden. With the aid of real-world air quality data collected hourly in different stations throughout Madrid, we present an empirical approach using data-driven techniques with the following goals: (1) Find parsimonious systems of ordinary differential equations via sparse identification of nonlinear dynamics (SINDy) that model the concentration of pollutants and their changes over time; (2) assess the performance and limitations of our models using stability analysis; (3) reconstruct the time series of chemical pollutants not measured in certain stations using delay coordinate embedding results. Our results show that Akaike's Information Criterion can work well in conjunction with best subset regression as to find an equilibrium between sparsity and goodness of fit. We also find that, due to the complexity of the chemical system under study, identifying the dynamics of this system over longer periods of time require higher levels of data filtering and smoothing. Stability analysis for the reconstructed ordinary differential equations (ODEs) reveals that more than half of the physically relevant critical points are saddle points, suggesting that the system is unstable even under the idealized assumption that all environmental conditions are constant over time.
Can Federated Learning Save The Planet?
Authors: Xinchi Qiu, Titouan Parcollet, Daniel J. Beutel, Taner Topal, Akhil Mathur, Nicholas D. Lane
Date: 2020-10-13 00:00:00
Tasks: Federated Learning

Despite impressive results, deep learning-based technologies also raise severe privacy and environmental concerns induced by the training procedure often conducted in data centers. In response, alternatives to centralized training such as Federated Learning (FL) have emerged. Perhaps unexpectedly, FL, in particular, is starting to be deployed at a global scale by companies that must adhere to new legal demands and policies originating from governments and the civil society for privacy protection. However, the potential environmental impact related to FL remains unclear and unexplored. This paper offers the first-ever systematic study of the carbon footprint of FL. First, we propose a rigorous model to quantify the carbon footprint, hence facilitating the investigation of the relationship between FL design and carbon emissions. Then, we compare the carbon footprint of FL to traditional centralized learning. Our findings show FL, despite being slower to converge, can be a greener technology than data center GPUs. Finally, we highlight and connect the reported results to the future challenges and trends in FL to reduce its environmental impact, including algorithms efficiency, hardware capabilities, and stronger industry transparency.
On the Fairness of Causal Algorithmic Recourse
Authors: Julius von Kügelgen, Amir-Hossein Karimi, Umang Bhatt, Isabel Valera, Adrian Weller, Bernhard Schölkopf
Date: 2020-10-13 00:00:00
Tasks: counterfactual, Fairness

Algorithmic fairness is typically studied from the perspective of predictions. Instead, here we investigate fairness from the perspective of recourse actions suggested to individuals to remedy an unfavourable classification. We propose two new fairness criteria at the group and individual level, which -- unlike prior work on equalising the average group-wise distance from the decision boundary -- explicitly account for causal relationships between features, thereby capturing downstream effects of recourse actions performed in the physical world. We explore how our criteria relate to others, such as counterfactual fairness, and show that fairness of recourse is complementary to fairness of prediction. We study theoretically and empirically how to enforce fair causal recourse by altering the classifier and perform a case study on the Adult dataset. Finally, we discuss whether fairness violations in the data generating process revealed by our criteria may be better addressed by societal interventions as opposed to constraints on the classifier.
A Deep Learning Forecaster with Exogenous Variables for Day-Ahead Locational Marginal Price
Authors: Dipanwita Saha, Felipe Lopez
Date: 2020-10-13 00:00:00
Tasks: Deep Learning, Time Series, Time Series Analysis

Several approaches have been proposed to forecast day-ahead locational marginal price (daLMP) in deregulated energy markets. The rise of deep learning has motivated its use in energy price forecasts but most deep learning approaches fail to accommodate for exogenous variables, which have significant influence in the peaks and valleys of the daLMP. Accurate forecasts of the daLMP valleys are of crucial importance for power generators since one of the most important decisions they face is whether to sell power at a loss to prevent incurring in shutdown and start-up costs, or to bid at production cost and face the risk of shutting down. In this article we propose a deep learning model that incorporates both the history of daLMP and the effect of exogenous variables (e.g., forecasted load, weather data). A numerical study at the PJM independent system operator (ISO) illustrates how the proposed model outperforms traditional time series techniques while supporting risk-based analysis of shutdown decisions.
Autotuning Search Space for Loop Transformations
Authors: Michael Kruse, Hal Finkel, Xingfu Wu
Date: 2020-10-13 00:00:00

One of the challenges for optimizing compilers is to predict whether applying an optimization will improve its execution speed. Programmers may override the compiler's profitability heuristic using optimization directives such as pragmas in the source code. Machine learning in the form of autotuning can assist users in finding the best optimizations for each platform. In this paper we propose a loop transformation search space that takes the form of a tree, in contrast to previous approaches that usually use vector spaces to represent loop optimization configurations. We implemented a simple autotuner exploring the search space and applied it to a selected set of PolyBench kernels. While the autotuner is capable of representing every possible sequence of loop transformations and their relations, the results motivate the use of better search strategies such as Monte Carlo tree search to find sophisticated loop transformations such as multilevel tiling.