pwc-archive/papers-with-abstracts huggingface dataset:
Traffic Queue Length and Pressure Estimation for Road Networks with Geometric Deep Learning Algorithms
Authors: Simon F. G. Ehlers
Date: 2019-05-09 00:00:00
Tasks: Management

Due to urbanization and the increase of individual mobility, in most metropolitan areas around the world congestion and inefficient traffic management occur. Highly necessary intelligent traffic control systems, which are able to reduce congestion, rely on measurements of traffic situations in urban road networks and freeways. Unfortunately, the instrumentation for accurate traffic measurement is expensive and not widely implemented. This thesis addresses this problem, where relatively inexpensive and easy to install loop-detectors are used by a geometric deep learning algorithm, which uses loop-detector data in a spatial context of a road network, to estimate queue length in front of signalized intersections, which can be then used for following traffic control tasks. Therefore, in the first part of this work a conventional estimation method for queue length (which does not use machine learning techniques) based on second-by-second loop-detector data is implemented, which uses detected shockwaves in queues to estimate the length and point of time for the maximum queue. The method is later used as reference but also as additional input information for the geometric deep learning approach. In the second part the geometric deep learning algorithm is developed, which uses spatial correlations in the road network but also temporal correlations in detector data time sequences by new attention mechanisms, to overcome the limitations of conventional methods like excess traffic demand, lane changing and stop-and-go traffic. Therefore, it is necessary to abstract the topology of the road network in a graph. Both approaches are compared regarding their performance, reliability as well as limitations and validated by usage of the traffic simulation software SUMO (Simulation of Urban MObility). Finally, the results are discussed in the conclusions and further investigations are suggested.
Differentially Private Learning with Adaptive Clipping
Authors: Galen Andrew, Om Thakkar, H. Brendan McMahan, Swaroop Ramaswamy
Date: 2019-05-09 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Federated Learning

Existing approaches for training neural networks with user-level differential privacy (e.g., DP Federated Averaging) in federated learning (FL) settings involve bounding the contribution of each user's model update by clipping it to some constant value. However there is no good a priori setting of the clipping norm across tasks and learning settings: the update norm distribution depends on the model architecture and loss, the amount of data on each device, the client learning rate, and possibly various other parameters. We propose a method wherein instead of a fixed clipping norm, one clips to a value at a specified quantile of the update norm distribution, where the value at the quantile is itself estimated online, with differential privacy. The method tracks the quantile closely, uses a negligible amount of privacy budget, is compatible with other federated learning technologies such as compression and secure aggregation, and has a straightforward joint DP analysis with DP-FedAvg. Experiments demonstrate that adaptive clipping to the median update norm works well across a range of realistic federated learning tasks, sometimes outperforming even the best fixed clip chosen in hindsight, and without the need to tune any clipping hyperparameter.
Adversarially Trained Autoencoders for Parallel-Data-Free Voice Conversion
Authors: Orhan Ocal, Oguz H. Elibol, Gokce Keskin, Cory Stephenson, Anil Thomas, Kannan Ramchandran
Date: 2019-05-09 00:00:00
Tasks: Voice Conversion

We present a method for converting the voices between a set of speakers. Our method is based on training multiple autoencoder paths, where there is a single speaker-independent encoder and multiple speaker-dependent decoders. The autoencoders are trained with an addition of an adversarial loss which is provided by an auxiliary classifier in order to guide the output of the encoder to be speaker independent. The training of the model is unsupervised in the sense that it does not require collecting the same utterances from the speakers nor does it require time aligning over phonemes. Due to the use of a single encoder, our method can generalize to converting the voice of out-of-training speakers to speakers in the training dataset. We present subjective tests corroborating the performance of our method.
Exploring the Hyperparameter Landscape of Adversarial Robustness
Authors: Evelyn Duesterwald, Anupama Murthi, Ganesh Venkataraman, Mathieu Sinn, Deepak Vijaykeerthy
Date: 2019-05-09 00:00:00
Tasks: Adversarial Robustness, Hyperparameter Optimization, Navigate

Adversarial training shows promise as an approach for training models that are robust towards adversarial perturbation. In this paper, we explore some of the practical challenges of adversarial training. We present a sensitivity analysis that illustrates that the effectiveness of adversarial training hinges on the settings of a few salient hyperparameters. We show that the robustness surface that emerges across these salient parameters can be surprisingly complex and that therefore no effective one-size-fits-all parameter settings exist. We then demonstrate that we can use the same salient hyperparameters as tuning knob to navigate the tension that can arise between robustness and accuracy. Based on these findings, we present a practical approach that leverages hyperparameter optimization techniques for tuning adversarial training to maximize robustness while keeping the loss in accuracy within a defined budget.
Universal Adversarial Perturbations for Speech Recognition Systems
Authors: Paarth Neekhara, Shehzeen Hussain, Prakhar Pandey, Shlomo Dubnov, Julian McAuley, Farinaz Koushanfar
Date: 2019-05-09 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

In this work, we demonstrate the existence of universal adversarial audio perturbations that cause mis-transcription of audio signals by automatic speech recognition (ASR) systems. We propose an algorithm to find a single quasi-imperceptible perturbation, which when added to any arbitrary speech signal, will most likely fool the victim speech recognition model. Our experiments demonstrate the application of our proposed technique by crafting audio-agnostic universal perturbations for the state-of-the-art ASR system -- Mozilla DeepSpeech. Additionally, we show that such perturbations generalize to a significant extent across models that are not available during training, by performing a transferability test on a WaveNet based ASR system.
Random Function Priors for Correlation Modeling
Authors: Aonan Zhang, John Paisley
Date: 2019-05-09 00:00:00
Tasks: Variational Inference

The likelihood model of high dimensional data $X_n$ can often be expressed as $p(X_n|Z_n,\theta)$, where $\theta\mathrel{\mathop:}=(\theta_k)_{k\in[K]}$ is a collection of hidden features shared across objects, indexed by $n$, and $Z_n$ is a non-negative factor loading vector with $K$ entries where $Z_{nk}$ indicates the strength of $\theta_k$ used to express $X_n$. In this paper, we introduce random function priors for $Z_n$ for modeling correlations among its $K$ dimensions $Z_{n1}$ through $Z_{nK}$, which we call \textit{population random measure embedding} (PRME). Our model can be viewed as a generalized paintbox model~\cite{Broderick13} using random functions, and can be learned efficiently with neural networks via amortized variational inference. We derive our Bayesian nonparametric method by applying a representation theorem on separately exchangeable discrete random measures.
Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss
Authors: Lele Chen, Ross K. Maddox, Zhiyao Duan, Chenliang Xu
Date: 2019-05-09 00:00:00

We devise a cascade GAN approach to generate talking face video, which is robust to different face shapes, view angles, facial characteristics, and noisy audio conditions. Instead of learning a direct mapping from audio to video frames, we propose first to transfer audio to high-level structure, i.e., the facial landmarks, and then to generate video frames conditioned on the landmarks. Compared to a direct audio-to-image approach, our cascade approach avoids fitting spurious correlations between audiovisual signals that are irrelevant to the speech content. We, humans, are sensitive to temporal discontinuities and subtle artifacts in video. To avoid those pixel jittering problems and to enforce the network to focus on audiovisual-correlated regions, we propose a novel dynamically adjustable pixel-wise loss with an attention mechanism. Furthermore, to generate a sharper image with well-synchronized facial movements, we propose a novel regression-based discriminator structure, which considers sequence-level information along with frame-level information. Thoughtful experiments on several datasets and real-world samples demonstrate significantly better results obtained by our method than the state-of-the-art methods in both quantitative and qualitative comparisons.
Beta Survival Models
Authors: David Hubbard, Benoit Rostykus, Yves Raimond, Tony Jebara
Date: 2019-05-09 00:00:00

This article analyzes the problem of estimating the time until an event occurs, also known as survival modeling. We observe through substantial experiments on large real-world datasets and use-cases that populations are largely heterogeneous. Sub-populations have different mean and variance in their survival rates requiring flexible models that capture heterogeneity. We leverage a classical extension of the logistic function into the survival setting to characterize unobserved heterogeneity using the beta distribution. This yields insights into the geometry of the problem as well as efficient estimation methods for linear, tree and neural network models that adjust the beta distribution based on observed covariates. We also show that the additional information captured by the beta distribution leads to interesting ranking implications as we determine who is most-at-risk. We show theoretically that the ranking is variable as we forecast forward in time and prove that pairwise comparisons of survival remain transitive. Empirical results using large-scale datasets across two use-cases (online conversions and retention modeling), demonstrate the competitiveness of the method. The simplicity of the method and its ability to capture skew in the data makes it a viable alternative to standard techniques particularly when we are interested in the time to event and when the underlying probabilities are heterogeneous.
On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization
Authors: Hao Yu, Rong Jin, Sen yang
Date: 2019-05-09 00:00:00
Tasks: BIG-bench Machine Learning

Recent developments on large-scale distributed machine learning applications, e.g., deep neural networks, benefit enormously from the advances in distributed non-convex optimization techniques, e.g., distributed Stochastic Gradient Descent (SGD). A series of recent works study the linear speedup property of distributed SGD variants with reduced communication. The linear speedup property enable us to scale out the computing capability by adding more computing nodes into our system. The reduced communication complexity is desirable since communication overhead is often the performance bottleneck in distributed systems. Recently, momentum methods are more and more widely adopted in training machine learning models and can often converge faster and generalize better. For example, many practitioners use distributed SGD with momentum to train deep neural networks with big data. However, it remains unclear whether any distributed momentum SGD possesses the same linear speedup property as distributed SGD and has reduced communication complexity. This paper fills the gap by considering a distributed communication efficient momentum SGD method and proving its linear speedup property.
Non-Asymptotic Gap-Dependent Regret Bounds for Tabular MDPs
Authors: Max Simchowitz, Kevin Jamieson
Date: 2019-05-09 00:00:00
Proceeding: NeurIPS 2019 12

This paper establishes that optimistic algorithms attain gap-dependent and non-asymptotic logarithmic regret for episodic MDPs. In contrast to prior work, our bounds do not suffer a dependence on diameter-like quantities or ergodicity, and smoothly interpolate between the gap dependent logarithmic-regret, and the $\widetilde{\mathcal{O}}(\sqrt{HSAT})$-minimax rate. The key technique in our analysis is a novel "clipped" regret decomposition which applies to a broad family of recent optimistic algorithms for episodic MDPs.
When Deep Learning Met Code Search
Authors: Jose Cambronero, Hongyu Li, Seohyun Kim, Koushik Sen, Satish Chandra
Date: 2019-05-09 00:00:00
Tasks: Code Search, Deep Learning, Natural Language Queries

There have been multiple recent proposals on using deep neural networks for code search using natural language. Common across these proposals is the idea of $\mathit{embedding}$ code and natural language queries, into real vectors and then using vector distance to approximate semantic correlation between code and the query. Multiple approaches exist for learning these embeddings, including $\mathit{unsupervised}$ techniques, which rely only on a corpus of code examples, and $\mathit{supervised}$ techniques, which use an $\mathit{aligned}$ corpus of paired code and natural language descriptions. The goal of this supervision is to produce embeddings that are more similar for a query and the corresponding desired code snippet. Clearly, there are choices in whether to use supervised techniques at all, and if one does, what sort of network and training to use for supervision. This paper is the first to evaluate these choices systematically. To this end, we assembled implementations of state-of-the-art techniques to run on a common platform, training and evaluation corpora. To explore the design space in network complexity, we also introduced a new design point that is a $\mathit{minimal}$ supervision extension to an existing unsupervised technique. Our evaluation shows that: 1. adding supervision to an existing unsupervised technique can improve performance, though not necessarily by much; 2. simple networks for supervision can be more effective that more sophisticated sequence-based networks for code search; 3. while it is common to use docstrings to carry out supervision, there is a sizeable gap between the effectiveness of docstrings and a more query-appropriate supervision corpus. The evaluation dataset is now available at arXiv:1908.09804
Convolutional Neural Networks Utilizing Multifunctional Spin-Hall MTJ Neurons
Authors: Andrew W. Stephan, Steven J. Koester
Date: 2019-05-09 00:00:00

We propose a new network architecture for standard spin-Hall magnetic tunnel junction-based spintronic neurons that allows them to compute multiple critical convolutional neural network functionalities simultaneously and in parallel, saving space and time. An approximation to the Rectified Linear Unit transfer function and the local pooling function are computed simultaneously with the convolution operation itself. A proof-of-concept simulation is performed on the MNIST dataset, achieving up to 98% accuracy at a cost of less than 1 nJ for all convolution, activation and pooling operations combined. The simulations are remarkably robust to thermal noise, performing well even with very small magnetic layers.
Wearable Sensor Data Based Human Activity Recognition using Machine Learning: A new approach
Authors: H. D. Nguyen, K. P. Tran, X. Zeng, L. Koehl, G. Tartare
Date: 2019-05-09 00:00:00
Tasks: Activity Recognition, BIG-bench Machine Learning, Ensemble Learning, Human Activity Recognition

Recent years have witnessed the rapid development of human activity recognition (HAR) based on wearable sensor data. One can find many practical applications in this area, especially in the field of health care. Many machine learning algorithms such as Decision Trees, Support Vector Machine, Naive Bayes, K-Nearest Neighbor, and Multilayer Perceptron are successfully used in HAR. Although these methods are fast and easy for implementation, they still have some limitations due to poor performance in a number of situations. In this paper, we propose a novel method based on the ensemble learning to boost the performance of these machine learning methods for HAR.
Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting
Authors: Rajat Sen, Hsiang-Fu Yu, Inderjit Dhillon
Date: 2019-05-09 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Demand Forecasting, Time Series, Time Series Analysis, Time Series Forecasting

Forecasting high-dimensional time series plays a crucial role in many applications such as demand forecasting and financial predictions. Modern datasets can have millions of correlated time-series that evolve together, i.e they are extremely high dimensional (one dimension for each individual time-series). There is a need for exploiting global patterns and coupling them with local calibration for better prediction. However, most recent deep learning approaches in the literature are one-dimensional, i.e, even though they are trained on the whole dataset, during prediction, the future forecast for a single dimension mainly depends on past values from the same dimension. In this paper, we seek to correct this deficiency and propose DeepGLO, a deep forecasting model which thinks globally and acts locally. In particular, DeepGLO is a hybrid model that combines a global matrix factorization model regularized by a temporal convolution network, along with another temporal network that can capture local properties of each time-series and associated covariates. Our model can be trained effectively on high-dimensional but diverse time series, where different time series can have vastly different scales, without a priori normalization or rescaling. Empirical results demonstrate that DeepGLO can outperform state-of-the-art approaches; for example, we see more than 25% improvement in WAPE over other methods on a public dataset that contains more than 100K-dimensional time series.
Two-layer Near-lossless HDR Coding with Backward Compatibility to JPEG
Authors: Hiroyuki Kobayashi, Osamu Watanabe, Hitoshi Kiya
Date: 2019-05-09 00:00:00
Tasks: Image Compression, Quantization, Vocal Bursts Valence Prediction

We propose an efficient two-layer near-lossless coding method using an extended histogram packing technique with backward compatibility to the legacy JPEG standard. The JPEG XT, which is the international standard to compress HDR images, adopts a two-layer coding method for backward compatibility to the legacy JPEG standard. However, there are two problems with this two-layer coding method. One is that it does not exhibit better near-lossless performance than other methods for HDR image compression with single-layer structure. The other problem is that the determining the appropriate values of the coding parameters may be required for each input image to achieve good compression performance of near-lossless compression with the two-layer coding method of the JPEG XT. To solve these problems, we focus on a histogram-packing technique that takes into account the histogram sparseness of HDR images. We used zero-skip quantization, which is an extension of the histogram-packing technique proposed for lossless coding, for implementing the proposed near-lossless coding method. The experimental results indicate that the proposed method exhibits not only a better near-lossless compression performance than that of the two-layer coding method of the JPEG XT, but also there are no issue regarding the combination of parameter values without losing backward compatibility to the JPEG standard.
Kindling the Darkness: A Practical Low-light Image Enhancer
Authors: Yonghua Zhang, Jiawan Zhang, Xiaojie Guo
Date: 2019-05-04 00:00:00
Tasks: GPU, Low-Light Image Enhancement

Images captured under low-light conditions often suffer from (partially) poor visibility. Besides unsatisfactory lightings, multiple types of degradations, such as noise and color distortion due to the limited quality of cameras, hide in the dark. In other words, solely turning up the brightness of dark regions will inevitably amplify hidden artifacts. This work builds a simple yet effective network for \textbf{Kin}dling the \textbf{D}arkness (denoted as KinD), which, inspired by Retinex theory, decomposes images into two components. One component (illumination) is responsible for light adjustment, while the other (reflectance) for degradation removal. In such a way, the original space is decoupled into two smaller subspaces, expecting to be better regularized/learned. It is worth to note that our network is trained with paired images shot under different exposure conditions, instead of using any ground-truth reflectance and illumination information. Extensive experiments are conducted to demonstrate the efficacy of our design and its superiority over state-of-the-art alternatives. Our KinD is robust against severe visual defects, and user-friendly to arbitrarily adjust light levels. In addition, our model spends less than 50ms to process an image in VGA resolution on a 2080Ti GPU. All the above merits make our KinD attractive for practical use.
Solving zero-sum extensive-form games with arbitrary payoff uncertainty models
Authors: Juan Leni, John Levine, John Quigley
Date: 2019-04-24 00:00:00
Tasks: counterfactual, Form

Modeling strategic conflict from a game theoretical perspective involves
dealing with epistemic uncertainty. Payoff uncertainty models are typically
restricted to simple probability models due to computational restrictions.
Recent breakthroughs Artificial Intelligence (AI) research applied to Poker
have resulted in novel approximation approaches such as counterfactual regret
minimization, that can successfully deal with large-scale imperfect games. By
drawing from these ideas, this work addresses the problem of arbitrary
continuous payoff distributions. We propose a method, Harsanyi-Counterfactual
Regret Minimization, to solve two-player zero-sum extensive-form games with
arbitrary payoff distribution models. Given a game $\Gamma$, using a Harsanyi
transformation we generate a new game $\Gamma^\#$ to which we later apply
Counterfactual Regret Minimization to obtain $\varepsilon$-Nash equilibria. We
include numerical experiments showing how the method can be applied to a
previously published problem.
A reconstruction of the multipreference closure
Authors: Laura Giordano, Valentina Gliozzi
Date: 2019-05-05 00:00:00
Tasks: Relation

The paper describes a preferential approach for dealing with exceptions in KLM preferential logics, based on the rational closure. It is well known that the rational closure does not allow an independent handling of the inheritance of different defeasible properties of concepts. Several solutions have been proposed to face this problem and the lexicographic closure is the most notable one. In this work, we consider an alternative closure construction, called the Multi Preference closure (MP-closure), that has been first considered for reasoning with exceptions in DLs. Here, we reconstruct the notion of MP-closure in the propositional case and we show that it is a natural variant of Lehmann's lexicographic closure. Abandoning Maximal Entropy (an alternative route already considered but not explored by Lehmann) leads to a construction which exploits a different lexicographic ordering w.r.t. the lexicographic closure, and determines a preferential consequence relation rather than a rational consequence relation. We show that, building on the MP-closure semantics, rationality can be recovered, at least from the semantic point of view, resulting in a rational consequence relation which is stronger than the rational closure, but incomparable with the lexicographic closure. We also show that the MP-closure is stronger than the Relevant Closure.