pwc-archive/papers-with-abstracts huggingface dataset:
URSA: A Neural Network for Unordered Point Clouds Using Constellations
Authors: Mark B. Skouson, Brett J. Borghetti, Robert C. Leishman
Date: 2018-08-14 00:00:00
Tasks: Classification, General Classification, Point Cloud Classification

This paper describes a neural network layer, named Ursa, that uses a
constellation of points to learn classification information from point cloud
data. Unlike other machine learning classification problems where the task is
to classify an individual high-dimensional observation, in a point-cloud
classification problem the goal is to classify a set of d-dimensional
observations. Because a point cloud is a set, there is no ordering to the
collection of points in a point-cloud classification problem. Thus, the
challenge of classifying point clouds inputs is in building a classifier which
is agnostic to the ordering of the observations, yet preserves the
d-dimensional information of each point in the set. This research presents
Ursa, a new layer type for an artificial neural network which achieves these
two properties. Similar to new methods for this task, this architecture works
directly on d-dimensional points rather than first converting the points to a
d-dimensional volume. The Ursa layer is followed by a series of dense layers to
classify 2D and 3D objects from point clouds. Experiments on ModelNet40 and
MNIST data show classification results comparable with current methods, while
reducing the training parameters by over 50 percent.
Gradient descent in some simple settings
Authors: Y. Cooper
Date: 2018-08-14 00:00:00

In this note, we observe the behavior of gradient flow and discrete and noisy
gradient descent in some simple settings. It is commonly noted that addition of
noise to gradient descent can affect the trajectory of gradient descent. Here,
we run some computer experiments for gradient descent on some simple functions,
and observe this principle in some concrete examples.
VizML: A Machine Learning Approach to Visualization Recommendation
Authors: Kevin Z. Hu, Michiel A. Bakker, Stephen Li, Tim Kraska, César A. Hidalgo
Date: 2018-08-14 00:00:00
Tasks: BIG-bench Machine Learning, Data Visualization, Recommendation Systems

Data visualization should be accessible for all analysts with data, not just
the few with technical expertise. Visualization recommender systems aim to
lower the barrier to exploring basic visualizations by automatically generating
results for analysts to search and select, rather than manually specify. Here,
we demonstrate a novel machine learning-based approach to visualization
recommendation that learns visualization design choices from a large corpus of
datasets and associated visualizations. First, we identify five key design
choices made by analysts while creating visualizations, such as selecting a
visualization type and choosing to encode a column along the X- or Y-axis. We
train models to predict these design choices using one million
dataset-visualization pairs collected from a popular online visualization
platform. Neural networks predict these design choices with high accuracy
compared to baseline models. We report and interpret feature importances from
one of these baseline models. To evaluate the generalizability and uncertainty
of our approach, we benchmark with a crowdsourced test set, and show that the
performance of our model is comparable to human performance when predicting
consensus visualization type, and exceeds that of other ML-based systems.
ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder
Authors: Hirokazu Kameoka, Takuhiro Kaneko, Kou Tanaka, Nobukatsu Hojo
Date: 2018-08-13 00:00:00
Tasks: Attribute, Decoder, Voice Conversion

This paper proposes a non-parallel many-to-many voice conversion (VC) method using a variant of the conditional variational autoencoder (VAE) called an auxiliary classifier VAE (ACVAE). The proposed method has three key features. First, it adopts fully convolutional architectures to construct the encoder and decoder networks so that the networks can learn conversion rules that capture time dependencies in the acoustic feature sequences of source and target speech. Second, it uses an information-theoretic regularization for the model training to ensure that the information in the attribute class label will not be lost in the conversion process. With regular CVAEs, the encoder and decoder are free to ignore the attribute class label input. This can be problematic since in such a situation, the attribute class label will have little effect on controlling the voice characteristics of input speech at test time. Such situations can be avoided by introducing an auxiliary classifier and training the encoder and decoder so that the attribute classes of the decoder outputs are correctly predicted by the classifier. Third, it avoids producing buzzy-sounding speech at test time by simply transplanting the spectral details of the input speech into its converted version. Subjective evaluation experiments revealed that this simple method worked reasonably well in a non-parallel many-to-many speaker identity conversion task.
A Framework for Automated Cellular Network Tuning with Reinforcement Learning
Authors: Faris B. Mismar, Jinseok Choi, Brian L. Evans
Date: 2018-08-13 00:00:00
Tasks: Management, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Tuning cellular network performance against always occurring wireless impairments can dramatically improve reliability to end users. In this paper, we formulate cellular network performance tuning as a reinforcement learning (RL) problem and provide a solution to improve the performance for indoor and outdoor environments. By leveraging the ability of Q-learning to estimate future performance improvement rewards, we propose two algorithms: (1) closed loop power control (PC) for downlink voice over LTE (VoLTE) and (2) self-organizing network (SON) fault management. The VoLTE PC algorithm uses RL to adjust the indoor base station transmit power so that the signal to interference plus noise ratio (SINR) of a user equipment (UE) meets the target SINR. It does so without the UE having to send power control requests. The SON fault management algorithm uses RL to improve the performance of an outdoor base station cluster by resolving faults in the network through configuration management. Both algorithms exploit measurements from the connected users, wireless impairments, and relevant configuration parameters to solve a non-convex performance optimization problem using RL. Simulation results show that our proposed RL based algorithms outperform the industry standards today in realistic cellular communication environments.
Ghost imaging with the human eye
Authors: Alessandro Boccolini, Alessandro Fedrizzi, Daniele Faccio
Date: 2018-08-13 00:00:00

Computational ghost imaging relies on the decomposition of an image into
patterns that are summed together with weights that measure the overlap of each
pattern with the scene being imaged. These tasks rely on a computer. Here we
demonstrate that the computational integration can be performed directly with
the human eye. We use this human ghost imaging technique to evaluate the
temporal response of the eye and establish the image persistence time to be
around 20 ms followed by a further 20 ms exponential decay. These persistence
times are in agreement with previous studies but can now potentially be
extended to include a more precise characterisation of visual stimuli and
provide a new experimental tool for the study of visual perception.
COLA: Decentralized Linear Learning
Authors: Lie He, An Bian, Martin Jaggi
Date: 2018-08-13 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: BIG-bench Machine Learning, CoLA, General Classification, regression

Decentralized machine learning is a promising emerging paradigm in view of global challenges of data ownership and privacy. We consider learning of linear classification and regression models, in the setting where the training data is decentralized over many user devices, and the learning algorithm must run on-device, on an arbitrary communication network, without a central coordinator. We propose COLA, a new decentralized training algorithm with strong theoretical guarantees and superior practical performance. Our framework overcomes many limitations of existing methods, and achieves communication efficiency, scalability, elasticity as well as resilience to changes in data and participating devices.
R$^3$-Net: A Deep Network for Multi-oriented Vehicle Detection in Aerial Images and Videos
Authors: Qingpeng Li, Lichao Mou, Qizhi Xu, Yun Zhang, Xiao Xiang Zhu
Date: 2018-08-16 00:00:00
Tasks: Motion Estimation, Position, Region Proposal, vehicle detection

Vehicle detection is a significant and challenging task in aerial remote
sensing applications. Most existing methods detect vehicles with regular
rectangle boxes and fail to offer the orientation of vehicles. However, the
orientation information is crucial for several practical applications, such as
the trajectory and motion estimation of vehicles. In this paper, we propose a
novel deep network, called rotatable region-based residual network (R$^3$-Net),
to detect multi-oriented vehicles in aerial images and videos. More specially,
R$^3$-Net is utilized to generate rotatable rectangular target boxes in a half
coordinate system. First, we use a rotatable region proposal network (R-RPN) to
generate rotatable region of interests (R-RoIs) from feature maps produced by a
deep convolutional neural network. Here, a proposed batch averaging rotatable
anchor (BAR anchor) strategy is applied to initialize the shape of vehicle
candidates. Next, we propose a rotatable detection network (R-DN) for the final
classification and regression of the R-RoIs. In R-DN, a novel rotatable
position sensitive pooling (R-PS pooling) is designed to keep the position and
orientation information simultaneously while downsampling the feature maps of
R-RoIs. In our model, R-RPN and R-DN can be trained jointly. We test our
network on two open vehicle detection image datasets, namely DLR 3K Munich
Dataset and VEDAI Dataset, demonstrating the high precision and robustness of
our method. In addition, further experiments on aerial videos show the good
generalization capability of the proposed method and its potential for vehicle
tracking in aerial videos. The demo video is available at
https://youtu.be/xCYD-tYudN0.
Distributionally Adversarial Attack
Authors: Tianhang Zheng, Changyou Chen, Kui Ren
Date: 2018-08-16 00:00:00
Tasks: Adversarial Attack

Recent work on adversarial attack has shown that Projected Gradient Descent
(PGD) Adversary is a universal first-order adversary, and the classifier
adversarially trained by PGD is robust against a wide range of first-order
attacks. It is worth noting that the original objective of an attack/defense
model relies on a data distribution $p(\mathbf{x})$, typically in the form of
risk maximization/minimization, e.g.,
$\max/\min\mathbb{E}_{p(\mathbf(x))}\mathcal{L}(\mathbf{x})$ with
$p(\mathbf{x})$ some unknown data distribution and $\mathcal{L}(\cdot)$ a loss
function. However, since PGD generates attack samples independently for each
data sample based on $\mathcal{L}(\cdot)$, the procedure does not necessarily
lead to good generalization in terms of risk optimization. In this paper, we
achieve the goal by proposing distributionally adversarial attack (DAA), a
framework to solve an optimal {\em adversarial-data distribution}, a perturbed
distribution that satisfies the $L_\infty$ constraint but deviates from the
original data distribution to increase the generalization risk maximally.
Algorithmically, DAA performs optimization on the space of potential data
distributions, which introduces direct dependency between all data points when
generating adversarial samples. DAA is evaluated by attacking state-of-the-art
defense models, including the adversarially-trained models provided by {\em MIT
MadryLab}. Notably, DAA ranks {\em the first place} on MadryLab's white-box
leaderboards, reducing the accuracy of their secret MNIST model to $88.79\%$
(with $l_\infty$ perturbations of $\epsilon = 0.3$) and the accuracy of their
secret CIFAR model to $44.71\%$ (with $l_\infty$ perturbations of $\epsilon =
8.0$). Code for the experiments is released on
\url{https://github.com/tianzheng4/Distributionally-Adversarial-Attack}.
Performance, Power, and Area Design Trade-offs in Millimeter-Wave Transmitter Beamforming Architectures
Authors: Han Yan, Sridhar Ramesh, Timothy Gallagher, Curtis Ling, Danijela Cabric
Date: 2018-07-19 00:00:00
Tasks: Quantization

Millimeter wave (mmWave) communications is viewed as the key enabler of 5G
cellular networks due to vast spectrum availability that could boost peak rate
and capacity. Due to increased propagation loss in mmWave band, transceivers
with massive antenna array are required to meet link budget, but their power
consumption and cost become limiting factors for commercial systems. Radio
designs based on hybrid digital and analog array architectures and the usage of
radio frequency (RF) signal processing via phase shifters have emerged as
potential solutions to improve radio energy efficiency and deliver performances
close to conventional digital antenna arrays. In this paper, we provide an
overview of the state-of-the-art mmWave massive antenna array designs and
comparison among three array architectures, namely digital array,
partially-connected hybrid array (sub-array), and fully-connected hybrid array.
The comparison of performance, power, and area for these three architectures is
performed for three typical 5G downlink use cases. This is the first study to
comprehensively model and quantitatively analyze all design aspects and
criteria including: 1) optimal beamforming precoder, 2) quantization accuracy
in digital-to-analog converter (DAC) and phase shifters, 3) RF signal
distribution losses, 4) power and area based on state-of-the-art mmWave
circuits including high-speed DACs, mixers, phase shifters, and power
amplifiers. The analysis shows that the hybrid architecture provides marginal,
if any, benefits over the digital array. It also reveals that sub-array
architecture suffers from reduced beamforming gain due to array partitioning,
which has to be compensated with additional transmission power and signal
processing. Fully-connected hybrid architecture is limited by significant RF
signal distribution loss and corresponding cost of RF amplifiers needed to
compensate this loss.
Learning Graph Embeddings from WordNet-based Similarity Measures
Authors: Andrey Kutuzov, Mohammad Dorgham, Oleksiy Oliynyk, Chris Biemann, Alexander Panchenko
Date: 2018-08-16 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Graph Embedding, Semantic Similarity, Semantic Textual Similarity, Word Sense Disambiguation

We present path2vec, a new approach for learning graph embeddings that relies
on structural measures of pairwise node similarities. The model learns
representations for nodes in a dense space that approximate a given
user-defined graph distance measure, such as e.g. the shortest path distance or
distance measures that take information beyond the graph structure into
account. Evaluation of the proposed model on semantic similarity and word sense
disambiguation tasks, using various WordNet-based similarity measures, show
that our approach yields competitive results, outperforming strong graph
embedding baselines. The model is computationally efficient, being orders of
magnitude faster than the direct computation of graph-based distances.
Improving Conditional Sequence Generative Adversarial Networks by Stepwise Evaluation
Authors: Yi-Lin Tuan, Hung-Yi Lee
Date: 2018-08-16 00:00:00
Tasks: Dialogue Generation

Sequence generative adversarial networks (SeqGAN) have been used to improve
conditional sequence generation tasks, for example, chit-chat dialogue
generation. To stabilize the training of SeqGAN, Monte Carlo tree search (MCTS)
or reward at every generation step (REGS) is used to evaluate the goodness of a
generated subsequence. MCTS is computationally intensive, but the performance
of REGS is worse than MCTS. In this paper, we propose stepwise GAN (StepGAN),
in which the discriminator is modified to automatically assign scores
quantifying the goodness of each subsequence at every generation step. StepGAN
has significantly less computational costs than MCTS. We demonstrate that
StepGAN outperforms previous GAN-based methods on both synthetic experiment and
chit-chat dialogue generation.
Deep Convolutional Networks as shallow Gaussian Processes
Authors: Adrià Garriga-Alonso, Carl Edward Rasmussen, Laurence Aitchison
Date: 2018-08-16 00:00:00
Proceeding: ICLR 2019 5
Tasks: Gaussian Processes, General Classification

We show that the output of a (residual) convolutional neural network (CNN) with an appropriate prior over the weights and biases is a Gaussian process (GP) in the limit of infinitely many convolutional filters, extending similar results for dense networks. For a CNN, the equivalent kernel can be computed exactly and, unlike "deep kernels", has very few parameters: only the hyperparameters of the original CNN. Further, we show that this kernel has two properties that allow it to be computed efficiently; the cost of evaluating the kernel for a pair of images is similar to a single forward pass through the original CNN with only one filter per layer. The kernel equivalent to a 32-layer ResNet obtains 0.84% classification error on MNIST, a new record for GPs with a comparable number of parameters.
BlockQNN: Efficient Block-wise Neural Network Architecture Generation
Authors: Zhao Zhong, Zichen Yang, Boyang Deng, Junjie Yan, Wei Wu, Jing Shao, Cheng-Lin Liu
Date: 2018-08-16 00:00:00
Tasks: GPU, image-classification, Image Classification, Q-Learning

Convolutional neural networks have gained a remarkable success in computer
vision. However, most usable network architectures are hand-crafted and usually
require expertise and elaborate design. In this paper, we provide a block-wise
network generation pipeline called BlockQNN which automatically builds
high-performance networks using the Q-Learning paradigm with epsilon-greedy
exploration strategy. The optimal network block is constructed by the learning
agent which is trained to choose component layers sequentially. We stack the
block to construct the whole auto-generated network. To accelerate the
generation process, we also propose a distributed asynchronous framework and an
early stop strategy. The block-wise generation brings unique advantages: (1) it
yields state-of-the-art results in comparison to the hand-crafted networks on
image classification, particularly, the best network generated by BlockQNN
achieves 2.35% top-1 error rate on CIFAR-10. (2) it offers tremendous reduction
of the search space in designing networks, spending only 3 days with 32 GPUs. A
faster version can yield a comparable result with only 1 GPU in 20 hours. (3)
it has strong generalizability in that the network built on CIFAR also performs
well on the larger-scale dataset. The best network achieves very competitive
accuracy of 82.0% top-1 and 96.0% top-5 on ImageNet.
LARNN: Linear Attention Recurrent Neural Network
Authors: Guillaume Chevalier
Date: 2018-08-16 00:00:00
Tasks: Time Series, Time Series Analysis

The Linear Attention Recurrent Neural Network (LARNN) is a recurrent
attention module derived from the Long Short-Term Memory (LSTM) cell and ideas
from the consciousness Recurrent Neural Network (RNN). Yes, it LARNNs. The
LARNN uses attention on its past cell state values for a limited window size
$k$. The formulas are also derived from the Batch Normalized LSTM (BN-LSTM)
cell and the Transformer Network for its Multi-Head Attention Mechanism. The
Multi-Head Attention Mechanism is used inside the cell such that it can query
its own $k$ past values with the attention window. This has the effect of
augmenting the rank of the tensor with the attention mechanism, such that the
cell can perform complex queries to question its previous inner memories, which
should augment the long short-term effect of the memory. With a clever trick,
the LARNN cell with attention can be easily used inside a loop on the cell
state, just like how any other Recurrent Neural Network (RNN) cell can be
looped linearly through time series. This is due to the fact that its state,
which is looped upon throughout time steps within time series, stores the inner
states in a "first in, first out" queue which contains the $k$ most recent
states and on which it is easily possible to add static positional encoding
when the queue is represented as a tensor. This neural architecture yields
better results than the vanilla LSTM cells. It can obtain results of 91.92% for
the test accuracy, compared to the previously attained 91.65% using vanilla
LSTM cells. Note that this is not to compare to other research, where up to
93.35% is obtained, but costly using 18 LSTM cells rather than with 2 to 3
cells as analyzed here. Finally, an interesting discovery is made, such that
adding activation within the multi-head attention mechanism's linear layers can
yield better results in the context researched hereto.
Deeper Image Quality Transfer: Training Low-Memory Neural Networks for 3D Images
Authors: Stefano B. Blumberg, Ryutaro Tanno, Iasonas Kokkinos, Daniel C. Alexander
Date: 2018-08-16 00:00:00
Tasks: Deep Learning

In this paper we address the memory demands that come with the processing of
3-dimensional, high-resolution, multi-channeled medical images in deep
learning. We exploit memory-efficient backpropagation techniques, to reduce the
memory complexity of network training from being linear in the network's depth,
to being roughly constant $ - $ permitting us to elongate deep architectures
with negligible memory increase. We evaluate our methodology in the paradigm of
Image Quality Transfer, whilst noting its potential application to various
tasks that use deep learning. We study the impact of depth on accuracy and show
that deeper models have more predictive power, which may exploit larger
training sets. We obtain substantially better results than the previous
state-of-the-art model with a slight memory increase, reducing the
root-mean-squared-error by $ 13\% $. Our code is publicly available.
The linear hidden subset problem for the (1+1) EA with scheduled and adaptive mutation rates
Authors: Hafsteinn Einarsson, Marcelo Matheus Gauy, Johannes Lengler, Florian Meier, Asier Mujika, Angelika Steger, Felix Weissenberger
Date: 2018-08-16 00:00:00
Tasks: Evolutionary Algorithms

We study unbiased $(1+1)$ evolutionary algorithms on linear functions with an
unknown number $n$ of bits with non-zero weight. Static algorithms achieve an
optimal runtime of $O(n (\ln n)^{2+\epsilon})$, however, it remained unclear
whether more dynamic parameter policies could yield better runtime guarantees.
We consider two setups: one where the mutation rate follows a fixed schedule,
and one where it may be adapted depending on the history of the run. For the
first setup, we give a schedule that achieves a runtime of $(1\pm o(1))\beta n
\ln n$, where $\beta \approx 3.552$, which is an asymptotic improvement over
the runtime of the static setup. Moreover, we show that no schedule admits a
better runtime guarantee and that the optimal schedule is essentially unique.
For the second setup, we show that the runtime can be further improved to
$(1\pm o(1)) e n \ln n$, which matches the performance of algorithms that know
$n$ in advance.
  Finally, we study the related model of initial segment uncertainty with
static position-dependent mutation rates, and derive asymptotically optimal
lower bounds. This answers a question by Doerr, Doerr, and K\"otzing.