pwc-archive/papers-with-abstracts huggingface dataset:
Multi-Level Sequence GAN for Group Activity Recognition
Authors: Harshala Gammulle, Simon Denman, Sridha Sridharan, Clinton Fookes
Date: 2018-12-18 00:00:00
Tasks: Action Classification, Activity Prediction, Activity Recognition, General Classification, Generative Adversarial Network, Group Activity Recognition

We propose a novel semi-supervised, Multi-Level Sequential Generative
Adversarial Network (MLS-GAN) architecture for group activity recognition. In
contrast to previous works which utilise manually annotated individual human
action predictions, we allow the models to learn it's own internal
representations to discover pertinent sub-activities that aid the final group
activity recognition task. The generator is fed with person-level and
scene-level features that are mapped temporally through LSTM networks.
Action-based feature fusion is performed through novel gated fusion units that
are able to consider long-term dependencies, exploring the relationships among
all individual actions, to learn an intermediate representation or `action
code' for the current group activity. The network achieves its semi-supervised
behaviour by allowing it to perform group action classification together with
the adversarial real/fake validation. We perform extensive evaluations on
different architectural variants to demonstrate the importance of the proposed
architecture. Furthermore, we show that utilising both person-level and
scene-level features facilitates the group activity prediction better than
using only person-level features. Our proposed architecture outperforms current
state-of-the-art results for sports and pedestrian based classification tasks
on Volleyball and Collective Activity datasets, showing it's flexible nature
for effective learning of group activities.
Composing Text and Image for Image Retrieval - An Empirical Odyssey
Authors: Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, James Hays
Date: 2018-12-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Retrieval, Image Retrieval with Multi-Modal Query, Retrieval

In this paper, we study the task of image retrieval, where the input query is
specified in the form of an image plus some text that describes desired
modifications to the input image. For example, we may present an image of the
Eiffel tower, and ask the system to find images which are visually similar but
are modified in small ways, such as being taken at nighttime instead of during
the day. To tackle this task, we learn a similarity metric between a target
image and a source image plus source text, an embedding and composing function
such that target image feature is close to the source image plus text
composition feature. We propose a new way to combine image and text using such
function that is designed for the retrieval task. We show this outperforms
existing approaches on 3 different datasets, namely Fashion-200k, MIT-States
and a new synthetic dataset we create based on CLEVR. We also show that our
approach can be used to classify input queries, in addition to image retrieval.
Retinal vessel segmentation based on Fully Convolutional Neural Networks
Authors: Américo Oliveira, Sérgio Pereira, Carlos A. Silva
Date: 2018-12-18 00:00:00
Tasks: Data Augmentation, Retinal Vessel Segmentation, Segmentation

The retinal vascular condition is a reliable biomarker of several
ophthalmologic and cardiovascular diseases, so automatic vessel segmentation
may be crucial to diagnose and monitor them. In this paper, we propose a novel
method that combines the multiscale analysis provided by the Stationary Wavelet
Transform with a multiscale Fully Convolutional Neural Network to cope with the
varying width and direction of the vessel structure in the retina. Our proposal
uses rotation operations as the basis of a joint strategy for both data
augmentation and prediction, which allows us to explore the information learned
during training to refine the segmentation. The method was evaluated on three
publicly available databases, achieving an average accuracy of 0.9576, 0.9694,
and 0.9653, and average area under the ROC curve of 0.9821, 0.9905, and 0.9855
on the DRIVE, STARE, and CHASE_DB1 databases, respectively. It also appears to
be robust to the training set and to the inter-rater variability, which shows
its potential for real-world applications.
Learning Private Neural Language Modeling with Attentive Aggregation
Authors: Shaoxiong Ji, Shirui Pan, Guodong Long, Xue Li, Jing Jiang, Zi Huang
Date: 2018-12-17 00:00:00
Tasks: Federated Learning, Language Modeling, Language Modelling

Mobile keyboard suggestion is typically regarded as a word-level language
modeling problem. Centralized machine learning technique requires massive user
data collected to train on, which may impose privacy concerns for sensitive
personal typing data of users. Federated learning (FL) provides a promising
approach to learning private language modeling for intelligent personalized
keyboard suggestion by training models in distributed clients rather than
training in a central server. To obtain a global model for prediction, existing
FL algorithms simply average the client models and ignore the importance of
each client during model aggregation. Furthermore, there is no optimization for
learning a well-generalized global model on the central server. To solve these
problems, we propose a novel model aggregation with the attention mechanism
considering the contribution of clients models to the global model, together
with an optimization technique during server aggregation. Our proposed
attentive aggregation method minimizes the weighted distance between the server
model and client models through iterative parameters updating while attends the
distance between the server model and client models. Through experiments on two
popular language modeling datasets and a social media dataset, our proposed
method outperforms its counterparts in terms of perplexity and communication
cost in most settings of comparison.
OCTID: Optical Coherence Tomography Image Database
Authors: Peyman Gholami, Priyanka Roy, Mohana Kuppuswamy Parthasarathy, Vasudevan Lakshminarayanan
Date: 2018-12-17 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Optical coherence tomography (OCT) is a non-invasive imaging modality which is widely used in clinical ophthalmology. OCT images are capable of visualizing deep retinal layers which is crucial for early diagnosis of retinal diseases. In this paper, we describe a comprehensive open-access database containing more than 500 highresolution images categorized into different pathological conditions. The image classes include Normal (NO), Macular Hole (MH), Age-related Macular Degeneration (AMD), Central Serous Retinopathy (CSR), and Diabetic Retinopathy (DR). The images were obtained from a raster scan protocol with a 2mm scan length and 512x1024 pixel resolution. We have also included 25 normal OCT images with their corresponding ground truth delineations which can be used for an accurate evaluation of OCT image segmentation. In addition, we have provided a user-friendly GUI which can be used by clinicians for manual (and semi-automated) segmentation.
Learning Constraints from Demonstrations
Authors: Glen Chou, Dmitry Berenson, Necmiye Ozay
Date: 2018-12-17 00:00:00

We extend the learning from demonstration paradigm by providing a method for
learning unknown constraints shared across tasks, using demonstrations of the
tasks, their cost functions, and knowledge of the system dynamics and control
constraints. Given safe demonstrations, our method uses hit-and-run sampling to
obtain lower cost, and thus unsafe, trajectories. Both safe and unsafe
trajectories are used to obtain a consistent representation of the unsafe set
via solving an integer program. Our method generalizes across system dynamics
and learns a guaranteed subset of the constraint. We also provide theoretical
analysis on what subset of the constraint can be learnable from safe
demonstrations. We demonstrate our method on linear and nonlinear system
dynamics, show that it can be modified to work with suboptimal demonstrations,
and that it can also be used to learn constraints in a feature space.
Rethinking Epistemic Logic with Belief Bases
Authors: Emiliano Lorini
Date: 2018-12-17 00:00:00

We introduce a new semantics for a logic of explicit and implicit beliefs
based on the concept of multi-agent belief base. Differently from existing
Kripke-style semantics for epistemic logic in which the notions of possible
world and doxastic/epistemic alternative are primitive, in our semantics they
are non-primitive but are defined from the concept of belief base. We provide a
complete axiomatization and prove decidability for our logic via a finite model
argument. We also provide a polynomial embedding of our logic into Fagin &
Halpern's logic of general awareness and establish a complexity result for our
logic via the embedding.
Fuzzy Hashing as Perturbation-Consistent Adversarial Kernel Embedding
Authors: Ari Azarafrooz, John Brock
Date: 2018-12-17 00:00:00
Tasks: Malware Analysis

Measuring the similarity of two files is an important task in malware
analysis, with fuzzy hash functions being a popular approach. Traditional fuzzy
hash functions are data agnostic: they do not learn from a particular dataset
how to determine similarity; their behavior is fixed across all datasets. In
this paper, we demonstrate that fuzzy hash functions can be learned in a novel
minimax training framework and that these learned fuzzy hash functions
outperform traditional fuzzy hash functions at the file similarity task for
Portable Executable files. In our approach, hash digests can be extracted from
the kernel embeddings of two kernel networks, trained in a minimax framework,
where the roles of players during training (i.e adversary versus generator)
alternate along with the input data. We refer to this new minimax architecture
as perturbation-consistent. The similarity score for a pair of files is the
utility of the minimax game in equilibrium. Our experiments show that learned
fuzzy hash functions generalize well, capable of determining that two files are
similar even when one of those files was generated using insertion and deletion
operations.
An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents
Authors: Felipe Petroski Such, Vashisht Madhavan, Rosanne Liu, Rui Wang, Pablo Samuel Castro, Yulun Li, Jiale Zhi, Ludwig Schubert, Marc G. Bellemare, Jeff Clune, Joel Lehman
Date: 2018-12-17 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, Friction, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Much human and computational effort has aimed to improve how deep reinforcement learning algorithms perform on benchmarks such as the Atari Learning Environment. Comparatively less effort has focused on understanding what has been learned by such methods, and investigating and comparing the representations learned by different families of reinforcement learning (RL) algorithms. Sources of friction include the onerous computational requirements, and general logistical and architectural complications for running Deep RL algorithms at scale. We lessen this friction, by (1) training several algorithms at scale and releasing trained models, (2) integrating with a previous Deep RL model release, and (3) releasing code that makes it easy for anyone to load, visualize, and analyze such models. This paper introduces the Atari Zoo framework, which contains models trained across benchmark Atari games, in an easy-to-use format, as well as code that implements common modes of analysis and connects such models to a popular neural network visualization library. Further, to demonstrate the potential of this dataset and software package, we show initial quantitative and qualitative comparisons between the performance and representations of several deep RL algorithms, highlighting interesting and previously unknown distinctions between them.
Probabilistic Attribute Tree in Convolutional Neural Networks for Facial Expression Recognition
Authors: Jie Cai, Zibo Meng, Ahmed Shehab Khan, Zhiyuan Li, James O'Reilly, Yan Tong
Date: 2018-12-17 00:00:00
Tasks: Attribute, Facial Expression Recognition, Facial Expression Recognition (FER)

In this paper, we proposed a novel Probabilistic Attribute Tree-CNN (PAT-CNN)
to explicitly deal with the large intra-class variations caused by
identity-related attributes, e.g., age, race, and gender. Specifically, a novel
PAT module with an associated PAT loss was proposed to learn features in a
hierarchical tree structure organized according to attributes, where the final
features are less affected by the attributes. Then, expression-related features
are extracted from leaf nodes. Samples are probabilistically assigned to tree
nodes at different levels such that expression-related features can be learned
from all samples weighted by probabilities. We further proposed a
semi-supervised strategy to learn the PAT-CNN from limited attribute-annotated
samples to make the best use of available data. Experimental results on five
facial expression datasets have demonstrated that the proposed PAT-CNN
outperforms the baseline models by explicitly modeling attributes. More
impressively, the PAT-CNN using a single model achieves the best performance
for faces in the wild on the SFEW dataset, compared with the state-of-the-art
methods using an ensemble of hundreds of CNNs.
A stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs
Authors: Rohit Kannan, James Luedtke
Date: 2018-12-17 00:00:00

We propose a stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs. Our approach is based on a bi-objective viewpoint of chance-constrained programs that seeks solutions on the efficient frontier of optimal objective value versus risk of constraint violation. To this end, we construct a reformulated problem whose objective is to minimize the probability of constraints violation subject to deterministic convex constraints (which includes a bound on the objective function value). We adapt existing smoothing-based approaches for chance-constrained problems to derive a convergent sequence of smooth approximations of our reformulated problem, and apply a projected stochastic subgradient algorithm to solve it. In contrast with exterior sampling-based approaches (such as sample average approximation) that approximate the original chance-constrained program with one having finite support, our proposal converges to stationary solutions of a smooth approximation of the original problem, thereby avoiding poor local solutions that may be an artefact of a fixed sample. Our proposal also includes a tailored implementation of the smoothing-based approach that chooses key algorithmic parameters based on problem data. Computational results on four test problems from the literature indicate that our proposed approach can efficiently determine good approximations of the efficient frontier.
Deep learning incorporating biologically-inspired neural dynamics
Authors: Stanisław Woźniak, Angeliki Pantazi, Thomas Bohnstingl, Evangelos Eleftheriou
Date: 2018-12-17 00:00:00
Tasks: Deep Learning, Language Modelling

Neural networks have become the key technology of artificial intelligence and have contributed to breakthroughs in several machine learning tasks, primarily owing to advances in deep learning applied to Artificial Neural Networks (ANNs). Simultaneously, Spiking Neural Networks (SNNs) incorporating biologically-feasible spiking neurons have held great promise because of their rich temporal dynamics and high-power efficiency. However, the developments in SNNs were proceeding separately from those in ANNs, effectively limiting the adoption of deep learning research insights. Here we show an alternative perspective on the spiking neuron that casts it as a particular ANN construct called Spiking Neural Unit (SNU), and a soft SNU (sSNU) variant that generalizes its dynamics to a novel recurrent ANN unit. SNUs bridge the biologically-inspired SNNs with ANNs and provide a methodology for seamless inclusion of spiking neurons in deep learning architectures. Furthermore, SNU enables highly-efficient in-memory acceleration of SNNs trained with backpropagation through time, implemented with the hardware in-the-loop. We apply SNUs to tasks ranging from hand-written digit recognition, language modelling, to music prediction. We obtain accuracy comparable to, or better than, that of state-of-the-art ANNs, and we experimentally verify the efficacy of the in-memory-based SNN realization for the music-prediction task using 52,800 phase-change memory devices. The new generation of neural units introduced in this paper incorporate biologically-inspired neural dynamics in deep learning. In addition, they provide a systematic methodology for training neuromorphic computing hardware. Thus, they open a new avenue for a widespread adoption of SNNs in practical applications.
On the Continuity of Rotation Representations in Neural Networks
Authors: Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, Hao Li
Date: 2018-12-17 00:00:00
Proceeding: CVPR 2019 6

In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In this paper, we advance a definition of a continuous representation, which can be helpful for training deep neural networks. We relate this to topological concepts such as homeomorphism and embedding. We then investigate what are continuous and discontinuous representations for 2D, 3D, and n-dimensional rotations. We demonstrate that for 3D rotations, all representations are discontinuous in the real Euclidean spaces of four or fewer dimensions. Thus, widely used representations such as quaternions and Euler angles are discontinuous and difficult for neural networks to learn. We show that the 3D rotations have continuous representations in 5D and 6D, which are more suitable for learning. We also present continuous representations for the general case of the n-dimensional rotation group SO(n). While our main focus is on rotations, we also show that our constructions apply to other groups such as the orthogonal group and similarity transforms. We finally present empirical results, which show that our continuous rotation representations outperform discontinuous ones for several practical problems in graphics and vision, including a simple autoencoder sanity test, a rotation estimator for 3D point clouds, and an inverse kinematics solver for 3D human poses.
From Satellite Imagery to Disaster Insights
Authors: Jigar Doshi, Saikat Basu, Guan Pang
Date: 2018-12-17 00:00:00
Tasks: Change Detection, Disaster Response

The use of satellite imagery has become increasingly popular for disaster
monitoring and response. After a disaster, it is important to prioritize rescue
operations, disaster response and coordinate relief efforts. These have to be
carried out in a fast and efficient manner since resources are often limited in
disaster-affected areas and it's extremely important to identify the areas of
maximum damage. However, most of the existing disaster mapping efforts are
manual which is time-consuming and often leads to erroneous results. In order
to address these issues, we propose a framework for change detection using
Convolutional Neural Networks (CNN) on satellite images which can then be
thresholded and clustered together into grids to find areas which have been
most severely affected by a disaster. We also present a novel metric called
Disaster Impact Index (DII) and use it to quantify the impact of two natural
disasters - the Hurricane Harvey flood and the Santa Rosa fire. Our framework
achieves a top F1 score of 81.2% on the gridded flood dataset and 83.5% on the
gridded fire dataset.
Boundary loss for highly unbalanced segmentation
Authors: Hoel Kervadec, Jihene Bouchtiba, Christian Desrosiers, Eric Granger, Jose Dolz, Ismail Ben Ayed
Date: 2018-12-17 00:00:00
Tasks: Brain Lesion Segmentation From Mri, Ischemic Stroke Lesion Segmentation, Lesion Segmentation, Medical Image Analysis, Medical Image Segmentation, Segmentation, Semantic Segmentation, Unbalanced Segmentation

Widely used loss functions for CNN segmentation, e.g., Dice or cross-entropy, are based on integrals over the segmentation regions. Unfortunately, for highly unbalanced segmentations, such regional summations have values that differ by several orders of magnitude across classes, which affects training performance and stability. We propose a boundary loss, which takes the form of a distance metric on the space of contours, not regions. This can mitigate the difficulties of highly unbalanced problems because it uses integrals over the interface between regions instead of unbalanced integrals over the regions. Furthermore, a boundary loss complements regional information. Inspired by graph-based optimization techniques for computing active-contour flows, we express a non-symmetric $L_2$ distance on the space of contours as a regional integral, which avoids completely local differential computations involving contour points. This yields a boundary loss expressed with the regional softmax probability outputs of the network, which can be easily combined with standard regional losses and implemented with any existing deep network architecture for N-D segmentation. We report comprehensive evaluations and comparisons on different unbalanced problems, showing that our boundary loss can yield significant increases in performances while improving training stability. Our code is publicly available: https://github.com/LIVIAETS/surface-loss .
Fuzzy Controller of Reward of Reinforcement Learning For Handwritten Digit Recognition
Authors: Saber Malekzadeh
Date: 2018-12-17 00:00:00
Tasks: Handwriting Recognition, Handwritten Digit Recognition, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recognition of human environment with computer systems always was a big deal
in artificial intelligence. In this area handwriting recognition and
conceptualization of it to computer is an important area in it. In the past
years with growth of machine learning in artificial intelligence, efforts to
using this technique increased. In this paper is tried to using fuzzy
controller, to optimizing amount of reward of reinforcement learning for
recognition of handwritten digits. For this aim first a sample of every digit
with 10 standard computer fonts, given to actor and then actor is trained. In
the next level is tried to test the actor with dataset and then results show
improvement of recognition when using fuzzy controller of reinforcement
learning.
From FiLM to Video: Multi-turn Question Answering with Multi-modal Context
Authors: Dat Tien Nguyen, Shikhar Sharma, Hannes Schulz, Layla El Asri
Date: 2018-12-17 00:00:00
Tasks: Decoder, Question Answering

Understanding audio-visual content and the ability to have an informative
conversation about it have both been challenging areas for intelligent systems.
The Audio Visual Scene-aware Dialog (AVSD) challenge, organized as a track of
the Dialog System Technology Challenge 7 (DSTC7), proposes a combined task,
where a system has to answer questions pertaining to a video given a dialogue
with previous question-answer pairs and the video itself. We propose for this
task a hierarchical encoder-decoder model which computes a multi-modal
embedding of the dialogue context. It first embeds the dialogue history using
two LSTMs. We extract video and audio frames at regular intervals and compute
semantic features using pre-trained I3D and VGGish models, respectively. Before
summarizing both modalities into fixed-length vectors using LSTMs, we use FiLM
blocks to condition them on the embeddings of the current question, which
allows us to reduce the dimensionality considerably. Finally, we use an LSTM
decoder that we train with scheduled sampling and evaluate using beam search.
Compared to the modality-fusing baseline model released by the AVSD challenge
organizers, our model achieves a relative improvements of more than 16%,
scoring 0.36 BLEU-4 and more than 33%, scoring 0.997 CIDEr.