pwc-archive/papers-with-abstracts huggingface dataset:
Task Driven Generative Modeling for Unsupervised Domain Adaptation: Application to X-ray Image Segmentation
Authors: Yue Zhang, Shun Miao, Tommaso Mansi, Rui Liao
Date: 2018-06-11 00:00:00
Tasks: Anatomy, Domain Adaptation, Generative Adversarial Network, Image Segmentation, Organ Segmentation, Semantic Segmentation, Style Transfer, Unsupervised Domain Adaptation

Automatic parsing of anatomical objects in X-ray images is critical to many
clinical applications in particular towards image-guided invention and workflow
automation. Existing deep network models require a large amount of labeled
data. However, obtaining accurate pixel-wise labeling in X-ray images relies
heavily on skilled clinicians due to the large overlaps of anatomy and the
complex texture patterns. On the other hand, organs in 3D CT scans preserve
clearer structures as well as sharper boundaries and thus can be easily
delineated. In this paper, we propose a novel model framework for learning
automatic X-ray image parsing from labeled CT scans. Specifically, a Dense
Image-to-Image network (DI2I) for multi-organ segmentation is first trained on
X-ray like Digitally Reconstructed Radiographs (DRRs) rendered from 3D CT
volumes. Then we introduce a Task Driven Generative Adversarial Network
(TD-GAN) architecture to achieve simultaneous style transfer and parsing for
unseen real X-ray images. TD-GAN consists of a modified cycle-GAN substructure
for pixel-to-pixel translation between DRRs and X-ray images and an added
module leveraging the pre-trained DI2I to enforce segmentation consistency. The
TD-GAN framework is general and can be easily adapted to other learning tasks.
In the numerical experiments, we validate the proposed model on 815 DRRs and
153 topograms. While the vanilla DI2I without any adaptation fails completely
on segmenting the topograms, the proposed model does not require any topogram
labels and is able to provide a promising average dice of 85% which achieves
the same level accuracy of supervised training (88%).
Full deep neural network training on a pruned weight budget
Authors: Maximilian Golub, Guy Lemieux, Mieszko Lis
Date: 2018-06-11 00:00:00

We introduce a DNN training technique that learns only a fraction of the full parameter set without incurring an accuracy penalty. To do this, our algorithm constrains the total number of weights updated during backpropagation to those with the highest total gradients. The remaining weights are not tracked, and their initial value is regenerated at every access to avoid storing them in memory. This can dramatically reduce the number of off-chip memory accesses during both training and inference, a key component of the energy needs of DNN accelerators. By ensuring that the total weight diffusion remains close to that of baseline unpruned SGD, networks pruned using our technique are able to retain state-of-the-art accuracy across network architectures -- including networks previously identified as difficult to compress, such as Densenet and WRN. With ResNet18 on ImageNet, we observe an 11.7$\times$ weight reduction with no accuracy loss, and up to 24.4$\times$ with a small accuracy impact.
Multilingual Scene Character Recognition System using Sparse Auto-Encoder for Efficient Local Features Representation in Bag of Features
Authors: Maroua Tounsi, Ikram Moalla, Frank Lebourgeois, Adel M. ALIMI
Date: 2018-06-11 00:00:00
Tasks: Deep Learning, Scene Text Recognition

The recognition of texts existing in camera-captured images has become an
important issue for a great deal of research during the past few decades. This
give birth to Scene Character Recognition (SCR) which is an important step in
scene text recognition pipeline. In this paper, we extended the Bag of Features
(BoF)-based model using deep learning for representing features for accurate
SCR of different languages. In the features coding step, a deep Sparse
Auto-encoder (SAE)-based strategy was applied to enhance the representative and
discriminative abilities of image features. This deep learning architecture
provides more efficient features representation and therefore a better
recognition accuracy. Our system was evaluated extensively on all the scene
character datasets of five different languages. The experimental results proved
the efficiency of our system for a multilingual SCR.
An optimized system to solve text-based CAPTCHA
Authors: Ye Wang, Mi Lu
Date: 2018-06-11 00:00:00
Tasks: Segmentation

CAPTCHA(Completely Automated Public Turing test to Tell Computers and Humans
Apart) can be used to protect data from auto bots. Countless kinds of CAPTCHAs
are thus designed, while we most frequently utilize text-based scheme because
of most convenience and user-friendly way \cite{bursztein2011text}. Currently,
various types of CAPTCHAs need corresponding segmentation to identify single
character due to the numerous different segmentation ways. Our goal is to
defeat the CAPTCHA, thus firstly the CAPTCHAs need to be split into character
by character. There isn't a regular segmentation algorithm to obtain the
divided characters in all kinds of examples, which means that we have to treat
the segmentation individually. In this paper, we build a whole system to defeat
the CAPTCHAs as well as achieve state-of-the-art performance. In detail, we
present our self-adaptive algorithm to segment different kinds of characters
optimally, and then utilize both the existing methods and our own constructed
convolutional neural network as an extra classifier. Results are provided
showing how our system work well towards defeating these CAPTCHAs.
Meta Continual Learning
Authors: Risto Vuorio, Dong-Yeon Cho, Daejoong Kim, Jiwon Kim
Date: 2018-06-11 00:00:00
Tasks: Continual Learning

Using neural networks in practical settings would benefit from the ability of
the networks to learn new tasks throughout their lifetimes without forgetting
the previous tasks. This ability is limited in the current deep neural networks
by a problem called catastrophic forgetting, where training on new tasks tends
to severely degrade performance on previous tasks. One way to lessen the impact
of the forgetting problem is to constrain parameters that are important to
previous tasks to stay close to the optimal parameters. Recently, multiple
competitive approaches for computing the importance of the parameters with
respect to the previous tasks have been presented. In this paper, we propose a
learning to optimize algorithm for mitigating catastrophic forgetting. Instead
of trying to formulate a new constraint function ourselves, we propose to train
another neural network to predict parameter update steps that respect the
importance of parameters to the previous tasks. In the proposed meta-training
scheme, the update predictor is trained to minimize loss on a combination of
current and past tasks. We show experimentally that the proposed approach works
in the continual learning setting.
DFNet: Semantic Segmentation on Panoramic Images with Dynamic Loss Weights and Residual Fusion Block
Authors: Wei Jiang, Yan Wu
Date: 2018-06-11 00:00:00
Tasks: Semantic Segmentation

For the self-driving and automatic parking, perception is the basic and
critical technique, moreover, the detection of lane markings and parking slots
is an important part of visual perception. In this paper, we use the semantic
segmentation method to segment the area and classify the class of lane makings
and parking slots on panoramic surround view (PSV) dataset. We propose the
DFNet and make two main contributions, one is dynamic loss weights, and the
other is residual fusion block (RFB). Dynamic loss weights are varying from
classes, calculated according to the pixel number of each class in a batch. RFB
is composed of two convolutional layers, one pooling layer, and a fusion layer
to combine the feature maps by pixel multiplication. We evaluate our method on
PSV dataset, and achieve an advanced result.
Auto-Meta: Automated Gradient Based Meta Learner Search
Authors: Jaehong Kim, Sangyeul Lee, Sungwan Kim, Moonsu Cha, Jung Kwon Lee, Youngduck Choi, Yongseok Choi, Dong-Yeon Cho, Jiwon Kim
Date: 2018-06-11 00:00:00
Tasks: BIG-bench Machine Learning, Meta-Learning, Neural Architecture Search

Fully automating machine learning pipelines is one of the key challenges of
current artificial intelligence research, since practical machine learning
often requires costly and time-consuming human-powered processes such as model
design, algorithm development, and hyperparameter tuning. In this paper, we
verify that automated architecture search synergizes with the effect of
gradient-based meta learning. We adopt the progressive neural architecture
search \cite{liu:pnas_google:DBLP:journals/corr/abs-1712-00559} to find optimal
architectures for meta-learners. The gradient based meta-learner whose
architecture was automatically found achieved state-of-the-art results on the
5-shot 5-way Mini-ImageNet classification problem with $74.65\%$ accuracy,
which is $11.54\%$ improvement over the result obtained by the first
gradient-based meta-learner called MAML
\cite{finn:maml:DBLP:conf/icml/FinnAL17}. To our best knowledge, this work is
the first successful neural architecture search implementation in the context
of meta learning.
Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation
Authors: Shani Gamrian, Yoav Goldberg
Date: 2018-05-31 00:00:00
Proceeding: ICLR 2019 5
Tasks: Image-to-Image Translation, Imitation Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning, Translation

Despite the remarkable success of Deep RL in learning control policies from raw pixels, the resulting models do not generalize. We demonstrate that a trained agent fails completely when facing small visual changes, and that fine-tuning---the common transfer learning paradigm---fails to adapt to these changes, to the extent that it is faster to re-train the model from scratch. We show that by separating the visual transfer task from the control policy we achieve substantially better sample efficiency and transfer behavior, allowing an agent trained on the source task to transfer well to the target tasks. The visual mapping from the target to the source domain is performed using unaligned GANs, resulting in a control policy that can be further improved using imitation learning from imperfect demonstrations. We demonstrate the approach on synthetic visual variants of the Breakout game, as well as on transfer between subsequent levels of Road Fighter, a Nintendo car-driving game. A visualization of our approach can be seen in https://youtu.be/4mnkzYyXMn4 and https://youtu.be/KCGTrQi6Ogo .
Collaborative Memory Network for Recommendation Systems
Authors: Travis Ebesu, Bin Shen, Yi Fang
Date: 2018-04-29 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Recommendation systems play a vital role to keep users engaged with
personalized content in modern online platforms. Deep learning has
revolutionized many research fields and there is a recent surge of interest in
applying it to collaborative filtering (CF). However, existing methods compose
deep learning architectures with the latent factor model ignoring a major class
of CF models, neighborhood or memory-based approaches. We propose Collaborative
Memory Networks (CMN), a deep architecture to unify the two classes of CF
models capitalizing on the strengths of the global structure of latent factor
model and local neighborhood-based structure in a nonlinear fashion. Motivated
by the success of Memory Networks, we fuse a memory component and neural
attention mechanism as the neighborhood component. The associative addressing
scheme with the user and item memories in the memory module encodes complex
user-item relations coupled with the neural attention mechanism to learn a
user-item specific neighborhood. Finally, the output module jointly exploits
the neighborhood with the user and item memories to produce the ranking score.
Stacking multiple memory modules together yield deeper architectures capturing
increasingly complex user-item relations. Furthermore, we show strong
connections between CMN components, memory networks and the three classes of CF
models. Comprehensive experimental results demonstrate the effectiveness of CMN
on three public datasets outperforming competitive baselines. Qualitative
visualization of the attention weights provide insight into the model's
recommendation process and suggest the presence of higher order interactions.
Creation and analysis of biochemical constraint-based models: the COBRA Toolbox v3.0
Authors: Laurent Heirendt, Sylvain Arreckx, Thomas Pfau, Sebastián N. Mendoza, Anne Richelle, Almut Heinken, Hulda S. Haraldsdóttir, Jacek Wachowiak, Sarah M. Keating, Vanja Vlasov, Stefania Magnusdóttir, Chiam Yu Ng, German Preciat, Alise Žagare, Siu H. J. Chan, Maike K. Aurich, Catherine M. Clancy, Jennifer Modamio, John T. Sauls, Alberto Noronha, Aarash Bordbar, Benjamin Cousins, Diana C. El Assal, Luis V. Valcarcel, Iñigo Apaolaza, Susan Ghaderi, Masoud Ahookhosh, Marouen Ben Guebila, Andrejs Kostromins, Nicolas Sompairac, Hoai M. Le, Ding Ma, Yuekai Sun, Lin Wang, James T. Yurkovich, Miguel A. P. Oliveira, Phan T. Vuong, Lemmer P. El Assal, Inna Kuperstein, Andrei Zinovyev, H. Scott Hinton, William A. Bryant, Francisco J. Aragón Artacho, Francisco J. Planes, Egils Stalidzans, Alejandro Maass, Santosh Vempala, Michael Hucka, Michael A. Saunders, Costas D. Maranas, Nathan E. Lewis, Thomas Sauter, Bernhard Ø. Palsson, Ines Thiele, Ronan M. T. Fleming
Date: 2017-10-11 00:00:00
Tasks: Experimental Design

COnstraint-Based Reconstruction and Analysis (COBRA) provides a molecular
mechanistic framework for integrative analysis of experimental data and
quantitative prediction of physicochemically and biochemically feasible
phenotypic states. The COBRA Toolbox is a comprehensive software suite of
interoperable COBRA methods. It has found widespread applications in biology,
biomedicine, and biotechnology because its functions can be flexibly combined
to implement tailored COBRA protocols for any biochemical network. Version 3.0
includes new methods for quality controlled reconstruction, modelling,
topological analysis, strain and experimental design, network visualisation as
well as network integration of chemoinformatic, metabolomic, transcriptomic,
proteomic, and thermochemical data. New multi-lingual code integration also
enables an expansion in COBRA application scope via high-precision,
high-performance, and nonlinear numerical optimisation solvers for multi-scale,
multi-cellular and reaction kinetic modelling, respectively. This protocol can
be adapted for the generation and analysis of a constraint-based model in a
wide variety of molecular systems biology scenarios. This protocol is an update
to the COBRA Toolbox 1.0 and 2.0. The COBRA Toolbox 3.0 provides an
unparalleled depth of constraint-based reconstruction and analysis methods.
Natural Language Processing - A Survey
Authors: Kevin Mote
Date: 2012-09-25 00:00:00
Tasks: Survey

The utility and power of Natural Language Processing (NLP) seems destined to
change our technological society in profound and fundamental ways. However
there are, to date, few accessible descriptions of the science of NLP that have
been written for a popular audience, or even for an audience of intelligent,
but uninitiated scientists. This paper aims to provide just such an overview.
In short, the objective of this article is to describe the purpose, procedures
and practical applications of NLP in a clear, balanced, and readable way. We
will examine the most recent literature describing the methods and processes of
NLP, analyze some of the challenges that researchers are faced with, and
briefly survey some of the current and future applications of this science to
IT research in general.
Identifying Metaphoric Antonyms in a Corpus Analysis of Finance Articles
Authors: Aaron Gerow, Mark Keane
Date: 2012-12-13 00:00:00
Tasks: Articles

Using a corpus of 17,000+ financial news reports (involving over 10M words),
we perform an analysis of the argument-distributions of the UP and DOWN verbs
used to describe movements of indices, stocks and shares. In Study 1
participants identified antonyms of these verbs in a free-response task and a
matching task from which the most commonly identified antonyms were compiled.
In Study 2, we determined whether the argument-distributions for the verbs in
these antonym-pairs were sufficiently similar to predict the most
frequently-identified antonym. Cosine similarity correlates moderately with the
proportions of antonym-pairs identified by people (r = 0.31). More
impressively, 87% of the time the most frequently-identified antonym is either
the first- or second-most similar pair in the set of alternatives. The
implications of these results for distributional approaches to determining
metaphoric knowledge are discussed.
Identifying Metaphor Hierarchies in a Corpus Analysis of Finance Articles
Authors: Aaron Georw, Mark Keane
Date: 2012-12-13 00:00:00
Tasks: Articles, Clustering

Using a corpus of over 17,000 financial news reports (involving over 10M
words), we perform an analysis of the argument-distributions of the UP- and
DOWN-verbs used to describe movements of indices, stocks, and shares. Using
measures of the overlap in the argument distributions of these verbs and
k-means clustering of their distributions, we advance evidence for the proposal
that the metaphors referred to by these verbs are organised into hierarchical
structures of superordinate and subordinate groups.
Variational Bayesian Inference with Stochastic Search
Authors: John Paisley, David Blei, Michael Jordan
Date: 2012-06-27 00:00:00
Tasks: Bayesian Inference, Stochastic Optimization, Variational Inference

Mean-field variational inference is a method for approximate Bayesian
posterior inference. It approximates a full posterior distribution with a
factorized set of distributions by maximizing a lower bound on the marginal
likelihood. This requires the ability to integrate a sum of terms in the log
joint likelihood using this factorized distribution. Often not all integrals
are in closed form, which is typically handled by using a lower bound. We
present an alternative algorithm based on stochastic optimization that allows
for direct optimization of the variational lower bound. This method uses
control variates to reduce the variance of the stochastic search gradient, in
which existing lower bounds can play an important role. We demonstrate the
approach on two non-conjugate models: logistic regression and an approximation
to the HDP.
Few-Shot Segmentation Propagation with Guided Networks
Authors: Kate Rakelly, Evan Shelhamer, Trevor Darrell, Alexei A. Efros, Sergey Levine
Date: 2018-05-25 00:00:00
Tasks: Interactive Segmentation, Segmentation, Semantic Segmentation, Video Segmentation, Video Semantic Segmentation

Learning-based methods for visual segmentation have made progress on
particular types of segmentation tasks, but are limited by the necessary
supervision, the narrow definitions of fixed tasks, and the lack of control
during inference for correcting errors. To remedy the rigidity and annotation
burden of standard approaches, we address the problem of few-shot segmentation:
given few image and few pixel supervision, segment any images accordingly. We
propose guided networks, which extract a latent task representation from any
amount of supervision, and optimize our architecture end-to-end for fast,
accurate few-shot segmentation. Our method can switch tasks without further
optimization and quickly update when given more guidance. We report the first
results for segmentation from one pixel per concept and show real-time
interactive video segmentation. Our unified approach propagates pixel
annotations across space for interactive segmentation, across time for video
segmentation, and across scenes for semantic segmentation. Our guided segmentor
is state-of-the-art in accuracy for the amount of annotation and time. See
http://github.com/shelhamer/revolver for code, models, and more details.
Utilizing Static Analysis and Code Generation to Accelerate Neural Networks
Authors: Lawrence McAfee, Kunle Olukotun
Date: 2012-06-27 00:00:00
Tasks: C++ code, Code Generation

As datasets continue to grow, neural network (NN) applications are becoming
increasingly limited by both the amount of available computational power and
the ease of developing high-performance applications. Researchers often must
have expert systems knowledge to make their algorithms run efficiently.
Although available computing power increases rapidly each year, algorithm
efficiency is not able to keep pace due to the use of general purpose
compilers, which are not able to fully optimize specialized application
domains. Within the domain of NNs, we have the added knowledge that network
architecture remains constant during training, meaning the architecture's data
structure can be statically optimized by a compiler. In this paper, we present
SONNC, a compiler for NNs that utilizes static analysis to generate optimized
parallel code. We show that SONNC's use of static optimizations make it able to
outperform hand-optimized C++ code by up to 7.8X, and MATLAB code by up to 24X.
Additionally, we show that use of SONNC significantly reduces code complexity
when using structurally sparse networks.
An efficient high-quality hierarchical clustering algorithm for automatic inference of software architecture from the source code of a software system
Authors: Sarge Rogatch
Date: 2012-02-15 00:00:00
Tasks: Clustering

It is a high-quality algorithm for hierarchical clustering of large software
source code. This effectively allows to break the complexity of tens of
millions lines of source code, so that a human software engineer can comprehend
a software system at high level by means of looking at its architectural
diagram that is reconstructed automatically from the source code of the
software system. The architectural diagram shows a tree of subsystems having
OOP classes in its leaves (in the other words, a nested software
decomposition). The tool reconstructs the missing
(inconsistent/incomplete/inexistent) architectural documentation for a software
system from its source code. This facilitates software maintenance: change
requests can be performed substantially faster. Simply speaking, this unique
tool allows to lift the comprehensible grain of object-oriented software
systems from OOP class-level to subsystem-level. It is estimated that a
commercial tool, developed on the basis of this work, will reduce software
maintenance expenses 10 times on the current needs, and will allow to implement
next-generation software systems which are currently too complex to be within
the range of human comprehension, therefore can't yet be designed or
implemented. Implemented prototype in Open Source:
http://sourceforge.net/p/insoar/code-0/1/tree/
Beaming Binaries - a New Observational Category of Photometric Binary Stars
Authors: Shay Zucker, Tsevi Mazeh, Tal Alexander
Date: 2007-08-15 00:00:00

The new photometric space-borne survey missions CoRoT and Kepler will be able
to detect minute flux variations in binary stars due to relativistic beaming
caused by the line-of-sight motion of their components. In all but very short
period binaries (P>10d), these variations will dominate over the ellipsoidal
and reflection periodic variability. Thus, CoRoT and Kepler will discover a new
observational class: photometric beaming binary stars. We examine this new
category and the information that the photometric variations can provide. The
variations that result from the observatory heliocentric velocity can be used
to extract some spectral information even for single stars.