pwc-archive/papers-with-abstracts huggingface dataset:
Integrating NVIDIA Deep Learning Accelerator (NVDLA) with RISC-V SoC on FireSim
Authors: Farzad Farshchi, Qijing Huang, Heechul Yun
Date: 2019-03-05 00:00:00
Tasks: object-detection, Object Detection

NVDLA is an open-source deep neural network (DNN) accelerator which has received a lot of attention by the community since its introduction by Nvidia. It is a full-featured hardware IP and can serve as a good reference for conducting research and development of SoCs with integrated accelerators. However, an expensive FPGA board is required to do experiments with this IP in a real SoC. Moreover, since NVDLA is clocked at a lower frequency on an FPGA, it would be hard to do accurate performance analysis with such a setup. To overcome these limitations, we integrate NVDLA into a real RISC-V SoC on the Amazon cloud FPGA using FireSim, a cycle-exact FPGA-accelerated simulator. We then evaluate the performance of NVDLA by running YOLOv3 object-detection algorithm. Our results show that NVDLA can sustain 7.5 fps when running YOLOv3. We further analyze the performance by showing that sharing the last-level cache with NVDLA can result in up to 1.56x speedup. We then identify that sharing the memory system with the accelerator can result in unpredictable execution time for the real-time tasks running on this platform. We believe this is an important issue that must be addressed in order for on-chip DNN accelerators to be incorporated in real-time embedded systems.
TinBiNN: Tiny Binarized Neural Network Overlay in about 5,000 4-LUTs and 5mW
Authors: Guy G. F. Lemieux, Joe Edwards, Joel Vandergriendt, Aaron Severance, Ryan De Iaco, Abdullah Raouf, Hussein Osman, Tom Watzka, Satwant Singh
Date: 2019-03-05 00:00:00

Reduced-precision arithmetic improves the size, cost, power and performance of neural networks in digital logic. In convolutional neural networks, the use of 1b weights can achieve state-of-the-art error rates while eliminating multiplication, reducing storage and improving power efficiency. The BinaryConnect binary-weighted system, for example, achieves 9.9% error using floating-point activations on the CIFAR-10 dataset. In this paper, we introduce TinBiNN, a lightweight vector processor overlay for accelerating inference computations with 1b weights and 8b activations. The overlay is very small -- it uses about 5,000 4-input LUTs and fits into a low cost iCE40 UltraPlus FPGA from Lattice Semiconductor. To show this can be useful, we build two embedded 'person detector' systems by shrinking the original BinaryConnect network. The first is a 10-category classifier with a 89% smaller network that runs in 1,315ms and achieves 13.6% error. The other is a 1-category classifier that is even smaller, runs in 195ms, and has only 0.4% error. In both classifiers, the error can be attributed entirely to training and not reduced precision.
Application-level Studies of Cellular Neural Network-based Hardware Accelerators
Authors: Qiuwen Lou, Indranil Palit, Tang Li, Andras Horvath, Michael Niemier, X. Sharon Hu
Date: 2019-02-28 00:00:00
Tasks: CPU

As cost and performance benefits associated with Moore's Law scaling slow, researchers are studying alternative architectures (e.g., based on analog and/or spiking circuits) and/or computational models (e.g., convolutional and recurrent neural networks) to perform application-level tasks faster, more energy efficiently, and/or more accurately. We investigate cellular neural network (CeNN)-based co-processors at the application-level for these metrics. While it is well-known that CeNNs can be well-suited for spatio-temporal information processing, few (if any) studies have quantified the energy/delay/accuracy of a CeNN-friendly algorithm and compared the CeNN-based approach to the best von Neumann algorithm at the application level. We present an evaluation framework for such studies. As a case study, a CeNN-friendly target-tracking algorithm was developed and mapped to an array architecture developed in conjunction with the algorithm. We compare the energy, delay, and accuracy of our architecture/algorithm (assuming all overheads) to the most accurate von Neumann algorithm (Struck). Von Neumann CPU data is measured on an Intel i5 chip. The CeNN approach is capable of matching the accuracy of Struck, and can offer approximately 1000x improvements in energy-delay product.
Splitter: Learning Node Representations that Capture Multiple Social Contexts
Authors: Alessandro Epasto, Bryan Perozzi
Date: 2019-03-18 00:00:00
Proceeding: WWW 2019 3
Tasks: Graph Embedding, Link Prediction, Network Embedding

Recent interest in graph embedding methods has focused on learning a single representation for each node in the graph. But can nodes really be best described by a single vector representation? In this work, we propose a method for learning multiple representations of the nodes in a graph (e.g., the users of a social network). Based on a principled decomposition of the ego-network, each representation encodes the role of the node in a different local community in which the nodes participate. These representations allow for improved reconstruction of the nuanced relationships that occur in the graph a phenomenon that we illustrate through state-of-the-art results on link prediction tasks on a variety of graphs, reducing the error by up to 90%. In addition, we show that these embeddings allow for effective visual analysis of the learned community structure.
Beyond Word2Vec: Embedding Words and Phrases in Same Vector Space
Authors: Vijay Prakash Dwivedi, Manish Shrivastava
Date: 2017-12-18 00:00:00
Proceeding: International Conference on Natural Language Processing (ICON) 2017, Kolkata, India 2017 12
Tasks: Phrase Vector Embedding, Semantic Textual Similarity, Word Embeddings

Word embeddings are being used for several linguistic problems and NLP tasks. Improvements in solutions to such problems are great because of the recent breakthroughs in vector representation of words and research in vector space models. However, vector embeddings of phrases keeping semantics intact with words has been challenging. We propose a novel methodology using Siamese deep neural networks to embed multi-word units and fine-tune the current state-of-the-art word embed-dings keeping both in the same vector space. We show several semantic relations between words and phrases using the embeddings generated by our system and evaluate that the similarity of words and their corresponding paraphrases are maximized using the modified embeddings.
A Comprehensive Review of Image Enhancement Techniques
Authors: Raman Maini, Himanshu Aggarwal
Date: 2010-03-22 00:00:00
Tasks: Image Enhancement

Principle objective of Image enhancement is to process an image so that
result is more suitable than original image for specific application. Digital
image enhancement techniques provide a multitude of choices for improving the
visual quality of images. Appropriate choice of such techniques is greatly
influenced by the imaging modality, task at hand and viewing conditions. This
paper will provide an overview of underlying concepts, along with algorithms
commonly used for image enhancement. The paper focuses on spatial domain
techniques for image enhancement, with particular reference to point processing
methods and histogram processing.
Domain Generalization by Solving Jigsaw Puzzles
Authors: Fabio Maria Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, Tatiana Tommasi
Date: 2019-03-16 00:00:00
Tasks: Domain Generalization, Image Classification, Object Recognition

Human adaptability relies crucially on the ability to learn and merge
knowledge both from supervised and unsupervised learning: the parents point out
few important concepts, but then the children fill in the gaps on their own.
This is particularly effective, because supervised learning can never be
exhaustive and thus learning autonomously allows to discover invariances and
regularities that help to generalize. In this paper we propose to apply a
similar approach to the task of object recognition across domains: our model
learns the semantic labels in a supervised fashion, and broadens its
understanding of the data by learning from self-supervised signals how to solve
a jigsaw puzzle on the same images. This secondary task helps the network to
learn the concepts of spatial correlation while acting as a regularizer for the
classification task. Multiple experiments on the PACS, VLCS, Office-Home and
digits datasets confirm our intuition and show that this simple method
outperforms previous domain generalization and adaptation solutions. An
ablation study further illustrates the inner workings of our approach.
Fast Interactive Object Annotation with Curve-GCN
Authors: Huan Ling, Jun Gao, Amlan Kar, Wenzheng Chen, Sanja Fidler
Date: 2019-03-16 00:00:00
Proceeding: CVPR 2019 6
Tasks: Object

Manually labeling objects by tracing their boundaries is a laborious process.
In Polygon-RNN++ the authors proposed Polygon-RNN that produces polygonal
annotations in a recurrent manner using a CNN-RNN architecture, allowing
interactive correction via humans-in-the-loop. We propose a new framework that
alleviates the sequential nature of Polygon-RNN, by predicting all vertices
simultaneously using a Graph Convolutional Network (GCN). Our model is trained
end-to-end. It supports object annotation by either polygons or splines,
facilitating labeling efficiency for both line-based and curved objects. We
show that Curve-GCN outperforms all existing approaches in automatic mode,
including the powerful PSP-DeepLab and is significantly more efficient in
interactive mode than Polygon-RNN++. Our model runs at 29.3ms in automatic, and
2.6ms in interactive mode, making it 10x and 100x faster than Polygon-RNN++.
A Cross-Season Correspondence Dataset for Robust Semantic Segmentation
Authors: Måns Larsson, Erik Stenborg, Lars Hammarstrand, Torsten Sattler, Mark Pollefeys, Fredrik Kahl
Date: 2019-03-16 00:00:00
Tasks: Segmentation, Semantic Segmentation

In this paper, we present a method to utilize 2D-2D point matches between images taken during different image conditions to train a convolutional neural network for semantic segmentation. Enforcing label consistency across the matches makes the final segmentation algorithm robust to seasonal changes. We describe how these 2D-2D matches can be generated with little human interaction by geometrically matching points from 3D models built from images. Two cross-season correspondence datasets are created providing 2D-2D matches across seasonal changes as well as from day to night. The datasets are made publicly available to facilitate further research. We show that adding the correspondences as extra supervision during training improves the segmentation performance of the convolutional neural network, making it more robust to seasonal changes and weather conditions.
Unsupervised Part-Based Disentangling of Object Shape and Appearance
Authors: Dominik Lorenz, Leonard Bereska, Timo Milbich, Björn Ommer
Date: 2019-03-16 00:00:00
Proceeding: CVPR 2019 6
Tasks: Appearance Transfer, Image Generation, Object, Pose Prediction, Translation, Unsupervised Facial Landmark Detection, Unsupervised Human Pose Estimation

Large intra-class variation is the result of changes in multiple object characteristics. Images, however, only show the superposition of different variable factors such as appearance or shape. Therefore, learning to disentangle and represent these different characteristics poses a great challenge, especially in the unsupervised case. Moreover, large object articulation calls for a flexible part-based model. We present an unsupervised approach for disentangling appearance and shape by learning parts consistently over all instances of a category. Our model for learning an object representation is trained by simultaneously exploiting invariance and equivariance constraints between synthetically transformed images. Since no part annotation or prior information on an object class is required, the approach is applicable to arbitrary classes. We evaluate our approach on a wide range of object categories and diverse tasks including pose prediction, disentangled image synthesis, and video-to-video translation. The approach outperforms the state-of-the-art on unsupervised keypoint prediction and compares favorably even against supervised approaches on the task of shape and appearance transfer.
AdaGraph: Unifying Predictive and Continuous Domain Adaptation through Graphs
Authors: Massimiliano Mancini, Samuel Rota Bulò, Barbara Caputo, Elisa Ricci
Date: 2019-03-17 00:00:00
Proceeding: CVPR 2019 6
Tasks: Domain Adaptation

The ability to categorize is a cornerstone of visual intelligence, and a key functionality for artificial, autonomous visual machines. This problem will never be solved without algorithms able to adapt and generalize across visual domains. Within the context of domain adaptation and generalization, this paper focuses on the predictive domain adaptation scenario, namely the case where no target data are available and the system has to learn to generalize from annotated source images plus unlabeled samples with associated metadata from auxiliary domains. Our contributionis the first deep architecture that tackles predictive domainadaptation, able to leverage over the information broughtby the auxiliary domains through a graph. Moreover, we present a simple yet effective strategy that allows us to take advantage of the incoming target data at test time, in a continuous domain adaptation scenario. Experiments on three benchmark databases support the value of our approach.
Inverse Path Tracing for Joint Material and Lighting Estimation
Authors: Dejan Azinović, Tzu-Mao Li, Anton Kaplanyan, Matthias Nießner
Date: 2019-03-17 00:00:00
Tasks: 3D geometry, Lighting Estimation, Retrieval

Modern computer vision algorithms have brought significant advancement to 3D
geometry reconstruction. However, illumination and material reconstruction
remain less studied, with current approaches assuming very simplified models
for materials and illumination. We introduce Inverse Path Tracing, a novel
approach to jointly estimate the material properties of objects and light
sources in indoor scenes by using an invertible light transport simulation. We
assume a coarse geometry scan, along with corresponding images and camera
poses. The key contribution of this work is an accurate and simultaneous
retrieval of light sources and physically based material properties (e.g.,
diffuse reflectance, specular reflectance, roughness, etc.) for the purpose of
editing and re-rendering the scene under new conditions. To this end, we
introduce a novel optimization method using a differentiable Monte Carlo
renderer that computes derivatives with respect to the estimated unknown
illumination and material properties. This enables joint optimization for
physically correct light transport and material models using a tailored
stochastic gradient descent.
QATM: Quality-Aware Template Matching For Deep Learning
Authors: Jiaxin Cheng, Yue Wu, Wael Abd-Almageed, Premkumar Natarajan
Date: 2019-03-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: Deep Learning, Image-To-Gps Verification, Template Matching

Finding a template in a search image is one of the core problems many
computer vision, such as semantic image semantic, image-to-GPS verification
\etc. We propose a novel quality-aware template matching method, QATM, which is
not only used as a standalone template matching algorithm, but also a trainable
layer that can be easily embedded into any deep neural network. Specifically,
we assess the quality of a matching pair using soft-ranking among all matching
pairs, and thus different matching scenarios such as 1-to-1, 1-to-many, and
many-to-many will be all reflected to different values. Our extensive
evaluation on classic template matching benchmarks and deep learning tasks
demonstrate the effectiveness of QATM. It not only outperforms state-of-the-art
template matching methods when used alone, but also largely improves existing
deep network solutions.
Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection
Authors: Jia-Xing Zhong, Nannan Li, Weijie Kong, Shan Liu, Thomas H. Li, Ge Li
Date: 2019-03-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: Anomaly Detection, Anomaly Detection In Surveillance Videos, Multiple Instance Learning, Supervised Anomaly Detection, Video Anomaly Detection, Weakly-supervised Anomaly Detection

Video anomaly detection under weak labels is formulated as a typical
multiple-instance learning problem in previous works. In this paper, we provide
a new perspective, i.e., a supervised learning task under noisy labels. In such
a viewpoint, as long as cleaning away label noise, we can directly apply fully
supervised action classifiers to weakly supervised anomaly detection, and take
maximum advantage of these well-developed classifiers. For this purpose, we
devise a graph convolutional network to correct noisy labels. Based upon
feature similarity and temporal consistency, our network propagates supervisory
signals from high-confidence snippets to low-confidence ones. In this manner,
the network is capable of providing cleaned supervision for action classifiers.
During the test phase, we only need to obtain snippet-wise predictions from the
action classifier without any extra post-processing. Extensive experiments on 3
datasets at different scales with 2 types of action classifiers demonstrate the
efficacy of our method. Remarkably, we obtain the frame-level AUC score of
82.12% on UCF-Crime.
Semantic Image Synthesis with Spatially-Adaptive Normalization
Authors: Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu
Date: 2019-03-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Generation, Image-to-Image Translation, Sketch-to-Image Translation

We propose spatially-adaptive normalization, a simple but effective layer for synthesizing photorealistic images given an input semantic layout. Previous methods directly feed the semantic layout as input to the deep network, which is then processed through stacks of convolution, normalization, and nonlinearity layers. We show that this is suboptimal as the normalization layers tend to ``wash away'' semantic information. To address the issue, we propose using the input layout for modulating the activations in normalization layers through a spatially-adaptive, learned transformation. Experiments on several challenging datasets demonstrate the advantage of the proposed method over existing approaches, regarding both visual fidelity and alignment with input layouts. Finally, our model allows user control over both semantic and style. Code is available at https://github.com/NVlabs/SPADE .
Self-calibrating Deep Photometric Stereo Networks
Authors: Guan-Ying Chen, Kai Han, Boxin Shi, Yasuyuki Matsushita, Kwan-Yee K. Wong
Date: 2019-03-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: Deep Learning

This paper proposes an uncalibrated photometric stereo method for
non-Lambertian scenes based on deep learning. Unlike previous approaches that
heavily rely on assumptions of specific reflectances and light source
distributions, our method is able to determine both shape and light directions
of a scene with unknown arbitrary reflectances observed under unknown varying
light directions. To achieve this goal, we propose a two-stage deep learning
architecture, called SDPS-Net, which can effectively take advantage of
intermediate supervision, resulting in reduced learning difficulty compared to
a single-stage model. Experiments on both synthetic and real datasets show that
our proposed approach significantly outperforms previous uncalibrated
photometric stereo methods.
Understanding the Limitations of CNN-based Absolute Camera Pose Regression
Authors: Torsten Sattler, Qunjie Zhou, Marc Pollefeys, Laura Leal-Taixe
Date: 2019-03-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D geometry, Camera Pose Estimation, Image Retrieval, Mixed Reality, Pose Estimation, regression, Retrieval, Self-Driving Cars, Visual Localization

Visual localization is the task of accurate camera pose estimation in a known
scene. It is a key problem in computer vision and robotics, with applications
including self-driving cars, Structure-from-Motion, SLAM, and Mixed Reality.
Traditionally, the localization problem has been tackled using 3D geometry.
Recently, end-to-end approaches based on convolutional neural networks have
become popular. These methods learn to directly regress the camera pose from an
input image. However, they do not achieve the same level of pose accuracy as 3D
structure-based methods. To understand this behavior, we develop a theoretical
model for camera pose regression. We use our model to predict failure cases for
pose regression techniques and verify our predictions through experiments. We
furthermore use our model to show that pose regression is more closely related
to pose approximation via image retrieval than to accurate pose estimation via
3D structure. A key result is that current approaches do not consistently
outperform a handcrafted image retrieval baseline. This clearly shows that
additional research is needed before pose regression algorithms are ready to
compete with structure-based methods.
Learning Correspondence from the Cycle-Consistency of Time
Authors: Xiaolong Wang, Allan Jabri, Alexei A. Efros
Date: 2019-03-18 00:00:00
Proceeding: CVPR 2019 6
Tasks: Optical Flow Estimation, Semantic Segmentation, Semi-Supervised Video Object Segmentation, Unsupervised Video Object Segmentation, Video Object Segmentation, Video Semantic Segmentation

We introduce a self-supervised method for learning visual correspondence from
unlabeled video. The main idea is to use cycle-consistency in time as free
supervisory signal for learning visual representations from scratch. At
training time, our model learns a feature map representation to be useful for
performing cycle-consistent tracking. At test time, we use the acquired
representation to find nearest neighbors across space and time. We demonstrate
the generalizability of the representation -- without finetuning -- across a
range of visual correspondence tasks, including video object segmentation,
keypoint tracking, and optical flow. Our approach outperforms previous
self-supervised methods and performs competitively with strongly supervised
methods.
Multi-Differential Fairness Auditor for Black Box Classifiers
Authors: Xavier Gitiaux, Huzefa Rangwala
Date: 2019-03-18 00:00:00
Tasks: Decision Making, Fairness

Machine learning algorithms are increasingly involved in sensitive
decision-making process with adversarial implications on individuals. This
paper presents mdfa, an approach that identifies the characteristics of the
victims of a classifier's discrimination. We measure discrimination as a
violation of multi-differential fairness. Multi-differential fairness is a
guarantee that a black box classifier's outcomes do not leak information on the
sensitive attributes of a small group of individuals. We reduce the problem of
identifying worst-case violations to matching distributions and predicting
where sensitive attributes and classifier's outcomes coincide. We apply mdfa to
a recidivism risk assessment classifier and demonstrate that individuals
identified as African-American with little criminal history are three-times
more likely to be considered at high risk of violent recidivism than similar
individuals but not African-American.