pwc-archive/papers-with-abstracts huggingface dataset:
Spatial-Spectral Feature Extraction via Deep ConvLSTM Neural Networks for Hyperspectral Image Classification
Authors: Wen-Shuai Hu, Heng-Chao Li, Lei Pan, Wei Li, Ran Tao, Qian Du
Date: 2019-05-09 00:00:00
Tasks: General Classification, Hyperspectral Image Classification, image-classification, Image Classification

In recent years, deep learning has presented a great advance in hyperspectral image (HSI) classification. Particularly, long short-term memory (LSTM), as a special deep learning structure, has shown great ability in modeling long-term dependencies in the time dimension of video or the spectral dimension of HSIs. However, the loss of spatial information makes it quite difficult to obtain the better performance. In order to address this problem, two novel deep models are proposed to extract more discriminative spatial-spectral features by exploiting the Convolutional LSTM (ConvLSTM). By taking the data patch in a local sliding window as the input of each memory cell band by band, the 2-D extended architecture of LSTM is considered for building the spatial-spectral ConvLSTM 2-D Neural Network (SSCL2DNN) to model long-range dependencies in the spectral domain. To better preserve the intrinsic structure information of the hyperspectral data, the spatial-spectral ConvLSTM 3-D Neural Network (SSCL3DNN) is proposed by extending LSTM to 3-D version for further improving the classification performance. The experiments, conducted on three commonly used HSI data sets, demonstrate that the proposed deep models have certain competitive advantages and can provide better classification performance than other state-of-the-art approaches.
Fast and Efficient Zero-Learning Image Fusion
Authors: Fayez Lahoud, Sabine Süsstrunk
Date: 2019-05-09 00:00:00

We propose a real-time image fusion method using pre-trained neural networks. Our method generates a single image containing features from multiple sources. We first decompose images into a base layer representing large scale intensity variations, and a detail layer containing small scale changes. We use visual saliency to fuse the base layers, and deep feature maps extracted from a pre-trained neural network to fuse the detail layers. We conduct ablation studies to analyze our method's parameters such as decomposition filters, weight construction methods, and network depth and architecture. Then, we validate its effectiveness and speed on thermal, medical, and multi-focus fusion. We also apply it to multiple image inputs such as multi-exposure sequences. The experimental results demonstrate that our technique achieves state-of-the-art performance in visual quality, objective assessment, and runtime efficiency.
Intra-frame Object Tracking by Deblatting
Authors: Jan Kotera, Denys Rozumnyi, Filip Šroubek, Jiří Matas
Date: 2019-05-09 00:00:00
Tasks: Deblurring, Image Matting, Object, Object Tracking, Video Super-Resolution

Objects moving at high speed along complex trajectories often appear in videos, especially videos of sports. Such objects elapse non-negligible distance during exposure time of a single frame and therefore their position in the frame is not well defined. They appear as semi-transparent streaks due to the motion blur and cannot be reliably tracked by standard trackers. We propose a novel approach called Tracking by Deblatting based on the observation that motion blur is directly related to the intra-frame trajectory of an object. Blur is estimated by solving two intertwined inverse problems, blind deblurring and image matting, which we call deblatting. The trajectory is then estimated by fitting a piecewise quadratic curve, which models physically justifiable trajectories. As a result, tracked objects are precisely localized with higher temporal resolution than by conventional trackers. The proposed TbD tracker was evaluated on a newly created dataset of videos with ground truth obtained by a high-speed camera using a novel Trajectory-IoU metric that generalizes the traditional Intersection over Union and measures the accuracy of the intra-frame trajectory. The proposed method outperforms baseline both in recall and trajectory accuracy.
Liver Lesion Segmentation with slice-wise 2D Tiramisu and Tversky loss function
Authors: Karsten Roth, Tomasz Konopczyński, Jürgen Hesser
Date: 2019-05-09 00:00:00
Tasks: Lesion Segmentation, Liver Segmentation, Segmentation, Sensitivity, Tumor Segmentation

At present, lesion segmentation is still performed manually (or semi-automatically) by medical experts. To facilitate this process, we contribute a fully-automatic lesion segmentation pipeline. This work proposes a method as a part of the LiTS (Liver Tumor Segmentation Challenge) competition for ISBI 17 and MICCAI 17 comparing methods for automatics egmentation of liver lesions in CT scans. By utilizing cascaded, densely connected 2D U-Nets and a Tversky-coefficient based loss function, our framework achieves very good shape extractions with high detection sensitivity, with competitive scores at time of publication. In addition, adjusting hyperparameters in our Tversky-loss allows to tune the network towards higher sensitivity or robustness.
TE141K: Artistic Text Benchmark for Text Effect Transfer
Authors: Shuai Yang, Wenjing Wang, Jiaying Liu
Date: 2019-05-08 00:00:00
Tasks: Style Transfer, Text Effects Transfer

Text effects are combinations of visual elements such as outlines, colors and textures of text, which can dramatically improve its artistry. Although text effects are extensively utilized in the design industry, they are usually created by human experts due to their extreme complexity; this is laborious and not practical for normal users. In recent years, some efforts have been made toward automatic text effect transfer; however, the lack of data limits the capabilities of transfer models. To address this problem, we introduce a new text effects dataset, TE141K, with 141,081 text effect/glyph pairs in total. Our dataset consists of 152 professionally designed text effects rendered on glyphs, including English letters, Chinese characters, and Arabic numerals. To the best of our knowledge, this is the largest dataset for text effect transfer to date. Based on this dataset, we propose a baseline approach called text effect transfer GAN (TET-GAN), which supports the transfer of all 152 styles in one model and can efficiently extend to new styles. Finally, we conduct a comprehensive comparison in which 14 style transfer models are benchmarked. Experimental results demonstrate the superiority of TET-GAN both qualitatively and quantitatively and indicate that our dataset is effective and challenging.
S4L: Self-Supervised Semi-Supervised Learning
Authors: Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, Lucas Beyer
Date: 2019-05-09 00:00:00
Proceeding: ICCV 2019 10
Tasks: General Classification, image-classification, Image Classification, Representation Learning, Semi-Supervised Image Classification

This work tackles the problem of semi-supervised learning of image classifiers. Our main insight is that the field of semi-supervised learning can benefit from the quickly advancing field of self-supervised visual representation learning. Unifying these two approaches, we propose the framework of self-supervised semi-supervised learning and use it to derive two novel semi-supervised image classification methods. We demonstrate the effectiveness of these methods in comparison to both carefully tuned baselines, and existing semi-supervised learning methods. We then show that our approach and existing semi-supervised methods can be jointly trained, yielding a new state-of-the-art result on semi-supervised ILSVRC-2012 with 10% of labels.
Seesaw-Net: Convolution Neural Network With Uneven Group Convolution
Authors: Jintao Zhang
Date: 2019-05-09 00:00:00
Tasks: image-classification, Image Classification, Neural Architecture Search

In this paper, we are interested in boosting the representation capability of convolution neural networks which utilizing the inverted residual structure. Based on the success of Inverted Residual structure[Sandler et al. 2018] and Interleaved Low-Rank Group Convolutions[Sun et al. 2018], we rethink this two pattern of neural network structure, rather than NAS(Neural architecture search) method[Zoph and Le 2017; Pham et al. 2018; Liu et al. 2018b], we introduce uneven point-wise group convolution, which provide a novel search space for designing basic blocks to obtain better trade-off between representation capability and computational cost. Meanwhile, we propose two novel information flow patterns that will enable cross-group information flow for multiple group convolution layers with and without any channel permute/shuffle operation. Dense experiments on image classification task show that our proposed model, named Seesaw-Net, achieves state-of-the-art(SOTA) performance with limited computation and memory cost. Our code will be open-source and available together with pre-trained models.
Learning Loss for Active Learning
Authors: Donggeun Yoo, In So Kweon
Date: 2019-05-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Active Learning, image-classification, Image Classification, object-detection, Object Detection, Pose Estimation

The performance of deep neural networks improves with more annotated data. The problem is that the budget for annotation is limited. One solution to this is active learning, where a model asks human to annotate data that it perceived as uncertain. A variety of recent methods have been proposed to apply active learning to deep networks but most of them are either designed specific for their target tasks or computationally inefficient for large networks. In this paper, we propose a novel active learning method that is simple but task-agnostic, and works efficiently with the deep networks. We attach a small parametric module, named "loss prediction module," to a target network, and learn it to predict target losses of unlabeled inputs. Then, this module can suggest data that the target model is likely to produce a wrong prediction. This method is task-agnostic as networks are learned from a single loss regardless of target tasks. We rigorously validate our method through image classification, object detection, and human pose estimation, with the recent network architectures. The results demonstrate that our method consistently outperforms the previous methods over the tasks.
What Do Single-view 3D Reconstruction Networks Learn?
Authors: Maxim Tatarchenko, Stephan R. Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, Thomas Brox
Date: 2019-05-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Reconstruction, Decoder, General Classification, image-classification, Image Classification, Object Reconstruction, Retrieval, Single-View 3D Reconstruction

Convolutional networks for single-view object reconstruction have shown impressive performance and have become a popular subject of research. All existing techniques are united by the idea of having an encoder-decoder network that performs non-trivial reasoning about the 3D structure of the output space. In this work, we set up two alternative approaches that perform image classification and retrieval respectively. These simple baselines yield better results than state-of-the-art methods, both qualitatively and quantitatively. We show that encoder-decoder methods are statistically indistinguishable from these baselines, thus indicating that the current state of the art in single-view object reconstruction does not actually perform reconstruction but image classification. We identify aspects of popular experimental procedures that elicit this behavior and discuss ways to improve the current state of research.
Forecasting Pedestrian Trajectory with Machine-Annotated Training Data
Authors: Olly Styles, Arun Ross, Victor Sanchez
Date: 2019-05-09 00:00:00
Tasks: Autonomous Vehicles, Pedestrian Detection, Trajectory Forecasting

Reliable anticipation of pedestrian trajectory is imperative for the operation of autonomous vehicles and can significantly enhance the functionality of advanced driver assistance systems. While significant progress has been made in the field of pedestrian detection, forecasting pedestrian trajectories remains a challenging problem due to the unpredictable nature of pedestrians and the huge space of potentially useful features. In this work, we present a deep learning approach for pedestrian trajectory forecasting using a single vehicle-mounted camera. Deep learning models that have revolutionized other areas in computer vision have seen limited application to trajectory forecasting, in part due to the lack of richly annotated training data. We address the lack of training data by introducing a scalable machine annotation scheme that enables our model to be trained using a large dataset without human annotation. In addition, we propose Dynamic Trajectory Predictor (DTP), a model for forecasting pedestrian trajectory up to one second into the future. DTP is trained using both human and machine-annotated data, and anticipates dynamic motion that is not captured by linear models. Experimental evaluation confirms the benefits of the proposed model.
Deep AutoEncoder-based Lossy Geometry Compression for Point Clouds
Authors: Wei Yan, Yiting shao, Shan Liu, Thomas H. Li, Zhu Li, Ge Li
Date: 2019-04-18 00:00:00
Tasks: Autonomous Driving, Image Compression

Point cloud is a fundamental 3D representation which is widely used in real
world applications such as autonomous driving. As a newly-developed media
format which is characterized by complexity and irregularity, point cloud
creates a need for compression algorithms which are more flexible than existing
codecs. Recently, autoencoders(AEs) have shown their effectiveness in many
visual analysis tasks as well as image compression, which inspires us to employ
it in point cloud compression. In this paper, we propose a general
autoencoder-based architecture for lossy geometry point cloud compression. To
the best of our knowledge, it is the first autoencoder-based geometry
compression codec that directly takes point clouds as input rather than voxel
grids or collections of images. Compared with handcrafted codecs, this approach
adapts much more quickly to previously unseen media contents and media formats,
meanwhile achieving competitive performance. Our architecture consists of a
pointnet-based encoder, a uniform quantizer, an entropy estimation block and a
nonlinear synthesis transformation module. In lossy geometry compression of
point cloud, results show that the proposed method outperforms the test model
for categories 1 and 3 (TMC13) published by MPEG-3DG group on the 125th
meeting, and on average a 73.15\% BD-rate gain is achieved.
Supervised Online Hashing via Hadamard Codebook Learning
Authors: Mingbao Lin, Rongrong Ji, Hong Liu, Yongjian Liu
Date: 2019-04-28 00:00:00
Tasks: Retrieval, Semantic Similarity, Semantic Textual Similarity

In recent years, binary code learning, a.k.a hashing, has received extensive attention in large-scale multimedia retrieval. It aims to encode high-dimensional data points to binary codes, hence the original high-dimensional metric space can be efficiently approximated via Hamming space. However, most existing hashing methods adopted offline batch learning, which is not suitable to handle incremental datasets with streaming data or new instances. In contrast, the robustness of the existing online hashing remains as an open problem, while the embedding of supervised/semantic information hardly boosts the performance of the online hashing, mainly due to the defect of unknown category numbers in supervised learning. In this paper, we proposed an online hashing scheme, termed Hadamard Codebook based Online Hashing (HCOH), which aims to solve the above problems towards robust and supervised online hashing. In particular, we first assign an appropriate high-dimensional binary codes to each class label, which is generated randomly by Hadamard codes to each class label, which is generated randomly by Hadamard codes. Subsequently, LSH is adopted to reduce the length of such Hadamard codes in accordance with the hash bits, which can adapt the predefined binary codes online, and theoretically guarantee the semantic similarity. Finally, we consider the setting of stochastic data acquisition, which facilitates our method to efficiently learn the corresponding hashing functions via stochastic gradient descend (SGD) online. Notably, the proposed HCOH can be embedded with supervised labels and it not limited to a predefined category number. Extensive experiments on three widely-used benchmarks demonstrate the merits of the proposed scheme over the state-of-the-art methods. The code is available at https://github.com/lmbxmu/mycode/tree/master/2018ACMMM_HCOH.
Measuring similarity between geo-tagged videos using largest common view
Authors: Wei Ding, KwangSoo Yang, Kwang Woo Nam
Date: 2019-04-28 00:00:00

This paper presents a novel problem for discovering the similar trajectories
based on the field of view (FoV) of the video data. The problem is important
for many societal applications such as grouping moving objects, classifying
geo-images, and identifying the interesting trajectory patterns. Prior work
consider only either spatial locations or spatial relationship between two
line-segments. However, these approaches show a limitation to find the similar
moving objects with common views. In this paper, we propose new algorithm that
can group both spatial locations and points of view to identify similar
trajectories. We also propose novel methods that reduce the computational cost
for the proposed work. Experimental results using real-world datasets
demonstrates that the proposed approach outperforms prior work and reduces the
computational cost.
HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
Authors: Zhen Dong, Zhewei Yao, Amir Gholami, Michael Mahoney, Kurt Keutzer
Date: 2019-04-29 00:00:00
Proceeding: ICCV 2019 10
Tasks: Quantization

Model size and inference speed/power have become a major challenge in the
deployment of Neural Networks for many applications. A promising approach to
address these problems is quantization. However, uniformly quantizing a model
to ultra low precision leads to significant accuracy degradation. A novel
solution for this is to use mixed-precision quantization, as some parts of the
network may allow lower precision as compared to other layers. However, there
is no systematic way to determine the precision of different layers. A brute
force approach is not feasible for deep networks, as the search space for
mixed-precision is exponential in the number of layers. Another challenge is a
similar factorial complexity for determining block-wise fine-tuning order when
quantizing the model to a target precision. Here, we introduce Hessian AWare
Quantization (HAWQ), a novel second-order quantization method to address these
problems. HAWQ allows for the automatic selection of the relative quantization
precision of each layer, based on the layer's Hessian spectrum. Moreover, HAWQ
provides a deterministic fine-tuning order for quantizing layers, based on
second-order information. We show the results of our method on Cifar-10 using
ResNet20, and on ImageNet using Inception-V3, ResNet50 and SqueezeNext models.
Comparing HAWQ with state-of-the-art shows that we can achieve similar/better
accuracy with $8\times$ activation compression ratio on ResNet20, as compared
to DNAS~\cite{wu2018mixed}, and up to $1\%$ higher accuracy with up to $14\%$
smaller models on ResNet50 and Inception-V3, compared to recently proposed
methods of RVQuant~\cite{park2018value} and HAQ~\cite{wang2018haq}.
Furthermore, we show that we can quantize SqueezeNext to just 1MB model size
while achieving above $68\%$ top1 accuracy on ImageNet.
On Applying Machine Learning/Object Detection Models for Analysing Digitally Captured Physical Prototypes from Engineering Design Projects
Authors: Jorgen F. Erichsen, Sampsa Kohtala, Martin Steinert, Torgeir Welo
Date: 2019-05-07 00:00:00
Tasks: Object, object-detection, Object Detection

While computer vision has received increasing attention in computer science over the last decade, there are few efforts in applying this to leverage engineering design research. Existing datasets and technologies allow researchers to capture and access more observations and video files, hence analysis is becoming a limiting factor. Therefore, this paper is investigating the application of machine learning, namely object detection methods to aid in the analysis of physical porotypes. With access to a large dataset of digitally captured physical prototypes from early-stage development projects (5950 images from 850 prototypes), the authors investigate applications that can be used for analysing this dataset. The authors retrained two pre-trained object detection models from two known frameworks, the TensorFlow Object Detection API and Darknet, using custom image sets of images of physical prototypes. As a result, a proof-of-concept of four trained models are presented; two models for detecting samples of wood-based sheet materials and two models for detecting samples containing microcontrollers. All models are evaluated using standard metrics for object detection model performance and the applicability of using object detection models in engineering design research is discussed. Results indicate that the models can successfully classify the type of material and type of pre-made component, respectively. However, more work is needed to fully integrate object detection models in the engineering design analysis workflow. The authors also extrapolate that the use of object detection for analysing images of physical prototypes will substantially reduce the effort required for analysing large datasets in engineering design research.
Inferring the Importance of Product Appearance: A Step Towards the Screenless Revolution
Authors: Yongshun Gong, Jin-Feng Yi, Dong-Dong Chen, Jian Zhang, Jiayu Zhou, ZhiHua Zhou
Date: 2019-05-01 00:00:00
Tasks: Decision Making

Nowadays, almost all the online orders were placed through screened devices such as mobile phones, tablets, and computers. With the rapid development of the Internet of Things (IoT) and smart appliances, more and more screenless smart devices, e.g., smart speaker and smart refrigerator, appear in our daily lives. They open up new means of interaction and may provide an excellent opportunity to reach new customers and increase sales. However, not all the items are suitable for screenless shopping, since some items' appearance play an important role in consumer decision making. Typical examples include clothes, dolls, bags, and shoes. In this paper, we aim to infer the significance of every item's appearance in consumer decision making and identify the group of items that are suitable for screenless shopping. Specifically, we formulate the problem as a classification task that predicts if an item's appearance has a significant impact on people's purchase behavior. To solve this problem, we extract features from three different views, namely items' intrinsic properties, items' images, and users' comments, and collect a set of necessary labels via crowdsourcing. We then propose an iterative semi-supervised learning framework with three carefully designed loss functions. We conduct extensive experiments on a real-world transaction dataset collected from the online retail giant JD.com. Experimental results verify the effectiveness of the proposed method.