pwc-archive/papers-with-abstracts huggingface dataset:
Semi Supervised Semantic Segmentation Using Generative Adversarial Network
Authors: Nasim Souly, Concetto Spampinato, Mubarak Shah
Date: 2017-10-01 00:00:00
Proceeding: ICCV 2017 10
Tasks: Benchmarking, General Classification, Generative Adversarial Network, Semantic Segmentation, Semi-Supervised Semantic Segmentation

Semantic segmentation has been a long standing challenging task in computer vision. It aims at assigning a label to each image pixel and needs a significant number of pixel-level annotated data, which is often unavailable. To address this lack of annotations, in this paper, we leverage, on one hand, a massive amount of available unlabeled or weakly labeled data, and on the other hand, non-realimages created through Generative Adversarial Networks. In particular, we propose a semi-supervised framework -based on Generative Adversarial Networks (GANs) - which consists of a generator network to provide extra training examples to a multi-class classifier, acting as discriminator in the GAN framework, that assigns sample a label y from the K possible classes or marks it as a fake sample (extra class). The underlying idea is that adding large fake visual data forces real samples to be close in the feature space, which, in turn, improves multiclass pixel classification. To ensure a higher quality of generated images by GANs with consequently improved pixel classification, we extend the above framework by adding weakly annotated data, i.e., we provide class level information to the generator. We test our approaches on several challenging benchmarking visual datasets, i.e. PASCAL, SiftFLow, Stanford and CamVid, achieving competitive performance compared to state-of-the-art semantic segmentation methods.
Deep Scene Image Classification With the MFAFVNet
Authors: Yunsheng Li, Mandar Dixit, Nuno Vasconcelos
Date: 2017-10-01 00:00:00
Proceeding: ICCV 2017 10
Tasks: Classification, General Classification, image-classification, Image Classification, Object Recognition, Scene Classification

The problem of transferring a deep convolutional network trained for object recognition to the task of scene image classification is considered. An embedded implementation of the recently proposed mixture of factor analyzers Fisher vector (MFA-FV) is proposed. This enables the design of a network architecture, the MFAFVNet, that can be trained in an end to end manner. The new architecture involves the design of an MFA-FV layer that implements a statistically correct version of the MFA-FV, through a combination of network computations and regularization. When compared to previous neural implementations of Fisher vectors, the MFAFVNet relies on a more powerful statistical model and a more accurate implementation. When compared to previous non-embedded models, the MFAFVNet relies on a state of the art model, which is now embedded into a CNN. This enables end to end training. Experiments show that the MFAFVNet has state of the art performance on scene classification.
Joint Prediction of Activity Labels and Starting Times in Untrimmed Videos
Authors: Tahmida Mahmud, Mahmudul Hasan, Amit K. Roy-Chowdhury
Date: 2017-10-01 00:00:00
Proceeding: ICCV 2017 10

Most of the existing works on human activity analysis focus on recognition or early recognition of the activity labels from complete or partial observations. Predicting the labels of future unobserved activities where no frames of the predicted activities have been observed is a challenging problem, with important applications, which has not been explored much. Associated with the future label prediction problem is the problem of predicting the starting time of the next activity. In this work, we propose a system that is able to infer about the labels and the starting times of future activities. Activities are characterized by the previous activity sequence (which is observed), as well as the objects present in the scene during their occurrence. We propose a network similar to a hybrid Siamese network with three branches to jointly learn both the future label and the starting time. The first branch takes visual features from the objects present in the scene using a fully connected network, the second branch takes previous activity features using a LSTM network to model long-term sequential relationships and the third branch captures the last observed activity features to model the context of inter-activity time using another fully connected network. These concatenated features are used for both label and time prediction. Experiments on two challenging datasets demonstrate that our framework for joint prediction of activity label and starting time improves the performance of both, and outperforms the state-of-the-arts.
TORNADO: A Spatio-Temporal Convolutional Regression Network for Video Action Proposal
Authors: Hongyuan Zhu, Romain Vial, Shijian Lu
Date: 2017-10-01 00:00:00
Proceeding: ICCV 2017 10
Tasks: Action Detection, regression

Given a video clip, action proposal aims to quickly generate a number of spatio-temporal tubes that enclose candidate human activities. Recently, the regression-based object detectors and long-term recurrent convolutional network (LRCN) have demonstrated superior performance in human action detection and recognition. However, the regression-based detectors performs inference without considering the temporal context among neighboring frames, and the LRCN using global visual percepts lacks the capability to capture local temporal dynamics. In this paper, we present a novel framework called TORNADO for human action proposal detection in un-trimmed video clips. Specifically, we propose a spatial-temporal convolutional network that combines the advantages of regression-based detector and LRCN by empowering Convolutional LSTM with regression capability. Our approach consists of a temporal convolutional regression network (T-CRN) and a spatial regression network (S-CRN) which are trained end-to-end on both RGB and OpticalFlow streams. They fuse appearance, motion and temporal contexts to regress the bounding boxes of candidate human actions simultaneously in 28 FPS. The action proposals are constructed by solving dynamic programming with peak trimming of the generated action boxes. Extensive experiments on the challenging UCF-101 and UCF-Sports datasets show that our method achieves superior performance as compared with the state-of-the-arts.
Learning Action Recognition Model From Depth and Skeleton Videos
Authors: Hossein Rahmani, Mohammed Bennamoun
Date: 2017-10-01 00:00:00
Proceeding: ICCV 2017 10
Tasks: Action Recognition, Human-Object Interaction Detection, Temporal Action Localization

Depth sensors open up possibilities of dealing with the human action recognition problem by providing 3D human skeleton data and depth images of the scene. Analysis of human actions based on 3D skeleton data has become popular recently, due to its robustness and view-invariant representation. However, the skeleton alone is insufficient to distinguish actions which involve human-object interactions. In this paper, we propose a deep model which efficiently models human-object interactions and intra-class variations under viewpoint changes. First, a human body-part model is introduced to transfer the depth appearances of body-parts to a shared view-invariant space. Second, an end-to-end learning framework is proposed which is able to effectively combine the view-invariant body-part representation from skeletal and depth images, and learn the relations between the human body-parts and the environmental objects, the interactions between different human body-parts, and the temporal structure of human actions. We have evaluated the performance of our proposed model against 15 existing techniques on two large benchmark human action recognition datasets including NTU RGB+D and UWA3DII. The Experimental results show that our technique provides a significant improvement over state-of-the-art methods.
Scale Recovery for Monocular Visual Odometry Using Depth Estimated With Deep Convolutional Neural Fields
Authors: Xiaochuan Yin, Xiangwei Wang, Xiaoguo Du, Qijun Chen
Date: 2017-10-01 00:00:00
Proceeding: ICCV 2017 10
Tasks: Depth Estimation, Monocular Visual Odometry, Visual Odometry

Scale recovery is one of the central problems for monocular visual odometry. Normally, road plane and camera height are specified as reference to recover the scale. The performances of these methods depend on the plane recognition and height measurement of camera. In this work, we propose a novel method to recover the scale by incorporating the depths estimated from images using deep convolutional neural fields. Our method considers the whole environmental structure as reference rather than a specified plane. The accuracy of depth estimation contributes to the scale recovery. We improve the performance of depth estimation by considering two consecutive frames and egomotion of camera into our networks. The depth refinement and scale recovery are obtained iteratively. In this way, our method can eliminate the scale drift and improve the depth estimation simultaneously. The effectiveness of our method is verified on the KITTI dataset for both visual odometry and depth estimation tasks.
Deep Adaptive Image Clustering
Authors: Jianlong Chang, Lingfeng Wang, Gaofeng Meng, Shiming Xiang, Chunhong Pan
Date: 2017-10-01 00:00:00
Proceeding: ICCV 2017 10
Tasks: Clustering, Image Clustering

Image clustering is a crucial but challenging task in machine learning and computer vision. Existing methods often ignore the combination between feature learning and clustering. To tackle this problem, we propose Deep Adaptive Clustering (DAC) that recasts the clustering problem into a binary pairwise-classification framework to judge whether pairs of images belong to the same clusters. In DAC, the similarities are calculated as the cosine distance between label features of images which are generated by a deep convolutional network (ConvNet). By introducing a constraint into DAC, the learned label features tend to be one-hot vectors that can be utilized for clustering images. The main challenge is that the ground-truth similarities are unknown in image clustering. We handle this issue by presenting an alternating iterative Adaptive Learning algorithm where each iteration alternately selects labeled samples and trains the ConvNet. Conclusively, images are automatically clustered based on the label features. Experimental results show that DAC achieves state-of-the-art performance on five popular datasets, e.g., yielding 97.75% clustering accuracy on MNIST, 52.18% on CIFAR-10 and 46.99% on STL-10.
One Network to Solve Them All -- Solving Linear Inverse Problems Using Deep Projection Models
Authors: J. H. Rick Chang, Chun-Liang Li, Barnabas Poczos, B. V. K. Vijaya Kumar, Aswin C. Sankaranarayanan
Date: 2017-10-01 00:00:00
Proceeding: ICCV 2017 10
Tasks: All, Compressive Sensing, Image Inpainting, Super-Resolution

While deep learning methods have achieved state-of-the-art performance in many challenging inverse problems like image inpainting and super-resolution, they invariably involve problem-specific training of the networks. Under this approach, each inverse problem requires its own dedicated network. In scenarios where we need to solve a wide variety of problems, e.g., on a mobile camera, it is inefficient and expensive to use these problem-specific networks. On the other hand, traditional methods using analytic signal priors can be used to solve any linear inverse problem; this often comes with a performance that is worse than learning-based methods. In this work, we provide a middle ground between the two kinds of methods -- we propose a general framework to train a single deep neural network that solves arbitrary linear inverse problems. We achieve this by training a network that acts as a quasi-projection operator for the set of natural images and show that any linear inverse problem involving natural images can be solved using iterative methods. We empirically show that the proposed framework demonstrates superior performance over traditional methods using wavelet sparsity prior while achieving performance comparable to specially-trained networks on tasks including compressive sensing and pixel-wise inpainting.
Initialization of Self-Organizing Maps: Principal Components Versus Random Initialization. A Case Study
Authors: A. A. Akinduko, E. M. Mirkes
Date: 2012-10-22 00:00:00

The performance of the Self-Organizing Map (SOM) algorithm is dependent on
the initial weights of the map. The different initialization methods can
broadly be classified into random and data analysis based initialization
approach. In this paper, the performance of random initialization (RI) approach
is compared to that of principal component initialization (PCI) in which the
initial map weights are chosen from the space of the principal component.
Performance is evaluated by the fraction of variance unexplained (FVU).
Datasets were classified into quasi-linear and non-linear and it was observed
that RI performed better for non-linear datasets; however the performance of
PCI approach remains inconclusive for quasi-linear datasets.
Glyce: Glyph-vectors for Chinese Character Representations
Authors: Yuxian Meng, Wei Wu, Fei Wang, Xiaoya Li, Ping Nie, Fan Yin, Muyu Li, Qinghong Han, Xiaofei Sun, Jiwei Li
Date: 2019-01-29 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Chinese Dependency Parsing, Chinese Named Entity Recognition, Chinese Part-of-Speech Tagging, Chinese Semantic Role Labeling, Chinese Sentence Pair Classification, Chinese Word Segmentation, Classification, Dependency Parsing, Document Classification, General Classification, image-classification, Image Classification, Language Modelling, Machine Translation, Multi-Task Learning, NER, Part-Of-Speech Tagging, POS, Semantic Role Labeling, Semantic Textual Similarity, Sentence, Sentence Classification, Sentence-Pair Classification, Sentiment Analysis, text-classification, Text Classification

It is intuitive that NLP tasks for logographic languages like Chinese should benefit from the use of the glyph information in those languages. However, due to the lack of rich pictographic evidence in glyphs and the weak generalization ability of standard computer vision models on character data, an effective way to utilize the glyph information remains to be found. In this paper, we address this gap by presenting Glyce, the glyph-vectors for Chinese character representations. We make three major innovations: (1) We use historical Chinese scripts (e.g., bronzeware script, seal script, traditional Chinese, etc) to enrich the pictographic evidence in characters; (2) We design CNN structures (called tianzege-CNN) tailored to Chinese character image processing; and (3) We use image-classification as an auxiliary task in a multi-task learning setup to increase the model's ability to generalize. We show that glyph-based models are able to consistently outperform word/char ID-based models in a wide range of Chinese NLP tasks. We are able to set new state-of-the-art results for a variety of Chinese NLP tasks, including tagging (NER, CWS, POS), sentence pair classification, single sentence classification tasks, dependency parsing, and semantic role labeling. For example, the proposed model achieves an F1 score of 80.6 on the OntoNotes dataset of NER, +1.5 over BERT; it achieves an almost perfect accuracy of 99.8\% on the Fudan corpus for text classification. Code found at https://github.com/ShannonAI/glyce.
Diversity in Faces
Authors: Michele Merler, Nalini Ratha, Rogerio S. Feris, John R. Smith
Date: 2019-01-29 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction, Diversity, Face Recognition, Fairness

Face recognition is a long standing challenge in the field of Artificial
Intelligence (AI). The goal is to create systems that accurately detect,
recognize, verify, and understand human faces. There are significant technical
hurdles in making these systems accurate, particularly in unconstrained
settings due to confounding factors related to pose, resolution, illumination,
occlusion, and viewpoint. However, with recent advances in neural networks,
face recognition has achieved unprecedented accuracy, largely built on
data-driven deep learning methods. While this is encouraging, a critical aspect
that is limiting facial recognition accuracy and fairness is inherent facial
diversity. Every face is different. Every face reflects something unique about
us. Aspects of our heritage - including race, ethnicity, culture, geography -
and our individual identify - age, gender, and other visible manifestations of
self-expression, are reflected in our faces. We expect face recognition to work
equally accurately for every face. Face recognition needs to be fair. As we
rely on data-driven methods to create face recognition technology, we need to
ensure necessary balance and coverage in training data. However, there are
still scientific questions about how to represent and extract pertinent facial
features and quantitatively measure facial diversity. Towards this goal,
Diversity in Faces (DiF) provides a data set of one million annotated human
face images for advancing the study of facial diversity. The annotations are
generated using ten well-established facial coding schemes from the scientific
literature. The facial coding schemes provide human-interpretable quantitative
measures of facial features. We believe that by making the extracted coding
schemes available on a large set of faces, we can accelerate research and
development towards creating more fair and accurate facial recognition systems.
Progressive Augmentation of GANs
Authors: Dan Zhang, Anna Khoreva
Date: 2019-01-29 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Image Generation

Training of Generative Adversarial Networks (GANs) is notoriously fragile, requiring to maintain a careful balance between the generator and the discriminator in order to perform well. To mitigate this issue we introduce a new regularization technique - progressive augmentation of GANs (PA-GAN). The key idea is to gradually increase the task difficulty of the discriminator by progressively augmenting its input or feature space, thus enabling continuous learning of the generator. We show that the proposed progressive augmentation preserves the original GAN objective, does not compromise the discriminator's optimality and encourages a healthy competition between the generator and discriminator, leading to the better-performing generator. We experimentally demonstrate the effectiveness of PA-GAN across different architectures and on multiple benchmarks for the image synthesis task, on average achieving ~3 point improvement of the FID score.
Unsupervised Person Re-identification by Deep Asymmetric Metric Embedding
Authors: Hong-Xing Yu, An-Cong Wu, Wei-Shi Zheng
Date: 2019-01-29 00:00:00
Tasks: Clustering, Deep Clustering, Metric Learning, Person Re-Identification, Unsupervised Person Re-Identification

Person re-identification (Re-ID) aims to match identities across
non-overlapping camera views. Researchers have proposed many supervised Re-ID
models which require quantities of cross-view pairwise labelled data. This
limits their scalabilities to many applications where a large amount of data
from multiple disjoint camera views is available but unlabelled. Although some
unsupervised Re-ID models have been proposed to address the scalability
problem, they often suffer from the view-specific bias problem which is caused
by dramatic variances across different camera views, e.g., different
illumination, viewpoints and occlusion. The dramatic variances induce specific
feature distortions in different camera views, which can be very disturbing in
finding cross-view discriminative information for Re-ID in the unsupervised
scenarios, since no label information is available to help alleviate the bias.
We propose to explicitly address this problem by learning an unsupervised
asymmetric distance metric based on cross-view clustering. The asymmetric
distance metric allows specific feature transformations for each camera view to
tackle the specific feature distortions. We then design a novel unsupervised
loss function to embed the asymmetric metric into a deep neural network, and
therefore develop a novel unsupervised deep framework named the DEep
Clustering-based Asymmetric MEtric Learning (DECAMEL). In such a way, DECAMEL
jointly learns the feature representation and the unsupervised asymmetric
metric. DECAMEL learns a compact cross-view cluster structure of Re-ID data,
and thus help alleviate the view-specific bias and facilitate mining the
potential cross-view discriminative information for unsupervised Re-ID.
Extensive experiments on seven benchmark datasets whose sizes span several
orders show the effectiveness of our framework.
Two-Stream Multi-Task Network for Fashion Recognition
Authors: Peizhao Li, Yanjing Li, Xiao-Long Jiang, Xian-Tong Zhen
Date: 2019-01-29 00:00:00
Tasks: Attribute, Multi-Task Learning, Vocal Bursts Valence Prediction

In this paper, we present a two-stream multi-task network for fashion recognition. This task is challenging as fashion clothing always contain multiple attributes, which need to be predicted simultaneously for real-time industrial systems. To handle these challenges, we formulate fashion recognition into a multi-task learning problem, including landmark detection, category and attribute classifications, and solve it with the proposed deep convolutional neural network. We design two knowledge sharing strategies which enable information transfer between tasks and improve the overall performance. The proposed model achieves state-of-the-art results on large-scale fashion dataset comparing to the existing methods, which demonstrates its great effectiveness and superiority for fashion recognition.
Attention-based Context Aggregation Network for Monocular Depth Estimation
Authors: Yuru Chen, Haitao Zhao, Zhengwei Hu
Date: 2019-01-29 00:00:00
Tasks: Depth Estimation, Monocular Depth Estimation, Multi-class Classification

Depth estimation is a traditional computer vision task, which plays a crucial
role in understanding 3D scene geometry. Recently,
deep-convolutional-neural-networks based methods have achieved promising
results in the monocular depth estimation field. Specifically, the framework
that combines the multi-scale features extracted by the dilated convolution
based block (atrous spatial pyramid pooling, ASPP) has gained the significant
improvement in the dense labeling task. However, the discretized and predefined
dilation rates cannot capture the continuous context information that differs
in diverse scenes and easily introduce the grid artifacts in depth estimation.
In this paper, we propose an attention-based context aggregation network (ACAN)
to tackle these difficulties. Based on the self-attention model, ACAN
adaptively learns the task-specific similarities between pixels to model the
context information. First, we recast the monocular depth estimation as a dense
labeling multi-class classification problem. Then we propose a soft ordinal
inference to transform the predicted probabilities to continuous depth values,
which can reduce the discretization error (about 1% decrease in RMSE). Second,
the proposed ACAN aggregates both the image-level and pixel-level context
information for depth estimation, where the former expresses the statistical
characteristic of the whole image and the latter extracts the long-range
spatial dependencies for each pixel. Third, for further reducing the
inconsistency between the RGB image and depth map, we construct an attention
loss to minimize their information entropy. We evaluate on public monocular
depth-estimation benchmark datasets (including NYU Depth V2, KITTI). The
experiments demonstrate the superiority of our proposed ACAN and achieve the
competitive results with the state of the arts.