pwc-archive/papers-with-abstracts huggingface dataset:
Dynamic Curriculum Learning for Low-Resource Neural Machine Translation
Authors: Chen Xu, Bojie Hu, Yufan Jiang, Kai Feng, Zeyang Wang, Shen Huang, Qi Ju, Tong Xiao, Jingbo Zhu
Date: 2020-11-30 00:00:00
Proceeding: COLING 2020 8
Tasks: Low Resource Neural Machine Translation, Low-Resource Neural Machine Translation, Low Resource NMT, Machine Translation, NMT, Translation

Large amounts of data has made neural machine translation (NMT) a big success in recent years. But it is still a challenge if we train these models on small-scale corpora. In this case, the way of using data appears to be more important. Here, we investigate the effective use of training data for low-resource NMT. In particular, we propose a dynamic curriculum learning (DCL) method to reorder training samples in training. Unlike previous work, we do not use a static scoring function for reordering. Instead, the order of training samples is dynamically determined in two ways - loss decline and model competence. This eases training by highlighting easy samples that the current model has enough competence to learn. We test our DCL method in a Transformer-based system. Experimental results show that DCL outperforms several strong baselines on three low-resource machine translation benchmarks and different sized data of WMT' 16 En-De.
Zero-Shot Calibration of Fisheye Cameras
Authors: Jae-Yeong Lee
Date: 2020-11-30 00:00:00
Tasks: Camera Calibration, Common Sense Reasoning

In this paper, we present a novel zero-shot camera calibration method that estimates camera parameters with no calibration image. It is common sense that we need at least one or more pattern images for camera calibration. However, the proposed method estimates camera parameters from the horizontal and vertical field of view information of the camera without any image acquisition. The proposed method is particularly useful for wide-angle or fisheye cameras that have large image distortion. Image distortion is modeled in the way fisheye lenses are designed and estimated based on the square pixel assumption of the image sensors. The calibration accuracy of the proposed method is evaluated on eight different commercial cameras qualitatively and quantitatively, and compared with conventional calibration methods. The experimental results show that the calibration accuracy of the zero-shot method is comparable to conventional full calibration results. The method can be used as a practical alternative in real applications where individual calibration is difficult or impractical, and in most field applications where calibration accuracy is less critical. Moreover, the estimated camera parameters by the method can also be used to provide proper initialization of any existing calibration methods, making them to converge more stably and avoid local minima.
REaL: Real-time Face Detection and Recognition Using Euclidean Space and Likelihood Estimation
Authors: Sandesh Ramesh, Manoj Kumar M V, K Aditya Shastry
Date: 2020-11-30 00:00:00
Tasks: Face Detection

Detecting and recognizing faces accurately has always been a challenge. Differentiating facial features, training images, and producing quick results require a lot of computation. The REaL system we have proposed in this paper discusses its functioning and ways in which computations can be carried out in a short period. REaL experiments are carried out on live images and the recognition rates are promising. The system is also successful in removing non-human objects from its calculations. The system uses a local database to store captured images and feeds the neural network frequently. The captured images are cropped automatically to remove unwanted noise. The system calculates the Euler angles and the probability of whether the face is smiling, has its left eye, and right eyes open or not.
Video Self-Stitching Graph Network for Temporal Action Localization
Authors: Chen Zhao, Ali Thabet, Bernard Ghanem
Date: 2020-11-30 00:00:00
Proceeding: ICCV 2021 10
Tasks: Action Localization, Temporal Action Localization

Temporal action localization (TAL) in videos is a challenging task, especially due to the large variation in action temporal scales. Short actions usually occupy a major proportion in the datasets, but tend to have the lowest performance. In this paper, we confront the challenge of short actions and propose a multi-level cross-scale solution dubbed as video self-stitching graph network (VSGN). We have two key components in VSGN: video self-stitching (VSS) and cross-scale graph pyramid network (xGPN). In VSS, we focus on a short period of a video and magnify it along the temporal dimension to obtain a larger scale. We stitch the original clip and its magnified counterpart in one input sequence to take advantage of the complementary properties of both scales. The xGPN component further exploits the cross-scale correlations by a pyramid of cross-scale graph networks, each containing a hybrid module to aggregate features from across scales as well as within the same scale. Our VSGN not only enhances the feature representations, but also generates more positive anchors for short actions and more short training samples. Experiments demonstrate that VSGN obviously improves the localization performance of short actions as well as achieving the state-of-the-art overall performance on THUMOS-14 and ActivityNet-v1.3.
A CRF-based Framework for Tracklet Inactivation in Online Multi-Object Tracking
Authors: Tianze Gao, Huihui Pan, Zidong Wang, Huijun Gao
Date: 2020-11-30 00:00:00
Tasks: Multi-Object Tracking, Object Tracking, Online Multi-Object Tracking

Online multi-object tracking (MOT) is an active research topic in the domain of computer vision. Although many previously proposed algorithms have exhibited decent results, the issue of tracklet inactivation has not been sufficiently studied. Simple strategies such as using a fixed threshold on classification scores are adopted, yielding undesirable tracking mistakes and limiting the overall performance. In this paper, a conditional random field (CRF) based framework is put forward to tackle the tracklet inactivation issue in online MOT problems. A discrete CRF which exploits the intra-frame relationship between tracking hypotheses is developed to improve the robustness of tracklet inactivation. Separate sets of feature functions are designed for the unary and binary terms in the CRF, which take into account various tracking challenges in practical scenarios. To handle the problem of varying CRF nodes in the MOT context, two strategies named as hypothesis filtering and dummy nodes are employed. In the proposed framework, the inference stage is conducted by using the loopy belief propagation algorithm, and the CRF parameters are determined by utilizing the maximum likelihood estimation method followed by slight manual adjustment. Experimental results show that the tracker combined with the CRF-based framework outperforms the baseline on the MOT16 and MOT17 benchmarks. The extensibility of the proposed framework is further validated by an extensive experiment.
Incremental Learning via Rate Reduction
Authors: Ziyang Wu, Christina Baek, Chong You, Yi Ma
Date: 2020-11-30 00:00:00
Proceeding: CVPR 2021 1
Tasks: Deep Learning, Incremental Learning

Current deep learning architectures suffer from catastrophic forgetting, a failure to retain knowledge of previously learned classes when incrementally trained on new classes. The fundamental roadblock faced by deep learning methods is that deep learning models are optimized as "black boxes," making it difficult to properly adjust the model parameters to preserve knowledge about previously seen data. To overcome the problem of catastrophic forgetting, we propose utilizing an alternative "white box" architecture derived from the principle of rate reduction, where each layer of the network is explicitly computed without back propagation. Under this paradigm, we demonstrate that, given a pre-trained network and new data classes, our approach can provably construct a new network that emulates joint training with all past and new classes. Finally, our experiments show that our proposed learning algorithm observes significantly less decay in classification performance, outperforming state of the art methods on MNIST and CIFAR-10 by a large margin and justifying the use of "white box" algorithms for incremental learning even for sufficiently complex image data.
Monocular 3D Object Detection with Sequential Feature Association and Depth Hint Augmentation
Authors: Tianze Gao, Huihui Pan, Huijun Gao
Date: 2020-11-30 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, Depth Estimation, Monocular 3D Object Detection, Object, object-detection, Object Detection

Monocular 3D object detection, with the aim of predicting the geometric properties of on-road objects, is a promising research topic for the intelligent perception systems of autonomous driving. Most state-of-the-art methods follow a keypoint-based paradigm, where the keypoints of objects are predicted and employed as the basis for regressing the other geometric properties. In this work, a unified network named as FADNet is presented to address the task of monocular 3D object detection. In contrast to previous keypoint-based methods, we propose to divide the output modalities into different groups according to the estimation difficulty of object properties. Different groups are treated differently and sequentially associated by a convolutional Gated Recurrent Unit. Another contribution of this work is the strategy of depth hint augmentation. To provide characterized depth patterns as hints for depth estimation, a dedicated depth hint module is designed to generate row-wise features named as depth hints, which are explicitly supervised in a bin-wise manner. The contributions of this work are validated by conducting experiments and ablation study on the KITTI benchmark. Without utilizing depth priors, post optimization, or other refinement modules, our network performs competitively against state-of-the-art methods while maintaining a decent running speed.
FactorizeNet: Progressive Depth Factorization for Efficient Network Architecture Exploration Under Quantization Constraints
Authors: Stone Yun, Alexander Wong
Date: 2020-11-30 00:00:00
Tasks: Quantization

Depth factorization and quantization have emerged as two of the principal strategies for designing efficient deep convolutional neural network (CNN) architectures tailored for low-power inference on the edge. However, there is still little detailed understanding of how different depth factorization choices affect the final, trained distributions of each layer in a CNN, particularly in the situation of quantized weights and activations. In this study, we introduce a progressive depth factorization strategy for efficient CNN architecture exploration under quantization constraints. By algorithmically increasing the granularity of depth factorization in a progressive manner, the proposed strategy enables a fine-grained, low-level analysis of layer-wise distributions. Thus enabling the gain of in-depth, layer-level insights on efficiency-accuracy tradeoffs under fixed-precision quantization. Such a progressive depth factorization strategy also enables efficient identification of the optimal depth-factorized macroarchitecture design (which we will refer to here as FactorizeNet) based on the desired efficiency-accuracy requirements.
Just One Moment: Structural Vulnerability of Deep Action Recognition against One Frame Attack
Authors: Jaehui Hwang, Jun-Hyuk Kim, Jun-Ho Choi, Jong-Seok Lee
Date: 2020-11-30 00:00:00
Proceeding: ICCV 2021 10
Tasks: Action Recognition, Adversarial Attack

The video-based action recognition task has been extensively studied in recent years. In this paper, we study the structural vulnerability of deep learning-based action recognition models against the adversarial attack using the one frame attack that adds an inconspicuous perturbation to only a single frame of a given video clip. Our analysis shows that the models are highly vulnerable against the one frame attack due to their structural properties. Experiments demonstrate high fooling rates and inconspicuous characteristics of the attack. Furthermore, we show that strong universal one frame perturbations can be obtained under various scenarios. Our work raises the serious issue of adversarial vulnerability of the state-of-the-art action recognition models in various perspectives.
ScaleNAS: One-Shot Learning of Scale-Aware Representations for Visual Recognition
Authors: Hsin-Pai Cheng, Feng Liang, Meng Li, Bowen Cheng, Feng Yan, Hai Li, Vikas Chandra, Yiran Chen
Date: 2020-11-30 00:00:00
Tasks: Multi-Person Pose Estimation, Neural Architecture Search, One-Shot Learning, Pose Estimation, Semantic Segmentation

Scale variance among different sizes of body parts and objects is a challenging problem for visual recognition tasks. Existing works usually design dedicated backbone or apply Neural architecture Search(NAS) for each task to tackle this challenge. However, existing works impose significant limitations on the design or search space. To solve these problems, we present ScaleNAS, a one-shot learning method for exploring scale-aware representations. ScaleNAS solves multiple tasks at a time by searching multi-scale feature aggregation. ScaleNAS adopts a flexible search space that allows an arbitrary number of blocks and cross-scale feature fusions. To cope with the high search cost incurred by the flexible space, ScaleNAS employs one-shot learning for multi-scale supernet driven by grouped sampling and evolutionary search. Without further retraining, ScaleNet can be directly deployed for different visual recognition tasks with superior performance. We use ScaleNAS to create high-resolution models for two different tasks, ScaleNet-P for human pose estimation and ScaleNet-S for semantic segmentation. ScaleNet-P and ScaleNet-S outperform existing manually crafted and NAS-based methods in both tasks. When applying ScaleNet-P to bottom-up human pose estimation, it surpasses the state-of-the-art HigherHRNet. In particular, ScaleNet-P4 achieves 71.6% AP on COCO test-dev, achieving new state-of-the-art result.
Robust and Private Learning of Halfspaces
Authors: Badih Ghazi, Ravi Kumar, Pasin Manurangsi, Thao Nguyen
Date: 2020-11-30 00:00:00
Tasks: Adversarial Robustness

In this work, we study the trade-off between differential privacy and adversarial robustness under L2-perturbations in the context of learning halfspaces. We prove nearly tight bounds on the sample complexity of robust private learning of halfspaces for a large regime of parameters. A highlight of our results is that robust and private learning is harder than robust or private learning alone. We complement our theoretical analysis with experimental results on the MNIST and USPS datasets, for a learning algorithm that is both differentially private and adversarially robust.
End-to-End 3D Point Cloud Learning for Registration Task Using Virtual Correspondences
Authors: Zhijian Qiao, Huanshu Wei, Zhe Liu, Chuanzhe Suo, Hesheng Wang
Date: 2020-11-30 00:00:00
Tasks: Point Cloud Registration

3D Point cloud registration is still a very challenging topic due to the difficulty in finding the rigid transformation between two point clouds with partial correspondences, and it's even harder in the absence of any initial estimation information. In this paper, we present an end-to-end deep-learning based approach to resolve the point cloud registration problem. Firstly, the revised LPD-Net is introduced to extract features and aggregate them with the graph network. Secondly, the self-attention mechanism is utilized to enhance the structure information in the point cloud and the cross-attention mechanism is designed to enhance the corresponding information between the two input point clouds. Based on which, the virtual corresponding points can be generated by a soft pointer based method, and finally, the point cloud registration problem can be solved by implementing the SVD method. Comparison results in ModelNet40 dataset validate that the proposed approach reaches the state-of-the-art in point cloud registration tasks and experiment resutls in KITTI dataset validate the effectiveness of the proposed approach in real applications.Our source code is available at \url{https://github.com/qiaozhijian/VCR-Net.git}
Where Should We Begin? A Low-Level Exploration of Weight Initialization Impact on Quantized Behaviour of Deep Neural Networks
Authors: Stone Yun, Alexander Wong
Date: 2020-11-30 00:00:00
Tasks: Quantization

With the proliferation of deep convolutional neural network (CNN) algorithms for mobile processing, limited precision quantization has become an essential tool for CNN efficiency. Consequently, various works have sought to design fixed precision quantization algorithms and quantization-focused optimization techniques that minimize quantization induced performance degradation. However, there is little concrete understanding of how various CNN design decisions/best practices affect quantized inference behaviour. Weight initialization strategies are often associated with solving issues such as vanishing/exploding gradients but an often-overlooked aspect is their impact on the final trained distributions of each layer. We present an in-depth, fine-grained ablation study of the effect of different weights initializations on the final distributions of weights and activations of different CNN architectures. The fine-grained, layerwise analysis enables us to gain deep insights on how initial weights distributions will affect final accuracy and quantized behaviour. To our best knowledge, we are the first to perform such a low-level, in-depth quantitative analysis of weights initialization and its effect on quantized behaviour.
Gradient Sparsification Can Improve Performance of Differentially-Private Convex Machine Learning
Authors: Farhad Farokhi
Date: 2020-11-30 00:00:00
Tasks: BIG-bench Machine Learning, compressed sensing, Privacy Preserving

We use gradient sparsification to reduce the adverse effect of differential privacy noise on performance of private machine learning models. To this aim, we employ compressed sensing and additive Laplace noise to evaluate differentially-private gradients. Noisy privacy-preserving gradients are used to perform stochastic gradient descent for training machine learning models. Sparsification, achieved by setting the smallest gradient entries to zero, can reduce the convergence speed of the training algorithm. However, by sparsification and compressed sensing, the dimension of communicated gradient and the magnitude of additive noise can be reduced. The interplay between these effects determines whether gradient sparsification improves the performance of differentially-private machine learning models. We investigate this analytically in the paper. We prove that, for small privacy budgets, compression can improve performance of privacy-preserving machine learning models. However, for large privacy budgets, compression does not necessarily improve the performance. Intuitively, this is because the effect of privacy-preserving noise is minimal in large privacy budget regime and thus improvements from gradient sparsification cannot compensate for its slower convergence.
Deep Implicit Templates for 3D Shape Representation
Authors: Zerong Zheng, Tao Yu, Qionghai Dai, Yebin Liu
Date: 2020-11-30 00:00:00
Proceeding: CVPR 2021 1
Tasks: 3D Shape Representation

Deep implicit functions (DIFs), as a kind of 3D shape representation, are becoming more and more popular in the 3D vision community due to their compactness and strong representation power. However, unlike polygon mesh-based templates, it remains a challenge to reason dense correspondences or other semantic relationships across shapes represented by DIFs, which limits its applications in texture transfer, shape analysis and so on. To overcome this limitation and also make DIFs more interpretable, we propose Deep Implicit Templates, a new 3D shape representation that supports explicit correspondence reasoning in deep implicit representations. Our key idea is to formulate DIFs as conditional deformations of a template implicit function. To this end, we propose Spatial Warping LSTM, which decomposes the conditional spatial transformation into multiple affine transformations and guarantees generalization capability. Moreover, the training loss is carefully designed in order to achieve high reconstruction accuracy while learning a plausible template with accurate correspondences in an unsupervised manner. Experiments show that our method can not only learn a common implicit template for a collection of shapes, but also establish dense correspondences across all the shapes simultaneously without any supervision.
Learnable Motion Coherence for Correspondence Pruning
Authors: YuAn Liu, Lingjie Liu, Cheng Lin, Zhen Dong, Wenping Wang
Date: 2020-11-30 00:00:00
Proceeding: CVPR 2021 1
Tasks: Camera Pose Estimation, Pose Estimation

Motion coherence is an important clue for distinguishing true correspondences from false ones. Modeling motion coherence on sparse putative correspondences is challenging due to their sparsity and uneven distributions. Existing works on motion coherence are sensitive to parameter settings and have difficulty in dealing with complex motion patterns. In this paper, we introduce a network called Laplacian Motion Coherence Network (LMCNet) to learn motion coherence property for correspondence pruning. We propose a novel formulation of fitting coherent motions with a smooth function on a graph of correspondences and show that this formulation allows a closed-form solution by graph Laplacian. This closed-form solution enables us to design a differentiable layer in a learning framework to capture global motion coherence from putative correspondences. The global motion coherence is further combined with local coherence extracted by another local layer to robustly detect inlier correspondences. Experiments demonstrate that LMCNet has superior performances to the state of the art in relative camera pose estimation and correspondences pruning of dynamic scenes.
A Selective Survey on Versatile Knowledge Distillation Paradigm for Neural Network Models
Authors: Jeong-Hoe Ku, Jihun Oh, YoungYoon Lee, Gaurav Pooniwala, SangJeong Lee
Date: 2020-11-30 00:00:00
Tasks: Knowledge Distillation, Model Compression, Self-Supervised Learning, Survey

This paper aims to provide a selective survey about knowledge distillation(KD) framework for researchers and practitioners to take advantage of it for developing new optimized models in the deep neural network field. To this end, we give a brief overview of knowledge distillation and some related works including learning using privileged information(LUPI) and generalized distillation(GD). Even though knowledge distillation based on the teacher-student architecture was initially devised as a model compression technique, it has found versatile applications over various frameworks. In this paper, we review the characteristics of knowledge distillation from the hypothesis that the three important ingredients of knowledge distillation are distilled knowledge and loss,teacher-student paradigm, and the distillation process. In addition, we survey the versatility of the knowledge distillation by studying its direct applications and its usage in combination with other deep learning paradigms. Finally we present some future works in knowledge distillation including explainable knowledge distillation where the analytical analysis of the performance gain is studied and the self-supervised learning which is a hot research topic in deep learning community.