pwc-archive/papers-with-abstracts huggingface dataset:
Fine-Grained Vehicle Classification with Unsupervised Parts Co-occurrence Learning
Authors: Sara Elkerdawy, Nilanjan Ray, Hong Zhang
Date: 2019-01-23 00:00:00
Tasks: Classification, Fine-Grained Image Classification, Fine-Grained Vehicle Classification, General Classification, Transfer Learning

Vehicle fine-grained classification is a challenging research problem with little attention in the field. In this paper, we propose a deep network architecture for vehicles fine-grained classification without the need of parts or 3D bounding boxes annotation. Co-occurrence layer (COOC) layer is exploited for unsupervised parts discovery. In addition, a two-step procedure with transfer learning and fine-tuning is utilized. This enables us to better fine-tune models with pre-trained weights on ImageNet in some layers while having random initialization in some others. Our model achieves 86.5% accuracy outperforming the state of the art methods in BoxCars116K by 4%. In addition, we achieve 95.5% and 93.19% on CompCars on both train-test splits, 70-30 and 50-50, outperforming the other methods by 4.5% and 8% respectively.
Deep CNNs With Spatially Weighted Pooling for Fine-Grained Car Recognition
Authors: Qichang Hu, Huibing Wang, Teng Li, Chunhua Shen
Date: 2017-04-04 00:00:00
Tasks: Fine-Grained Image Classification

Fine-grained car recognition aims to recognize the category information of a car, such as car make, car model, or even the year of manufacture. A number of recent studies have shown that a deep convolutional neural network (DCNN) trained on a large-scale data set can achieve impressive results at a range of generic object classification tasks. In this paper, we propose a spatially weighted pooling (SWP) strategy, which considerably improves the robustness and effectiveness of the feature representation of most dominant DCNNs. More specifically, the SWP is a novel pooling layer, which contains a predefined number of spatially weighted masks or pooling channels. The SWP pools the extracted features of DCNNs with the guidance of its learnt masks, which measures the importance of the spatial units in terms of discriminative power. As the existing methods that apply uniform grid pooling on the convolutional feature maps of DCNNs, the proposed method can extract the convolutional features and generate the pooling channels from a single DCNN. Thus minimal modification is needed in terms of implementation. Moreover, the parameters of the SWP layer can be learned in the end-to-end training process of the DCNN. By applying our method to several fine-grained car recognition data sets, we demonstrate that the proposed method can achieve better performance than recent approaches in the literature. We advance the state-of-the-art results by improving the accuracy from 92.6% to 93.1% on the Stanford Cars-196 data set and 91.2% to 97.6% on the recent CompCars data set. We have also tested the proposed method on two additional large-scale data sets with impressive results observed.
Detecting earthquakes: a novel deep learning-based approach for effective disaster response
Authors: Shakeel Muhammad, Katsutoshi Itoyama, Kenji Nishida, Kazuhiro Nakadai
Date: 2021-04-01 00:00:00
Tasks: Disaster Response, Specificity

In the present study, we present an intelligent earthquake signal detector that provides added assistance to automate traditional disaster responses. To effectively respond in a crisis scenario, additional sensors and automation are always necessary. Deep learning has achieved success in various low signal-to-noise ratio tasks, which motivated us to propose a novel 3-dimensional (3D) CNN-RNN-based earthquake detector from a demonstration paradigm to real-time implementation. Data taken from the STanford EArthquake Dataset (STEAD) are used to train the network. After preprocessing the raw earthquake signals, features such as log-mel spectrograms are extracted. Once the model has learned spatial and temporal information from low-frequency earthquake waves, it can be employed in real time to distinguish small and large earthquakes from seismic noise with an accuracy, sensitivity, and specificity of 99.057%, 98.488%, and 99.621%, respectively. We also observe that the choice of filters in log-mel spectrogram impacts the results much more than the model complexity. Furthermore, we implement and test the model on data collected continuously over two months by a personal seismometer in the laboratory. The inference speed for a single prediction is 2.27 seconds, and the system delivers a stable detection of all 63 major earthquakes from November 2019 to December 2019 reported by the Japan Meteorological Agency.
Using deep convolutional neural networks for neonatal brain image segmentation
Authors: Ding Y, Acosta R, Enguix V, Suffren S, Ortmann J, Luck D, Dolz J, Lodygensky GA.
Date: 2020-03-26 00:00:00
Tasks: Brain Image Segmentation, Image Segmentation, Semantic Segmentation

Introduction: Deep learning neural networks are especially potent at dealing with structured data, such as images and volumes. Both modified LiviaNET and HyperDense-Net performed well at a prior competition segmenting 6-month-old infant magnetic resonance images, but neonatal cerebral tissue type identification is challenging given its uniquely inverted tissue contrasts. The current study aims to evaluate the two architectures to segment neonatal brain tissue types at term equivalent age.

Methods: Both networks were retrained over 24 pairs of neonatal T1 and T2 data from the Developing Human Connectome Project public data set and validated on another eight pairs against ground truth. We then reported the best-performing model from training and its performance by computing the Dice similarity coefficient (DSC) for each tissue type against eight test subjects.

Results: During the testing phase, among the segmentation approaches tested, the dual-modality HyperDense-Net achieved the best statistically significantly test mean DSC values, obtaining 0.94/0.95/0.92 for the tissue types and took 80 h to train and 10 min to segment, including preprocessing. The single-modality LiviaNET was better at processing T2-weighted images than processing T1-weighted images across all tissue types, achieving mean DSC values of 0.90/0.90/0.88 for gray matter, white matter, and cerebrospinal fluid, respectively, while requiring 30 h to train and 8 min to segment each brain, including preprocessing.

Discussion: Our evaluation demonstrates that both neural networks can segment neonatal brains, achieving previously reported performance. Both networks will be continuously retrained over an increasingly larger repertoire of neonatal brain data and be made available through the Canadian Neonatal Brain Platform to better serve the neonatal brain imaging research community.
No Gestures Left Behind: Learning Relationships between Spoken Language and Freeform Gestures
Authors: Chaitanya Ahuja, Dong Won Lee, Ryo Ishii, Louis-Philippe Morency
Date: 2020-10-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Gesture Generation

We study relationships between spoken language and co-speech gestures in context of two key challenges. First, distributions of text and gestures are inherently skewed making it important to model the long tail. Second, gesture predictions are made at a subword level, making it important to learn relationships between language and audio. We introduce Adversarial Importance Sampled Learning, which combines adversarial learning with importance sampling to strike a balance between precision and coverage. We substantiate the effectiveness of our approach through large-scale quantitative and user studies, which show that our proposed methodology significantly outperforms previous stateof-the-art approaches for gesture generation.
The art of defense: letting networks fool the attacker
Authors: Jinlai Zhang, Yinpeng Dong, Binbin Liu, Bo Ouyang, Jihong Zhu, Minchi Kuang, Houqing Wang, Yanmei Meng
Date: 2021-04-07 00:00:00
Tasks: Adversarial Defense

Robust environment perception is critical for autonomous cars, and adversarial defenses are the most effective and widely studied ways to improve the robustness of environment perception. However, all of previous defense methods decrease the natural accuracy, and the nature of the DNNs itself has been overlooked. To this end, in this paper, we propose a novel adversarial defense for 3D point cloud classifier that makes full use of the nature of the DNNs. Due to the disorder of point cloud, all point cloud classifiers have the property of permutation invariant to the input point cloud. Based on this nature, we design invariant transformations defense (IT-Defense). We show that, even after accounting for obfuscated gradients, our IT-Defense is a resilient defense against state-of-the-art (SOTA) 3D attacks. Moreover, IT-Defense do not hurt clean accuracy compared to previous SOTA 3D defenses. Our code is available at: {\footnotesize{\url{https://github.com/cuge1995/IT-Defense}}}.
ARC: A Vision-based Automatic Retail Checkout System
Authors: Syed Talha Bukhari, Abdul Wahab Amin, Muhammad Abdullah Naveed, Muhammad Rzi Abbas
Date: 2021-04-07 00:00:00
Tasks: ARC

Retail checkout systems employed at supermarkets primarily rely on barcode scanners, with some utilizing QR codes, to identify the items being purchased. These methods are time-consuming in practice, require a certain level of human supervision, and involve waiting in long queues. In this regard, we propose a system, that we call ARC, which aims at making the process of check-out at retail store counters faster, autonomous, and more convenient, while reducing dependency on a human operator. The approach makes use of a computer vision-based system, with a Convolutional Neural Network at its core, which scans objects placed beneath a webcam for identification. To evaluate the proposed system, we curated an image dataset of one-hundred local retail items of various categories. Within the given assumptions and considerations, the system achieves a reasonable test-time accuracy, pointing towards an ambitious future for the proposed setup. The project code and the dataset are made publicly available.
TenSEAL: A Library for Encrypted Tensor Operations Using Homomorphic Encryption
Authors: Ayoub Benaissa, Bilal Retiat, Bogdan Cebere, Alaa Eddine Belfedhal
Date: 2021-04-07 00:00:00
Tasks: BIG-bench Machine Learning, Privacy Preserving

Machine learning algorithms have achieved remarkable results and are widely applied in a variety of domains. These algorithms often rely on sensitive and private data such as medical and financial records. Therefore, it is vital to draw further attention regarding privacy threats and corresponding defensive techniques applied to machine learning models. In this paper, we present TenSEAL, an open-source library for Privacy-Preserving Machine Learning using Homomorphic Encryption that can be easily integrated within popular machine learning frameworks. We benchmark our implementation using MNIST and show that an encrypted convolutional neural network can be evaluated in less than a second, using less than half a megabyte of communication.
Self-Supervised Learning for Gastritis Detection with Gastric X-ray Images
Authors: Guang Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
Date: 2021-04-07 00:00:00
Tasks: Self-Supervised Learning, Specificity

Purpose: Manual annotation of gastric X-ray images by doctors for gastritis detection is time-consuming and expensive. To solve this, a self-supervised learning method is developed in this study. The effectiveness of the proposed self-supervised learning method in gastritis detection is verified using a few annotated gastric X-ray images. Methods: In this study, we develop a novel method that can perform explicit self-supervised learning and learn discriminative representations from gastric X-ray images. Models trained based on the proposed method were fine-tuned on datasets comprising a few annotated gastric X-ray images. Five self-supervised learning methods, i.e., SimSiam, BYOL, PIRL-jigsaw, PIRL-rotation, and SimCLR, were compared with the proposed method. Furthermore, three previous methods, one pretrained on ImageNet, one trained from scratch, and one semi-supervised learning method, were compared with the proposed method. Results: The proposed method's harmonic mean score of sensitivity and specificity after fine-tuning with the annotated data of 10, 20, 30, and 40 patients were 0.875, 0.911, 0.915, and 0.931, respectively. The proposed method outperformed all comparative methods, including the five self-supervised learning and three previous methods. Experimental results showed the effectiveness of the proposed method in gastritis detection using a few annotated gastric X-ray images. Conclusions: This paper proposes a novel self-supervised learning method based on a teacher-student architecture for gastritis detection using gastric X-ray images. The proposed method can perform explicit self-supervised learning and learn discriminative representations from gastric X-ray images. The proposed method exhibits potential clinical use in gastritis detection using a few annotated gastric X-ray images.
Soft-Label Anonymous Gastric X-ray Image Distillation
Authors: Guang Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
Date: 2021-04-07 00:00:00
Tasks: valid

This paper presents a soft-label anonymous gastric X-ray image distillation method based on a gradient descent approach. The sharing of medical data is demanded to construct high-accuracy computer-aided diagnosis (CAD) systems. However, the large size of the medical dataset and privacy protection are remaining problems in medical data sharing, which hindered the research of CAD systems. The idea of our distillation method is to extract the valid information of the medical dataset and generate a tiny distilled dataset that has a different data distribution. Different from model distillation, our method aims to find the optimal distilled images, distilled labels and the optimized learning rate. Experimental results show that the proposed method can not only effectively compress the medical dataset but also anonymize medical images to protect the patient's private information. The proposed approach can improve the efficiency and security of medical data sharing.
TSception: Capturing Temporal Dynamics and Spatial Asymmetry from EEG for Emotion Recognition
Authors: Yi Ding, Neethu Robinson, Su Zhang, Qiuhao Zeng, Cuntai Guan
Date: 2021-04-07 00:00:00
Tasks: EEG, Emotion Recognition

The high temporal resolution and the asymmetric spatial activations are essential attributes of electroencephalogram (EEG) underlying emotional processes in the brain. To learn the temporal dynamics and spatial asymmetry of EEG towards accurate and generalized emotion recognition, we propose TSception, a multi-scale convolutional neural network that can classify emotions from EEG. TSception consists of dynamic temporal, asymmetric spatial, and high-level fusion layers, which learn discriminative representations in the time and channel dimensions simultaneously. The dynamic temporal layer consists of multi-scale 1D convolutional kernels whose lengths are related to the sampling rate of EEG, which learns the dynamic temporal and frequency representations of EEG. The asymmetric spatial layer takes advantage of the asymmetric EEG patterns for emotion, learning the discriminative global and hemisphere representations. The learned spatial representations will be fused by a high-level fusion layer. Using more generalized cross-validation settings, the proposed method is evaluated on two publicly available datasets DEAP and MAHNOB-HCI. The performance of the proposed network is compared with prior reported methods such as SVM, KNN, FBFgMDM, FBTSC, Unsupervised learning, DeepConvNet, ShallowConvNet, and EEGNet. TSception achieves higher classification accuracies and F1 scores than other methods in most of the experiments. The codes are available at https://github.com/yi-ding-cs/TSception
Optimizing Memory Efficiency of Graph Neural Networks on Edge Computing Platforms
Authors: Ao Zhou, Jianlei Yang, Yeqi Gao, Tong Qiao, Yingjie Qi, Xiaoyi Wang, Yunli Chen, Pengcheng Dai, Weisheng Zhao, Chunming Hu
Date: 2021-04-07 00:00:00
Tasks: Edge-computing

Graph neural networks (GNN) have achieved state-of-the-art performance on various industrial tasks. However, the poor efficiency of GNN inference and frequent Out-Of-Memory (OOM) problem limit the successful application of GNN on edge computing platforms. To tackle these problems, a feature decomposition approach is proposed for memory efficiency optimization of GNN inference. The proposed approach could achieve outstanding optimization on various GNN models, covering a wide range of datasets, which speeds up the inference by up to 3x. Furthermore, the proposed feature decomposition could significantly reduce the peak memory usage (up to 5x in memory efficiency improvement) and mitigate OOM problems during GNN inference.
PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics
Authors: Zhiao Huang, Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua B. Tenenbaum, Chuang Gan
Date: 2021-04-07 00:00:00
Proceeding: ICLR 2021 1
Tasks: Reinforcement Learning (RL)

Simulated virtual environments serve as one of the main driving forces behind developing and evaluating skill learning algorithms. However, existing environments typically only simulate rigid body physics. Additionally, the simulation process usually does not provide gradients that might be useful for planning and control optimizations. We introduce a new differentiable physics benchmark called PasticineLab, which includes a diverse collection of soft body manipulation tasks. In each task, the agent uses manipulators to deform the plasticine into the desired configuration. The underlying physics engine supports differentiable elastic and plastic deformation using the DiffTaichi system, posing many under-explored challenges to robotic agents. We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark. Experimental results suggest that 1) RL-based approaches struggle to solve most of the tasks efficiently; 2) gradient-based approaches, by optimizing open-loop control sequences with the built-in differentiable physics engine, can rapidly find a solution within tens of iterations, but still fall short on multi-stage tasks that require long-term planning. We expect that PlasticineLab will encourage the development of novel algorithms that combine differentiable physics and RL for more complex physics-based skill learning tasks.
Streaming Self-Training via Domain-Agnostic Unlabeled Images
Authors: Zhiqiu Lin, Deva Ramanan, Aayush Bansal
Date: 2021-04-07 00:00:00
Tasks: Fine-Grained Image Classification, image-classification, Image Classification, Semantic Segmentation, Surface Normal Estimation

We present streaming self-training (SST) that aims to democratize the process of learning visual recognition models such that a non-expert user can define a new task depending on their needs via a few labeled examples and minimal domain knowledge. Key to SST are two crucial observations: (1) domain-agnostic unlabeled images enable us to learn better models with a few labeled examples without any additional knowledge or supervision; and (2) learning is a continuous process and can be done by constructing a schedule of learning updates that iterates between pre-training on novel segments of the streams of unlabeled data, and fine-tuning on the small and fixed labeled dataset. This allows SST to overcome the need for a large number of domain-specific labeled and unlabeled examples, exorbitant computational resources, and domain/task-specific knowledge. In this setting, classical semi-supervised approaches require a large amount of domain-specific labeled and unlabeled examples, immense resources to process data, and expert knowledge of a particular task. Due to these reasons, semi-supervised learning has been restricted to a few places that can house required computational and human resources. In this work, we overcome these challenges and demonstrate our findings for a wide range of visual recognition tasks including fine-grained image classification, surface normal estimation, and semantic segmentation. We also demonstrate our findings for diverse domains including medical, satellite, and agricultural imagery, where there does not exist a large amount of labeled or unlabeled data.
Modern Hopfield Networks for Few- and Zero-Shot Reaction Template Prediction
Authors: Philipp Seidl, Philipp Renz, Natalia Dyubankova, Paulo Neves, Jonas Verhoeven, Marwin Segler, Jörg K. Wegner, Sepp Hochreiter, Günter Klambauer
Date: 2021-04-07 00:00:00
Tasks: Retrosynthesis, Single-step retrosynthesis, Zero-Shot Learning

Finding synthesis routes for molecules of interest is an essential step in the discovery of new drugs and materials. To find such routes, computer-assisted synthesis planning (CASP) methods are employed which rely on a model of chemical reactivity. In this study, we model single-step retrosynthesis in a template-based approach using modern Hopfield networks (MHNs). We adapt MHNs to associate different modalities, reaction templates and molecules, which allows the model to leverage structural information about reaction templates. This approach significantly improves the performance of template relevance prediction, especially for templates with few or zero training examples. With inference speed several times faster than that of baseline methods, we improve predictive performance for top-k exact match accuracy for $\mathrm{k}\geq5$ in the retrosynthesis benchmark USPTO-50k.
Leaving Goals on the Pitch: Evaluating Decision Making in Soccer
Authors: Maaike Van Roy, Pieter Robberechts, Wen-Chi Yang, Luc De Raedt, Jesse Davis
Date: 2021-04-07 00:00:00
Tasks: counterfactual, Decision Making

Analysis of the popular expected goals (xG) metric in soccer has determined that a (slightly) smaller number of high-quality attempts will likely yield more goals than a slew of low-quality ones. This observation has driven a change in shooting behavior. Teams are passing up on shots from outside the penalty box, in the hopes of generating a better shot closer to goal later on. This paper evaluates whether this decrease in long-distance shots is warranted. Therefore, we propose a novel generic framework to reason about decision-making in soccer by combining techniques from machine learning and artificial intelligence (AI). First, we model how a team has behaved offensively over the course of two seasons by learning a Markov Decision Process (MDP) from event stream data. Second, we use reasoning techniques arising from the AI literature on verification to each team's MDP. This allows us to reason about the efficacy of certain potential decisions by posing counterfactual questions to the MDP. Our key conclusion is that teams would score more goals if they shot more often from outside the penalty box in a small number of team-specific locations. The proposed framework can easily be extended and applied to analyze other aspects of the game.