pwc-archive/papers-with-abstracts huggingface dataset:
D3DLO: Deep 3D LiDAR Odometry
Authors: Philipp Adis, Nicolas Horst, Mathias Wien
Date: 2021-01-28 00:00:00

LiDAR odometry (LO) describes the task of finding an alignment of subsequent LiDAR point clouds. This alignment can be used to estimate the motion of the platform where the LiDAR sensor is mounted on. Currently, on the well-known KITTI Vision Benchmark Suite state-of-the-art algorithms are non-learning approaches. We propose a network architecture that learns LO by directly processing 3D point clouds. It is trained on the KITTI dataset in an end-to-end manner without the necessity of pre-defining corresponding pairs of points. An evaluation on the KITTI Vision Benchmark Suite shows similar performance to a previously published work, DeepCLR [1], even though our model uses only around 3.56% of the number of network parameters thereof. Furthermore, a plane point extraction is applied which leads to a marginal performance decrease while simultaneously reducing the input size by up to 50%.
Position, Padding and Predictions: A Deeper Look at Position Information in CNNs
Authors: Md Amirul Islam, Matthew Kowal, Sen Jia, Konstantinos G. Derpanis, Neil D. B. Bruce
Date: 2021-01-28 00:00:00
Tasks: Position

In contrast to fully connected networks, Convolutional Neural Networks (CNNs) achieve efficiency by learning weights associated with local filters with a finite spatial extent. An implication of this is that a filter may know what it is looking at, but not where it is positioned in the image. In this paper, we first test this hypothesis and reveal that a surprising degree of absolute position information is encoded in commonly used CNNs. We show that zero padding drives CNNs to encode position information in their internal representations, while a lack of padding precludes position encoding. This gives rise to deeper questions about the role of position information in CNNs: (i) What boundary heuristics enable optimal position encoding for downstream tasks?; (ii) Does position encoding affect the learning of semantic representations?; (iii) Does position encoding always improve performance? To provide answers, we perform the largest case study to date on the role that padding and border heuristics play in CNNs. We design novel tasks which allow us to quantify boundary effects as a function of the distance to the border. Numerous semantic objectives reveal the effect of the border on semantic representations. Finally, we demonstrate the implications of these findings on multiple real-world tasks to show that position information can both help or hurt performance.
NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation
Authors: Angtian Wang, Adam Kortylewski, Alan Yuille
Date: 2021-01-29 00:00:00
Proceeding: ICLR 2021 1
Tasks: 3D geometry, 3D Pose Estimation, Contrastive Learning, Pose Estimation

3D pose estimation is a challenging but important task in computer vision. In this work, we show that standard deep learning approaches to 3D pose estimation are not robust when objects are partially occluded or viewed from a previously unseen pose. Inspired by the robustness of generative vision models to partial occlusion, we propose to integrate deep neural networks with 3D generative representations of objects into a unified neural architecture that we term NeMo. In particular, NeMo learns a generative model of neural feature activations at each vertex on a dense 3D mesh. Using differentiable rendering we estimate the 3D object pose by minimizing the reconstruction error between NeMo and the feature representation of the target image. To avoid local optima in the reconstruction loss, we train the feature extractor to maximize the distance between the individual feature representations on the mesh using contrastive learning. Our extensive experiments on PASCAL3D+, occluded-PASCAL3D+ and ObjectNet3D show that NeMo is much more robust to partial occlusion and unseen pose compared to standard deep networks, while retaining competitive performance on regular data. Interestingly, our experiments also show that NeMo performs reasonably well even when the mesh representation only crudely approximates the true object geometry with a cuboid, hence revealing that the detailed 3D geometry is not needed for accurate 3D pose estimation. The code is publicly available at https://github.com/Angtian/NeMo.
[Re] Learning Memory Guided Normality for Anomaly Detection
Authors: Kevin Stephen, Varun Menon
Date: 2021-01-29 00:00:00
Proceeding: RC 2020
Tasks: Anomaly Detection, Hyperparameter Optimization, Unsupervised Anomaly Detection

Scope of Reproducibility 

The authors have introduced a novel method for unsupervised anomaly detection that utilises a newly introduced “Memory Module” in their paper [1]. We validate the authorsʼ claim that this helps improve performance by helping the network learn prototypical patterns, and uses the learnt memory to reduce the representation capacity of Convolutional Neural Networks. Further, we validate the efficacy of two losses introduced by the authors, “Separateness Loss” and “Compactness Loss” presented to increase the discriminative power of the memory items and the deeply learned features. We test the efficacy with the help of t-SNE plots of the memory items. 

Methodology 

The authors provide a codebase available at https://github.com/cvlab-yonsei/MNAD with scripts for the Prediction task. We reused the available code to build scripts for the Reconstruction task and variants with and without memory. The completed codebase for all tasks available at https://github.com/alchemi5t/MNADrc. 

Results 

We obtain results that are within a 3% range of the reported results for all datasets other than on the ShanghaiTech dataset[2]. The anomalous run stuck out since the ”Non Memory” of the same run could score markedly better than the ”Memory” variant. This led us to investigate the behaviour of the memory module and found valuable insights which are presented in Section 5. 

What was easy 

• The authors provide codes to run the Prediction task with memory. 
• The ideas presented in the paper were clear and easy to understand. 
• The datasets were easily available, with the exception of the ShanghaiTech dataset[2].

What was difficult 

• After training the model on the ShanghaiTech dataset[2], we found that the “Memory” variants of the model had issues as described in Sections 4 and 5. We investigated the behaviour of the memory module and introduced additional supervision to solve the problem. 
• We required hyperparameter optimization to hit the reported scores. 
• The paper also lacked information as to which parts of the pipeline were actually to be credited for the improvement in performance, as discussed in Section 5. 

Communication with original authors 

The codebase has scripts only for the Prediction task ”Memory” variant of the benchmarks. We reused portions of the codebase to create the other variants. We then validated our modifications with the authors. We also communicated the discrepancies we saw in parts of the results and the authors asked us to wait until they update their repository with the missing scripts.
Reliable COVID-19 Detection Using Chest X-ray Images
Authors: Aysen Degerli, Mete Ahishali, Serkan Kiranyaz, Muhammad E. H. Chowdhury, Moncef Gabbouj
Date: 2021-01-28 00:00:00
Tasks: COVID-19 Diagnosis, Specificity

Coronavirus disease 2019 (COVID-19) has emerged the need for computer-aided diagnosis with automatic, accurate, and fast algorithms. Recent studies have applied Machine Learning algorithms for COVID-19 diagnosis over chest X-ray (CXR) images. However, the data scarcity in these studies prevents a reliable evaluation with the potential of overfitting and limits the performance of deep networks. Moreover, these networks can discriminate COVID-19 pneumonia usually from healthy subjects only or occasionally, from limited pneumonia types. Thus, there is a need for a robust and accurate COVID-19 detector evaluated over a large CXR dataset. To address this need, in this study, we propose a reliable COVID-19 detection network: ReCovNet, which can discriminate COVID-19 pneumonia from 14 different thoracic diseases and healthy subjects. To accomplish this, we have compiled the largest COVID-19 CXR dataset: QaTa-COV19 with 124,616 images including 4603 COVID-19 samples. The proposed ReCovNet achieved a detection performance with 98.57% sensitivity and 99.77% specificity.
A Petri Dish for Histopathology Image Analysis
Authors: Jerry Wei, Arief Suriawinata, Bing Ren, Xiaoying Liu, Mikhail Lisovsky, Louis Vaickus, Charles Brown, Michael Baker, Naofumi Tomita, Lorenzo Torresani, Jason Wei, Saeed Hassanpour
Date: 2021-01-29 00:00:00
Tasks: Binary Classification, Natural Questions, Transfer Learning

With the rise of deep learning, there has been increased interest in using neural networks for histopathology image analysis, a field that investigates the properties of biopsy or resected specimens traditionally manually examined under a microscope by pathologists. However, challenges such as limited data, costly annotation, and processing high-resolution and variable-size images make it difficult to quickly iterate over model designs. Throughout scientific history, many significant research directions have leveraged small-scale experimental setups as petri dishes to efficiently evaluate exploratory ideas. In this paper, we introduce a minimalist histopathology image analysis dataset (MHIST), an analogous petri dish for histopathology image analysis. MHIST is a binary classification dataset of 3,152 fixed-size images of colorectal polyps, each with a gold-standard label determined by the majority vote of seven board-certified gastrointestinal pathologists and annotator agreement level. MHIST occupies less than 400 MB of disk space, and a ResNet-18 baseline can be trained to convergence on MHIST in just 6 minutes using 3.5 GB of memory on a NVIDIA RTX 3090. As example use cases, we use MHIST to study natural questions such as how dataset size, network depth, transfer learning, and high-disagreement examples affect model performance. By introducing MHIST, we hope to not only help facilitate the work of current histopathology imaging researchers, but also make the field more-accessible to the general community. Our dataset is available at https://bmirds.github.io/MHIST.
Differential Privacy Meets Federated Learning under Communication Constraints
Authors: Nima Mohammadi, Jianan Bai, Qiang Fan, Yifei Song, Yang Yi, Lingjia Liu
Date: 2021-01-28 00:00:00
Tasks: Federated Learning, Model Compression

The performance of federated learning systems is bottlenecked by communication costs and training variance. The communication overhead problem is usually addressed by three communication-reduction techniques, namely, model compression, partial device participation, and periodic aggregation, at the cost of increased training variance. Different from traditional distributed learning systems, federated learning suffers from data heterogeneity (since the devices sample their data from possibly different distributions), which induces additional variance among devices during training. Various variance-reduced training algorithms have been introduced to combat the effects of data heterogeneity, while they usually cost additional communication resources to deliver necessary control information. Additionally, data privacy remains a critical issue in FL, and thus there have been attempts at bringing Differential Privacy to this framework as a mediator between utility and privacy requirements. This paper investigates the trade-offs between communication costs and training variance under a resource-constrained federated system theoretically and experimentally, and how communication reduction techniques interplay in a differentially private setting. The results provide important insights into designing practical privacy-aware federated learning systems.
Revisiting Non-Specific Syndromic Surveillance
Authors: Moritz Kulessa, Eneldo Loza Mencía, Johannes Fürnkranz
Date: 2021-01-28 00:00:00

Infectious disease surveillance is of great importance for the prevention of major outbreaks. Syndromic surveillance aims at developing algorithms which can detect outbreaks as early as possible by monitoring data sources which allow to capture the occurrences of a certain disease. Recent research mainly focuses on the surveillance of specific, known diseases, putting the focus on the definition of the disease pattern under surveillance. Until now, only little effort has been devoted to what we call non-specific syndromic surveillance, i.e., the use of all available data for detecting any kind of outbreaks, including infectious diseases which are unknown beforehand. In this work, we revisit published approaches for non-specific syndromic surveillance and present a set of simple statistical modeling techniques which can serve as benchmarks for more elaborate machine learning approaches. Our experimental comparison on established synthetic data and real data in which we injected synthetic outbreaks shows that these benchmarks already achieve very competitive results and often outperform more elaborate algorithms.
Adversarial Learning with Cost-Sensitive Classes
Authors: Haojing Shen, Sihong Chen, Ran Wang, XiZhao Wang
Date: 2021-01-29 00:00:00
Tasks: Adversarial Robustness

It is necessary to improve the performance of some special classes or to particularly protect them from attacks in adversarial learning. This paper proposes a framework combining cost-sensitive classification and adversarial learning together to train a model that can distinguish between protected and unprotected classes, such that the protected classes are less vulnerable to adversarial examples. We find in this framework an interesting phenomenon during the training of deep neural networks, called Min-Max property, that is, the absolute values of most parameters in the convolutional layer approach zero while the absolute values of a few parameters are significantly larger becoming bigger. Based on this Min-Max property which is formulated and analyzed in a view of random distribution, we further build a new defense model against adversarial examples for adversarial robustness improvement. An advantage of the built model is that it performs better than the standard one and can combine with adversarial training to achieve an improved performance. It is experimentally confirmed that, regarding the average accuracy of all classes, our model is almost as same as the existing models when an attack does not occur and is better than the existing models when an attack occurs. Specifically, regarding the accuracy of protected classes, the proposed model is much better than the existing models when an attack occurs.
Fair Resource Allocation for Demands with Sharp Lower Tail Inequalities
Authors: Vacharapat Mettanant, Jittat Fakcharoenphol
Date: 2021-01-29 00:00:00
Tasks: Fairness

We consider a fairness problem in resource allocation where multiple groups demand resources from a common source with the total fixed amount. The general model was introduced by Elzayn et al. [FAT*'19]. We follow Donahue and Kleinberg [FAT*'20] who considered the case when the demand distribution is known. We show that for many common demand distributions that satisfy sharp lower tail inequalities, a natural allocation that provides resources proportional to each group's average demand performs very well. More specifically, this natural allocation is approximately fair and efficient (i.e., it provides near maximum utilization). We also show that, when small amount of unfairness is allowed, the Price of Fairness (PoF), in this case, is close to 1.
An adaptive artificial neural network-based generative design method for layout designs
Authors: Chao Qian, Renkai Tan, Wenjing Ye
Date: 2021-01-29 00:00:00
Tasks: BIG-bench Machine Learning, Generative Adversarial Network, Layout Design

Layout designs are encountered in a variety of fields. For problems with many design degrees of freedom, efficiency of design methods becomes a major concern. In recent years, machine learning methods such as artificial neural networks have been used increasingly to speed up the design process. A main issue of many such approaches is the need for a large corpus of training data that are generated using high-dimensional simulations. The high computational cost associated with training data generation largely diminishes the efficiency gained by using machine learning methods. In this work, an adaptive artificial neural network-based generative design approach is proposed and developed. This method uses a generative adversarial network to generate design candidates and thus the number of design variables is greatly reduced. To speed up the evaluation of the objective function, a convolutional neural network is constructed as the surrogate model for function evaluation. The inverse design is carried out using the genetic algorithm in conjunction with two neural networks. A novel adaptive learning and optimization strategy is proposed, which allows the design space to be effectively explored for the search for optimal solutions. As such the number of training data needed is greatly reduced. The performance of the proposed design method is demonstrated on two heat source layout design problems. In both problems, optimal designs have been obtained. Compared with several existing approaches, the proposed approach has the best performance in terms of accuracy and efficiency.
Learning Twofold Heterogeneous Multi-Task by Sharing Similar Convolution Kernel Pairs
Authors: Quan Feng, Songcan Chen
Date: 2021-01-29 00:00:00
Tasks: Multi-Task Learning

Heterogeneous multi-task learning (HMTL) is an important topic in multi-task learning (MTL). Most existing HMTL methods usually solve either scenario where all tasks reside in the same input (feature) space yet unnecessarily the consistent output (label) space or scenario where their input (feature) spaces are heterogeneous while the output (label) space is consistent. However, to the best of our knowledge, there is limited study on twofold heterogeneous MTL (THMTL) scenario where the input and the output spaces are both inconsistent or heterogeneous. In order to handle this complicated scenario, in this paper, we design a simple and effective multi-task adaptive learning (MTAL) network to learn multiple tasks in such THMTL setting. Specifically, we explore and utilize the inherent relationship between tasks for knowledge sharing from similar convolution kernels in individual layers of the MTAL network. Then in order to realize the sharing, we weightedly aggregate any pair of convolutional kernels with their similarity greater than some threshold $\rho$, consequently, our model effectively performs cross-task learning while suppresses the intra-redundancy of the entire network. Finally, we conduct end-to-end training. Our experimental results demonstrate the effectiveness of our method in comparison with the state-of-the-art counterparts.
Peptipedia: a comprehensive database for peptide research supported by Assembled predictive models and Data Mining approaches
Authors: Cristofer Quiroz, Yasna Barrera Saavedra, Benjamín Armijo-Galdames, Juan Amado-Hinojosa, Álvaro Olivera-Nappa, Anamaria Sanchez-Daza, David Medina-Ortiz
Date: 2021-01-28 00:00:00
Tasks: Drug Design

Motivation: Peptides have attracted the attention in this century due to their remarkable therapeutic properties. Computational tools are being developed to take advantage of existing information, encapsulating knowledge and making it available in a simple way for general public use. However, these are property-specific redundant data systems, and usually do not display the data in a clear way. In some cases, information download is not even possible. This data needs to be available in a simple form for drug design and other biotechnological applications. Results: We developed Peptipedia, a user-friendly database and web application to search, characterise and analyse peptide sequences. Our tool integrates the information from thirty previously reported databases, making it the largest repository of peptides with recorded activities so far. Besides, we implemented a variety of services to increase our tool's usability. The significant differences of our tools with other existing alternatives becomes a substantial contribution to develop biotechnological and bioengineering applications for peptides. Availability: Peptipedia is available for non-commercial use as an open-access software, licensed under the GNU General Public License, version GPL 3.0. The web platform is publicly available at pesb2.cl/peptipedia. Both the source code and sample datasets are available in the GitHub repository https://github.com/CristoferQ/PeptideDatabase. Contact: david.medina@cebib.cl, ana.sanchez@ing.uchile.cl
Gaussian Process Latent Class Choice Models
Authors: Georges Sfeir, Filipe Rodrigues, Maya Abou-Zeid
Date: 2021-01-28 00:00:00
Tasks: Discrete Choice Models, Gaussian Processes

We present a Gaussian Process - Latent Class Choice Model (GP-LCCM) to integrate a non-parametric class of probabilistic machine learning within discrete choice models (DCMs). Gaussian Processes (GPs) are kernel-based algorithms that incorporate expert knowledge by assuming priors over latent functions rather than priors over parameters, which makes them more flexible in addressing nonlinear problems. By integrating a Gaussian Process within a LCCM structure, we aim at improving discrete representations of unobserved heterogeneity. The proposed model would assign individuals probabilistically to behaviorally homogeneous clusters (latent classes) using GPs and simultaneously estimate class-specific choice models by relying on random utility models. Furthermore, we derive and implement an Expectation-Maximization (EM) algorithm to jointly estimate/infer the hyperparameters of the GP kernel function and the class-specific choice parameters by relying on a Laplace approximation and gradient-based numerical optimization methods, respectively. The model is tested on two different mode choice applications and compared against different LCCM benchmarks. Results show that GP-LCCM allows for a more complex and flexible representation of heterogeneity and improves both in-sample fit and out-of-sample predictive power. Moreover, behavioral and economic interpretability is maintained at the class-specific choice model level while local interpretation of the latent classes can still be achieved, although the non-parametric characteristic of GPs lessens the transparency of the model.