pwc-archive/papers-with-abstracts huggingface dataset:
Transfer of Manure as Fertilizer from Livestock Farms to Crop Fields: The Case of Catalonia
Authors: Andreas Kamilaris, Andries Engelbrecht, Andreas Pitsillides, Francesc X. Prenafeta-Boldu
Date: 2020-06-14 00:00:00

Intensive livestock production might have a negative environmental impact, by producing large amounts of animal manure, which, if not properly managed, can contaminate nearby water bodies with nutrient excess. However, if animal manure is exported to nearby crop fields, to be used as organic fertilizer, pollution can be mitigated. It is a single-objective optimization problem, in regards to finding the best solution for the logistics process of satisfying nutrient needs of crops by means of livestock manure. This paper proposes three different approaches to solve the problem: a centralized optimal algorithm (COA), a decentralized nature-inspired cooperative technique, based on the foraging behaviour of ants (AIA), as well as a naive neighbour-based method (NBS), which constitutes the existing practice used today in an ad hoc, uncoordinated manner in Catalonia. Results show that the COA approach is 8.5% more efficient than the AIA. However, the AIA approach is fairer to the farmers and more balanced in terms of average transportation distances that need to be covered by each livestock farmer, while it is 1.07 times more eefficient than the NBS. Our work constitutes the first application of a decentralized AIA to this interesting real-world problem, in a domain where swarm intelligence methods are still under-exploited.
Targeted Adversarial Perturbations for Monocular Depth Prediction
Authors: Alex Wong, Safa Cicek, Stefano Soatto
Date: 2020-06-12 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Adversarial Attack, Adversarial Defense, Depth Estimation, Depth Prediction, Monocular Depth Estimation, Prediction

We study the effect of adversarial perturbations on the task of monocular depth prediction. Specifically, we explore the ability of small, imperceptible additive perturbations to selectively alter the perceived geometry of the scene. We show that such perturbations can not only globally re-scale the predicted distances from the camera, but also alter the prediction to match a different target scene. We also show that, when given semantic or instance information, perturbations can fool the network to alter the depth of specific categories or instances in the scene, and even remove them while preserving the rest of the scene. To understand the effect of targeted perturbations, we conduct experiments on state-of-the-art monocular depth prediction methods. Our experiments reveal vulnerabilities in monocular depth prediction networks, and shed light on the biases and context learned by them.
FCOS: A simple and strong anchor-free object detector
Authors: Zhi Tian, Chunhua Shen, Hao Chen, Tong He
Date: 2020-06-14 00:00:00
Tasks: Object, Object Detection, Semantic Segmentation

In computer vision, object detection is one of most important tasks, which underpins a few instance-level recognition tasks and many downstream applications. Recently one-stage methods have gained much attention over two-stage approaches due to their simpler design and competitive performance. Here we propose a fully convolutional one-stage object detector (FCOS) to solve object detection in a per-pixel prediction fashion, analogue to other dense prediction problems such as semantic segmentation. Almost all state-of-the-art object detectors such as RetinaNet, SSD, YOLOv3, and Faster R-CNN rely on pre-defined anchor boxes. In contrast, our proposed detector FCOS is anchor box free, as well as proposal free. By eliminating the pre-defined set of anchor boxes, FCOS completely avoids the complicated computation related to anchor boxes such as calculating the intersection over union (IoU) scores during training. More importantly, we also avoid all hyper-parameters related to anchor boxes, which are often sensitive to the final detection performance. With the only post-processing non-maximum suppression (NMS), we demonstrate a much simpler and flexible detection framework achieving improved detection accuracy. We hope that the proposed FCOS framework can serve as a simple and strong alternative for many other instance-level tasks. Code and pre-trained models are available at: https://git.io/AdelaiDet
Sentiment Analysis of Arabic Algerian Dialect Using a Supervised Method
Authors: Adel Abdelli, Fayçal Guerrouf, Okba Tibermacine, Belkacem Abdelli
Date: 2020-04-16 00:00:00
Proceeding: International Conference on Intelligent Systems and Advanced Computing Sciences (ISACS) 2020 4
Tasks: Sentiment Analysis

Sentiment analysis holds an important place in Natural Language Processing (NLP) due to its utility in resolving different issues in many fields such as e-commerce, politic sciences, social media analysis, cybersecurity, etc. In fact, most of the work done in the field has been dedicated to the sentiment of English texts. However, adapting research findings to the Arabic language and its dialects is not trivial because of their linguistic features. In this work, we apply two supervised methods, namely, Deep Learning and Support Vector Machines (SVM), for sentiment analysis of Modern Arabic and the Algerian dialect. These methods have been applied on a huge annotated dataset collected from different Arabic Algerian sources. Findings are showing promising results in sentiment analysis of the Arabic Algerian dialect.
A Semantic Matching Energy Function for Learning with Multi-relational Data
Authors: Xavier Glorot, Antoine Bordes, Jason Weston, Yoshua Bengio
Date: 2013-01-15 00:00:00
Tasks: Information Retrieval, Link Prediction, Relational Reasoning, Retrieval

Large-scale relational learning becomes crucial for handling the huge amounts of structured data generated daily in many application domains ranging from computational biology or information retrieval, to natural language processing. In this paper, we present a new neural network architecture designed to embed multi-relational graphs into a flexible continuous vector space in which the original data is kept and enhanced. The network is trained to encode the semantics of these graphs in order to assign high probabilities to plausible components. We empirically show that it reaches competitive performance in link prediction on standard datasets from the literature.
Topology-aware Differential Privacy for Decentralized Image Classification
Authors: Shangwei Guo, Tianwei Zhang, Guowen Xu, Han Yu, Tao Xiang, Yang Liu
Date: 2020-06-14 00:00:00
Tasks: Classification, image-classification, Image Classification

In this paper, we design Top-DP, a novel solution to optimize the differential privacy protection of decentralized image classification systems. The key insight of our solution is to leverage the unique features of decentralized communication topologies to reduce the noise scale and improve the model usability. (1) We enhance the DP-SGD algorithm with this topology-aware noise reduction strategy, and integrate the time-aware noise decay technique. (2) We design two novel learning protocols (synchronous and asynchronous) to protect systems with different network connectivities and topologies. We formally analyze and prove the DP requirement of our proposed solutions. Experimental evaluations demonstrate that our solution achieves a better trade-off between usability and privacy than prior works. To the best of our knowledge, this is the first DP optimization work from the perspective of network topologies.
Learning to Detect 3D Reflection Symmetry for Single-View Reconstruction
Authors: Yichao Zhou, Shichen Liu, Yi Ma
Date: 2020-06-17 00:00:00
Tasks: 3D Reconstruction, Single-View 3D Reconstruction

3D reconstruction from a single RGB image is a challenging problem in computer vision. Previous methods are usually solely data-driven, which lead to inaccurate 3D shape recovery and limited generalization capability. In this work, we focus on object-level 3D reconstruction and present a geometry-based end-to-end deep learning framework that first detects the mirror plane of reflection symmetry that commonly exists in man-made objects and then predicts depth maps by finding the intra-image pixel-wise correspondence of the symmetry. Our method fully utilizes the geometric cues from symmetry during the test time by building plane-sweep cost volumes, a powerful tool that has been used in multi-view stereopsis. To our knowledge, this is the first work that uses the concept of cost volumes in the setting of single-image 3D reconstruction. We conduct extensive experiments on the ShapeNet dataset and find that our reconstruction method significantly outperforms the previous state-of-the-art single-view 3D reconstruction networks in term of the accuracy of camera poses and depth maps, without requiring objects being completely symmetric. Code is available at https://github.com/zhou13/symmetrynet.
LSD-C: Linearly Separable Deep Clusters
Authors: Sylvestre-Alvise Rebuffi, Sebastien Ehrhardt, Kai Han, Andrea Vedaldi, Andrew Zisserman
Date: 2020-06-17 00:00:00
Tasks: Clustering, Data Augmentation, Deep Clustering, Document Classification, Image Classification, Image Clustering

We present LSD-C, a novel method to identify clusters in an unlabeled dataset. Our algorithm first establishes pairwise connections in the feature space between the samples of the minibatch based on a similarity metric. Then it regroups in clusters the connected samples and enforces a linear separation between clusters. This is achieved by using the pairwise connections as targets together with a binary cross-entropy loss on the predictions that the associated pairs of samples belong to the same cluster. This way, the feature representation of the network will evolve such that similar samples in this feature space will belong to the same linearly separated cluster. Our method draws inspiration from recent semi-supervised learning practice and proposes to combine our clustering algorithm with self-supervised pretraining and strong data augmentation. We show that our approach significantly outperforms competitors on popular public image benchmarks including CIFAR 10/100, STL 10 and MNIST, as well as the document classification dataset Reuters 10K.
Semantic Visual Navigation by Watching YouTube Videos
Authors: Matthew Chang, Arjun Gupta, Saurabh Gupta
Date: 2020-06-17 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Q-Learning, Visual Navigation

Semantic cues and statistical regularities in real-world environment layouts can improve efficiency for navigation in novel environments. This paper learns and leverages such semantic cues for navigating to objects of interest in novel environments, by simply watching YouTube videos. This is challenging because YouTube videos don't come with labels for actions or goals, and may not even showcase optimal behavior. Our method tackles these challenges through the use of Q-learning on pseudo-labeled transition quadruples (image, action, next image, reward). We show that such off-policy Q-learning from passive data is able to learn meaningful semantic cues for navigation. These cues, when used in a hierarchical navigation policy, lead to improved efficiency at the ObjectGoal task in visually realistic simulations. We observe a relative improvement of 15-83% over end-to-end RL, behavior cloning, and classical methods, while using minimal direct interaction.
Self-training Avoids Using Spurious Features Under Domain Shift
Authors: Yining Chen, Colin Wei, Ananya Kumar, Tengyu Ma
Date: 2020-06-17 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

In unsupervised domain adaptation, existing theory focuses on situations where the source and target domains are close. In practice, conditional entropy minimization and pseudo-labeling work even when the domain shifts are much larger than those analyzed by existing theory. We identify and analyze one particular setting where the domain shift can be large, but these algorithms provably work: certain spurious features correlate with the label in the source domain but are independent of the label in the target. Our analysis considers linear classification where the spurious features are Gaussian and the non-spurious features are a mixture of log-concave distributions. For this setting, we prove that entropy minimization on unlabeled target data will avoid using the spurious feature if initialized with a decently accurate source classifier, even though the objective is non-convex and contains multiple bad local minima using the spurious features. We verify our theory for spurious domain shift tasks on semi-synthetic Celeb-A and MNIST datasets. Our results suggest that practitioners collect and self-train on large, diverse datasets to reduce biases in classifiers even if labeling is impractical.
Big Self-Supervised Models are Strong Semi-Supervised Learners
Authors: Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, Geoffrey Hinton
Date: 2020-06-17 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Self-Supervised Image Classification, Semi-Supervised Image Classification

One paradigm for learning from few labeled examples while making best use of a large amount of unlabeled data is unsupervised pretraining followed by supervised fine-tuning. Although this paradigm uses unlabeled data in a task-agnostic way, in contrast to common approaches to semi-supervised learning for computer vision, we show that it is surprisingly effective for semi-supervised learning on ImageNet. A key ingredient of our approach is the use of big (deep and wide) networks during pretraining and fine-tuning. We find that, the fewer the labels, the more this approach (task-agnostic use of unlabeled data) benefits from a bigger network. After fine-tuning, the big network can be further improved and distilled into a much smaller one with little loss in classification accuracy by using the unlabeled examples for a second time, but in a task-specific way. The proposed semi-supervised learning algorithm can be summarized in three steps: unsupervised pretraining of a big ResNet model using SimCLRv2, supervised fine-tuning on a few labeled examples, and distillation with unlabeled examples for refining and transferring the task-specific knowledge. This procedure achieves 73.9% ImageNet top-1 accuracy with just 1% of the labels ($\le$13 labeled images per class) using ResNet-50, a $10\times$ improvement in label efficiency over the previous state-of-the-art. With 10% of labels, ResNet-50 trained with our method achieves 77.5% top-1 accuracy, outperforming standard supervised training with all of the labels.
Deep Learning Meets SAR
Authors: Xiao Xiang Zhu, Sina Montazeri, Mohsin Ali, Yuansheng Hua, Yuanyuan Wang, Lichao Mou, Yilei Shi, Feng Xu, Richard Bamler
Date: 2020-06-17 00:00:00
Tasks: Deep Learning

Deep learning in remote sensing has become an international hype, but it is mostly limited to the evaluation of optical data. Although deep learning has been introduced in Synthetic Aperture Radar (SAR) data processing, despite successful first attempts, its huge potential remains locked. In this paper, we provide an introduction to the most relevant deep learning models and concepts, point out possible pitfalls by analyzing special characteristics of SAR data, review the state-of-the-art of deep learning applied to SAR in depth, summarize available benchmarks, and recommend some important future research directions. With this effort, we hope to stimulate more research in this interesting yet under-exploited research field and to pave the way for use of deep learning in big SAR data processing workflows.
Analytical Probability Distributions and EM-Learning for Deep Generative Networks
Authors: Randall Balestriero, Sebastien Paris, Richard G. Baraniuk
Date: 2020-06-17 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Anomaly Detection, Imputation, Variational Inference

Deep Generative Networks (DGNs) with probabilistic modeling of their output and latent space are currently trained via Variational Autoencoders (VAEs). In the absence of a known analytical form for the posterior and likelihood expectation, VAEs resort to approximations, including (Amortized) Variational Inference (AVI) and Monte-Carlo (MC) sampling. We exploit the Continuous Piecewise Affine (CPA) property of modern DGNs to derive their posterior and marginal distributions as well as the latter's first moments. These findings enable us to derive an analytical Expectation-Maximization (EM) algorithm that enables gradient-free DGN learning. We demonstrate empirically that EM training of DGNs produces greater likelihood than VAE training. Our findings will guide the design of new VAE AVI that better approximate the true posterior and open avenues to apply standard statistical tools for model comparison, anomaly detection, and missing data imputation.
Conversational Neuro-Symbolic Commonsense Reasoning
Authors: Forough Arabshahi, Jennifer Lee, Mikayla Gawarecki, Kathryn Mazaitis, Amos Azaria, Tom Mitchell
Date: 2020-06-17 00:00:00

In order for conversational AI systems to hold more natural and broad-ranging conversations, they will require much more commonsense, including the ability to identify unstated presumptions of their conversational partners. For example, in the command "If it snows at night then wake me up early because I don't want to be late for work" the speaker relies on commonsense reasoning of the listener to infer the implicit presumption that they wish to be woken only if it snows enough to cause traffic slowdowns. We consider here the problem of understanding such imprecisely stated natural language commands given in the form of "if-(state), then-(action), because-(goal)" statements. More precisely, we consider the problem of identifying the unstated presumptions of the speaker that allow the requested action to achieve the desired goal from the given state (perhaps elaborated by making the implicit presumptions explicit). We release a benchmark data set for this task, collected from humans and annotated with commonsense presumptions. We present a neuro-symbolic theorem prover that extracts multi-hop reasoning chains, and apply it to this problem. Furthermore, to accommodate the reality that current AI commonsense systems lack full coverage, we also present an interactive conversational framework built on our neuro-symbolic system, that conversationally evokes commonsense knowledge from humans to complete its reasoning chains.
Generalising Recursive Neural Models by Tensor Decomposition
Authors: Daniele Castellana, Davide Bacciu
Date: 2020-06-17 00:00:00
Tasks: Tensor Decomposition

Most machine learning models for structured data encode the structural knowledge of a node by leveraging simple aggregation functions (in neural models, typically a weighted sum) of the information in the node's neighbourhood. Nevertheless, the choice of simple context aggregation functions, such as the sum, can be widely sub-optimal. In this work we introduce a general approach to model aggregation of structural context leveraging a tensor-based formulation. We show how the exponential growth in the size of the parameter space can be controlled through an approximation based on the Tucker tensor decomposition. This approximation allows limiting the parameters space size, decoupling it from its strict relation with the size of the hidden encoding space. By this means, we can effectively regulate the trade-off between expressivity of the encoding, controlled by the hidden size, computational complexity and model generalisation, influenced by parameterisation. Finally, we introduce a new Tensorial Tree-LSTM derived as an instance of our framework and we use it to experimentally assess our working hypotheses on tree classification scenarios.
The Nyström method for convex loss functions
Authors: Andrea Della Vecchia, Ernesto de Vito, Jaouad Mourtada, Lorenzo Rosasco
Date: 2020-06-17 00:00:00
Tasks: Computational Efficiency

We investigate an extension of classical empirical risk minimization, where the hypothesis space consists of a random subspace within a given Hilbert space. Specifically, we examine the Nystr\"om method where the subspaces are defined by a random subset of the data. This approach recovers Nystr\"om approximations used in kernel methods as a specific case. Using random subspaces naturally leads to computational advantages, but a key question is whether it compromises the learning accuracy. Recently, the tradeoffs between statistics and computation have been explored for the square loss and self-concordant losses, such as the logistic loss. In this paper, we extend these analyses to general convex Lipschitz losses, which may lack smoothness, such as the hinge loss used in support vector machines. Our main results show the existence of various scenarios where computational gains can be achieved without sacrificing learning performance. When specialized to smooth loss functions, our analysis recovers most previous results. Moreover, it allows to consider classification problems and translate the surrogate risk bounds into classification error bounds. Indeed, this gives the opportunity to compare the effect of Nystr\"om approximations when combined with different loss functions such as the hinge or the square loss.
Adversarial Examples Detection and Analysis with Layer-wise Autoencoders
Authors: Bartosz Wójcik, Paweł Morawiecki, Marek Śmieja, Tomasz Krzyżek, Przemysław Spurek, Jacek Tabor
Date: 2020-06-17 00:00:00

We present a mechanism for detecting adversarial examples based on data representations taken from the hidden layers of the target network. For this purpose, we train individual autoencoders at intermediate layers of the target network. This allows us to describe the manifold of true data and, in consequence, decide whether a given example has the same characteristics as true data. It also gives us insight into the behavior of adversarial examples and their flow through the layers of a deep neural network. Experimental results show that our method outperforms the state of the art in supervised and unsupervised settings.
Robust Persistence Diagrams using Reproducing Kernels
Authors: Siddharth Vishwanath, Kenji Fukumizu, Satoshi Kuriki, Bharath Sriperumbudur
Date: 2020-06-17 00:00:00
Proceeding: NeurIPS 2020 12

Persistent homology has become an important tool for extracting geometric and topological features from data, whose multi-scale features are summarized in a persistence diagram. From a statistical perspective, however, persistence diagrams are very sensitive to perturbations in the input space. In this work, we develop a framework for constructing robust persistence diagrams from superlevel filtrations of robust density estimators constructed using reproducing kernels. Using an analogue of the influence function on the space of persistence diagrams, we establish the proposed framework to be less sensitive to outliers. The robust persistence diagrams are shown to be consistent estimators in bottleneck distance, with the convergence rate controlled by the smoothness of the kernel. This, in turn, allows us to construct uniform confidence bands in the space of persistence diagrams. Finally, we demonstrate the superiority of the proposed approach on benchmark datasets.