pwc-archive/papers-with-abstracts huggingface dataset:
Disentangled Neural Architecture Search
Authors: Xinyue Zheng, Peng Wang, Qigang Wang, Zhongchao shi
Date: 2020-09-24 00:00:00
Tasks: Neural Architecture Search

Neural architecture search has shown its great potential in various areas recently. However, existing methods rely heavily on a black-box controller to search architectures, which suffers from the serious problem of lacking interpretability. In this paper, we propose disentangled neural architecture search (DNAS) which disentangles the hidden representation of the controller into semantically meaningful concepts, making the neural architecture search process interpretable. Based on systematical study, we discover the correlation between network architecture and its performance, and propose a dense-sampling strategy to conduct a targeted search in promising regions that may generate well-performing architectures. We show that: 1) DNAS successfully disentangles the architecture representations, including operation selection, skip connections, and number of layers. 2) Benefiting from interpretability, DNAS can find excellent architectures under different FLOPS restrictions flexibly. 3) Dense-sampling leads to neural architecture search with higher efficiency and better performance. On the NASBench-101 dataset, DNAS achieves state-of-the-art performance of 94.21% using less than 1/13 computational cost of baseline methods. On ImageNet dataset, DNAS discovers the competitive architectures that achieves 22.7% test error. our method provides a new perspective of understanding neural architecture search.
Embedding and generation of indoor climbing routes with variational autoencoder
Authors: K. H. Lo
Date: 2020-09-16 00:00:00

Recent increase in popularity of indoor climbing allows possible applications of deep learning algorthms to classify and generate climbing routes. In this work, we employ a variational autoencoder to climbing routes in a standardized training apparatus MoonBoard, a well-known training tool within the climbing community. By sampling the encoded latent space, it is observed that the algorithm can generate high quality climbing routes. 22 generated problems are uploaded to the Moonboard app for user review. This algorithm could serve as a first step to facilitate indoor climbing route setting.
Physics Informed Neural Networks for Simulating Radiative Transfer
Authors: Siddhartha Mishra, Roberto Molinaro
Date: 2020-09-25 00:00:00
Tasks: BIG-bench Machine Learning

We propose a novel machine learning algorithm for simulating radiative transfer. Our algorithm is based on physics informed neural networks (PINNs), which are trained by minimizing the residual of the underlying radiative tranfer equations. We present extensive experiments and theoretical error estimates to demonstrate that PINNs provide a very easy to implement, fast, robust and accurate method for simulating radiative transfer. We also present a PINN based algorithm for simulating inverse problems for radiative transfer efficiently.
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Authors: Wenshuai Zhao, Jorge Peña Queralta, Tomi Westerlund
Date: 2020-09-24 00:00:00
Tasks: Deep Reinforcement Learning, Domain Adaptation, Imitation Learning, Knowledge Distillation, Meta-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning has recently seen huge success across multiple areas in the robotics domain. Owing to the limitations of gathering real-world data, i.e., sample inefficiency and the cost of collecting it, simulation environments are utilized for training the different agents. This not only aids in providing a potentially infinite data source, but also alleviates safety concerns with real robots. Nonetheless, the gap between the simulated and real worlds degrades the performance of the policies once the models are transferred into real robots. Multiple research efforts are therefore now being directed towards closing this sim-to-real gap and accomplish more efficient policy transfer. Recent years have seen the emergence of multiple methods applicable to different domains, but there is a lack, to the best of our knowledge, of a comprehensive review summarizing and putting into context the different methods. In this survey paper, we cover the fundamental background behind sim-to-real transfer in deep reinforcement learning and overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation. We categorize some of the most relevant recent works, and outline the main application scenarios. Finally, we discuss the main opportunities and challenges of the different approaches and point to the most promising directions.
Fair Meta-Learning For Few-Shot Classification
Authors: Chen Zhao, Changbin Li, Jincheng Li, Feng Chen
Date: 2020-09-23 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Fairness, General Classification, Meta-Learning

Artificial intelligence nowadays plays an increasingly prominent role in our life since decisions that were once made by humans are now delegated to automated systems. A machine learning algorithm trained based on biased data, however, tends to make unfair predictions. Developing classification algorithms that are fair with respect to protected attributes of the data thus becomes an important problem. Motivated by concerns surrounding the fairness effects of sharing and few-shot machine learning tools, such as the Model Agnostic Meta-Learning framework, we propose a novel fair fast-adapted few-shot meta-learning approach that efficiently mitigates biases during meta-train by ensuring controlling the decision boundary covariance that between the protected variable and the signed distance from the feature vectors to the decision boundary. Through extensive experiments on two real-world image benchmarks over three state-of-the-art meta-learning algorithms, we empirically demonstrate that our proposed approach efficiently mitigates biases on model output and generalizes both accuracy and fairness to unseen tasks with a limited amount of training samples.
Sketch-based community detection in evolving networks
Authors: Andre Beckus, George K. Atia
Date: 2020-09-24 00:00:00
Tasks: Community Detection, Stochastic Block Model

We consider an approach for community detection in time-varying networks. At its core, this approach maintains a small sketch graph to capture the essential community structure found in each snapshot of the full network. We demonstrate how the sketch can be used to explicitly identify six key community events which typically occur during network evolution: growth, shrinkage, merging, splitting, birth and death. Based on these detection techniques, we formulate a community detection algorithm which can process a network concurrently exhibiting all processes. One advantage afforded by the sketch-based algorithm is the efficient handling of large networks. Whereas detecting events in the full graph may be computationally expensive, the small size of the sketch allows changes to be quickly assessed. A second advantage occurs in networks containing clusters of disproportionate size. The sketch is constructed such that there is equal representation of each cluster, thus reducing the possibility that the small clusters are lost in the estimate. We present a new standardized benchmark based on the stochastic block model which models the addition and deletion of nodes, as well as the birth and death of communities. When coupled with existing benchmarks, this new benchmark provides a comprehensive suite of tests encompassing all six community events. We provide analysis and a set of numerical results demonstrating the advantages of our approach both in run time and in the handling of small clusters.
DeepControl: 2D RF pulses facilitating $B_1^+$ inhomogeneity and $B_0$ off-resonance compensation in vivo at 7T
Authors: Mads Sloth Vinding, Christoph Stefan Aigner, Sebastian Schmitter, Torben Ellegaard Lund
Date: 2020-09-25 00:00:00

Purpose: Rapid 2D RF pulse design with subject specific $B_1^+$ inhomogeneity and $B_0$ off-resonance compensation at 7 T predicted from convolutional neural networks is presented. Methods: The convolution neural network was trained on half a million single-channel transmit, 2D RF pulses optimized with an optimal control method using artificial 2D targets, $B_1^+$ and $B_0$ maps. Predicted pulses were tested in a phantom and in vivo at 7 T with measured $B_1^+$ and $B_0$ maps from a high-resolution GRE sequence. Results: Pulse prediction by the trained convolutional neural network was done on the fly during the MR session in approximately 9 ms for multiple hand drawn ROIs and the measured $B_1^+$ and $B_0$ maps. Compensation of $B_1^+$ inhomogeneity and $B_0$ off-resonances has been confirmed in the phantom and in vivo experiments. The reconstructed image data agrees well with the simulations using the acquired $B_1^+$ and $B_0$ maps and the 2D RF pulse predicted by the convolutional neural networks is as good as the conventional RF pulse obtained by optimal control. Conclusion: The proposed convolutional neural network based 2D RF pulse design method predicts 2D RF pulses with an excellent excitation pattern and compensated $B_1^+$ and $B_0$ variations at 7 T. The rapid 2D RF pulse prediction (9 ms) enables subject-specific high-quality 2D RF pulses without the need to run lengthy optimizations.
A Big Data Lake for Multilevel Streaming Analytics
Authors: Ruoran Liu, Haruna Isah, Farhana Zulkernine
Date: 2020-09-25 00:00:00
Tasks: Management

Large organizations are seeking to create new architectures and scalable platforms to effectively handle data management challenges due to the explosive nature of data rarely seen in the past. These data management challenges are largely posed by the availability of streaming data at high velocity from various sources in multiple formats. The changes in data paradigm have led to the emergence of new data analytics and management architecture. This paper focuses on storing high volume, velocity and variety data in the raw formats in a data storage architecture called a data lake. First, we present our study on the limitations of traditional data warehouses in handling recent changes in data paradigms. We discuss and compare different open source and commercial platforms that can be used to develop a data lake. We then describe our end-to-end data lake design and implementation approach using the Hadoop Distributed File System (HDFS) on the Hadoop Data Platform (HDP). Finally, we present a real-world data lake development use case for data stream ingestion, staging, and multilevel streaming analytics which combines structured and unstructured data. This study can serve as a guide for individuals or organizations planning to implement a data lake solution for their use cases.
Towards General Purpose Geometry-Preserving Single-View Depth Estimation
Authors: Mikhail Romanov, Nikolay Patatkin, Anna Vorontsova, Sergey Nikolenko, Anton Konushin, Dmitry Senyushkin
Date: 2020-09-25 00:00:00
Tasks: Depth Estimation, Diversity, Monocular Depth Estimation, Scene Understanding

Single-view depth estimation (SVDE) plays a crucial role in scene understanding for AR applications, 3D modeling, and robotics, providing the geometry of a scene based on a single image. Recent works have shown that a successful solution strongly relies on the diversity and volume of training data. This data can be sourced from stereo movies and photos. However, they do not provide geometrically complete depth maps (as disparities contain unknown shift value). Therefore, existing models trained on this data are not able to recover correct 3D representations. Our work shows that a model trained on this data along with conventional datasets can gain accuracy while predicting correct scene geometry. Surprisingly, only a small portion of geometrically correct depth maps are required to train a model that performs equally to a model trained on the full geometrically correct dataset. After that, we train computationally efficient models on a mixture of datasets using the proposed method. Through quantitative comparison on completely unseen datasets and qualitative comparison of 3D point clouds, we show that our model defines the new state of the art in general-purpose SVDE.
Learning Sparse Sentence Encoding without Supervision: An Exploration of Sparsity in Variational Autoencoders
Authors: Victor Prokhorov, Yingzhen Li, Ehsan Shareghi, Nigel Collier
Date: 2020-09-25 00:00:00
Proceeding: ACL (RepL4NLP) 2021 8
Tasks: Inductive Bias, Representation Learning, Sentence, text-classification, Text Classification

It has been long known that sparsity is an effective inductive bias for learning efficient representation of data in vectors with fixed dimensionality, and it has been explored in many areas of representation learning. Of particular interest to this work is the investigation of the sparsity within the VAE framework which has been explored a lot in the image domain, but has been lacking even a basic level of exploration in NLP. Additionally, NLP is also lagging behind in terms of learning sparse representations of large units of text e.g., sentences. We use the VAEs that induce sparse latent representations of large units of text to address the aforementioned shortcomings. First, we move in this direction by measuring the success of unsupervised state-of-the-art (SOTA) and other strong VAE-based sparsification baselines for text and propose a hierarchical sparse VAE model to address the stability issue of SOTA. Then, we look at the implications of sparsity on text classification across 3 datasets, and highlight a link between performance of sparse latent representations on downstream tasks and its ability to encode task-related information.
Lateral Force Prediction using Gaussian Process Regression for Intelligent Tire Systems
Authors: Bruno Henrique Groenner Barbosa, Nan Xu, Hassan Askari, Amir Khajepour
Date: 2020-09-25 00:00:00
Tasks: GPR, regression

Understanding the dynamic behavior of tires and their interactions with road plays an important role in designing integrated vehicle control strategies. Accordingly, having access to reliable information about the tire-road interactions through tire embedded sensors is very demanding for developing enhanced vehicle control systems. Thus, the main objectives of the present research work are i. to analyze data from an experimental accelerometer-based intelligent tire acquired over a wide range of maneuvers, with different vertical loads, velocities, and high slip angles; and ii. to develop a lateral force predictor based on a machine learning tool, more specifically the Gaussian Process Regression (GPR) technique. It is delineated that the proposed intelligent tire system can provide reliable information about the tire-road interactions even in the case of high slip angles. Besides, the lateral forces model based on GPR can predict forces with acceptable accuracy and provide level of uncertainties that can be very useful for designing vehicle control strategies.
Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Coding
Authors: David Burth Kurka, Deniz Gündüz
Date: 2020-09-26 00:00:00

We propose deep learning based communication methods for adaptive-bandwidth transmission of images over wireless channels. We consider the scenario in which images are transmitted progressively in layers over time or frequency, and such layers can be aggregated by receivers in order to increase the quality of their reconstructions. We investigate two scenarios, one in which the layers are sent sequentially, and incrementally contribute to the refinement of a reconstruction, and another in which the layers are independent and can be retrieved in any order. Those scenarios correspond to the well known problems of \textit{successive refinement} and \textit{multiple descriptions}, respectively, in the context of joint source-channel coding (JSCC). We propose DeepJSCC-$l$, an innovative solution that uses convolutional autoencoders, and present three architectures with different complexity trade-offs. To the best of our knowledge, this is the first practical multiple-description JSCC scheme developed and tested for practical information sources and channels. Numerical results show that DeepJSCC-$l$ can learn to transmit the source progressively with negligible losses in the end-to-end performance compared with a single transmission. Moreover, DeepJSCC-$l$ has comparable performance with state of the art digital progressive transmission schemes in the challenging low signal-to-noise ratio (SNR) and small bandwidth regimes, with the additional advantage of graceful degradation with channel SNR.
Machine Learning Algorithms for Active Monitoring of High Performance Computing as a Service (HPCaaS) Cloud Environments
Authors: Gianluca Longoni, Ryan LaMothe, Jeremy Teuton, Mark Greaves, Nicole Nichols, William Smith
Date: 2020-09-26 00:00:00
Tasks: BIG-bench Machine Learning, Cloud Computing, Management, Privacy Preserving

Cloud computing provides ubiquitous and on-demand access to vast reconfigurable resources that can meet any computational need. Many service models are available, but the Infrastructure as a Service (IaaS) model is particularly suited to operate as a high performance computing (HPC) platform, by networking large numbers of cloud computing nodes. We used the Pacific Northwest National Laboratory (PNNL) cloud computing environment to perform our experiments. A number of cloud computing providers such as Amazon Web Services, Microsoft Azure, or IBM Cloud, offer flexible and scalable computing resources. This paper explores the viability identifying types of engineering applications running on a cloud infrastructure configured as an HPC platform using privacy preserving features as input to statistical models. The engineering applications considered in this work include MCNP6, a radiation transport code developed by Los Alamos National Laboratory, OpenFOAM, an open source computational fluid dynamics code, and CADO-NFS, a numerical implementation of the general number field sieve algorithm used for prime number factorization. Our experiments use the OpenStack cloud management tool to create a cloud HPC environment and the privacy preserving Ceilometer billing meters as classification features to demonstrate identification of these applications.
QuatRE: Relation-Aware Quaternions for Knowledge Graph Embeddings
Authors: Dai Quoc Nguyen, Thanh Vu, Tu Dinh Nguyen, Dinh Phung
Date: 2020-09-26 00:00:00
Tasks: Knowledge Graph Completion, Knowledge Graph Embeddings, Knowledge Graphs, Relation

We propose a simple yet effective embedding model to learn quaternion embeddings for entities and relations in knowledge graphs. Our model aims to enhance correlations between head and tail entities given a relation within the Quaternion space with Hamilton product. The model achieves this goal by further associating each relation with two relation-aware rotations, which are used to rotate quaternion embeddings of the head and tail entities, respectively. Experimental results show that our proposed model produces state-of-the-art performances on well-known benchmark datasets for knowledge graph completion. Our code is available at: \url{https://github.com/daiquocnguyen/QuatRE}.
A light-weight method to foster the (Grad)CAM interpretability and explainability of classification networks
Authors: Alfred Schöttl
Date: 2020-09-26 00:00:00

We consider a light-weight method which allows to improve the explainability of localized classification networks. The method considers (Grad)CAM maps during the training process by modification of the training loss and does not require additional structural elements. It is demonstrated that the (Grad)CAM interpretability, as measured by several indicators, can be improved in this way. Since the method shall be applicable on embedded systems and on standard deeper architectures, it essentially takes advantage of second order derivatives during the training and does not require additional model layers.
Metaphor Detection using Deep Contextualized Word Embeddings
Authors: Shashwat Aggarwal, Ramesh Singh
Date: 2020-09-26 00:00:00
Tasks: Sentiment Analysis, Word Embeddings

Metaphors are ubiquitous in natural language, and their detection plays an essential role in many natural language processing tasks, such as language understanding, sentiment analysis, etc. Most existing approaches for metaphor detection rely on complex, hand-crafted and fine-tuned feature pipelines, which greatly limit their applicability. In this work, we present an end-to-end method composed of deep contextualized word embeddings, bidirectional LSTMs and multi-head attention mechanism to address the task of automatic metaphor detection. Our method, unlike many other existing approaches, requires only the raw text sequences as input features to detect the metaphoricity of a phrase. We compare the performance of our method against the existing baselines on two benchmark datasets, TroFi, and MOH-X respectively. Experimental evaluations confirm the effectiveness of our approach.
DT-Net: A novel network based on multi-directional integrated convolution and threshold convolution
Authors: Hongfeng You, Long Yu, Shengwei Tian, Xiang Ma, Yan Xing, Xiaojie Ma
Date: 2020-09-26 00:00:00
Tasks: Segmentation, Semantic Segmentation

Since medical image data sets contain few samples and singular features, lesions are viewed as highly similar to other tissues. The traditional neural network has a limited ability to learn features. Even if a host of feature maps is expanded to obtain more semantic information, the accuracy of segmenting the final medical image is slightly improved, and the features are excessively redundant. To solve the above problems, in this paper, we propose a novel end-to-end semantic segmentation algorithm, DT-Net, and use two new convolution strategies to better achieve end-to-end semantic segmentation of medical images. 1. In the feature mining and feature fusion stage, we construct a multi-directional integrated convolution (MDIC). The core idea is to use the multi-scale convolution to enhance the local multi-directional feature maps to generate enhanced feature maps and to mine the generated features that contain more semantics without increasing the number of feature maps. 2. We also aim to further excavate and retain more meaningful deep features reduce a host of noise features in the training process. Therefore, we propose a convolution thresholding strategy. The central idea is to set a threshold to eliminate a large number of redundant features and reduce computational complexity. Through the two strategies proposed above, the algorithm proposed in this paper produces state-of-the-art results on two public medical image datasets. We prove in detail that our proposed strategy plays an important role in feature mining and eliminating redundant features. Compared with the existing semantic segmentation algorithms, our proposed algorithm has better robustness.