pwc-archive/papers-with-abstracts huggingface dataset:
Curvature Regularization to Prevent Distortion in Graph Embedding
Authors: Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Chunxu Zhang, Bo Yang
Date: 2020-11-28 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Graph Embedding

Recent research on graph embedding has achieved success in various applications. Most graph embedding methods preserve the proximity in a graph into a manifold in an embedding space. We argue an important but neglected problem about this proximity-preserving strategy: Graph topology patterns, while preserved well into an embedding manifold by preserving proximity, may distort in the ambient embedding Euclidean space, and hence to detect them becomes difficult for machine learning models. To address the problem, we propose curvature regularization, to enforce flatness for embedding manifolds, thereby preventing the distortion. We present a novel angle-based sectional curvature, termed ABS curvature, and accordingly three kinds of curvature regularization to induce flat embedding manifolds during graph embedding. We integrate curvature regularization into five popular proximity-preserving embedding methods, and empirical results in two applications show significant improvements on a wide range of open graph datasets.
Class-agnostic Object Detection
Authors: Ayush Jaiswal, Yue Wu, Pradeep Natarajan, Premkumar Natarajan
Date: 2020-11-28 00:00:00
Tasks: Benchmarking, Class-agnostic Object Detection, Image Classification, Object, object-detection, Object Detection, Relation Prediction, Retrieval, Visual Grounding

Object detection models perform well at localizing and classifying objects that they are shown during training. However, due to the difficulty and cost associated with creating and annotating detection datasets, trained models detect a limited number of object types with unknown objects treated as background content. This hinders the adoption of conventional detectors in real-world applications like large-scale object matching, visual grounding, visual relation prediction, obstacle detection (where it is more important to determine the presence and location of objects than to find specific types), etc. We propose class-agnostic object detection as a new problem that focuses on detecting objects irrespective of their object-classes. Specifically, the goal is to predict bounding boxes for all objects in an image but not their object-classes. The predicted boxes can then be consumed by another system to perform application-specific classification, retrieval, etc. We propose training and evaluation protocols for benchmarking class-agnostic detectors to advance future research in this domain. Finally, we propose (1) baseline methods and (2) a new adversarial learning framework for class-agnostic detection that forces the model to exclude class-specific information from features used for predictions. Experimental results show that adversarial learning improves class-agnostic detection efficacy.
EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference
Authors: Thierry Tambe, Coleman Hooper, Lillian Pentecost, Tianyu Jia, En-Yu Yang, Marco Donato, Victor Sanh, Paul N. Whatmough, Alexander M. Rush, David Brooks, Gu-Yeon Wei
Date: 2020-11-28 00:00:00
Tasks: Edge-computing, GPU, Network Pruning, Quantization, Sentence

Transformer-based language models such as BERT provide significant accuracy improvement for a multitude of natural language processing (NLP) tasks. However, their hefty computational and memory demands make them challenging to deploy to resource-constrained edge platforms with strict latency requirements. We present EdgeBERT, an in-depth algorithm-hardware co-design for latency-aware energy optimization for multi-task NLP. EdgeBERT employs entropy-based early exit predication in order to perform dynamic voltage-frequency scaling (DVFS), at a sentence granularity, for minimal energy consumption while adhering to a prescribed target latency. Computation and memory footprint overheads are further alleviated by employing a calibrated combination of adaptive attention span, selective network pruning, and floating-point quantization. Furthermore, in order to maximize the synergistic benefits of these algorithms in always-on and intermediate edge computing settings, we specialize a 12nm scalable hardware accelerator system, integrating a fast-switching low-dropout voltage regulator (LDO), an all-digital phase-locked loop (ADPLL), as well as, high-density embedded non-volatile memories (eNVMs) wherein the sparse floating-point bit encodings of the shared multi-task parameters are carefully stored. Altogether, latency-aware multi-task NLP inference acceleration on the EdgeBERT hardware system generates up to 7x, 2.5x, and 53x lower energy compared to the conventional inference without early stopping, the latency-unbounded early exit approach, and CUDA adaptations on an Nvidia Jetson Tegra X2 mobile GPU, respectively.
E-Pro: Euler Angle and Probabilistic Model for Face Detection and Recognition
Authors: Sandesh Ramesh, Manoj Kumar M V, Sanjay H A
Date: 2020-11-28 00:00:00
Tasks: Face Detection, Face Recognition

It is human nature to give prime importance to facial appearances. Often, to look good is to feel good. Also, facial features are unique to every individual on this planet, which means it is a source of vital information. This work proposes a framework named E-Pro for the detection and recognition of faces by taking facial images as inputs. E-Pro has its potential application in various domains, namely attendance, surveillance, crowd monitoring, biometric-based authentication etc. E-Pro is developed here as a mobile application that aims to aid lecturers to mark attendance in a classroom by detecting and recognizing the faces of students from a picture clicked through the app. E-Pro has been developed using Google Firebase Face Recognition APIs, which uses Euler Angles, and Probabilistic Model. E-Pro has been tested on stock images and the experimental results are promising.
Privacy-Preserving Federated Learning for UAV-Enabled Networks: Learning-Based Joint Scheduling and Resource Management
Authors: Helin Yang, Jun Zhao, Zehui Xiong, Kwok-Yan Lam, Sumei Sun, Liang Xiao
Date: 2020-11-28 00:00:00
Tasks: Distributed Computing, Federated Learning, Management, Privacy Preserving, Scheduling

Unmanned aerial vehicles (UAVs) are capable of serving as flying base stations (BSs) for supporting data collection, artificial intelligence (AI) model training, and wireless communications. However, due to the privacy concerns of devices and limited computation or communication resource of UAVs, it is impractical to send raw data of devices to UAV servers for model training. Moreover, due to the dynamic channel condition and heterogeneous computing capacity of devices in UAV-enabled networks, the reliability and efficiency of data sharing require to be further improved. In this paper, we develop an asynchronous federated learning (AFL) framework for multi-UAV-enabled networks, which can provide asynchronous distributed computing by enabling model training locally without transmitting raw sensitive data to UAV servers. The device selection strategy is also introduced into the AFL framework to keep the low-quality devices from affecting the learning efficiency and accuracy. Moreover, we propose an asynchronous advantage actor-critic (A3C) based joint device selection, UAVs placement, and resource management algorithm to enhance the federated convergence speed and accuracy. Simulation results demonstrate that our proposed framework and algorithm achieve higher learning accuracy and faster federated execution time compared to other existing solutions.
Lattice Fusion Networks for Image Denoising
Authors: Seyed Mohsen Hosseini
Date: 2020-11-28 00:00:00
Tasks: Denoising, Image Denoising

A novel method for feature fusion in convolutional neural networks is proposed in this paper. Different feature fusion techniques are suggested to facilitate the flow of information and improve the training of deep neural networks. Some of these techniques as well as the proposed network can be considered a type of Directed Acyclic Graph (DAG) Network, where a layer can receive inputs from other layers and have outputs to other layers. In the proposed general framework of Lattice Fusion Network (LFNet), feature maps of each convolutional layer are passed to other layers based on a lattice graph structure, where nodes are convolutional layers. To evaluate the performance of the proposed architecture, different designs based on the general framework of LFNet are implemented for the task of image denoising. This task is used as an example where training deep convolutional networks is needed. Results are compared with state of the art methods. The proposed network is able to achieve better results with far fewer learnable parameters, which shows the effectiveness of LFNets for training of deep neural networks.
LocKedge: Low-Complexity Cyberattack Detection in IoT Edge Computing
Authors: Truong Thu Huong, Ta Phuong Bac, Dao M. Long, Bui D. Thang, Nguyen T. Binh, Tran D. Luong, Tran Kim Phuc
Date: 2020-11-28 00:00:00
Tasks: Edge-computing, Federated Learning

Internet of Things and its applications are becoming commonplace with more devices, but always at risk of network security. It is therefore crucial for an IoT network design to identify attackers accurately, quickly and promptly. Many solutions have been proposed, mainly concerning secure IoT architectures and classification algorithms, but none of them have paid enough attention to reducing the complexity. Our proposal in this paper is an edge cloud architecture that fulfills the detection task right at the edge layer, near the source of the attacks for quick response, versatility, as well as reducing the workload of the cloud. We also propose a multi attack detection mechanism called LocKedge Low Complexity Cyberattack Detection in IoT Edge Computing, which has low complexity for deployment at the edge zone while still maintaining high accuracy. LocKedge is implemented in two manners: centralized and federated learning manners in order to verify the performance of the architecture from different perspectives. The performance of our proposed mechanism is compared with that of other machine learning and deep learning methods using the most updated BoT IoT data set. The results show that LocKedge outperforms other algorithms such as NN, CNN, RNN, KNN, SVM, KNN, RF and Decision Tree in terms of accuracy and NN in terms of complexity.
Using Multiple Subwords to Improve English-Esperanto Automated Literary Translation Quality
Authors: Alberto Poncelas, Jan Buts, James Hadley, Andy Way
Date: 2020-11-28 00:00:00
Proceeding: loresmt (AACL) 2020 12
Tasks: Machine Translation, Translation

Building Machine Translation (MT) systems for low-resource languages remains challenging. For many language pairs, parallel data are not widely available, and in such cases MT models do not achieve results comparable to those seen with high-resource languages. When data are scarce, it is of paramount importance to make optimal use of the limited material available. To that end, in this paper we propose employing the same parallel sentences multiple times, only changing the way the words are split each time. For this purpose we use several Byte Pair Encoding models, with various merge operations used in their configuration. In our experiments, we use this technique to expand the available data and improve an MT system involving a low-resource language pair, namely English-Esperanto. As an additional contribution, we made available a set of English-Esperanto parallel data in the literary domain.
Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning
Authors: Siyi Deng, Yang Ning, Jiwei Zhao, Heping Zhang
Date: 2020-11-28 00:00:00
Tasks: parameter estimation, regression, Vocal Bursts Intensity Prediction

We consider the estimation problem in high-dimensional semi-supervised learning. Our goal is to investigate when and how the unlabeled data can be exploited to improve the estimation of the regression parameters of linear model in light of the fact that such linear models may be misspecified in data analysis. We first establish the minimax lower bound for parameter estimation in the semi-supervised setting, and show that this lower bound cannot be achieved by supervised estimators using the labeled data only. We propose an optimal semi-supervised estimator that can attain this lower bound and therefore improves the supervised estimators, provided that the conditional mean function can be consistently estimated with a proper rate. We further propose a safe semi-supervised estimator. We view it safe, because this estimator is always at least as good as the supervised estimators. We also extend our idea to the aggregation of multiple semi-supervised estimators caused by different misspecifications of the conditional mean function. Extensive numerical simulations and a real data analysis are conducted to illustrate our theoretical results.
Scalable Deep-Learning-Accelerated Topology Optimization for Additively Manufactured Materials
Authors: Sirui Bi, Jiaxin Zhang, Guannan Zhang
Date: 2020-11-28 00:00:00
Tasks: Deep Learning

Topology optimization (TO) is a popular and powerful computational approach for designing novel structures, materials, and devices. Two computational challenges have limited the applicability of TO to a variety of industrial applications. First, a TO problem often involves a large number of design variables to guarantee sufficient expressive power. Second, many TO problems require a large number of expensive physical model simulations, and those simulations cannot be parallelized. To address these issues, we propose a general scalable deep-learning (DL) based TO framework, referred to as SDL-TO, which utilizes parallel schemes in high performance computing (HPC) to accelerate the TO process for designing additively manufactured (AM) materials. Unlike the existing studies of DL for TO, our framework accelerates TO by learning the iterative history data and simultaneously training on the mapping between the given design and its gradient. The surrogate gradient is learned by utilizing parallel computing on multiple CPUs incorporated with a distributed DL training on multiple GPUs. The learned TO gradient enables a fast online update scheme instead of an expensive update based on the physical simulator or solver. Using a local sampling strategy, we achieve to reduce the intrinsic high dimensionality of the design space and improve the training accuracy and the scalability of the SDL-TO framework. The method is demonstrated by benchmark examples and AM materials design for heat conduction. The proposed SDL-TO framework shows competitive performance compared to the baseline methods but significantly reduces the computational cost by a speed up of around 8.6x over the standard TO implementation.
Thermodynamic Consistent Neural Networks for Learning Material Interfacial Mechanics
Authors: Jiaxin Zhang, Congjie Wei, Chenglin Wu
Date: 2020-11-28 00:00:00

For multilayer materials in thin substrate systems, interfacial failure is one of the most challenges. The traction-separation relations (TSR) quantitatively describe the mechanical behavior of a material interface undergoing openings, which is critical to understand and predict interfacial failures under complex loadings. However, existing theoretical models have limitations on enough complexity and flexibility to well learn the real-world TSR from experimental observations. A neural network can fit well along with the loading paths but often fails to obey the laws of physics, due to a lack of experimental data and understanding of the hidden physical mechanism. In this paper, we propose a thermodynamic consistent neural network (TCNN) approach to build a data-driven model of the TSR with sparse experimental data. The TCNN leverages recent advances in physics-informed neural networks (PINN) that encode prior physical information into the loss function and efficiently train the neural networks using automatic differentiation. We investigate three thermodynamic consistent principles, i.e., positive energy dissipation, steepest energy dissipation gradient, and energy conservative loading path. All of them are mathematically formulated and embedded into a neural network model with a novel defined loss function. A real-world experiment demonstrates the superior performance of TCNN, and we find that TCNN provides an accurate prediction of the whole TSR surface and significantly reduces the violated prediction against the laws of physics.
Towards Robust Partially Supervised Multi-Structure Medical Image Segmentation on Small-Scale Data
Authors: Nanqing Dong, Michael Kampffmeyer, Xiaodan Liang, Min Xu, Irina Voiculescu, Eric P. Xing
Date: 2020-11-28 00:00:00
Tasks: Data Augmentation, Image Segmentation, Medical Image Segmentation, Multi-Task Learning, Organ Segmentation, Segmentation, Semantic Segmentation

The data-driven nature of deep learning (DL) models for semantic segmentation requires a large number of pixel-level annotations. However, large-scale and fully labeled medical datasets are often unavailable for practical tasks. Recently, partially supervised methods have been proposed to utilize images with incomplete labels in the medical domain. To bridge the methodological gaps in partially supervised learning (PSL) under data scarcity, we propose Vicinal Labels Under Uncertainty (VLUU), a simple yet efficient framework utilizing the human structure similarity for partially supervised medical image segmentation. Motivated by multi-task learning and vicinal risk minimization, VLUU transforms the partially supervised problem into a fully supervised problem by generating vicinal labels. We systematically evaluate VLUU under the challenges of small-scale data, dataset shift, and class imbalance on two commonly used segmentation datasets for the tasks of chest organ segmentation and optic disc-and-cup segmentation. The experimental results show that VLUU can consistently outperform previous partially supervised models in these settings. Our research suggests a new research direction in label-efficient deep learning with partial supervision.
Inter-slice Context Residual Learning for 3D Medical Image Segmentation
Authors: Jianpeng Zhang, Yutong Xie, Yan Wang, Yong Xia
Date: 2020-11-28 00:00:00
Tasks: Brain Tumor Segmentation, Decoder, Image Segmentation, Medical Image Segmentation, Pancreas Segmentation, Segmentation, Semantic Segmentation, Tumor Segmentation

Automated and accurate 3D medical image segmentation plays an essential role in assisting medical professionals to evaluate disease progresses and make fast therapeutic schedules. Although deep convolutional neural networks (DCNNs) have widely applied to this task, the accuracy of these models still need to be further improved mainly due to their limited ability to 3D context perception. In this paper, we propose the 3D context residual network (ConResNet) for the accurate segmentation of 3D medical images. This model consists of an encoder, a segmentation decoder, and a context residual decoder. We design the context residual module and use it to bridge both decoders at each scale. Each context residual module contains both context residual mapping and context attention mapping, the formal aims to explicitly learn the inter-slice context information and the latter uses such context as a kind of attention to boost the segmentation accuracy. We evaluated this model on the MICCAI 2018 Brain Tumor Segmentation (BraTS) dataset and NIH Pancreas Segmentation (Pancreas-CT) dataset. Our results not only demonstrate the effectiveness of the proposed 3D context residual learning scheme but also indicate that the proposed ConResNet is more accurate than six top-ranking methods in brain tumor segmentation and seven top-ranking methods in pancreas segmentation. Code is available at https://git.io/ConResNet
Batch Normalization with Enhanced Linear Transformation
Authors: Yuhui Xu, Lingxi Xie, Cihang Xie, Jieru Mei, Siyuan Qiao, Wei Shen, Hongkai Xiong, Alan Yuille
Date: 2020-11-28 00:00:00

Batch normalization (BN) is a fundamental unit in modern deep networks, in which a linear transformation module was designed for improving BN's flexibility of fitting complex data distributions. In this paper, we demonstrate properly enhancing this linear transformation module can effectively improve the ability of BN. Specifically, rather than using a single neuron, we propose to additionally consider each neuron's neighborhood for calculating the outputs of the linear transformation. Our method, named BNET, can be implemented with 2-3 lines of code in most deep learning libraries. Despite the simplicity, BNET brings consistent performance gains over a wide range of backbones and visual benchmarks. Moreover, we verify that BNET accelerates the convergence of network training and enhances spatial information by assigning the important neurons with larger weights accordingly. The code is available at https://github.com/yuhuixu1993/BNET.
Towards Combating Pandemic-related Misinformation in Social Media
Authors: Isa Inuwa-Dutse
Date: 2020-11-28 00:00:00
Tasks: Misinformation

Conventional preventive measures during pandemic include social distancing and lockdown. Such measures in the time of social media brought about a new set of challenges - vulnerability to the toxic impact of online misinformation is high. A case in point is the prevailing COVID-19; as the virus propagates, so does the associated misinformation and fake news about it leading to infodemic. Since the outbreak, there has been a surge of studies investigating various aspects of the pandemic. Of interest to this chapter include studies centring on datasets from online social media platforms where the bulk of the public discourse happen. Consequently, the main goal is to support the fight against negative infodemic by (1) contributing a diverse set of curated relevant datasets (2) recommending relevant areas to study using the datasets (3) discussion on how relevant datasets, strategies and state-of-the-art IT tools can be leveraged in managing the pandemic.
A Backward SDE Method for Uncertainty Quantification in Deep Learning
Authors: Richard Archibald, Feng Bao, Yanzhao Cao, He Zhang
Date: 2020-11-28 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Uncertainty Quantification

We develop a probabilistic machine learning method, which formulates a class of stochastic neural networks by a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced under the stochastic maximum principle framework. Numerical experiments for applications of stochastic neural networks are carried out to validate the effectiveness of our methodology.