pwc-archive/papers-with-abstracts huggingface dataset:
Deep Neural Network Based Ensemble learning Algorithms for the healthcare system (diagnosis of chronic diseases)
Authors: Jafar Abdollahi, Babak Nouri-Moghaddam, Mehdi Ghazanfari
Date: 2021-03-15 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Ensemble Learning, General Classification

learning algorithms. In this paper, we review the classification algorithms used in the health care system (chronic diseases) and present the neural network-based Ensemble learning method. We briefly describe the commonly used algorithms and describe their critical properties. Materials and Methods: In this study, modern classification algorithms used in healthcare, examine the principles of these methods and guidelines, and to accurately diagnose and predict chronic diseases, superior machine learning algorithms with the neural network-based ensemble learning Is used. To do this, we use experimental data, real data on chronic patients (diabetes, heart, cancer) available on the UCI site. Results: We found that group algorithms designed to diagnose chronic diseases can be more effective than baseline algorithms. It also identifies several challenges to further advancing the classification of machine learning in the diagnosis of chronic diseases. Conclusion: The results show the high performance of the neural network-based Ensemble learning approach for the diagnosis and prediction of chronic diseases, which in this study reached 98.5, 99, and 100% accuracy, respectively.
Hybrid stacked ensemble combined with genetic algorithms for Prediction of Diabetes
Authors: Jafar Abdollahi, Babak Nouri-Moghaddam
Date: 2021-03-15 00:00:00

Diabetes is currently one of the most common, dangerous, and costly diseases in the world that is caused by an increase in blood sugar or a decrease in insulin in the body. Diabetes can have detrimental effects on people's health if diagnosed late. Today, diabetes has become one of the challenges for health and government officials. Prevention is a priority, and taking care of people's health without compromising their comfort is an essential need. In this study, the Ensemble training methodology based on genetic algorithms are used to accurately diagnose and predict the outcomes of diabetes mellitus. In this study, we use the experimental data, real data on Indian diabetics on the University of California website. Current developments in ICT, such as the Internet of Things, machine learning, and data mining, allow us to provide health strategies with more intelligent capabilities to accurately predict the outcomes of the disease in daily life and the hospital and prevent the progression of this disease and its many complications. The results show the high performance of the proposed method in diagnosing the disease, which has reached 98.8%, and 99% accuracy in this study.
Geometric Change Detection in Digital Twins using 3D Machine Learning
Authors: Tiril Sundby, Julia Maria Graham, Adil Rasheed, Mandar Tabib, Omer San
Date: 2021-03-15 00:00:00
Tasks: 3D Pose Estimation, BIG-bench Machine Learning, Change Detection, Descriptive, Motion Detection, Object, object-detection, Object Detection, Pose Estimation

Digital twins are meant to bridge the gap between real-world physical systems and virtual representations. Both stand-alone and descriptive digital twins incorporate 3D geometric models, which are the physical representations of objects in the digital replica. Digital twin applications are required to rapidly update internal parameters with the evolution of their physical counterpart. Due to an essential need for having high-quality geometric models for accurate physical representations, the storage and bandwidth requirements for storing 3D model information can quickly exceed the available storage and bandwidth capacity. In this work, we demonstrate a novel approach to geometric change detection in the context of a digital twin. We address the issue through a combined solution of Dynamic Mode Decomposition (DMD) for motion detection, YOLOv5 for object detection, and 3D machine learning for pose estimation. DMD is applied for background subtraction, enabling detection of moving foreground objects in real-time. The video frames containing detected motion are extracted and used as input to the change detection network. The object detection algorithm YOLOv5 is applied to extract the bounding boxes of detected objects in the video frames. Furthermore, the rotational pose of each object is estimated in a 3D pose estimation network. A series of convolutional neural networks conducts feature extraction from images and 3D model shapes. Then, the network outputs the estimated Euler angles of the camera orientation with respect to the object in the input image. By only storing data associated with a detected change in pose, we minimize necessary storage and bandwidth requirements while still being able to recreate the 3D scene on demand.
Robust MAML: Prioritization task buffer with adaptive learning process for model-agnostic meta-learning
Authors: Thanh Nguyen, Tung Luu, Trung Pham, Sanzhar Rakhimkul, Chang D. Yoo
Date: 2021-03-15 00:00:00
Tasks: Meta-Learning, Meta Reinforcement Learning

Model agnostic meta-learning (MAML) is a popular state-of-the-art meta-learning algorithm that provides good weight initialization of a model given a variety of learning tasks. The model initialized by provided weight can be fine-tuned to an unseen task despite only using a small amount of samples and within a few adaptation steps. MAML is simple and versatile but requires costly learning rate tuning and careful design of the task distribution which affects its scalability and generalization. This paper proposes a more robust MAML based on an adaptive learning scheme and a prioritization task buffer(PTB) referred to as Robust MAML (RMAML) for improving scalability of training process and alleviating the problem of distribution mismatch. RMAML uses gradient-based hyper-parameter optimization to automatically find the optimal learning rate and uses the PTB to gradually adjust train-ing task distribution toward testing task distribution over the course of training. Experimental results on meta reinforcement learning environments demonstrate a substantial performance gain as well as being less sensitive to hyper-parameter choice and robust to distribution mismatch.
Generating Synthetic Handwritten Historical Documents With OCR Constrained GANs
Authors: Lars Vögtlin, Manuel Drazyk, Vinaychandran Pondenkandath, Michele Alberti, Rolf Ingold
Date: 2021-03-15 00:00:00
Tasks: Optical Character Recognition (OCR), Synthetic Data Generation

We present a framework to generate synthetic historical documents with precise ground truth using nothing more than a collection of unlabeled historical images. Obtaining large labeled datasets is often the limiting factor to effectively use supervised deep learning methods for Document Image Analysis (DIA). Prior approaches towards synthetic data generation either require expertise or result in poor accuracy in the synthetic documents. To achieve high precision transformations without requiring expertise, we tackle the problem in two steps. First, we create template documents with user-specified content and structure. Second, we transfer the style of a collection of unlabeled historical images to these template documents while preserving their text and layout. We evaluate the use of our synthetic historical documents in a pre-training setting and find that we outperform the baselines (randomly initialized and pre-trained). Additionally, with visual examples, we demonstrate a high-quality synthesis that makes it possible to generate large labeled historical document datasets with precise ground truth.
Boosting ship detection in SAR images with complementary pretraining techniques
Authors: Wei Bao, Meiyu Huang, Yaqin Zhang, Yao Xu, Xuejiao Liu, Xueshuang Xiang
Date: 2021-03-15 00:00:00
Tasks: Representation Learning, SAR Ship Detection

Deep learning methods have made significant progress in ship detection in synthetic aperture radar (SAR) images. The pretraining technique is usually adopted to support deep neural networks-based SAR ship detectors due to the scarce labeled SAR images. However, directly leveraging ImageNet pretraining is hardly to obtain a good ship detector because of different imaging perspective and geometry. In this paper, to resolve the problem of inconsistent imaging perspective between ImageNet and earth observations, we propose an optical ship detector (OSD) pretraining technique, which transfers the characteristics of ships in earth observations to SAR images from a large-scale aerial image dataset. On the other hand, to handle the problem of different imaging geometry between optical and SAR images, we propose an optical-SAR matching (OSM) pretraining technique, which transfers plentiful texture features from optical images to SAR images by common representation learning on the optical-SAR matching task. Finally, observing that the OSD pretraining based SAR ship detector has a better recall on sea area while the OSM pretraining based SAR ship detector can reduce false alarms on land area, we combine the predictions of the two detectors through weighted boxes fusion to further improve detection results. Extensive experiments on four SAR ship detection datasets and two representative CNN-based detection benchmarks are conducted to show the effectiveness and complementarity of the two proposed detectors, and the state-of-the-art performance of the combination of the two detectors. The proposed method won the sixth place of ship detection in SAR images in 2020 Gaofen challenge.
Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Model
Authors: Thanh Nguyen, Tung M. Luu, Thang Vu, Chang D. Yoo
Date: 2021-03-15 00:00:00
Tasks: Contrastive Learning, Data Augmentation, reinforcement-learning, Reinforcement Learning (RL), Representation Learning

Developing an agent in reinforcement learning (RL) that is capable of performing complex control tasks directly from high-dimensional observation such as raw pixels is yet a challenge as efforts are made towards improving sample efficiency and generalization. This paper considers a learning framework for Curiosity Contrastive Forward Dynamics Model (CCFDM) in achieving a more sample-efficient RL based directly on raw pixels. CCFDM incorporates a forward dynamics model (FDM) and performs contrastive learning to train its deep convolutional neural network-based image encoder (IE) to extract conducive spatial and temporal information for achieving a more sample efficiency for RL. In addition, during training, CCFDM provides intrinsic rewards, produced based on FDM prediction error, encourages the curiosity of the RL agent to improve exploration. The diverge and less-repetitive observations provide by both our exploration strategy and data augmentation available in contrastive learning improve not only the sample efficiency but also the generalization. Performance of existing model-free RL methods such as Soft Actor-Critic built on top of CCFDM outperforms prior state-of-the-art pixel-based RL methods on the DeepMind Control Suite benchmark.
ReinforceBug: A Framework to Generate Adversarial Textual Examples
Authors: Bushra Sabir, M. Ali Babar, Raj Gaire
Date: 2021-03-11 00:00:00
Proceeding: NAACL 2021 6
Tasks: Semantic Similarity, Semantic Textual Similarity

Adversarial Examples (AEs) generated by perturbing original training examples are useful in improving the robustness of Deep Learning (DL) based models. Most prior works, generate AEs that are either unconscionable due to lexical errors or semantically or functionally deviant from original examples. In this paper, we present ReinforceBug, a reinforcement learning framework, that learns a policy that is transferable on unseen datasets and generates utility-preserving and transferable (on other models) AEs. Our results show that our method is on average 10% more successful as compared to the state-of-the-art attack TextFooler. Moreover, the target models have on average 73.64% confidence in the wrong prediction, the generated AEs preserve the functional equivalence and semantic similarity (83.38% ) to their original counterparts, and are transferable on other models with an average success rate of 46%.
Partial Differential Equations is All You Need for Generating Neural Architectures -- A Theory for Physical Artificial Intelligence Systems
Authors: Ping Guo, Kaizhu Huang, Zenglin Xu
Date: 2021-03-10 00:00:00
Tasks: All

In this work, we generalize the reaction-diffusion equation in statistical physics, Schr\"odinger equation in quantum mechanics, Helmholtz equation in paraxial optics into the neural partial differential equations (NPDE), which can be considered as the fundamental equations in the field of artificial intelligence research. We take finite difference method to discretize NPDE for finding numerical solution, and the basic building blocks of deep neural network architecture, including multi-layer perceptron, convolutional neural network and recurrent neural networks, are generated. The learning strategies, such as Adaptive moment estimation, L-BFGS, pseudoinverse learning algorithms and partial differential equation constrained optimization, are also presented. We believe it is of significance that presented clear physical image of interpretable deep neural networks, which makes it be possible for applying to analog computing device design, and pave the road to physical artificial intelligence.
Neural Networks and Denotation
Authors: Eric E. Allen
Date: 2021-03-15 00:00:00

We introduce a framework for reasoning about what meaning is captured by the neurons in a trained neural network. We provide a strategy for discovering meaning by training a second model (referred to as an observer model) to classify the state of the model it observes (an object model) in relation to attributes of the underlying dataset. We implement and evaluate observer models in the context of a specific set of classification problems, employ heat maps for visualizing the relevance of components of an object model in the context of linear observer models, and use these visualizations to extract insights about the manner in which neural networks identify salient characteristics of their inputs. We identify important properties captured decisively in trained neural networks; some of these properties are denoted by individual neurons. Finally, we observe that the label proportion of a property denoted by a neuron is dependent on the depth of a neuron within a network; we analyze these dependencies, and provide an interpretation of them.
Probabilistic Grammatical Evolution
Authors: Jessica Mégane, Nuno Lourenço, Penousal Machado
Date: 2021-03-15 00:00:00

Grammatical Evolution (GE) is one of the most popular Genetic Programming (GP) variants, and it has been used with success in several problem domains. Since the original proposal, many enhancements have been proposed to GE in order to address some of its main issues and improve its performance. In this paper we propose Probabilistic Grammatical Evolution (PGE), which introduces a new genotypic representation and new mapping mechanism for GE. Specifically, we resort to a Probabilistic Context-Free Grammar (PCFG) where its probabilities are adapted during the evolutionary process, taking into account the productions chosen to construct the fittest individual. The genotype is a list of real values, where each value represents the likelihood of selecting a derivation rule. We evaluate the performance of PGE in two regression problems and compare it with GE and Structured Grammatical Evolution (SGE). The results show that PGE has a a better performance than GE, with statistically significant differences, and achieved similar performance when comparing with SGE.
Gradient Policy on "CartPole" game and its' expansibility to F1Tenth Autonomous Vehicles
Authors: Mingwei Shi
Date: 2021-03-15 00:00:00
Tasks: Autonomous Vehicles

Policy gradient is an effective way to estimate continuous action on the environment. This paper, it about explaining the mathematical formula and code implementation. In the end, comparing between the rotation angle of the stick on CartPole , and the angle of the Autonomous vehicle when turning, and utilizing the Bicycle Model, a simple Kinematic dynamic model, are the purpose to discover the similarity between these two models, so as to facilitate the model transfer from CartPole to the F1tenth Autonomous vehicle.
A Fast Heuristic for Gateway Location in Wireless Backhaul of 5G Ultra-Dense Networks
Authors: Mital Raithatha, Aizaz U. Chaudhry, Roshdy H. M. Hafez, John W. Chinneck
Date: 2021-02-22 00:00:00
Tasks: Clustering

In 5G Ultra-Dense Networks, a distributed wireless backhaul is an attractive solution for forwarding traffic to the core. The macro-cell coverage area is divided into many small cells. A few of these cells are designated as gateways and are linked to the core by high-capacity fiber optic links. Each small cell is associated with one gateway and all small cells forward their traffic to their respective gateway through multi-hop mesh networks. We investigate the gateway location problem and show that finding near-optimal gateway locations improves the backhaul network capacity. An exact p-median integer linear program is formulated for comparison with our novel K-GA heuristic that combines a Genetic Algorithm (GA) with K-means clustering to find near-optimal gateway locations. We compare the performance of KGA with six other approaches in terms of average number of hops and backhaul network capacity at different node densities through extensive Monte Carlo simulations. All approaches are tested in various user distribution scenarios, including uniform distribution, bivariate Gaussian distribution, and cluster distribution. In all cases K-GA provides near-optimal results, achieving average number of hops and backhaul network capacity within 2% of optimal while saving an average of 95% of the execution time.
BrainNetGAN: Data augmentation of brain connectivity using generative adversarial network for dementia classification
Authors: Chao Li, Yiran Wei, Xi Chen, Carola-Bibiane Schonlieb
Date: 2021-03-10 00:00:00
Tasks: Binary Classification, Classification, Data Augmentation, Diffusion  MRI, General Classification, Generative Adversarial Network, Management

Alzheimer's disease (AD) is the most common age-related dementia. It remains a challenge to identify the individuals at risk of dementia for precise management. Brain MRI offers a noninvasive biomarker to detect brain aging. Previous evidence shows that the brain structural change detected by diffusion MRI is associated with dementia. Mounting studies has conceptualised the brain as a complex network, which has shown the utility of this approach in characterising various neurological and psychiatric disorders. Therefore, the structural connectivity shows promise in dementia classification. The proposed BrainNetGAN is a generative adversarial network variant to augment the brain structural connectivity matrices for binary dementia classification tasks. Structural connectivity matrices between separated brain regions are constructed using tractography on diffusion MRI data. The BrainNetGAN model is trained to generate fake brain connectivity matrices, which are expected to reflect latent distribution of the real brain network data. Finally, a convolutional neural network classifier is proposed for binary dementia classification. Numerical results show that the binary classification performance in the testing set was improved using the BrainNetGAN augmented dataset. The proposed methodology allows quick synthesis of an arbitrary number of augmented connectivity matrices and can be easily transferred to similar classification tasks.
Distance Metric-Based Learning with Interpolated Latent Features for Location Classification in Endoscopy Image and Video
Authors: Mohammad Reza Mohebbian, Khan A. Wahid, Anh Dinh, Paul Babyn
Date: 2021-03-15 00:00:00
Tasks: Anomaly Detection, Few-Shot Learning, image-classification, Image Classification, Meta-Learning, Metric Learning, Transfer Learning, Video Summarization

Conventional Endoscopy (CE) and Wireless Capsule Endoscopy (WCE) are known tools for diagnosing gastrointestinal (GI) tract disorders. Detecting the anatomical location of GI tract can help clinicians to determine a more appropriate treatment plan, can reduce repetitive endoscopy and is important in drug-delivery. There are few research that address detecting anatomical location of WCE and CE images using classification, mainly because of difficulty in collecting data and anotating them. In this study, we present a few-shot learning method based on distance metric learning which combines transfer-learning and manifold mixup scheme for localizing endoscopy frames and can be trained on few samples. The manifold mixup process improves few-shot learning by increasing the number of training epochs while reducing overfitting, as well as providing more accurate decision boundaries. A dataset is collected from 10 different anatomical positions of human GI tract. Two models were trained using only 78 CE and 27 WCE annotated frames to predict the location of 25700 and 1825 video frames from CE and WCE, respectively. In addition, we performed subjective evaluation using nine gastroenterologists to show the necessaity of having an AI system for localization. Various ablation studies and interpretations are performed to show the importance of each step, such effect of transfer-learning approach, and impact of manifold mixup on performance. The proposed method is also compared with various methods trained on categorical cross-entropy loss and produced better results which show that proposed method has potential to be used for endoscopy image classification.
Stack of discriminative autoencoders for multiclass anomaly detection in endoscopy images
Authors: Mohammad Reza Mohebbian, Khan A. Wahid, Paul Babyn
Date: 2021-03-15 00:00:00
Tasks: Anomaly Detection, Clustering, General Classification

Wireless Capsule Endoscopy (WCE) helps physicians examine the gastrointestinal (GI) tract noninvasively. There are few studies that address pathological assessment of endoscopy images in multiclass classification and most of them are based on binary anomaly detection or aim to detect a specific type of anomaly. Multiclass anomaly detection is challenging, especially when the dataset is poorly sampled or imbalanced. Many available datasets in endoscopy field, such as KID2, suffer from an imbalance issue, which makes it difficult to train a high-performance model. Additionally, increasing the number of classes makes classification more difficult. We proposed a multiclass classification algorithm that is extensible to any number of classes and can handle an imbalance issue. The proposed method uses multiple autoencoders where each one is trained on one class to extract features with the most discrimination from other classes. The loss function of autoencoders is set based on reconstruction, compactness, distance from other classes, and Kullback-Leibler (KL) divergence. The extracted features are clustered and then classified using an ensemble of support vector data descriptors. A total of 1,778 normal, 227 inflammation, 303 vascular, and 44 polyp images from the KID2 dataset are used for evaluation. The entire algorithm ran 5 times and achieved F1-score of 96.3 +- 0.2% and 85.0 +- 0.4% on the test set for binary and multiclass anomaly detection, respectively. The impact of each step of the algorithm was investigated by various ablation studies and the results were compared with published works. The suggested approach is a competitive option for detecting multiclass anomalies in the GI field.