pwc-archive/papers-with-abstracts huggingface dataset:
Whole page recognition of historical handwriting
Authors: Hans J. G. A. Dolfing
Date: 2020-09-22 00:00:00
Tasks: Segmentation

Historical handwritten documents guard an important part of human knowledge only within reach of a few scholars and experts. Recent developments in machine learning and handwriting research have the potential of rendering this information accessible and searchable to a larger audience. To this end, we investigate an end-to-end inference approach without text localization which takes a handwritten page and transcribes its full text. No explicit character, word or line segmentation is involved in inference which is why we call this approach "segmentation free". We explore its robustness and accuracy compared to a line-by-line segmented approach based on the IAM, RODRIGO and ScribbleLens corpora, in three languages with handwriting styles spanning 400 years. We concentrate on model types and sizes which can be deployed on a hand-held or embedded device. We conclude that a whole page inference approach without text localization and segmentation is competitive.
What Do You See? Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors
Authors: Yi-Shan Lin, Wen-Chuan Lee, Z. Berkay Celik
Date: 2020-09-22 00:00:00
Tasks: Explainable artificial intelligence, Explainable Artificial Intelligence (XAI)

EXplainable AI (XAI) methods have been proposed to interpret how a deep neural network predicts inputs through model saliency explanations that highlight the parts of the inputs deemed important to arrive a decision at a specific target. However, it remains challenging to quantify correctness of their interpretability as current evaluation approaches either require subjective input from humans or incur high computation cost with automated evaluation. In this paper, we propose backdoor trigger patterns--hidden malicious functionalities that cause misclassification--to automate the evaluation of saliency explanations. Our key observation is that triggers provide ground truth for inputs to evaluate whether the regions identified by an XAI method are truly relevant to its output. Since backdoor triggers are the most important features that cause deliberate misclassification, a robust XAI method should reveal their presence at inference time. We introduce three complementary metrics for systematic evaluation of explanations that an XAI method generates and evaluate seven state-of-the-art model-free and model-specific posthoc methods through 36 models trojaned with specifically crafted triggers using color, shape, texture, location, and size. We discovered six methods that use local explanation and feature relevance fail to completely highlight trigger regions, and only a model-free approach can uncover the entire trigger region.
A Generative Adversarial Approach with Residual Learning for Dust and Scratches Artifacts Removal
Authors: Ionuţ Mironică
Date: 2020-09-22 00:00:00
Tasks: Denoising, Image Enhancement, Photo Retouching

Retouching can significantly elevate the visual appeal of photos, but many casual photographers lack the expertise to operate in a professional manner. One particularly challenging task for old photo retouching remains the removal of dust and scratches artifacts. Traditionally, this task has been completed manually with special image enhancement software and represents a tedious task that requires special know-how of photo editing applications. However, recent research utilizing Generative Adversarial Networks (GANs) has been proven to obtain good results in various automated image enhancement tasks compared to traditional methods. This motivated us to explore the use of GANs in the context of film photo editing. In this paper, we present a GAN based method that is able to remove dust and scratches errors from film scans. Specifically, residual learning is utilized to speed up the training process, as well as boost the denoising performance. An extensive evaluation of our model on a community provided dataset shows that it generalizes remarkably well, not being dependent on any particular type of image. Finally, we significantly outperform the state-of-the-art methods and software applications, providing superior results.
An embedded deep learning system for augmented reality in firefighting applications
Authors: Manish Bhattarai, Aura Rose Jensen-Curtis, Manel Martínez-Ramón
Date: 2020-09-22 00:00:00
Tasks: Decision Making

Firefighting is a dynamic activity, in which numerous operations occur simultaneously. Maintaining situational awareness (i.e., knowledge of current conditions and activities at the scene) is critical to the accurate decision-making necessary for the safe and successful navigation of a fire environment by firefighters. Conversely, the disorientation caused by hazards such as smoke and extreme heat can lead to injury or even fatality. This research implements recent advancements in technology such as deep learning, point cloud and thermal imaging, and augmented reality platforms to improve a firefighter's situational awareness and scene navigation through improved interpretation of that scene. We have designed and built a prototype embedded system that can leverage data streamed from cameras built into a firefighter's personal protective equipment (PPE) to capture thermal, RGB color, and depth imagery and then deploy already developed deep learning models to analyze the input data in real time. The embedded system analyzes and returns the processed images via wireless streaming, where they can be viewed remotely and relayed back to the firefighter using an augmented reality platform that visualizes the results of the analyzed inputs and draws the firefighter's attention to objects of interest, such as doors and windows otherwise invisible through smoke and flames.
TSV Extrusion Morphology Classification Using Deep Convolutional Neural Networks
Authors: Brendan Reidy, Golareh Jalilvand, Tengfei Jiang, Ramtin Zand
Date: 2020-09-22 00:00:00
Tasks: Classification, Data Augmentation, General Classification, Morphology classification

In this paper, we utilize deep convolutional neural networks (CNNs) to classify the morphology of through-silicon via (TSV) extrusion in three dimensional (3D) integrated circuits (ICs). TSV extrusion is a crucial reliability concern which can deform and crack interconnect layers in 3D ICs and cause device failures. Herein, the white light interferometry (WLI) technique is used to obtain the surface profile of the extruded TSVs. We have developed a program that uses raw data obtained from WLI to create a TSV extrusion morphology dataset, including TSV images with 54x54 pixels that are labeled and categorized into three morphology classes. Four CNN architectures with different network complexities are implemented and trained for TSV extrusion morphology classification application. Data augmentation and dropout approaches are utilized to realize a balance between overfitting and underfitting in the CNN models. Results obtained show that the CNN model with optimized complexity, dropout, and data augmentation can achieve a classification accuracy comparable to that of a human expert.
MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video
Authors: Donglai Xiang, Fabian Prada, Chenglei Wu, Jessica Hodgins
Date: 2020-09-22 00:00:00
Tasks: Surface Reconstruction, valid

We present a method to capture temporally coherent dynamic clothing deformation from a monocular RGB video input. In contrast to the existing literature, our method does not require a pre-scanned personalized mesh template, and thus can be applied to in-the-wild videos. To constrain the output to a valid deformation space, we build statistical deformation models for three types of clothing: T-shirt, short pants and long pants. A differentiable renderer is utilized to align our captured shapes to the input frames by minimizing the difference in both silhouette, segmentation, and texture. We develop a UV texture growing method which expands the visible texture region of the clothing sequentially in order to minimize drift in deformation tracking. We also extract fine-grained wrinkle detail from the input videos by fitting the clothed surface to the normal maps estimated by a convolutional neural network. Our method produces temporally coherent reconstruction of body and clothing from monocular video. We demonstrate successful clothing capture results from a variety of challenging videos. Extensive quantitative experiments demonstrate the effectiveness of our method on metrics including body pose error and surface reconstruction error of the clothing.
CCBlock: An Effective Use of Deep Learning for Automatic Diagnosis of COVID-19 Using X-Ray Images
Authors: Ali Al-Bawi, Karrar Ali Al-Kaabi, Mohammed Jeryo, Ahmad Al-Fatlawi
Date: 2020-09-21 00:00:00
Tasks: COVID-19 Diagnosis

Propose: Troubling countries one after another, the COVID-19 pandemic has dramatically affected the health and well-being of the world's population. The disease may continue to persist more extensively due to the increasing number of new cases daily, the rapid spread of the virus, and delay in the PCR analysis results. Therefore, it is necessary to consider developing assistive methods for detecting and diagnosing the COVID-19 to eradicate the spread of the novel coronavirus among people. Based on convolutional neural networks (CNNs), automated detection systems have shown promising results of diagnosing patients with the COVID-19 through radiography; thus, they are introduced as a workable solution to the COVID-19 diagnosis. Materials and Methods: Based on the enhancement of the classical visual geometry group (VGG) network with the convolutional COVID block (CCBlock), an efficient screening model was proposed in this study to diagnose and distinguish patients with the COVID-19 from those with pneumonia and the healthy people through radiography. The model testing dataset included 1,828 x-ray images available on public platforms. 310 images were showing confirmed COVID-19 cases, 864 images indicating pneumonia cases, and 654 images showing healthy people. Results: According to the test results, enhancing the classical VGG network with radiography provided the highest diagnosis performance and overall accuracy of 98.52% for two classes as well as accuracy of 95.34% for three classes. Conclusions: According to the results, using the enhanced VGG deep neural network can help radiologists automatically diagnose the COVID-19 through radiography.
Operator-valued formulas for Riemannian Gradient and Hessian and families of tractable metrics
Authors: Du Nguyen
Date: 2020-09-21 00:00:00
Tasks: Riemannian optimization

We provide an explicit formula for the Levi-Civita connection and Riemannian Hessian for a Riemannian manifold that is a quotient of a manifold embedded in an inner product space with a non-constant metric function. Together with a classical formula for projection, this allows us to evaluate Riemannian gradient and Hessian for several families of metrics on classical manifolds, including a family of metrics on Stiefel manifolds connecting both the constant and canonical ambient metrics with closed-form geodesics. Using these formulas, we derive Riemannian optimization frameworks on quotients of Stiefel manifolds, including flag manifolds, and a new family of complete quotient metrics on the manifold of positive-semidefinite matrices of fixed rank, considered as a quotient of a product of Stiefel and positive-definite matrix manifold with affine-invariant metrics. The method is procedural, and in many instances, the Riemannian gradient and Hessian formulas could be derived by symbolic calculus. The method extends the list of potential metrics that could be used in manifold optimization and machine learning.
Federated Learning for Computational Pathology on Gigapixel Whole Slide Images
Authors: Ming Y. Lu, Dehan Kong, Jana Lipkova, Richard J. Chen, Rajendra Singh, Drew F. K. Williamson, Tiffany Y. Chen, Faisal Mahmood
Date: 2020-09-21 00:00:00
Tasks: Deep Learning, Diagnostic, Federated Learning, Multiple Instance Learning, Privacy Preserving, Survival Prediction, Weakly-supervised Learning, whole slide images

Deep Learning-based computational pathology algorithms have demonstrated profound ability to excel in a wide array of tasks that range from characterization of well known morphological phenotypes to predicting non-human-identifiable features from histology such as molecular alterations. However, the development of robust, adaptable, and accurate deep learning-based models often rely on the collection and time-costly curation large high-quality annotated training data that should ideally come from diverse sources and patient populations to cater for the heterogeneity that exists in such datasets. Multi-centric and collaborative integration of medical data across multiple institutions can naturally help overcome this challenge and boost the model performance but is limited by privacy concerns amongst other difficulties that may arise in the complex data sharing process as models scale towards using hundreds of thousands of gigapixel whole slide images. In this paper, we introduce privacy-preserving federated learning for gigapixel whole slide images in computational pathology using weakly-supervised attention multiple instance learning and differential privacy. We evaluated our approach on two different diagnostic problems using thousands of histology whole slide images with only slide-level labels. Additionally, we present a weakly-supervised learning framework for survival prediction and patient stratification from whole slide images and demonstrate its effectiveness in a federated setting. Our results show that using federated learning, we can effectively develop accurate weakly supervised deep learning models from distributed data silos without direct data sharing and its associated complexities, while also preserving differential privacy using randomized noise generation.
Survey of explainable machine learning with visual and granular methods beyond quasi-explanations
Authors: Boris Kovalerchuk, Muhammad Aurangzeb Ahmad, Ankur Teredesai
Date: 2020-09-21 00:00:00
Tasks: BIG-bench Machine Learning, LEMMA, Time Series Analysis

This paper surveys visual methods of explainability of Machine Learning (ML) with focus on moving from quasi-explanations that dominate in ML to domain-specific explanation supported by granular visuals. ML interpretation is fundamentally a human activity and visual methods are more readily interpretable. While efficient visual representations of high-dimensional data exist, the loss of interpretable information, occlusion, and clutter continue to be a challenge, which lead to quasi-explanations. We start with the motivation and the different definitions of explainability. The paper focuses on a clear distinction between quasi-explanations and domain specific explanations, and between explainable and an actually explained ML model that are critically important for the explainability domain. We discuss foundations of interpretability, overview visual interpretability and present several types of methods to visualize the ML models. Next, we present methods of visual discovery of ML models, with the focus on interpretable models, based on the recently introduced concept of General Line Coordinates (GLC). These methods take the critical step of creating visual explanations that are not merely quasi-explanations but are also domain specific visual explanations while these methods themselves are domain-agnostic. The paper includes results on theoretical limits to preserve n-D distances in lower dimensions, based on the Johnson-Lindenstrauss lemma, point-to-point and point-to-graph GLC approaches, and real-world case studies. The paper also covers traditional visual methods for understanding ML models, which include deep learning and time series models. We show that many of these methods are quasi-explanations and need further enhancement to become domain specific explanations. We conclude with outlining open problems and current research frontiers.
ALICE: Active Learning with Contrastive Natural Language Explanations
Authors: Weixin Liang, James Zou, Zhou Yu
Date: 2020-09-22 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Active Learning, Classification, General Classification

Training a supervised neural network classifier typically requires many annotated training samples. Collecting and annotating a large number of data points are costly and sometimes even infeasible. Traditional annotation process uses a low-bandwidth human-machine communication interface: classification labels, each of which only provides several bits of information. We propose Active Learning with Contrastive Explanations (ALICE), an expert-in-the-loop training framework that utilizes contrastive natural language explanations to improve data efficiency in learning. ALICE learns to first use active learning to select the most informative pairs of label classes to elicit contrastive natural language explanations from experts. Then it extracts knowledge from these explanations using a semantic parser. Finally, it incorporates the extracted knowledge through dynamically changing the learning model's structure. We applied ALICE in two visual recognition tasks, bird species classification and social relationship classification. We found by incorporating contrastive explanations, our models outperform baseline models that are trained with 40-100% more training data. We found that adding 1 explanation leads to similar performance gain as adding 13-30 labeled training data points.
Semantic Workflows and Machine Learning for the Assessment of Carbon Storage by Urban Trees
Authors: Juan Carrillo, Daniel Garijo, Mark Crowley, Rober Carrillo, Yolanda Gil, Katherine Borda
Date: 2020-09-22 00:00:00
Tasks: BIG-bench Machine Learning

Climate science is critical for understanding both the causes and consequences of changes in global temperatures and has become imperative for decisive policy-making. However, climate science studies commonly require addressing complex interoperability issues between data, software, and experimental approaches from multiple fields. Scientific workflow systems provide unparalleled advantages to address these issues, including reproducibility of experiments, provenance capture, software reusability and knowledge sharing. In this paper, we introduce a novel workflow with a series of connected components to perform spatial data preparation, classification of satellite imagery with machine learning algorithms, and assessment of carbon stored by urban trees. To the best of our knowledge, this is the first study that estimates carbon storage for a region in Africa following the guidelines from the Intergovernmental Panel on Climate Change (IPCC).
Stochastic Neighbor Embedding with Gaussian and Student-t Distributions: Tutorial and Survey
Authors: Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray, Mark Crowley
Date: 2020-09-22 00:00:00
Tasks: Dimensionality Reduction

Stochastic Neighbor Embedding (SNE) is a manifold learning and dimensionality reduction method with a probabilistic approach. In SNE, every point is consider to be the neighbor of all other points with some probability and this probability is tried to be preserved in the embedding space. SNE considers Gaussian distribution for the probability in both the input and embedding spaces. However, t-SNE uses the Student-t and Gaussian distributions in these spaces, respectively. In this tutorial and survey paper, we explain SNE, symmetric SNE, t-SNE (or Cauchy-SNE), and t-SNE with general degrees of freedom. We also cover the out-of-sample extension and acceleration for these methods.
Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning
Authors: Alejandro R. Martinez
Date: 2020-09-22 00:00:00
Tasks: General Classification, Transfer Learning

Since December of 2019, novel coronavirus disease COVID-19 has spread around the world infecting millions of people and upending the global economy. One of the driving reasons behind its high rate of infection is due to the unreliability and lack of RT-PCR testing. At times the turnaround results span as long as a couple of days, only to yield a roughly 70% sensitivity rate. As an alternative, recent research has investigated the use of Computer Vision with Convolutional Neural Networks (CNNs) for the classification of COVID-19 from CT scans. Due to an inherent lack of available COVID-19 CT data, these research efforts have been forced to leverage the use of Transfer Learning. This commonly employed Deep Learning technique has shown to improve model performance on tasks with relatively small amounts of data, as long as the Source feature space somewhat resembles the Target feature space. Unfortunately, a lack of similarity is often encountered in the classification of medical images as publicly available Source datasets usually lack the visual features found in medical images. In this study, we propose the use of Multi-Source Transfer Learning (MSTL) to improve upon traditional Transfer Learning for the classification of COVID-19 from CT scans. With our multi-source fine-tuning approach, our models outperformed baseline models fine-tuned with ImageNet. We additionally, propose an unsupervised label creation process, which enhances the performance of our Deep Residual Networks. Our best performing model was able to achieve an accuracy of 0.893 and a Recall score of 0.897, outperforming its baseline Recall score by 9.3%.
EI-MTD:Moving Target Defense for Edge Intelligence against Adversarial Attacks
Authors: Ya-guan Qian, Qiqi Shao, Jia-min Wang, Xiang Lin, Yankai Guo, Zhaoquan Gu, Bin Wang, Chunming Wu
Date: 2020-09-19 00:00:00
Tasks: Knowledge Distillation, Scheduling

With the boom of edge intelligence, its vulnerability to adversarial attacks becomes an urgent problem. The so-called adversarial example can fool a deep learning model on the edge node to misclassify. Due to the property of transferability, the adversary can easily make a black-box attack using a local substitute model. Nevertheless, the limitation of resource of edge nodes cannot afford a complicated defense mechanism as doing on the cloud data center. To overcome the challenge, we propose a dynamic defense mechanism, namely EI-MTD. It first obtains robust member models with small size through differential knowledge distillation from a complicated teacher model on the cloud data center. Then, a dynamic scheduling policy based on a Bayesian Stackelberg game is applied to the choice of a target model for service. This dynamic defense can prohibit the adversary from selecting an optimal substitute model for black-box attacks. Our experimental result shows that this dynamic scheduling can effectively protect edge intelligence against adversarial attacks under the black-box setting.