pwc-archive/papers-with-abstracts huggingface dataset:
Comprehensive Efficiency Analysis of Machine Learning Algorithms for Developing Hardware-Based Cybersecurity Countermeasures
Authors: Darren Cobian
Date: 2022-01-05 00:00:00
Tasks: BIG-bench Machine Learning, Ensemble Learning, Malware Classification

Modern computing systems have led cyber adversaries to create more sophisticated malware than was previously available in the early days of technology. Dated detection techniques such as Anti-Virus Software (AVS) based on signature-based methods could no longer keep up with the demand that computer systems required of them. The complexity of modern malware has led to the development of contemporary detection techniques that use the machine learning field and hardware to boost the detection rates of malicious software. These new techniques use Hardware Performance Counters (HPCs) that form a digital signature of sorts. After the models are fed training data, they can reference these HPCs to classify zero-day malware samples. A problem emerges when malware with no comparable HPC values comes into contact with these new techniques. We provide an analysis of several machine learning and deep learning models that run zero-day samples and evaluate the results from the conversion of C++ algorithms to a hardware description language (HDL) used to begin a hardware implementation. Our results present a lack of accuracy from the models when running zero-day malware data as our highest detector, decision tree, was only able to reach 91.2% accuracy and had an F1-Score of 91.5% in the form of a decision tree. Next, through the Receiver Operating Curve (ROC) and area-under-the-curve (AUC), we can also determine that the algorithms did not present significant robustness as the largest AUC was only 0.819. In addition, we viewed relatively high overhead for our ensemble learning algorithm while also only having an 86.3% accuracy and 86% F1-Score. Finally, as an additional task, we adapted the one rule algorithm to fit many rules to make malware classification understandable to everyday users by allowing them to view the regulations while maintaining relatively high accuracy.
DSNet: Dynamic Skin Deformation Prediction by Recurrent Neural Network
Authors: Hyewon Seo, Kaifeng Zou, Frederic Cordier
Date: 2021-11-26 00:00:00
Tasks: Prediction

Skin dynamics contributes to the enriched realism of human body models in rendered scenes. Traditional methods rely on physics-based simulations to accurately reproduce the dynamic behavior of soft tissues. Due to the model complexity and thus the heavy computation, however, they do not directly offer practical solutions to domains where real-time performance is desirable. The quality shapes obtained by physics-based simulations are not fully exploited by example-based or more recent datadriven methods neither, with most of them having focused on the modeling of static skin shapes by leveraging quality data. To address these limitations, we present a learningbased method for dynamic skin deformation. At the core of our work is a recurrent neural network that learns to predict the nonlinear, dynamics-dependent shape change over time from pre-existing mesh deformation sequence data. Our network also learns to predict the variation of skin dynamics across different individuals with varying body shapes. After training the network delivers realistic, high-quality skin dynamics that is specific to a person in a real-time course. We obtain results that significantly saves the computational time, while maintaining comparable prediction quality compared to state-of-the-art results.
Portrait Segmentation Using Ensemble of Heterogeneous Deep-Learning Models
Authors: Yong-Woon Kim 1, *, Yung-Cheol Byun 2, * and Addapalli V. N. Krishna 3
Date: 2021-02-23 00:00:00
Proceeding: Information Theory in Signal Processing and Image Processing 2021 2
Tasks: Autonomous Vehicles, Deep Learning, Image Segmentation, Medical Image Analysis, Portrait Segmentation, Segmentation, Semantic Segmentation

Abstract: Image segmentation plays a central role in a broad range of applications, such as medical
image analysis, autonomous vehicles, video surveillance and augmented reality. Portrait segmentation, which is a subset of semantic image segmentation, is widely used as a preprocessing step in
multiple applications such as security systems, entertainment applications, video conferences, etc. A
substantial amount of deep learning-based portrait segmentation approaches have been developed,
since the performance and accuracy of semantic image segmentation have improved significantly
due to the recent introduction of deep learning technology. However, these approaches are limited to
a single portrait segmentation model. In this paper, we propose a novel approach using an ensemble
method by combining multiple heterogeneous deep-learning based portrait segmentation models to
improve the segmentation performance. The Two-Models ensemble and Three-Models ensemble,
using a simple soft voting method and weighted soft voting method, were experimented. Intersection
over Union (IoU) metric, IoU standard deviation and false prediction rate were used to evaluate
the performance. Cost efficiency was calculated to analyze the efficiency of segmentation. The
experiment results show that the proposed ensemble approach can perform with higher accuracy and
lower errors than single deep-learning-based portrait segmentation models. The results also show
that the ensemble of deep-learning models typically increases the use of memory and computing
power, although it also shows that the ensemble of deep-learning models can perform more efficiently
than a single model with higher accuracy using less memory and less computing power.
OpenIPDM: A Probabilistic Framework for Estimating the Deterioration and Effect of Interventions on Bridges
Authors: Zachary Hamida, Blanche Laurent, James-A. Goulet
Date: 2022-01-20 00:00:00

This paper describes OpenIPDM software for modelling the deterioration process of infrastructures using network-scale visual inspection data. In addition to the deterioration state estimates, OpenIPDM provides functions for quantifying the effect of interventions, estimating the service life of an intervention, and generating synthetic data for verification purposes. Each of the aforementioned functions are accessible by an interactive graphical user interface. OpenIPDM is designed based on the research work done on a network of bridges in Quebec province, so that the concepts presented in the software have been validated for applications in a real-world context. In addition, this software provides foundations for future developments in the subject area of modelling the deterioration as well as intervention planning.
Fine-Grained Trajectory-based Travel Time Estimation for Multi-city Scenarios Based on Deep Meta-Learning
Authors: Chenxing Wang, Fang Zhao, Haichao Zhang, Haiyong Luo, Yanjun Qin, Yuchen Fang
Date: 2022-01-20 00:00:00
Tasks: Decoder, Meta-Learning, Travel Time Estimation

Travel Time Estimation (TTE) is indispensable in intelligent transportation system (ITS). It is significant to achieve the fine-grained Trajectory-based Travel Time Estimation (TTTE) for multi-city scenarios, namely to accurately estimate travel time of the given trajectory for multiple city scenarios. However, it faces great challenges due to complex factors including dynamic temporal dependencies and fine-grained spatial dependencies. To tackle these challenges, we propose a meta learning based framework, MetaTTE, to continuously provide accurate travel time estimation over time by leveraging well-designed deep neural network model called DED, which consists of Data preprocessing module and Encoder-Decoder network module. By introducing meta learning techniques, the generalization ability of MetaTTE is enhanced using small amount of examples, which opens up new opportunities to increase the potential of achieving consistent performance on TTTE when traffic conditions and road networks change over time in the future. The DED model adopts an encoder-decoder network to capture fine-grained spatial and temporal representations. Extensive experiments on two real-world datasets are conducted to confirm that our MetaTTE outperforms six state-of-art baselines, and improve 29.35% and 25.93% accuracy than the best baseline on Chengdu and Porto datasets, respectively.
Adaptability of Improved NEAT in Variable Environments
Authors: Destiny Bailey
Date: 2021-10-22 00:00:00
Tasks: feature selection

A large challenge in Artificial Intelligence (AI) is training control agents that can properly adapt to variable environments. Environments in which the conditions change can cause issues for agents trying to operate in them. Building algorithms that can train agents to operate in these environments and properly deal with the changing conditions is therefore important. NeuroEvolution of Augmenting Topologies (NEAT) was a novel Genetic Algorithm (GA) when it was created, but has fallen aside with newer GAs outperforming it. This paper furthers the research on this subject by implementing various versions of improved NEAT in a variable environment to determine if NEAT can perform well in these environments. The improvements included, in every combination, are: recurrent connections, automatic feature selection, and increasing population size. The recurrent connections improvement performed extremely well. The automatic feature selection improvement was found to be detrimental to performance, and the increasing population size improvement lowered performance a small amount, but decreased computation requirements noticeably.
Cross-Domain Few-Shot Graph Classification
Authors: Kaveh Hassani
Date: 2022-01-20 00:00:00
Tasks: Classification, Cross-Domain Few-Shot, Graph Classification, Meta-Learning, Transfer Learning

We study the problem of few-shot graph classification across domains with nonequivalent feature spaces by introducing three new cross-domain benchmarks constructed from publicly available datasets. We also propose an attention-based graph encoder that uses three congruent views of graphs, one contextual and two topological views, to learn representations of task-specific information for fast adaptation, and task-agnostic information for knowledge transfer. We run exhaustive experiments to evaluate the performance of contrastive and meta-learning strategies. We show that when coupled with metric-based meta-learning frameworks, the proposed encoder achieves the best average meta-test classification accuracy across all benchmarks. The source code and data will be released here: https://github.com/kavehhassani/metagrl
Enhanced Performance of Pre-Trained Networks by Matched Augmentation Distributions
Authors: Touqeer Ahmad, Mohsen Jafarzadeh, Akshay Raj Dhamija, Ryan Rabinowitz, Steve Cruz, Chunchun Li, Terrance E. Boult
Date: 2022-01-19 00:00:00

There exists a distribution discrepancy between training and testing, in the way images are fed to modern CNNs. Recent work tried to bridge this gap either by fine-tuning or re-training the network at different resolutions. However re-training a network is rarely cheap and not always viable. To this end, we propose a simple solution to address the train-test distributional shift and enhance the performance of pre-trained models -- which commonly ship as a package with deep learning platforms \eg, PyTorch. Specifically, we demonstrate that running inference on the center crop of an image is not always the best as important discriminatory information may be cropped-off. Instead we propose to combine results for multiple random crops for a test image. This not only matches the train time augmentation but also provides the full coverage of the input image. We explore combining representation of random crops through averaging at different levels \ie, deep feature level, logit level, and softmax level. We demonstrate that, for various families of modern deep networks, such averaging results in better validation accuracy compared to using a single central crop per image. The softmax averaging results in the best performance for various pre-trained networks without requiring any re-training or fine-tuning whatsoever. On modern GPUs with batch processing, the paper's approach to inference of pre-trained networks, is essentially free as all images in a batch can all be processed at once.
The Role of Facial Expressions and Emotion in ASL
Authors: Lee Kezar, Pei Zhou
Date: 2022-01-19 00:00:00

There is little prior work on quantifying the relationships between facial expressions and emotionality in American Sign Language. In this final report, we provide two methods for studying these relationships through probability and prediction. Using a large corpus of natural signing manually annotated with facial features paired with lexical emotion datasets, we find that there exist many relationships between emotionality and the face, and that a simple classifier can predict what someone is saying in terms of broad emotional categories only by looking at the face.
Learning-by-Novel-View-Synthesis for Full-Face Appearance-Based 3D Gaze Estimation
Authors: Jiawei Qin, Takuru Shimoyama, Yusuke Sugano
Date: 2022-01-20 00:00:00
Tasks: 3D Face Reconstruction, Data Augmentation, Face Reconstruction, Gaze Estimation, Novel View Synthesis

Despite recent advances in appearance-based gaze estimation techniques, the need for training data that covers the target head pose and gaze distribution remains a crucial challenge for practical deployment. This work examines a novel approach for synthesizing gaze estimation training data based on monocular 3D face reconstruction. Unlike prior works using multi-view reconstruction, photo-realistic CG models, or generative neural networks, our approach can manipulate and extend the head pose range of existing training data without any additional requirements. We introduce a projective matching procedure to align the reconstructed 3D facial mesh with the camera coordinate system and synthesize face images with accurate gaze labels. We also propose a mask-guided gaze estimation model and data augmentation strategies to further improve the estimation accuracy by taking advantage of synthetic training data. Experiments using multiple public datasets show that our approach significantly improves the estimation performance on challenging cross-dataset settings with non-overlapping gaze distributions.
Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision
Authors: Jian Wang, Lingjie Liu, Weipeng Xu, Kripasindhu Sarkar, Diogo Luvizon, Christian Theobalt
Date: 2022-01-20 00:00:00
Proceeding: CVPR 2022 1
Tasks: 3D Human Pose Estimation, Egocentric Pose Estimation, Pose Estimation

Egocentric 3D human pose estimation with a single fisheye camera has drawn a significant amount of attention recently. However, existing methods struggle with pose estimation from in-the-wild images, because they can only be trained on synthetic data due to the unavailability of large-scale in-the-wild egocentric datasets. Furthermore, these methods easily fail when the body parts are occluded by or interacting with the surrounding scene. To address the shortage of in-the-wild data, we collect a large-scale in-the-wild egocentric dataset called Egocentric Poses in the Wild (EgoPW). This dataset is captured by a head-mounted fisheye camera and an auxiliary external camera, which provides an additional observation of the human body from a third-person perspective during training. We present a new egocentric pose estimation method, which can be trained on the new dataset with weak external supervision. Specifically, we first generate pseudo labels for the EgoPW dataset with a spatio-temporal optimization method by incorporating the external-view supervision. The pseudo labels are then used to train an egocentric pose estimation network. To facilitate the network training, we propose a novel learning strategy to supervise the egocentric features with the high-quality features extracted by a pretrained external-view pose estimation model. The experiments show that our method predicts accurate 3D poses from a single in-the-wild egocentric image and outperforms the state-of-the-art methods both quantitatively and qualitatively.
Experimental Large-Scale Jet Flames' Geometrical Features Extraction for Risk Management Using Infrared Images and Deep Learning Segmentation Methods
Authors: Carmina Pérez-Guerrero, Adriana Palacios, Gilberto Ochoa-Ruiz, Christian Mata, Joaquim Casal, Miguel Gonzalez-Mendoza, Luis Eduardo Falcón-Morales
Date: 2022-01-20 00:00:00
Tasks: Management, Semantic Segmentation

Jet fires are relatively small and have the least severe effects among the diverse fire accidents that can occur in industrial plants; however, they are usually involved in a process known as the domino effect, that leads to more severe events, such as explosions or the initiation of another fire, making the analysis of such fires an important part of risk analysis. This research work explores the application of deep learning models in an alternative approach that uses the semantic segmentation of jet fires flames to extract main geometrical attributes, relevant for fire risk assessments. A comparison is made between traditional image processing methods and some state-of-the-art deep learning models. It is found that the best approach is a deep learning architecture known as UNet, along with its two improvements, Attention UNet and UNet++. The models are then used to segment a group of vertical jet flames of varying pipe outlet diameters to extract their main geometrical characteristics. Attention UNet obtained the best general performance in the approximation of both height and area of the flames, while also showing a statistically significant difference between it and UNet++. UNet obtained the best overall performance for the approximation of the lift-off distances; however, there is not enough data to prove a statistically significant difference between Attention UNet and UNet++. The only instance where UNet++ outperformed the other models, was while obtaining the lift-off distances of the jet flames with 0.01275 m pipe outlet diameter. In general, the explored models show good agreement between the experimental and predicted values for relatively large turbulent propane jet flames, released in sonic and subsonic regimes; thus, making these radiation zones segmentation models, a suitable approach for different jet flame risk management scenarios.
GASCN: Graph Attention Shape Completion Network
Authors: Haojie Huang, ZiYi Yang, Robert Platt
Date: 2022-01-20 00:00:00
Tasks: Graph Attention

Shape completion, the problem of inferring the complete geometry of an object given a partial point cloud, is an important problem in robotics and computer vision. This paper proposes the Graph Attention Shape Completion Network (GASCN), a novel neural network model that solves this problem. This model combines a graph-based model for encoding local point cloud information with an MLP-based architecture for encoding global information. For each completed point, our model infers the normal and extent of the local surface patch which is used to produce dense yet precise shape completions. We report experiments that demonstrate that GASCN outperforms standard shape completion methods on a standard benchmark drawn from the Shapenet dataset.
Self-supervised Video Representation Learning with Cascade Positive Retrieval
Authors: Cheng-En Wu, Farley Lai, Yu Hen Hu, Asim Kadav
Date: 2022-01-20 00:00:00
Tasks: Action Recognition, Contrastive Learning, Representation Learning, Retrieval, Video Retrieval

Self-supervised video representation learning has been shown to effectively improve downstream tasks such as video retrieval and action recognition. In this paper, we present the Cascade Positive Retrieval (CPR) that successively mines positive examples w.r.t. the query for contrastive learning in a cascade of stages. Specifically, CPR exploits multiple views of a query example in different modalities, where an alternative view may help find another positive example dissimilar in the query view. We explore the effects of possible CPR configurations in ablations including the number of mining stages, the top similar example selection ratio in each stage, and progressive training with an incremental number of the final Top-k selection. The overall mining quality is measured to reflect the recall across training set classes. CPR reaches a median class mining recall of 83.3%, outperforming previous work by 5.5%. Implementation-wise, CPR is complementary to pretext tasks and can be easily applied to previous work. In the evaluation of pretraining on UCF101, CPR consistently improves existing work and even achieves state-of-the-art R@1 of 56.7% and 24.4% in video retrieval as well as 83.8% and 54.8% in action recognition on UCF101 and HMDB51. The code is available at https://github.com/necla-ml/CPR.
CELESTIAL: Classification Enabled via Labelless Embeddings with Self-supervised Telescope Image Analysis Learning
Authors: Suhas Kotha, Anirudh Koul, Siddha Ganju, Meher Kasam
Date: 2022-01-20 00:00:00
Tasks: Image Retrieval, Retrieval, Scene Classification, Self-Supervised Learning

A common class of problems in remote sensing is scene classification, a fundamentally important task for natural hazards identification, geographic image retrieval, and environment monitoring. Recent developments in this field rely label-dependent supervised learning techniques which is antithetical to the 35 petabytes of unlabelled satellite imagery in NASA GIBS. To solve this problem, we establish CELESTIAL-a self-supervised learning pipeline for effectively leveraging sparsely-labeled satellite imagery. This pipeline successfully adapts SimCLR, an algorithm that first learns image representations on unlabelled data and then fine-tunes this knowledge on the provided labels. Our results show CELESTIAL requires only a third of the labels that the supervised method needs to attain the same accuracy on an experimental dataset. The first unsupervised tier can enable applications such as reverse image search for NASA Worldview (i.e. searching similar atmospheric phenomenon over years of unlabelled data with minimal samples) and the second supervised tier can lower the necessity of expensive data annotation significantly. In the future, we hope we can generalize the CELESTIAL pipeline to other data types, algorithms, and applications.
PRMI: A Dataset of Minirhizotron Images for Diverse Plant Root Study
Authors: Weihuang Xu, Guohao Yu, Yiming Cui, Romain Gloaguen, Alina Zare, Jason Bonnette, Joel Reyes-Cabrera, Ashish Rajurkar, Diane Rowland, Roser Matamala, Julie D. Jastrow, Thomas E. Juenger, Felix B. Fritschi
Date: 2022-01-20 00:00:00
Tasks: Segmentation, Semantic Segmentation, Weakly-supervised Learning

Understanding a plant's root system architecture (RSA) is crucial for a variety of plant science problem domains including sustainability and climate adaptation. Minirhizotron (MR) technology is a widely-used approach for phenotyping RSA non-destructively by capturing root imagery over time. Precisely segmenting roots from the soil in MR imagery is a critical step in studying RSA features. In this paper, we introduce a large-scale dataset of plant root images captured by MR technology. In total, there are over 72K RGB root images across six different species including cotton, papaya, peanut, sesame, sunflower, and switchgrass in the dataset. The images span a variety of conditions including varied root age, root structures, soil types, and depths under the soil surface. All of the images have been annotated with weak image-level labels indicating whether each image contains roots or not. The image-level labels can be used to support weakly supervised learning in plant root segmentation tasks. In addition, 63K images have been manually annotated to generate pixel-level binary masks indicating whether each pixel corresponds to root or not. These pixel-level binary masks can be used as ground truth for supervised learning in semantic segmentation tasks. By introducing this dataset, we aim to facilitate the automatic segmentation of roots and the research of RSA with deep learning and other image analysis algorithms.