pwc-archive/papers-with-abstracts huggingface dataset:
Hyperspectral Image Classification With Contrastive Graph Convolutional Network
Authors: Wentao Yu, Sheng Wan, Guangyu Li, Jian Yang, Chen Gong
Date: 2022-05-11 00:00:00
Tasks: Classification, Contrastive Learning, Hyperspectral Image Classification, image-classification, Image Classification, Representation Learning

Recently, Graph Convolutional Network (GCN) has been widely used in Hyperspectral Image (HSI) classification due to its satisfactory performance. However, the number of labeled pixels is very limited in HSI, and thus the available supervision information is usually insufficient, which will inevitably degrade the representation ability of most existing GCN-based methods. To enhance the feature representation ability, in this paper, a GCN model with contrastive learning is proposed to explore the supervision signals contained in both spectral information and spatial relations, which is termed Contrastive Graph Convolutional Network (ConGCN), for HSI classification. First, in order to mine sufficient supervision signals from spectral information, a semi-supervised contrastive loss function is utilized to maximize the agreement between different views of the same node or the nodes from the same land cover category. Second, to extract the precious yet implicit spatial relations in HSI, a graph generative loss function is leveraged to explore supplementary supervision signals contained in the graph topology. In addition, an adaptive graph augmentation technique is designed to flexibly incorporate the spectral-spatial priors of HSI, which helps facilitate the subsequent contrastive representation learning. The extensive experimental results on four typical benchmark datasets firmly demonstrate the effectiveness of the proposed ConGCN in both qualitative and quantitative aspects.
Around View Monitoring System for Hydraulic Excavators
Authors: Dong Jun Yeom, Yu Na Hong, Yoojun Kim, Hyun Seok Yoo, Youngsuk Kim
Date: 2022-04-04 00:00:00

This paper describes the Around View Monitoring (AVM) system for hydraulic excavators that prevents the safety accidents caused by blind spots and increases the operational efficiency. To verify the developed system, experiments were conducted with its prototype. The experimental results demonstrate its applicability in the field with the following values: 7m of a visual range, 15fps of image refresh rate, 300ms of working information data reception rate, and 300ms of surface condition data reception rate.
A Deep Learning Ensemble Framework for Off-Nadir Geocentric Pose Prediction
Authors: Christopher Sun, Jai Sharma, Milind Maiti
Date: 2022-05-04 00:00:00
Tasks: Change Detection, Disaster Response, Feature Engineering, Pose Prediction

Computational methods to accelerate natural disaster response include change detection, map alignment, and vision-aided navigation. Current software functions optimally only on near-nadir images, though off-nadir images are often the first sources of information following a natural disaster. The use of off-nadir images for the aforementioned tasks requires the computation of geocentric pose, which is an aerial vehicle's spatial orientation with respect to gravity. This study proposes a deep learning ensemble framework to predict geocentric pose using 5,923 near-nadir and off-nadir RGB satellite images of cities worldwide. First, a U-Net Fully Convolutional Neural Network predicts the pixel-wise above-ground elevation mask of the RGB images. Then, the elevation masks are concatenated with the RGB images to form four-channel inputs fed into a second convolutional model, which predicts orientation angle and magnification scale. A performance accuracy of R2=0.917 significantly outperforms previous methodologies. In addition, outlier removal is performed through supervised interpolation, and a sensitivity analysis of elevation masks is conducted to gauge the usefulness of data features, motivating future avenues of feature engineering. The high-accuracy software built in this study contributes to mapping and navigation procedures for effective disaster response to save lives.
2-d signature of images and texture classification
Authors: Sheng Zhang, Guang Lin, Samy Tindel
Date: 2022-05-10 00:00:00
Tasks: Classification, Object, Texture Classification

We introduce a proper notion of 2-dimensional signature for images. This object is inspired by the so-called rough paths theory, and it captures many essential features of a 2-dimensional object such as an image. It thus serves as a low-dimensional feature for pattern classification. Here we implement a simple procedure for texture classification. In this context, we show that a low dimensional set of features based on signatures produces an excellent accuracy.
Vision Transformer: Vit and its Derivatives
Authors: Zujun Fu
Date: 2022-05-12 00:00:00
Tasks: Decoder, Word Embeddings

Transformer, an attention-based encoder-decoder architecture, has not only revolutionized the field of natural language processing (NLP), but has also done some pioneering work in the field of computer vision (CV). Compared to convolutional neural networks (CNNs), the Vision Transformer (ViT) relies on excellent modeling capabilities to achieve very good performance on several benchmarks such as ImageNet, COCO, and ADE20k. ViT is inspired by the self-attention mechanism in natural language processing, where word embeddings are replaced with patch embeddings. This paper reviews the derivatives of ViT and the cross-applications of ViT with other fields.
A Novel Face-Anti Spoofing Neural Network Model For Face Recognition And Detection
Authors: Soham S. Sarpotdar
Date: 2022-05-14 00:00:00
Tasks: Face Anti-Spoofing, Face Recognition

Face Recognition (FR) systems are being used in a variety of applications, including road crossings, banking, and mobile banking. The widespread use of FR systems has raised concerns about the safety of face biometrics against spoofing attacks, which use the use of a photo or video of a legitimate user's face to gain illegal access to the resources or activities. Despite the development of several FAS or liveness detection methods (which determine whether a face is live or spoofed at the time of acquisition), the problem remains unsolved due to the difficulty of identifying discrimination and operationally reasonably priced spoof characteristics but also approaches. Additionally, certain facial portions are frequently repeated or correlate to image clutter, resulting in poor performance overall. This research proposes a face-anti-spoofing neural network model that outperforms existing models and has an efficiency of 0.89 percent.
Dual Branch Prior-SegNet: CNN for Interventional CBCT using Planning Scan and Auxiliary Segmentation Loss
Authors: Philipp Ernst, Suhita Ghosh, Georg Rose, Andreas Nürnberger
Date: 2022-05-11 00:00:00

This paper proposes an extension to the Dual Branch Prior-Net for sparse view interventional CBCT reconstruction incorporating a high quality planning scan. An additional head learns to segment interventional instruments and thus guides the reconstruction task. The prior scans are misaligned by up to +-5deg in-plane during training. Experiments show that the proposed model, Dual Branch Prior-SegNet, significantly outperforms any other evaluated model by >2.8dB PSNR. It also stays robust wrt. rotations of up to +-5.5deg.
Prediction of stent under-expansion in calcified coronary arteries using machine-learning on intravascular optical coherence tomography
Authors: Yazan Gharaibeh, Juhwan Lee, Vladislav N. Zimin, Chaitanya Kolluru, Luis A. P. Dallan, Gabriel T. R. Pereira, Armando Vergara-Martel, Justin N. Kim, Ammar Hoori, Pengfei Dong, Peshala T. Gamage, Linxia Gu, Hiram G. Bezerra, Sadeer Al-Kindi, David L. Wilson
Date: 2022-05-16 00:00:00
Tasks: regression

BACKGROUND Careful evaluation of the risk of stent under-expansions before the intervention will aid treatment planning, including the application of a pre-stent plaque modification strategy. OBJECTIVES It remains challenging to achieve a proper stent expansion in the presence of severely calcified coronary lesions. Building on our work in deep learning segmentation, we created an automated machine learning approach that uses lesion attributes to predict stent under-expansion from pre-stent images, suggesting the need for plaque modification. METHODS Pre- and post-stent intravascular optical coherence tomography image data were obtained from 110 coronary lesions. Lumen and calcifications in pre-stent images were segmented using deep learning, and numerous features per lesion were extracted. We analyzed stent expansion along the lesion, enabling frame, segmental, and whole-lesion analyses. We trained regression models to predict the poststent lumen area and then to compute the stent expansion index (SEI). Stents with an SEI < or >/= 80% were classified as "under-expanded" and "well-expanded," respectively. RESULTS Best performance (root-mean-square-error = 0.04+/-0.02 mm2, r = 0.94+/-0.04, p < 0.0001) was achieved when we used features from both the lumen and calcification to train a Gaussian regression model for a segmental analysis over a segment length of 31 frames. Under-expansion classification results (AUC=0.85+/-0.02) were significantly improved over other approaches. CONCLUSIONS We used calcifications and lumen features to identify lesions at risk of stent under-expansion. Results suggest that the use of pre-stent images can inform physicians of the need to apply plaque modification approaches.
A Hardware-Aware Framework for Accelerating Neural Architecture Search Across Modalities
Authors: Daniel Cummings, Anthony Sarah, Sharath Nittur Sridhar, Maciej Szankin, Juan Pablo Munoz, Sairam Sundaresan
Date: 2022-05-19 00:00:00
Tasks: Evolutionary Algorithms, image-classification, Image Classification, Machine Translation, Neural Architecture Search

Recent advances in Neural Architecture Search (NAS) such as one-shot NAS offer the ability to extract specialized hardware-aware sub-network configurations from a task-specific super-network. While considerable effort has been employed towards improving the first stage, namely, the training of the super-network, the search for derivative high-performing sub-networks is still under-explored. Popular methods decouple the super-network training from the sub-network search and use performance predictors to reduce the computational burden of searching on different hardware platforms. We propose a flexible search framework that automatically and efficiently finds optimal sub-networks that are optimized for different performance metrics and hardware configurations. Specifically, we show how evolutionary algorithms can be paired with lightly trained objective predictors in an iterative cycle to accelerate architecture search in a multi-objective setting for various modalities including machine translation and image classification.
Manifold-aligned Neighbor Embedding
Authors: Mohammad Tariqul Islam, Jason W. Fleischer
Date: 2022-05-19 00:00:00

In this paper, we introduce a neighbor embedding framework for manifold alignment. We demonstrate the efficacy of the framework using a manifold-aligned version of the uniform manifold approximation and projection algorithm. We show that our algorithm can learn an aligned manifold that is visually competitive to embedding of the whole dataset.
Iterative Thresholding Algorithm for Sparse Inverse Covariance Estimation
Authors: Dominique Guillot, Bala Rajaratnam, Benjamin T. Rolfs, Arian Maleki, Ian Wong
Date: 2012-11-12 00:00:00
Proceeding: NeurIPS 2012 12

The L1-regularized maximum likelihood estimation problem has recently become a topic of great interest within the machine learning, statistics, and optimization communities as a method for producing sparse inverse covariance estimators. In this paper, a proximal gradient method (G-ISTA) for performing L1-regularized covariance matrix estimation is presented. Although numerous algorithms have been proposed for solving this problem, this simple proximal gradient method is found to have attractive theoretical and numerical properties. G-ISTA has a linear rate of convergence, resulting in an O(log e) iteration complexity to reach a tolerance of e. This paper gives eigenvalue bounds for the G-ISTA iterates, providing a closed-form linear convergence rate. The rate is shown to be closely related to the condition number of the optimal point. Numerical convergence results and timing comparisons for the proposed method are presented. G-ISTA is shown to perform very well, especially when the optimal point is well-conditioned.
Scale Invariance of Immune System Response Rates and Times: Perspectives on Immune System Architecture and Implications for Artificial Immune Systems
Authors: Soumya Banerjee, Melanie Moses
Date: 2010-08-08 00:00:00

Most biological rates and times decrease systematically with organism body size. We use an ordinary differential equation (ODE) model of West Nile Virus in birds to show that pathogen replication rates decline with host body size, but natural immune system (NIS) response rates do not change systematically with body size. This is surprising since the NIS has to search for small quantities of pathogens through larger physical spaces in larger organisms, and also respond by producing larger absolute quantities of antibody in larger organisms. We call this scale-invariant detection and response. We hypothesize that the NIS has evolved an architecture to efficiently neutralize pathogens. We investigate a range of architectures using an Agent Based Model (ABM). We find that a sub-modular NIS architecture, in which lymph node number and size both increase sublinearly with body size, efficiently balances the tradeoff between local pathogen detection and global response using antibodies. This leads to nearly scale-invariant detection and response, consistent with experimental data. Similar to the NIS, physical space and resources are also important constraints on Artificial Immune Systems (AIS), especially distributed systems applications used to connect low-powered sensors using short-range wireless communication. We show that AIS problems, like distributed robot control, will also require a sub-modular architecture to efficiently balance the tradeoff between local search for a solution and global response or proliferation of the solution between different components. This research has wide applicability in other distributed systems AIS applications.
AI-based thermal bridge detection of building rooftops on district scale using aerial images
Authors: Zoe Mayer, Yu Hou, James Kahn, Rebekka Volk, Frank Schultmann
Date: 2021-08-09 00:00:00
Proceeding: EG-ICE Workshop on Intelligent Computing in Engineering 2021 8
Tasks: 2D Object Detection, Instance Segmentation, Object Detection

Thermal bridges are weak areas of building envelopes that conduct more heat to the outside than surrounding envelope areas. They lead to increased energy consumption and the formation of mold. With a neural network approach, we demonstrate a method of automatically detecting thermal bridges on building rooftops from panorama drone images of whole city districts. To train the neural network, we created a dataset including 917 images and 6895 annotations. The images in the dataset contain thermal information for detecting thermal bridges and a height map for rooftop recognition in addition to regular RGB information. Due to the small dataset, our approach currently only has an average recall of 9.4% @IoU:0.5-0.95 (14.4% for large objects). Nevertheless, our approach reliably detects structures only on rooftops and not on other parts of buildings, without any additional segmentation effort of building parts.
Towards a Defense Against Federated Backdoor Attacks Under Continuous Training
Authors: Shuaiqi Wang, Jonathan Hayase, Giulia Fanti, Sewoong Oh
Date: 2022-05-24 00:00:00
Tasks: Continual Learning, Federated Learning

Backdoor attacks are dangerous and difficult to prevent in federated learning (FL), where training data is sourced from untrusted clients over long periods of time. These difficulties arise because: (a) defenders in FL do not have access to raw training data, and (b) a new phenomenon we identify called backdoor leakage causes models trained continuously to eventually suffer from backdoors due to cumulative errors in defense mechanisms. We propose shadow learning, a framework for defending against backdoor attacks in the FL setting under long-range training. Shadow learning trains two models in parallel: a backbone model and a shadow model. The backbone is trained without any defense mechanism to obtain good performance on the main task. The shadow model combines filtering of malicious clients with early-stopping to control the attack success rate even as the data distribution changes. We theoretically motivate our design and show experimentally that our framework significantly improves upon existing defenses against backdoor attacks.
Compressing Deep Graph Neural Networks via Adversarial Knowledge Distillation
Authors: Huarui He, Jie Wang, Zhanqiu Zhang, Feng Wu
Date: 2022-05-24 00:00:00
Tasks: Graph Classification, Knowledge Distillation, Node Classification

Deep graph neural networks (GNNs) have been shown to be expressive for modeling graph-structured data. Nevertheless, the over-stacked architecture of deep graph models makes it difficult to deploy and rapidly test on mobile or embedded systems. To compress over-stacked GNNs, knowledge distillation via a teacher-student architecture turns out to be an effective technique, where the key step is to measure the discrepancy between teacher and student networks with predefined distance functions. However, using the same distance for graphs of various structures may be unfit, and the optimal distance formulation is hard to determine. To tackle these problems, we propose a novel Adversarial Knowledge Distillation framework for graph models named GraphAKD, which adversarially trains a discriminator and a generator to adaptively detect and decrease the discrepancy. Specifically, noticing that the well-captured inter-node and inter-class correlations favor the success of deep GNNs, we propose to criticize the inherited knowledge from node-level and class-level views with a trainable discriminator. The discriminator distinguishes between teacher knowledge and what the student inherits, while the student GNN works as a generator and aims to fool the discriminator. To our best knowledge, GraphAKD is the first to introduce adversarial training to knowledge distillation in graph domains. Experiments on node-level and graph-level classification benchmarks demonstrate that GraphAKD improves the student performance by a large margin. The results imply that GraphAKD can precisely transfer knowledge from a complicated teacher GNN to a compact student GNN.
Diffuse Map Guiding Unsupervised Generative Adversarial Network for SVBRDF Estimation
Authors: Zhiyao Luo, Hongnan Chen
Date: 2022-05-24 00:00:00
Tasks: Generative Adversarial Network, SVBRDF Estimation

Reconstructing materials in the real world has always been a difficult problem in computer graphics. Accurately reconstructing the material in the real world is critical in the field of realistic rendering. Traditionally, materials in computer graphics are mapped by an artist, then mapped onto a geometric model by coordinate transformation, and finally rendered with a rendering engine to get realistic materials. For opaque objects, the industry commonly uses physical-based bidirectional reflectance distribution function (BRDF) rendering models for material modeling. The commonly used physical-based rendering models are Cook-Torrance BRDF, Disney BRDF. In this paper, we use the Cook-Torrance model to reconstruct the materials. The SVBRDF material parameters include Normal, Diffuse, Specular and Roughness. This paper presents a Diffuse map guiding material estimation method based on the Generative Adversarial Network(GAN). This method can predict plausible SVBRDF maps with global features using only a few pictures taken by the mobile phone. The main contributions of this paper are: 1) We preprocess a small number of input pictures to produce a large number of non-repeating pictures for training to reduce over-fitting. 2) We use a novel method to directly obtain the guessed diffuse map with global characteristics, which provides more prior information for the training process. 3) We improve the network architecture of the generator so that it can generate fine details of normal maps and reduce the possibility to generate over-flat normal maps. The method used in this paper can obtain prior knowledge without using dataset training, which greatly reduces the difficulty of material reconstruction and saves a lot of time to generate and calibrate datasets.
Multi-Agent Collaborative Inference via DNN Decoupling: Intermediate Feature Compression and Edge Learning
Authors: Zhiwei Hao, Guanyu Xu, Yong Luo, Han Hu, Jianping An, Shiwen Mao
Date: 2022-05-24 00:00:00
Tasks: Collaborative Inference, CPU, Feature Compression

Recently, deploying deep neural network (DNN) models via collaborative inference, which splits a pre-trained model into two parts and executes them on user equipment (UE) and edge server respectively, becomes attractive. However, the large intermediate feature of DNN impedes flexible decoupling, and existing approaches either focus on the single UE scenario or simply define tasks considering the required CPU cycles, but ignore the indivisibility of a single DNN layer. In this paper, we study the multi-agent collaborative inference scenario, where a single edge server coordinates the inference of multiple UEs. Our goal is to achieve fast and energy-efficient inference for all UEs. To achieve this goal, we first design a lightweight autoencoder-based method to compress the large intermediate feature. Then we define tasks according to the inference overhead of DNNs and formulate the problem as a Markov decision process (MDP). Finally, we propose a multi-agent hybrid proximal policy optimization (MAHPPO) algorithm to solve the optimization problem with a hybrid action space. We conduct extensive experiments with different types of networks, and the results show that our method can reduce up to 56\% of inference latency and save up to 72\% of energy consumption.
CDFKD-MFS: Collaborative Data-free Knowledge Distillation via Multi-level Feature Sharing
Authors: Zhiwei Hao, Yong Luo, Zhi Wang, Han Hu, Jianping An
Date: 2022-05-24 00:00:00
Tasks: Data-free Knowledge Distillation, Knowledge Distillation

Recently, the compression and deployment of powerful deep neural networks (DNNs) on resource-limited edge devices to provide intelligent services have become attractive tasks. Although knowledge distillation (KD) is a feasible solution for compression, its requirement on the original dataset raises privacy concerns. In addition, it is common to integrate multiple pretrained models to achieve satisfactory performance. How to compress multiple models into a tiny model is challenging, especially when the original data are unavailable. To tackle this challenge, we propose a framework termed collaborative data-free knowledge distillation via multi-level feature sharing (CDFKD-MFS), which consists of a multi-header student module, an asymmetric adversarial data-free KD module, and an attention-based aggregation module. In this framework, the student model equipped with a multi-level feature-sharing structure learns from multiple teacher models and is trained together with a generator in an asymmetric adversarial manner. When some real samples are available, the attention module adaptively aggregates predictions of the student headers, which can further improve performance. We conduct extensive experiments on three popular computer visual datasets. In particular, compared with the most competitive alternative, the accuracy of the proposed framework is 1.18\% higher on the CIFAR-100 dataset, 1.67\% higher on the Caltech-101 dataset, and 2.99\% higher on the mini-ImageNet dataset.