pwc-archive/papers-with-abstracts huggingface dataset:
RSG-Net: Towards Rich Sematic Relationship Prediction for Intelligent Vehicle in Complex Environments
Authors: Yafu Tian, Alexander Carballo, Ruifeng Li, Kazuya Takeda
Date: 2022-07-16 00:00:00
Tasks: Decision Making, Position

Behavioral and semantic relationships play a vital role on intelligent self-driving vehicles and ADAS systems. Different from other research focused on trajectory, position, and bounding boxes, relationship data provides a human understandable description of the object's behavior, and it could describe an object's past and future status in an amazingly brief way. Therefore it is a fundamental method for tasks such as risk detection, environment understanding, and decision making. In this paper, we propose RSG-Net (Road Scene Graph Net): a graph convolutional network designed to predict potential semantic relationships from object proposals, and produces a graph-structured result, called "Road Scene Graph". The experimental results indicate that this network, trained on Road Scene Graph dataset, could efficiently predict potential semantic relationships among objects around the ego-vehicle.
Brain tumor detection using artificial convolutional neural networks
Authors: Javier Melchor, Balam Sotelo, Jorge Vera, Horacio Corral
Date: 2022-06-22 00:00:00

In this paper, a convolutional neural network (CNN) was used to classify NMR images of human brains with 4 different types of tumors: meningioma, glioma and pituitary gland tumors. During the training phase of this project, an accuracy of 100% was obtained, meanwhile, in the evaluation phase the precision was 96%.
Cov3d: Detection of the presence and severity of COVID-19 from CT scans using 3D ResNets
Authors: Robert Turnbull
Date: 2022-07-05 00:00:00
Tasks: Computed Tomography (CT), COVID-19 Diagnosis, Medical Image Analysis

Deep learning has been used to assist in the analysis of medical imaging. One such use is the classification of Computed Tomography (CT) scans when detecting for COVID-19 in subjects. This paper presents Cov3d, a three dimensional convolutional neural network for detecting the presence and severity of COVID19 from chest CT scans. Trained on the COV19-CT-DB dataset with human expert annotations, it achieves a macro f1 score of 0.9476 on the validation set for the task of detecting the presence of COVID19. For the task of classifying the severity of COVID19, it achieves a macro f1 score of 0.7552. Both results improve on the baseline results of the `AI-enabled Medical Image Analysis Workshop and Covid-19 Diagnosis Competition' (MIA-COV19D) in 2022.
Continual Few-Shot Learning with Adversarial Class Storage
Authors: Kun Wu, Chengxiang Yin, Jian Tang, Zhiyuan Xu, Yanzhi Wang, Dejun Yang
Date: 2022-07-10 00:00:00
Tasks: continual few-shot learning, Few-Shot Learning, Meta-Learning, Task 2

Humans have a remarkable ability to quickly and effectively learn new concepts in a continuous manner without forgetting old knowledge. Though deep learning has made tremendous successes on various computer vision tasks, it faces challenges for achieving such human-level intelligence. In this paper, we define a new problem called continual few-shot learning, in which tasks arrive sequentially and each task is associated with a few training samples. We propose Continual Meta-Learner (CML) to solve this problem. CML integrates metric-based classification and a memory-based mechanism along with adversarial learning into a meta-learning framework, which leads to the desirable properties: 1) it can quickly and effectively learn to handle a new task; 2) it overcomes catastrophic forgetting; 3) it is model-agnostic. We conduct extensive experiments on two image datasets, MiniImageNet and CIFAR100. Experimental results show that CML delivers state-of-the-art performance in terms of classification accuracy on few-shot learning tasks without catastrophic forgetting.
Lagrangian Density Space-Time Deep Neural Network Topology
Authors: Bhupesh Bishnoi
Date: 2022-06-30 00:00:00

As a network-based functional approximator, we have proposed a "Lagrangian Density Space-Time Deep Neural Networks" (LDDNN) topology. It is qualified for unsupervised training and learning to predict the dynamics of underlying physical science governed phenomena. The prototypical network respects the fundamental conservation laws of nature through the succinctly described Lagrangian and Hamiltonian density of the system by a given data-set of generalized nonlinear partial differential equations. The objective is to parameterize the Lagrangian density over a neural network and directly learn from it through data instead of hand-crafting an exact time-dependent "Action solution" of Lagrangian density for the physical system. With this novel approach, can understand and open up the information inference aspect of the "Black-box deep machine learning representation" for the physical dynamics of nature by constructing custom-tailored network interconnect topologies, activation, and loss/cost functions based on the underlying physical differential operators. This article will discuss statistical physics interpretation of neural networks in the Lagrangian and Hamiltonian domains.
Self-Explaining Deviations for Coordination
Authors: Hengyuan Hu, Samuel Sokota, David Wu, Anton Bakhtin, Andrei Lupu, Brandon Cui, Jakob N. Foerster
Date: 2022-07-13 00:00:00

Fully cooperative, partially observable multi-agent problems are ubiquitous in the real world. In this paper, we focus on a specific subclass of coordination problems in which humans are able to discover self-explaining deviations (SEDs). SEDs are actions that deviate from the common understanding of what reasonable behavior would be in normal circumstances. They are taken with the intention of causing another agent or other agents to realize, using theory of mind, that the circumstance must be abnormal. We first motivate SED with a real world example and formalize its definition. Next, we introduce a novel algorithm, improvement maximizing self-explaining deviations (IMPROVISED), to perform SEDs. Lastly, we evaluate IMPROVISED both in an illustrative toy setting and the popular benchmark setting Hanabi, where it is the first method to produce so called finesse plays, which are regarded as one of the more iconic examples of human theory of mind.
NN2Rules: Extracting Rule List from Neural Networks
Authors: G Roshan Lal, Varun Mithal
Date: 2022-07-04 00:00:00
Tasks: Decision Making, Explainable artificial intelligence, Explanation Generation

We present an algorithm, NN2Rules, to convert a trained neural network into a rule list. Rule lists are more interpretable since they align better with the way humans make decisions. NN2Rules is a decompositional approach to rule extraction, i.e., it extracts a set of decision rules from the parameters of the trained neural network model. We show that the decision rules extracted have the same prediction as the neural network on any input presented to it, and hence the same accuracy. A key contribution of NN2Rules is that it allows hidden neuron behavior to be either soft-binary (eg. sigmoid activation) or rectified linear (ReLU) as opposed to existing decompositional approaches that were developed with the assumption of soft-binary activation.
A Polyphone BERT for Polyphone Disambiguation in Mandarin Chinese
Authors: Song Zhang, Ken Zheng, Xiaoxu Zhu, Baoxiang Li
Date: 2022-07-01 00:00:00
Tasks: Polyphone disambiguation, text-to-speech, Text to Speech

Grapheme-to-phoneme (G2P) conversion is an indispensable part of the Chinese Mandarin text-to-speech (TTS) system, and the core of G2P conversion is to solve the problem of polyphone disambiguation, which is to pick up the correct pronunciation for several candidates for a Chinese polyphonic character. In this paper, we propose a Chinese polyphone BERT model to predict the pronunciations of Chinese polyphonic characters. Firstly, we create 741 new Chinese monophonic characters from 354 source Chinese polyphonic characters by pronunciation. Then we get a Chinese polyphone BERT by extending a pre-trained Chinese BERT with 741 new Chinese monophonic characters and adding a corresponding embedding layer for new tokens, which is initialized by the embeddings of source Chinese polyphonic characters. In this way, we can turn the polyphone disambiguation task into a pre-training task of the Chinese polyphone BERT. Experimental results demonstrate the effectiveness of the proposed model, and the polyphone BERT model obtain 2% (from 92.1% to 94.1%) improvement of average accuracy compared with the BERT-based classifier model, which is the prior state-of-the-art in polyphone disambiguation.
A Hybrid Model and Learning-Based Adaptive Navigation Filter
Authors: Barak Or, Itzik Klein
Date: 2022-06-14 00:00:00

The fusion between an inertial navigation system and global navigation satellite systems is regularly used in many platforms such as drones, land vehicles, and marine vessels. The fusion is commonly carried out in a model-based extended Kalman filter framework. One of the critical parameters of the filter is the process noise covariance. It is responsible for the real-time solution accuracy, as it considers both vehicle dynamics uncertainty and the inertial sensors quality. In most situations, the process noise is covariance assumed to be constant. Yet, due to vehicle dynamics and sensor measurement variations throughout the trajectory, the process noise covariance is subject to change. To cope with such situations, several adaptive model-based Kalman filters were suggested in the literature. In this paper, we propose a hybrid model and learning-based adaptive navigation filter. We rely on the model-based Kalman filter and design a deep neural network model to tune the momentary system noise covariance matrix, based only on the inertial sensor readings. Once the process noise covariance is learned, it is plugged into the well-established, model-based Kalman filter. After deriving the proposed hybrid framework, field experiment results using a quadrotor are presented and a comparison to model-based adaptive approaches is given. We show that the proposed method obtained an improvement of 25% in the position error. Furthermore, the proposed hybrid learning method can be used in any navigation filter and also in any relevant estimation problem.
Change Detection in VHR Imagery With Severe Co-Registration Errors Using Deep Learning: A Comparative Study
Authors: Viktoria Kristollari, Vassilia Karathanassi
Date: 2022-03-24 00:00:00
Proceeding: IEEE Access 2022 3
Tasks: Change Detection, Change detection for remote sensing images, Earth Observation

Change detection (CD) through Earth observation techniques can offer very significant information for monitoring tasks in a time-efficient manner. Very high-resolution (VHR) images can display objects in fine detail, thus making it possible to rapidly perceive isolated changes. However, this is a challenging task because of the increased within-class variance and geometric registration errors caused by different satellite view directions and angles. Lately, deep learning (DL) CD methods have proven very appealing for the CD problem because of their flexibility to combine and process different types of information along with the increased availability of higher processing power systems. Even though previous research has developed several notable DL methodologies, it has mostly focused on images with minor co-registration errors. Based on that, the goal of this study is to evaluate the performance of five state-of-the-art DL CD methods, two unsupervised and three supervised, on VHR images with severe co-registration errors. The methods are implemented on four urban European areas of versatile morphology. In addition, before applying the CD process, four popular automatic co-registration methods were evaluated because of the importance of this pre-processing step for the successful output of the CD problem. It was shown that phase correlation used on the Fourier-Mellin Transform produced the most satisfactory co-registration results and STANet detected building-related changes most successfully. Its success can be attributed to its particular attention mechanism and its training dataset. The rest of the co-registration and CD methods showed low performance.
Pansharpening PRISMA Data for Marine Plastic Litter Detection Using Plastic Indexes
Authors: Maria Kremezi, Viktoria Kristollari, Vassilia Karathanassi, Konstantinos Topouzelis, Pol Kolokoussis, Nicolò Taggio, Antonello Aiello, Giulio Ceriola, Enrico Barbone, Paolo Corradi
Date: 2021-04-19 00:00:00
Proceeding: IEEE Access 2021 4
Tasks: Hyperspectral Image Super-Resolution, Image Super-Resolution, Pansharpening

Hyperspectral PRISMA images are new and have not yet been evaluated for their ability to detect marine plastic litter. The hyperspectral PRISMA images have a fine spectral resolution, however, their spatial resolution is not high enough to enable the discrimination of small plastic objects in the ocean. Pansharpening with the panchromatic data enhances their spatial resolution and makes their detection capabilities a technological challenge. This study exploits, for the first time, the potential of using satellite hyperspectral data in detecting small-sized marine plastic litter. Controlled experiments with plastic targets of various sizes constructed from several materials have been conducted. The required pre-processing steps have been defined and 13 pansharpening methods have been applied and evaluated for their ability to spectrally discriminate plastics from water. Among them, the PCA-based substitution efficiently separates plastic spectra from water without producing duplicate edges, or pixelation. Plastic targets with size equivalent to 8% of the original hyperspectral image pixel coverage are easily detected. The same targets can also be observed in the panchromatic image, however, they cannot be detected solely by the panchromatic information as they are confused with other appearances. Exploiting spectra derived from the pan-sharpened hyperspectral images, an index combining methodology has been developed, which enables the detection of plastic objects. Although spectra of plastic materials present similarities with water spectra, some spectral characteristics can be utilized for producing marine plastic litter indexes. Based on these indexes, the index combining methodology has successfully detected the plastic targets and differentiated them from other materials.
Hierarchic Temporal Convolutional Network With Cross-Domain Encoder for Music Source Separation
Authors: Ying Hu, Yadong Chen, Wenzhong Yang, Liang He, Hao Huang
Date: 2022-06-30 00:00:00
Proceeding: IEEE Signal Processing Letters 2022 6
Tasks: Audio Source Separation, Music Source Separation, Time Series, Time Series Analysis

Recently, the time-domain-based methods (i.e., the method of modeling the raw waveform directly) for audio source separation have shown tremendous potential. In this paper, we propose a model which combines the complexed spectrogram domain feature and time-domain feature by a cross-domain encoder (CDE) and adopts the hierarchic temporal convolutional network (HTCN) for multiple music sources separation. The CDE is designed to enable the network to code the interactive information of the time-domain and complexed spectrogram domain features. HTCN enables it to learn the long-time series dependence effectively. We also designed a feature calibration unit (FCU) to be applied in the HTCN and adopted the multi-stage training strategy during the training stage. The ablation study demonstrates the effectiveness of each designed component in the model. We conducted the experiments on the MUSDB18 dataset. The experimental results indicate that our proposed CDE-HTCN model outperforms the top-of-the-line methods and, compared with the state-of-the-art method, DEMUCS, achieves the improvement of the average SDR score of 0.61 dB. Significantly, the improvement of the SDR score for the bass source has a sizable margin of 0.91 dB.
3D-RadViz: Three Dimensional Radial Visualization for Large-Scale Data Visualization
Authors: Abdelrahman Elewah, Abeer A. Badawi, Haytham Khalil, Shahryar Rahnamayan, Khalid Elgazzar
Date: 2021-08-09 00:00:00
Proceeding: IEEE Congress on Evolutionary Computation 2021 8
Tasks: Data Visualization

This paper presents 3D-RadViz, a visualization method for high dimensional data using a three dimensional radial visualization technique. The proposed technique extends the capabilities of the classical two dimensional radial visualization (RadViz) in order to reduce overlapping data points. For evaluation purposes, the paper applies the proposed 3D-RadViz alongside with t-SNE and a recently published 3-dimensional radial visualization technique to the same real-world datasets from the UCI machine learning repository. The contribution of the work is an interactive, deterministic, and high-performance library, implemented in Python, that can be utilized to realize better high dimensional data visualization using the proposed 3D-RadViz technique. The paper also suggests some directions for future enhancements to the proposed visualization approach.
LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injection
Authors: Zhuo Chen, Yufeng Huang, Jiaoyan Chen, Yuxia Geng, Yin Fang, Jeff Pan, Ningyu Zhang, Wen Zhang
Date: 2022-07-26 00:00:00
Tasks: Decoder, Knowledge Graphs, Question Answering, Text Generation, Visual Question Answering, Visual Question Answering (VQA)

Visual question answering (VQA) often requires an understanding of visual concepts and language semantics, which relies on external knowledge. Most existing methods exploit pre-trained language models or/and unstructured text, but the knowledge in these resources are often incomplete and noisy. Some other methods prefer to use knowledge graphs (KGs) which often have intensive structured knowledge, but the research is still quite preliminary. In this paper, we propose LaKo, a knowledge-driven VQA method via Late Knowledge-to-text Injection. To effectively incorporate an external KG, we transfer triples into textual format and propose a late injection mechanism for knowledge fusion. Finally we address VQA as a text generation task with an effective encoder-decoder paradigm, which achieves state-of-the-art results on OKVQA dataset.
Class-agnostic-Few-shot-Object-Counting
Authors: Shuo-Diao Yang, Hung-Ting Su, Winston H. Hsu, Wen-Chin Chen
Date: 2021-06-14 00:00:00
Proceeding: WACV 2021 6
Tasks: Object, Object Counting

Object counting which aims to calculate the number of total instances of the given class is a classic but crucial task that can be applied to many applications. Most of the prior works only focus on counting certain classes of objects such as people, cars, animals, etc. However, in recent years, there are lots of applications that need to get the count of the unseen class of objects such as a mechanical arm commanded to grab the novel object. In this paper, we present an effective object counting network, Class-agnostic Few-shot Object Counting Network (CFOCNet), that supports counting arbitrary classes of object unseen during training stage. Instead of counting a pre-defined class, our model is able to count instances based on input reference images and reduces the huge cost of data collection, training and parameter tuning for each new object class. Our model utilizes not only the similarity between query image and reference images but self attending the query image to learn the self-repeatedness. Using a two-stream Resnet that matches features in different scales, our network can automatically learn to aggregate different scales of the matching scores. We evaluate our method on the subset of the COCO dataset that contains 80 classes of objects and many diverse scenes. In the experiments, our network outperforms other methods including detection and some previous works by a large margin. To the best of our knowledge, we are the first that mainly focuses on few-shot object counting in the class-agnostic manner.
Task Agnostic and Post-hoc Unseen Distribution Detection
Authors: Radhika Dua, Seongjun Yang, Yixuan Li, Edward Choi
Date: 2022-07-26 00:00:00
Tasks: Anomaly Detection, Out of Distribution (OOD) Detection

Despite the recent advances in out-of-distribution(OOD) detection, anomaly detection, and uncertainty estimation tasks, there do not exist a task-agnostic and post-hoc approach. To address this limitation, we design a novel clustering-based ensembling method, called Task Agnostic and Post-hoc Unseen Distribution Detection (TAPUDD) that utilizes the features extracted from the model trained on a specific task. Explicitly, it comprises of TAP-Mahalanobis, which clusters the training datasets' features and determines the minimum Mahalanobis distance of the test sample from all clusters. Further, we propose the Ensembling module that aggregates the computation of iterative TAP-Mahalanobis for a different number of clusters to provide reliable and efficient cluster computation. Through extensive experiments on synthetic and real-world datasets, we observe that our approach can detect unseen samples effectively across diverse tasks and performs better or on-par with the existing baselines. To this end, we eliminate the necessity of determining the optimal value of the number of clusters and demonstrate that our method is more viable for large-scale classification tasks.
Offline Reinforcement Learning at Multiple Frequencies
Authors: Kaylee Burns, Tianhe Yu, Chelsea Finn, Karol Hausman
Date: 2022-07-26 00:00:00
Tasks: Offline RL, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Leveraging many sources of offline robot data requires grappling with the heterogeneity of such data. In this paper, we focus on one particular aspect of heterogeneity: learning from offline data collected at different control frequencies. Across labs, the discretization of controllers, sampling rates of sensors, and demands of a task of interest may differ, giving rise to a mixture of frequencies in an aggregated dataset. We study how well offline reinforcement learning (RL) algorithms can accommodate data with a mixture of frequencies during training. We observe that the $Q$-value propagates at different rates for different discretizations, leading to a number of learning challenges for off-the-shelf offline RL. We present a simple yet effective solution that enforces consistency in the rate of $Q$-value updates to stabilize learning. By scaling the value of $N$ in $N$-step returns with the discretization size, we effectively balance $Q$-value propagation, leading to more stable convergence. On three simulated robotic control problems, we empirically find that this simple approach outperforms na\"ive mixing by 50% on average.