pwc-archive/papers-with-abstracts huggingface dataset:
Combining Rules and Embeddings via Neuro-Symbolic AI for Knowledge Base Completion
Authors: Prithviraj Sen, Breno W. S. R. Carvalho, Ibrahim Abdelaziz, Pavan Kapanipathi, Francois Luus, Salim Roukos, Alexander Gray
Date: 2021-09-16 00:00:00
Tasks: Inductive logic programming, Knowledge Base Completion, Relation

Recent interest in Knowledge Base Completion (KBC) has led to a plethora of approaches based on reinforcement learning, inductive logic programming and graph embeddings. In particular, rule-based KBC has led to interpretable rules while being comparable in performance with graph embeddings. Even within rule-based KBC, there exist different approaches that lead to rules of varying quality and previous work has not always been precise in highlighting these differences. Another issue that plagues most rule-based KBC is the non-uniformity of relation paths: some relation sequences occur in very few paths while others appear very frequently. In this paper, we show that not all rule-based KBC models are the same and propose two distinct approaches that learn in one case: 1) a mixture of relations and the other 2) a mixture of paths. When implemented on top of neuro-symbolic AI, which learns rules by extending Boolean logic to real-valued logic, the latter model leads to superior KBC accuracy outperforming state-of-the-art rule-based KBC by 2-10% in terms of mean reciprocal rank. Furthermore, to address the non-uniformity of relation paths, we combine rule-based KBC with graph embeddings thus improving our results even further and achieving the best of both worlds.
Decoupling Long- and Short-Term Patterns in Spatiotemporal Inference
Authors: Junfeng Hu, Yuxuan Liang, Zhencheng Fan, Li Liu, Yifang Yin, Roger Zimmermann
Date: 2021-09-16 00:00:00
Tasks: Graph Attention

Sensors are the key to environmental monitoring, which impart benefits to smart cities in many aspects, such as providing real-time air quality information to assist human decision-making. However, it is impractical to deploy massive sensors due to the expensive costs, resulting in sparse data collection. Therefore, how to get fine-grained data measurement has long been a pressing issue. In this paper, we aim to infer values at non-sensor locations based on observations from available sensors (termed spatiotemporal inference), where capturing spatiotemporal relationships among the data plays a critical role. Our investigations reveal two significant insights that have not been explored by previous works. Firstly, data exhibits distinct patterns at both long- and short-term temporal scales, which should be analyzed separately. Secondly, short-term patterns contain more delicate relations including those across spatial and temporal dimensions simultaneously, while long-term patterns involve high-level temporal trends. Based on these observations, we propose to decouple the modeling of short-term and long-term patterns. Specifically, we introduce a joint spatiotemporal graph attention network to learn the relations across space and time for short-term patterns. Furthermore, we propose a graph recurrent network with a time skip strategy to alleviate the gradient vanishing problem and model the long-term dependencies. Experimental results on four public real-world datasets demonstrate that our method effectively captures both long- and short-term relations, achieving state-of-the-art performance against existing methods.
Reconstructing Cosmic Polarization Rotation with ResUNet-CMB
Authors: Eric Guzman, Joel Meyers
Date: 2021-09-20 00:00:00

Cosmic polarization rotation, which may result from parity-violating new physics or the presence of primordial magnetic fields, converts $E$-mode polarization of the cosmic microwave background (CMB) into $B$-mode polarization. Anisotropic cosmic polarization rotation leads to statistical anisotropy in CMB polarization and can be reconstructed with quadratic estimator techniques similar to those designed for gravitational lensing of the CMB. At the sensitivity of upcoming CMB surveys, lensing-induced $B$-mode polarization will act as a limiting factor in the search for anisotropic cosmic polarization rotation, meaning that an analysis which incorporates some form of delensing will be required to improve constraints on the effect with future surveys. In this paper we extend the ResUNet-CMB convolutional neural network to reconstruct anisotropic cosmic polarization rotation in the presence of gravitational lensing and patchy reionization, and we show that the network simultaneously reconstructs all three effects with variance that is lower than that from the standard quadratic estimator nearly matching the performance of an iterative reconstruction method.
BabelCalib: A Universal Approach to Calibrating Central Cameras
Authors: Yaroslava Lochman, Kostiantyn Liepieshov, Jianhui Chen, Michal Perdoch, Christopher Zach, James Pritts
Date: 2021-09-20 00:00:00
Proceeding: ICCV 2021 10
Tasks: Camera Calibration, Pose Estimation

Existing calibration methods occasionally fail for large field-of-view cameras due to the non-linearity of the underlying problem and the lack of good initial values for all parameters of the used camera model. This might occur because a simpler projection model is assumed in an initial step, or a poor initial guess for the internal parameters is pre-defined. A lot of the difficulties of general camera calibration lie in the use of a forward projection model. We side-step these challenges by first proposing a solver to calibrate the parameters in terms of a back-projection model and then regress the parameters for a target forward model. These steps are incorporated in a robust estimation framework to cope with outlying detections. Extensive experiments demonstrate that our approach is very reliable and returns the most accurate calibration parameters as measured on the downstream task of absolute pose estimation on test sets. The code is released at https://github.com/ylochman/babelcalib.
Deep Anomaly Generation: An Image Translation Approach of Synthesizing Abnormal Banded Chromosome Images
Authors: Lukas Uzolas, Javier Rico, Pierrick Coupé, Juan C. SanMiguel György Cserey
Date: 2021-09-20 00:00:00
Tasks: Data Augmentation, Image-to-Image Translation, Medical Image Analysis, Translation

Advances in deep-learning-based pipelines have led to breakthroughs in a variety of microscopy image diagnostics. However, a sufficiently big training data set is usually difficult to obtain due to high annotation costs. In the case of banded chromosome images, the creation of big enough libraries is difficult for multiple pathologies due to the rarity of certain genetic disorders. Generative Adversarial Networks (GANs) have proven to be effective in generating synthetic images and extending training data sets. In our work, we implement a conditional adversarial network that allows generation of realistic single chromosome images following user-defined banding patterns. To this end, an image-to-image translation approach based on self-generated 2D chromosome segmentation label maps is used. Our validation shows promising results when synthesizing chromosomes with seen as well as unseen banding patterns. We believe that this approach can be exploited for data augmentation of chromosome data sets with structural abnormalities. Therefore, the proposed method could help to tackle medical image analysis problems such as data simulation, segmentation, detection, or classification in the field of cytogenetics.
DEM Super-Resolution with EfficientNetV2
Authors: Bekir Z Demiray, Muhammed Sit, Ibrahim Demir
Date: 2021-09-20 00:00:00
Tasks: Super-Resolution

Efficient climate change monitoring and modeling rely on high-quality geospatial and environmental datasets. Due to limitations in technical capabilities or resources, the acquisition of high-quality data for many environmental disciplines is costly. Digital Elevation Model (DEM) datasets are such examples whereas their low-resolution versions are widely available, high-resolution ones are scarce. In an effort to rectify this problem, we propose and assess an EfficientNetV2 based model. The proposed model increases the spatial resolution of DEMs up to 16times without additional information.
Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR
Authors: Ziyue Feng, Longlong Jing, Peng Yin, YingLi Tian, Bing Li
Date: 2021-09-20 00:00:00
Tasks: 3D Object Detection, Depth Completion, Depth Estimation, Depth Prediction, Monocular 3D Object Detection, Monocular Depth Estimation, object-detection, Object Detection

Self-supervised monocular depth prediction provides a cost-effective solution to obtain the 3D location of each pixel. However, the existing approaches usually lead to unsatisfactory accuracy, which is critical for autonomous robots. In this paper, we propose FusionDepth, a novel two-stage network to advance the self-supervised monocular dense depth learning by leveraging low-cost sparse (e.g. 4-beam) LiDAR. Unlike the existing methods that use sparse LiDAR mainly in a manner of time-consuming iterative post-processing, our model fuses monocular image features and sparse LiDAR features to predict initial depth maps. Then, an efficient feed-forward refine network is further designed to correct the errors in these initial depth maps in pseudo-3D space with real-time performance. Extensive experiments show that our proposed model significantly outperforms all the state-of-the-art self-supervised methods, as well as the sparse-LiDAR-based methods on both self-supervised monocular depth prediction and completion tasks. With the accurate dense depth prediction, our model outperforms the state-of-the-art sparse-LiDAR-based method (Pseudo-LiDAR++) by more than 68% for the downstream task monocular 3D object detection on the KITTI Leaderboard. Code is available at https://github.com/AutoAILab/FusionDepth
Superquadric Object Representation for Optimization-based Semantic SLAM
Authors: Florian Tschopp, Juan Nieto, Roland Siegwart, Cesar Cadena
Date: 2021-09-20 00:00:00
Tasks: Object, Object Recognition, Open-Ended Question Answering, Semantic SLAM, Simultaneous Localization and Mapping

Introducing semantically meaningful objects to visual Simultaneous Localization And Mapping (SLAM) has the potential to improve both the accuracy and reliability of pose estimates, especially in challenging scenarios with significant view-point and appearance changes. However, how semantic objects should be represented for an efficient inclusion in optimization-based SLAM frameworks is still an open question. Superquadrics(SQs) are an efficient and compact object representation, able to represent most common object types to a high degree, and typically retrieved from 3D point-cloud data. However, accurate 3D point-cloud data might not be available in all applications. Recent advancements in machine learning enabled robust object recognition and semantic mask measurements from camera images under many different appearance conditions. We propose a pipeline to leverage such semantic mask measurements to fit SQ parameters to multi-view camera observations using a multi-stage initialization and optimization procedure. We demonstrate the system's ability to retrieve randomly generated SQ parameters from multi-view mask observations in preliminary simulation experiments and evaluate different initialization stages and cost functions.
Real-Time Trash Detection for Modern Societies using CCTV to Identifying Trash by utilizing Deep Convolutional Neural Network
Authors: Syed Muhammad Raza, Syed Muhammad Ghazi Hassan, Syed Ali Hassan, Soo Young Shin
Date: 2021-09-20 00:00:00

To protect the environment from trash pollution, especially in societies, and to take strict action against the red-handed people who throws the trash. As modern societies are developing and these societies need a modern solution to make the environment clean. Artificial intelligence (AI) evolution, especially in Deep Learning, gives an excellent opportunity to develop real-time trash detection using CCTV cameras. The inclusion of this project is real-time trash detection using a deep model of Convolutional Neural Network (CNN). It is used to obtain eight classes mask, tissue papers, shoppers, boxes, automobile parts, pampers, bottles, and juices boxes. After detecting the trash, the camera records the video of that person for ten seconds who throw trash in society. The challenging part of this paper is preparing a complex custom dataset that took too much time. The dataset consists of more than 2100 images. The CNN model was created, labeled, and trained. The detection time accuracy and average mean precision (mAP) benchmark both models' performance. In experimental phase the mAP performance and accuracy of the improved CNN model was superior in all aspects. The model is used on a CCTV camera to detect trash in real-time.
Fine-Context Shadow Detection using Shadow Removal
Authors: Jeya Maria Jose Valanarasu, Vishal M. Patel
Date: 2021-09-20 00:00:00
Tasks: Shadow Detection, Shadow Detection And Removal, Shadow Removal

Current shadow detection methods perform poorly when detecting shadow regions that are small, unclear or have blurry edges. In this work, we attempt to address this problem on two fronts. First, we propose a Fine Context-aware Shadow Detection Network (FCSD-Net), where we constraint the receptive field size and focus on low-level features to learn fine context features better. Second, we propose a new learning strategy, called Restore to Detect (R2D), where we show that when a deep neural network is trained for restoration (shadow removal), it learns meaningful features to delineate the shadow masks as well. To make use of this complementary nature of shadow detection and removal tasks, we train an auxiliary network for shadow removal and propose a complementary feature learning block (CFL) to learn and fuse meaningful features from shadow removal network to the shadow detection network. We train the proposed network, FCSD-Net, using the R2D learning strategy across multiple datasets. Experimental results on three public shadow detection datasets (ISTD, SBU and UCF) show that our method improves the shadow detection performance while being able to detect fine context better compared to the other recent methods.
Parameter Decoupling Strategy for Semi-supervised 3D Left Atrium Segmentation
Authors: Xuanting Hao, Shengbo Gao, Lijie Sheng, Jicong Zhang
Date: 2021-09-20 00:00:00
Tasks: Image Segmentation, Left Atrium Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Consistency training has proven to be an advanced semi-supervised framework and achieved promising results in medical image segmentation tasks through enforcing an invariance of the predictions over different views of the inputs. However, with the iterative updating of model parameters, the models would tend to reach a coupled state and eventually lose the ability to exploit unlabeled data. To address the issue, we present a novel semi-supervised segmentation model based on parameter decoupling strategy to encourage consistent predictions from diverse views. Specifically, we first adopt a two-branch network to simultaneously produce predictions for each image. During the training process, we decouple the two prediction branch parameters by quadratic cosine distance to construct different views in latent space. Based on this, the feature extractor is constrained to encourage the consistency of probability maps generated by classifiers under diversified features. In the overall training process, the parameters of feature extractor and classifiers are updated alternately by consistency regularization operation and decoupling operation to gradually improve the generalization performance of the model. Our method has achieved a competitive result over the state-of-the-art semi-supervised methods on the Atrial Segmentation Challenge dataset, demonstrating the effectiveness of our framework. Code is available at https://github.com/BX0903/PDC.
Audio-Visual Speech Recognition is Worth 32$\times$32$\times$8 Voxels
Authors: Dmitriy Serdyuk, Otavio Braga, Olivier Siohan
Date: 2021-09-20 00:00:00
Tasks: Audio-Visual Speech Recognition, Automatic Speech Recognition, Automatic Speech Recognition (ASR), image-classification, Image Classification, Lip Reading, speech-recognition, Speech Recognition, Visual Speech Recognition

Audio-visual automatic speech recognition (AV-ASR) introduces the video modality into the speech recognition process, often by relying on information conveyed by the motion of the speaker's mouth. The use of the video signal requires extracting visual features, which are then combined with the acoustic features to build an AV-ASR system [1]. This is traditionally done with some form of 3D convolutional network (e.g. VGG) as widely used in the computer vision community. Recently, image transformers [2] have been introduced to extract visual features useful for image classification tasks. In this work, we propose to replace the 3D convolutional visual front-end with a video transformer front-end. We train our systems on a large-scale dataset composed of YouTube videos and evaluate performance on the publicly available LRS3-TED set, as well as on a large set of YouTube videos. On a lip-reading task, the transformer-based front-end shows superior performance compared to a strong convolutional baseline. On an AV-ASR task, the transformer front-end performs as well as (or better than) the convolutional baseline. Fine-tuning our model on the LRS3-TED training set matches previous state of the art. Thus, we experimentally show the viability of the convolution-free model for AV-ASR.
Modeling Annotation Uncertainty with Gaussian Heatmaps in Landmark Localization
Authors: Franz Thaler, Christian Payer, Martin Urschler, Darko Stern
Date: 2021-09-20 00:00:00
Tasks: Decision Making, Position

In landmark localization, due to ambiguities in defining their exact position, landmark annotations may suffer from large observer variabilities, which result in uncertain annotations. To model the annotation ambiguities of the training dataset, we propose to learn anisotropic Gaussian parameters modeling the shape of the target heatmap during optimization. Furthermore, our method models the prediction uncertainty of individual samples by fitting anisotropic Gaussian functions to the predicted heatmaps during inference. Besides state-of-the-art results, our experiments on datasets of hand radiographs and lateral cephalograms also show that Gaussian functions are correlated with both localization accuracy and observer variability. As a final experiment, we show the importance of integrating the uncertainty into decision making by measuring the influence of the predicted location uncertainty on the classification of anatomical abnormalities in lateral cephalograms.
On Circuit-based Hybrid Quantum Neural Networks for Remote Sensing Imagery Classification
Authors: Alessandro Sebastianelli, Daniela A. Zaidenberg, Dario Spiller, Bertrand Le Saux, Silvia Liberata Ullo
Date: 2021-09-20 00:00:00
Tasks: Classification, Earth Observation

This article aims to investigate how circuit-based hybrid Quantum Convolutional Neural Networks (QCNNs) can be successfully employed as image classifiers in the context of remote sensing. The hybrid QCNNs enrich the classical architecture of CNNs by introducing a quantum layer within a standard neural network. The novel QCNN proposed in this work is applied to the Land Use and Land Cover (LULC) classification, chosen as an Earth Observation (EO) use case, and tested on the EuroSAT dataset used as reference benchmark. The results of the multiclass classification prove the effectiveness of the presented approach, by demonstrating that the QCNN performances are higher than the classical counterparts. Moreover, investigation of various quantum circuits shows that the ones exploiting quantum entanglement achieve the best classification scores. This study underlines the potentialities of applying quantum computing to an EO case study and provides the theoretical and experimental background for futures investigations.
Beyond Semantic to Instance Segmentation: Weakly-Supervised Instance Segmentation via Semantic Knowledge Transfer and Self-Refinement
Authors: Beomyoung Kim, Youngjoon Yoo, Chaeeun Rhee, Junmo Kim
Date: 2021-09-20 00:00:00
Proceeding: CVPR 2022 1
Tasks: Image-level Supervised Instance Segmentation, Instance Segmentation, Point-Supervised Instance Segmentation, Segmentation, Semantic Segmentation, Transfer Learning, Weakly-supervised instance segmentation, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

Weakly-supervised instance segmentation (WSIS) has been considered as a more challenging task than weakly-supervised semantic segmentation (WSSS). Compared to WSSS, WSIS requires instance-wise localization, which is difficult to extract from image-level labels. To tackle the problem, most WSIS approaches use off-the-shelf proposal techniques that require pre-training with instance or object level labels, deviating the fundamental definition of the fully-image-level supervised setting. In this paper, we propose a novel approach including two innovative components. First, we propose a semantic knowledge transfer to obtain pseudo instance labels by transferring the knowledge of WSSS to WSIS while eliminating the need for the off-the-shelf proposals. Second, we propose a self-refinement method to refine the pseudo instance labels in a self-supervised scheme and to use the refined labels for training in an online manner. Here, we discover an erroneous phenomenon, semantic drift, that occurred by the missing instances in pseudo instance labels categorized as background class. This semantic drift occurs confusion between background and instance in training and consequently degrades the segmentation performance. We term this problem as semantic drift problem and show that our proposed self-refinement method eliminates the semantic drift problem. The extensive experiments on PASCAL VOC 2012 and MS COCO demonstrate the effectiveness of our approach, and we achieve a considerable performance without off-the-shelf proposal techniques. The code is available at https://github.com/clovaai/BESTIE.
Predicting Visual Improvement after Macular Hole Surgery: a Cautionary Tale on Deep Learning with Very Limited Data
Authors: M. Godbout, A. Lachance, F. Antaki, A. Dirani, A. Durand
Date: 2021-09-20 00:00:00
Tasks: Deep Learning, regression

We investigate the potential of machine learning models for the prediction of visual improvement after macular hole surgery from preoperative data (retinal images and clinical features). Collecting our own data for the task, we end up with only 121 total samples, putting our work in the very limited data regime. We explore a variety of deep learning methods for limited data to train deep computer vision models, finding that all tested deep vision models are outperformed by a simple regression model on the clinical features. We believe this is compelling evidence of the extreme difficulty of using deep learning on very limited data.