pwc-archive/papers-with-abstracts huggingface dataset:
Guidance and Teaching Network for Video Salient Object Detection
Authors: Yingxia Jiao, Xiao Wang, Yu-Cheng Chou, Shouyuan Yang, Ge-Peng Ji, Rong Zhu, Ge Gao
Date: 2021-05-21 00:00:00
Tasks: GPU, Object, object-detection, Object Detection, Salient Object Detection, Video Salient Object Detection

Owing to the difficulties of mining spatial-temporal cues, the existing approaches for video salient object detection (VSOD) are limited in understanding complex and noisy scenarios, and often fail in inferring prominent objects. To alleviate such shortcomings, we propose a simple yet efficient architecture, termed Guidance and Teaching Network (GTNet), to independently distil effective spatial and temporal cues with implicit guidance and explicit teaching at feature- and decision-level, respectively. To be specific, we (a) introduce a temporal modulator to implicitly bridge features from motion into the appearance branch, which is capable of fusing cross-modal features collaboratively, and (b) utilise motion-guided mask to propagate the explicit cues during the feature aggregation. This novel learning strategy achieves satisfactory results via decoupling the complex spatial-temporal cues and mapping informative cues across different modalities. Extensive experiments on three challenging benchmarks show that the proposed method can run at ~28 fps on a single TITAN Xp GPU and perform competitively against 14 cutting-edge baselines.
DSR: Direct Simultaneous Registration for Multiple 3D Images
Authors: Zhehua Mao, Liang Zhao, Shoudong Huang, Yiting Fan, Alex Pui-Wai Lee
Date: 2021-05-21 00:00:00
Tasks: Image Registration, Pose Estimation

This paper presents a novel algorithm named Direct Simultaneous Registration (DSR) that registers a collection of 3D images in a simultaneous fashion without specifying any reference image, feature extraction and matching, or information loss or reuse. The algorithm optimizes the global poses of local image frames by maximizing the similarity between a predefined panoramic image and local images. Although we formulate the problem as a Direct Bundle Adjustment (DBA) that jointly optimizes the poses of local frames and the intensities of the panoramic image, by investigating the independence of pose estimation from the panoramic image in the solving process, DSR is proposed to solve the poses only and proved to be able to obtain the same optimal poses as DBA. The proposed method is particularly suitable for the scenarios where distinct features are not available, such as Transesophageal Echocardiography (TEE) images. DSR is evaluated by comparing it with four widely used methods via simulated and in-vivo 3D TEE images. It is shown that the proposed method outperforms these four methods in terms of accuracy and requires much fewer computational resources than the state-of-the-art accumulated pairwise estimates (APE).
An interpretable object detection based model for the diagnosis of neonatal lung diseases using Ultrasound images
Authors: Rodina Bassiouny, Adel Mohamed, Karthi Umapathy, Naimul Khan
Date: 2021-05-21 00:00:00
Tasks: object-detection, Object Detection, Region Proposal

Over the last few decades, Lung Ultrasound (LUS) has been increasingly used to diagnose and monitor different lung diseases in neonates. It is a non invasive tool that allows a fast bedside examination while minimally handling the neonate. Acquiring a LUS scan is easy, but understanding the artifacts concerned with each respiratory disease is challenging. Mixed artifact patterns found in different respiratory diseases may limit LUS readability by the operator. While machine learning (ML), especially deep learning can assist in automated analysis, simply feeding the ultrasound images to an ML model for diagnosis is not enough to earn the trust of medical professionals. The algorithm should output LUS features that are familiar to the operator instead. Therefore, in this paper we present a unique approach for extracting seven meaningful LUS features that can be easily associated with a specific pathological lung condition: Normal pleura, irregular pleura, thick pleura, Alines, Coalescent B-lines, Separate B-lines and Consolidations. These artifacts can lead to early prediction of infants developing later respiratory distress symptoms. A single multi-class region proposal-based object detection model faster-RCNN (fRCNN) was trained on lower posterior lung ultrasound videos to detect these LUS features which are further linked to four common neonatal diseases. Our results show that fRCNN surpasses single stage models such as RetinaNet and can successfully detect the aforementioned LUS features with a mean average precision of 86.4%. Instead of a fully automatic diagnosis from images without any interpretability, detection of such LUS features leave the ultimate control of diagnosis to the clinician, which can result in a more trustworthy intelligent system.
An Optical physics inspired CNN approach for intrinsic image decomposition
Authors: Harshana Weligampola, Gihan Jayatilaka, Suren Sritharan, Parakrama Ekanayake, Roshan Ragel, Vijitha Herath, Roshan Godaliyadda
Date: 2021-05-21 00:00:00
Tasks: Intrinsic Image Decomposition

Intrinsic Image Decomposition is an open problem of generating the constituents of an image. Generating reflectance and shading from a single image is a challenging task specifically when there is no ground truth. There is a lack of unsupervised learning approaches for decomposing an image into reflectance and shading using a single image. We propose a neural network architecture capable of this decomposition using physics-based parameters derived from the image. Through experimental results, we show that (a) the proposed methodology outperforms the existing deep learning-based IID techniques and (b) the derived parameters improve the efficacy significantly. We conclude with a closer analysis of the results (numerical and example images) showing several avenues for improvement.
Uma implementação do jogo Pedra, Papel e Tesoura utilizando Visao Computacional
Authors: Ezequiel França dos Santos, Gabriel Fontenelle
Date: 2021-05-20 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

This paper presents a game, controlled by computer vision, in identification of hand gestures (hand-tracking). The proposed work is based on image segmentation and construction of a convex hull with Jarvis Algorithm , and determination of the pattern based on the extraction of area characteristics in the convex hull.
Opening Deep Neural Networks with Generative Models
Authors: Marcos Vendramini, Hugo Oliveira, Alexei Machado, Jefersson A. dos Santos
Date: 2021-05-20 00:00:00
Tasks: image-classification, Image Classification, Object Recognition, Open Set Learning

Image classification methods are usually trained to perform predictions taking into account a predefined group of known classes. Real-world problems, however, may not allow for a full knowledge of the input and label spaces, making failures in recognition a hazard to deep visual learning. Open set recognition methods are characterized by the ability to correctly identify inputs of known and unknown classes. In this context, we propose GeMOS: simple and plug-and-play open set recognition modules that can be attached to pretrained Deep Neural Networks for visual recognition. The GeMOS framework pairs pre-trained Convolutional Neural Networks with generative models for open set recognition to extract open set scores for each sample, allowing for failure recognition in object recognition tasks. We conduct a thorough evaluation of the proposed method in comparison with state-of-the-art open set algorithms, finding that GeMOS either outperforms or is statistically indistinguishable from more complex and costly models.
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding
Authors: Hu Xu, Gargi Ghosh, Po-Yao Huang, Prahal Arora, Masoumeh Aminzadeh, Christoph Feichtenhofer, Florian Metze, Luke Zettlemoyer
Date: 2021-05-20 00:00:00
Proceeding: Findings (ACL) 2021 8
Tasks: Action Segmentation, Language Modeling, Language Modelling, Retrieval, Temporal Action Localization, Video Captioning, Video Retrieval, Video Understanding

We present a simplified, task-agnostic multi-modal pre-training approach that can accept either video or text input, or both for a variety of end tasks. Existing pre-training are task-specific by adopting either a single cross-modal encoder that requires both modalities, limiting their use for retrieval-style end tasks or more complex multitask learning with two unimodal encoders, limiting early cross-modal fusion. We instead introduce new pretraining masking schemes that better mix across modalities (e.g. by forcing masks for text to predict the closest video embeddings) while also maintaining separability (e.g. unimodal predictions are sometimes required, without using all the input). Experimental results show strong performance across a wider range of tasks than any previous methods, often outperforming task-specific pre-training. Code is made available at https://github.com/pytorch/fairseq/tree/main/examples/MMPT.
Dense Reconstruction of Transparent Objects by Altering Incident Light Paths Through Refraction
Authors: Kai Han, Kwan-Yee K. Wong, Miaomiao Liu
Date: 2021-05-20 00:00:00
Tasks: Object, Surface Reconstruction, Transparent objects

This paper addresses the problem of reconstructing the surface shape of transparent objects. The difficulty of this problem originates from the viewpoint dependent appearance of a transparent object, which quickly makes reconstruction methods tailored for diffuse surfaces fail disgracefully. In this paper, we introduce a fixed viewpoint approach to dense surface reconstruction of transparent objects based on refraction of light. We present a simple setup that allows us to alter the incident light paths before light rays enter the object by immersing the object partially in a liquid, and develop a method for recovering the object surface through reconstructing and triangulating such incident light paths. Our proposed approach does not need to model the complex interactions of light as it travels through the object, neither does it assume any parametric form for the object shape nor the exact number of refractions and reflections taken place along the light paths. It can therefore handle transparent objects with a relatively complex shape and structure, with unknown and inhomogeneous refractive index. We also show that for thin transparent objects, our proposed acquisition setup can be further simplified by adopting a single refraction approximation. Experimental results on both synthetic and real data demonstrate the feasibility and accuracy of our proposed approach.
Happy Dance, Slow Clap: Using Reaction GIFs to Predict Induced Affect on Twitter
Authors: Boaz Shmueli, Soumya Ray, Lun-Wei Ku
Date: 2021-05-20 00:00:00
Proceeding: ACL 2021 5

Datasets with induced emotion labels are scarce but of utmost importance for many NLP tasks. We present a new, automated method for collecting texts along with their induced reaction labels. The method exploits the online use of reaction GIFs, which capture complex affective states. We show how to augment the data with induced emotion and induced sentiment labels. We use our method to create and publish ReactionGIF, a first-of-its-kind affective dataset of 30K tweets. We provide baselines for three new tasks, including induced sentiment prediction and multilabel classification of induced emotions. Our method and dataset open new research opportunities in emotion detection and affective computing.
Error Resilient Collaborative Intelligence via Low-Rank Tensor Completion
Authors: Lior Bragilevsky, Ivan V. Bajić
Date: 2021-05-20 00:00:00

In the race to bring Artificial Intelligence (AI) to the edge, collaborative intelligence has emerged as a promising way to lighten the computation load on edge devices that run applications based on Deep Neural Networks (DNNs). Typically, a deep model is split at a certain layer into edge and cloud sub-models. The deep feature tensor produced by the edge sub-model is transmitted to the cloud, where the remaining computationally intensive workload is performed by the cloud sub-model. The communication channel between the edge and cloud is imperfect, which will result in missing data in the deep feature tensor received at the cloud side. In this study, we examine the effectiveness of four low-rank tensor completion methods in recovering missing data in the deep feature tensor. We consider both sparse tensors, such as those produced by the VGG16 model, as well as non-sparse tensors, such as those produced by ResNet34 model. We study tensor completion effectiveness in both conplexity-constrained and unconstrained scenario.
Correlated Input-Dependent Label Noise in Large-Scale Image Classification
Authors: Mark Collier, Basil Mustafa, Efi Kokiopoulou, Rodolphe Jenatton, Jesse Berent
Date: 2021-05-19 00:00:00
Proceeding: CVPR 2021 1
Tasks: Classification, image-classification, Image Classification

Large scale image classification datasets often contain noisy labels. We take a principled probabilistic approach to modelling input-dependent, also known as heteroscedastic, label noise in these datasets. We place a multivariate Normal distributed latent variable on the final hidden layer of a neural network classifier. The covariance matrix of this latent variable, models the aleatoric uncertainty due to label noise. We demonstrate that the learned covariance structure captures known sources of label noise between semantically similar and co-occurring classes. Compared to standard neural network training and other baselines, we show significantly improved accuracy on Imagenet ILSVRC 2012 79.3% (+2.6%), Imagenet-21k 47.0% (+1.1%) and JFT 64.7% (+1.6%). We set a new state-of-the-art result on WebVision 1.0 with 76.6% top-1 accuracy. These datasets range from over 1M to over 300M training examples and from 1k classes to more than 21k classes. Our method is simple to use, and we provide an implementation that is a drop-in replacement for the final fully-connected layer in a deep classifier.
Spatial-temporal Conv-sequence Learning with Accident Encoding for Traffic Flow Prediction
Authors: Zichuan Liu, Rui Zhang, Chen Wang, Zhu Xiao, Hongbo Jiang
Date: 2021-05-21 00:00:00

In an intelligent transportation system, the key problem of traffic forecasting is how to extract periodic temporal dependencies and complex spatial correlations. Current state-of-the-art methods for predicting traffic flow are based on graph architectures and sequence learning models, but they do not fully exploit dynamic spatial-temporal information in the traffic system. Specifically, the temporal dependencies in the short-range are diluted by recurrent neural networks. Moreover, local spatial information is also ignored by existing sequence models, because their convolution operation uses global average pooling. Besides, accidents may occur during object transition, which will cause congestion in the real world and further decrease prediction accuracy. To overcome these challenges, we propose Spatial-Temporal Conv-sequence Learning (STCL), where a focused temporal block uses unidirectional convolution to capture short-term periodic temporal dependencies effectively, and a patial-temporal fusion module is responsible for extracting dependencies of interactions and decreasing the feature dimensions. Moreover, as the accidents features have an impact on local traffic congestion, we employ position encoding to detect anomalies in complex traffic situations. We have conducted a large number of experiments on real-world tasks and verified the effectiveness of our proposed method.
Generalization Error Bound for Hyperbolic Ordinal Embedding
Authors: Atsushi Suzuki, Atsushi Nitanda, Jing Wang, Linchuan Xu, Marc Cavazza, Kenji Yamanishi
Date: 2021-05-21 00:00:00

Hyperbolic ordinal embedding (HOE) represents entities as points in hyperbolic space so that they agree as well as possible with given constraints in the form of entity i is more similar to entity j than to entity k. It has been experimentally shown that HOE can obtain representations of hierarchical data such as a knowledge base and a citation network effectively, owing to hyperbolic space's exponential growth property. However, its theoretical analysis has been limited to ideal noiseless settings, and its generalization error in compensation for hyperbolic space's exponential representation ability has not been guaranteed. The difficulty is that existing generalization error bound derivations for ordinal embedding based on the Gramian matrix do not work in HOE, since hyperbolic space is not inner-product space. In this paper, through our novel characterization of HOE with decomposed Lorentz Gramian matrices, we provide a generalization error bound of HOE for the first time, which is at most exponential with respect to the embedding space's radius. Our comparison between the bounds of HOE and Euclidean ordinal embedding shows that HOE's generalization error is reasonable as a cost for its exponential representation ability.
Automated Detection of Abnormalities from an EEG Recording of Epilepsy Patients With a Compact Convolutional Neural Network
Authors: Taku Shoji, Noboru Yoshida, Toshihisa Tanaka
Date: 2021-05-21 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

Electroencephalography (EEG) is essential for the diagnosis of epilepsy, but it requires expertise and experience to identify abnormalities. It is thus crucial to develop automated models for the detection of abnormalities in EEGs related to epilepsy. This paper describes the development of a novel class of compact convolutional neural networks (CNNs) for detecting abnormal patterns and electrodes in EEGs for epilepsy. The designed model is inspired by a CNN developed for brain-computer interfacing called multichannel EEGNet (mEEGNet). Unlike the EEGNet, the proposed model, mEEGNet, has the same number of electrode inputs and outputs to detect abnormal patterns. The mEEGNet was evaluated with a clinical dataset consisting of 29 cases of juvenile and childhood absence epilepsy labeled by a clinical expert. The labels were given to paroxysmal discharges visually observed in both ictal (seizure) and interictal (nonseizure) durations. Results showed that the mEEGNet detected abnormalities with the area under the curve, F1-values, and sensitivity equivalent to or higher than those of existing CNNs. Moreover, the number of parameters is much smaller than other CNN models. To our knowledge, the dataset of absence epilepsy validated with machine learning through this research is the largest in the literature.
Covariance-Free Sparse Bayesian Learning
Authors: Alexander Lin, Andrew H. Song, Berkin Bilgic, Demba Ba
Date: 2021-05-21 00:00:00
Tasks: compressed sensing, MRI Reconstruction, Uncertainty Quantification

Sparse Bayesian learning (SBL) is a powerful framework for tackling the sparse coding problem while also providing uncertainty quantification. The most popular inference algorithms for SBL exhibit prohibitively large computational costs for high-dimensional problems due to the need to maintain a large covariance matrix. To resolve this issue, we introduce a new method for accelerating SBL inference -- named covariance-free expectation maximization (CoFEM) -- that avoids explicit computation of the covariance matrix. CoFEM solves multiple linear systems to obtain unbiased estimates of the posterior statistics needed by SBL. This is accomplished by exploiting innovations from numerical linear algebra such as preconditioned conjugate gradient and a little-known diagonal estimation rule. For a large class of compressed sensing matrices, we provide theoretical justifications for why our method scales well in high-dimensional settings. Through simulations, we show that CoFEM can be up to thousands of times faster than existing baselines without sacrificing coding accuracy. Through applications to calcium imaging deconvolution and multi-contrast MRI reconstruction, we show that CoFEM enables SBL to tractably tackle high-dimensional sparse coding problems of practical interest.
Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units
Authors: Zihao Zhang, Stefan Zohren
Date: 2021-05-21 00:00:00
Tasks: Decoder

We design multi-horizon forecasting models for limit order book (LOB) data by using deep learning techniques. Unlike standard structures where a single prediction is made, we adopt encoder-decoder models with sequence-to-sequence and Attention mechanisms to generate a forecasting path. Our methods achieve comparable performance to state-of-art algorithms at short prediction horizons. Importantly, they outperform when generating predictions over long horizons by leveraging the multi-horizon setup. Given that encoder-decoder models rely on recurrent neural layers, they generally suffer from slow training processes. To remedy this, we experiment with utilising novel hardware, so-called Intelligent Processing Units (IPUs) produced by Graphcore. IPUs are specifically designed for machine intelligence workload with the aim to speed up the computation process. We show that in our setup this leads to significantly faster training times when compared to training models with GPUs.
Low-Memory Implementations of Ridge Solutions for Broad Learning System with Incremental Learning
Authors: Hufei Zhu
Date: 2021-05-21 00:00:00
Tasks: Incremental Learning

The existing low-memory BLS implementation proposed recently avoids the need for storing and inverting large matrices, to achieve efficient usage of memories. However, the existing low-memory BLS implementation sacrifices the testing accuracy as a price for efficient usage of memories, since it can no longer obtain the generalized inverse or ridge solution for the output weights during incremental learning, and it cannot work under the very small ridge parameter that is utilized in the original BLS. Accordingly, it is required to develop the low-memory BLS implementations, which can work under very small ridge parameters and compute the generalized inverse or ridge solution for the output weights in the process of incremental learning. In this paper, firstly we propose the low-memory implementations for the recently proposed recursive and square-root BLS algorithms on added inputs and the recently proposed squareroot BLS algorithm on added nodes, by simply processing a batch of inputs or nodes in each recursion. Since the recursive BLS implementation includes the recursive updates of the inverse matrix that may introduce numerical instabilities after a large number of iterations, and needs the extra computational load to decompose the inverse matrix into the Cholesky factor when cooperating with the proposed low-memory implementation of the square-root BLS algorithm on added nodes, we only improve the low-memory implementations of the square-root BLS algorithms on added inputs and nodes, to propose the full lowmemory implementation of the square-root BLS algorithm. All the proposed low-memory BLS implementations compute the ridge solution for the output weights in the process of incremental learning, and most of them can work under very small ridge parameters.