pwc-archive/papers-with-abstracts huggingface dataset:
GIFnets: Differentiable GIF Encoding Framework
Authors: Innfarn Yoo, Xiyang Luo, Yilin Wang, Feng Yang, Peyman Milanfar
Date: 2020-06-24 00:00:00
Proceeding: CVPR 2020 6

Graphics Interchange Format (GIF) is a widely used image file format. Due to the limited number of palette colors, GIF encoding often introduces color banding artifacts. Traditionally, dithering is applied to reduce color banding, but introducing dotted-pattern artifacts. To reduce artifacts and provide a better and more efficient GIF encoding, we introduce a differentiable GIF encoding pipeline, which includes three novel neural networks: PaletteNet, DitherNet, and BandingNet. Each of these three networks provides an important functionality within the GIF encoding pipeline. PaletteNet predicts a near-optimal color palette given an input image. DitherNet manipulates the input image to reduce color banding artifacts and provides an alternative to traditional dithering. Finally, BandingNet is designed to detect color banding, and provides a new perceptual loss specifically for GIF images. As far as we know, this is the first fully differentiable GIF encoding pipeline based on deep neural networks and compatible with existing GIF decoders. User study shows that our algorithm is better than Floyd-Steinberg based GIF encoding.
Flexible Image Denoising with Multi-layer Conditional Feature Modulation
Authors: Jiazhi Du, Xin Qiao, Zifei Yan, Hongzhi Zhang, WangMeng Zuo
Date: 2020-06-24 00:00:00
Tasks: Denoising, Image Denoising

For flexible non-blind image denoising, existing deep networks usually take both noisy image and noise level map as the input to handle various noise levels with a single model. However, in this kind of solution, the noise variance (i.e., noise level) is only deployed to modulate the first layer of convolution feature with channel-wise shifting, which is limited in balancing noise removal and detail preservation. In this paper, we present a novel flexible image enoising network (CFMNet) by equipping an U-Net backbone with multi-layer conditional feature modulation (CFM) modules. In comparison to channel-wise shifting only in the first layer, CFMNet can make better use of noise level information by deploying multiple layers of CFM. Moreover, each CFM module takes onvolutional features from both noisy image and noise level map as input for better trade-off between noise removal and detail preservation. Experimental results show that our CFMNet is effective in exploiting noise level information for flexible non-blind denoising, and performs favorably against the existing deep image denoising methods in terms of both quantitative metrics and visual quality.
Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model
Authors: Ren Yang, Fabian Mentzer, Luc van Gool, Radu Timofte
Date: 2020-06-24 00:00:00
Tasks: Decoder, MS-SSIM, SSIM, Video Compression

The past few years have witnessed increasing interests in applying deep learning to video compression. However, the existing approaches compress a video frame with only a few number of reference frames, which limits their ability to fully exploit the temporal correlation among video frames. To overcome this shortcoming, this paper proposes a Recurrent Learned Video Compression (RLVC) approach with the Recurrent Auto-Encoder (RAE) and Recurrent Probability Model (RPM). Specifically, the RAE employs recurrent cells in both the encoder and decoder. As such, the temporal information in a large range of frames can be used for generating latent representations and reconstructing compressed outputs. Furthermore, the proposed RPM network recurrently estimates the Probability Mass Function (PMF) of the latent representation, conditioned on the distribution of previous latent representations. Due to the correlation among consecutive frames, the conditional cross entropy can be lower than the independent cross entropy, thus reducing the bit-rate. The experiments show that our approach achieves the state-of-the-art learned video compression performance in terms of both PSNR and MS-SSIM. Moreover, our approach outperforms the default Low-Delay P (LDP) setting of x265 on PSNR, and also has better performance on MS-SSIM than the SSIM-tuned x265 and the slowest setting of x265. The codes are available at https://github.com/RenYang-home/RLVC.git.
GMMLoc: Structure Consistent Visual Localization with Gaussian Mixture Models
Authors: Huaiyang Huang, Haoyang Ye, Yuxiang Sun, Ming Liu
Date: 2020-06-24 00:00:00
Tasks: Simultaneous Localization and Mapping, State Estimation, Visual Localization

Incorporating prior structure information into the visual state estimation could generally improve the localization performance. In this letter, we aim to address the paradox between accuracy and efficiency in coupling visual factors with structure constraints. To this end, we present a cross-modality method that tracks a camera in a prior map modelled by the Gaussian Mixture Model (GMM). With the pose estimated by the front-end initially, the local visual observations and map components are associated efficiently, and the visual structure from the triangulation is refined simultaneously. By introducing the hybrid structure factors into the joint optimization, the camera poses are bundle-adjusted with the local visual structure. By evaluating our complete system, namely GMMLoc, on the public dataset, we show how our system can provide a centimeter-level localization accuracy with only trivial computational overhead. In addition, the comparative studies with the state-of-the-art vision-dominant state estimators demonstrate the competitive performance of our method.
A Novel Approach for Correcting Multiple Discrete Rigid In-Plane Motions Artefacts in MRI Scans
Authors: Michael Rotman, Rafi Brada, Israel Beniaminy, Sangtae Ahn, Christopher J. Hardy, Lior Wolf
Date: 2020-06-24 00:00:00
Tasks: compressed sensing

Motion artefacts created by patient motion during an MRI scan occur frequently in practice, often rendering the scans clinically unusable and requiring a re-scan. While many methods have been employed to ameliorate the effects of patient motion, these often fall short in practice. In this paper we propose a novel method for removing motion artefacts using a deep neural network with two input branches that discriminates between patient poses using the motion's timing. The first branch receives a subset of the $k$-space data collected during a single patient pose, and the second branch receives the remaining part of the collected $k$-space data. The proposed method can be applied to artefacts generated by multiple movements of the patient. Furthermore, it can be used to correct motion for the case where $k$-space has been under-sampled, to shorten the scan time, as is common when using methods such as parallel imaging or compressed sensing. Experimental results on both simulated and real MRI data show the efficacy of our approach.
Does Non-COVID19 Lung Lesion Help? Investigating Transferability in COVID-19 CT Image Segmentation
Authors: Yixin Wang, Yao Zhang, Yang Liu, Jiang Tian, Cheng Zhong, Zhongchao Shi, Yang Zhang, Zhiqiang He
Date: 2020-06-23 00:00:00
Tasks: Computed Tomography (CT), Image Segmentation, Segmentation, Semantic Segmentation, Transfer Learning

Coronavirus disease 2019 (COVID-19) is a highly contagious virus spreading all around the world. Deep learning has been adopted as an effective technique to aid COVID-19 detection and segmentation from computed tomography (CT) images. The major challenge lies in the inadequate public COVID-19 datasets. Recently, transfer learning has become a widely used technique that leverages the knowledge gained while solving one problem and applying it to a different but related problem. However, it remains unclear whether various non-COVID19 lung lesions could contribute to segmenting COVID-19 infection areas and how to better conduct this transfer procedure. This paper provides a way to understand the transferability of non-COVID19 lung lesions. Based on a publicly available COVID-19 CT dataset and three public non-COVID19 datasets, we evaluate four transfer learning methods using 3D U-Net as a standard encoder-decoder method. The results reveal the benefits of transferring knowledge from non-COVID19 lung lesions, and learning from multiple lung lesion datasets can extract more general features, leading to accurate and robust pre-trained models. We further show the capability of the encoder to learn feature representations of lung lesions, which improves segmentation accuracy and facilitates training convergence. In addition, our proposed Hybrid-encoder learning method incorporates transferred lung lesion features from non-COVID19 datasets effectively and achieves significant improvement. These findings promote new insights into transfer learning for COVID-19 CT image segmentation, which can also be further generalized to other medical tasks.
Microstructure Generation via Generative Adversarial Network for Heterogeneous, Topologically Complex 3D Materials
Authors: Tim Hsu, William K. Epting, Hokon Kim, Harry W. Abernathy, Gregory A. Hackett, Anthony D. Rollett, Paul A. Salvador, Elizabeth A. Holm
Date: 2020-06-22 00:00:00
Tasks: Generative Adversarial Network

Using a large-scale, experimentally captured 3D microstructure dataset, we implement the generative adversarial network (GAN) framework to learn and generate 3D microstructures of solid oxide fuel cell electrodes. The generated microstructures are visually, statistically, and topologically realistic, with distributions of microstructural parameters, including volume fraction, particle size, surface area, tortuosity, and triple phase boundary density, being highly similar to those of the original microstructure. These results are compared and contrasted with those from an established, grain-based generation algorithm (DREAM.3D). Importantly, simulations of electrochemical performance, using a locally resolved finite element model, demonstrate that the GAN generated microstructures closely match the performance distribution of the original, while DREAM.3D leads to significant differences. The ability of the generative machine learning model to recreate microstructures with high fidelity suggests that the essence of complex microstructures may be captured and represented in a compact and manipulatable form.
Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search
Authors: Jaehyeon Kim, Sungwon Kim, Jungil Kong, Sungroh Yoon
Date: 2020-05-22 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: text-to-speech, Text to Speech, Text-To-Speech Synthesis

Recently, text-to-speech (TTS) models such as FastSpeech and ParaNet have been proposed to generate mel-spectrograms from text in parallel. Despite the advantages, the parallel TTS models cannot be trained without guidance from autoregressive TTS models as their external aligners. In this work, we propose Glow-TTS, a flow-based generative model for parallel TTS that does not require any external aligner. We introduce Monotonic Alignment Search (MAS), an internal alignment search algorithm for training Glow-TTS. By leveraging the properties of flows, MAS searches for the most probable monotonic alignment between text and the latent representation of speech. Glow-TTS obtains an order-of-magnitude speed-up over the autoregressive TTS model, Tacotron 2, at synthesis with comparable speech quality, requiring only 1.5 seconds to synthesize one minute of speech in end-to-end. We further show that our model can be easily extended to a multi-speaker setting. Our demo page and code are available at public.
Improving LPCNet-based Text-to-Speech with Linear Prediction-structured Mixture Density Network
Authors: Min-Jae Hwang, Eunwoo Song, Ryuichi Yamamoto, Frank Soong, Hong-Goo Kang
Date: 2020-01-31 00:00:00
Tasks: Quantization, Speech Synthesis, text-to-speech, Text to Speech

In this paper, we propose an improved LPCNet vocoder using a linear prediction (LP)-structured mixture density network (MDN). The recently proposed LPCNet vocoder has successfully achieved high-quality and lightweight speech synthesis systems by combining a vocal tract LP filter with a WaveRNN-based vocal source (i.e., excitation) generator. However, the quality of synthesized speech is often unstable because the vocal source component is insufficiently represented by the mu-law quantization method, and the model is trained without considering the entire speech production mechanism. To address this problem, we first introduce LP-MDN, which enables the autoregressive neural vocoder to structurally represent the interactions between the vocal tract and vocal source components. Then, we propose to incorporate the LP-MDN to the LPCNet vocoder by replacing the conventional discretized output with continuous density distribution. The experimental results verify that the proposed system provides high quality synthetic speech by achieving a mean opinion score of 4.41 within a text-to-speech framework.
Learning Diagnosis of COVID-19 from a Single Radiological Image
Authors: Pengyi Zhang, Yunxin Zhong, Xiaoying Tang, Yunlin Deng, Xiaoqiong Li
Date: 2020-06-06 00:00:00
Proceeding: arXiv:2006.12220 2020 6
Tasks: COVID-19 Diagnosis, Data Augmentation

Radiological image is currently adopted as the visual evidence for COVID-19 diagnosis in clinical. Using deep models to realize automated infection measurement and COVID-19 diagnosis is important for faster examination based on radiological imaging. Unfortunately, collecting large training data systematically in the early stage is difficult. To address this problem, we explore the feasibility of learning deep models for COVID-19 diagnosis from a single radiological image by resorting to synthesizing diverse radiological images. Specifically, we propose a novel conditional generative model, called CoSinGAN, which can be learned from a single radiological image with a given condition, i.e., the annotations of the lung and COVID-19 infection. Our CoSinGAN is able to capture the conditional distribution of visual finds of COVID-19 infection, and further synthesize diverse and high-resolution radiological images that match the input conditions precisely. Both deep classification and segmentation networks trained on synthesized samples from CoSinGAN achieve notable detection accuracy of COVID-19 infection. Such results are significantly better than the counterparts trained on the same extremely small number of real samples (1 or 2 real samples) by using strong data augmentation, and approximate to the counterparts trained on large dataset (2846 real images). It confirms our method can significantly reduce the performance gap between deep models trained on extremely small dataset and on large dataset, and thus has the potential to realize learning COVID-19 diagnosis from few radiological images in the early stage of COVID-19 pandemic. Our codes are made publicly available at https://github.com/PengyiZhang/CoSinGAN.
Iterative Entity Alignment via Joint Knowledge Embeddings
Authors: Hao Zhu, Ruobing Xie, Zhiyuan Liu, Maosong Sun
Date: 2017-07-01 00:00:00
Proceeding: International Joint Conference on Artificial Intelligence 2017 7
Tasks: Entity Alignment, Knowledge Graph Completion, Knowledge Graphs

Entity alignment aims to link entities and their counterparts among multiple knowledge graphs (KGs). Most existing methods typically rely on external information of entities such as Wikipedia links and require costly manual feature construction to complete alignment. In this paper, we present a novel approach for entity alignment via joint knowledge embeddings. Our method jointly encodes both entities and relations of various KGs into a unified low-dimensional semantic space according to a small seed set of aligned entities. During this process, we can align entities according to their semantic distance in this joint semantic space. More specifically, we present an iterative and parameter sharing method to improve alignment performance. Experiment results on real-world datasets show that, as compared to baselines, our method achieves significant improvements on entity alignment, and can further improve knowledge graph completion performance on various KGs with the favor of joint knowledge embeddings.
Moore's Paradox and the logic of belief
Authors: Andrés Páez
Date: 2020-06-19 00:00:00

Moores Paradox is a test case for any formal theory of belief. In Knowledge and Belief, Hintikka developed a multimodal logic for statements that express sentences containing the epistemic notions of knowledge and belief. His account purports to offer an explanation of the paradox. In this paper I argue that Hintikkas interpretation of one of the doxastic operators is philosophically problematic and leads to an unnecessarily strong logical system. I offer a weaker alternative that captures in a more accurate way our logical intuitions about the notion of belief without sacrificing the possibility of providing an explanation for problematic cases such as Moores Paradox.
Unsupervised Learning of Global Registration of Temporal Sequence of Point Clouds
Authors: Lingjing Wang, Yi Shi, Xiang Li, Yi Fang
Date: 2020-06-17 00:00:00

Global registration of point clouds aims to find an optimal alignment of a sequence of 2D or 3D point sets. In this paper, we present a novel method that takes advantage of current deep learning techniques for unsupervised learning of global registration from a temporal sequence of point clouds. Our key novelty is that we introduce a deep Spatio-Temporal REPresentation (STREP) feature, which describes the geometric essence of both temporal and spatial relationship of the sequence of point clouds acquired with sensors in an unknown environment. In contrast to the previous practice that treats each time step (pair-wise registration) individually, our unsupervised model starts with optimizing a sequence of latent STREP feature, which is then decoded to a temporally and spatially continuous sequence of geometric transformations to globally align multiple point clouds. We have evaluated our proposed approach over both simulated 2D and real 3D datasets and the experimental results demonstrate that our method can beat other techniques by taking into account the temporal information in deep feature learning.
Enhancing Few-Shot Image Classification with Unlabelled Examples
Authors: Peyman Bateni, Jarred Barber, Jan-Willem van de Meent, Frank Wood
Date: 2020-06-17 00:00:00
Tasks: Classification, Clustering, Few-Shot Image Classification, Few-Shot Learning, General Classification, image-classification, Image Classification, Meta-Learning, Object Recognition

We develop a transductive meta-learning method that uses unlabelled instances to improve few-shot image classification performance. Our approach combines a regularized Mahalanobis-distance-based soft k-means clustering procedure with a modified state of the art neural adaptive feature extractor to achieve improved test-time classification accuracy using unlabelled data. We evaluate our method on transductive few-shot learning tasks, in which the goal is to jointly predict labels for query (test) examples given a set of support (training) examples. We achieve state of the art performance on the Meta-Dataset, mini-ImageNet and tiered-ImageNet benchmarks. All trained models and code have been made publicly available at github.com/plai-group/simple-cnaps.
Pain Intensity Estimation from Mobile Video Using 2D and 3D Facial Keypoints
Authors: Matthew Lee, Lyndon Kennedy, Andreas Girgensohn, Lynn Wilcox, John Song En Lee, Chin Wen Tan, Ban Leong Sng
Date: 2020-06-17 00:00:00
Tasks: Management, Multiple Instance Learning

Managing post-surgical pain is critical for successful surgical outcomes. One of the challenges of pain management is accurately assessing the pain level of patients. Self-reported numeric pain ratings are limited because they are subjective, can be affected by mood, and can influence the patient's perception of pain when making comparisons. In this paper, we introduce an approach that analyzes 2D and 3D facial keypoints of post-surgical patients to estimate their pain intensity level. Our approach leverages the previously unexplored capabilities of a smartphone to capture a dense 3D representation of a person's face as input for pain intensity level estimation. Our contributions are adata collection study with post-surgical patients to collect ground-truth labeled sequences of 2D and 3D facial keypoints for developing a pain estimation algorithm, a pain estimation model that uses multiple instance learning to overcome inherent limitations in facial keypoint sequences, and the preliminary results of the pain estimation model using 2D and 3D features with comparisons of alternate approaches.
Geometric Models of Rolling-Shutter Cameras
Authors: Marci Meingast, Christopher Geyer, Shankar Sastry
Date: 2005-03-29 00:00:00

Cameras with rolling shutters are becoming more common as low-power, low-cost CMOS sensors are being used more frequently in cameras. The rolling shutter means that not all scanlines are exposed over the same time interval. The effects of a rolling shutter are noticeable when either the camera or objects in the scene are moving and can lead to systematic biases in projection estimation. We develop a general projection equation for a rolling shutter camera and show how it is affected by different types of camera motion. In the case of fronto-parallel motion, we show how that camera can be modeled as an X-slit camera. We also develop approximate projection equations for a non-zero angular velocity about the optical axis and approximate the projection equation for a constant velocity screw motion. We demonstrate how the rolling shutter effects the projective geometry of the camera and in turn the structure-from-motion.
Efficient graph construction for image representation
Authors: Sarath Shekkizhar, Antonio Ortega
Date: 2020-02-16 00:00:00
Tasks: Denoising, graph construction

Graphs are useful to interpret widely used image processing methods, e.g., bilateral filtering, or to develop new ones, e.g., kernel based techniques. However, simple graph constructions are often used, where edge weight and connectivity depend on a few parameters. In particular, the sparsity of the graph is determined by the choice of a window size. As an alternative, we extend and adapt to images recently introduced non negative kernel regression (NNK) graph construction. In NNK graphs sparsity adapts to intrinsic data properties. Moreover, while previous work considered NNK graphs in generic settings, here we develop novel algorithms that take advantage of image properties so that the NNK approach can scale to large images. Our experiments show that sparse NNK graphs achieve improved energy compaction and denoising performance when compared to using graphs directly derived from the bilateral filter.
A Novel Graph Based Clustering Approach to Document Topic Modeling
Authors: Prateek Chanda, Asit Kumar Das
Date: 2018-04-21 00:00:00
Proceeding: ICCCNT 2018 4
Tasks: Clustering

Clustering is the task of assigning a set of objects into groups so that the objects within the same
cluster are more similar to each other than to those in other clusters based on some similarity measures.
Clustering of documents is an important task in text mining based on their research topics. In this field, cluster analysis is the task of grouping a set of documents in such a way that the documents in the same cluster have similar topic and documents of di fferent clusters have
different topics. The proposed method introduces a novel
graph based clustering method which uses the
importance factor of a document based on a better
mathematical approach than well known classical
methods. Document with the maximum importance factor
in a cluster is considered as the centroid of the cluster.
Publicly available synthetic dataset is used to evaluate the
performance of the proposed algorithm and the method is
compared with some traditional graph based methods to
demonstrate its accuracy.