pwc-archive/papers-with-abstracts huggingface dataset:
Painting Style-Aware Manga Colorization Based on Generative Adversarial Networks
Authors: Yugo Shimizu, Ryosuke Furuta, Delong Ouyang, Yukinobu Taniguchi, Ryota Hinami, Shonosuke Ishiwatari
Date: 2021-07-16 00:00:00
Tasks: Colorization

Japanese comics (called manga) are traditionally created in monochrome format. In recent years, in addition to monochrome comics, full color comics, a more attractive medium, have appeared. Unfortunately, color comics require manual colorization, which incurs high labor costs. Although automatic colorization methods have been recently proposed, most of them are designed for illustrations, not for comics. Unlike illustrations, since comics are composed of many consecutive images, the painting style must be consistent. To realize consistent colorization, we propose here a semi-automatic colorization method based on generative adversarial networks (GAN); the method learns the painting style of a specific comic from small amount of training data. The proposed method takes a pair of a screen tone image and a flat colored image as input, and outputs a colorized image. Experiments show that the proposed method achieves better performance than the existing alternatives.
Deep Learning to Ternary Hash Codes by Continuation
Authors: Mingrui Chen, Weiyu Li, Weizhi Lu
Date: 2021-07-16 00:00:00
Tasks: Deep Learning, Image Retrieval, Quantization, Retrieval

Recently, it has been observed that {0,1,-1}-ternary codes which are simply generated from deep features by hard thresholding, tend to outperform {-1,1}-binary codes in image retrieval. To obtain better ternary codes, we for the first time propose to jointly learn the features with the codes by appending a smoothed function to the networks. During training, the function could evolve into a non-smoothed ternary function by a continuation method. The method circumvents the difficulty of directly training discrete functions and reduces the quantization errors of ternary codes. Experiments show that the generated codes indeed could achieve higher retrieval accuracy.
All the attention you need: Global-local, spatial-channel attention for image retrieval
Authors: Chull Hwan Song, Hye Joo Han, Yannis Avrithis
Date: 2021-07-16 00:00:00
Tasks: All, Image Retrieval, Representation Learning, Retrieval

We address representation learning for large-scale instance-level image retrieval. Apart from backbone, training pipelines and loss functions, popular approaches have focused on different spatial pooling and attention mechanisms, which are at the core of learning a powerful global image representation. There are different forms of attention according to the interaction of elements of the feature tensor (local and global) and the dimensions where it is applied (spatial and channel). Unfortunately, each study addresses only one or two forms of attention and applies it to different problems like classification, detection or retrieval. We present global-local attention module (GLAM), which is attached at the end of a backbone network and incorporates all four forms of attention: local and global, spatial and channel. We obtain a new feature tensor and, by spatial pooling, we learn a powerful embedding for image retrieval. Focusing on global descriptors, we provide empirical evidence of the interaction of all forms of attention and improve the state of the art on standard benchmarks.
CCVS: Context-aware Controllable Video Synthesis
Authors: Guillaume Le Moing, Jean Ponce, Cordelia Schmid
Date: 2021-07-16 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Decoder, Optical Flow Estimation, Self-Supervised Learning, Video Generation, Video Prediction

This presentation introduces a self-supervised learning approach to the synthesis of new video clips from old ones, with several new key elements for improved spatial resolution and realism: It conditions the synthesis process on contextual information for temporal continuity and ancillary information for fine control. The prediction model is doubly autoregressive, in the latent space of an autoencoder for forecasting, and in image space for updating contextual information, which is also used to enforce spatio-temporal consistency through a learnable optical flow module. Adversarial training of the autoencoder in the appearance and temporal domains is used to further improve the realism of its output. A quantizer inserted between the encoder and the transformer in charge of forecasting future frames in latent space (and its inverse inserted between the transformer and the decoder) adds even more flexibility by affording simple mechanisms for handling multimodal ancillary information for controlling the synthesis process (eg, a few sample frames, an audio track, a trajectory in image space) and taking into account the intrinsically uncertain nature of the future by allowing multiple predictions. Experiments with an implementation of the proposed approach give very good qualitative and quantitative results on multiple tasks and standard benchmarks.
Probabilistic Appearance-Invariant Topometric Localization with New Place Awareness
Authors: Ming Xu, Tobias Fischer, Niko Sünderhauf, Michael Milford
Date: 2021-07-16 00:00:00
Tasks: Loop Closure Detection, State Estimation, Visual Place Recognition

Probabilistic state-estimation approaches offer a principled foundation for designing localization systems, because they naturally integrate sequences of imperfect motion and exteroceptive sensor data. Recently, probabilistic localization systems utilizing appearance-invariant visual place recognition (VPR) methods as the primary exteroceptive sensor have demonstrated state-of-the-art performance in the presence of substantial appearance change. However, existing systems 1) do not fully utilize odometry data within the motion models, and 2) are unable to handle route deviations, due to the assumption that query traverses exactly repeat the mapping traverse. To address these shortcomings, we present a new probabilistic topometric localization system which incorporates full 3-dof odometry into the motion model and furthermore, adds an "off-map" state within the state-estimation framework, allowing query traverses which feature significant route detours from the reference map to be successfully localized. We perform extensive evaluation on multiple query traverses from the Oxford RobotCar dataset exhibiting both significant appearance change and deviations from routes previously traversed. In particular, we evaluate performance on two practically relevant localization tasks: loop closure detection and global localization. Our approach achieves major performance improvements over both existing and improved state-of-the-art systems.
Optical Inspection of the Silicon Micro-strip Sensors for the CBM Experiment employing Artificial Intelligence
Authors: E. Lavrik, M. Shiroya, H. R. Schmidt, A. Toia, J. M. Heuser
Date: 2021-07-16 00:00:00
Tasks: Defect Detection

Optical inspection of 1191 silicon micro-strip sensors was performed using a custom made optical inspection setup, employing a machine-learning based approach for the defect analysis and subsequent quality assurance. Furthermore, metrological control of the sensor's surface was performed. In this manuscript, we present the analysis of various sensor surface defects. Among these are implant breaks, p-stop breaks, aluminium strip opens, aluminium strip shorts, surface scratches, double metallization layer defects, passivation layer defects, bias resistor defects as well as dust particle identification. The defect detection was done using the application of Convolutional Deep Neural Networks (CDNNs). From this, defective strips and defect clusters were identified, as well as a 2D map of the defects using their geometrical positions on the sensor was performed. Based on the total number of defects found on the sensor's surface, a method for the estimation of sensor's overall quality grade and quality score was proposed.
Exploiting generative self-supervised learning for the assessment of biological images with lack of annotations: a COVID-19 case-study
Authors: Alessio Mascolini, Dario Cardamone, Francesco Ponzio, Santa Di Cataldo, Elisa Ficarra
Date: 2021-07-16 00:00:00
Tasks: Representation Learning, Self-Supervised Learning, Zero-Shot Learning

Computer-aided analysis of biological images typically requires extensive training on large-scale annotated datasets, which is not viable in many situations. In this paper we present GAN-DL, a Discriminator Learner based on the StyleGAN2 architecture, which we employ for self-supervised image representation learning in the case of fluorescent biological images. We show that Wasserstein Generative Adversarial Networks combined with linear Support Vector Machines enable high-throughput compound screening based on raw images. We demonstrate this by classifying active and inactive compounds tested for the inhibition of SARS-CoV-2 infection in VERO and HRCE cell lines. In contrast to previous methods, our deep learning based approach does not require any annotation besides the one that is normally collected during the sample preparation process. We test our technique on the RxRx19a Sars-CoV-2 image collection. The dataset consists of fluorescent images that were generated to assess the ability of regulatory-approved or in late-stage clinical trials compound to modulate the in vitro infection from SARS-CoV-2 in both VERO and HRCE cell lines. We show that our technique can be exploited not only for classification tasks, but also to effectively derive a dose response curve for the tested treatments, in a self-supervised manner. Lastly, we demonstrate its generalization capabilities by successfully addressing a zero-shot learning task, consisting in the categorization of four different cell types of the RxRx1 fluorescent images collection.
Wasserstein Distances, Geodesics and Barycenters of Merge Trees
Authors: Mathieu Pont, Jules Vidal, Julie Delon, Julien Tierny
Date: 2021-07-16 00:00:00

This paper presents a unified computational framework for the estimation of distances, geodesics and barycenters of merge trees. We extend recent work on the edit distance [106] and introduce a new metric, called the Wasserstein distance between merge trees, which is purposely designed to enable efficient computations of geodesics and barycenters. Specifically, our new distance is strictly equivalent to the L2-Wasserstein distance between extremum persistence diagrams, but it is restricted to a smaller solution space, namely, the space of rooted partial isomorphisms between branch decomposition trees. This enables a simple extension of existing optimization frameworks [112] for geodesics and barycenters from persistence diagrams to merge trees. We introduce a task-based algorithm which can be generically applied to distance, geodesic, barycenter or cluster computation. The task-based nature of our approach enables further accelerations with shared-memory parallelism. Extensive experiments on public ensembles and SciVis contest benchmarks demonstrate the efficiency of our approach -- with barycenter computations in the orders of minutes for the largest examples -- as well as its qualitative ability to generate representative barycenter merge trees, visually summarizing the features of interest found in the ensemble. We show the utility of our contributions with dedicated visualization applications: feature tracking, temporal reduction and ensemble clustering. We provide a lightweight C++ implementation that can be used to reproduce our results.
Lightness Modulated Deep Inverse Tone Mapping
Authors: Kanglin Liu, Gaofeng Cao, Jiang Duan, Guoping Qiu
Date: 2021-07-16 00:00:00
Tasks: HDR Reconstruction, Image Reconstruction, inverse tone mapping, Inverse-Tone-Mapping, Tone Mapping

Single-image HDR reconstruction or inverse tone mapping (iTM) is a challenging task. In particular, recovering information in over-exposed regions is extremely difficult because details in such regions are almost completely lost. In this paper, we present a deep learning based iTM method that takes advantage of the feature extraction and mapping power of deep convolutional neural networks (CNNs) and uses a lightness prior to modulate the CNN to better exploit observations in the surrounding areas of the over-exposed regions to enhance the quality of HDR image reconstruction. Specifically, we introduce a Hierarchical Synthesis Network (HiSN) for inferring a HDR image from a LDR input and a Lightness Adpative Modulation Network (LAMN) to incorporate the the lightness prior knowledge in the inferring process. The HiSN hierarchically synthesizes the high-brightness component and the low-brightness component of the HDR image whilst the LAMN uses a lightness adaptive mask that separates detail-less saturated bright pixels from well-exposed lower light pixels to enable HiSN to better infer the missing information, particularly in the difficult over-exposed detail-less areas. We present experimental results to demonstrate the effectiveness of the new technique based on quantitative measures and visual comparisons. In addition, we present ablation studies of HiSN and visualization of the activation maps inside LAMN to help gain a deeper understanding of the internal working of the new iTM algorithm and explain why it can achieve much improved performance over state-of-the-art algorithms.
Joint Semi-supervised 3D Super-Resolution and Segmentation with Mixed Adversarial Gaussian Domain Adaptation
Authors: Nicolo Savioli, Antonio de Marvao, Wenjia Bai, Shuo Wang, Stuart A. Cook, Calvin W. L. Chin, Daniel Rueckert, Declan P. O'Regan
Date: 2021-07-16 00:00:00
Tasks: Domain Adaptation, Domain Generalization, Generative Adversarial Network, Super-Resolution

Optimising the analysis of cardiac structure and function requires accurate 3D representations of shape and motion. However, techniques such as cardiac magnetic resonance imaging are conventionally limited to acquiring contiguous cross-sectional slices with low through-plane resolution and potential inter-slice spatial misalignment. Super-resolution in medical imaging aims to increase the resolution of images but is conventionally trained on features from low resolution datasets and does not super-resolve corresponding segmentations. Here we propose a semi-supervised multi-task generative adversarial network (Gemini-GAN) that performs joint super-resolution of the images and their labels using a ground truth of high resolution 3D cines and segmentations, while an unsupervised variational adversarial mixture autoencoder (V-AMA) is used for continuous domain adaptation. Our proposed approach is extensively evaluated on two transnational multi-ethnic populations of 1,331 and 205 adults respectively, delivering an improvement on state of the art methods in terms of Dice index, peak signal to noise ratio, and structural similarity index measure. This framework also exceeds the performance of state of the art generative domain adaptation models on external validation (Dice index 0.81 vs 0.74 for the left ventricle). This demonstrates how joint super-resolution and segmentation, trained on 3D ground-truth data with cross-domain generalization, enables robust precision phenotyping in diverse populations.
Unpaired cross-modality educed distillation (CMEDL) for medical image segmentation
Authors: Jue Jiang, Andreas Rimner, Joseph O. Deasy, Harini Veeraraghavan
Date: 2021-07-16 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Organ Segmentation, Segmentation, Semantic Segmentation, Translation, Tumor Segmentation

Accurate and robust segmentation of lung cancers from CT, even those located close to mediastinum, is needed to more accurately plan and deliver radiotherapy and to measure treatment response. Therefore, we developed a new cross-modality educed distillation (CMEDL) approach, using unpaired CT and MRI scans, whereby an informative teacher MRI network guides a student CT network to extract features that signal the difference between foreground and background. Our contribution eliminates two requirements of distillation methods: (i) paired image sets by using an image to image (I2I) translation and (ii) pre-training of the teacher network with a large training set by using concurrent training of all networks. Our framework uses an end-to-end trained unpaired I2I translation, teacher, and student segmentation networks. Architectural flexibility of our framework is demonstrated using 3 segmentation and 2 I2I networks. Networks were trained with 377 CT and 82 T2w MRI from different sets of patients, with independent validation (N=209 tumors) and testing (N=609 tumors) datasets. Network design, methods to combine MRI with CT information, distillation learning under informative (MRI to CT), weak (CT to MRI) and equal teacher (MRI to MRI), and ablation tests were performed. Accuracy was measured using Dice similarity (DSC), surface Dice (sDSC), and Hausdorff distance at the 95$^{th}$ percentile (HD95). The CMEDL approach was significantly (p $<$ 0.001) more accurate (DSC of 0.77 vs. 0.73) than non-CMEDL methods with an informative teacher for CT lung tumor, with a weak teacher (DSC of 0.84 vs. 0.81) for MRI lung tumor, and with equal teacher (DSC of 0.90 vs. 0.88) for MRI multi-organ segmentation. CMEDL also reduced inter-rater lung tumor segmentation variabilities..
In-Bed Person Monitoring Using Thermal Infrared Sensors
Authors: Elias Josse, Amanda Nerborg, Kevin Hernandez-Diaz, Fernando Alonso-Fernandez
Date: 2021-07-16 00:00:00

The world is expecting an aging population and shortage of healthcare professionals. This poses the problem of providing a safe and dignified life for the elderly. Technological solutions involving cameras can contribute to safety, comfort and efficient emergency responses, but they are invasive of privacy. We use 'Griddy', a prototype with a Panasonic Grid-EYE, a low-resolution infrared thermopile array sensor, which offers more privacy. Mounted over a bed, it can determine if the user is on the bed or not without human interaction. For this purpose, two datasets were captured, one (480 images) under constant conditions, and a second one (200 images) under different variations such as use of a duvet, sleeping with a pet, or increased room temperature. We test three machine learning algorithms: Support Vector Machines (SVM), k-Nearest Neighbors (k-NN) and Neural Network (NN). With 10-fold cross validation, the highest accuracy in the main dataset is for both SVM and k-NN (99%). The results with variable data show a lower reliability under certain circumstances, highlighting the need of extra work to meet the challenge of variations in the environment.
Feature selection and multi-kernel learning for adaptive graph regularized nonnegative matrix factorization
Authors: Jim Jing-Yan Wang, Jianhua Z. Huang, Yijun Sun, Xin Gao
Date: 2014-09-20 00:00:00
Proceeding: Elsevier Ltd 2014 9
Tasks: feature selection

Nonnegative matrix factorization (NMF), a popular part-based representation technique, does not capture
the intrinsic local geometric structure of the data space. Graph regularized NMF (GNMF) was recently
proposed to avoid this limitation by regularizing NMF with a nearest neighbor graph constructed from
the input data set. However, GNMF has two main bottlenecks. First, using the original feature space
directly to construct the graph is not necessarily optimal because of the noisy and irrelevant features
and nonlinear distributions of data samples. Second, one possible way to handle the nonlinear distribution of data samples is by kernel embedding. However, it is often difficult to choose the most suitable kernel. To solve these bottlenecks, we propose two novel graph-regularized NMF methods, AGNMFFS and
AGNMFMK, by introducing feature selection and multiple-kernel learning to the graph regularized NMF,
respectively. Instead of using a fixed graph as in GNMF, the two proposed methods learn the nearest
neighbor graph that is adaptive to the selected features and learned multiple kernels, respectively. For
each method, we propose a unified objective function to conduct feature selection/multi-kernel learning,
NMF and adaptive graph regularization simultaneously. We further develop two iterative algorithms to
solve the two optimization problems. Experimental results on two challenging pattern classification tasks
demonstrate that the proposed methods significantly outperform state-of-the-art data representation
methods.
Robust Real-Time Violence Detection in Video Using CNN And LSTM
Authors: almamon rasool abdali, Rana F. Al-Tuma
Date: 2019-03-27 00:00:00
Proceeding: IEEE Scientific Conference of Computer Sciences (SCCS) 2019 3
Tasks: Action Recognition In Videos, Video Classification

Detection of a violence event in surveillance systems is playing a significant role in law enforcement and city safety. The effectiveness of violence event detectors measures by the speed of response and the accuracy and the generality over different kind of video sources with a different format. Several studies worked on the violence detection with focus either on speed or accuracy or both but not taking into account the generality over different kind of video sources. In this paper, we proposed a real-time violence detector based on deep-learning methods. The proposed model consists of CNN as a spatial feature extractor and LSTM as temporal relation learning method with a focus on the three-factor (overall generality - accuracy - fast response time). The suggested model achieved 98% accuracy with speed of 131 frames/sec. Comparison of the accuracy and the speed of the proposed model with previous works illustrated that the proposed model provides the highest accuracy and the fastest speed among all the previous works in the field of violence detection.
Predicting Daily Trading Volume via Various Hidden States
Authors: Shaojun Ma, Pengcheng Li
Date: 2021-07-16 00:00:00

Predicting intraday trading volume plays an important role in trading alpha research. Existing methods such as rolling means(RM) and a two-states based Kalman Filtering method have been presented in this topic. We extend two states into various states in Kalman Filter framework to improve the accuracy of prediction. Specifically, for different stocks we utilize cross validation and determine best states number by minimizing mean squared error of the trading volume. We demonstrate the effectivity of our method through a series of comparison experiments and numerical analysis.
Public Health, Technology, and Human Rights: Lessons from Digital Contact Tracing
Authors: Maria Carnovale, Khahlil Louisy
Date: 2021-07-15 00:00:00

To mitigate inefficiencies in manual contact tracing processes, Digital Contact Tracing and Exposure Notifications Systems were developed for use as public-interest technologies during the SARS-CoV-2 global pandemic. Effective implementation of these tools requires alignment across several factors, including local regulations and policies and trust in government and public health officials. Careful consideration should also be made to minimize any potential conflicts with existing processes in public health which has demonstrated effectiveness. Four unique cases-of Ireland, Guayaquil, Haiti, and the Philippines-detailed in this paper will highlight the importance of upholding the principles of Scientific Validity, Necessity, Time Boundedness, and Proportionality.