pwc-archive/papers-with-abstracts huggingface dataset:
Segmentation of the Left Ventricle by SDD double threshold selection and CHT
Authors: ZiHao Wang, ZhenZhou Wang
Date: 2020-07-21 00:00:00
Tasks: Classification, General Classification, LV Segmentation, Object, object-detection, Object Detection, Segmentation

Automatic and robust segmentation of the left ventricle (LV) in magnetic resonance images (MRI) has remained challenging for many decades. With the great success of deep learning in object detection and classification, the research focus of LV segmentation has changed to convolutional neural network (CNN) in recent years. However, LV segmentation is a pixel-level classification problem and its categories are intractable compared to object detection and classification. In this paper, we proposed a robust LV segmentation method based on slope difference distribution (SDD) double threshold selection and circular Hough transform (CHT). The proposed method achieved 96.51% DICE score on the test set of automated cardiac diagnosis challenge (ACDC) which is higher than the best accuracy reported in recently published literatures.
Fine-Grained Image Captioning with Global-Local Discriminative Objective
Authors: Jie Wu, Tianshui Chen, Hefeng Wu, Zhi Yang, Guangchun Luo, Liang Lin
Date: 2020-07-21 00:00:00
Tasks: Descriptive, Image Captioning, Retrieval, Sentence

Significant progress has been made in recent years in image captioning, an active topic in the fields of vision and language. However, existing methods tend to yield overly general captions and consist of some of the most frequent words/phrases, resulting in inaccurate and indistinguishable descriptions (see Figure 1). This is primarily due to (i) the conservative characteristic of traditional training objectives that drives the model to generate correct but hardly discriminative captions for similar images and (ii) the uneven word distribution of the ground-truth captions, which encourages generating highly frequent words/phrases while suppressing the less frequent but more concrete ones. In this work, we propose a novel global-local discriminative objective that is formulated on top of a reference model to facilitate generating fine-grained descriptive captions. Specifically, from a global perspective, we design a novel global discriminative constraint that pulls the generated sentence to better discern the corresponding image from all others in the entire dataset. From the local perspective, a local discriminative constraint is proposed to increase attention such that it emphasizes the less frequent but more concrete words/phrases, thus facilitating the generation of captions that better describe the visual details of the given images. We evaluate the proposed method on the widely used MS-COCO dataset, where it outperforms the baseline methods by a sizable margin and achieves competitive performance over existing leading approaches. We also conduct self-retrieval experiments to demonstrate the discriminability of the proposed method.
Accounting for Unobserved Confounding in Domain Generalization
Authors: Alexis Bellot, Mihaela van der Schaar
Date: 2020-07-21 00:00:00
Tasks: Domain Generalization

This paper investigates the problem of learning robust, generalizable prediction models from a combination of multiple datasets and qualitative assumptions about the underlying data-generating model. Part of the challenge of learning robust models lies in the influence of unobserved confounders that void many of the invariances and principles of minimum error presently used for this problem. Our approach is to define a different invariance property of causal solutions in the presence of unobserved confounders which, through a relaxation of this invariance, can be connected with an explicit distributionally robust optimization problem over a set of affine combination of data distributions. Concretely, our objective takes the form of a standard loss, plus a regularization term that encourages partial equality of error derivatives with respect to model parameters. We demonstrate the empirical performance of our approach on healthcare data from different modalities, including image, speech and tabular data.
Gasper: GrAph Signal ProcEssing in R
Authors: Basile de Loynes, Fabien Navarro, Baptiste Olivier
Date: 2020-07-21 00:00:00

We present a short tutorial on to the use of the R gasper package. Gasper is a package dedicated to signal processing on graphs. It also provides an interface to the SuiteSparse Matrix Collection.
Multi-modal Transformer for Video Retrieval
Authors: Valentin Gabeur, Chen Sun, Karteek Alahari, Cordelia Schmid
Date: 2020-07-21 00:00:00
Proceeding: ECCV 2020 8
Tasks: Natural Language Queries, Retrieval, Video Retrieval, Zero-Shot Video Retrieval

The task of retrieving video content relevant to natural language queries plays a critical role in effectively handling internet-scale datasets. Most of the existing methods for this caption-to-video retrieval problem do not fully exploit cross-modal cues present in video. Furthermore, they aggregate per-frame visual features with limited or no temporal information. In this paper, we present a multi-modal transformer to jointly encode the different modalities in video, which allows each of them to attend to the others. The transformer architecture is also leveraged to encode and model the temporal information. On the natural language side, we investigate the best practices to jointly optimize the language embedding together with the multi-modal transformer. This novel framework allows us to establish state-of-the-art results for video retrieval on three datasets. More details are available at http://thoth.inrialpes.fr/research/MMT.
Distributed Associative Memory Network with Memory Refreshing Loss
Authors: Taewon Park, Inchul Choi, Minho Lee
Date: 2020-07-21 00:00:00
Tasks: Memorization, Question Answering, Relational Reasoning

Despite recent progress in memory augmented neural network (MANN) research, associative memory networks with a single external memory still show limited performance on complex relational reasoning tasks. Especially the content-based addressable memory networks often fail to encode input data into rich enough representation for relational reasoning and this limits the relation modeling performance of MANN for long temporal sequence data. To address these problems, here we introduce a novel Distributed Associative Memory architecture (DAM) with Memory Refreshing Loss (MRL) which enhances the relation reasoning performance of MANN. Inspired by how the human brain works, our framework encodes data with distributed representation across multiple memory blocks and repeatedly refreshes the contents for enhanced memorization similar to the rehearsal process of the brain. For this procedure, we replace a single external memory with a set of multiple smaller associative memory blocks and update these sub-memory blocks simultaneously and independently for the distributed representation of input data. Moreover, we propose MRL which assists a task's target objective while learning relational information existing in data. MRL enables MANN to reinforce an association between input data and task objective by reproducing stochastically sampled input data from stored memory contents. With this procedure, MANN further enriches the stored representations with relational information. In experiments, we apply our approaches to Differential Neural Computer (DNC), which is one of the representative content-based addressing memory models and achieves the state-of-the-art performance on both memorization and relational reasoning tasks.
Analysis and Optimization of Service Delay for Multi-quality Videos in Multi-tier Heterogeneous Network with Random Caching
Authors: Xuewei Zhang, Tiejun Lv, Yuan Ren, Wei Ni, Norman C. Beaulieu
Date: 2020-07-21 00:00:00

Aiming to minimize service delay, we propose a new random caching scheme in device-to-device (D2D)-assisted heterogeneous network. To support diversified viewing qualities of multimedia video services, each video file is encoded into a base layer (BL) and multiple enhancement layers (ELs) by scalable video coding (SVC). A super layer, including the BL and several ELs, is transmitted to every user. We define and quantify the service delay of multi-quality videos by deriving successful transmission probabilities when a user is served by a D2D helper, a small-cell base station (SBS) and a macro-cell base station (MBS). We formulate a delay minimization problem subject to the limited cache sizes of D2D helpers and SBSs. The structure of the optimal solutions to the problem is revealed, and then an improved standard gradient projection method is designed to effectively obtain the solutions. Both theoretical analysis and Monte-Carlo simulations validate the successful transmission probabilities. Compared with three benchmark caching policies, the proposed SVC-based random caching scheme is superior in terms of reducing the service delay.
Learning Person Re-identification Models from Videos with Weak Supervision
Authors: Xueping Wang, Sujoy Paul, Dripta S. Raychaudhuri, Min Liu, Yaonan Wang, Amit K. Roy-Chowdhury, Fellow, IEEE
Date: 2020-07-21 00:00:00
Tasks: Multiple Instance Learning, Person Re-Identification, Video-Based Person Re-Identification

Most person re-identification methods, being supervised techniques, suffer from the burden of massive annotation requirement. Unsupervised methods overcome this need for labeled data, but perform poorly compared to the supervised alternatives. In order to cope with this issue, we introduce the problem of learning person re-identification models from videos with weak supervision. The weak nature of the supervision arises from the requirement of video-level labels, i.e. person identities who appear in the video, in contrast to the more precise framelevel annotations. Towards this goal, we propose a multiple instance attention learning framework for person re-identification using such video-level labels. Specifically, we first cast the video person re-identification task into a multiple instance learning setting, in which person images in a video are collected into a bag. The relations between videos with similar labels can be utilized to identify persons, on top of that, we introduce a co-person attention mechanism which mines the similarity correlations between videos with person identities in common. The attention weights are obtained based on all person images instead of person tracklets in a video, making our learned model less affected by noisy annotations. Extensive experiments demonstrate the superiority of the proposed method over the related methods on two weakly labeled person re-identification datasets.
CSLNSpeech: solving extended speech separation problem with the help of Chinese sign language
Authors: Jiasong Wu, Xuan Li, Taotao Li, Fanman Meng, Youyong Kong, Guanyu Yang, Lotfi Senhadji, Huazhong Shu
Date: 2020-07-21 00:00:00
Tasks: Self-Supervised Learning, Speech Separation

Previous audio-visual speech separation methods use the synchronization of the speaker's facial movement and speech in the video to supervise the speech separation in a self-supervised way. In this paper, we propose a model to solve the speech separation problem assisted by both face and sign language, which we call the extended speech separation problem. We design a general deep learning network for learning the combination of three modalities, audio, face, and sign language information, for better solving the speech separation problem. To train the model, we introduce a large-scale dataset named the Chinese Sign Language News Speech (CSLNSpeech) dataset, in which three modalities of audio, face, and sign language coexist. Experiment results show that the proposed model has better performance and robustness than the usual audio-visual system. Besides, sign language modality can also be used alone to supervise speech separation tasks, and the introduction of sign language is helpful for hearing-impaired people to learn and communicate. Last, our model is a general speech separation framework and can achieve very competitive separation performance on two open-source audio-visual datasets. The code is available at https://github.com/iveveive/SLNSpeech
Sparse Nonnegative Tensor Factorization and Completion with Noisy Observations
Authors: Xiongjun Zhang, Michael K. Ng
Date: 2020-07-21 00:00:00
Tasks: Denoising

In this paper, we study the sparse nonnegative tensor factorization and completion problem from partial and noisy observations for third-order tensors. Because of sparsity and nonnegativity, the underlying tensor is decomposed into the tensor-tensor product of one sparse nonnegative tensor and one nonnegative tensor. We propose to minimize the sum of the maximum likelihood estimation for the observations with nonnegativity constraints and the tensor $\ell_0$ norm for the sparse factor. We show that the error bounds of the estimator of the proposed model can be established under general noise observations. The detailed error bounds under specific noise distributions including additive Gaussian noise, additive Laplace noise, and Poisson observations can be derived. Moreover, the minimax lower bounds are shown to be matched with the established upper bounds up to a logarithmic factor of the sizes of the underlying tensor. These theoretical results for tensors are better than those obtained for matrices, and this illustrates the advantage of the use of nonnegative sparse tensor models for completion and denoising. Numerical experiments are provided to validate the superiority of the proposed tensor-based method compared with the matrix-based approach.
Learning Structured Latent Factors from Dependent Data:A Generative Model Framework from Information-Theoretic Perspective
Authors: Ruixiang Zhang, Masanori Koyama, katsuhiko Ishiguro
Date: 2020-07-21 00:00:00
Proceeding: ICML 2020 1
Tasks: Fairness, Inductive Bias

Learning controllable and generalizable representation of multivariate data with desired structural properties remains a fundamental problem in machine learning. In this paper, we present a novel framework for learning generative models with various underlying structures in the latent space. We represent the inductive bias in the form of mask variables to model the dependency structure in the graphical model and extend the theory of multivariate information bottleneck to enforce it. Our model provides a principled approach to learn a set of semantically meaningful latent factors that reflect various types of desired structures like capturing correlation or encoding invariance, while also offering the flexibility to automatically estimate the dependency structure from data. We show that our framework unifies many existing generative models and can be applied to a variety of tasks including multi-modal data modeling, algorithmic fairness, and invariant risk minimization.
Novel View Synthesis on Unpaired Data by Conditional Deformable Variational Auto-Encoder
Authors: Mingyu Yin, Li Sun, Qingli Li
Date: 2020-07-21 00:00:00
Proceeding: ECCV 2020 8
Tasks: Decoder, Disentanglement, Novel View Synthesis, Translation

Novel view synthesis often needs the paired data from both the source and target views. This paper proposes a view translation model under cVAE-GAN framework without requiring the paired data. We design a conditional deformable module (CDM) which uses the view condition vectors as the filters to convolve the feature maps of the main branch in VAE. It generates several pairs of displacement maps to deform the features, like the 2D optical flows. The results are fed into the deformed feature based normalization module (DFNM), which scales and offsets the main branch feature, given its deformed one as the input from the side branch. Taking the advantage of the CDM and DFNM, the encoder outputs a view-irrelevant posterior, while the decoder takes the code drawn from it to synthesize the reconstructed and the viewtranslated images. To further ensure the disentanglement between the views and other factors, we add adversarial training on the code. The results and ablation studies on MultiPIE and 3D chair datasets validate the effectiveness of the framework in cVAE and the designed module.
Word Representation for Rhythms
Authors: Tongyu Lu, Lyucheng Yan, Gus Xia
Date: 2020-07-21 00:00:00
Tasks: Rhythm

This paper proposes a word representation strategy for rhythm patterns. Using 1034 pieces of Nottingham Dataset, a rhythm word dictionary whose size is 450 (without control tokens) is generated. BERT model is created to explore syntactic potentials of rhythm words. Our model is able to find overall music structures and cluster different meters. In a larger scheme, a think mode - music as language - is proposed for systematic considerations.
Inferring Political Preferences from Twitter
Authors: Mohd Zeeshan Ansari, Areesha Fatima Siddiqui, Mohammad Anas
Date: 2020-07-21 00:00:00
Tasks: Sentiment Analysis, text-classification, Text Classification

Sentiment analysis is the task of automatic analysis of opinions and emotions of users towards an entity or some aspect of that entity. Political Sentiment Analysis of social media helps the political strategists to scrutinize the performance of a party or candidate and improvise their weaknesses far before the actual elections. During the time of elections, the social networks get flooded with blogs, chats, debates and discussions about the prospects of political parties and politicians. The amount of data generated is much large to study, analyze and draw inferences using the latest techniques. Twitter is one of the most popular social media platforms enables us to perform domain-specific data preparation. In this work, we chose to identify the inclination of political opinions present in Tweets by modelling it as a text classification problem using classical machine learning. The tweets related to the Delhi Elections in 2020 are extracted and employed for the task. Among the several algorithms, we observe that Support Vector Machines portrays the best performance.
Feature-metric Loss for Self-supervised Learning of Depth and Egomotion
Authors: Chang Shu, Kun Yu, Zhixiang Duan, Kuiyuan Yang
Date: 2020-07-21 00:00:00
Proceeding: ECCV 2020 8
Tasks: Depth Estimation, Monocular Depth Estimation, Self-Supervised Learning, Visual Odometry

Photometric loss is widely used for self-supervised depth and egomotion estimation. However, the loss landscapes induced by photometric differences are often problematic for optimization, caused by plateau landscapes for pixels in textureless regions or multiple local minima for less discriminative pixels. In this work, feature-metric loss is proposed and defined on feature representation, where the feature representation is also learned in a self-supervised manner and regularized by both first-order and second-order derivatives to constrain the loss landscapes to form proper convergence basins. Comprehensive experiments and detailed analysis via visualization demonstrate the effectiveness of the proposed feature-metric loss. In particular, our method improves state-of-the-art methods on KITTI from 0.885 to 0.925 measured by $\delta_1$ for depth estimation, and significantly outperforms previous method for visual odometry.
Graph-PCNN: Two Stage Human Pose Estimation with Graph Pose Refinement
Authors: Jian Wang, Xiang Long, Yuan Gao, Errui Ding, Shilei Wen
Date: 2020-07-21 00:00:00
Proceeding: ECCV 2020 8
Tasks: Pose Estimation, regression, Vocal Bursts Valence Prediction

Recently, most of the state-of-the-art human pose estimation methods are based on heatmap regression. The final coordinates of keypoints are obtained by decoding heatmap directly. In this paper, we aim to find a better approach to get more accurate localization results. We mainly put forward two suggestions for improvement: 1) different features and methods should be applied for rough and accurate localization, 2) relationship between keypoints should be considered. Specifically, we propose a two-stage graph-based and model-agnostic framework, called Graph-PCNN, with a localization subnet and a graph pose refinement module added onto the original heatmap regression network. In the first stage, heatmap regression network is applied to obtain a rough localization result, and a set of proposal keypoints, called guided points, are sampled. In the second stage, for each guided point, different visual feature is extracted by the localization subnet. The relationship between guided points is explored by the graph pose refinement module to get more accurate localization results. Experiments show that Graph-PCNN can be used in various backbones to boost the performance by a large margin. Without bells and whistles, our best model can achieve a new state-of-the-art 76.8% AP on COCO test-dev split.
Incentives for Federated Learning: a Hypothesis Elicitation Approach
Authors: Yang Liu, Jiaheng Wei
Date: 2020-07-21 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning, scoring rule

Federated learning provides a promising paradigm for collecting machine learning models from distributed data sources without compromising users' data privacy. The success of a credible federated learning system builds on the assumption that the decentralized and self-interested users will be willing to participate to contribute their local models in a trustworthy way. However, without proper incentives, users might simply opt out the contribution cycle, or will be mis-incentivized to contribute spam/false information. This paper introduces solutions to incentivize truthful reporting of a local, user-side machine learning model for federated learning. Our results build on the literature of information elicitation, but focus on the questions of eliciting hypothesis (rather than eliciting human predictions). We provide a scoring rule based framework that incentivizes truthful reporting of local hypotheses at a Bayesian Nash Equilibrium. We study the market implementation, accuracy as well as robustness properties of our proposed solution too. We verify the effectiveness of our methods using MNIST and CIFAR-10 datasets. Particularly we show that by reporting low-quality hypotheses, users will receive decreasing scores (rewards, or payments).
Video Super-resolution with Temporal Group Attention
Authors: Takashi Isobe, Songjiang Li, Xu Jia, Shanxin Yuan, Gregory Slabaugh, Chunjing Xu, Ya-Li Li, Shengjin Wang, Qi Tian
Date: 2020-07-21 00:00:00
Proceeding: CVPR 2020 6
Tasks: Super-Resolution, Video Super-Resolution

Video super-resolution, which aims at producing a high-resolution video from its corresponding low-resolution version, has recently drawn increasing attention. In this work, we propose a novel method that can effectively incorporate temporal information in a hierarchical way. The input sequence is divided into several groups, with each one corresponding to a kind of frame rate. These groups provide complementary information to recover missing details in the reference frame, which is further integrated with an attention module and a deep intra-group fusion module. In addition, a fast spatial alignment is proposed to handle videos with large motion. Extensive results demonstrate the capability of the proposed model in handling videos with various motion. It achieves favorable performance against state-of-the-art methods on several benchmark datasets.