pwc-archive/papers-with-abstracts huggingface dataset:
Stay on Topic, Please: Aligning User Comments to the Content of a News Article
Authors: Jumanah Alshehri, Marija Stanojevic, Eduard Dragut, Zoran Obradovic
Date: 2021-03-03 00:00:00
Tasks: Articles, Classification, General Classification, Ordinal Classification

Social scientists have shown that up to 50% if the content posted to a news article have no relation to its journalistic content. In this study we propose a classification algorithm to categorize user comments posted to a new article base don their alignment to its content. The alignment seek to match user comments to an article based on similarity off content, entities in discussion, and topic. We proposed a BERTAC, BAERT-based approach that learn jointly article-comment embeddings and infers the relevance class of comments. We introduce an ordinal classification loss that penalizes the difference between the predicted and true label. We conduct a thorough study to show influence of the proposed loss on the learning process. The results on five representative news outlets show that our approach can learn the comment class with up to 36% average accuracy improvement compering to the baselines, and up to 25% compering to the BA-BC model. BA-BC is out approach that consists of two models aimed to capture dis-jointly the formal language of news articles and the informal language of comments. We also conduct a user study to evaluate human labeling performance to understand the difficulty of the classification task. The user agreement on comment-article alignment is "moderate" per Krippendorff's alpha score, which suggests that the classification task is difficult.
Semi-supervised Learning for COVID-19 Image Classification via ResNet
Authors: Lucy Nwosu, Xiangfang Li, Lijun Qian, Seungchan Kim, Xishuang Dong
Date: 2021-02-27 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Coronavirus disease 2019 (COVID-19) is an ongoing global pandemic in over 200 countries and territories, which has resulted in a great public health concern across the international community. Analysis of X-ray imaging data can play a critical role in timely and accurate screening and fighting against COVID-19. Supervised deep learning has been successfully applied to recognize COVID-19 pathology from X-ray imaging datasets. However, it requires a substantial amount of annotated X-ray images to train models, which is often not applicable to data analysis for emerging events such as COVID-19 outbreak, especially in the early stage of the outbreak. To address this challenge, this paper proposes a two-path semi-supervised deep learning model, ssResNet, based on Residual Neural Network (ResNet) for COVID-19 image classification, where two paths refer to a supervised path and an unsupervised path, respectively. Moreover, we design a weighted supervised loss that assigns higher weight for the minority classes in the training process to resolve the data imbalance. Experimental results on a large-scale of X-ray image dataset COVIDx demonstrate that the proposed model can achieve promising performance even when trained on very few labeled training images.
Reservoir Computing as a Tool for Climate Predictability Studies
Authors: B. T. Nadiga
Date: 2021-02-24 00:00:00

Reduced-order dynamical models play a central role in developing our understanding of predictability of climate irrespective of whether we are dealing with the actual climate system or surrogate climate-models. In this context, the Linear-Inverse-Modeling (LIM) approach, by capturing a few essential interactions between dynamical components of the full system, has proven valuable in providing insights into predictability of the full system. We demonstrate that Reservoir Computing (RC), a form of learning suitable for systems with chaotic dynamics, provides an alternative nonlinear approach that improves on the predictive skill of the LIM approach. We do this in the example setting of predicting sea-surface-temperature in the North Atlantic in the pre-industrial control simulation of a popular earth system model, the Community-Earth-System-Model so that we can compare the performance of the new RC based approach with the traditional LIM approach both when learning data is plentiful and when such data is more limited. The improved predictive skill of the RC approach over a wide range of conditions -- larger number of retained EOF coefficients, extending well into the limited data regime, etc. -- suggests that this machine-learning technique may have a use in climate predictability studies. While the possibility of developing a climate emulator -- the ability to continue the evolution of the system on the attractor long after failing to be able to track the reference trajectory -- is demonstrated in the Lorenz-63 system, it is suggested that further development of the RC approach may permit such uses of the new approach in more realistic predictability studies.
u-cf2vec: Representation Learning for Personalized Algorithm Selection in Recommender Systems
Authors: Tomas Sousa-Pereira, Tiago Cunha, Carlos Soares
Date: 2021-03-09 00:00:00
Tasks: Collaborative Filtering, Meta-Learning, Recommendation Systems, Representation Learning

Collaborative Filtering (CF) has become the standard approach to solve recommendation systems (RS) problems. Collaborative Filtering algorithms try to make predictions about interests of a user by collecting the personal interests from multiple users. There are multiple CF algorithms, each one of them with its own biases. It is the Machine Learning practitioner that has to choose the best algorithm for each task beforehand. In Recommender Systems, different algorithms have different performance for different users within the same dataset. Meta Learning (MtL) has been used to choose the best algorithm for a given problem. Meta Learning is usually applied to select algorithms for a whole dataset. Adapting it to select the to the algorithm for a single user in a RS involves several challenges. The most important is the design of the metafeatures which, in typical meta learning, characterize datasets while here, they must characterize a single user. This work presents a new meta-learning based framework named $\mu$-cf2vec to select the best algorithm for each user. We propose using Representation Learning techniques to extract the metafeatures. Representation Learning tries to extract representations that can be reused in other learning tasks. In this work we also implement the framework using different RL techniques to evaluate which one can be more useful to solve this task. In the meta level, the meta learning model will use the metafeatures to extract knowledge that will be used to predict the best algorithm for each user. We evaluated an implementation of this framework using MovieLens 20M dataset. Our implementation achieved consistent gains in the meta level, however, in the base level we only achieved marginal gains.
Improving Sequential Recommendation with Attribute-augmented Graph Neural Networks
Authors: Xinzhou Dong, Beihong Jin, Wei Zhuo, Beibei Li, Taofeng Xue
Date: 2021-03-10 00:00:00
Tasks: Attribute, Graph Neural Network, Recommendation Systems, Sequential Recommendation

Many practical recommender systems provide item recommendation for different users only via mining user-item interactions but totally ignoring the rich attribute information of items that users interact with. In this paper, we propose an attribute-augmented graph neural network model named Murzim. Murzim takes as input the graphs constructed from the user-item interaction sequences and corresponding item attribute sequences. By combining the GNNs with node aggregation and an attention network, Murzim can capture user preference patterns, generate embeddings for user-item interaction sequences, and then generate recommendations through next-item prediction. We conduct extensive experiments on multiple datasets. Experimental results show that Murzim outperforms several state-of-the-art methods in terms of recall and MRR, which illustrates that Murzim can make use of item attribute information to produce better recommendations. At present, Murzim has been deployed in MX Player, one of India's largest streaming platforms, and is recommending videos for tens of thousands of users.
Task-adaptive Neural Process for User Cold-Start Recommendation
Authors: Xixun Lin, Jia Wu, Chuan Zhou, Shirui Pan, Yanan Cao, Bin Wang
Date: 2021-02-26 00:00:00
Tasks: Meta-Learning, Recommendation Systems

User cold-start recommendation is a long-standing challenge for recommender systems due to the fact that only a few interactions of cold-start users can be exploited. Recent studies seek to address this challenge from the perspective of meta learning, and most of them follow a manner of parameter initialization, where the model parameters can be learned by a few steps of gradient updates. While these gradient-based meta-learning models achieve promising performances to some extent, a fundamental problem of them is how to adapt the global knowledge learned from previous tasks for the recommendations of cold-start users more effectively. In this paper, we develop a novel meta-learning recommender called task-adaptive neural process (TaNP). TaNP is a new member of the neural process family, where making recommendations for each user is associated with a corresponding stochastic process. TaNP directly maps the observed interactions of each user to a predictive distribution, sidestepping some training issues in gradient-based meta-learning models. More importantly, to balance the trade-off between model capacity and adaptation reliability, we introduce a novel task-adaptive mechanism. It enables our model to learn the relevance of different tasks and customize the global knowledge to the task-related decoder parameters for estimating user preferences. We validate TaNP on multiple benchmark datasets in different experimental settings. Empirical results demonstrate that TaNP yields consistent improvements over several state-of-the-art meta-learning recommenders.
SMIL: Multimodal Learning with Severely Missing Modality
Authors: Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, Xi Peng
Date: 2021-03-09 00:00:00
Tasks: Meta-Learning

A common assumption in multimodal learning is the completeness of training data, i.e., full modalities are available in all training examples. Although there exists research endeavor in developing novel methods to tackle the incompleteness of testing data, e.g., modalities are partially missing in testing examples, few of them can handle incomplete training modalities. The problem becomes even more challenging if considering the case of severely missing, e.g., 90% training examples may have incomplete modalities. For the first time in the literature, this paper formally studies multimodal learning with missing modality in terms of flexibility (missing modalities in training, testing, or both) and efficiency (most training data have incomplete modality). Technically, we propose a new method named SMIL that leverages Bayesian meta-learning in uniformly achieving both objectives. To validate our idea, we conduct a series of experiments on three popular benchmarks: MM-IMDb, CMU-MOSI, and avMNIST. The results prove the state-of-the-art performance of SMIL over existing methods and generative baselines including autoencoders and generative adversarial networks. Our code is available at https://github.com/mengmenm/SMIL.
Capturing Omni-Range Context for Omnidirectional Segmentation
Authors: Kailun Yang, Jiaming Zhang, Simon Reiß, Xinxin Hu, Rainer Stiefelhagen
Date: 2021-03-09 00:00:00
Proceeding: CVPR 2021 1
Tasks: Autonomous Driving, Image Segmentation, Segmentation, Semantic Segmentation

Convolutional Networks (ConvNets) excel at semantic segmentation and have become a vital component for perception in autonomous driving. Enabling an all-encompassing view of street-scenes, omnidirectional cameras present themselves as a perfect fit in such systems. Most segmentation models for parsing urban environments operate on common, narrow Field of View (FoV) images. Transferring these models from the domain they were designed for to 360-degree perception, their performance drops dramatically, e.g., by an absolute 30.0% (mIoU) on established test-beds. To bridge the gap in terms of FoV and structural distribution between the imaging domains, we introduce Efficient Concurrent Attention Networks (ECANets), directly capturing the inherent long-range dependencies in omnidirectional imagery. In addition to the learned attention-based contextual priors that can stretch across 360-degree images, we upgrade model training by leveraging multi-source and omni-supervised learning, taking advantage of both: Densely labeled and unlabeled data originating from multiple datasets. To foster progress in panoramic image segmentation, we put forward and extensively evaluate models on Wild PAnoramic Semantic Segmentation (WildPASS), a dataset designed to capture diverse scenes from all around the globe. Our novel model, training regimen and multi-source prediction fusion elevate the performance (mIoU) to new state-of-the-art results on the public PASS (60.2%) and the fresh WildPASS (69.0%) benchmarks.
Multitask 3D CBCT-to-CT Translation and Organs-at-Risk Segmentation Using Physics-Based Data Augmentation
Authors: Navdeep Dahiya, Sadegh R Alam, Pengpeng Zhang, Si-Yuan Zhang, Anthony Yezzi, Saad Nadeem
Date: 2021-03-09 00:00:00
Tasks: Data Augmentation, Translation

In current clinical practice, noisy and artifact-ridden weekly cone-beam computed tomography (CBCT) images are only used for patient setup during radiotherapy. Treatment planning is done once at the beginning of the treatment using high-quality planning CT (pCT) images and manual contours for organs-at-risk (OARs) structures. If the quality of the weekly CBCT images can be improved while simultaneously segmenting OAR structures, this can provide critical information for adapting radiotherapy mid-treatment as well as for deriving biomarkers for treatment response. Using a novel physics-based data augmentation strategy, we synthesize a large dataset of perfectly/inherently registered planning CT and synthetic-CBCT pairs for locally advanced lung cancer patient cohort, which are then used in a multitask 3D deep learning framework to simultaneously segment and translate real weekly CBCT images to high-quality planning CT-like images. We compared the synthetic CT and OAR segmentations generated by the model to real planning CT and manual OAR segmentations and showed promising results. The real week 1 (baseline) CBCT images which had an average MAE of 162.77 HU compared to pCT images are translated to synthetic CT images that exhibit a drastically improved average MAE of 29.31 HU and average structural similarity of 92% with the pCT images. The average DICE scores of the 3D organs-at-risk segmentations are: lungs 0.96, heart 0.88, spinal cord 0.83 and esophagus 0.66. This approach could allow clinicians to adjust treatment plans using only the routine low-quality CBCT images, potentially improving patient outcomes. Our code, data, and pre-trained models will be made available via our physics-based data augmentation library, Physics-ArX, at https://github.com/nadeemlab/Physics-ArX.
Pixel-wise Distance Regression for Glacier Calving Front Detection and Segmentation
Authors: Amirabbas Davari, Christoph Baller, Thorsten Seehaus, Matthias Braun, Andreas Maier, Vincent Christlein
Date: 2021-03-09 00:00:00
Tasks: Distance regression, regression

Glacier calving front position (CFP) is an important glaciological variable. Traditionally, delineating the CFPs has been carried out manually, which was subjective, tedious and expensive. Automating this process is crucial for continuously monitoring the evolution and status of glaciers. Recently, deep learning approaches have been investigated for this application. However, the current methods get challenged by a severe class-imbalance problem. In this work, we propose to mitigate the class-imbalance between the calving front class and the non-calving front class by reformulating the segmentation problem into a pixel-wise regression task. A Convolutional Neural Network gets optimized to predict the distance values to the glacier front for each pixel in the image. The resulting distance map localizes the CFP and is further post-processed to extract the calving front line. We propose three post-processing methods, one method based on statistical thresholding, a second method based on conditional random fields (CRF), and finally the use of a second U-Net. The experimental results confirm that our approach significantly outperforms the state-of-the-art methods and produces accurate delineation. The Second U-Net obtains the best performance results, resulting in an average improvement of about 21% dice coefficient enhancement.
A Multi-resolution Approach to Expression Recognition in the Wild
Authors: Fabio Valerio Massoli, Donato Cafarelli, Giuseppe Amato, Fabrizio Falchi
Date: 2021-03-09 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

Facial expressions play a fundamental role in human communication. Indeed, they typically reveal the real emotional status of people beyond the spoken language. Moreover, the comprehension of human affect based on visual patterns is a key ingredient for any human-machine interaction system and, for such reasons, the task of Facial Expression Recognition (FER) draws both scientific and industrial interest. In the recent years, Deep Learning techniques reached very high performance on FER by exploiting different architectures and learning paradigms. In such a context, we propose a multi-resolution approach to solve the FER task. We ground our intuition on the observation that often faces images are acquired at different resolutions. Thus, directly considering such property while training a model can help achieve higher performance on recognizing facial expressions. To our aim, we use a ResNet-like architecture, equipped with Squeeze-and-Excitation blocks, trained on the Affect-in-the-Wild 2 dataset. Not being available a test set, we conduct tests and models selection by employing the validation set only on which we achieve more than 90\% accuracy on classifying the seven expressions that the dataset comprises.
Structural Connectome Atlas Construction in the Space of Riemannian Metrics
Authors: Kristen M. Campbell, Haocheng Dai, Zhe Su, Martin Bauer, P. Thomas Fletcher, Sarang C. Joshi
Date: 2021-03-09 00:00:00

The structural connectome is often represented by fiber bundles generated from various types of tractography. We propose a method of analyzing connectomes by representing them as a Riemannian metric, thereby viewing them as points in an infinite-dimensional manifold. After equipping this space with a natural metric structure, the Ebin metric, we apply object-oriented statistical analysis to define an atlas as the Fr\'echet mean of a population of Riemannian metrics. We demonstrate connectome registration and atlas formation using connectomes derived from diffusion tensors estimated from a subset of subjects from the Human Connectome Project.
Learning to compose 6-DoF omnidirectional videos using multi-sphere images
Authors: Jisheng Li, Yuze He, Yubin Hu, Yuxing Han, Jiangtao Wen
Date: 2021-03-10 00:00:00
Tasks: Depth Estimation

Omnidirectional video is an essential component of Virtual Reality. Although various methods have been proposed to generate content that can be viewed with six degrees of freedom (6-DoF), existing systems usually involve complex depth estimation, image in-painting or stitching pre-processing. In this paper, we propose a system that uses a 3D ConvNet to generate a multi-sphere images (MSI) representation that can be experienced in 6-DoF VR. The system utilizes conventional omnidirectional VR camera footage directly without the need for a depth map or segmentation mask, thereby significantly simplifying the overall complexity of the 6-DoF omnidirectional video composition. By using a newly designed weighted sphere sweep volume (WSSV) fusing technique, our approach is compatible with most panoramic VR camera setups. A ground truth generation approach for high-quality artifact-free 6-DoF contents is proposed and can be used by the research and development community for 6-DoF content generation.
Novel tile segmentation scheme for omnidirectional video
Authors: Jisheng Li, Ziyu Wen, Sihan Li, Yikai Zhao, Bichuan Guo, Jiangtao Wen
Date: 2021-03-10 00:00:00
Tasks: Video Segmentation, Video Semantic Segmentation

Regular omnidirectional video encoding technics use map projection to flatten a scene from a spherical shape into one or several 2D shapes. Common projection methods including equirectangular and cubic projection have varying levels of interpolation that create a large number of non-information-carrying pixels that lead to wasted bitrate. In this paper, we propose a tile based omnidirectional video segmentation scheme which can save up to 28% of pixel area and 20% of BD-rate averagely compared to the traditional equirectangular projection based approach.
Manifold Regularized Dynamic Network Pruning
Authors: Yehui Tang, Yunhe Wang, Yixing Xu, Yiping Deng, Chao Xu, DaCheng Tao, Chang Xu
Date: 2021-03-10 00:00:00
Proceeding: CVPR 2021 1
Tasks: Network Pruning

Neural network pruning is an essential approach for reducing the computational complexity of deep models so that they can be well deployed on resource-limited devices. Compared with conventional methods, the recently developed dynamic pruning methods determine redundant filters variant to each input instance which achieves higher acceleration. Most of the existing methods discover effective sub-networks for each instance independently and do not utilize the relationship between different inputs. To maximally excavate redundancy in the given network architecture, this paper proposes a new paradigm that dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks (dubbed as ManiDP). We first investigate the recognition complexity and feature similarity between images in the training set. Then, the manifold relationship between instances and the pruned sub-networks will be aligned in the training procedure. The effectiveness of the proposed method is verified on several benchmarks, which shows better performance in terms of both accuracy and computational cost compared to the state-of-the-art methods. For example, our method can reduce 55.3% FLOPs of ResNet-34 with only 0.57% top-1 accuracy degradation on ImageNet.
Tuna Nutriment Tracking using Trajectory Mapping in Application to Aquaculture Fish Tank
Authors: Hilmil Pradana, Keiichi Horio
Date: 2021-03-10 00:00:00

The cost of fish feeding is usually around 40 percent of total production cost. Estimating a state of fishes in a tank and adjusting an amount of nutriments play an important role to manage cost of fish feeding system. Our approach is based on tracking nutriments on videos collected from an active aquaculture fish farm. Tracking approach is applied to acknowledge movement of nutriment to understand more about the fish behavior. Recently, there has been increasing number of researchers focused on developing tracking algorithms to generate more accurate and faster determination of object. Unfortunately, recent studies have shown that efficient and robust tracking of multiple objects with complex relations remain unsolved. Hence, focusing to develop tracking algorithm in aquaculture is more challenging because tracked object has a lot of aquatic variant creatures. By following aforementioned problem, we develop tuna nutriment tracking based on the classical minimum cost problem which consistently performs well in real environment datasets. In evaluation, the proposed method achieved 21.32 pixels and 3.08 pixels for average error distance and standard deviation, respectively. Quantitative evaluation based on the data generated by human annotators shows that the proposed method is valuable for aquaculture fish farm and can be widely applied to real environment datasets.