pwc-archive/papers-with-abstracts huggingface dataset:
Deep-VFX: Deep Action Recognition Driven VFX for Short Video
Authors: Ao Luo, Ning Xie, Zhijia Tao, Feng Jiang
Date: 2020-07-22 00:00:00
Tasks: Action Recognition, Rhythm, Template Matching

Human motion is a key function to communicate information. In the application, short-form mobile video is so popular all over the world such as Tik Tok. The users would like to add more VFX so as to pursue creativity and personlity. Many special effects are added on the short video platform. These gives the users more possibility to show off these personality. The common and traditional way is to create the template of VFX. However, in order to synthesis the perfect, the users have to tedious attempt to grasp the timing and rhythm of new templates. It is not easy-to-use especially for the mobile app. This paper aims to change the VFX synthesis by motion driven instead of the traditional template matching. We propose the AI method to improve this VFX synthesis. In detail, in order to add the special effect on the human body. The skeleton extraction is essential in this system. We also propose a novel form of LSTM to find out the user's intention by action recognition. The experiment shows that our system enables to generate VFX for short video more easier and efficient.
Improving Monocular Depth Estimation by Leveraging Structural Awareness and Complementary Datasets
Authors: Tian Chen, Shijie An, Yuan Zhang, Chongyang Ma, Huayan Wang, Xiaoyan Guo, Wen Zheng
Date: 2020-07-22 00:00:00
Proceeding: ECCV 2020 8
Tasks: Depth Estimation, Monocular Depth Estimation

Monocular depth estimation plays a crucial role in 3D recognition and understanding. One key limitation of existing approaches lies in their lack of structural information exploitation, which leads to inaccurate spatial layout, discontinuous surface, and ambiguous boundaries. In this paper, we tackle this problem in three aspects. First, to exploit the spatial relationship of visual features, we propose a structure-aware neural network with spatial attention blocks. These blocks guide the network attention to global structures or local details across different feature layers. Second, we introduce a global focal relative loss for uniform point pairs to enhance spatial constraint in the prediction, and explicitly increase the penalty on errors in depth-wise discontinuous regions, which helps preserve the sharpness of estimation results. Finally, based on analysis of failure cases for prior methods, we collect a new Hard Case (HC) Depth dataset of challenging scenes, such as special lighting conditions, dynamic objects, and tilted camera angles. The new dataset is leveraged by an informed learning curriculum that mixes training examples incrementally to handle diverse data distributions. Experimental results show that our method outperforms state-of-the-art approaches by a large margin in terms of both prediction accuracy on NYUDv2 dataset and generalization performance on unseen datasets.
DeepCLR: Correspondence-Less Architecture for Deep End-to-End Point Cloud Registration
Authors: Markus Horn, Nico Engel, Vasileios Belagiannis, Michael Buchholz, Klaus Dietmayer
Date: 2020-07-22 00:00:00
Tasks: Point Cloud Registration

This work addresses the problem of point cloud registration using deep neural networks. We propose an approach to predict the alignment between two point clouds with overlapping data content, but displaced origins. Such point clouds originate, for example, from consecutive measurements of a LiDAR mounted on a moving platform. The main difficulty in deep registration of raw point clouds is the fusion of template and source point cloud. Our proposed architecture applies flow embedding to tackle this problem, which generates features that describe the motion of each template point. These features are then used to predict the alignment in an end-to-end fashion without extracting explicit point correspondences between both input clouds. We rely on the KITTI odometry and ModelNet40 datasets for evaluating our method on various point distributions. Our approach achieves state-of-the-art accuracy and the lowest run-time of the compared methods.
Fragments-Expert: A Graphical User Interface MATLAB Toolbox for Classification of File Fragments
Authors: Mehdi Teimouri, Zahra Seyedghorban, Fatemeh Amirjani
Date: 2020-07-22 00:00:00
Proceeding: Concurrency and Computation Practice and Experience 2020 12
Tasks: BIG-bench Machine Learning, Classification, General Classification, Intrusion Detection

The classification of file fragments of various file formats is an essential task in various applications such as firewalls, intrusion detection systems, anti-viruses, web content filtering, and digital forensics. However, the community lacks a suitable software tool that can integrate major methods for feature extraction from file fragments and classification among various file formats. In this paper, we present Fragments-Expert that is a graphical user interface MATLAB toolbox for the classification of file fragments. It provides users with 22 categories of features extracted from file fragments. These features can be employed by 7 categories of machine learning algorithms for the task of classification among various file formats.
Learnable Descent Algorithm for Nonsmooth Nonconvex Image Reconstruction
Authors: Yunmei Chen, Hongcheng Liu, Xiaojing Ye, Qingchao Zhang
Date: 2020-07-22 00:00:00
Tasks: Image Reconstruction

We propose a general learning based framework for solving nonsmooth and nonconvex image reconstruction problems. We model the regularization function as the composition of the $l_{2,1}$ norm and a smooth but nonconvex feature mapping parametrized as a deep convolutional neural network. We develop a provably convergent descent-type algorithm to solve the nonsmooth nonconvex minimization problem by leveraging the Nesterov's smoothing technique and the idea of residual learning, and learn the network parameters such that the outputs of the algorithm match the references in training data. Our method is versatile as one can employ various modern network structures into the regularization, and the resulting network inherits the guaranteed convergence of the algorithm. We also show that the proposed network is parameter-efficient and its performance compares favorably to the state-of-the-art methods in a variety of image reconstruction problems in practice.
Edge-aware Graph Representation Learning and Reasoning for Face Parsing
Authors: Gusi Te, Yinglu Liu, Wei Hu, Hailin Shi, Tao Mei
Date: 2020-07-22 00:00:00
Proceeding: ECCV 2020 8
Tasks: Face Parsing, Graph Representation Learning, Representation Learning

Face parsing infers a pixel-wise label to each facial component, which has drawn much attention recently. Previous methods have shown their efficiency in face parsing, which however overlook the correlation among different face regions. The correlation is a critical clue about the facial appearance, pose, expression etc., and should be taken into account for face parsing. To this end, we propose to model and reason the region-wise relations by learning graph representations, and leverage the edge information between regions for optimized abstraction. Specifically, we encode a facial image onto a global graph representation where a collection of pixels ("regions") with similar features are projected to each vertex. Our model learns and reasons over relations between the regions by propagating information across vertices on the graph. Furthermore, we incorporate the edge information to aggregate the pixel-wise features onto vertices, which emphasizes on the features around edges for fine segmentation along edges. The finally learned graph representation is projected back to pixel grids for parsing. Experiments demonstrate that our model outperforms state-of-the-art methods on the widely used Helen dataset, and also exhibits the superior performance on the large-scale CelebAMask-HQ and LaPa dataset. The code is available at https://github.com/tegusi/EAGRNet.
MetAL: Active Semi-Supervised Learning on Graphs via Meta Learning
Authors: Kaushalya Madhawa, Tsuyoshi Murata
Date: 2020-07-22 00:00:00
Tasks: Active Learning, Bilevel Optimization, Meta-Learning

The objective of active learning (AL) is to train classification models with less number of labeled instances by selecting only the most informative instances for labeling. The AL algorithms designed for other data types such as images and text do not perform well on graph-structured data. Although a few heuristics-based AL algorithms have been proposed for graphs, a principled approach is lacking. In this paper, we propose MetAL, an AL approach that selects unlabeled instances that directly improve the future performance of a classification model. For a semi-supervised learning problem, we formulate the AL task as a bilevel optimization problem. Based on recent work in meta-learning, we use the meta-gradients to approximate the impact of retraining the model with any unlabeled instance on the model performance. Using multiple graph datasets belonging to different domains, we demonstrate that MetAL efficiently outperforms existing state-of-the-art AL algorithms.
Greenhouse Segmentation on High-Resolution Optical Satellite Imagery using Deep Learning Techniques
Authors: Orkhan Baghirli, Imran Ibrahimli, Tarlan Mammadzada
Date: 2020-07-22 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Greenhouse segmentation has pivotal importance for climate-smart agricultural land-use planning. Deep learning-based approaches provide state-of-the-art performance in natural image segmentation. However, semantic segmentation on high-resolution optical satellite imagery is a challenging task because of the complex environment. In this paper, a sound methodology is proposed for pixel-wise classification on images acquired by the Azersky (SPOT-7) optical satellite. In particular, customized variations of U-Net-like architectures are employed to identify greenhouses. Two models are proposed which uniquely incorporate dilated convolutions and skip connections, and the results are compared to that of the baseline U-Net model. The dataset used consists of pan-sharpened orthorectified Azersky images (red, green, blue,and near infrared channels) with 1.5-meter resolution and annotation masks, collected from 15 regions in Azerbaijan where the greenhouses are densely congested. The images cover the cumulative area of 1008 $km^2$ and annotation masks contain 47559 polygons in total. The $F_1, Kappa, AUC$, and $IOU$ scores are used for performance evaluation. It is observed that the use of the deconvolutional layers alone throughout the expansive path does not yield satisfactory results; therefore, they are either replaced or coupled with bilinear interpolation. All models benefit from the hard example mining (HEM) strategy. It is also reported that the best accuracy of $93.29\%$ ($F_1\,score$) is recorded when the weighted binary cross-entropy loss is coupled with the dice loss. Experimental results showed that both of the proposed models outperformed the baseline U-Net architecture such that the best model proposed scored $4.48\%$ higher in comparison to the baseline architecture.
Regulating human control over autonomous systems
Authors: Mikolaj firlej, Araz Taeihagh
Date: 2020-07-22 00:00:00
Tasks: Autonomous Vehicles

In recent years, many sectors have experienced significant progress in automation, associated with the growing advances in artificial intelligence and machine learning. There are already automated robotic weapons, which are able to evaluate and engage with targets on their own, and there are already autonomous vehicles that do not need a human driver. It is argued that the use of increasingly autonomous systems (AS) should be guided by the policy of human control, according to which humans should execute a certain significant level of judgment over AS. While in the military sector there is a fear that AS could mean that humans lose control over life and death decisions, in the transportation domain, on the contrary, there is a strongly held view that autonomy could bring significant operational benefits by removing the need for a human driver. This article explores the notion of human control in the United States in the two domains of defense and transportation. The operationalization of emerging policies of human control results in the typology of direct and indirect human controls exercised over the use of AS. The typology helps to steer the debate away from the linguistic complexities of the term autonomy. It identifies instead where human factors are undergoing important changes and ultimately informs about more detailed rules and standards formulation, which differ across domains, applications, and sectors.
IBM Federated Learning: an Enterprise Framework White Paper V0.1
Authors: Heiko Ludwig, Nathalie Baracaldo, Gegi Thomas, Yi Zhou, Ali Anwar, Shashank Rajamoni, Yuya Ong, Jayaram Radhakrishnan, Ashish Verma, Mathieu Sinn, Mark Purcell, Ambrish Rawat, Tran Minh, Naoise Holohan, Supriyo Chakraborty, Shalisha Whitherspoon, Dean Steuer, Laura Wynter, Hifaz Hassan, Sean Laguna, Mikhail Yurochkin, Mayank Agarwal, Ebube Chuba, Annie Abay
Date: 2020-07-22 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning

Federated Learning (FL) is an approach to conduct machine learning without centralizing training data in a single place, for reasons of privacy, confidentiality or data volume. However, solving federated machine learning problems raises issues above and beyond those of centralized machine learning. These issues include setting up communication infrastructure between parties, coordinating the learning process, integrating party results, understanding the characteristics of the training data sets of different participating parties, handling data heterogeneity, and operating with the absence of a verification data set. IBM Federated Learning provides infrastructure and coordination for federated learning. Data scientists can design and run federated learning jobs based on existing, centralized machine learning models and can provide high-level instructions on how to run the federation. The framework applies to both Deep Neural Networks as well as ``traditional'' approaches for the most common machine learning libraries. {\proj} enables data scientists to expand their scope from centralized to federated machine learning, minimizing the learning curve at the outset while also providing the flexibility to deploy to different compute environments and design custom fusion algorithms.
Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains
Authors: Ziqi Liu, Wei Cai, Zhi-Qin John Xu
Date: 2020-07-22 00:00:00

In this paper, we propose multi-scale deep neural networks (MscaleDNNs) using the idea of radial scaling in frequency domain and activation functions with compact support. The radial scaling converts the problem of approximation of high frequency contents of PDEs' solutions to a problem of learning about lower frequency functions, and the compact support activation functions facilitate the separation of frequency contents of the target function to be approximated by corresponding DNNs. As a result, the MscaleDNNs achieve fast uniform convergence over multiple scales. The proposed MscaleDNNs are shown to be superior to traditional fully connected DNNs and be an effective mesh-less numerical method for Poisson-Boltzmann equations with ample frequency contents over complex and singular domains.
SOCRATES: Towards a Unified Platform for Neural Network Analysis
Authors: Long H. Pham, Jiaying Li, Jun Sun
Date: 2020-07-22 00:00:00
Tasks: Face Recognition, Fairness, Self-Driving Cars

Studies show that neural networks, not unlike traditional programs, are subject to bugs, e.g., adversarial samples that cause classification errors and discriminatory instances that demonstrate the lack of fairness. Given that neural networks are increasingly applied in critical applications (e.g., self-driving cars, face recognition systems and personal credit rating systems), it is desirable that systematic methods are developed to analyze (e.g., test or verify) neural networks against desirable properties. Recently, a number of approaches have been developed for analyzing neural networks. These efforts are however scattered (i.e., each approach tackles some restricted classes of neural networks against certain particular properties), incomparable (i.e., each approach has its own assumptions and input format) and thus hard to apply, reuse or extend. In this project, we aim to build a unified framework for developing techniques to analyze neural networks. Towards this goal, we develop a platform called SOCRATES which supports a standardized format for a variety of neural network models, an assertion language for property specification as well as multiple neural network analysis algorithms including two novel ones for falsifying and probabilistic verification of neural network models. SOCRATES is extensible and thus existing approaches can be easily integrated. Experiment results show that our platform can handle a wide range of networks models and properties. More importantly, it provides a platform for synergistic research on neural network analysis.
MathNet: Haar-Like Wavelet Multiresolution-Analysis for Graph Representation and Learning
Authors: Xuebin Zheng, Bingxin Zhou, Ming Li, Yu Guang Wang, Junbin Gao
Date: 2020-07-22 00:00:00
Tasks: Graph Classification

Graph Neural Networks (GNNs) have recently caught great attention and achieved significant progress in graph-level applications. In this paper, we propose a framework for graph neural networks with multiresolution Haar-like wavelets, or MathNet, with interrelated convolution and pooling strategies. The underlying method takes graphs in different structures as input and assembles consistent graph representations for readout layers, which then accomplishes label prediction. To achieve this, the multiresolution graph representations are first constructed and fed into graph convolutional layers for processing. The hierarchical graph pooling layers are then involved to downsample graph resolution while simultaneously remove redundancy within graph signals. The whole workflow could be formed with a multi-level graph analysis, which not only helps embed the intrinsic topological information of each graph into the GNN, but also supports fast computation of forward and adjoint graph transforms. We show by extensive experiments that the proposed framework obtains notable accuracy gains on graph classification and regression tasks with performance stability. The proposed MathNet outperforms various existing GNN models, especially on big data sets.
IITK at the FinSim Task: Hypernym Detection in Financial Domain via Context-Free and Contextualized Word Embeddings
Authors: Vishal Keswani, Sakshi Singh, Ashutosh Modi
Date: 2020-07-22 00:00:00
Proceeding: FinNLP (COLING) 2020 1
Tasks: Learning Semantic Representations, Word Embeddings

In this paper, we present our approaches for the FinSim 2020 shared task on "Learning Semantic Representations for the Financial Domain". The goal of this task is to classify financial terms into the most relevant hypernym (or top-level) concept in an external ontology. We leverage both context-dependent and context-independent word embeddings in our analysis. Our systems deploy Word2vec embeddings trained from scratch on the corpus (Financial Prospectus in English) along with pre-trained BERT embeddings. We divide the test dataset into two subsets based on a domain rule. For one subset, we use unsupervised distance measures to classify the term. For the second subset, we use simple supervised classifiers like Naive Bayes, on top of the embeddings, to arrive at a final prediction. Finally, we combine both the results. Our system ranks 1st based on both the metrics, i.e., mean rank and accuracy.
Exploratory Search with Sentence Embeddings
Authors: Austin Silveria
Date: 2020-07-22 00:00:00
Tasks: Sentence, Sentence Embeddings

Exploratory search aims to guide users through a corpus rather than pinpointing exact information. We propose an exploratory search system based on hierarchical clusters and document summaries using sentence embeddings. With sentence embeddings, we represent documents as the mean of their embedded sentences, extract summaries containing sentences close to this document representation and extract keyphrases close to the document representation. To evaluate our search system, we scrape our personal search history over the past year and report our experience with the system. We then discuss motivating use cases of an exploratory search system of this nature and conclude with possible directions of future work.
Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networks
Authors: Ping Wang, Khushbu Agarwal, Colby Ham, Sutanay Choudhury, Chandan K. Reddy
Date: 2020-07-22 00:00:00
Tasks: Link Prediction, Representation Learning, Self-Supervised Learning

Representation learning methods for heterogeneous networks produce a low-dimensional vector embedding for each node that is typically fixed for all tasks involving the node. Many of the existing methods focus on obtaining a static vector representation for a node in a way that is agnostic to the downstream application where it is being used. In practice, however, downstream tasks such as link prediction require specific contextual information that can be extracted from the subgraphs related to the nodes provided as input to the task. To tackle this challenge, we develop SLiCE, a framework bridging static representation learning methods using global information from the entire graph with localized attention driven mechanisms to learn contextual node representations. We first pre-train our model in a self-supervised manner by introducing higher-order semantic associations and masking nodes, and then fine-tune our model for a specific link prediction task. Instead of training node representations by aggregating information from all semantic neighbors connected via metapaths, we automatically learn the composition of different metapaths that characterize the context for a specific task without the need for any pre-defined metapaths. SLiCE significantly outperforms both static and contextual embedding learning methods on several publicly available benchmark network datasets. We also interpret the semantic association matrix and provide its utility and relevance in making successful link predictions between heterogeneous nodes in the network.
Predicting Job-Hopping Motive of Candidates Using Answers to Open-ended Interview Questions
Authors: Madhura Jayaratne, Buddhi Jayatilleke
Date: 2020-07-22 00:00:00
Tasks: Word Embeddings

A significant proportion of voluntary employee turnover includes people who frequently move from job to job, known as job-hopping. Our work shows that language used in responding to interview questions on past behaviour and situational judgement is predictive of job-hopping motive as measured by the Job-Hopping Motives (JHM) Scale. The study is based on responses from over 45,000 job applicants who completed an online chat interview and self-rated themselves on JHM Scale. Five different methods of text representation were evaluated, namely four open-vocabulary approaches (TF-IDF, LDA, Glove word embeddings and Doc2Vec document embeddings) and one closed-vocabulary approach (LIWC). The Glove embeddings provided the best results with a correlation of r = 0.35 between sequences of words used and the JHM Scale. Further analysis also showed a correlation of r = 0.25 between language-based job-hopping motive and the personality trait Openness to experience and a correlation of r = -0.09 with the trait Agreeableness.