pwc-archive/papers-with-abstracts huggingface dataset:
Language-Agnostic Representation Learning of Source Code from Structure and Context
Authors: Daniel Zügner, Tobias Kirschstein, Michele Catasta, Jure Leskovec, Stephan Günnemann
Date: 2021-03-21 00:00:00
Proceeding: ICLR 2021 1
Tasks: Code Summarization, Representation Learning, Source Code Summarization

Source code (Context) and its parsed abstract syntax tree (AST; Structure) are two complementary representations of the same computer program. Traditionally, designers of machine learning models have relied predominantly either on Structure or Context. We propose a new model, which jointly learns on Context and Structure of source code. In contrast to previous approaches, our model uses only language-agnostic features, i.e., source code and features that can be computed directly from the AST. Besides obtaining state-of-the-art on monolingual code summarization on all five programming languages considered in this work, we propose the first multilingual code summarization model. We show that jointly training on non-parallel data from multiple programming languages improves results on all individual languages, where the strongest gains are on low-resource languages. Remarkably, multilingual training only from Context does not lead to the same improvements, highlighting the benefits of combining Structure and Context for representation learning on code.
A Learned Compact and Editable Light Field Representation
Authors: Menghan Xia, Jose Echevarria, Minshan Xie, Tien-Tsin Wong
Date: 2021-03-21 00:00:00
Tasks: Decoder

Light fields are 4D scene representation typically structured as arrays of views, or several directional samples per pixel in a single view. This highly correlated structure is not very efficient to transmit and manipulate (especially for editing), though. To tackle these problems, we present a novel compact and editable light field representation, consisting of a set of visual channels (i.e. the central RGB view) and a complementary meta channel that encodes the residual geometric and appearance information. The visual channels in this representation can be edited using existing 2D image editing tools, before accurately reconstructing the whole edited light field back. We propose to learn this representation via an autoencoder framework, consisting of an encoder for learning the representation, and a decoder for reconstructing the light field. To handle the challenging occlusions and propagation of edits, we specifically designed an editing-aware decoding network and its associated training strategy, so that the edits to the visual channels can be consistently propagated to the whole light field upon reconstruction.Experimental results show that our proposed method outperforms related existing methods in reconstruction accuracy, and achieves visually pleasant performance in editing propagation.
PGT: A Progressive Method for Training Models on Long Videos
Authors: Bo Pang, Gao Peng, Yizhuo Li, Cewu Lu
Date: 2021-03-21 00:00:00
Proceeding: CVPR 2021 1

Convolutional video models have an order of magnitude larger computational complexity than their counterpart image-level models. Constrained by computational resources, there is no model or training method that can train long video sequences end-to-end. Currently, the main-stream method is to split a raw video into clips, leading to incomplete fragmentary temporal information flow. Inspired by natural language processing techniques dealing with long sentences, we propose to treat videos as serial fragments satisfying Markov property, and train it as a whole by progressively propagating information through the temporal dimension in multiple steps. This progressive training (PGT) method is able to train long videos end-to-end with limited resources and ensures the effective transmission of information. As a general and robust training method, we empirically demonstrate that it yields significant performance improvements on different models and datasets. As an illustrative example, the proposed method improves SlowOnly network by 3.7 mAP on Charades and 1.9 top-1 accuracy on Kinetics with negligible parameter and computation overhead. Code is available at https://github.com/BoPang1996/PGT.
Common Sense Knowledge, Ontology and Text Mining for Implicit Requirements
Authors: Onyeka Emebo, Aparna S. Varde, Olawande Daramola
Date: 2021-03-21 00:00:00
Tasks: Common Sense Reasoning, Management

The ability of a system to meet its requirements is a strong determinant of success. Thus effective requirements specification is crucial. Explicit Requirements are well-defined needs for a system to execute. IMplicit Requirements (IMRs) are assumed needs that a system is expected to fulfill though not elicited during requirements gathering. Studies have shown that a major factor in the failure of software systems is the presence of unhandled IMRs. Since relevance of IMRs is important for efficient system functionality, there are methods developed to aid the identification and management of IMRs. In this paper, we emphasize that Common Sense Knowledge, in the field of Knowledge Representation in AI, would be useful to automatically identify and manage IMRs. This paper is aimed at identifying the sources of IMRs and also proposing an automated support tool for managing IMRs within an organizational context. Since this is found to be a present gap in practice, our work makes a contribution here. We propose a novel approach for identifying and managing IMRs based on combining three core technologies: common sense knowledge, text mining and ontology. We claim that discovery and handling of unknown and non-elicited requirements would reduce risks and costs in software development.
A Modular and Unified Framework for Detecting and Localizing Video Anomalies
Authors: Keval Doshi, Yasin Yilmaz
Date: 2021-03-21 00:00:00
Tasks: Anomaly Detection, Event Detection, Online Video Anomaly Detection, Transfer Learning, Video Anomaly Detection

Anomaly detection in videos has been attracting an increasing amount of attention. Despite the competitive performance of recent methods on benchmark datasets, they typically lack desirable features such as modularity, cross-domain adaptivity, interpretability, and real-time anomalous event detection. Furthermore, current state-of-the-art approaches are evaluated using the standard instance-based detection metric by considering video frames as independent instances, which is not ideal for video anomaly detection. Motivated by these research gaps, we propose a modular and unified approach to the online video anomaly detection and localization problem, called MOVAD, which consists of a novel transfer learning based plug-and-play architecture, a sequential anomaly detector, a mathematical framework for selecting the detection threshold, and a suitable performance metric for real-time anomalous event detection in videos. Extensive performance evaluations on benchmark datasets show that the proposed framework significantly outperforms the current state-of-the-art approaches.
Deep Dense Multi-scale Network for Snow Removal Using Semantic and Geometric Priors
Authors: Kaihao Zhang, Rongqing Li, Yanjiang Yu, Wenhan Luo, Changsheng Li, Hongdong Li
Date: 2021-03-21 00:00:00
Tasks: Image Restoration, Snow Removal

Images captured in snowy days suffer from noticeable degradation of scene visibility, which degenerates the performance of current vision-based intelligent systems. Removing snow from images thus is an important topic in computer vision. In this paper, we propose a Deep Dense Multi-Scale Network (\textbf{DDMSNet}) for snow removal by exploiting semantic and geometric priors. As images captured in outdoor often share similar scenes and their visibility varies with depth from camera, such semantic and geometric information provides a strong prior for snowy image restoration. We incorporate the semantic and geometric maps as input and learn the semantic-aware and geometry-aware representation to remove snow. In particular, we first create a coarse network to remove snow from the input images. Then, the coarsely desnowed images are fed into another network to obtain the semantic and geometric labels. Finally, we design a DDMSNet to learn semantic-aware and geometry-aware representation via a self-attention mechanism to produce the final clean images. Experiments evaluated on public synthetic and real-world snowy images verify the superiority of the proposed method, offering better results both quantitatively and qualitatively.
Insight-centric Visualization Recommendation
Authors: Camille Harris, Ryan A. Rossi, Sana Malik, Jane Hoffswell, Fan Du, Tak Yeon Lee, Eunyee Koh, Handong Zhao
Date: 2021-03-21 00:00:00
Tasks: Attribute, Recommendation Systems

Visualization recommendation systems simplify exploratory data analysis (EDA) and make understanding data more accessible to users of all skill levels by automatically generating visualizations for users to explore. However, most existing visualization recommendation systems focus on ranking all visualizations into a single list or set of groups based on particular attributes or encodings. This global ranking makes it difficult and time-consuming for users to find the most interesting or relevant insights. To address these limitations, we introduce a novel class of visualization recommendation systems that automatically rank and recommend both groups of related insights as well as the most important insights within each group. Our proposed approach combines results from many different learning-based methods to discover insights automatically. A key advantage is that this approach generalizes to a wide variety of attribute types such as categorical, numerical, and temporal, as well as complex non-trivial combinations of these different attribute types. To evaluate the effectiveness of our approach, we implemented a new insight-centric visualization recommendation system, SpotLight, which generates and ranks annotated visualizations to explain each insight. We conducted a user study with 12 participants and two datasets which showed that users are able to quickly understand and find relevant insights in unfamiliar data.
ActivationNet: Representation learning to predict contact quality of interacting 3-D surfaces in engineering designs
Authors: Rishikesh Ranade, Jay Pathak
Date: 2021-03-21 00:00:00
Tasks: BIG-bench Machine Learning, Representation Learning

Engineering simulations for analysis of structural and fluid systems require information of contacts between various 3-D surfaces of the geometry to accurately model the physics between them. In machine learning applications, 3-D surfaces are most suitably represented with point clouds or meshes and learning representations of interacting geometries form point-based representations is challenging. The objective of this work is to introduce a machine learning algorithm, ActivationNet, that can learn from point clouds or meshes of interacting 3-D surfaces and predict the quality of contact between these surfaces. The ActivationNet generates activation states from point-based representation of surfaces using a multi-dimensional binning approach. The activation states are further used to contact quality between surfaces using deep neural networks. The performance of our model is demonstrated using several experiments, including tests on interacting surfaces extracted from engineering geometries. In all the experiments presented in this paper, the contact quality predictions of ActivationNet agree well with the expectations.
Geo-Spatiotemporal Features and Shape-Based Prior Knowledge for Fine-grained Imbalanced Data Classification
Authors: Charles A. Kantor, Marta Skreta, Brice Rauby, Léonard Boussioux, Emmanuel Jehanno, Alexandra Luccioni, David Rolnick, Hugues Talbot
Date: 2021-03-21 00:00:00
Tasks: Classification, General Classification

Fine-grained classification aims at distinguishing between items with similar global perception and patterns, but that differ by minute details. Our primary challenges come from both small inter-class variations and large intra-class variations. In this article, we propose to combine several innovations to improve fine-grained classification within the use-case of wildlife, which is of practical interest for experts. We utilize geo-spatiotemporal data to enrich the picture information and further improve the performance. We also investigate state-of-the-art methods for handling the imbalanced data issue.
Learning Optimal Fronthauling and Decentralized Edge Computation in Fog Radio Access Networks
Authors: Hoon Lee, Junbeom Kim, Seok-Hwan Park
Date: 2021-03-21 00:00:00
Tasks: Edge-computing

Fog radio access networks (F-RANs), which consist of a cloud and multiple edge nodes (ENs) connected via fronthaul links, have been regarded as promising network architectures. The F-RAN entails a joint optimization of cloud and edge computing as well as fronthaul interactions, which is challenging for traditional optimization techniques. This paper proposes a Cloud-Enabled Cooperation-Inspired Learning (CECIL) framework, a structural deep learning mechanism for handling a generic F-RAN optimization problem. The proposed solution mimics cloud-aided cooperative optimization policies by including centralized computing at the cloud, distributed decision at the ENs, and their uplink-downlink fronthaul interactions. A group of deep neural networks (DNNs) are employed for characterizing computations of the cloud and ENs. The forwardpass of the DNNs is carefully designed such that the impacts of the practical fronthaul links, such as channel noise and signling overheads, can be included in a training step. As a result, operations of the cloud and ENs can be jointly trained in an end-to-end manner, whereas their real-time inferences are carried out in a decentralized manner by means of the fronthaul coordination. To facilitate fronthaul cooperation among multiple ENs, the optimal fronthaul multiple access schemes are designed. Training algorithms robust to practical fronthaul impairments are also presented. Numerical results validate the effectiveness of the proposed approaches.
High precision control and deep learning-based corn stand counting algorithms for agricultural robot
Authors: Zhongzhong Zhang, Erkan Kayacan, Benjamin Thompson, Girish Chowdhary
Date: 2021-03-21 00:00:00
Tasks: Model Predictive Control, Visual Tracking

This paper presents high precision control and deep learning-based corn stand counting algorithms for a low-cost, ultra-compact 3D printed and autonomous field robot for agricultural operations. Currently, plant traits, such as emergence rate, biomass, vigor, and stand counting, are measured manually. This is highly labor-intensive and prone to errors. The robot, termed TerraSentia, is designed to automate the measurement of plant traits for efficient phenotyping as an alternative to manual measurements. In this paper, we formulate a Nonlinear Moving Horizon Estimator (NMHE) that identifies key terrain parameters using onboard robot sensors and a learning-based Nonlinear Model Predictive Control (NMPC) that ensures high precision path tracking in the presence of unknown wheel-terrain interaction. Moreover, we develop a machine vision algorithm designed to enable an ultra-compact ground robot to count corn stands by driving through the fields autonomously. The algorithm leverages a deep network to detect corn plants in images, and a visual tracking model to re-identify detected objects at different time steps. We collected data from 53 corn plots in various fields for corn plants around 14 days after emergence (stage V3 - V4). The robot predictions have agreed well with the ground truth with $C_{robot}=1.02 \times C_{human}-0.86$ and a correlation coefficient $R=0.96$. The mean relative error given by the algorithm is $-3.78\%$, and the standard deviation is $6.76\%$. These results indicate a first and significant step towards autonomous robot-based real-time phenotyping using low-cost, ultra-compact ground robots for corn and potentially other crops.
Self-supervised Representation Learning with Relative Predictive Coding
Authors: Yao-Hung Hubert Tsai, Martin Q. Ma, Muqiao Yang, Han Zhao, Louis-Philippe Morency, Ruslan Salakhutdinov
Date: 2021-03-21 00:00:00
Proceeding: ICLR 2021 1
Tasks: Representation Learning, Self-Supervised Learning, Sensitivity

This paper introduces Relative Predictive Coding (RPC), a new contrastive representation learning objective that maintains a good balance among training stability, minibatch size sensitivity, and downstream task performance. The key to the success of RPC is two-fold. First, RPC introduces the relative parameters to regularize the objective for boundedness and low variance. Second, RPC contains no logarithm and exponential score functions, which are the main cause of training instability in prior contrastive objectives. We empirically verify the effectiveness of RPC on benchmark vision and speech self-supervised learning tasks. Lastly, we relate RPC with mutual information (MI) estimation, showing RPC can be used to estimate MI with low variance.
Sliding Mode Learning Control of Uncertain Nonlinear Systems with Lyapunov Stability Analysis
Authors: Erkan Kayacan
Date: 2021-03-21 00:00:00

This paper addresses to Sliding Mode Learning Control (SMLC) of uncertain nonlinear systems with Lyapunov stability analysis. In the control scheme, a conventional control term is used to provide the system stability in compact space while a Type-2 Neuro-Fuzzy Controller (T2NFC) learns system behavior so that the T2NFC takes the overall control of the system completely in a very short time period. The stability of the sliding mode learning algorithm was proven in literature; however, it is so restrictive for systems without the overall system stability. To address this shortcoming, a novel control structure with a novel sliding surface is proposed in this paper and the stability of the overall system is proven for nth-order uncertain nonlinear systems. To investigate the capability and effectiveness of the proposed learning and control algorithms, the simulation studies have been achieved under noisy conditions. The simulation results confirm that the developed SMLC algorithm can learn the system behavior in the absence of any mathematical model knowledge and exhibit robust control performance against external disturbances.
Structural Textile Pattern Recognition and Processing Based on Hypergraphs
Authors: Vuong M. Ngo, Sven Helmer, Nhien-An Le-Khac, M-Tahar Kechadi
Date: 2021-03-21 00:00:00
Tasks: Clustering, Retrieval

The humanities, like many other areas of society, are currently undergoing major changes in the wake of digital transformation. However, in order to make collection of digitised material in this area easily accessible, we often still lack adequate search functionality. For instance, digital archives for textiles offer keyword search, which is fairly well understood, and arrange their content following a certain taxonomy, but search functionality at the level of thread structure is still missing. To facilitate the clustering and search, we introduce an approach for recognising similar weaving patterns based on their structures for textile archives. We first represent textile structures using hypergraphs and extract multisets of k-neighbourhoods describing weaving patterns from these graphs. Then, the resulting multisets are clustered using various distance measures and various clustering algorithms (K-Means for simplicity and hierarchical agglomerative algorithms for precision). We evaluate the different variants of our approach experimentally, showing that this can be implemented efficiently (meaning it has linear complexity), and demonstrate its quality to query and cluster datasets containing large textile samples. As, to the est of our knowledge, this is the first practical approach for explicitly modelling complex and irregular weaving patterns usable for retrieval, we aim at establishing a solid baseline.
Development and Validation of a Deep Learning Model for Prediction of Severe Outcomes in Suspected COVID-19 Infection
Authors: Varun Buch, Aoxiao Zhong, Xiang Li, Marcio Aloisio Bezerra Cavalcanti Rockenbach, Dufan Wu, Hui Ren, Jiahui Guan, Andrew Liteplo, Sayon Dutta, Ittai Dayan, Quanzheng Li
Date: 2021-03-21 00:00:00
Tasks: Management, Prognosis

COVID-19 patient triaging with predictive outcome of the patients upon first present to emergency department (ED) is crucial for improving patient prognosis, as well as better hospital resources management and cross-infection control. We trained a deep feature fusion model to predict patient outcomes, where the model inputs were EHR data including demographic information, co-morbidities, vital signs and laboratory measurements, plus patient's CXR images. The model output was patient outcomes defined as the most insensitive oxygen therapy required. For patients without CXR images, we employed Random Forest method for the prediction. Predictive risk scores for COVID-19 severe outcomes ("CO-RISK" score) were derived from model output and evaluated on the testing dataset, as well as compared to human performance. The study's dataset (the "MGB COVID Cohort") was constructed from all patients presenting to the Mass General Brigham (MGB) healthcare system from March 1st to June 1st, 2020. ED visits with incomplete or erroneous data were excluded. Patients with no test order for COVID or confirmed negative test results were excluded. Patients under the age of 15 were also excluded. Finally, electronic health record (EHR) data from a total of 11060 COVID-19 confirmed or suspected patients were used in this study. Chest X-ray (CXR) images were also collected from each patient if available. Results show that CO-RISK score achieved area under the Curve (AUC) of predicting MV/death (i.e. severe outcomes) in 24 hours of 0.95, and 0.92 in 72 hours on the testing dataset. The model shows superior performance to the commonly used risk scores in ED (CURB-65 and MEWS). Comparing with physician's decisions, CO-RISK score has demonstrated superior performance to human in making ICU/floor decisions.
Bayesian Distributional Policy Gradients
Authors: Luchen Li, A. Aldo Faisal
Date: 2021-03-20 00:00:00
Tasks: Atari Games, Contrastive Learning, Distributional Reinforcement Learning, MuJoCo, Reinforcement Learning (RL)

Distributional Reinforcement Learning (RL) maintains the entire probability distribution of the reward-to-go, i.e. the return, providing more learning signals that account for the uncertainty associated with policy performance, which may be beneficial for trading off exploration and exploitation and policy learning in general. Previous works in distributional RL focused mainly on computing the state-action-return distributions, here we model the state-return distributions. This enables us to translate successful conventional RL algorithms that are based on state values into distributional RL. We formulate the distributional Bellman operation as an inference-based auto-encoding process that minimises Wasserstein metrics between target/model return distributions. The proposed algorithm, BDPG (Bayesian Distributional Policy Gradients), uses adversarial training in joint-contrastive learning to estimate a variational posterior from the returns. Moreover, we can now interpret the return prediction uncertainty as an information gain, which allows to obtain a new curiosity measure that helps BDPG steer exploration actively and efficiently. We demonstrate in a suite of Atari 2600 games and MuJoCo tasks, including well known hard-exploration challenges, how BDPG learns generally faster and with higher asymptotic performance than reference distributional RL algorithms.