pwc-archive/papers-with-abstracts huggingface dataset:
Spatial Attention-based Non-reference Perceptual Quality Prediction Network for Omnidirectional Images
Authors: Li Yang, Mai Xu, Deng Xin, Bo Feng
Date: 2021-03-10 00:00:00
Tasks: Image Quality Assessment

Due to the strong correlation between visual attention and perceptual quality, many methods attempt to use human saliency information for image quality assessment. Although this mechanism can get good performance, the networks require human saliency labels, which is not easily accessible for omnidirectional images (ODI). To alleviate this issue, we propose a spatial attention-based perceptual quality prediction network for non-reference quality assessment on ODIs (SAP-net). To drive our SAP-net, we establish a large-scale IQA dataset of ODIs (IQA-ODI), which is composed of subjective scores of 200 subjects on 1,080 ODIs. In IQA-ODI, there are 120 high quality ODIs as reference, and 960 ODIs with impairments in both JPEG compression and map projection. Without any human saliency labels, our network can adaptively estimate human perceptual quality on impaired ODIs through a self-attention manner, which significantly promotes the prediction performance of quality scores. Moreover, our method greatly reduces the computational complexity in quality assessment task on ODIs. Extensive experiments validate that our network outperforms 9 state-of-the-art methods for quality assessment on ODIs. The dataset and code have been available on \url{ https://github.com/yanglixiaoshen/SAP-Net}.
Symmetry meets AI
Authors: Gabriela Barenboim, Johannes Hirn, Veronica Sanz
Date: 2021-03-10 00:00:00

We explore whether Neural Networks (NNs) can {\it discover} the presence of symmetries as they learn to perform a task. For this, we train hundreds of NNs on a {\it decoy task} based on well-controlled Physics templates, where no information on symmetry is provided. We use the output from the last hidden layer of all these NNs, projected to fewer dimensions, as the input for a symmetry classification task, and show that information on symmetry had indeed been identified by the original NN without guidance. As an interdisciplinary application of this procedure, we identify the presence and level of symmetry in artistic paintings from different styles such as those of Picasso, Pollock and Van Gogh.
A critical reappraisal of predicting suicidal ideation using fMRI
Authors: Timothy Verstynen, Konrad Kording
Date: 2021-03-10 00:00:00
Tasks: BIG-bench Machine Learning

For many psychiatric disorders, neuroimaging offers a potential for revolutionizing diagnosis, and potentially treatment, by providing access to preverbal mental processes. In their study "Machine learning of neural representations of suicide and emotion concepts identifies suicidal youth."1, Just and colleagues report that a Naive Bayes classifier, trained on voxelwise fMRI responses in human participants during the presentation of words and concepts related to mortality, can predict whether an individual had reported having suicidal ideations with a classification accuracy of 91%. Here we report a reappraisal of the methods employed by the authors, including re-analysis of the same data set, that calls into question the accuracy of the authors findings. The analysis is a case study in the dangers of overfitting in machine learning.
Session-based Social and Dependency-aware Software Recommendation
Authors: Dengcheng Yan, Tianyi Tang, Wenxin Xie, Yiwen Zhang, Qiang He
Date: 2021-03-10 00:00:00
Tasks: Graph Attention, Recommendation Systems

With the increase of complexity of modern software, social collaborative coding and reuse of open source software packages become more and more popular, which thus greatly enhances the development efficiency and software quality. However, the explosive growth of open source software packages exposes developers to the challenge of information overload. While this can be addressed by conventional recommender systems, they usually do not consider particular constraints of social coding such as social influence among developers and dependency relations among software packages. In this paper, we aim to model the dynamic interests of developers with both social influence and dependency constraints, and propose the Session-based Social and Dependency-aware software Recommendation (SSDRec) model. This model integrates recurrent neural network (RNN) and graph attention network (GAT) into a unified framework. An RNN is employed to model the short-term dynamic interests of developers in each session and two GATs are utilized to capture social influence from friends and dependency constraints from dependent software packages, respectively. Extensive experiments are conducted on real-world datasets and the results demonstrate that our model significantly outperforms the competitive baselines.
Time-Ordered Recent Event (TORE) Volumes for Event Cameras
Authors: R. Wes Baldwin, Ruixu Liu, Mohammed Almatrafi, Vijayan Asari, Keigo Hirakawa
Date: 2021-03-10 00:00:00
Tasks: Blocking, Denoising, Image Reconstruction, Pose Estimation

Event cameras are an exciting, new sensor modality enabling high-speed imaging with extremely low-latency and wide dynamic range. Unfortunately, most machine learning architectures are not designed to directly handle sparse data, like that generated from event cameras. Many state-of-the-art algorithms for event cameras rely on interpolated event representations - obscuring crucial timing information, increasing the data volume, and limiting overall network performance. This paper details an event representation called Time-Ordered Recent Event (TORE) volumes. TORE volumes are designed to compactly store raw spike timing information with minimal information loss. This bio-inspired design is memory efficient, computationally fast, avoids time-blocking (i.e. fixed and predefined frame rates), and contains "local memory" from past data. The design is evaluated on a wide range of challenging tasks (e.g. event denoising, image reconstruction, classification, and human pose estimation) and is shown to dramatically improve state-of-the-art performance. TORE volumes are an easy-to-implement replacement for any algorithm currently utilizing event representations.
Cheapest-to-Deliver Collateral: A Common Factor Approach
Authors: Felix L. Wolf, Lech A. Grzelak, Griselda Deelstra
Date: 2021-03-10 00:00:00

The collateral choice option gives the collateral posting party the opportunity to switch between different collateral currencies which is well-known to impact the asset price. Quantification of the option's value is of practical importance but remains challenging under the assumption of stochastic rates, as it is determined by an intractable distribution which requires involved approximations. Indeed, many practitioners still rely on deterministic spreads between the rates for valuation. We develop a scalable and stable stochastic model of the collateral spreads under the assumption of conditional independence. This allows for a common factor approximation which admits analytical results from which further estimators are obtained. We show that in modelling the spreads between collateral rates, a second order model yields accurate results for the value of the collateral choice option. The model remains precise for a wide range of model parameters and is numerically efficient even for a large number of collateral currencies.
BCFNet: A Balanced Collaborative Filtering Network with Attention Mechanism
Authors: Zi-Yuan Hu, Jin Huang, Zhi-Hong Deng, Chang-Dong Wang, Ling Huang, Jian-Huang Lai, Philip S. Yu
Date: 2021-03-10 00:00:00
Tasks: Collaborative Filtering, Representation Learning

Collaborative Filtering (CF) based recommendation methods have been widely studied, which can be generally categorized into two types, i.e., representation learning-based CF methods and matching function learning-based CF methods. Representation learning tries to learn a common low dimensional space for the representations of users and items. In this case, a user and item match better if they have higher similarity in that common space. Matching function learning tries to directly learn the complex matching function that maps user-item pairs to matching scores. Although both methods are well developed, they suffer from two fundamental flaws, i.e., the representation learning resorts to applying a dot product which has limited expressiveness on the latent features of users and items, while the matching function learning has weakness in capturing low-rank relations. To overcome such flaws, we propose a novel recommendation model named Balanced Collaborative Filtering Network (BCFNet), which has the strengths of the two types of methods. In addition, an attention mechanism is designed to better capture the hidden information within implicit feedback and strengthen the learning ability of the neural network. Furthermore, a balance module is designed to alleviate the over-fitting issue in DNNs. Extensive experiments on eight real-world datasets demonstrate the effectiveness of the proposed model.
U-Net Transformer: Self and Cross Attention for Medical Image Segmentation
Authors: Olivier Petit, Nicolas Thome, Clément Rambour, Luc Soler
Date: 2021-03-10 00:00:00
Tasks: Decoder, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Medical image segmentation remains particularly challenging for complex and low-contrast anatomical structures. In this paper, we introduce the U-Transformer network, which combines a U-shaped architecture for image segmentation with self- and cross-attention from Transformers. U-Transformer overcomes the inability of U-Nets to model long-range contextual interactions and spatial dependencies, which are arguably crucial for accurate segmentation in challenging contexts. To this end, attention mechanisms are incorporated at two main levels: a self-attention module leverages global interactions between encoder features, while cross-attention in the skip connections allows a fine spatial recovery in the U-Net decoder by filtering out non-semantic features. Experiments on two abdominal CT-image datasets show the large performance gain brought out by U-Transformer compared to U-Net and local Attention U-Nets. We also highlight the importance of using both self- and cross-attention, and the nice interpretability features brought out by U-Transformer.
Physics-Guided Neural Networks for Inversion-based Feedforward Control applied to Linear Motors
Authors: Max Bolderman, Mircea Lazar, Hans Butler
Date: 2021-03-10 00:00:00

Ever-increasing throughput specifications in semiconductor manufacturing require operating high-precision mechatronics, such as linear motors, at higher accelerations. In turn this creates higher nonlinear parasitic forces that cannot be handled by industrial feedforward controllers. Motivated by this problem, in this paper we develop a general framework for inversion-based feedforward controller design using physics-guided neural networks (PGNNs). In contrast with black-box neural networks, the developed PGNNs embed prior physical knowledge in the input and hidden layers, which results in improved training convergence and learning of underlying physical laws. The PGNN inversion-based feedforward control framework is validated in simulation on an industrial linear motor, for which it achieves a mean average tracking error twenty times smaller than mass-acceleration feedforward in simulation.
Variable-rate discrete representation learning
Authors: Sander Dieleman, Charlie Nash, Jesse Engel, Karen Simonyan
Date: 2021-03-10 00:00:00
Tasks: Representation Learning

Semantically meaningful information content in perceptual signals is usually unevenly distributed. In speech signals for example, there are often many silences, and the speed of pronunciation can vary considerably. In this work, we propose slow autoencoders (SlowAEs) for unsupervised learning of high-level variable-rate discrete representations of sequences, and apply them to speech. We show that the resulting event-based representations automatically grow or shrink depending on the density of salient information in the input signals, while still allowing for faithful signal reconstruction. We develop run-length Transformers (RLTs) for event-based representation modelling and use them to construct language models in the speech domain, which are able to generate grammatical and semantically coherent utterances and continuations.
On the Dual Implementation of Collision-Avoidance Constraints in Path-Following MPC for Underactuated Surface Vessels
Authors: Simon Helling, Christian Roduner, Thomas Meurer
Date: 2021-03-10 00:00:00
Tasks: Collision Avoidance, Model Predictive Control

A path-following collision-avoidance model predictive control (MPC) method is proposed which approximates obstacle shapes as convex polygons. Collision-avoidance is ensured by means of the signed distance function which is calculated efficiently as part of the MPC problem by making use of a dual formulation. The overall MPC problem can be solved by standard nonlinear programming (NLP) solvers. The dual signed distance formulation yields, besides the (dual) collision-avoidance constraints, norm, and consistency constraints. A novel approach is presented that combines the arising norm equality with the dual collision-avoidance inequality constraints to yield an alternative formulation reduced in complexity. Moving obstacles are considered using separate convex sets of linearly predicted obstacle positions in the dual problem. The theoretical findings and simplifications are compared with the often-used ellipsoidal obstacle formulation and are analyzed with regard to efficiency by the example of a simulated path-following autonomous surface vessel during a realistic maneuver and AIS obstacle data from the Kiel bay area.
Impacts of the Numbers of Colors and Shapes on Outlier Detection: from Automated to User Evaluation
Authors: Loann Giovannangeli, Romain Giot, David Auber, Romain Bourqui
Date: 2021-03-10 00:00:00
Tasks: Attribute, Outlier Detection

The design of efficient representations is well established as a fruitful way to explore and analyze complex or large data. In these representations, data are encoded with various visual attributes depending on the needs of the representation itself. To make coherent design choices about visual attributes, the visual search field proposes guidelines based on the human brain perception of features. However, information visualization representations frequently need to depict more data than the amount these guidelines have been validated on. Since, the information visualization community has extended these guidelines to a wider parameter space. This paper contributes to this theme by extending visual search theories to an information visualization context. We consider a visual search task where subjects are asked to find an unknown outlier in a grid of randomly laid out distractor. Stimuli are defined by color and shape features for the purpose of visually encoding categorical data. The experimental protocol is made of a parameters space reduction step (i.e., sub-sampling) based on a machine learning model, and a user evaluation to measure capacity limits and validate hypotheses. The results show that the major difficulty factor is the number of visual attributes that are used to encode the outlier. When redundantly encoded, the display heterogeneity has no effect on the task. When encoded with one attribute, the difficulty depends on that attribute heterogeneity until its capacity limit (7 for color, 5 for shape) is reached. Finally, when encoded with two attributes simultaneously, performances drop drastically even with minor heterogeneity.
Knowledge-based Extraction of Cause-Effect Relations from Biomedical Text
Authors: Sachin Pawar, Ravina More, Girish K. Palshikar, Pushpak Bhattacharyya, Vasudeva Varma
Date: 2021-03-10 00:00:00
Tasks: Triplet

We propose a knowledge-based approach for extraction of Cause-Effect (CE) relations from biomedical text. Our approach is a combination of an unsupervised machine learning technique to discover causal triggers and a set of high-precision linguistic rules to identify cause/effect arguments of these causal triggers. We evaluate our approach using a corpus of 58,761 Leukaemia-related PubMed abstracts consisting of 568,528 sentences. We could extract 152,655 CE triplets from this corpus where each triplet consists of a cause phrase, an effect phrase and a causal trigger. As compared to the existing knowledge base - SemMedDB (Kilicoglu et al., 2012), the number of extractions are almost twice. Moreover, the proposed approach outperformed the existing technique SemRep (Rindflesch and Fiszman, 2003) on a dataset of 500 sentences.
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Authors: Solon Barocas, Anhong Guo, Ece Kamar, Jacquelyn Krones, Meredith Ringel Morris, Jennifer Wortman Vaughan, Duncan Wadsworth, Hanna Wallach
Date: 2021-03-10 00:00:00

Disaggregated evaluations of AI systems, in which system performance is assessed and reported separately for different groups of people, are conceptually simple. However, their design involves a variety of choices. Some of these choices influence the results that will be obtained, and thus the conclusions that can be drawn; others influence the impacts -- both beneficial and harmful -- that a disaggregated evaluation will have on people, including the people whose data is used to conduct the evaluation. We argue that a deeper understanding of these choices will enable researchers and practitioners to design careful and conclusive disaggregated evaluations. We also argue that better documentation of these choices, along with the underlying considerations and tradeoffs that have been made, will help others when interpreting an evaluation's results and conclusions.
Extension of the Lagrange multiplier test for error cross-section independence to large panels with non normal errors
Authors: Zhaoyuan Li, Jianfeng Yao
Date: 2021-03-10 00:00:00
Tasks: Time Series, Time Series Analysis, valid

This paper reexamines the seminal Lagrange multiplier test for cross-section independence in a large panel model where both the number of cross-sectional units n and the number of time series observations T can be large. The first contribution of the paper is an enlargement of the test with two extensions: firstly the new asymptotic normality is derived in a simultaneous limiting scheme where the two dimensions (n, T) tend to infinity with comparable magnitudes; second, the result is valid for general error distribution (not necessarily normal). The second contribution of the paper is a new test statistic based on the sum of the fourth powers of cross-section correlations from OLS residuals, instead of their squares used in the Lagrange multiplier statistic. This new test is generally more powerful, and the improvement is particularly visible against alternatives with weak or sparse cross-section dependence. Both simulation study and real data analysis are proposed to demonstrate the advantages of the enlarged Lagrange multiplier test and the power enhanced test in comparison with the existing procedures.
Grid-Graph Signal Processing (Grid-GSP): A Graph Signal Processing Framework for the Power Grid
Authors: Raksha Ramakrishna, Anna Scaglione
Date: 2021-03-10 00:00:00
Tasks: Data Compression

The underlying theme of this paper is to explore the various facets of power systems data through the lens of graph signal processing (GSP), laying down the foundations of the Grid-GSP framework. Grid-GSP provides an interpretation for the spatio-temporal properties of voltage phasor measurements, by showing how the well-known power systems modeling supports a generative low-pass graph filter model for the state variables, namely the voltage phasors. Using the model we formalize the empirical observation that voltage phasor measurement data lie in a low-dimensional subspace and tie their spatio-temporal structure to generator voltage dynamics. The Grid-GSP generative model is then successfully employed to investigate the problems pertaining to the grid of data sampling and interpolation, network inference, detection of anomalies and data compression. Numerical results on a large synthetic grid that mimics the real-grid of the state of Texas, ACTIVSg2000, and on real-world measurements from ISO-New England verify the efficacy of applying Grid-GSP methods to electric grid data.
Graph Neural Networks Inspired by Classical Iterative Algorithms
Authors: Yongyi Yang, Tang Liu, Yangkun Wang, Jinjing Zhou, Quan Gan, Zhewei Wei, Zheng Zhang, Zengfeng Huang, David Wipf
Date: 2021-03-10 00:00:00
Tasks: Node Classification

Despite the recent success of graph neural networks (GNN), common architectures often exhibit significant limitations, including sensitivity to oversmoothing, long-range dependencies, and spurious edges, e.g., as can occur as a result of graph heterophily or adversarial attacks. To at least partially address these issues within a simple transparent framework, we consider a new family of GNN layers designed to mimic and integrate the update rules of two classical iterative algorithms, namely, proximal gradient descent and iterative reweighted least squares (IRLS). The former defines an extensible base GNN architecture that is immune to oversmoothing while nonetheless capturing long-range dependencies by allowing arbitrary propagation steps. In contrast, the latter produces a novel attention mechanism that is explicitly anchored to an underlying end-to-end energy function, contributing stability with respect to edge uncertainty. When combined we obtain an extremely simple yet robust model that we evaluate across disparate scenarios including standardized benchmarks, adversarially-perturbated graphs, graphs with heterophily, and graphs involving long-range dependencies. In doing so, we compare against SOTA GNN approaches that have been explicitly designed for the respective task, achieving competitive or superior node classification accuracy. Our code is available at https://github.com/FFTYYY/TWIRLS.
Endurance-Aware Mapping of Spiking Neural Networks to Neuromorphic Hardware
Authors: Twisha Titirsha, Shihao Song, Anup Das, Jeffrey Krichmar, Nikil Dutt, Nagarajan Kandasamy, Francky Catthoor
Date: 2021-03-09 00:00:00
Tasks: graph partitioning

Neuromorphic computing systems are embracing memristors to implement high density and low power synaptic storage as crossbar arrays in hardware. These systems are energy efficient in executing Spiking Neural Networks (SNNs). We observe that long bitlines and wordlines in a memristive crossbar are a major source of parasitic voltage drops, which create current asymmetry. Through circuit simulations, we show the significant endurance variation that results from this asymmetry. Therefore, if the critical memristors (ones with lower endurance) are overutilized, they may lead to a reduction of the crossbar's lifetime. We propose eSpine, a novel technique to improve lifetime by incorporating the endurance variation within each crossbar in mapping machine learning workloads, ensuring that synapses with higher activation are always implemented on memristors with higher endurance, and vice versa. eSpine works in two steps. First, it uses the Kernighan-Lin Graph Partitioning algorithm to partition a workload into clusters of neurons and synapses, where each cluster can fit in a crossbar. Second, it uses an instance of Particle Swarm Optimization (PSO) to map clusters to tiles, where the placement of synapses of a cluster to memristors of a crossbar is performed by analyzing their activation within the workload. We evaluate eSpine for a state-of-the-art neuromorphic hardware model with phase-change memory (PCM)-based memristors. Using 10 SNN workloads, we demonstrate a significant improvement in the effective lifetime.