pwc-archive/papers-with-abstracts huggingface dataset:
Generalisation in Neural Networks Does not Require Feature Overlap
Authors: Jeff Mitchell, Jeffrey S. Bowers
Date: 2021-07-04 00:00:00

That shared features between train and test data are required for generalisation in artificial neural networks has been a common assumption of both proponents and critics of these models. Here, we show that convolutional architectures avoid this limitation by applying them to two well known challenges, based on learning the identity function and learning rules governing sequences of words. In each case, successful performance on the test set requires generalising to features that were not present in the training data, which is typically not feasible for standard connectionist models. However, our experiments demonstrate that neural networks can succeed on such problems when they incorporate the weight sharing employed by convolutional architectures. In the image processing domain, such architectures are intended to reflect the symmetry under spatial translations of the natural world that such images depict. We discuss the role of symmetry in the two tasks and its connection to generalisation.
Exploiting Spiking Dynamics with Spatial-temporal Feature Normalization in Graph Learning
Authors: Mingkun Xu, Yujie Wu, Lei Deng, Faqiang Liu, Guoqi Li, Jing Pei
Date: 2021-06-30 00:00:00
Tasks: Graph Attention, Graph Learning, Graph Neural Network, Node Classification

Biological spiking neurons with intrinsic dynamics underlie the powerful representation and learning capabilities of the brain for processing multimodal information in complex environments. Despite recent tremendous progress in spiking neural networks (SNNs) for handling Euclidean-space tasks, it still remains challenging to exploit SNNs in processing non-Euclidean-space data represented by graph data, mainly due to the lack of effective modeling framework and useful training techniques. Here we present a general spike-based modeling framework that enables the direct training of SNNs for graph learning. Through spatial-temporal unfolding for spiking data flows of node features, we incorporate graph convolution filters into spiking dynamics and formalize a synergistic learning paradigm. Considering the unique features of spike representation and spiking dynamics, we propose a spatial-temporal feature normalization (STFN) technique suitable for SNN to accelerate convergence. We instantiate our methods into two spiking graph models, including graph convolution SNNs and graph attention SNNs, and validate their performance on three node-classification benchmarks, including Cora, Citeseer, and Pubmed. Our model can achieve comparable performance with the state-of-the-art graph neural network (GNN) models with much lower computation costs, demonstrating great benefits for the execution on neuromorphic hardware and prompting neuromorphic applications in graphical scenarios.
Differentiable Programming of Reaction-Diffusion Patterns
Authors: Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson
Date: 2021-06-22 00:00:00
Tasks: Texture Synthesis

Reaction-Diffusion (RD) systems provide a computational framework that governs many pattern formation processes in nature. Current RD system design practices boil down to trial-and-error parameter search. We propose a differentiable optimization method for learning the RD system parameters to perform example-based texture synthesis on a 2D plane. We do this by representing the RD system as a variant of Neural Cellular Automata and using task-specific differentiable loss functions. RD systems generated by our method exhibit robust, non-trivial 'life-like' behavior.
Backpropagated Neighborhood Aggregation for Accurate Training of Spiking Neural Networks
Authors: Yukun Yang, Wenrui Zhang, Peng Li
Date: 2021-06-22 00:00:00

While backpropagation (BP) has been applied to spiking neural networks (SNNs) achieving encouraging results, a key challenge involved is to backpropagate a continuous-valued loss over layers of spiking neurons exhibiting discontinuous all-or-none firing activities. Existing methods deal with this difficulty by introducing compromises that come with their own limitations, leading to potential performance degradation. We propose a novel BP-like method, called neighborhood aggregation (NA), which computes accurate error gradients guiding weight updates that may lead to discontinuous modifications of firing activities. NA achieves this goal by aggregating finite differences of the loss over multiple perturbed membrane potential waveforms in the neighborhood of the present membrane potential of each neuron while utilizing a new membrane potential distance function. Our experiments show that the proposed NA algorithm delivers the state-of-the-art performance for SNN training on several datasets.
Sequential Recommendation for Cold-start Users with Meta Transitional Learning
Authors: Jianling Wang, Kaize Ding, James Caverlee
Date: 2021-07-13 00:00:00
Tasks: Few-Shot Learning, Meta-Learning, Sequential Recommendation, Translation

A fundamental challenge for sequential recommenders is to capture the sequential patterns of users toward modeling how users transit among items. In many practical scenarios, however, there are a great number of cold-start users with only minimal logged interactions. As a result, existing sequential recommendation models will lose their predictive power due to the difficulties in learning sequential patterns over users with only limited interactions. In this work, we aim to improve sequential recommendation for cold-start users with a novel framework named MetaTL, which learns to model the transition patterns of users through meta-learning. Specifically, the proposed MetaTL: (i) formulates sequential recommendation for cold-start users as a few-shot learning problem; (ii) extracts the dynamic transition patterns among users with a translation-based architecture; and (iii) adopts meta transitional learning to enable fast learning for cold-start users with only limited interactions, leading to accurate inference of sequential interactions.
Learning to Recommend Items to Wikidata Editors
Authors: Kholoud Alghamdi, Miaojing Shi, Elena Simperl
Date: 2021-07-13 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Wikidata is an open knowledge graph built by a global community of volunteers. As it advances in scale, it faces substantial challenges around editor engagement. These challenges are in terms of both attracting new editors to keep up with the sheer amount of work and retaining existing editors. Experience from other online communities and peer-production systems, including Wikipedia, suggests that personalised recommendations could help, especially newcomers, who are sometimes unsure about how to contribute best to an ongoing effort. For this reason, we propose a recommender system WikidataRec for Wikidata items. The system uses a hybrid of content-based and collaborative filtering techniques to rank items for editors relying on both item features and item-editor previous interaction. A neural network, named a neural mixture of representations, is designed to learn fine weights for the combination of item-based representations and optimize them with editor-based representation by item-editor interaction. To facilitate further research in this space, we also create two benchmark datasets, a general-purpose one with 220,000 editors responsible for 14 million interactions with 4 million items and a second one focusing on the contributions of more than 8,000 more active editors. We perform an offline evaluation of the system on both datasets with promising results. Our code and datasets are available at https://github.com/WikidataRec-developer/Wikidata_Recommender.
Multi-Step Critiquing User Interface for Recommender Systems
Authors: Diana Petrescu, Diego Antognini, Boi Faltings
Date: 2021-07-13 00:00:00
Tasks: Recommendation Systems

Recommendations with personalized explanations have been shown to increase user trust and perceived quality and help users make better decisions. Moreover, such explanations allow users to provide feedback by critiquing them. Several algorithms for recommender systems with multi-step critiquing have therefore been developed. However, providing a user-friendly interface based on personalized explanations and critiquing has not been addressed in the last decade. In this paper, we introduce four different web interfaces (available under https://lia.epfl.ch/critiquing/) helping users making decisions and finding their ideal item. We have chosen the hotel recommendation domain as a use case even though our approach is trivially adaptable for other domains. Moreover, our system is model-agnostic (for both recommender systems and critiquing models) allowing a great flexibility and further extensions. Our interfaces are above all a useful tool to help research in recommendation with critiquing. They allow to test such systems on a real use case and also to highlight some limitations of these approaches to find solutions to overcome them.
A Review on Edge Analytics: Issues, Challenges, Opportunities, Promises, Future Directions, and Applications
Authors: Sabuzima Nayak, Ripon Patgiri, Lilapati Waikhom, Arif Ahmed
Date: 2021-07-01 00:00:00
Tasks: Edge-computing

Edge technology aims to bring Cloud resources (specifically, the compute, storage, and network) to the closed proximity of the Edge devices, i.e., smart devices where the data are produced and consumed. Embedding computing and application in Edge devices lead to emerging of two new concepts in Edge technology, namely, Edge computing and Edge analytics. Edge analytics uses some techniques or algorithms to analyze the data generated by the Edge devices. With the emerging of Edge analytics, the Edge devices have become a complete set. Currently, Edge analytics is unable to provide full support for the execution of the analytic techniques. The Edge devices cannot execute advanced and sophisticated analytic algorithms following various constraints such as limited power supply, small memory size, limited resources, etc. This article aims to provide a detailed discussion on Edge analytics. A clear explanation to distinguish between the three concepts of Edge technology, namely, Edge devices, Edge computing, and Edge analytics, along with their issues. Furthermore, the article discusses the implementation of Edge analytics to solve many problems in various areas such as retail, agriculture, industry, and healthcare. In addition, the research papers of the state-of-the-art edge analytics are rigorously reviewed in this article to explore the existing issues, emerging challenges, research opportunities and their directions, and applications.
Uncertainty Modeling of Emerging Device-based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Search
Authors: Zheyu Yan, Da-Cheng Juan, Xiaobo Sharon Hu, Yiyu Shi
Date: 2021-07-06 00:00:00
Tasks: Neural Architecture Search

Emerging device-based Computing-in-memory (CiM) has been proved to be a promising candidate for high-energy efficiency deep neural network (DNN) computations. However, most emerging devices suffer uncertainty issues, resulting in a difference between actual data stored and the weight value it is designed to be. This leads to an accuracy drop from trained models to actually deployed platforms. In this work, we offer a thorough analysis of the effect of such uncertainties-induced changes in DNN models. To reduce the impact of device uncertainties, we propose UAE, an uncertainty-aware Neural Architecture Search scheme to identify a DNN model that is both accurate and robust against device uncertainties.
Forecasting Thermoacoustic Instabilities in Liquid Propellant Rocket Engines Using Multimodal Bayesian Deep Learning
Authors: Ushnish Sengupta, Günther Waxenegger-Wilfing, Jan Martin, Justin Hardi, Matthew P. Juniper
Date: 2021-07-01 00:00:00
Tasks: Time Series Analysis

The 100 MW cryogenic liquid oxygen/hydrogen multi-injector combustor BKD operated by the DLR Institute of Space Propulsion is a research platform that allows the study of thermoacoustic instabilities under realistic conditions, representative of small upper stage rocket engines. We use data from BKD experimental campaigns in which the static chamber pressure and fuel-oxidizer ratio are varied such that the first tangential mode of the combustor is excited under some conditions. We train an autoregressive Bayesian neural network model to forecast the amplitude of the dynamic pressure time series, inputting multiple sensor measurements (injector pressure/ temperature measurements, static chamber pressure, high-frequency dynamic pressure measurements, high-frequency OH* chemiluminescence measurements) and future flow rate control signals. The Bayesian nature of our algorithms allows us to work with a dataset whose size is restricted by the expense of each experimental run, without making overconfident extrapolations. We find that the networks are able to accurately forecast the evolution of the pressure amplitude and anticipate instability events on unseen experimental runs 500 milliseconds in advance. We compare the predictive accuracy of multiple models using different combinations of sensor inputs. We find that the high-frequency dynamic pressure signal is particularly informative. We also use the technique of integrated gradients to interpret the influence of different sensor inputs on the model prediction. The negative log-likelihood of data points in the test dataset indicates that predictive uncertainties are well-characterized by our Bayesian model and simulating a sensor failure event results as expected in a dramatic increase in the epistemic component of the uncertainty.
Semi-Random Impossibilities of Condorcet Criterion
Authors: Lirong Xia
Date: 2021-07-14 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction

The Condorcet criterion (CC) is a classical and well-accepted criterion for voting. Unfortunately, it is incompatible with many other desiderata including participation (Par), half-way monotonicity (HM), Maskin monotonicity (MM), and strategy-proofness (SP). Such incompatibilities are often known as impossibility theorems, and are proved by worst-case analysis. Previous work has investigated the likelihood for these impossibilities to occur under certain models, which are often criticized of being unrealistic. We strengthen previous work by proving the first set of semi-random impossibilities for voting rules to satisfy CC and the more general, group versions of the four desiderata: for any sufficiently large number of voters $n$, any size of the group $1\le B\le \sqrt n$, any voting rule $r$, and under a large class of {\em semi-random} models that include Impartial Culture, the likelihood for $r$ to satisfy CC and Par, CC and HM, CC and MM, or CC and SP is $1-\Omega(\frac{B}{\sqrt n})$. This matches existing lower bounds for CC and Par ($B=1$) and CC and SP ($B\le \sqrt n$), showing that many commonly-studied voting rules are already asymptotically optimal in such cases.
Simulating transmission scenarios of the Delta variant of SARS-CoV-2 in Australia
Authors: Sheryl L. Chang, Oliver M. Cliff, Cameron Zachreson, Mikhail Prokopenko
Date: 2021-07-14 00:00:00

An outbreak of the Delta (B.1.617.2) variant of SARS-CoV-2 that began around mid-June 2021 in Sydney, Australia, quickly developed into a nation-wide epidemic. The ongoing epidemic is of major concern as the Delta variant is more infectious than previous variants that circulated in Australia in 2020. Using a re-calibrated agent-based model, we explored a feasible range of non-pharmaceutical interventions, including case isolation, home quarantine, school closures, and stay-at-home restrictions (i.e., "social distancing"). Our modelling indicated that the levels of reduced interactions in workplaces and across communities attained in Sydney and other parts of the nation were inadequate for controlling the outbreak. A counter-factual analysis suggested that if 70% of the population followed tight stay-at-home restrictions, then at least 45 days would have been needed for new daily cases to fall from their peak to below ten per day. Our model predicted that, under a progressive vaccination rollout, if 40-50% of the Australian population follow stay-at-home restrictions, the incidence will peak by mid-October 2021: the peak in incidence across the nation was indeed observed in mid-October. We also quantified an expected burden on the healthcare system and potential fatalities across Australia.
A Diffuse Interface Model for Cell Blebbing Including Membrane-Cortex Coupling with Linker Dynamics
Authors: Philipp Werner, Martin Burger, Florian Frank, Harald Garcke
Date: 2021-07-13 00:00:00

The aim of this paper is to develop suitable models for the phenomenon of cell blebbing, which allow for computational predictions of mechanical effects including the crucial interaction of the cell membrane and the actin cortex. For this sake we resort to a two phase-field model that uses diffuse descriptions of both the membrane and the cortex, which in particular allows for a suitable description of the interaction via linker protein densities. Besides the detailed modelling we discuss some energetic aspects of the models and present a numerical scheme, which allows to carry out several computational studies. In those we demonstrate that several effects found in experiments can be reproduced, in particular bleb formation by cortex rupture, which was not possible by previous models without the linker dynamics.
You Only Write Thrice: Creating Documents, Computational Notebooks and Presentations From a Single Source
Authors: Kacper Sokol, Peter Flach
Date: 2021-07-02 00:00:00
Tasks: Management

Academic trade requires juggling multiple variants of the same content published in different formats: manuscripts, presentations, posters and computational notebooks. The need to track versions to accommodate for the write--review--rebut--revise life-cycle adds another layer of complexity. We propose to significantly reduce this burden by maintaining a single source document in a version-controlled environment (such as git), adding functionality to generate a collection of output formats popular in academia. To this end, we utilise various open-source tools from the Jupyter scientific computing ecosystem and operationalise selected software engineering concepts. We offer a proof-of-concept workflow that composes Jupyter Book (an online document), Jupyter Notebook (a computational narrative) and reveal.js slides from a single markdown source file. Hosted on GitHub, our approach supports change tracking and versioning, as well as a transparent review process based on the underlying code issue management infrastructure. An exhibit of our workflow can be previewed at https://so-cool.github.io/you-only-write-thrice/.
Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection
Authors: Ruijie Tao, Zexu Pan, Rohan Kumar Das, Xinyuan Qian, Mike Zheng Shou, Haizhou Li
Date: 2021-07-14 00:00:00
Tasks: Active Speaker Detection, Audio-Visual Active Speaker Detection

Active speaker detection (ASD) seeks to detect who is speaking in a visual scene of one or more speakers. The successful ASD depends on accurate interpretation of short-term and long-term audio and visual information, as well as audio-visual interaction. Unlike the prior work where systems make decision instantaneously using short-term features, we propose a novel framework, named TalkNet, that makes decision by taking both short-term and long-term features into consideration. TalkNet consists of audio and visual temporal encoders for feature representation, audio-visual cross-attention mechanism for inter-modality interaction, and a self-attention mechanism to capture long-term speaking evidence. The experiments demonstrate that TalkNet achieves 3.5% and 2.2% improvement over the state-of-the-art systems on the AVA-ActiveSpeaker dataset and Columbia ASD dataset, respectively. Code has been made available at: https://github.com/TaoRuijie/TalkNet_ASD.
Multi-Task Audio Source Separation
Authors: Lu Zhang, Chenxing Li, Feng Deng, Xiaorui Wang
Date: 2021-07-14 00:00:00
Tasks: Audio Source Separation, Multi-task Audio Source Seperation, Music Source Separation, Speech Enhancement, Speech Separation

The audio source separation tasks, such as speech enhancement, speech separation, and music source separation, have achieved impressive performance in recent studies. The powerful modeling capabilities of deep neural networks give us hope for more challenging tasks. This paper launches a new multi-task audio source separation (MTASS) challenge to separate the speech, music, and noise signals from the monaural mixture. First, we introduce the details of this task and generate a dataset of mixtures containing speech, music, and background noises. Then, we propose an MTASS model in the complex domain to fully utilize the differences in spectral characteristics of the three audio signals. In detail, the proposed model follows a two-stage pipeline, which separates the three types of audio signals and then performs signal compensation separately. After comparing different training targets, the complex ratio mask is selected as a more suitable target for the MTASS. The experimental results also indicate that the residual signal compensation module helps to recover the signals further. The proposed model shows significant advantages in separation performance over several well-known separation models.
Dynamic Power Systems Line Outage Detection Using Particle Filter and Partially Observed States
Authors: Xiaozhou Yang, Nan Chen, Chao Zhai
Date: 2021-07-14 00:00:00
Tasks: Diagnostic

Real-time transmission line outage detection is difficult because of partial phasor measurement unit (PMU) deployment and varying outage signal strength. Existing detection approaches focus on monitoring PMU-measured nodal algebraic states, i.e., voltage phase angle and magnitude. The success of such approaches, however, is largely predicated on strong outage signals and the presence of PMUs in the outage location's vicinity. To overcome these limitations, a unified framework is proposed in this work by utilizing both nodal voltage information and generator dynamic states, e.g., rotor angular position. The proposed scheme is shown to be faster and more robust to unknown outage locations through the incorporation of generator dynamics. Using the IEEE 39-bus system simulation data, the proposed scheme's properties and performances compared to existing approaches are presented. The new approach could help improve operators' real-time situational awareness by detecting outages faster and providing a breakdown of outage signals for diagnostic purposes, making future power systems more resilient.
Exploring DMD-type Algorithms for Modeling Signalised Intersections
Authors: Kazi Redwan Shabab, Shakib Mustavee, Shaurya Agarwal, Mohamed H. Zaki, Sajal Das
Date: 2021-07-13 00:00:00
Tasks: Vocal Bursts Type Prediction

This paper explores a novel data-driven approach based on recent developments in Koopman operator theory and dynamic mode decomposition (DMD) for modeling signalized intersections. Vehicular flow and queue formation on signalized intersections have complex nonlinear dynamics, making system identification, modeling, and controller design tasks challenging. We employ a Koopman theoretic approach to transform the original nonlinear dynamics into locally linear infinite-dimensional dynamics. The data-driven approach relies entirely on spatio-temporal snapshots of the traffic data. We investigate several key aspects of the approach and provide insights into the usage of DMD-type algorithms for application in adaptive signalized intersections. To demonstrate the utility of the obtained linearized dynamics, we perform prediction of the queue lengths at the intersection; and compare the results with the state-of-the-art long short term memory (LSTM) method. The case study involves the morning peak vehicle movements and queue lengths at two Orlando area signalized intersections. It is observed that DMD-based algorithms are able to capture complex dynamics with a linear approximation to a reasonable extent.