pwc-archive/papers-with-abstracts huggingface dataset:
A Step Toward More Inclusive People Annotations for Fairness
Authors: Candice Schumann, Susanna Ricco, Utsav Prabhu, Vittorio Ferrari, Caroline Pantofaru
Date: 2021-05-05 00:00:00
Tasks: Attribute, Fairness

The Open Images Dataset contains approximately 9 million images and is a widely accepted dataset for computer vision research. As is common practice for large datasets, the annotations are not exhaustive, with bounding boxes and attribute labels for only a subset of the classes in each image. In this paper, we present a new set of annotations on a subset of the Open Images dataset called the MIAP (More Inclusive Annotations for People) subset, containing bounding boxes and attributes for all of the people visible in those images. The attributes and labeling methodology for the MIAP subset were designed to enable research into model fairness. In addition, we analyze the original annotation methodology for the person class and its subclasses, discussing the resulting patterns in order to inform future annotation efforts. By considering both the original and exhaustive annotation sets, researchers can also now study how systematic patterns in training annotations affect modeling.
R2U3D: Recurrent Residual 3D U-Net for Lung Segmentation
Authors: Dhaval D. Kadia, Md Zahangir Alom, Ranga Burada, Tam V. Nguyen, Vijayan K. Asari
Date: 2021-05-05 00:00:00
Tasks: Data Augmentation, Image Segmentation, Segmentation, Semantic Segmentation

3D lung segmentation is essential since it processes the volumetric information of the lungs, removes the unnecessary areas of the scan, and segments the actual area of the lungs in a 3D volume. Recently, the deep learning model, such as U-Net outperforms other network architectures for biomedical image segmentation. In this paper, we propose a novel model, namely, Recurrent Residual 3D U-Net (R2U3D), for the 3D lung segmentation task. In particular, the proposed model integrates 3D convolution into the Recurrent Residual Neural Network based on U-Net. It helps learn spatial dependencies in 3D and increases the propagation of 3D volumetric information. The proposed R2U3D network is trained on the publicly available dataset LUNA16 and it achieves state-of-the-art performance on both LUNA16 (testing set) and VESSEL12 dataset. In addition, we show that training the R2U3D model with a smaller number of CT scans, i.e., 100 scans, without applying data augmentation achieves an outstanding result in terms of Soft Dice Similarity Coefficient (Soft-DSC) of 0.9920.
A Metamodel Structure For Regression Analysis: Application To Prediction Of Autism Spectrum Disorder Severity
Authors: Shiyu Wang, Nicha C. Dvornek
Date: 2021-05-06 00:00:00
Tasks: regression

Traditional regression models do not generalize well when learning from small and noisy datasets. Here we propose a novel metamodel structure to improve the regression result. The metamodel is composed of multiple classification base models and a regression model built upon the base models. We test this structure on the prediction of autism spectrum disorder (ASD) severity as measured by the ADOS communication (ADOS COMM) score from resting-state fMRI data, using a variety of base models. The metamodel outperforms traditional regression models as measured by the Pearson correlation coefficient between true and predicted scores and stability. In addition, we found that the metamodel is more flexible and more generalizable.
Estimating Reproducible Functional Networks Associated with Task Dynamics using Unsupervised LSTMs
Authors: Nicha C. Dvornek, Pamela Ventola, James S. Duncan
Date: 2021-05-06 00:00:00
Tasks: Time Series, Time Series Analysis

We propose a method for estimating more reproducible functional networks that are more strongly associated with dynamic task activity by using recurrent neural networks with long short term memory (LSTMs). The LSTM model is trained in an unsupervised manner to learn to generate the functional magnetic resonance imaging (fMRI) time-series data in regions of interest. The learned functional networks can then be used for further analysis, e.g., correlation analysis to determine functional networks that are strongly associated with an fMRI task paradigm. We test our approach and compare to other methods for decomposing functional networks from fMRI activity on 2 related but separate datasets that employ a biological motion perception task. We demonstrate that the functional networks learned by the LSTM model are more strongly associated with the task activity and dynamics compared to other approaches. Furthermore, the patterns of network association are more closely replicated across subjects within the same dataset as well as across datasets. More reproducible functional networks are essential for better characterizing the neural correlates of a target task.
Membership Inference Attacks on Deep Regression Models for Neuroimaging
Authors: Umang Gupta, Dimitris Stripelis, Pradeep K. Lam, Paul M. Thompson, José Luis Ambite, Greg Ver Steeg
Date: 2021-05-06 00:00:00
Tasks: Deep Learning, Federated Learning, regression

Ensuring the privacy of research participants is vital, even more so in healthcare environments. Deep learning approaches to neuroimaging require large datasets, and this often necessitates sharing data between multiple sites, which is antithetical to the privacy objectives. Federated learning is a commonly proposed solution to this problem. It circumvents the need for data sharing by sharing parameters during the training process. However, we demonstrate that allowing access to parameters may leak private information even if data is never directly shared. In particular, we show that it is possible to infer if a sample was used to train the model given only access to the model prediction (black-box) or access to the model itself (white-box) and some leaked samples from the training data distribution. Such attacks are commonly referred to as Membership Inference attacks. We show realistic Membership Inference attacks on deep learning models trained for 3D neuroimaging tasks in a centralized as well as decentralized setup. We demonstrate feasible attacks on brain age prediction models (deep learning models that predict a person's age from their brain MRI scan). We correctly identified whether an MRI scan was used in model training with a 60% to over 80% success rate depending on model complexity and security assumptions.
KuraNet: Systems of Coupled Oscillators that Learn to Synchronize
Authors: Matthew Ricci, Minju Jung, Yuwei Zhang, Mathieu Chalvidal, Aneri Soni, Thomas Serre
Date: 2021-05-06 00:00:00

Networks of coupled oscillators are some of the most studied objects in the theory of dynamical systems. Two important areas of current interest are the study of synchrony in highly disordered systems and the modeling of systems with adaptive network structures. Here, we present a single approach to both of these problems in the form of "KuraNet", a deep-learning-based system of coupled oscillators that can learn to synchronize across a distribution of disordered network conditions. The key feature of the model is the replacement of the traditionally static couplings with a coupling function which can learn optimal interactions within heterogeneous oscillator populations. We apply our approach to the eponymous Kuramoto model and demonstrate how KuraNet can learn data-dependent coupling structures that promote either global or cluster synchrony. For example, we show how KuraNet can be used to empirically explore the conditions of global synchrony in analytically impenetrable models with disordered natural frequencies, external field strengths, and interaction delays. In a sequence of cluster synchrony experiments, we further show how KuraNet can function as a data classifier by synchronizing into coherent assemblies. In all cases, we show how KuraNet can generalize to both new data and new network scales, making it easy to work with small systems and form hypotheses about the thermodynamic limit. Our proposed learning-based approach is broadly applicable to arbitrary dynamical systems with wide-ranging relevance to modeling in physics and systems biology.
The layer-wise L1 Loss Landscape of Neural Nets is more complex around local minima
Authors: Peter Hinz
Date: 2021-05-06 00:00:00

For fixed training data and network parameters in the other layers the L1 loss of a ReLU neural network as a function of the first layer's parameters is a piece-wise affine function. We use the Deep ReLU Simplex algorithm to iteratively minimize the loss monotonically on adjacent vertices and analyze the trajectory of these vertex positions. We empirically observe that in a neighbourhood around a local minimum, the iterations behave differently such that conclusions on loss level and proximity of the local minimum can be made before it has been found: Firstly the loss seems to decay exponentially slow at iterated adjacent vertices such that the loss level at the local minimum can be estimated from the loss levels of subsequently iterated vertices, and secondly we observe a strong increase of the vertex density around local minima. This could have far-reaching consequences for the design of new gradient-descent algorithms that might improve convergence rate by exploiting these facts.
Pathloss modeling for in-body optical wireless communications
Authors: Stylianos E. Trevlakis, Alexandros-Apostolos A. Boulogeorgos, Nestor D. Chatzidiamantis
Date: 2021-05-06 00:00:00

Optical wireless communications (OWCs) have been recognized as a candidate enabler of next generation in-body nano-scale networks and implants. The development of an accurate channel model capable of accommodating the particularities of different type of tissues is expected to boost the design of optimized communication protocols for such applications. Motivated by this, this paper focuses on presenting a general pathloss model for in-body OWCs. In particular, we use experimental measurements in order to extract analytical expressions for the absorption coefficients of the five main tissues' constitutions, namely oxygenated and de-oxygenated blood, water, fat, and melanin. Building upon these expressions, we derive a general formula for the absorption coefficient evaluation of any biological tissue. To verify the validity of this formula, we compute the absorption coefficient of complex tissues and compare them against respective experimental results reported by independent research works. Interestingly, we observe that the analytical formula has high accuracy and is capable of modeling the pathloss and, therefore, the penetration depth in complex tissues.
Semidefinite Programming for Community Detection with Side Information
Authors: Mohammad Esmaeili, Hussein Metwaly Saad, Aria Nosratinia
Date: 2021-05-06 00:00:00
Tasks: Community Detection

This paper produces an efficient Semidefinite Programming (SDP) solution for community detection that incorporates non-graph data, which in this context is known as side information. SDP is an efficient solution for standard community detection on graphs. We formulate a semi-definite relaxation for the maximum likelihood estimation of node labels, subject to observing both graph and non-graph data. This formulation is distinct from the SDP solution of standard community detection, but maintains its desirable properties. We calculate the exact recovery threshold for three types of non-graph information, which in this paper are called side information: partially revealed labels, noisy labels, as well as multiple observations (features) per node with arbitrary but finite cardinality. We find that SDP has the same exact recovery threshold in the presence of side information as maximum likelihood with side information. Thus, the methods developed herein are computationally efficient as well as asymptotically accurate for the solution of community detection in the presence of side information. Simulations show that the asymptotic results of this paper can also shed light on the performance of SDP for graphs of modest size.
Practical and Rigorous Uncertainty Bounds for Gaussian Process Regression
Authors: Christian Fiedler, Carsten W. Scherer, Sebastian Trimpe
Date: 2021-05-06 00:00:00
Tasks: Gaussian Processes, regression

Gaussian Process Regression is a popular nonparametric regression method based on Bayesian principles that provides uncertainty estimates for its predictions. However, these estimates are of a Bayesian nature, whereas for some important applications, like learning-based control with safety guarantees, frequentist uncertainty bounds are required. Although such rigorous bounds are available for Gaussian Processes, they are too conservative to be useful in applications. This often leads practitioners to replacing these bounds by heuristics, thus breaking all theoretical guarantees. To address this problem, we introduce new uncertainty bounds that are rigorous, yet practically useful at the same time. In particular, the bounds can be explicitly evaluated and are much less conservative than state of the art results. Furthermore, we show that certain model misspecifications lead to only graceful degradation. We demonstrate these advantages and the usefulness of our results for learning-based control with numerical examples.
A probabilistic model for missing traffic volume reconstruction based on data fusion
Authors: Xintao Yan, Yan Zhao, Henry X. Liu
Date: 2021-05-06 00:00:00
Tasks: Traffic Signal Control

Traffic volume information is critical for intelligent transportation systems. It serves as a key input to transportation planning, roadway design, and traffic signal control. However, the traffic volume data collected by fixed-location sensors, such as loop detectors, often suffer from the missing data problem and low coverage problem. The missing data problem could be caused by hardware malfunction. The low coverage problem is due to the limited coverage of fixed-location sensors in the transportation network, which restrains our understanding of the traffic at the network level. To tackle these problems, we propose a probabilistic model for traffic volume reconstruction by fusing fixed-location sensor data and probe vehicle data. We apply the probabilistic principal component analysis (PPCA) to capture the correlations in traffic volume data. An innovative contribution of this work is that we also integrate probe vehicle data into the framework, which allows the model to solve both of the above-mentioned two problems. Using a real-world traffic volume dataset, we show that the proposed method outperforms state-of-the-art methods for the extensively studied missing data problem. Moreover, for the low coverage problem, which cannot be handled by most existing methods, the proposed model can also achieve high accuracy. The experiments also show that even when the missing ratio reaches 80%, the proposed method can still give an accurate estimate of the unknown traffic volumes with only a 10% probe vehicle penetration rate. The results validate the effectiveness and robustness of the proposed model and demonstrate its potential for practical applications.
Modeling the geospatial evolution of COVID-19 using spatio-temporal convolutional sequence-to-sequence neural networks
Authors: Mário Cardoso, André Cavalheiro, Alexandre Borges, Ana F. Duarte, Amílcar Soares, Maria João Pereira, Nuno J. Nunes, Leonardo Azevedo, Arlindo L. Oliveira
Date: 2021-05-06 00:00:00

Europe was hit hard by the COVID-19 pandemic and Portugal was one of the most affected countries, having suffered three waves in the first twelve months. Approximately between Jan 19th and Feb 5th 2021 Portugal was the country in the world with the largest incidence rate, with 14-days incidence rates per 100,000 inhabitants in excess of 1000. Despite its importance, accurate prediction of the geospatial evolution of COVID-19 remains a challenge, since existing analytical methods fail to capture the complex dynamics that result from both the contagion within a region and the spreading of the infection from infected neighboring regions. We use a previously developed methodology and official municipality level data from the Portuguese Directorate-General for Health (DGS), relative to the first twelve months of the pandemic, to compute an estimate of the incidence rate in each location of mainland Portugal. The resulting sequence of incidence rate maps was then used as a gold standard to test the effectiveness of different approaches in the prediction of the spatial-temporal evolution of the incidence rate. Four different methods were tested: a simple cell level autoregressive moving average (ARMA) model, a cell level vector autoregressive (VAR) model, a municipality-by-municipality compartmental SIRD model followed by direct block sequential simulation and a convolutional sequence-to-sequence neural network model based on the STConvS2S architecture. We conclude that the convolutional sequence-to-sequence neural network is the best performing method, when predicting the medium-term future incidence rate, using the available information.
Solve routing problems with a residual edge-graph attention neural network
Authors: Kun Lei, Peng Guo, Yi Wang, Xiao Wu, Wenchao Zhao
Date: 2021-05-06 00:00:00
Tasks: Combinatorial Optimization, Deep Reinforcement Learning, Graph Attention, Traveling Salesman Problem

For NP-hard combinatorial optimization problems, it is usually difficult to find high-quality solutions in polynomial time. The design of either an exact algorithm or an approximate algorithm for these problems often requires significantly specialized knowledge. Recently, deep learning methods provide new directions to solve such problems. In this paper, an end-to-end deep reinforcement learning framework is proposed to solve this type of combinatorial optimization problems. This framework can be applied to different problems with only slight changes of input (for example, for a traveling salesman problem (TSP), the input is the two-dimensional coordinates of nodes; while for a capacity-constrained vehicle routing problem (CVRP), the input is simply changed to three-dimensional vectors including the two-dimensional coordinates and the customer demands of nodes), masks and decoder context vectors. The proposed framework is aiming to improve the models in literacy in terms of the neural network model and the training algorithm. The solution quality of TSP and the CVRP up to 100 nodes are significantly improved via our framework. Specifically, the average optimality gap is reduced from 4.53\% (reported best \cite{R22}) to 3.67\% for TSP with 100 nodes and from 7.34\% (reported best \cite{R22}) to 6.68\% for CVRP with 100 nodes when using the greedy decoding strategy. Furthermore, our framework uses about 1/3$\sim$3/4 training samples compared with other existing learning methods while achieving better results. The results performed on randomly generated instances and the benchmark instances from TSPLIB and CVRPLIB confirm that our framework has a linear running time on the problem size (number of nodes) during the testing phase, and has a good generalization performance from random instance training to real-world instance testing.
CrossWalk: Fairness-enhanced Node Representation Learning
Authors: Ahmad Khajehnejad, Moein Khajehnejad, Mahmoudreza Babaei, Krishna P. Gummadi, Adrian Weller, Baharan Mirzasoleiman
Date: 2021-05-06 00:00:00
Tasks: Fairness, Link Prediction, Node Classification, Representation Learning

The potential for machine learning systems to amplify social inequities and unfairness is receiving increasing popular and academic attention. Much recent work has focused on developing algorithmic tools to assess and mitigate such unfairness. However, there is little work on enhancing fairness in graph algorithms. Here, we develop a simple, effective and general method, CrossWalk, that enhances fairness of various graph algorithms, including influence maximization, link prediction and node classification, applied to node embeddings. CrossWalk is applicable to any random walk based node representation learning algorithm, such as DeepWalk and Node2Vec. The key idea is to bias random walks to cross group boundaries, by upweighting edges which (1) are closer to the groups' peripheries or (2) connect different groups in the network. CrossWalk pulls nodes that are near groups' peripheries towards their neighbors from other groups in the embedding space, while preserving the necessary structural properties of the graph. Extensive experiments show the effectiveness of our algorithm to enhance fairness in various graph algorithms, including influence maximization, link prediction and node classification in synthetic and real networks, with only a very small decrease in performance.
Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks
Authors: Hidenori Tanaka, Daniel Kunin
Date: 2021-05-06 00:00:00

In nature, symmetry governs regularities, while symmetry breaking brings texture. In artificial neural networks, symmetry has been a central design principle to efficiently capture regularities in the world, but the role of symmetry breaking is not well understood. Here, we develop a theoretical framework to study the "geometry of learning dynamics" in neural networks, and reveal a key mechanism of explicit symmetry breaking behind the efficiency and stability of modern neural networks. To build this understanding, we model the discrete learning dynamics of gradient descent using a continuous-time Lagrangian formulation, in which the learning rule corresponds to the kinetic energy and the loss function corresponds to the potential energy. Then, we identify "kinetic symmetry breaking" (KSB), the condition when the kinetic energy explicitly breaks the symmetry of the potential function. We generalize Noether's theorem known in physics to take into account KSB and derive the resulting motion of the Noether charge: "Noether's Learning Dynamics" (NLD). Finally, we apply NLD to neural networks with normalization layers and reveal how KSB introduces a mechanism of "implicit adaptive optimization", establishing an analogy between learning dynamics induced by normalization layers and RMSProp. Overall, through the lens of Lagrangian mechanics, we have established a theoretical foundation to discover geometric design principles for the learning dynamics of neural networks.
MIMII DUE: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspection with Domain Shifts due to Changes in Operational and Environmental Conditions
Authors: Ryo Tanabe, Harsh Purohit, Kota Dohi, Takashi Endo, Yuki Nikaido, Toshiki Nakamura, Yohei Kawaguchi
Date: 2021-05-06 00:00:00
Tasks: Task 2

In this paper, we introduce MIMII DUE, a new dataset for malfunctioning industrial machine investigation and inspection with domain shifts due to changes in operational and environmental conditions. Conventional methods for anomalous sound detection face practical challenges because the distribution of features changes between the training and operational phases (called domain shift) due to various real-world factors. To check the robustness against domain shifts, we need a dataset that actually includes domain shifts, but such a dataset does not exist so far. The new dataset we created consists of the normal and abnormal operating sounds of five different types of industrial machines under two different operational/environmental conditions (source domain and target domain) independent of normal/abnormal, with domain shifts occurring between the two domains. Experimental results showed significant performance differences between the source and target domains, indicating that the dataset contains the domain shifts. These findings demonstrate that the dataset will be helpful for checking the robustness against domain shifts. The dataset is a subset of the dataset for DCASE 2021 Challenge Task 2 and freely available for download at https://zenodo.org/record/4740355