pwc-archive/papers-with-abstracts huggingface dataset:
A Novel End-To-End Network for Reconstruction of Non-Regularly Sampled Image Data Using Locally Fully Connected Layers
Authors: Simon Grosche, Fabian Brand, André Kaup
Date: 2022-03-17 00:00:00
Tasks: Super-Resolution

Quarter sampling and three-quarter sampling are novel sensor concepts that enable the acquisition of higher resolution images without increasing the number of pixels. This is achieved by non-regularly covering parts of each pixel of a low-resolution sensor such that only one quadrant or three quadrants of the sensor area of each pixel is sensitive to light. Combining a properly designed mask and a high-quality reconstruction algorithm, a higher image quality can be achieved than using a low-resolution sensor and subsequent upsampling. For the latter case, the image quality can be further enhanced using super resolution algorithms such as the very deep super resolution network (VDSR). In this paper, we propose a novel end-to-end neural network to reconstruct high resolution images from non-regularly sampled sensor data. The network is a concatenation of a locally fully connected reconstruction network (LFCR) and a standard VDSR network. Altogether, using a three-quarter sampling sensor with our novel neural network layout, the image quality in terms of PSNR for the Urban100 dataset can be increased by 2.96 dB compared to the state-of-the-art approach. Compared to a low-resolution sensor with VDSR, a gain of 1.11 dB is achieved.
SoK: Differential Privacy on Graph-Structured Data
Authors: Tamara T. Mueller, Dmitrii Usynin, Johannes C. Paetzold, Daniel Rueckert, Georgios Kaissis
Date: 2022-03-17 00:00:00
Tasks: BIG-bench Machine Learning, Graph Learning

In this work, we study the applications of differential privacy (DP) in the context of graph-structured data. We discuss the formulations of DP applicable to the publication of graphs and their associated statistics as well as machine learning on graph-based data, including graph neural networks (GNNs). The formulation of DP in the context of graph-structured data is difficult, as individual data points are interconnected (often non-linearly or sparsely). This connectivity complicates the computation of individual privacy loss in differentially private learning. The problem is exacerbated by an absence of a single, well-established formulation of DP in graph settings. This issue extends to the domain of GNNs, rendering private machine learning on graph-structured data a challenging task. A lack of prior systematisation work motivated us to study graph-based learning from a privacy perspective. In this work, we systematise different formulations of DP on graphs, discuss challenges and promising applications, including the GNN domain. We compare and separate works into graph analysis tasks and graph learning tasks with GNNs. Finally, we conclude our work with a discussion of open questions and potential directions for further research in this area.
On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels
Authors: Amnon Geifman, Meirav Galun, David Jacobs, Ronen Basri
Date: 2022-03-17 00:00:00

We study the properties of various over-parametrized convolutional neural architectures through their respective Gaussian process and neural tangent kernels. We prove that, with normalized multi-channel input and ReLU activation, the eigenfunctions of these kernels with the uniform measure are formed by products of spherical harmonics, defined over the channels of the different pixels. We next use hierarchical factorizable kernels to bound their respective eigenvalues. We show that the eigenvalues decay polynomially, quantify the rate of decay, and derive measures that reflect the composition of hierarchical features in these networks. Our results provide concrete quantitative characterization of over-parameterized convolutional network architectures.
Explainability in Graph Neural Networks: An Experimental Survey
Authors: Peibo Li, Yixing Yang, Maurice Pagnucco, Yang song
Date: 2022-03-17 00:00:00
Tasks: Graph Representation Learning, Representation Learning, Survey

Graph neural networks (GNNs) have been extensively developed for graph representation learning in various application domains. However, similar to all other neural networks models, GNNs suffer from the black-box problem as people cannot understand the mechanism underlying them. To solve this problem, several GNN explainability methods have been proposed to explain the decisions made by GNNs. In this survey, we give an overview of the state-of-the-art GNN explainability methods and how they are evaluated. Furthermore, we propose a new evaluation metric and conduct thorough experiments to compare GNN explainability methods on real world datasets. We also suggest future directions for GNN explainability.
Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Series
Authors: Kristoffer Wickstrøm, Michael Kampffmeyer, Karl Øyvind Mikalsen, Robert Jenssen
Date: 2022-03-17 00:00:00
Tasks: Contrastive Learning, Data Augmentation, Representation Learning, Time Series, Time Series Analysis, Transfer Learning

The lack of labeled data is a key challenge for learning useful representation from time series data. However, an unsupervised representation framework that is capable of producing high quality representations could be of great value. It is key to enabling transfer learning, which is especially beneficial for medical applications, where there is an abundance of data but labeling is costly and time consuming. We propose an unsupervised contrastive learning framework that is motivated from the perspective of label smoothing. The proposed approach uses a novel contrastive loss that naturally exploits a data augmentation scheme in which new samples are generated by mixing two data samples with a mixing component. The task in the proposed framework is to predict the mixing component, which is utilized as soft targets in the loss function. Experiments demonstrate the framework's superior performance compared to other representation learning approaches on both univariate and multivariate time series and illustrate its benefits for transfer learning for clinical time series.
Knowledge Graph Embedding Methods for Entity Alignment: An Experimental Review
Authors: Nikolaos Fanourakis, Vasilis Efthymiou, Dimitris Kotzinos, Vassilis Christophides
Date: 2022-03-17 00:00:00
Tasks: Attribute, Entity Alignment, Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Question Answering

In recent years, we have witnessed the proliferation of knowledge graphs (KG) in various domains, aiming to support applications like question answering, recommendations, etc. A frequent task when integrating knowledge from different KGs is to find which subgraphs refer to the same real-world entity. Recently, embedding methods have been used for entity alignment tasks, that learn a vector-space representation of entities which preserves their similarity in the original KGs. A wide variety of supervised, unsupervised, and semi-supervised methods have been proposed that exploit both factual (attribute based) and structural information (relation based) of entities in the KGs. Still, a quantitative assessment of their strengths and weaknesses in real-world KGs according to different performance metrics and KG characteristics is missing from the literature. In this work, we conduct the first meta-level analysis of popular embedding methods for entity alignment, based on a statistically sound methodology. Our analysis reveals statistically significant correlations of different embedding methods with various meta-features extracted by KGs and rank them in a statistically significant way according to their effectiveness across all real-world KGs of our testbed. Finally, we study interesting trade-offs in terms of methods' effectiveness and efficiency.
One-Shot Adaptation of GAN in Just One CLIP
Authors: Gihyun Kwon, Jong Chul Ye
Date: 2022-03-17 00:00:00
Tasks: Attribute, Image Retrieval

There are many recent research efforts to fine-tune a pre-trained generator with a few target images to generate images of a novel domain. Unfortunately, these methods often suffer from overfitting or under-fitting when fine-tuned with a single target image. To address this, here we present a novel single-shot GAN adaptation method through unified CLIP space manipulations. Specifically, our model employs a two-step training strategy: reference image search in the source generator using a CLIP-guided latent optimization, followed by generator fine-tuning with a novel loss function that imposes CLIP space consistency between the source and adapted generators. To further improve the adapted model to produce spatially consistent samples with respect to the source generator, we also propose contrastive regularization for patchwise relationships in the CLIP space. Experimental results show that our model generates diverse outputs with the target texture and outperforms the baseline models both qualitatively and quantitatively. Furthermore, we show that our CLIP space manipulation strategy allows more effective attribute editing.
CYBORGS: Contrastively Bootstrapping Object Representations by Grounding in Segmentation
Authors: Renhao Wang, Hang Zhao, Yang Gao
Date: 2022-03-17 00:00:00
Tasks: Contrastive Learning, Object, Segmentation

Many recent approaches in contrastive learning have worked to close the gap between pretraining on iconic images like ImageNet and pretraining on complex scenes like COCO. This gap exists largely because commonly used random crop augmentations obtain semantically inconsistent content in crowded scene images of diverse objects. Previous works use preprocessing pipelines to localize salient objects for improved cropping, but an end-to-end solution is still elusive. In this work, we propose a framework which accomplishes this goal via joint learning of representations and segmentation. We leverage segmentation masks to train a model with a mask-dependent contrastive loss, and use the partially trained model to bootstrap better masks. By iterating between these two components, we ground the contrastive updates in segmentation information, and simultaneously improve segmentation throughout pretraining. Experiments show our representations transfer robustly to downstream tasks in classification, detection and segmentation.
Context-Dependent Anomaly Detection with Knowledge Graph Embedding Models
Authors: Nathan Vaska, Kevin Leahy, Victoria Helus
Date: 2022-03-17 00:00:00
Tasks: Anomaly Detection, Graph Embedding, Knowledge Graph Embedding, Link Prediction

Increasing the semantic understanding and contextual awareness of machine learning models is important for improving robustness and reducing susceptibility to data shifts. In this work, we leverage contextual awareness for the anomaly detection problem. Although graphed-based anomaly detection has been widely studied, context-dependent anomaly detection is an open problem and without much current research. We develop a general framework for converting a context-dependent anomaly detection problem to a link prediction problem, allowing well-established techniques from this domain to be applied. We implement a system based on our framework that utilizes knowledge graph embedding models and demonstrates the ability to detect outliers using context provided by a semantic knowledge base. We show that our method can detect context-dependent anomalies with a high degree of accuracy and show that current object detectors can detect enough classes to provide the needed context for good performance within our example domain.
Euler State Networks: Non-dissipative Reservoir Computing
Authors: Claudio Gallicchio
Date: 2022-03-17 00:00:00
Tasks: Memorization, Time Series, Time Series Analysis, Time Series Classification

Inspired by the numerical solution of ordinary differential equations, in this paper we propose a novel Reservoir Computing (RC) model, called the Euler State Network (EuSN). The presented approach makes use of forward Euler discretization and antisymmetric recurrent matrices to design reservoir dynamics that are both stable and non-dissipative by construction. Our mathematical analysis shows that the resulting model is biased towards a unitary effective spectral radius and zero local Lyapunov exponents, intrinsically operating near to the edge of stability. Experiments on long-term memory tasks show the clear superiority of the proposed approach over standard RC models in problems requiring effective propagation of input information over multiple time-steps. Furthermore, results on time-series classification benchmarks indicate that EuSN is able to match (or even exceed) the accuracy of trainable Recurrent Neural Networks, while retaining the training efficiency of the RC family, resulting in up to $\approx$ 490-fold savings in computation time and $\approx$ 1750-fold savings in energy consumption.
Learning of Structurally Unambiguous Probabilistic Grammars
Authors: Dana Fisman, Dolav Nitay, Michal Ziv-Ukelson
Date: 2022-03-17 00:00:00

The problem of identifying a probabilistic context free grammar has two aspects: the first is determining the grammar's topology (the rules of the grammar) and the second is estimating probabilistic weights for each rule. Given the hardness results for learning context-free grammars in general, and probabilistic grammars in particular, most of the literature has concentrated on the second problem. In this work we address the first problem. We restrict attention to structurally unambiguous weighted context-free grammars (SUWCFG) and provide a query learning algorithm for \structurally unambiguous probabilistic context-free grammars (SUPCFG). We show that SUWCFG can be represented using \emph{co-linear multiplicity tree automata} (CMTA), and provide a polynomial learning algorithm that learns CMTAs. We show that the learned CMTA can be converted into a probabilistic grammar, thus providing a complete algorithm for learning a structurally unambiguous probabilistic context free grammar (both the grammar topology and the probabilistic weights) using structured membership queries and structured equivalence queries. A summarized version of this work was published at AAAI 21.
MolNet: A Chemically Intuitive Graph Neural Network for Prediction of Molecular Properties
Authors: YeJi Kim, Yoonho Jeong, Jihoo Kim, Eok Kyun Lee, Won June Kim, Insung S. Choi
Date: 2022-02-01 00:00:00
Tasks: Graph Neural Network

The graph neural network (GNN) has been a powerful deep-learning tool in chemistry domain, due to its close connection with molecular graphs. Most GNN models collect and update atom and molecule features from the fed atom (and, in some cases, bond) features, which are basically based on the two-dimensional (2D) graph representation of 3D molecules. Correspondingly, the adjacency matrix, containing the information on covalent bonds, or equivalent data structures (e.g., lists) have been the main core in the feature-updating processes, such as graph convolution. However, the 2D-based models do not faithfully represent 3D molecules and their physicochemical properties, exemplified by the overlooked field effect that is a "through-space" effect, not a "through-bond" effect. The GNN model proposed herein, denoted as MolNet, is chemically intuitive, accommodating the 3D non-bond information in a molecule, with a noncovalent adjacency matrix $\bf{\bar A}$, and also bond-strength information from a weighted bond matrix $\bf{B}$. The noncovalent atoms, not directly bonded to a given atom in a molecule, are identified within 5 $\unicode{x212B}$ of cut-off range for the construction of $\bf{\bar A}$, and $\bf{B}$ has edge weights of 1, 1.5, 2, and 3 for single, aromatic, double, and triple bonds, respectively. Comparative studies show that MolNet outperforms various baseline GNN models and gives a state-of-the-art performance in the classification task of BACE dataset and regression task of ESOL dataset. This work suggests a future direction of deep-learning chemistry in the construction of deep-learning models that are chemically intuitive and comparable with the existing chemistry concepts and tools.
Self-Normalized Density Map (SNDM) for Counting Microbiological Objects
Authors: Krzysztof M. Graczyk, Jaroslaw Pawlowski, Sylwia Majchrowska, Tomasz Golan
Date: 2022-03-15 00:00:00

The statistical properties of the density map (DM) approach to counting microbiological objects on images are studied in detail. The DM is given by U$^2$-Net. Two statistical methods for deep neural networks are utilized: the bootstrap and the Monte Carlo (MC) dropout. The detailed analysis of the uncertainties for the DM predictions leads to a deeper understanding of the DM model's deficiencies. Based on our investigation, we propose a self-normalization module in the network. The improved network model, called \textit{Self-Normalized Density Map} (SNDM), can correct its output density map by itself to accurately predict the total number of objects in the image. The SNDM architecture outperforms the original model. Moreover, both statistical frameworks -- bootstrap and MC dropout -- have consistent statistical results for SNDM, which were not observed in the original model. The SNDM efficiency is comparable with the detector-base models, such as Faster and Cascade R-CNN detectors.
Uncertainty with UAV Search of Multiple Goal-oriented Targets
Authors: Mor Sinay, Noa Agmon, Oleg Maksimov, Aviad Fux, Sarit Kraus
Date: 2022-03-03 00:00:00

This paper considers the complex problem of a team of UAVs searching targets under uncertainty. The goal of the UAV team is to find all of the moving targets as quickly as possible before they arrive at their selected goal. The uncertainty considered is threefold: First, the UAVs do not know the targets' locations and destinations. Second, the sensing capabilities of the UAVs are not perfect. Third, the targets' movement model is unknown. We suggest a real-time algorithmic framework for the UAVs, combining entropy and stochastic-temporal belief, that aims at optimizing the probability of a quick and successful detection of all of the targets. We have empirically evaluated the algorithmic framework, and have shown its efficiency and significant performance improvement compared to other solutions. Furthermore, we have evaluated our framework using Peer Designed Agents (PDAs), which are computer agents that simulate targets and show that our algorithmic framework outperforms other solutions in this scenario.
Towards Data-Efficient Detection Transformers
Authors: Wen Wang, Jing Zhang, Yang Cao, Yongliang Shen, DaCheng Tao
Date: 2022-03-17 00:00:00

Detection Transformers have achieved competitive performance on the sample-rich COCO dataset. However, we show most of them suffer from significant performance drops on small-size datasets, like Cityscapes. In other words, the detection transformers are generally data-hungry. To tackle this problem, we empirically analyze the factors that affect data efficiency, through a step-by-step transition from a data-efficient RCNN variant to the representative DETR. The empirical results suggest that sparse feature sampling from local image areas holds the key. Based on this observation, we alleviate the data-hungry issue of existing detection transformers by simply alternating how key and value sequences are constructed in the cross-attention layer, with minimum modifications to the original models. Besides, we introduce a simple yet effective label augmentation method to provide richer supervision and improve data efficiency. Experiments show that our method can be readily applied to different detection transformers and improve their performance on both small-size and sample-rich datasets. Code will be made publicly available at \url{https://github.com/encounter1997/DE-DETRs}.
DetMatch: Two Teachers are Better Than One for Joint 2D and 3D Semi-Supervised Object Detection
Authors: Jinhyung Park, Chenfeng Xu, Yiyang Zhou, Masayoshi Tomizuka, Wei Zhan
Date: 2022-03-17 00:00:00
Tasks: object-detection, Object Detection, Object Recognition, Semi-Supervised Object Detection

While numerous 3D detection works leverage the complementary relationship between RGB images and point clouds, developments in the broader framework of semi-supervised object recognition remain uninfluenced by multi-modal fusion. Current methods develop independent pipelines for 2D and 3D semi-supervised learning despite the availability of paired image and point cloud frames. Observing that the distinct characteristics of each sensor cause them to be biased towards detecting different objects, we propose DetMatch, a flexible framework for joint semi-supervised learning on 2D and 3D modalities. By identifying objects detected in both sensors, our pipeline generates a cleaner, more robust set of pseudo-labels that both demonstrates stronger performance and stymies single-modality error propagation. Further, we leverage the richer semantics of RGB images to rectify incorrect 3D class predictions and improve localization of 3D boxes. Evaluating on the challenging KITTI and Waymo datasets, we improve upon strong semi-supervised learning methods and observe higher quality pseudo-labels. Code will be released at https://github.com/Divadi/DetMatch
On Multi-Domain Long-Tailed Recognition, Imbalanced Domain Generalization and Beyond
Authors: Yuzhe Yang, Hao Wang, Dina Katabi
Date: 2022-03-17 00:00:00
Tasks: Domain Generalization

Real-world data often exhibit imbalanced label distributions. Existing studies on data imbalance focus on single-domain settings, i.e., samples are from the same data distribution. However, natural data can originate from distinct domains, where a minority class in one domain could have abundant instances from other domains. We formalize the task of Multi-Domain Long-Tailed Recognition (MDLT), which learns from multi-domain imbalanced data, addresses label imbalance, domain shift, and divergent label distributions across domains, and generalizes to all domain-class pairs. We first develop the domain-class transferability graph, and show that such transferability governs the success of learning in MDLT. We then propose BoDA, a theoretically grounded learning strategy that tracks the upper bound of transferability statistics, and ensures balanced alignment and calibration across imbalanced domain-class distributions. We curate five MDLT benchmarks based on widely-used multi-domain datasets, and compare BoDA to twenty algorithms that span different learning strategies. Extensive and rigorous experiments verify the superior performance of BoDA. Further, as a byproduct, BoDA establishes new state-of-the-art on Domain Generalization benchmarks, highlighting the importance of addressing data imbalance across domains, which can be crucial for improving generalization to unseen domains. Code and data are available at: https://github.com/YyzHarry/multi-domain-imbalance.
The Data Airlock: infrastructure for restricted data informatics
Authors: Gregory Rolan, Janis Dalins, Campbell Wilson
Date: 2022-03-17 00:00:00

Data science collaboration is problematic when access to operational data or models from outside the data-holding organisation is prohibited, for a variety of legal, security, ethical, or practical reasons. There are significant data privacy challenges when performing collaborative data science work against such restricted data. In this paper we describe a range of causes and risks associated with restricted data along with the social, environmental, data, and cryptographic measures that may be used to mitigate such issues. We then show how these are generally inadequate for restricted data contexts and introduce the 'Data Airlock' - secure infrastructure that facilitates 'eyes-off' data science workloads. After describing our use-case we detail the architecture and implementation of a first, single-organisation version of the Data Airlock infrastructure. We conclude with outcomes and learning from this implementation, and outline requirements for a second, federated version.