pwc-archive/papers-with-abstracts huggingface dataset:
Attention for Adversarial Attacks: Learning from your Mistakes
Authors: Florian Jaeckle, Aleksandr Agadzhanov, Jingyue Lu, M. Pawan Kumar
Date: 2021-11-22 00:00:00
Proceeding: AAAI Workshop AdvML 2022 2
Tasks: Autonomous Driving

In order to apply Neural Networks in safety-critical settings, such as healthcare or autonomous driving, we need to be able to analyse their robustness against adversarial attacks. As complete verification is often computationally prohibitive, we rely on cheap and effective adversarial attacks to estimate their robustness. However, state-of-the-art adversarial attacks, such as the frequently used PGD attack, often require many random restarts to generate adversarial examples. Each time we perform a restart we ignore all previous unsuccessful runs. In order to alleviate this inefficiency, we propose a method that learns from its mistakes. Specifically, our method uses Graph Neural Networks (GNNs) as an attention mechanism, to greatly reduce the search space for the attacks. The architecture of the GNN is based on the neural network we are attacking, and we perform forward and backward passes though the GNN mimicking the back-propagation algorithm of PGD attacks. The GNN outputs a smaller subspace for the PGD attack to focus on. Using our method, we manage to boost the attacks' performance: the GNN increases the success rate of PGD by over 35\% on a recent published dataset used for comparing adversarial attacks, while simultaneously reducing its average computation time.
Metamorphic Adversarial Detection Pipeline for Face Recognition Systems
Authors: Rohan Reddy Mekala, Sai Yerramreddy, Adam Porter
Date: 2021-11-22 00:00:00
Proceeding: AAAI Workshop AdvML 2022 2
Tasks: Adversarial Attack, Face Recognition, image-classification, Image Classification, object-detection, Object Detection, Semantic Similarity, Semantic Textual Similarity, Sentiment Analysis

Adversarial examples pose a serious threat to the robustness of machine learning models in general and deep learning models in particular. Computer vision tasks like image classification, facial recognition, object detection, etc. and natural language processing tasks like sentiment analysis and semantic similarity assessment have all been proven vulnerable to adversarial attacks. For computer vision tasks specifically, these carefully crafted perturbations to input images can cause targeted misclassifications to a label of the attacker's choice, without the perturbations being detectable to the naked eye. A particular class of adversarial attacks called black box attacks can be used to fool a model under attack despite not having access to the model parameters or input datasets used to train the model. As part of the research presented in this paper, we first deploy a range of state of the art adversarial attacks against multiple face recognition pipelines trained in a black box setup, and then generate pair-wise adversarial image sets to deceive the corresponding models under attack. Consequently, we propose a novel approach for adversarial detection that  utilizes statistical techniques to learn optimal thresholds of separation between clean and adversarial examples; achieving state of the art detection accuracies of over ~90%. Our proposed method has been exhaustively tested on multiple face recognition models under attack and adversarial attack type combinations with encouraging results.
Reinforcement Learning with General LTL Objectives is Intractable
Authors: Cambridge Yang, Michael Littman, Michael Carbin
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In recent years, researchers have made significant progress in devising reinforcement-learning algorithms for optimizing linear temporal logic (LTL) objectives and LTL-like objectives.
Despite these advancements, there are fundamental limitations to how well this problem can be solved that previous studies have alluded to but, to our knowledge, have not examined in depth.
In this paper, we address theoretically the hardness of learning with general LTL objectives.
We formalize the problem under the probably approximately correct learning in Markov decision processes (PAC-MDP) framework, a standard framework for measuring sample complexity in reinforcement learning.
In this formalization, we prove that the optimal policy for any LTL formula is PAC-MDP-learnable only if the formula is in the most limited class in the LTL hierarchy, consisting of only finite-horizon-decidable properties.
Practically, our result implies that it is impossible for a reinforcement-learning algorithm to obtain a PAC-MDP guarantee on the performance of its learned policy after finitely many interactions with an unconstrained environment for non-finite-horizon-decidable LTL objectives.
A Brain-Inspired Hierarchical Reasoning Framework for Cognition-Augmented Prosthetic Grasping
Authors: Laura Isabel Galindez Olascoaga, Alisha Menon, Mohamed Ibrahim, Jan Rabaey
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2

This paper proposes a hierarchical framework that enables reasoning across multiple levels of abstraction, from perception to high-level control. The framework relies primarily on Hyperdimensional (HD) computing, a versatile brain-inspired computing paradigm that can produce composite distributed representations for efficient classification and can also facilitate symbolic reasoning. We present this framework through an example use case corresponding to cognition-augmented prosthetic grasping, where user intent is probabilistically predicted to aid the user in successfully selecting and executing the most appropriate gripping mode. Overall, this paper aims to illustrate how HD computing can constitute a  mathematical formalism capable of integrating various levels of cognition under a common hierarchical framework.
FOLD-R++: A Toolset for Automated Inductive Learning of Default Theories from Mixed Data
Authors: Huaduo Wang, Gopal Gupta
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2
Tasks: feature selection, Inductive Learning

FOLD-R is an automated inductive learning algorithm for learning default rules with exceptions for mixed (numerical and categorical) data. It generates an (explainable) answer set programming (ASP) rule set for classification tasks. We present an improved FOLD-R algorithm, called FOLD- R++, that significantly increases the efficiency and scalability of FOLD-R. FOLD-R++ improves upon FOLD-R without compromising or losing information in the input training data during the encoding or feature selection phase. The FOLD-R++ algorithm is competitive in performance with the widely-used XGBoost algorithm, however, unlike XGBoost, the FOLD-R++ algorithm produces an explainable model. Next, we create a powerful tool-set by combining FOLD- R++ with s(CASP)—a goal-directed ASP execution engine— to make predictions on new data samples using the answer set program generated by FOLD-R++. The s(CASP) system also produces a justification for the prediction. Experiments presented in this paper show that our improved FOLD-R++ algorithm is a significant improvement over the original design and that the s(CASP) system can make predictions in an efficient manner as well.
NeuroBE: NN Approximations to Bucket Elimination
Authors: Sakshi Agarwal, Kalev Kask, Alexander Ihler, Rina Dechter
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2

A major limiting factor in graphical model inference is the complexity of computing the partition function. Exact message-passing algorithms such as Bucket Elimination (BE) require exponentially high levels of memory to evaluate the partition function, therefore approximations are often investigated. In this paper, we build upon one such recently introduced methodology called Deep Bucket Elimination (DBE) that uses classical Neural Networks (NNs) to approximate messages generated by BE in cases when buckets have large memory requirements. In the new scheme, called NeuroBE, we personalize the construction and learning of these NNs by utilizing prior information about message size and distribution. Our experiments demonstrate that this new scheme provides significant improvements over DBE, yielding more accurate estimates with less time. We also study the impact of the local errors in message approximations on the global accuracy of the estimand of the partition function.
Automatic Generation of Neural Architecture Search Spaces
Authors: David Calhas, Vasco M. Manquinho, Ines Lynce
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2
Tasks: Neural Architecture Search

Neural Architecture Search (NAS) is receiving growing attention as the need to remove the human bias from neural network models rises. There is extensive research in trying to beat state-of-the-art NAS algorithms. However, these advances do not focus directly on the search space these algorithms explore. Here, we propose a framework that encodes the structure of a convolutional neural network, respecting the arithmetical relation of the kernel and stride sizes with the input and output shapes. This framework consists of a formula with constraints that, if given the structure of the problem (input and output shapes), can produce specification properties of a neural architecture through a solver. We show that this methodology can assemble networks with arbitrary sizes and structures, that make for unique and uniform search spaces. To compare the resulting architectures, a metric that computes dissimilarity in terms of architectural structure is proposed. We empirically show that generating dissimilar architectures, implies dissimilarities in performance. In accordance, similar architectures are similar in performance.
A Confidence-Based Interface for Neuro-Symbolic Visual Question Answering
Authors: Thomas Eiter, Nelson Nicolas Higuera, Johannes Oetsch, Michael Pritz
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2
Tasks: Question Answering, Translation, Visual Question Answering, Visual Question Answering (VQA)

We present a neuro-symbolic visual question answering (VQA) approach for the CLEVR dataset that is based on the combination of deep neural networks and answer-set programming (ASP), a logic-based paradigm for declarative problem solving. We provide a translation mechanism for the questions included in CLEVR to ASP programs. By exploiting choice rules, we consider deterministic and non-deterministic scene encodings. In addition, we introduce a confidence-based interface between the ASP module and the neural network which allows us to restrict the non-determinism to objects classified by the network with high confidence. Our experiments show that the non-deterministic scene encoding achieves good results even if the neural networks are trained rather poorly in comparison with the deterministic approach. This is important for building robust VQA systems if network predictions are less-than perfect.
A Hybrid Neuro-Symbolic approach for Text-Based Games using Inductive Logic Programming
Authors: Kinjal Basu, Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Kartik Talamadupula, Tim Klinger, Murray Campbell, Mrinmaya Sachan, Gopal Gupta
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2
Tasks: Inductive logic programming, Natural Language Understanding, Reinforcement Learning (RL), text-based games

Text-based games (TBGs) have emerged as an important test-bed, requiring reinforcement learning (RL) agents to combine natural language understanding with reasoning. A key challenge for agents solving this task is to generalize across multiple games and shows good results on both seen and unseen objects. Currently, pure deep learning-based RL systems can perform well to known entities and states. They, however, perform poorly in novel situations e.g., when handling out-of-vocabulary (OOV) objects. In the perspective of generalization, recent efforts in infusing external commonsense knowledge into an RL agent show better results than pure deep-learning systems. However, the policies learned by these systems are not interpretable or easily transferable. To tackle these issues, we have designed a hybrid neuro-symbolic framework for TBGs that uses symbolic reasoning along with the neural RL model. It employs inductive logic programming (ILP) to learn the symbolic rules (policies) as default theory with exceptions and is represented in the form of an answer-set-program (ASP) that allows performing non-monotonic reasoning in the partially observable game environment. We use WordNet as an external knowledge source to lift the learned rules to their generalized versions. These rules are learned in an online manner and applied with an ASP solver to predict an action for the agent.  We show that the agents that incorporate the neuro-symbolic hybrid approach with the generalized rules outperform the baseline agents.
Imposing Hard Logical Constraints on Multi-label Classification Neural Networks
Authors: Eleonora Giunchiglia, Thomas Lukasiewicz
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2
Tasks: Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Machine Learning, and in particular deep learning, is becoming increasingly ubiquitous, and it is likely to be applied in almost every aspect of our lives in the next few years.  However, the careless application of such methods in the real world can have, and has already had, disastrous consequences. In order to avoid such undesirable and potentially dangerous scenarios, a standard approach is to formally specify the desired behavior of the system and then ensure
its compliancy to the specified properties.  

In this paper,  we thus propose to enhance deep learning models by incorporating background knowledge as hard logical constraints. The constraints rule out the models' undesired behaviors and can be exploited to gain better performance. In order to achieve the above, we propose CCN($h$), a novel model for multi-label classification problems with hard constraints expressed as normal logic rules. Given any multi-label classification neural network $h$, CCN($h$) is able to exploit the information expressed by the constraints to: (i) produce predictions that are guaranteed to satisfy the constraints, and (ii) improve the performances. We conduct an extensive experimental analysis showing the superior performance of CCN($h$) when compared to state-of-the-art models in the setting of multi-label classification problems with hard logical constraints.
On the Explainability of Convolutional Layers for Multi-Class Problems
Authors: Joe Townsend, Mateusz Kudla, Agnieszka Raszkowska, Theodoros Kasiousmis
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2

Neuro-symbolic reasoning systems support the goal of making the behaviour of trained neural networks more explainable. ERIC and SRAE are two such methods for CNNs that are similar in that they both provide decompositional, layer-wise explanations that can be extracted post-hoc and deployed as classifiers in their own right. However the two methods differ in how they represent knowledge and reason over those representations; ERIC reduces the layer's behaviour to a discrete logic program for symbolic reasoning over a vocabulary at most as large as the number of kernels in that layer; and SRAE reduces the layer's output to more limited but concise vocabulary represented by a set of continuous, orthogonal and sparse features. We compare both methods and show that despite these differences they yield similar results with respect to fidelity when deployed as approximations of the original CNN. SRAE offers marginally stronger fidelity than ERIC but in sacrificing some fidelity ERIC is able to offer a larger and more discrete set of symbols that more closely match what individual kernels actually see. Neither method has previously been demonstrated on multi-class problems but we show for the first time that under certain conditions they may yield high fidelity in such cases. However for both methods fidelity drops for those multi-class datasets in which images have less distinct edges. Similar results under different representations suggest challenges for layer-wise knowledge extraction in general and invite further investigation from the neuro-symbolic community, with our results offering an early benchmark for such research.
Post-processing Counterexample-guided Fairness Guarantees in Neural Networks
Authors: Kiarash Mohammadi, Aishwarya Sivaraman, Golnoosh Farnadi
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2
Tasks: Fairness

There is an increasing interest in adopting high-capacity machine learning models such as deep neural networks to semi-automate human decisions. Hence, it is crucial that these models guarantee similar decisions for similar individuals. To ensure such a fair decision, it is necessary to construct tools capable of verifying and enforcing fairness constraints. In this work, we propose methods to guarantee fairness of a neural network via verification using mixed-integer programming. We show, that it is possible to guarantee individual fair prediction without intervening in the model, efficiently and with little to no loss in accuracy.
Compositional Generalization and Neuro-Symbolic Architectures
Authors: Savitha Sam Abraham, Marjan Alirezaie
Date: 2021-11-21 00:00:00
Proceeding: AAAI Workshop CLeaR 2022 2

Compositional generalization is the ability to understand novel combinations of known concepts. Although it is considered as an innate skill for humans, recent studies have shown that neural networks lack this characteristic. In this paper, we focus on compositional generalization with respect to the two specific tasks of word problem solving and visual relation recognition and propose a neuro-symbolic solution, using DeepProbLog, that addresses the problem of compositionality in state-of-the-art neural systems for these tasks.
DeepDrug: A General Graph-Based Deep Learning Framework for Drug Relation Prediction
Authors: Xusheng Cao, Rui Fan, Wanwen Zeng
Date: 2022-02-10 00:00:00
Proceeding: Bioarxiv 2022 2
Tasks: Relation, Relation Prediction

Computational approaches for accurate predictions of drug-related interactions such as drug-drug interactions (DDIs) and drug-target interactions (DTIs) are highly demanding for biochemical researchers due to their efficiency and cost-effectiveness. Despite the fact that many methods have been proposed and developed to predict DDIs and DTIs respectively, their success is still limited due to a lack of systematic evaluation of the intrinsic properties embedded in their structure. In this paper, we develop a deep learning framework, named DeepDrug, to overcome these shortcomings by using graph convolutional networks to learn the graphical representations of drugs and proteins such as molecular fingerprints and residual structures in order to boost the prediction accuracy. We benchmark our methods in binary-class DDIs, multi-class DDIs and binary-class DTIs classification tasks using several datasets. We then demonstrate that DeepDrug outperforms other state-of-the-art published methods both in terms of accuracy and robustness in predicting DDIs and DTIs with varying ratios of positive to negative training data. Ultimately, we visualize the structural features learned by DeepDrug, which display compatible and accordant patterns in chemical properties, providing additional evidence to support the strong predictive power of DeepDrug. We believe that DeepDrug is an efficient tool in accurate prediction of DDIs and DTIs and provides a promising path in understanding the underlying mechanism of these biochemical relations. The source code of the DeepDrug can be downloaded from https://github.com/wanwenzeng/deepdrug .
GCN-BMP: Investigating Graph Representation Learning for DDI Prediction Task
Authors: Xin Chen, Xien Liu, Ji Wu
Date: 2020-07-01 00:00:00
Proceeding: Methods 2020 7
Tasks: Graph Representation Learning, Inductive Bias, Prediction, Representation Learning

One drug's pharmacological activity may be changed unexpectedly, owing to the concurrent administration of another drug. It is likely to cause unexpected drug-drug interactions (DDIs). Several machine learning approaches have been proposed to predict the occurrence of DDIs. However, existing approaches are almost dependent heavily on various drug-related features, which may incur noisy inductive bias. To alleviate this problem, we investigate the utilization of the end-to-end graph representation learning for the DDI prediction task. We establish a novel DDI prediction method named GCN-BMP (Graph Convolutional Network with Bond-aware Message Propagation) to conduct an accurate prediction for DDIs. Our experiments on two real-world datasets demonstrate that GCN-BMP can achieve higher performance compared to various baseline approaches. Moreover, in the light of the self-contained attention mechanism in our GCN-BMP, we could find the most vital local atoms that conform to domain knowledge with certain interpretability.
SSI–DDI: Substructure–Substructure Interactions for Drug–Drug Interaction Prediction
Authors: Arnold Nyamabo, Hui Yu, Jian-Yu Shi
Date: 2021-11-07 00:00:00
Proceeding: Briefings in Bioinformatics 2021 11
Tasks: Drug–drug Interaction Extraction, molecular representation

A major concern with co-administration of different drugs is the high risk of interference between their mechanisms of action, known as adverse drug–drug interactions (DDIs), which can cause serious injuries to the organism. Although several computational methods have been proposed for identifying potential adverse DDIs, there is still room for improvement. Existing methods are not explicitly based on the knowledge that DDIs are fundamentally caused by chemical substructure interactions instead of whole drugs’ chemical structures. Furthermore, most of existing methods rely on manually engineered molecular representation, which is limited by the domain expert’s knowledge.We propose substructure–substructure interaction–drug–drug interaction (SSI–DDI), a deep learning framework, which operates directly on the raw molecular graph representations of drugs for richer feature extraction; and, most importantly, breaks the DDI prediction task between two drugs down to identifying pairwise interactions between their respective substructures. SSI–DDI is evaluated on real-world data and improves DDI prediction performance compared to state-of-the-art methods. Source code is freely available at https://github.com/kanz76/SSI-DDI.
Stochastic Autograd
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Stochastic Optimization, Variational Inference

I don't think blog requires an abstract, so I guess you are assuming one of the paper: The successes of deep learning, variational inference, and many other fields have been aided by specialized implementations of reverse-mode automatic differentiation (AD) to compute gradients of mega-dimensional objectives. The AD techniques underlying these tools were designed to compute exact gradients to nu-
merical precision, but modern machine learning models are almost always trained with stochastic gradient descent. Why spend computation and memory on exact (minibatch) gradients only to use them for stochastic optimization? We develop a general framework and approach for randomized automatic differentiation (RAD), which can allow unbiased gradient estimates to be computed with reduced memory in return for variance. We examine limitations of the general approach, and argue that we must leverage problem specific structure to realize benefits. We develop RAD techniques for a variety of simple neural network architectures, and show that for a fixed memory budget, RAD converges in fewer iterations than using a small batch size for feedforward networks, and in a similar number for recurrent networks. We also show that RAD can be applied to scientific computing, and use it to develop a low-memory stochastic gradient method for optimizing the control parameters of a linear reaction-diffusion PDE representing a fission reactor.