pwc-archive/papers-with-abstracts huggingface dataset:
TimeSHAP: Explaining Recurrent Models through Sequence Perturbations
Authors: Anonymous
Date: 2020-10-16 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Decision Making, Feature Importance, Time Series, Time Series Analysis, Time Series Classification

Recurrent neural networks are a standard building block in numerous machine learning domains, from natural language processing to time-series classification. While their application has grown ubiquitous, understanding of their inner workings is still lacking.

In practice, the complex decision-making in these models is seen as a black-box, creating a tension between accuracy and interpretability. Moreover, the ability to understand the reasoning process of a model is important in order to debug it and, even more so, to build trust in its decisions.

Although considerable research effort has been guided towards explaining black-box models in recent years, recurrent models have received relatively little attention. Any method that aims to explain decisions from a sequence of instances should assess, not only feature importance, but also event importance, an ability that is missing from state-of-the-art explainers.

In this work, we contribute to filling these gaps by presenting TimeSHAP, a model-agnostic recurrent explainer that leverages KernelSHAP's sound theoretical footing and strong empirical results. As the input sequence may be arbitrarily long, we further propose a pruning method that is shown to dramatically improve its efficiency in practice.
The Reverse Turing Test for Evaluating Interpretability Methods on Unknown Tasks
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Prediction

The Turing Test evaluates a computer program’s ability to mimic human behaviour. The Reverse Turing Test, reversely, evaluates a human’s ability to mimic machine behaviour in a forward prediction task. We propose to use the Reverse Turing Test to evaluate the quality of interpretability methods. The Reverse Turing Test improves on previous experimental protocols for human evaluation of interpretability methods by a) including a training phase, and b) masking the task, which, combined, enables us to evaluate models independently of their quality, in a way that is unbiased by the participants' previous exposure to the task. We present a human evaluation of LIME across five NLP tasks in a Latin Square design and analyze the effect of masking the task in forward prediction experiments. Additionally, we demonstrate a fundamental limitation of LIME and show how this limitation is detrimental for human forward prediction in some NLP tasks.
Accounting for Human Learning when Inferring Human Preferences
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12

Inverse reinforcement learning (IRL) is a common technique for inferring human preferences from data. Standard IRL techniques tend to assume that the human demonstrator is stationary, that is that their policy $\pi$ doesn't change over time. In practice, humans interacting with a novel environment or performing well on a novel task will change their demonstrations as they learn more about the environment or task. We investigate the consequences of relaxing this assumption of stationarity, in particular by modelling the human as learning. Surprisingly, we find in some small examples that this can lead to better inference than if the human was stationary. That is, by observing a demonstrator who is themselves learning, a machine can infer \emph{more} than by observing a demonstrator who is noisily rational. In addition, we find evidence that misspecification can lead to poor inference, suggesting that modelling human learning is important, especially when the human is facing an unfamiliar environment.
On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Active Learning, Open-Domain Question Answering, QQP, Question Answering

Many pairwise classification tasks, such as paraphrase detection and open-domain question answering, naturally have extreme label imbalance (e.g., $99.99\%$ of examples are negatives). In contrast, many recent datasets heuristically choose examples to ensure label balance. We show that these heuristics lead to trained models that generalize poorly: State-of-the art models trained on QQP and WikiQA each have only $2.4\%$ average precision when evaluated on realistically imbalanced test data. We instead collect training data with active learning, using a BERT-based embedding model to efficiently retrieve uncertain points from a very large pool of unlabeled utterance pairs. By creating balanced training data with more informative negative examples, active learning greatly improves average precision to $32.5\%$ on QQP and $20.1\%$ on WikiQA.
Teaching the Machine to Explain Itself using Domain Knowledge
Authors: Anonymous
Date: 2020-10-16 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Decision Making, Fraud Detection

Machine Learning (ML) has been increasingly used to aid humans to make better and faster decisions.
However, non-technical humans-in-the-loop struggle to comprehend the rationale behind model predictions, hindering trust in algorithmic decision-making systems. Considerable research work on AI explainability attempts to win back trust in AI systems by developing explanation methods but there is still no major breakthrough. At the same time, popular explanation methods (\eg, LIME, and SHAP) produce explanations that are very hard to understand for non-data scientist persona.
To address this, we present JOEL, a neural network-based framework to jointly learn a decision-making task and associated explanations that convey domain knowledge. JOEL is tailored to human-in-the-loop domain experts that lack deep technical ML knowledge, providing high-level insights about the model's predictions that very much resemble the experts' own reasoning. Moreover, we collect the domain feedback from a pool of certified experts and use it to ameliorate the model (human teaching), hence promoting seamless and better suited explanations.
Lastly, we resort to semantic mappings between legacy expert systems and domain taxonomies to automatically annotate a bootstrap training set, overcoming the absence of concept-based human annotations. 
We validate JOEL empirically on a real-world fraud detection dataset. We show that JOEL can generalize the explanations from the bootstrap dataset. Furthermore, obtained results indicate that human teaching can further improve the explanations prediction quality by approximately $13.57\%$.
Conceptualization and Framework of Hybrid Intelligence Systems
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Fairness

As artificial intelligence (AI) systems are getting ubiquitous within our society, issues related to its fairness, accountability, and transparency are increasing rapidly. As a result, researchers are integrating humans with AI systems to build robust and reliable hybrid intelligence systems. However, a proper conceptualization of these systems does not underpin this rapid growth. This article provides a precise definition of hybrid intelligence systems as well as explains its relation with other similar concepts through our proposed framework and examples from contemporary literature. The framework breakdowns the relationship between a human and a machine in terms of the degree of coupling and the directive authority of each party. Finally, we argue that all AI systems are hybrid intelligence systems, so human factors need to be examined at every stage of such systems' lifecycle.
Robust Active Learning Strategies for Model Variability
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Active Learning, model

Active learning methods are useful when a limited budget for data labelling is available. However, the most widely used methods -- uncertainty sampling -- may suffer from problems derived from an excessive dependence on the model learned during data acquisition.
This results in datasets which are not optimal when they are used to train models very different from those used during data creation.
In this paper, we link this to the tendency of uncertainty sampling to select outliers and show that other methods that favour selection of representative sampling are more robust to changes in models.
We validate this experimentally on four NLP datasets.
The Design and Development of Games with a Purpose for AI Systems
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: BIG-bench Machine Learning

Collecting training data is often time-consuming, expensive and imposes a bottleneck on many machine learning tasks. Much of training data used to train ML systems is a result of the work of crowdworkers who are paid to do routinizable mundane tasks. Games with a purpose leverage game mechanics to use the perceptive capacities of users to collect data in a way that is more enjoyable to crowdworkers. Different machine learning tasks require different types of training data. In this paper, we discuss the design and development of building two games with a purpose: Guess the Word and Fool the AI, designed to collect data from both crowdworkers and domain experts for two very different machine learning problems. To make these games enjoyable and interactive, a team of engineers, research scientists and designers create new games with a purpose around various machine learning tasks. In this paper, we describe the design of these games, how we incorporate game mechanics within these games to make the collection of annotation tasks more efficient but also enjoyable.
MEME: Generating RNN Model Explanations via Model Extraction
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Decision Making, model, Model extraction, Mortality Prediction, Occupation prediction, Prediction

Recurrent Neural Networks (RNNs) have achieved remarkable performance on a range of tasks. A key step to further empowering RNN-based approaches is improving their explainability and interpretability. In this work we present MEME: a model extraction approach capable of approximating RNNs with interpretable models represented by human-understandable concepts and their interactions. We demonstrate how MEME can be applied to two multivariate, continuous data case studies: Room Occupation Prediction, and In-Hospital Mortality Prediction. Using these case-studies, we show how our extracted models can be used to interpret RNNs both locally and globally, by approximating RNN decision-making via interpretable concept interactions.
NetReAct: Interactive Learning for Network Summarization
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12

 Generating useful network summaries is a challenging and important problem with several applications like sensemaking, visualization, and compression. However, most current work in this space does not take human feedback into account while generating summaries. Consider an intelligence analysis scenario, where the analyst is exploring a similarity network between documents.  The analyst can express her agreement/disagreement with the visualization of the network summary via iterative feedback, e.g. closing or moving documents (``nodes'') together. How can we use this feedback to improve the network summary quality? In this paper, we present NetReact, a novel interactive network summarization algorithm which supports the visualization of networks induced by text corpora to perform sensemaking. NetReact incorporates human feedback with reinforcement learning to summarize and visualize document networks. It first summarizes the network by grouping relevant nodes, and then lays them out in groups in which spatial proximity is mapped to group similarity. Using scenarios from two datasets, we show how NetReact is successful in generating high-quality summaries and visualizations that reveal hidden patterns better than other non-trivial baselines.
Explanation Augmented Feedback in Human-in-the-Loop Reinforcement Learning
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Atari Games, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Human-in-the-loop Reinforcement Learning (HRL) aims to integrate human guidance with Reinforcement Learning (RL) algorithms to improve sample efficiency and performance. A common type of human guidance in HRL is binary evaluative "good" or "bad" feedback for queried states and actions. However, this type of learning scheme suffers from the problems of weak supervision and poor efficiency in leveraging human feedback. To address this, we present EXPAND (EXPlanation AugmeNted feeDback) which provides a visual explanation in the form of saliency maps from humans in addition to the binary feedback. EXPAND employs a state perturbation approach based on salient information in the state to augment the binary feedback. We choose five tasks, namely Pixel-Taxi and four Atari games, to evaluate this approach. We demonstrate the effectiveness of our method using two metrics: environment sample efficiency and human feedback sample efficiency. We show that our method significantly outperforms previous methods. We also analyze the results qualitatively by visualizing the agent's attention. Finally, we present an ablation study to confirm our hypothesis that augmenting binary feedback with state salient information results in a boost in performance.
What more can Entity Linking do for Question Answering?
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: coreference-resolution, Coreference Resolution, Entity Linking, Question Answering

We introduce a new NLP task–noun phrase linking (NPL)–which is a subset of entity linking and expands named entity linking (NEL) to link all noun phrases in a document to an external knowledge base. Our task is an expansion of NEL by linking not only named entities, but also references to named entities, and is distinct from coreference resolution in that references to unmentioned entities are also linked. Not only is this task more difficult, but performing well on this task would provide benefits to downstream systems, such as Question Answering systems (QA), which use entity linkers to assist with answering questions. By replacing these entity linkers with noun phrase linkers, the QA systems have more information, while shifting some of the difficulty of question answering to designing a good noun phrase linker. Our primary contribution is the introduction of the noun phrase linking task. To introduce NPL, we plan to collect an evaluation set based on annotating several QA datasets which we then use to compare NPL models, and estimate their effectiveness in improving end-to-end QA accuracy. This new entity linking task is more difficult than traditional entity linking, because of the difficulty connecting implicit references to named entities, and so requires a method to efficiently collect data. Our second contribution is that we develop an efficient method to collect annotation data by motivating domain experts to annotate and using human-in-the-loop annotation to assist annotators. Data collection is efficiently done by guiding human annotators towards examples where multiple entity linking models disagreed while maintaining accuracy on a gold set. We propose experiments to evaluate the effect of noun phrase linking on question answering systems, and also compare our new noun phrase linking systems against baseline coreference and entity linking systems. In summary, we introduce NPL, demonstrate a method to efficiently collect data, and propose experiments.
Human-in-the-loop solution for scoring economic development from geospatial data
Authors: Anonymous
Date: 2020-10-16 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Humanitarian

Reliable and timely measurements of economic activities are fundamental for understanding economic development and for delivering humanitarian aid and disaster relief to where needed. However, many developing countries still lack reliable data. This paper introduces a novel approach for measuring economic development from high-resolution satellite images in the absence of ground truth statistics. Our method's novelty is that we break down a computationally hard problem into sub-tasks, which involves a human-in-the-loop solution. With the combination of unsupervised learning and the partial orders of dozens of urban versus rural clusters, our method can estimate the economic development scores of over 10,000 satellite grids with less human labor than other baseline approaches (Spearman correlation of 0.851). We demonstrate how to apply our method to both developed and developing economies.
Local vs. Global interpretations for NLP
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Feature Importance, text-classification, Text Classification, valid

Recently, WordsWorth scores have been proposed for calculating feature importance in the context of traditional deep learning models trained for text classification tasks. Here, we experiment with the idea behind these scores and present them as a global explanation for a trained model. Interpretability literature shows that delete one method acts as a good explanation for NLP tasks. Since WW scores act as a good proxy to delete one scores for text classification, we extend the argument and utilize them for interpretation. We provide local and global explanations for a CNN trained on the IMDB reviews dataset by comparing these scores with LIME. Similarly to LIME, the global representation is a bag of words representation. Overall, we argue that evaluating a trained neural network on single words, at all possible locations in the input text one by one, gives powerful and valid insights into the workings of these otherwise black box models. This is a work in progress and we are looking for further tests to evaluate the usefulness of our method.
A Closed-Loop System for Improving Annotation Quality and Efficiency
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Autonomous Driving, Instance Segmentation, Semantic Segmentation

We present a general system and approach to improve the quality and efficiency of interactive annotation. A specific use case based on instance segmentation of vehicles for autonomous driving is used as an illustration.  Via incremental AB testing and a custom analytics pipeline, we show how to optimize human-ML interaction to systematically improve annotation efficiency, and address the shortcomings of ML models.
Training Shallow Phrase Structure Parsers From Semantic Frames and Simple NLG
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Natural Language Understanding, Text Generation

  Determining the meaning of customer utterances is an important part
  of fulfilling customer requests in task-oriented dialogue.
  Natural Language Understanding (NLU) models can determine this meaning,
  but typically require many customer utterances that are hand-annotated with
  meaning representations, which are difficult to obtain and must be
  repeated for each new target domain.  One way to reduce the labor
  involved in hand annotation is to have the human annotate a meaning
  representation (a ``semantic frame'' representation) separate from the
  corresponding utterance.  In this work, we investigate the use of this
  approach in conjunction with several simple natural language
  generation (NLG) approaches in order to train shallow parsers
  to extract phrase structure representations from customer
  utterances.  Our results show the effectivness of this approach for
  training NLU models.
Intent Discovery With Or Without Labeled Data Using Dependency Parser
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Clustering, Intent Discovery

In dialogue applications, machine learning classification models are often used to classify user utterances into different intents that help to understand the users. 
In real world scenarios, however, some utterances may not belong to any of the anticipated intent categories.
Furthermore, supervised classification models are not a viable solution when data of a new domain is introduced without the corresponding labels.
In this work, we present a clustering and evaluation approach that can be used in semi-supervised or unsupervised modes, depending on the (non-)availability of training data for new intent discovery.
This method assigns meaningful intent-labels by determining the optimal number of clusters and evaluating the performance of the clustering results. In addition, it assigns a TF-IDF score to individual samples within a cluster.
Machine learning for industrial safety culture in the developing world
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: BIG-bench Machine Learning, Cultural Vocal Bursts Intensity Prediction

A disproportionate number of deaths and injuries in the workplace are concentrated
in the developing world due to inadequate enforcement capabilities in such areas,
a lack of capital equipment and a paucity of expertise. This requires solutions
that enable automated detection of harmful violations of safety standards and
sending appropriate warnings at real-time. We propose an automated method for
highlighting such violations of safety measures by foregoing the usage of safety
helmets, hard hats and similar safety equipment in the factory setting. A pipeline is
proposed wherein camera feed at real-time is processed to detect the safety critical
objects and their placement using a set of deep learning based architectures tailored
for detection and localization under compute-constrained environments, with the
helmet compliance sub-task treated as an exemplar of our approach.
Police-In-The-Loop Person Re-identification
Authors: Anonymous
Date: 2020-10-15 00:00:00
Proceeding: NeurIPS Workshop HAMLETS 2020 12
Tasks: Clustering, Person Re-Identification, Re-Ranking

Person re-identification (Re-ID) has gained significant interest in the computer vision community. One important application of Re-ID is the analysis of CCTV footages for police investigations. This setting poses a challenge of identifying all the appearances of a given identity on time while also providing an opportunity to utilize feedback from a human investigator. In this paper, we propose a general framework for building human-in-the-loop (HITL) Re-ID systems for criminal investigations. Using this framework, we investigate various HITL re-ranking methods based on relevance feedback and hierarchical clustering. Quantitative evaluations on large scale datasets show significant improvements in Re-ID performance. Code is available at \url{https://}.