# _Pat-in-the-Loop_: Declarative Knowledge for Controlling Neural Networks
## 1 Introduction  
Neural networks are obtaining dazzling successes in natural language processing (NLP). General neural networks learned on terabytes of data are replacing decades of scientific investigations by showing unprecedented performances in a variety of NLP tasks [1]. Hence, systems based on NLP and on neural networks (NLP-NN) are everywhere.  
As a consequence of their success, public opinion is extremely fast in spotting possibly catastrophic, unwanted behavior on deployed NLP-NN systems (see, for example, [2, 3]). As many learned systems [4, 5], NLP-NN systems are also exposed to biased decisions or biased production of utterances. This problem is becoming so important that extensive analyses are performed, for example, for the tricky class of systems for sentiment analysis [6].  
To promptly recover from catastrophic failures, NLP-NN systems should be endowed with the possibility of modifying their behavior by using declarative languages. Deductive teaching is an extremely difficult task even in the human learning process [7, 8]. Active learning techniques [9, 10] can require too many examples and may focus the attention of NLP-NN systems on irrelevant peculiarities of datasets [11]. Usually we do not have the time or budget for human input on every data point, and so need strategies for deciding which data points are the most important for human review. Due to the high costs to obtain human-generated activity data using solutions for which a very limited number of examples with supervised information such as Few-Shot Learning or One-shot learning [12, 13] could be used. But the core issue of these techniques is the unreliable empirical risk minimizer that makes them hard to learn. Understanding the core issue helps categorize different works into data,model and algorithm according to how they solve the core issue using prior knowledge: data augments the supervised experience, model constrains the hypothesis space to be smaller, and algorithm alters the search strategy for the best hypothesis in the given hypothesis space [14]. But this is exactly what we want to fight in favour of approaches that understanding neural networks and trying to control their behavior besides using training examples.  
Looking into NLP-NN systems beyond the dazzling light is becoming an active area [15; 16], since traditional neural network visualization tools are obscure when applied to NLP-NN systems. Heatmaps are powerful tools for visualizing neural networks applied to image interpretation [17]. In fact, heatmaps can visualize how neural network layers treat specific subparts of images. Yet, when applied to NLP-NN systems [18] they are extremely difficult to interpret. For this reason, human involvement with the right interfaces could expedite the efficient labeling of tricky or novel data that a machine can't process, reducing the potential for data-related errors.  
In this paper, we propose _Pat-in-the-Loop_ as a model to include human control in specific NLP-NN systems that exploit syntactic information. The key contributions are: (1) _distributed tree encoders_ that directly exploit parse trees in neural networks; (2) _heat parse trees_ that visualize which parts of parse trees are responsible for the activation of specific neurons (see Figure 1); and (3) a declarative language for controlling the behavior of neural networks.  
Distributed tree encoders allow to produce heat parse trees and developers can explore activation of parse trees for specific decisions to derive rules for correcting system behavior.  
In the following work, we performed a pilot study on question classification where _Pat-in-the-Loop_ showed that human knowledge can be effectively used to control the behavior of a syntactic NLP-NN system.  
In the next section (Section 2) we report the related works about the visualization of neural networks models. Next follow a description of Pat-in-the-Loop works (Section 3) and finally (Section 4), we show the improvements achieved by the proposed model.