DaP 2018 Proceedings of the Workshop on Dialogue and Perception

Recent progress in processing raw perceptual data with deep learning techniques has revived interest in studies of emergent communication and interaction in multiagent settings with realistic visual input. Neural-network agents produce however high-dimensional “inner” representations of the input that can’t be directly interpreted. In our ongoing research, we design model-independent probing methods to gain insights into such inner representations, and on how the different tasks that agents are assigned affect their view of the same perceptual information. Our goals are to achieve a better understanding of how tasks affect representations, to make agent behavior more interpretable and, ultimately, to improve interactive agents by providing them with insights into each other’s representation. 1 Background and motivation Simulations in which multiple computational agents learn to communicate in order to solve a task collaboratively or adversarially in a grounded environment have a long history (Lewis, 1969; Briscoe, 2002; Cangelosi and Parisi, 2002; Skyrms, 2019; Steels, 2012, a.o.). However, early work was mostly limited by the technology of its times to simplified, typically hand-crafted smallscale inputs. The amazing success of deep neural networks in processing real-life perceptual data, in particular natural images (Russakovsky et al., 2015), has revitalized the area, with a new generation of studies that consider agents learning to interact when faced with tasks that require processing realistic visual input (e.g., Goodfellow et al., 2014; Lazaridou et al., 2017; Das et al., 2017; Lazaridou et al., 2018; Lee et al., 2017). In these multi-agent setups, agents typically process similar perceptual input, that they must use for different purposes. For example, in the classic referential game of Lazaridou et al. (2017), two players, a Sender and a Receiver, see the same pairs of images, one of them being the “target” image. The Sender sees the images and knows the target, and chooses to send one symbol from a fixed vocabulary to the Receiver. The Receiver sees only the images, receives the sent symbol, and tries to guess which image is the target. In case of success, both players receive a payoff of 1. Since the involved agents are neural networks, during training they produce distributed representations of the inputs, tuned to the task they are learning to solve. However, given the noisy natural input and the fact that these distributed representations are dense and high-dimensional, it is hard to understand what they are encoding. Our current research aims at developing methods to analyze the specifics of the representations of the same perceptual data developed by agents with different tasks. We have three main reasons to focus on this. First, from a cognitive point of view, understanding how different tasks affect perceptual representations might provide insights into how higher-level perception gets tuned to communicative or antagonistic objectives. For example, does categorical perception (Goldstone and Hendrickson, 2010) naturally arise in setups where agents must efficiently discriminate broad natural kinds? Second, as machine learning is tasked with increasingly important functions, being able to understand the representations that algorithms develop of the data they are exposed to is of the utmost importance (Doshi-Velez and Kim, 2017). This is particularly true in multi-agent settings, where the agents might develop their own opaque communication means (Lewis et al., 2017).