  This paper is preoccupied with the following question: given a (possibly
opaque) learning system, how can we understand whether its behaviour adheres to
governance constraints? The answer can be quite simple: we just need to "ask"
the system about it. We propose to construct an investigator agent to query a
learning agent -- the suspect agent -- to investigate its adherence to a given
ethical policy in the context of an information-seeking dialogue, modeled in
formal argumentation settings. This formal dialogue framework is the main
contribution of this paper. Through it, we break down compliance checking
mechanisms into three modular components, each of which can be tailored to
various needs in a vast amount of ways: an investigator agent, a suspect agent,
and an acceptance protocol determining whether the responses of the suspect
agent comply with the policy. This acceptance protocol presents a fundamentally
different approach to aggregation: rather than using quantitative methods to
deal with the non-determinism of a learning system, we leverage the use of
argumentation semantics to investigate the notion of properties holding
consistently. Overall, we argue that the introduced formal dialogue framework
opens many avenues both in the area of compliance checking and in the analysis
of properties of opaque systems.
