  Reinforcement Learning (RL) is a widely-used technique in many domains,
including autonomous driving, robotics, stock trading, and video games.
Unfortunately, the black box nature of RL agents, combined with increasing
legal and ethical considerations, makes it increasingly important that humans
understand the reasoning behind the actions taken by an RL agent, particularly
in safety-critical domains. To help address this challenge, we introduce
PolicyExplainer, a visual analytics interface which lets the user directly
query an RL agent. PolicyExplainer visualizes the states, policy, and expected
future rewards for an agent, and supports asking and answering questions such
as: "Why take this action? Why not this other action? When is this action
taken?". PolicyExplainer is designed based upon a domain analysis with RL
experts, and is evaluated via empirical assessments on a trio of domains: taxi
navigation, an inventory application, and the safety-critical domain of drug
recommendation for HIV patients.
