  A question answering (QA) system is a type of conversational AI that
generates natural language answers to questions posed by human users. QA
systems often form the backbone of interactive dialogue systems, and have been
studied extensively for a wide variety of tasks ranging from restaurant
recommendations to medical diagnostics. Dramatic progress has been made in
recent years, especially from the use of encoder-decoder neural architectures
trained with big data input. In this paper, we take initial steps to bringing
state-of-the-art neural QA technologies to Software Engineering applications by
designing a context-based QA system for basic questions about subroutines. We
curate a training dataset of 10.9 million question/context/answer tuples based
on rules we extract from recent empirical studies. Then, we train a custom
neural QA model with this dataset and evaluate the model in a study with
professional programmers. We demonstrate the strengths and weaknesses of the
system, and lay the groundwork for its use in eventual dialogue systems for
software engineering.
