  Our project task was to create a model that, given a speaker ID, chat
history, and an utterance query, can predict the response utterance in a
conversation. The model is personalized for each speaker. This task can be a
useful tool for building speech bots that talk in a human-like manner in a live
conversation. Further, we succeeded at using dense-vector encoding clustering
to be able to retrieve relevant historical dialogue context, a useful strategy
for overcoming the input limitations of neural-based models when predictions
require longer-term references from the dialogue history. In this paper, we
have implemented a state-of-the-art model using pre-training and fine-tuning
techniques built on transformer architecture and multi-headed attention blocks
for the Switchboard corpus. We also show how efficient vector clustering
algorithms can be used for real-time utterance predictions that require no
training and therefore work on offline and encrypted message histories.
