  This paper describes our submission for the End-to-end Multi-domain Task
Completion Dialog shared task at the 9th Dialog System Technology Challenge
(DSTC-9). Participants in the shared task build an end-to-end task completion
dialog system which is evaluated by human evaluation and a user simulator based
automatic evaluation. Different from traditional pipelined approaches where
modules are optimized individually and suffer from cascading failure, we
propose an end-to-end dialog system that 1) uses Generative Pretraining 2
(GPT-2) as the backbone to jointly solve Natural Language Understanding, Dialog
State Tracking, and Natural Language Generation tasks, 2) adopts Domain and
Task Adaptive Pretraining to tailor GPT-2 to the dialog domain before
finetuning, 3) utilizes heuristic pre/post-processing rules that greatly
simplify the prediction tasks and improve generalizability, and 4) equips a
fault tolerance module to correct errors and inappropriate responses. Our
proposed method significantly outperforms baselines and ties for first place in
the official evaluation. We make our source code publicly available.
