  Recent studies show the effectiveness of interview chatbots for information
elicitation. However, designing an effective interview chatbot is non-trivial.
Few tools exist to help designers design, evaluate, and improve an interview
chatbot iteratively. Based on a formative study and literature reviews, we
propose a computational framework for quantifying the performance of interview
chatbots. Incorporating the framework, we have developed iChatProfile, an
assistive chatbot design tool that can automatically generate a profile of an
interview chatbot with quantified performance metrics and offer design
suggestions for improving the chatbot based on such metrics. To validate the
effectiveness of iChatProfile, we designed and conducted a between-subject
study that compared the performance of 10 interview chatbots designed with or
without using iChatProfile. Based on the live chats between the 10 chatbots and
1349 users, our results show that iChatProfile helped the designers build
significantly more effective interview chatbots, improving both interview
quality and user experience.
