Large Language Models as Zero-shot Dialogue State Tracker through
  Function Calling
Authors: name: Zekun Li; name: Zhiyu Zoey Chen; name: Mike Ross; name: Patrick Huber; name: Seungwhan Moon; name: Zhaojiang Lin; name: Xin Luna Dong; name: Adithya Sagar; name: Xifeng Yan; name: Paul A. Crook
Published: 2024-02-16 06:13:18

Abstract: Large language models (LLMs) are increasingly prevalent in conversational
systems due to their advanced understanding and generative capabilities in
general contexts. However, their effectiveness in task-oriented dialogues
(TOD), which requires not only response generation but also effective dialogue
state tracking (DST) within specific tasks and domains, remains less
satisfying. In this work, we propose a novel approach FnCTOD for solving DST
with LLMs through function calling. This method improves zero-shot DST,
allowing adaptation to diverse domains without extensive data collection or
model tuning. Our experimental results demonstrate that our approach achieves
exceptional performance with both modestly sized open-source and also
proprietary LLMs: with in-context prompting it enables various 7B or 13B
parameter models to surpass the previous state-of-the-art (SOTA) achieved by
ChatGPT, and improves ChatGPT's performance beating the SOTA by 5.6% Avg. JGA.
Individual model results for GPT-3.5 and GPT-4 are boosted by 4.8% and 14%,
respectively. We also show that by fine-tuning on a small collection of diverse
task-oriented dialogues, we can equip modestly sized models, specifically a 13B
parameter LLaMA2-Chat model, with function-calling capabilities and DST
performance comparable to ChatGPT while maintaining their chat capabilities. We
plan to open-source experimental code and model.