Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence
  Embeddings for Automatic Dialog Flow Extraction
Authors: name: Sergio Burdisso, name: Srikanth Madikeri, name: Petr Motlicek

Efficiently deriving structured workflows from unannotated dialogs remains an
underexplored and formidable challenge in computational linguistics. Automating
this process could significantly accelerate the manual design of workflows in
new domains and enable the grounding of large language models in
domain-specific flowcharts, enhancing transparency and controllability. In this
paper, we introduce Dialog2Flow (D2F) embeddings, which differ from
conventional sentence embeddings by mapping utterances to a latent space where
they are grouped according to their communicative and informative functions
(i.e., the actions they represent). D2F allows for modeling dialogs as
continuous trajectories in a latent space with distinct action-related regions.
By clustering D2F embeddings, the latent space is quantized, and dialogs can be
converted into sequences of region/action IDs, facilitating the extraction of
the underlying workflow. To pre-train D2F, we build a comprehensive dataset by
unifying twenty task-oriented dialog datasets with normalized per-turn action
annotations. We also introduce a novel soft contrastive loss that leverages the
semantic information of these actions to guide the representation learning
process, showing superior performance compared to standard supervised
contrastive loss. Evaluation against various sentence embeddings, including
dialog-specific ones, demonstrates that D2F yields superior qualitative and
quantitative results across diverse domains.