Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extraction
Abstract
Efficiently deriving structured workflows from unannotated dialogs remains an underexplored and formidable challenge in computational linguistics. Automating this process could significantly accelerate the manual design of workflows in new domains and enable the grounding of large language models in domain-specific flowcharts, enhancing transparency and controllability. In this paper, we introduce Dialog2Flow (D2F) embeddings, which differ from conventional sentence embeddings by mapping utterances to a latent space where they are grouped according to their communicative and informative functions (i.e., the actions they represent). D2F allows for modeling dialogs as continuous trajectories in a latent space with distinct action-related regions. By clustering D2F embeddings, the latent space is quantized, and dialogs can be converted into sequences of region/action IDs, facilitating the extraction of the underlying workflow. To pre-train D2F, we build a comprehensive dataset by unifying twenty task-oriented dialog datasets with normalized per-turn action annotations. We also introduce a novel soft contrastive loss that leverages the semantic information of these actions to guide the representation learning process, showing superior performance compared to standard supervised contrastive loss. Evaluation against various sentence embeddings, including dialog-specific ones, demonstrates that D2F yields superior qualitative and quantitative results across diverse domains.
Community
(paper accepted to EMNLP 2024 main conference)
This paper introduces sentence embedding models pre-trained to automatically extract the dialog flow from a collection of dialogs (i.e. to "discover" the sequences of “communicational steps" that represent the collection).
Besides the embeddings the other two main contributions of the papers are:
- The corpus built to train the models.
- A novel supervised soft-contrastive loss useful when working with many labels (e.g thousands) that could be semantically related.
Link to Hugging Face collection containing the models and the dataset available here.
Github repo will be also available within the next two weeks, before being presented at the EMNLP main conference, here: https://github.com/idiap/dialog2flow (undergoing internal paperwork mandatory before publishing).
This repo will contain, among other things, script to convert any given collection of dialogs to its representative dialog flow.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- An Efficient Self-Learning Framework For Interactive Spoken Dialog Systems (2024)
- Ruri: Japanese General Text Embeddings (2024)
- DFlow: Diverse Dialogue Flow Simulation with Large Language Models (2024)
- Making Text Embedders Few-Shot Learners (2024)
- Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 4
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper