A Token-pair Framework for Information Extraction from Dialog Transcripts in S ere TOD Challenge (2022)

Abstract: This paper describes our solution for Sere- TOD Challenge Track 1: Information extraction from dialog transcripts. We propose a token-pair framework to simultaneously identify entity and value mentions and link them into corresponding triples. As entity mentions are usually coreferent, we adopt a baseline model for coreference resolution. We exploit both annotated transcripts and unsupervised dialogs for training. With model ensemble and post-processing strategies, our system significantly outperforms the baseline solution and ranks first in triple f1 and third in entity f1.

## introduction task-oriented dialogs cover a wide range of daily application, such as ordering food, booking tickets, and querying services. with the development of deep learning and natural language processing, ai assistants start to replace human operators in a few basic scenarios. however, correctly extracting key information in complicated contexts and generating human-like yet informative responses remain a challenge for both academia and industry. the seretod 2022 workshop introduces a challenge on mobile customer-service scenario with real-world dialog dataset @xcite . we mainly participate in track 1: information extraction from dialog transcripts, and present our tokenpair framework based solution in this paper. ## background the challenge provides around 100k dialog transcripts between mobile service users and staff, titled as mobilecs dataset, of which 10k are annotated while the rest are unlabeled. the annotation includes service entities and the attributes or values of the service (e.g. package price) or the user (e.g. account balance) mentioned in the dialog. as the dialogs are generally colloquial, co-references are required to be resolved for entity mentions. moreover, values for an entity may scatter in multi-turn dialogs or nested inside the verbal expression of entities. track 1 is mainly formulated as an information extraction problem and contains two sub-tasks: (1) entity extraction, i.e., to extract entity mentions with their corresponding entity types as defined in the schema; (2) slot filling, i.e., to extract values for entity attributes and to match the slot-value pairs with the corresponding entity concepts. f1 score is the metric for system evaluation. ## conclusion and further work we present our solution for information extraction from dialog transcripts in seretod challenge. the system is trained on both annotated transcripts and unsupervised dialogs. various strategies and tricks are employed to further boost system performance, with their effects analyzed and discussed. compared with the baseline implementation, our tokenpair solution not only integrates multiple modules into a unified model framework, but also significantly outperforms the baseline result by more than 20 percent. in the official evaluation results, our system ranks first in triple-f1 and third in ent-f1. for further work, we plan to integrate the coreference resolution model into the token-pair framework. we will evaluate the proposed label denoising methods and expect a well-anotated dataset. detailed settings, such as multi-task weighting, shall also be tuned for better performance.