SlowGuess/NAACL_2022_OCR huggingface dataset:
We report the domain-wise results for SMD and MWOZ in Table 8 and Table 9 respectively. Baseline model's results are reported from (Raghu et al., 2021) and (Madotto et al., 2020). The MWOZ dialogue dataset contains conversations on the following domains as reported in the baseline works: attraction, restaurant, and hotel. The domain-wise results demonstrate that DialogoKG achieves improved performance in almost all domains in a multi-domain setup. This demonstrates DialogoKG's capacity to handle a dynamic knowledge base.
Models	BLEU	MoverScore	Entity F1	Schedule	Navigate	Weather
GLMP (Wu et al., 2019)	13.9	54.2	59.6	72.5	54.6	56.5
MLM (Gangi Reddy et al., 2019)	17.0	64.0	54.6	66.7	46.9	56.0
Ent. Const. (Qin et al., 2019)	13.9	53.8	53.7	55.6	54.5	52.2
GPT2+KE (Madotto et al., 2020)	17.4	66.4	59.8	72.6	53.5	57.7
TTOS (He et al., 2020a)	17.4	59.8	55.4	63.5	45.9	64.1
DF-Net (Qin et al., 2020)	14.4	56.3	62.7	73.1	57.9	57.6
EER (He et al., 2020c)	17.2	60.9	59.0	71.8	52.5	57.8
FG2Seq (He et al., 2020b)	16.8	60.2	61.1	73.3	56.1	57.4
CDNet (Raghu et al., 2021)	17.8	61.1	62.9	75.4	56.7	61.3
DialoKG (Ours)	20.0	70.6	65.9	77.9	58.4	72.7
Table 8: Domain-wise results on SMD dataset.
Models	BLEU	MoverScore	Entity F1	Attraction	Restaurant	Hotel
GLMP (Wu et al., 2019)	6.9	51.2	32.4	24.4	38.4	28.1
MLM (Gangi Reddy et al., 2019)	-	-	-	-	-	-
Ent. Const. (Qin et al., 2019)	-	-	-	-	-	-
GPT2+KE (Madotto et al., 2020)	15.0	60.9	39.6	43.3	37.1	33.4
TTOS (He et al., 2020a)	-	-	-	-	-	-
DF-Net (Qin et al., 2020)	9.4	54.2	35.1	28.1	40.9	30.6
EER (He et al., 2020c)	13.6	57.2	35.6	43.0	34.3	35.7
FG2Seq (He et al., 2020b)	14.6	58.4	36.5	37.2	38.9	34.4
CDNet (Raghu et al., 2021)	11.9	55.8	38.7	38.9	41.7	36.3
DialoKG (Ours)	12.6	62.6	43.5	39.8	46.7	37.9
Table 9: Domain-wise results on MWOZ dataset.
C Knowledge Triples to Sequence Transformation
Figure 8 depicts how we linearize a graph into a sequence. The sequence begins with a [BOS] token, followed by the token [S] and a subject (worth house). The token [S] indicates that the following word in the sequence is a subject (in this case worth house). Then we append all the triples that are connected to the subject worth house where the relation and object is separated by the token [R] and [O], respectively. Similarly, the second subject is appended to the sequence separated by a preceding [S] token.
Figure 9 shows the interface of the annotation tool used to obtain human annotation scores. The interface displays a set of knowledge triples, a user utterance, the ground truth response, and a system-generated response for each point. Given the information displayed on the annotation tool, we asked the annotators to rate the system-generated responses against the ground-truth on a scale of [1,5] (higher is better). We explained the participants about the purpose of this research. The first two participants are male (over 30 years old), and
the third participant is female (more than 35 years old), both with several years of experience in the domain.
E Example System Outputs
We show conversations performed by DialogKG on SMD and MWOZ 2.1 dataset in Figure 10 and Figure 11, respectively. The example conversations demonstrate that DialogKG can perform accurate and engaging conversations.
Figure 8: Illustration of graph to sequence transformation.
[B0S]	[S]	worth house	[R]	stars	[0]	4 star	[R]	type	[0]	guest house	[S]	archway house	[R]	type	[0]	guest house
Knowledge Base
Subject	Relation	Object
chevron	distance	5 miles
chevron	traffic info	moderate traffic
chevron	poi type	gas station
chevron	address	783 arcadia pl
town and country	distance	5 miles
town and country	address	383 university ave
jacks house	poi type	friends house
jacks house	address	864 almanor ln
the clement hotel	traffic info	no traffic
User utterance: What is the address of chevron?
Ground truth: 783 arcadia pl is the address for chevron gas station.
System generated response: chevron is located at 783 arcadia pl.
Naturalness	○ 1	○ 2	○ 3	○ 4	◎ 5
Correctness	○ 1	○ 2	○ 3	○ 4	◎ 5
Figure 9: The interface of the annotation tool to obtained the human annotation scores.
the westin	poi type	rest stop
the westin	address	329 el camino real
the westin	distance	4 miles
the westin	traffic info	no traffic
sigona farmers market	poi type	grocery store
sigona farmers market	address	638 amherst st
sigona farmers market	distance	4 miles
sigona farmers market	traffic info	no traffic
whole foods	distance	2 miles
whole foods	poi type	grocery store
whole foods	traffic info	heavy traffic
stanford shopping center	distance	2 miles
stanford shopping center	poi type	shopping center
stanford shopping center	traffic info	moderate traffic
Turn 1	Q	Give me the directions to the closest grocery store ?
Gold	There are whole foods 2 miles away and sigona farmers market 4 miles away .
DialoKG	The nearest grocery store is whole foods located at 2 miles away .
Turn 2	Q	I need the one that avoids all heavy traffic .
Gold	sigona farmers market is located 4 miles away with no traffic is located at 638 amherst st .
DialoKG	There is heavy traffic on our way to the nearest grocery store, but I could find another one nearby.
Turn 3	Q	Thank you .
Gold	Great, glad I could be help. Have a great day we will be there soon .
DialoKG	You are welcome .
Figure 10: An example dialogue performed by DiaoKG on SMD dataset.
acron guest house	type	guest house
acron guest house	address	154 chesterton road
acron guest house	price range	moderate
acron guest house	area	north
acron guest house	stars	4 star
warkworth house	type	guest house
warkworth house	address	warkworth terrace
warkworth house	price range	moderate
warkworth house	area	east
warkworth house	stars	4 stars
kirkwood house	type	guest house
kirkwood house	address	172 chesterton road
kirkwood house	price range	moderate
kirkwood house	stars	4 star
Turn 1	Q	Can I get some help locating a room for the night ?
Gold	Absolutely. What type of lodging did you have in mind ?
DialoKG	I have a great number of places to stay. Do you have a specific area you would like to stay in ?
Turn 2	Q	I am looking for a guesthouse with free wifi and parking, please .
Gold	You're in luck, there are a great number of guesthouse that might work. Did you have an area in mind ?
DialoKG	I have a great number of guest houses that offer both free wiki and parking. Do you have a specific area you would like to stay in ?
Turn 3	Q	I don't care about the area. Which one do you suggest ?
Gold	How about acron guest house ?
DialoKG	Alright, how about acron guest house ?
Figure 11: An example dialogue performed by DialoKG on MWOZ 2.1 dataset.