SlowGuess/NAACL_2022_OCR huggingface dataset:
Table 5: Hyperparameter grid search on the joint learning coefficients on BART-base. The coefficients are given in the first column in the form of  $((\lambda_{\mathrm{mm - disamb}},\lambda_{\mathrm{att}},\lambda_{\mathrm{empty - coref}}),\lambda_{\mathrm{mm - coref}},\lambda_{\mathrm{retrieval}})$ . Bolded row shows the most optimal hyperparameters in terms of model selection criteria.
UT	Can you show me a jacket that goes with my wardrobe?
Generated AT	There are several jackets which match your criteria, including a black jacket, a grey and white jacket, and a light grey jacket.
GT AT	There are a couple of jackets throughout the store that you might like. What do you think?
UT	Can you recommend a jacket to match my clothes?
Generated AT	Do you like the blue one on the bottom row of the wall? It is a jacket.
GT AT	Do you like the brown one on the top left of the wall or the blue one directly under it? They are jackets.
UT	Can you help me find a cheap blouse?
Generated AT	What do you think of the grey and white blouse on the wall, the maroon, white, and blue on the rack, or the white and grey on the back wall?
GT AT	There are three options on the left wall: a grey and white, a white and grey, and a maroon, white, and blue?
Table 6: Comparison between Generated  $A_{T}$  and ground truth (GT) in the case of recommendation. The differences between the generated and the ground-truth (in terms of semantics) are bolded.
UT	What is the rating and price of that gray and white shirt?
Generated AT	Which one?
GT AT	Which one?
UT	The gray and white one hanging on the left.
Generated AT	The grey one is $239.99 and the brown one is only $59.99.
GT AT	Sure! The brown one is $59.99 and the grey one is $239.99.
UT	The green and white blouse in the closet and the red, white, and yellow blouse on the rack.
Generated AT	The former is from Downtown Consignment and is available in S, XL, and XXL. The latter is from Pedals & Gears and is in stock in XS, S and XL.
GT AT	The first is available in XS, S, and XL and is from The Vegan Baker. The other is in XS, S, and XL and is from Downtown Consignment.
Table 7: Comparison between Generated and GT  $A_{T}$  in the case of disambiguation and informing object attributes. The differences between the generated and the ground-truth (in terms of semantics) are bolded.
Figure 4: Comparison between the model prediction and the ground-truth of a SIMMC 2.0 dialog instance.
Utterance	MM-Disamb	MM-Coref
GT	Pred	GT	Pred
U0	Are there any blouses that are cheap and in red and white that you can show me?	-	-	-	-
GT B0	REQUEST:GET, slots: { type: blouse, price: cheap, color: red, white}
Pred B0	REQUEST:GET, slots: { type: blouse, price: cheap, color: red, white}
GT A0	There is one on the left rack. What do you think?
Generated A0	There are several red and white blouses which match your search criteria.
Retrieved A0	Sorry but no matches were found. How else may I be of assistance?
U1	What are the available sizes and the brands of the red, white, and yellow blouse and the green and white blouse?	1	1	21, 10	21
GT B1	ASK:GET, slots: {}
Pred B1	ASK:GET, slots: {}
GT A1	Which items?
Generated A1	Which items are you talking about?
Retrieved A1	Sorry, but no other matches were found.
U2	The green and white blouse in the closet and the red, white, and yellow blouse on the rack.	-	-	10, 21	10, 21
GT B2	INFORM:DISAMBIGUATE, slots: {}
Pred B2	INFORM:DISAMBIGUATE, slots: {}
GT A2	The first is available in XS, S, and XL and is from The Vegan Baker. The other is in XS, S, and XL and is from Downtown Consignment.
Generated A2	The first is available in XXL, L, and XL and is from Cats Are Great. The other is available and is in from The Vegan Baker.
Retrieved A2	The grey and white blouse is by Nature Photographers and is available in XS, M, L, and XL. The maroon and white blouse is by Global Voyager and is available in S, L, XL, and XXL.
U3	What else do you have that has good ratings?	-	-	-	-
GT B3	INFORM:REFINE, slots: { customerReview: good, type: blouse, price: cheap, color: red, white}
Pred B3	INFORM:REFINE, slots: { customerReview: good, type = blouse, size = XS, color = red, white, yellow}
GT AT	Sorry, but no other matches were found.
Generated A3	Sorry, but no item matches your search criteria.
Retrieved A3	Sorry, but no other matches were found.
U4	Now tell me how the green and white blouse compares with the other red and white blouse.	-	-	10, 27	10, 27
GT B4	REQUEST:COMPARE, slots: {}
Pred B4	REQUEST:COMPARE, slots: {}
GT A4	The green blouse is 24.99. The red blouse is 24.99.
Generated A4	The first is from Cats Are Great and has a score of 3.9. The other is from the Vegan Baker and has short sleeves.
Retrieved A4	The first one is from North Lodge and is a size L, the second one is from Modern Arts in an XXL.
Figure 5: Comparison between the model prediction and the ground-truth of a SIMMC 2.0 dialog instance.
Utterance	MM-Disamb	MM-Coref
GT	Pred	GT	Pred
U0	Can you help me find a cheap blouse?	-	-	-	-
GT B0	REQUEST:GET, slots: {type: blouse, price: cheap}
Pred B0	REQUEST:GET, slots: {type: blouse, price: cheap}
GT A0	There are three options on the left wall: a grey and white, a white and grey, and a maroon, white, and blue.
Generated A0	"What do you think of the maroon, white, and blue blouse on the left, the grey and white on the right, or the dark grey on the back wall?"
Retrieved A0	There are three options on the left wall: a grey and white, a white and grey, and a maroon, white, and blue.
U1	Are any from Global Voyager and more expensive?	-	-	-	-
GT B1	INFORM:REFINE, slots: { brand: Global Voyager, price: cheap, type: blouse}
Pred B1	INFORM:REFINE, slots: { brand: Global Voyager, price: cheap, type: blouse}
GT A1	None of the options match that description.
Generated A1	Sorry, but no matches were found. What else can I do for you?
Retrieved A1	None that match your search terms.
U2	What can you tell me about the white and grey blouse and the maroon, white, and blue?	0	0	7,3	7,3
GT B2	REQUEST:COMPARE, slots: {}
Pred B2	REQUEST:COMPARE, slots: {}
GT A2	Both have long sleeves.
Generated A2	The white and grey blouse has a score of 2.9, is available in M, L, XL, and XXL, is from StyleNow Feed, is 39.99, and is in size L. The other has a rating of 3.8, is 54.99, available in XXL and S, is in a holiday pattern, has long sleeves, is a maroon, white, and blue, from Global Voyager, is 64.99, and comes in S.
U3	Will you add the maroon sweater from the display and the black and olive dress from the rack to my cart?	0	0	14,30	28,30
GT B3	REQUEST:AddTOCART, slots: {}
GT A3	No problem, both items have been added for you.
Generated A3	Of course. Both have been added.
Retrieved A3	Confirmed. Both items will be added to your cart.
U4	Will you also add the black dress and the maroon one from the rack to my cart.	-	-	27,20	20
GT B4	REQUEST:AddTOCART, slots: {}
Pred B4	REQUEST:AddTOCART, slots: {}
GT A4	Of course. Both have been added.
Generated A4	Of course, both have been added to your cart.
Retrieved A4	Of course, both are being added to your cart now.
Figure 6: Comparison between the model prediction and the ground-truth of a SIMMC 2.0 dialog instance.
Utterance	MM-Disamb	MM-Coref
GT	Pred	GT	Pred
U0	I want a coat with good reviews.	-	-	-	-
GT B0	REQUEST:GET, slots: {type: coat, customerReview: good}
Pred B0	REQUEST:GET, slots: {type: coat, customerReview: good}
GT A0	How is this black one in the top middle on the wall, or the other black one on the right side of the rack?
Generated A0	How about the black coat on the top left of the wall display?
Retrieved A0	How is this black one in the top middle on the wall, or the other black one on the right side of the rack?
U1	How does the red, white and yellow blouse on the top right compare to the white red and violet one on the bottom of the left wall?	-	-	0,8	0,8
GT B1	REQUEST:COMPARE, slots: {}
Pred B1	REQUEST:COMPARE, slots: {}
GT AT	The first one is rated 3.4, and the other one is rated 4.7
Generated A1	The one on the right is rated 3.4, and the other one is rated higher at 4.7.
Retrieved A1	The left is light grey in M with short sleeves and the right is short sleeve in XXL with colors green, violet, and pick.
U2	Do you have anything similar to those black coats, but made by Downtown Consignment?	-	-	2,12	2,12
GT B2	INFORM:GET, slots: {sleeveLength: full, brand: Downtown Consignment, size: L, price: 139.99 type: coat}
Pred B2	INFORM:GET, slots: {brand: Downtown Consignment, type: coat}
GT A2	I don't have anything like that.
Generated A2	I'm sorry, I don't have anything like that. Can I get you something different?
Retrieved A2	Sorry, I don't have anything like that.
U3	What is the size on that dress?	1	1	1	-
GT B3	ASK:GET, slots: {}
Pred B3	ASK:GET, slots: {}
GT A3	Which dress are you asking about?
Generated A3	Which dress?
Retrieved A3	Which dress are you asking about?
U4	The black one on the top middle of the wall display.	-	-	1	-
GT B4.	INFORM:DISAMBIGUATE, slots: {}
Pred B4.	INFORM:DISAMBIGUATE, slots: {}
GT A4	That dress is a size M.
Generated A4	That dress is a M.
Retrieved A4	This dress is available in S, M, L, XL, and XXL.
Figure 7: Comparison between the model prediction and the ground-truth of a SIMMC 2.0 dialog instance.
Utterance	MM-Disamb	MM-Coref
GT	Pred	GT	Pred
U0	I'm looking to buy some luxury jeans that have reviewers falling head over heels.	-	-	-	-
GT B0	REQUEST:GET, slots: {type: jeans, customerReview: good, price: expensive}
Pred B0	REQUEST:GET, slots: {type: jeans, customerReview: good, price: expensive}
GT A0	Those two jeans in the back to the left are a match! Do either of those suit your preferences?
Generated A0	There are some very high-quality jeans on the shelves to the right. Could you see yourself wearing that dark blue pair on the end or the light blue pair next to it?
Retrieved A0	Those two jeans in the back to the left are a match! Do either of those suit your preferences?
U1	Yeah, I could get on board with those. Which one is more popular with buyers?	0	0	6,7	6,7
GT B1	REQUEST:COMPARE, slots: {}
Pred B1	REQUEST:COMPARE, slots: {}
GT A1	The lighter pair is more highly rated, with a nearly unbeatable average customer rating of 4.9. The darker pair also comes highly recommended, with an average score of 4.6.
Generated A1	Both of those jeans have on average been rated a 4.6 by other customers.
Retrieved A1	The lighter pair is more highly rated, with a nearly unbeatable average customer rating of 4.9. The darker pair also comes highly recommended, with an average score of 4.6.
U2	Good to know. Can I also get you to look for something in the same vein as that violet blouse hanging up to the right or the green, violet, and pink one near it, but by the brand Downtown Consignment?	-	-	4,5	4,5
GT B2	INFORM:GET, slots: {brand: Downtown Consignment, type: blouse}
Pred B2	INFORM:GET, slots: {brand: Downtown Consignment, type: blouse}
GT A2	That brand makes the green and white shirt hanging above the light-wash jeans to the left.
Generated A2	I'm so sorry, but I can't find any similar blouses by that brand. Is there something else I might be able to do for you?
Retrieved A2	That brand makes the green and white shirt hanging above the light-wash jeans to the left.
U3	Hmm okay and what are the rating figures on that red jacket and the grey one?	1	1	9,0	9,0
GT B3	ASK:GET, slots: {}
Pred B3	ASK:GET, slots: {}
GT A3	Woops, I'm not certain which jackets you're talking about. Could you please be more specific?
Generated A3	Which jackets are you inquiring about?
Retrieved A3	Woops, I'm not certain which jackets you're talking about. Could you please be more specific?
U4	I mean that red and white one to the right by the purple shirt and the grey one hanging near the green shirt to the left.	-	-	0,9	0,9
GT B4	INFORM:DISAMBIGUATE, slots: {} Pred B4	INFORM:DISAMBIGUATE, slots: {}
GT A4	The red and white jacket is rated a 3.3, while the grey jacket is rated a 3.6.
Generated A4	The red and white jacket has an average customer rating of 3.3, while the grey jacket is rated a 3.6.
Retrieved A4	The red and white jacket is rated a 3.3, while the grey jacket is rated a 3.6.
Figure 8: Comparison between the model prediction and the ground-truth of a SIMMC 2.0 dialog instance.
Models	Utterance	MM-Disamb	MM-Coref
GT	Pred	GT	Pred
U0	Are there any highly rated pants around here?
GT B0	REQUEST:GET, slots: { type: trousers, customerReview: good}
Pred B0	REQUEST:GET, slots: { type: trousers, customerReview: good}
GT A0	The first two pairs on the left have great reviews!
Generated A0	I think you may like the blue pair on the left and the two grey pairs on the right.
Retrieved A0	The first two pairs on the left have great reviews!
U1	Okay the last thing I need right now is more t-shirts, but I can't help myself. I'm totally eyeing that white one in the back to the left and the black one hanging on the top row. Which of those is less expensive, and what sizes are those?	0	0	1, 11	1, 10, 11
GT B1	REQUEST:COMPARE, slots: {}
Pred B1	REQUEST:COMPARE, slots: {}
GT A1	The black shirt pictured here is an extra large, and it sells for $24.99. The white shirt pictured here is a large, and it is priced slightly higher, at $29.99.
Generated A1	The white shirt is less expensive, at $54.99, and it is a small. The black shirt costs $29.99 and is an extra large
Retrieved A1	The black shirt pictured here is an extra large, and it sells for $24.99. The white shirt pictured here is a large, and it is priced slightly higher, at $29.99.
U2	Gotcha. Got anything else for me to see?
GT B2	INFORM:REFINE, slots: {type: trousers, customerReview: good}
Pred B2	INFORM:REFINE, slots: {type: tshirt}
GT A2	My apologies, but there are no more highly rated trousers for you to view at this time.
Generated A2	I apologize, but those are the only t-shirts I can find here. Is there something else I might be able to help you with?
Retrieved A2	Unfortunately, those are the only ones I have. Can I show you something different?
U3	Hm okay well can you try to dig up something with the same sizes as this black blouse on the first rack but with a polka dot pattern of some kind?	0	0	40	-
GT B3	INFORM:GET, slots: {pattern: spots, availableSizes: [S, XS, M], type: blouse}
Pred B3	INFORM:GET, slots: {pattern: checkered, availableSizes: [XS], type: blouse}
GT A3	Oh no, it looks like there are no similar blouses in that pattern. What else can I do for you today?
Generated A3	I regret to inform you there are no similar blouses with that pattern.
Retrieved A3	Oh no, it looks like there are no similar blouses in that pattern. What else can I do for you today?
U4	Oh snap that black blouse back there might be just the thing actually. What's that cost, and who makes it?	3	-
GT B4	ASK:GET, slots: {}
Pred B4	ASK:GET, slots: {}
GT A4	Which blouse are you inquiring about?
Generated A4	Sorry, which blouse are you interested in learning more about?
Retrieved A4	Which blouse are you inquiring about?