SlowGuess/ACL_2020_OCR huggingface dataset:
E Highly Rated Examples from IGC
Image	IGC Round	Output
Context
Question	These flowers are growing in my back yard.
What type of flowers are they?
Human Response
TransResNet MM-Sum	I don't know but they are so pretty.
I don't know but these flowers are gorgeous and look so bright!
Context
Question	I bought this at a flea market.
What is this for?
Human Response
TransResNet MM-Sum	I think it's a mortar and pestle.
I'm not sure, but you could sell it for some cash!
Context
Question	I'd love to visit this city.
Is this some city in Europe?
Human Response
TransResNet MM-Sum	It definitely looks like it.
Oh my goodness, yes! I love Amsterdam!
Context
Question	These just flew over my house!
Wow, aren't those Blue Angels?
Human Response
TransResNet MM-Sum	Yeah, they practice in this area for their shows.
These are the Blue Angels, they are truly one of a kind
Context
Question	My nephew's choir did so well yesterday!
That is great. You must be proud
Human Response
TransResNet MM-Sum	I am so proud, because he's been working really hard.
I am so proud of all that they accomplished.
Context
Question	This would be a great place to have a picnic.
yes it would seem very relaxing doesn't it?
Human Response
TransResNet MM-Sum	Yes, and I would quite enjoy spending a day here.
I think a picnic is more than a great possibility!
Table 5: Highly rated examples from the IGC dataset test split where  $\mathrm{TRANSRESNET}_{\mathrm{RET}}$  MM-Sum responses were rated the highest (score of 3) by human evaluators.
F Poorly Rated Examples from IGC
Image	IGC Round	Output
Context
Question	These are Easter eggs we made for art class.
How did you make them?
Human Response
TransResNet MM-Sum	We started with a glass base and went from there.
OHMYGOSH EASTER EGG HUNT! I want to hunt for Easter eggs too!
Context
Question	I'm thinking of getting these shocks on my bike.
Are the shocks black in color?
Human Response
TransResNet MM-Sum	Yes, except for the silver portion at the end.
Just got my new wheels!
Context
Question	I cannot decide if this is milk or something else.
It looks like milk, but it could also be kefir.
Human Response
TransResNet MM-Sum	What is kefir?
You can get all the protein you want form the sugar.
Context
Question	This woman is visiting the grave of a loved one.
Where is the cemetery?
Human Response
TransResNet MM-Sum	The cemetery is at the foot a beautiful quartz mountain.
Well she's just a walking, talking garden!
Table 6: Low rated examples from the IGC dataset test split where  $\mathrm{{TRANSRESNET}}{}_{\mathrm{{RET}}}$  MM-Sum responses were rated the lowest (score of 1) by human evaluators.
G Longer Chat Example
A: TransResNetRET MM-Sum (Extraordinary) B: Human (no style)
A: It's the most beautiful mountain in the world!
B: It looks really cool! I wonder if you can ski on it.
A: They are a mountain range I've climed many times!
B: You have climbed that? That seems really hard.
A: I did it; i conquered this climb in only 7 hours and 10 minutes!
B: Wow, that's fast. I would ski down that but I would need a lift to take me up
A: But could you? Could you truly climb this?
B: I really don't think I could
A: Climbing a mountain can give one a special strength, you need to experience it.
B: Maybe one day on a smaller mountain
A: It would take hard will and determination to scale that mighty peak
H Additional Ablation Results
Figure 10: Long-form conversation with the model. The model is given a style here, while the human is not.
Modules	TRANSRESNETGEN (F1)	TRANSRESNETGEN (BLEU-4)
Turn 1	Turn 2	Turn 3	All	Turn 1	Turn 2	Turn 3	All
Image Only	10.8	11.0	11.2	11.0	1.1	1.3	1.2	1.2
Style Only	10.4	9.8	10.4	10.2	1.4	1.5	1.4	1.4
Dialogue History Only	9.9	11.4	12.2	11.2	1.0	1.9	1.8	1.6
Style + Dialogue (no image)	9.6	12.5	13.1	11.7	1.5	2.1	2.0	1.9
Image + Dialogue (no style)	10.7	11.1	11.7	11.2	1.1	1.7	1.6	1.5
Image + Style (no dialogue)	12.1	11.6	11.6	11.8	1.6	1.5	1.5	1.6
Style + Dialogue + Image (full model)	12.3	12.5	13.1	12.6	1.7	2.1	2.0	1.9
Table 7: Ablations on IMAGE-CHAT. We compare variants of our best TRANsRESNET generative model (ResNeXtIG-3.5B image encoder) where we remove modalities: image, dialogue history and style conditioning, reporting F1 and BLEU-4 for generation for dialogue turns 1, 2 and 3 independently, as well as the average over all turns.