SlowGuess/NAACL_2022_OCR huggingface dataset:
RoViST-C For ALBERT, we use the 'albert-large-  $\nu I$ ' configuration and the Adam optimizer with a 0.00001 weight decay for training. The learning rate was 0.00001 which we schedule to reduce by  $5\%$  every epoch. Additionally, the batch size was 32 and early stopping was employed after the validation loss failed to improve for 5 epochs. We note that  $85\%$  and  $15\%$  of the data was used in the training and validation set respectively. In total, we trained the model for 5 epochs, taking 14 hours with a Nvidia Tesla P100 GPU.
RoViST-NR For assessing intra-sentence non-redundancy,  $n$ -grams of size 4 were used as we found that repetition of words within sentences usually occurred in fours.
Example Story 1 (Gold): Two women sit at a restaurant table and eat their dinner. They walk through the restaurant and see statues standing in a corner. The beach is next and they take a bike ride. The coast of the beach has gorgeous mountains and trees. The two women pose by the ocean for a final picture.
mountains (0.320)
statues (0.295)
ocean (0.329)
bike (0.292)
trees (0.271)
picture (0.299)
restaurant (0.298)
ride (0.264)
dinner (0.332)
beach (0.333)
table (0.363)
coast (0.312)
women (0.298)
corner (0.335)
Example Story 2 (MCSM+BART): We all had a family dinner. Our food was delicious and cooked perfectly. Mom and dad were there too. Grandma bought her wine, [female] had a glass. The family took some photos together before their visit.
dinner (0.352)
glass (0.320)
Figure 6: Retrieved regions from RoViST-VG for an example gold story (top) and machine generated story (bottom).
family (0.328)
female (0.352)
food (0.349)
photos (0.227)
dad (0.291)
visit (0.291)
mom (0.300)
wine (0.347)
grandma (0.286)
C Human Evaluation Survey
Figure 7 shows the survey instructions used in the human evaluation study and the format of the survey questions. Participants recruited were volunteers from a variety of age groups (20-60 years old), education level and gender (10 female, 16 male).
Story Human Evaluation
The Visual Storytelling task aims to tell a story that relates to a photo sequence. This is a survey that requires you to judge a story based on the following aspects:
1. Visually Grounded: the story closely reflects the entities in the photos and tends not to be vague. Each sentence should match its corresponding image and may relate to concepts in other images.
2. Story Naturalness: Inter-sentence Coherence: the story should be well-structured and consecutive sentences should flow like a narrative. Each sentence should contain information that is relevant to the rest of the story with nothing random or irrelevant. Story should have a strong 'focus' and sentence ordering should be correct.
- Good example: The food was ready for the party. There were many different kinds of dishes.
- Good example: We went to the park. They had a great playground. We also got to eat ice cream.
- Bad example: We went to the park. I grew up in Sydney. This is my dog.
- Bad example: I like cats. The food was delicious. Then, we went to the beach.
3. No Repetition: the story does not contain repeating plots across sentences and within sentences e.g. "The food was delicious. The salad was delicious", "The streets were filled with people. There were alot of people there".
Additional Notes:
- [male], [female], [location], [organization] represents the name of a male, female, location or organization respectively.
- This survey has 20 pages in total.
- If you have any questions, please don't hesitate to contact me.
Visual Prompt (45595)
Story 4 (45595)
the family was having a great time at the beach
they played in the sand .
then they jumped on a rock.
then they went out .after that they had a picnic
Story 4 (45595)
Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
Visually Grounded	○	○	○	○	○
Inter-sentence Coherence	○	○	○	○	○
No Repetition	○	○	○	○	○
Which story do you think is the best? *
Figure 7: Survey instructions and form format for the human evaluation study.