SlowGuess/NAACL_2021_OCR huggingface dataset:
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual Question Answering. In ICCV.
Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, José MF Moura, Devi Parikh, and Dhruv Batra. 2017. Visual dialog. In CVPR.
Zhe Gan, Yu Cheng, Ahmed EI Kholy, Linjie Li, Jingjing Liu, and Jianfeng Gao. 2019. Multi-step reasoning via recurrent dual attention for visual dialog. ACL.
Donald Geman, Stuart Geman, Neil Hallonquist, and Laurent Younes. 2015. Visual turing test for computer vision systems. NAS.
Dalu Guo, Chang Xu, and Dacheng Tao. 2019. Image-question-answer synergistic network for visual dialog. In CVPR.
Ronghang Hu, Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Kate Saenko. 2017. Learning to reason: End-to-end module networks for visual question answering. ICCV.
Drew A Hudson and Christopher D Manning. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In CVPR.
Unnat Jain, Svetlana Lazebnik, and Alexander Schwing. 2019. Two can play this game: Visual dialog with discriminative question generation and answering. CVPR.
Xiaoze Jiang, Jing Yu, Zengchang Qin, Yingying Zhuang, Xingxing Zhang, Yue Hu, and Qi Wu. 2020. Dualvd: An adaptive dual encoding model for deep visual understanding in visual dialogue. AAAI.
Gi-Cheon Kang, Jaeseo Lim, and Byoung-Tak Zhang. 2019. Dual attention networks for visual reference resolution in visual dialog. ACL.
Satwik Kottur, José MF Moura, Devi Parikh, Dhruv Bartra, and Marcus Rohrbach. 2018. Visual coreference resolution in visual dialog using neural module networks. In ECCV.
Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023.
Mateusz Malinowski and Mario Fritz. 2014. A multi-world approach to question answering about real-world scenes based on uncertain input. In NIPS.
George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM.
Vishvak Murahari, Dhruv Batra, Devi Parikh, and Abhishek Das. 2020. Large-scale pretraining for visual dialog: A simple state-of-the-art baseline. ECCV.
Van-Quang Nguyen, Masanori Suganuma, and Takayuki Okatani. 2020. Efficient attention mechanism for handling all the interactions between many inputs with application to visual dialog. ECCV.
Yulei Niu, Hanwang Zhang, Manli Zhang, Jianhong Zhang, Zhiwu Lu, and Ji-Rong Wen. 2019. Recursive visual attention in visual dialog. In CVPR.
Jiaxin Qi, Yulei Niu, Jianqiang Huang, and Hanwang Zhang. 2020. Two causal principles for improving visual dialog. CVPR.
Idan Schwartz, Alexander G Schwing, and Tamir Hazan. 2019a. A simple baseline for audio-visual scene-aware dialog. In CVPR.
Idan Schwartz, Seunghak Yu, Tamir Hazan, and Alexander G Schwing. 2019b. Factor graph attention. In CVPR.
Paul Hongsuck Seo, Andreas Lehrmann, Bohyung Han, and Leonid Sigal. 2017. Visual reference resolution using attention memory for visual dialog. In NIPS.
Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In AAAI.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In ACL.
Harm de Vries, Florian Strub, Sarath Chandar, Olivier Pietquin, Hugo Larochelle, and Aaron C Courville. 2017. Guesswhat?! visual object discovery through multi-modal dialogue. In CVPR.
Yue Wang, Shafiq Joty, Michael R Lyu, Irwin King, Caiming Xiong, and Steven CH Hoi. 2020. Vd-bert: A unified vision and dialog transformer with bert. EMNLP.
Tianhao Yang, Zheng-Jun Zha, and Hanwang Zhang. 2019. Making history matter: History-advantage sequence training for visual dialog. In ICCV.
Zilong Zheng, Wenguan Wang, Siyuan Qi, and Song-Chun Zhu. 2019. Reasoning visual dialogs with structural and partial observations. In CVPR.
A Qualitative Analysis
In the following, we show additional 200 randomly picked visual dialog samples from test-standard. For each sample, we provide the ranked MRR candidate set and the next 10 NDCG candidates. We
colorize the high-certainty candidates  $(\mathcal{H})$  with orange, the NDCG-agreement candidates  $(\mathcal{N})$  with purple, and the top-answers subset  $(\mathcal{T})$  with red. Note, if a candidate belongs to more than one set, we sketch the colors in the following order: orange→red→purple.
Question: what color is the light? MRR candidate set
1. red
2. green
3. the light is currently green
4. the lights are n't visible
Top 10 from the remaining NDCG candidates
1. yellow
2. not sure
3. i can't tell
4. it's kinda dark out so i am unsure
5. i can not tell
6. can't tell
7. black and red
8. UNK
9. black
10. it's dark
Question: what ethnicity do they appear to be? MRR candidate set
1. all white
2. white
3. i can not tell
4. can't tell
5. not sure
1. black
2. i can't tell
3. both have brown hair
4. there are 2 teens, and 1 adult
5. i think like 40's maybe
6. maybe younger seniors
7. dark brown
8. no, the tv is not that visible
9. 1 looks older the other is quite young
10. she appears to be caucasian
Question:are there any other animals? MRR candidate set
1. no
2. not that i see
3. nope
4. not that i can see
1. no other animals
2. 0 that i can see
3.0
4. 0 visible
5. lol, no
6. non
7. i don't think so
8. i can not see
9. no, only animals
10. i do not see any animals
Question: how old is the woman? MRR candidate set
1. she's probably 60
2. middle aged
3. late UNK's
4. i can not tell
5. unable to tell
1. can't tell
2. looks middle aged
3. mid sixties
4. i can't tell
5. late 40's
6. about middle aged maybe 40 something
7. 70s
8. not sure
9. about 30
10. at least 30s
Question:do they all have any spots on them? MRR candidate set
1. yes, all but 1 has spots
2. no
3. yes
4. not that i can see
5. it's too hard to tell
6. nope
1. i do n't think so
2. i can not tell
3. very few
4. some do, yes
5. not really
6. some do
7. can not tell
8. i can't tell
9. can't tell
10. 0
Question:see any store names ?
MRR candidate set
1. no
2. yes
3. i can not tell
4. unable to tell
5. not that i can tell
6. not that i can see
1. nope
2. not really
3. i can't see it
4. not sure
5. i can't tell
6. can't tell
7. probably just 1
8. not at all
9. yes 1
10. i don't think so
Question: what color is their hair?
1. i can not tell
2. black
3. can't see them
4. can't tell
5. it doesn ’ t have hair it ’ s a light
6. i can't tell
1. not sure
2. dark is all i can tell
3. brown i believe
4. hard to tell, but it looks very blackened
5.brown
6. it looks brown
7. no people
8. dark brown
9. it is dark brown
10. it's brown
Question: is it daytime or night time?
1.daytime
2.day
3. it appears to be daytime
4. it is daytime
5. it looks like it's daytime
6. it's daytime
1. it is day time
2.day time
3. it is day
4. it is during the day
5. hard to tell probably day
6. looks like early morning
7. daytime maybe dusk
8. hello, fellow turkey: yes, it is day time
9. in the even or morning
10. i can't tell
Question: is this photo in color? MRR candidate set
1. yes
2. yes it is
3. yes, it is
4. yes it is in color
5. yes, it's in color
1. yes the picture is in color
2. yes in color
3. it is
4. it is in color
5. it's in color
6. yes, it's very colorful!
7. i think so
8. looks like it
9. it is, but there is not much color in the photo due to the objects
10. yes it is well lit
Question: what are the vegetables he has? MRR candidate set
1. salmon and greens, maybe spinach
2. it looks like greens , hard to tell
3. can n't really tell
4. i can not tell
5. i can't tell
6. can't tell
1. not sure
2. i'm not a  $100\%$  sure, look like it
3. yes
4. not that i can see
5. can not see grass
6. green
7. plate
8. looks old
9. i do n't think so
10. nope
Question:are they all facing the camera?
1. yes
2. no
3. slightly
4. yes they are
5. nope
1. not really
2. i don't think so
3. yes they all are
4. yes i believe so
5. no side view
6. i do n't think so
7. i think so
8. all but 1
9. not all of them
10. not that i can see
Question: is it a poodle?
1. no
2. i do n't think so
3. nope
1. does not look like it
2. i can't tell
3. i can not tell
4. not sure
5. i can not say
6. can't tell
7. not that i can see
8. i do n't think so, but it's hard to tell
9. i'm not sure what that is
10. no i am unable to tell
Question: how old is the man?
1. i can not tell
2. can not tell
3. unable to tell
4. can't see his face
5. i can't tell
6. can't tell
1. i can't tell i can't see their face
2. twenties maybe, hard to tell
3. hard to tell but i would say mid 20 ’s
4. hard to say but i guess around thirty
5. not sure
6. i can not see their face
7. maybe 25ish
8. he looks to be about 30ish
9. it's hard to tell, it's from a distance
10. he looks to be maybe 20 or so
Question: is there anything near it?
1. no
2. grass and tree 's
3. nothing
4. not that i can see
5. nope
6. can't see anything
1.yes
2. i do n't think so
3. i don't see any
4. not really
5. you can see a pitcher and maybe a wall
6. a few things
7. no, it's just dirt
8. in the distance
9. no but it has a lot of stuff on it
10. not sure there may be, but it does n't show enough to tell
Question: is there anything on the wallpaper of the laptop?
1. no
2. nope
3. not that i can see
1. no there is not
2. i can not tell
3. i don't see any
4. can't tell
5. i can't tell
6.0
7. i don't think so
8. just a screen over it
9. not that i can UNK for sure
10. not sure
Question: is it date stamped?
1. no
2. no it's not
3. no it is n't
4. nope
5. no , it 's not
1. not that i can see
2. i do n't think so
3. i don't know
4. do n't know
5. i can't tell
6. can't tell
7. not sure
8. i can not tell
9. not readable
10. nah
Question: is this inside? MRR candidate set
1. yes
2. yes it is
3. yes, it is
4. yes i think so
1. yes, it seems to be
2. i think so, yes
3. this is inside
4. it seems to be
5. it is indoors
6. i believe so
7. i think so
8. looks like it
9. no
10.ues
Question: does the food look appetizing? MRR candidate set
1. yes
2. no
3. not really
4. not at all
5. nope
1. yes it does
2. no, he does n't
3. yes, it does
4. i can not tell
5. can't tell
6. a little
7. not that i can see
8. sort of
9. kinda
10. i can't tell
Question: is the image in color?
1. yes
2. yes it is
3. yes, it is in color
4. yes it is in color
5. yes, it's in color
6. yes, the photo is in color
1. yes, it is
2. yes the photo is in color
3. yes, full color photo
4. yes it's color
5. it is
6. yes in color
7. it is yes
8.yess
9.yea
10. yyes
Question: is it daytime?
1. yes
2. yes it is
3. yes, it is daytime
4. yes it is daytime
5. yes, it is
1. it is
2. yes, it looks to be
3. it is daytime
4. yup
5. it is day time
6. yes, i think so
7. i believe so
8. yes it is, it looks sunny
9. daytime
10. i think so
Question: is the photo in color?
1. yes
2. yes it is
3. yes, it is in color
4. yes it is in color
5. yes it's in color
1. yes, it is
2. yes the picture is in color
3. yeah, this photo is in color
4. yes in color
5. it is
6. it is in color
7. yup
8. it is a beautiful color picture
9.ye
10. y
Question:any buildings visible?
1. yes a few
2. yes
3. yes, a few buildings
4. yess
5. i think so
1. yeah in the distance
2. lots
3. in the distance
4. yes there are some small ones
5. yes, 1 across the street
6. yes in the far background
7. yes, but i can't tell what kind
8. yes, it is
9. very far in the background
10. no
Question: is there anything made out of plastic?
1. no
2. yes
3. nope
4. not that i can see
5. i do n't think so
1. can't tell
2. no that i can see
3. not sure
4. can't see
5. i can't tell
6. i can not tell
7. maybe
8. i think so
9. not really
0. yes, looks like a box
1. she looks like in her 30 's
2. in her twenties i believe
3. not sure
1. 20's
2. early 20 's
3. i can't tell
4. can't tell
5. 20's maybe
6. looks to be late 20s
7. i can not tell
8. young adult
9.25
10. early 30s maybe
Question: is the image in color? MRR candidate set
1. yes
2. yes it is
3. yes, it is
4. yes, it is in color
5. yes it is in color
1. yes this is a color image
2. yes, it's in color
3. it is yes
4. yes this picture is in color
5. yes the photo is in color
6. yes in color
7. yep
8. it is in color
9. yes it's color
10.ye
Question: is anyone in the bathroom?
1. no
2. no the bathroom
3. not that i can see
4. no people
5. nope
6. i don't think so
1. no it appears empty
2. not it's not visible
3. no he's not
4. i would say no
5. no, there is no 1 in the kitchen
6. UNK
7. don't know
8.0
9. not really
10. no people this is the sky
Question: is he wearing a hat?
1. no
2. no he is not
3. nope
4. i don't see 1
1. i don't think so
2. no, no hats
3. no baseball hat
4. not that i can see
5. no can't see it
6. no he is not wearing a helmet
7. can't tell
8. i can't tell
9. can not tell
10. not really
Question:are there any people?
1. no
2. yes
3. not that i can see
4. i don't see any
5. nope
1. i can't see any, no
2. no people
3. no people are visible
4. no, there are n't any people in the image
5. no there are n't
6. no people in the photo
7. 0 i can see
8. not that you can see
9. i can't see
10. on the screens there are
Question:do they have shoes on? MRR candidate set
1. yes
2. i can not tell
3. i see 1 shoe the other person is to blurry to tell
4. it looks like it
5. it's hard to tell
6. i can't tell
1. looks like it
2. i guess so, it is hard to tell
3. yes i think so, you can n't really see
4. can't tell
5. i think so
6. they are
7. not sure
8. no, they do not
9. can't see
10. yes they are green
Question: does it look like a ski resort? MRR candidate set
1. no
2. i can not tell
3. i do n't think so
4. no it does n't
5. probably not
6. nope
1. not really
2. i can't tell
3. no i can't
4. i do n't think so but it 's hard to tell
5. not that i can see
6. not sure
7. no, there is n't
8. maybe, i can't really tell
9. possibly
10. not in the picture
Question: is the picture in color?
1. yes
2. yes it is
3. yes, it is
4. yes it is in color
5. yes it is a color image
1. yeah, this photo is in color
2. yes in color
3. yes, it is fully colored
4. yyes
5. full blown color
6. i think so
7. looks like it
8. it is, but there is not much color in the photo due to the objects
9. somewhat, yes
10. looks it
Question: is she wearing a blazer?
1. no
2. no , sweater
3. nope
4. not that i can tell
5. i do n't think so
1. no a sweater
2. not that i can see
3. yes, a black 1
4. a long sleeve blouse
5. not sure
6. yes
7. can not see 1
8. do n't know
9. 0 seen
Question: does he look angry or otherwise UNK? MRR candidate set
Question: are they UNK giraffes or UNK? MRR candidate set
1. i can not tell
2. there are both
3. i can't tell
4. not sure
5. can't tell
1. no
2. can't tell
3. you can n't really see his face
4. i can't tell
1. i can't see that
2. i can not tell
3. not real sure
4. nope
5. not really
6. not sure
7. i do n't think so
8. looks to be sleeping if anything
9. not that i can see
10. it looks determined, but not angry
1. some of each
2. i do n't think so
3. hard to tell, but i do n't think so
4. it looks to be that way
5. no
6. they are all various sizes
7. i think so
8. both
9. can't tell too close up
10. there are 2 UNK ones.
Question: what color is the reindeer?
1. brown, with a red nose
2. red
3.brown
4. brown and white
5. mostly white
1. white
2. golden
3. white and tan
4. brown, i think
5. white with a red stripe
6. brown, white and black
7. dark brown
8. gold
9. it is brownish gray
10. mostly white with some black
Question:can you tell what type they are, stone or laminate?
1. they are stone
2. i can not
3. i can not tell
4. i can't tell
5. no i'm not sure
6. no, i can not
1. can't tell
2. i can ' tell , it ' s just white
3. nope
4. no
5. not sure
6. not really
7. no, i'm bad at that
8. i can't te
9. no, i can only see the top
10. not that i can see
Question: any horses? MRR candidate set
1. no
2. yes
3.yes,1
4.1
5. nope
1. 2
2. i just see 2
3. yes, it looks like it
4. not that i can see
5.ye,1ifnotmore
6. not that i see
7. i think so
8. no there is n't
9. not visible
10. barely part of 1
Question:are they male? MRR candidate set
1. yes
2. no
3. yes they are
4. i think so
5. i believe so
6. it seems like it
1. 1 is
2. yes 1 is
3. hard to tell
4. yes!
5. i can't tell
6. can't tell
7. i can not tell
8. not sure
9. no idea not visible
10. can't tell they are far away in background
Question: is the bench in front of a building? MRR candidate set
1. yes, it is
2. yes
3. yes in front of glass
4. no it is n't
5. nope
1. no
2. yes it is
3. yes looks like it
4. not that i can see
5. looks like it
6. appears so
7. i think so
8. i see a building on the left side, but only a bit of
9. i can not tell
10. i think so, it's hard to tell
Question:how many pizzas ?
1. i can see 2 kinds of pizzas
2. 2
3. there are 3
4. there are 2
5. i see 2
1. 2 in the scene
2.3
3. i think 4
4. just 1
5. i can not tell
6.4
7. it looks like a box of them and 2 sitting out
8. they are like 4
9.1
0. i can't tell
Question: what color is the building?
1. white
2. grey
3. it is white
4. greyish
5. beige white
1. white and black
2. blue
3. beige
4. blue with a glass door
5.bright blue
6. off white with some UNK of gray and red brown throughout
7. there is 1 building that is a cream color
8. it is glass
9. crème
10. it's a brownish white color
Question: is the photo in color? MRR candidate set
1. yes
2. yes it is
3. yes, it is
4. yes, it is in color
5. yes, the photo is in color
6. yes it is in color
1. yes, the image is in color
2. yes in color
3. yes, the picture is in color
4. the photo is in color
5. yes the picture is in color
6. it is
7. sure is
8.yes,
9. yup
10.ye
Question: is the day sunny? MRR candidate set
1. yes
2. no
3. not really
4. nope
1. yes it is
2. looks like it
3. no, it is not
4. yes, it is
5. i think so
6. UNK
7. sort of
8. i do n't think so
9. i suppose
10. can't really see the sun think it's pretty overcast
Question:are they on a white plate? MRR candidate set
1. yes
2. yes , it is
3. yes they are
4. yes it is
5. i believe so
1. it's in color
2.yes,a brown 1
3. i think so
4. UNK
5. no it is in a paper boat
6. it's okay
7. white
8. no, just on the table
9. yes, i can see some cars
10. it is in a box
Question: is the sink made out of metal? MRR candidate set
1. yes
2. i can not tell
3. i can't tell
4. can n't really tell
5. can't tell
1. not sure
2. no
3. yes it looks like it is
4. yes it is
5. i think so
6. looks like it
7. yes, it is
8. it look like it
9. nope
10. too far away to see that
Question: is there people there apart from the man?
1. no
2. no just 1 person
3. not that i can see
4. nope
5. i do n't see any
1. no other people
2. i don't see anyone at all
3.0
4. no people are in the picture
5. i don't think so
6. no people
7. no, there are n't any people visible
8. not really
9. no, there are n't any people
10. no people in the room
Question:are the elephants eating?
1. no
2. yes
3. nope
4. no, don't think so
1. no does n't look like it
2. i do n't think so
3. not that i can see
4. no standing
5. it looks like it
6. only the 1 with its head down is
7. i think so
8. yes they appear to be
9. yes, they appear to be
10. 1 of the 2 of them are
Question: is it a cold drink?
1. no
2. yes
3. no it isn
4. nope
1. looks that way
2. i do n't think so
3. no not really
4. i can not tell
5. looks like it
6. looks like it is
7. no sure
8. yes it is
9. looks like
10. it appears to be
Question:do they look scary?
1. no
2. not really
3. yes kind of
4. yes
5. not particularly
6. nope
1. not sure
2. i do n't think so
3. not that i can see
4. i can not tell
5. hard to say
6. maybe
7. i can't tell
8. can't tell
9. i can't see any
10. they look serious
Question:are there any other animals in the picture?
1. no
2.0
3. nope
4. 0 at all
5. not that i can see
1. no, there are 0
2. i can't see any
3. no, just the 1
4. i do n't think so
5. nope, just 1
6. 0 sen
7. no animals
8. no animals either
9. no other people
0. no people
Question:does it look delicious? MRR candidate set
1. yes
2. yes it does
3. yes, it is
4. looks okay
5. it does yes
6. yes, making me hungry
1. it's okay
2. not really
3. no not really
4.UNK
5. no it does not
6. no
7. i think so
8. no really
9. nope
10. no it's not
Question:are the kids playing with anything? MRR candidate set
1. holding umbrellas
2. no
3. yes
4. looks like a boy has something in his hand
5. nope
6. not that i can see
1. no, all are kids
2. i do n't think so
3. i can not tell
4. not really
5. yes i think so
6. nothing visible
7. yes, they are
8. i can't tell that
9. yes, a bunch of umbrellas
10. yes they are
Question:are they older or younger? MRR candidate set
1. they look older
2. i think younger
3. young
4. can't tell
5. don't know
6. i am not sure
7. i can not tell
8. i can't tell
1. not sure
2. looks like 1 of each
3. medium
4. they look like adults mostly
5. can't see their faces
6. 1 looks old while the other looks mid 30s
7. very old
8. they look fairly big
9. no
10. i can't tell, but judging by the UNK, probably
Question: is it a metal fence?
1. yes
2. wire
3. yes it is
4. it appears to be
5. yes, it is
1. it is
2. yes, it looks like it is
3. it looks like it
4. looks like it is
5. i think so it
6. it has some lines, but yes
7. i think so
8. yes they are
9. yes, i can see a fence
10. yes i think so , hard to tell
Question:are there pictures on the walls?
1. yes
2. no
3. yes there are
4. yes for sure
1. i think so
2. yes, 1 that i can see
3. not that i can see
4. nope
5. no pictures on the wall
6. yeah, 3
7.1
8. might be but it is hard to see
9. no, i do n't see any
10. yes, but not too many
Question:can you see clouds? MRR candidate set
1. i see no clouds the sky is fairly grey though
2. no clouds are visible
3. yes i can see the sky cloudy
4. no
5. no , it looks cloudy
1.no,
2. there seems to be a few clouds
3. nope
4. yes they are clouds
5. very few
6. not that i can see
7. not really
8. i do n't think so
9. yes i can
Question:where are the onions? MRR candidate set
1. on the right of the plate
2. you can not really tell
3. i can not tell
4. not sure
5. i can't tell
1. yes
2. can't tell
3.UNK
4. sitting at the table
5. it looks like a box of them and 2 sitting out
6. there is squash, shredded carrots, and shredded cabbage
7. not that i can see
8. the UNK of heaven
9. yes, it is
10. i think os
Question: does she have brown hair? MRR candidate set
1. yes
2. yes, she does
3. yes, it is
4. i believe so
5. i think so
6. oh yeah
1. black
2. yes and black
3. no
4. no, her eyes are brown
5. nope
6. it looks to be maybe of shoulder length
7. can't tell
8. i can't tell
9. yes i do
10. i can not tell
Question:are there other people? MRR candidate set
1. there is a gentleman beside her
2. no
3. yes, there are 2 other people in the background
4. no, there is not
5. not that i can see
6. no there are not
7. nope
1. no other people in the photo
2. 0 that i can see in the picture
3. no, there are no other people com
4. 0 i can see
5. no there's not
6. not from what i can see
7. no, i can't see anyone
8. not within view
9. yes 2 people can be seen in the background
10. i don't think so
Question: how many zebras are there? MRR candidate set
1.9
2. i count twelve
3. 10
4. 7
5. 8 or so
6. too many to count
1. about ten, of different species
2. eleven
3.at least 20
4. 11 or 12
5. looks like around a dozen
6. a lot
7. over 20
8. about 12-14
9. about 25
10. i see 6 i think
Question: is the yard large?
1. i can not tell
2. yes
3. i can't tell
4. can't tell
5. can not tell too close up
6. not sure
1. from what i can see
2. seems like it very close up photo
3. i think so
4. maybe
5. no
6. no, but hard to tell from the close up
7. yes it is
8. yes, it is
9. not really
10. yes it's large
Question:does the picture match the caption?
1. yes
2. no
3. yes, it does
4. yes it does match the caption
5. yes it does
6. no it is not
1. nope
2. no,
3. not really
4. it does
5. yes, the picture is in color
6. yes, it's in color
7. it is
8. yes, it is
9. yes it is
10. i think so
Question:are there any lights?
1. no
2. i think it's light coming from a camera he is holding
3. yes
4. not that i can see
5. 0 that i can see
6. nope
1. not seen
2. i can not see them
3. i do n't think so
4. yes, well lit
5. i can't tell
6. 0 at all
7. i think so
8. i can not tell