SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 6: Detailed examples of different prompting strategies. "Zero-shot Prompting" refers to our default prompting strategy, where prompts are in each specific language. "Few-shot Prompting" involves few-shot in-context examples, which appear before the final test sample. "English-instruction" uses English instructions instead of instructions in the target language. "English-Translation" uses the translated English data of other languages.
A.4 Details on exam papers
We list the specific exams we collected for constructing the datasets in Table 5.
A.5 Examples questions of each language in M3Exam
We list example questions of each language from Figure 8 to Figure 16.
The following are multiple-choice questions about social-science.
Mahatma Gandhi Spinning Yarn in the Late 1920s
Source: Carnegie Council for Ethics in International Affairs, January 15, 1929"
This photograph is most often associated with the term
(A) appeasement
(B) containment
(C) boycott
(D) segregation
Fromage: D / Flamingo: D
BLIP-2: C / InstructBLIP: C
Fromage: This is a textile, and not a photograph.
Flamingo: The picture shows famous Bharat  
Ratra(Indiau2019s premiere civilian award) awardee,  
Rashtrapati Dr. A.P.J. Abdul Kalam, better known as Missile  
Man of India, preparing tea for his students!! How is this a  
question? No description is given. No mention is made of  
details such as the figures, texts, and numbers are showing.  
The important details in this question are only the figures and  
texts, and a hint concerning the nature of the numbers. The  
figures and texts refer to the details of the image: Dr Kalam  
is holding a tea
BLIP-2: hand spinner who is seen wearing saffron robes was satyagrahis
Instruct-BLIP: man spinning wheel
The following are multiple-choice questions about math.
In her math class, Carla used unit cubes to build a right rectangular prism with a volume of 24 cubic units. The height of the prism was two units. Which figure could be the bottom layer of the prism?
(C)
Figure 8: An example question in English, with the correct answer in bold and blue.
(D)
Figure 7: Two examples of questions involving images. We show the original questions, the model outputs, as well as the model responses when asked to describe the images in detail.
Fromage: A / Flamingo: B
BLIP-2:A /InstructBLIP:A
Fromage:
(A) The structure has equal squares which are all arranged according to their mental angle.
(C) The illustration indicates the general arrangements of a rectangle.
Flamingo:
(A) We can see a 12 cube. Then there are three figures making up the cube in various positions. One cube is facing up. One cube is facing right. And one cube is facing down.
(B) This CD shows a period of four years. The four years are represented by stages. Each stage is represented by a box
BLIP-2:
(A) a drawing of white square blocks each one square
(B) The image shows an illustration of a  $4 \times 4$  cube with a line coming from one side going over to the other
Instruct-BLIP:
(A)-(D) cubes
Gravity is the major force that ( )
(A) keeps planets in their orbits
(B) affects melting of a solid
(C) keeps sunlight in Earth's atmosphere
(D) affects evaporation of water from oceans
img:no
Table 5: List of exams (translated into English) for constructing the M3Exam dataset.
Language	Level	Exam
English	low	New York State Testing Program Grade 5
mid	New York State Testing Program Grade 8
high	High School Regents Examinations
Chinese	low	Beijing Admission Examination for Junior High Schools (grade 6)
mid	Beijing Senior High School Entrance Examination (grade 9)
high	National College Entrance Examination (NCEE) (grade 12)
Italian	low	State exam of Primary School (grade 5)
mid	State exam of First Level Secondary School (grade 8)
high	State exam of Second Level Secondary School (grade 10)
Portuguese	low	Primary School Exams (grade 5 and 6)
mid	National Youth and Adult Competency Certification Exam (Encejea), Primary Education (grade 9)
high	National High School Exam (ENEM)
Vietnamese	low	Primary School Semester II Final Exam (grade 5)
mid	Secondary Graduation Exam, Secondary School Semester II Final Exam (grade 9)
high	National High School Graduation Exam (grade 12)
Thai	low	O-NET (Ordinary National Educational Test) for Primary 6 (grade 6)
mid	O-NET (Ordinary National Educational Test) for Secondary 3 (grade 9)
high	O-NET (Ordinary National Educational Test) for Secondary 6 (grade 12)
Swahili	low	Kenya National Examinations Board Exam (KNEB), Kenya Primary School Education Assessment (KPSE) (grade 6)
mid	Kenya Certificate of Primary Education (KCPE) (grade 8)
Afrikaans	low	Provincial Examinations, Intermediate Phase (grade 6)
mid	Provincial Examinations, Senior Phase (grade 9)
high	National Senior Certificate (grade 12)
Javanese	low	End of Semester I/II Exam (UAS), School Final Exam for Elementary School (grade 6)
mid	End of Semester I/II Exam (UAS), School Final Exam for Junior High School (grade 9)
下图是1978年与1986年北京郊区男户主职业占比变化情况。这一变化的产生主要是由于（）
(A)城市经济体制改革开始酝酿
(B) 农村经济体制改革深化
(C) 城乡之间的差异呈缩小趋势
(D)城市产业结构日益完善
Translated to English:
The figure below shows the change in the occupational proportion of male household heads in the suburbs of Beijing between 1978 and 1986. This change is mainly because ( )
(A) The reform of the urban economic system is brewing
(B) The reform of the rural economic system is deepening
(C) The gap between urban and rural areas is narrowing
(D) The urban industrial structure is improving day by day
Figure 9: An example question in Chinese, with the correct answer in bold and blue.
Osserva la segunte figura.
Figure 10: An example question in Italian, with the correct answer in bold and blue.
Quale tra le seguenti misure rappresenta la stima migliorere dell'area della figura?
(A)  $20\mathrm{cm}^2$
(B)  $24~\mathrm{cm}^2$
(C)  $26~\mathrm{cm}^2$
(D)  $30~\mathrm{cm}^2$
Observe the following figure.
Which of the following measures is the best estimate of the area of the figure?
(A)  $20~\mathrm{cm}^2$
(B)  $24~\mathrm{cm}^2$
(C)  $26~\mathrm{cm}^2$
(D)  $30~\mathrm{cm}^2$
img:yes
Monday September 20, 2010
RIDGWAY, L. Disponível em: http://fborfw.com. Aceso em: 23 fev. 2012.
Figure 11: An example question in Portuguese, with the correct answer in bold and blue.
Figure 12: An example question in Vietnamese, with the correct answer in bold and blue.
Na tira da série For better or for worse, a comunicação entre as personagens fica comprometida em um determinado momento porque
(A) as dasasamigadivergemdeopinionsobrefutebol
(B) una das amigas decide as preferências da outras.
(C) una das amigas ignora que o outono é temporada de futebol.
(D) uma das amigas desconhece a razão pela qual a outras a maltrata.
(E) as dosasamigasattribuem sentidosdiferentesaPALAVRA season.
In the series strip For better or for worse, the communication between the characters is compromised at a certain moment because
(A) the two friends have different opinions about soccer
(B) one of the friends disregards the preferences of the other
(C) one of the friends ignores that fall is football season
(D) one of the friends does not know why the other mistreats her
(E) the two friends attribute different meanings to the word season
Trong hé tiêu hóa cua nguroi, duói tác dong cua enzim tiêu hóa, chat não sau day duyc bien doi thangh glixérol va axit béo?
(A) Protéin
(B)TinhBOT
(C) Saccaròzō
(D) Lipit
In the human digestive system, as an effect of digestive enzymes, which of the following substances is broken down into glycerol and fatty acids?
(A) Protein
(B) Starch
(C) Sucrose
(D) Lipids
"unnuuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnuunnu
(A)
(B)
(C)
(D)
"A group of tourists travel by boat, admiring beautiful landscapes, such as Tham Lot Cave, Khao Phing Kan, Khao Ta-Pu, and also visiting the fossils at the Shell Cemetery." In which part of Thailand are these tourists travelling?
(A) Gulf of Thailand coast
(B) Andaman Sea coast
(C) Eastern Seaboard coast
(D) Eastern coast of Southern Thailand
Figure 13: An example question in Thai, with the correct answer in bold and blue.
Mkono wazi ni kwa ukarimu ilhali _ ni kwa inda.
(A) jicho la nje
(B) joka la mdimu
(C) kifauongo
(D) kisebusebu
An open hand is for generosity whereas _______ is for kindness.
(A) an outsider's eye
(B) the snake of a lime tree
(C) the shame-plant (Memusa pudica)
(D) feigning dislike
Figure 14: An example question in Swahili, with the correct answer in bold and blue.
Watter van die volgende beskryf Groot depressie van 1929 die Beste?
(A)'n Ekonomiese insinking
(B) Begin in die VSA
(C) die hele wèrd geraak
(D) Al bogenoemde
Which of the following best describes the Great Depression of 1929?
(A) An economic slump
(B) Began in the USA
(C) Affected the whole world
(D) All of the above
Figure 15: An example question in Afrikaans, with the correct answer in bold and blue.
Tulisan aksara jawa ing dhuwur yen ditulis aksara latin ...
(A) ugur kuna sejati
(B) gugup rukun jati
(C) nunut kurun sejati
(D) guyup rukun sejati
The phrase in Javanese script above written in the Latin alphabet is ...
Figure 16: An example question in Javanese, with the correct answer in bold and blue.