SlowGuess/ACL_2020_OCR huggingface dataset:
Fei Dong, Yue Zhang, and Jie Yang. 2017. Attention-based recurrent convolutional neural network for automatic essay scoring. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 153-162.
Mohammad K Ebrahimpour, Jiayun Li, Yen-Yun Yu, Jackson Reese, Azadeh Moghtaderi, Ming-Hsuan Yang, and David C Noelle. 2019. Ventral-dorsal neural networks: Object detection via selective attention. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 986-994. IEEE.
Youmna Farag, Helen Yannakoudakis, and Ted Briscoe. 2018. Neural automated essay scoring and coherence modeling for adversarially crafted input. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 263-271.
Corina Florescu and Cornelia Caragea. 2017. Position-rank: An unsupervised approach to keyphrase extraction from scholarly documents. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1105-1115.
Corina Florescu and Wei Jin. 2018. Learning feature representations for keyphrase extraction. In Thirty-Second AAAI Conference on Artificial Intelligence.
Peter W Foltz and Mark Rosenstein. 2015. Analysis of a large-scale formative writing assessment system with automated feedback. In Proceedings of the Second (2015) ACM Conference on Learning@ Scale, pages 339-342. ACM.
Cancan Jin, Ben He, Kai Hui, and Le Sun. 2018. Tdnn: a two-stage deep neural network for prompt-independent automated essay scoring. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1088-1097.
Beata Beigman Klebanov, Nitin Madnani, Jill Burstein, and Swapna Somasundaran. 2014. Content importance models for scoring writing from sources. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 247-252.
Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 107-117.
Xia Li, Minping Chen, Jianyun Nie, Zhenxing Liu, Ziheng Feng, and Yingdan Cai. 2018. Coherence-based automated essay scoring using self-attention. In Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, pages 386-397. Springer.
Nitin Madnani, Jill Burstein, Norbert Elliot, Beata Beigman Klebanov, Diane Napolitano, Slava Andreyev, and Maxwell Schwartz. 2018. Writing mentor: Self-regulated writing feedback for struggling writers. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, pages 113-117.
Debanjan Mahata, John Kuriakose, Rajiv Ratn Shah, and Roger Zimmermann. 2018. Key2vec: Automatic ranked keyphrase extraction from scientific articles using phrase embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 634-639.
Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. 2017. Deep keyphrase generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 582-592.
Zahra Rahimi and Diane Litman. 2016. Automatically extracting topical components for a response-to-text writing assessment. In Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, pages 277-282.
Rod D Roscoe, Laura K Allen, Jennifer L Weston, Scott A Crossley, and Danielle S McNamara. 2014. The writing pal intelligent tutoring system: Usability testing and development. Computers and Composition, 34:39-59.
Antonette Shibani, Simon Knight, and Simon Buckingham Shum. 2019. Contextualizable learning analytics design: A generic model and writing analytics evaluations. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge, pages 210-219. ACM.
Kaveh Taghipour and Hwee Tou Ng. 2016. A neural approach to automated essay scoring. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1882-1891.
Yi Tay, Minh C Phan, Luu Anh Tuan, and Siu Cheung Hui. 2018. Skipflow: incorporating neural coherence features for end-to-end automatic text scoring. In Thirty-Second AAAI Conference on Artificial Intelligence.
Bronwyn Woods, David Adamson, Shayne Miel, and Elijah Mayfield. 2017. Formative essay feedback using predictive scoring models. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2071-2080. ACM.
Peng Yuxin, He Xiangteng, and Zhao Junjie. 2018. Object-part attention model for fine-grained image classification. IEEE transactions on image processing: a publication of the IEEE Signal Processing Society, 27(3):1487-1500.
8574
Haoran Zhang and Diane Litman. 2017. Word embedding for response-to-text assessment of evidence. In Proceedings of ACL 2017, Student Research Workshop, pages 75-81.
Haoran Zhang and Diane Litman. 2018. Co-attention based neural network for source-dependent essay scoring. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 399-409.
Haoran Zhang, Ahmed Magooda, Diane Litman, Richard Correnti, Elaine Wang, LC Matsmura, Emily Howe, and Rafael Quintana. 2019. erevise: Using natural language processing to provide formative feedback on text evidence usage in student writing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 9619-9625.
8575
A.1 Topic Words Results
Table 9 shows all topic words for the  $RTA_{MVP}$  from  $TC_{manual}$ . Table 10 shows all topic words for the  $RTA_{MVP}$  from  $TC_{lda}$ . Table 11 shows all topic words for the  $RTA_{MVP}$  from  $TC_{pr}$ . Table 12 shows all topic words for the  $RTA_{MVP}$  from  $TC_{attn}$ .
A.2 Specific Example Phrases Results
Table 13 shows all specific example phrases for the  $RTA_{MVP}$  from  $TC_{manual}$ . Table 14 shows all specific example phrases for the  $RTA_{MVP}$  from  $TC_{lda}$ . Table 15 shows all specific example phrases for the  $RTA_{MVP}$  from  $TC_{pr}$ . Table 16 shows all specific example phrases for the  $RTA_{MVP}$  from  $TC_{attn}$ .
8576
Topic 1	Topic 2	Topic 3	Topic 4
care	bed	farmer	school
health	net	fertilizer	supplies
hospital	malaria	irrigation	fee
treatment	infect	dying	student
doctor	bednet	crop	midday
electricity	mosquito	seed	meal
disease	bug	water	lunch
water	sleeping	harvest	supply
sick	die	hungry	book
medicine	cheap	feed	paper
generator	infect	food	pencil
no	biting	energy
die	free
kid	children
bed	kid
patient	go
clinical officer	attend
running
Table 9: Topic words of  $TC_{manual}$ .
8577
Topic 1	Topic 2	Topic 3	Topic 4	Topic 5	Topic 6	Topic 7	Topic 8	Topic 9
help	kenya	poverty	food	money	school	people	hospital	years
poor	like	think	fertilizer	need	kids	sauri	medicine	africa
world	better	author	crops	nets	supplies	malaria	hospitals	project
good	know	lifetime	water	thing	children	sick	water	villages
things	life	article	farmers	afford	schools	2008	free	sauri
time	help	possible	needed	donate	lunch	disease	electricity	village
work	think	convinced	grow	right	education	2004	diseases	helped
hard	sauri	fight	dying	dollar	afford	nets	medicines	change
going	live	proverty	problem	treatment	energy	mosquitoes	doctors	lives
alot	clothes	said	family	survive	learn	getting	2008	goals
reason	states	achievable	families	needs	students	says	gave	improved
happen	place	time	stop	stuff	went	years	doctor	2015
helping	health	convince	lack	person	adults	progress	examples	help
goal	important	believe	hunger	cause	fees	died	2004	changed
believe	feel	hannah	tools	patients	parents	text	shape	year
problems	happy	shows	seeds	provide	2004	away	cure	changes
countries	tell	reasons	plants	cost	lunches	mosquitoes	running	started
difference	care	convincing	fertilizers	beds	books	prevent	treat	great
places	shoes	fighting	farming	means	home	treated	support	millennium
change	story	wrote	able	dont	wanted	dieing	common	progress
little	america	story	solved	dollars	chores	said	beds	came
improve	ways	agree	supply	medical	meal	come	patients	girl
country	wants	saying	irrigation	jobs	wood	night	said	2025
achieve	makes	opinion	won't	everyday	materials	bite	generator	place
hope	clothing	winning	afford	gone	learning	death	clean	program
helps	community	sachs	hungry	doctors	able	sleep	electricity	tells
everybody	economy	progress	plant	lots	supplies	impoverished	giving	small
start	history	conclusion	look	sickness	meals	living	drink	millenium
easy	paragraph	says	farms	live	paper	amazing	cures	read
making	that's	future	feed	fact	attendance	easily	evidence	happened
Table 10: Topic words of  $TC_{lda}$ .
8578
Topic 1	Topic 2	Topic 3	Topic 4	Topic 5	Topic 6	Topic 7	Topic 8	Topic 9	Topic 10	Topic 11	Topic 12	Topic 13	Topic 14	Topic 15	Topic 16	Topic 17	Topic 18	Topic 19
irrigation	road	diseases	adults	fight	development	joyful	people	midday	village	millennium	backs	plenty	doctor	thing	paper	end	work	sleeping
fertilizer	brighter	medicine	lifetime	villages	dirt	kids	school	women	access	hospital	supplies	world	bed
farmers	future	malaria	project	jump	fees	ground	care	shape	chores	net
crops	hannah	disease	goals	bar	students	bananas	medicines	patients	books	nets
plant	car	mosquitoes	plan	music	meal	cloth	schools	treatment	pencils	site
seeds	sauri	charge	economy	singing	energy	mothers	today	officer
outcome	market	quality	everyone	lunch	feet	supply	water
lack	year	supporters	dancing	clothing	areas	electricity
tools	time	help	day	kind	generator
place	health	rooms
years	advice	family
poverty	items
life	targets
communities	death
leaders	night
glimpse	costs
africa	die
chemicals	knowledge
solutions	food
millions	parents
Table 11: Topic words of  $TC_{pr}$ .
8579
Topic 1	Topic 2	Topic 3	Topic 4	Topic 5	Topic 6	Topic 7	Topic 8	Topic 9	Topic 10	Topic 11	Topic 12	Topic 13	Topic 14	Topic 15	Topic 16
poverty
fight
winning	way
would
rate
attendance	years
four
villages
80
progress
last	lunch
serves
parents
attend
passed	goals
problems
day
cloth
three
made
books	electricity
water
generator
also
running
energy
connected	supplies
food
net
rooms
packed
patients
needed
5
keep
poor
five
like
come
little
treatment
treat
harvest
showed
cheap	afford
lifetime
could
achieve
together
malaria
take
future
worked
care
family
hard
good
doctor
either
whole
save
money
easy
met	many
people
kenya
sauri
pencils
africa
yet
sachs
though
feed
two
health
set
crisis
areas
items	free
medicine
crops
charge
farmers
medicines	school
schools
fees
students	hospital
2004
disease
yala	bed
nets
used
every
sleeping
site
midday
meal
dramatic
change
clinical
officer
tattered
clothing
chemicals
malarial
preventable
treatable
costs	project
world
millennium
village
across
work
end
worry
supporters
time
2025	supply
maintain
diseases
hunger
lives
adults
life
dying
death
away
treated	fertilizer
seeds
addressed
irrigation
necessary
tools
lack
plenty
plant
common
become
go
get
place
solutions
really
targets
see
die
hungry
dancing
walked
bare
feet
hannah
impoverished
encouraging
probably	80
occurred
year
changes
outcome
today
first
along
Table 12: Topic words of  $TC_{attn}$ .
8580
Category 1	Category 2	Category 3	Category 4
unpaved roads	united nations intervention	yala sub district hospital	malaria common disease preventable treatable
tattered clothing	safer healthier better life	three kids bed two adults rooms packed patients	mosquitoes carry malaria infect people biting
bare feet	out poverty stabilize economy quality life communities	not medicine treatment could afford	kids die malaria adults sick 20 000 day
less than 1 dollar day	africa kenya sauri	no doctor only clinical officer running hospital	bed nets mosquitoes away people save millions lives
goals met 2015 2025	no running water electricity	bed nets cost 5 dollar
80 villages across sub-sahara africa	sad people dying near death preventable	cheap medicines treat malaria
Category 5	Category 6	Category 7	Category 8
crops dying	kids not attend go school	progress just four years	progress encouraging supporters
not afford fertilizer irrigation	not afford school fees	yala sub district hospital has medicine	solutions problems keep people impoverished
outcome poor crops	kids help chores fetching water wood	medicine free charge	change poverty stricken areas good
lack fertilizer water	schools minimal supplies books paper pencils	medicine most common diseases	poverty history not easy task hard
enough food crops harvest feed whole family hungry sick	concentrate not energy	water connected hospital	winning against poverty possible achievable lifetime
no midday meal lunch	hospital generator electricity
bed nets used every sleeping site
hunger crisis addressed fertilizer seeds
tools needed maintain food supply
kids go school now
no school fees
now serves lunch students
school attendance rate way up
Table 13: Specific example phrases of  $TC_{manual}$ .
8581
Table 14: Specific example phrases of  $TC_{lda}$ .
8582
Category 1
brighter future hannah
millennium villages project
unpaved dirt road
bar sauri primary school
future hannah
sauri primary school
villages project
millennium development goals
village leaders
dirt road
car jump
little kids
preventable diseases people
many kids
diseases people
kids die
school supplies
primary school
school fees
infect people
Table 15: Specific example phrases of  $TC_{pr}$ .
Table 16: Specific example phrases of  $TC_{attn}$ .
8584