SlowGuess/NAACL_2022_OCR huggingface dataset:
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Wei Lu and Hwee Tou Ng. 2010. Better punctuation prediction with dynamic conditional random fields. In Proceedings of the 2010 conference on empirical methods in natural language processing, pages 177-186.
Karan Makhija, Thi-Nga Ho, and Eng-Siong Chng. 2019. Transfer learning for punctuation prediction. In 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pages 268-273.
Iain McCowan, Jean Carletta, Wessel Kraaij, Simone Ashby, S Bourban, M Flynn, M Guillemot, Thomas Hain, J Kadlec, Vasilis Karaiskos, et al. 2005. The ami meeting corpus. In Proceedings of the 5th international conference on methods and techniques in behavioral research, volume 88, page 100. CiteSeer.
Minh Van Nguyen, Viet Duc Lai, Amir Pouran Ben Vey-seh, and Thien Huu Nguyen. 2021. Trankit: A lightweight transformer-based toolkit for multilingual natural language processing. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 80-90, Online. Association for Computational Linguistics.
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 1659-1666, Portoroz, Slovenia. European Language Resources Association (ELRA).
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206-5210. IEEE.
Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101-108, Online. Association for Computational Linguistics.
György Szaszák and Mate Akos Tündik. 2019. Leveraging a character, word and prosody triplet for an asr error robust and agglutination friendly punctuation approach. In *INTERSPEECH*, pages 2988-2992.
Jörg Tiedemann. 2016. Finding alternative translations in a large corpus of movie subtitle. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 3518-3522.
Chenglin Xu, Lei Xie, Guangpu Huang, Xiong Xiao, Eng Siong Chng, and Haizhou Li. 2014. A deep neural network approach for sentence boundary detection in broadcast news. In Fifteenth annual conference of the international speech communication association. Citeseer.
Dongdong Zhang, Shuangzhi Wu, Nan Yang, and Mu Li. 2013. Punctuation prediction with transition-based parsing. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 752-760.
A Annotation Tool
We develop a customized web-based annotation tool for this work. The annotation tool focuses on improving the readability of the annotated text, as the result, improves the annotation quality. Toward this end, we use color coding for punctuation markers. More importantly, whenever a sentence ending marker is assigned, such as period, question mark, and exclamation, it automatically creates a new line to separate sentences. Figure 2 shows the interface and an annotated text using our tool.
B Annotation Guideline
This section summarizes the taxonomy, annotation guideline, and annotation examples. Examples are shown in figure 2.
A period is used for:
- Marking the end of a declarative sentence.
- Separating independent clauses without a conjunction when a semi-colon is usually used (to distinguish with the case that a comma is used when a conjunction presents).
A question mark is used for:
- Marking the end of a question.
An exclamation mark is used for:
- Exclaiming something. They are commonly used after interjections (words or phrases that are used to exclaim, command, or protest like "wow" or "oh").
- Express the following emotions: excitement, surprise, astonishment, emphasizing a point, and other types of strong emotions.
A comma is used for:
- Separating independent clauses when they are joined by any of these seven coordinating conjunctions: and, but, for, or, nor, so, yet.
- Separating introductory clauses, phrases, or words from the main clause.
- Setting off clauses, phrases, and words that are not essential to the meaning of the sentence. Use one comma before to indicate the beginning of the pause and one at the end to indicate the end of the pause.
- Separating three or more words, phrases, or clauses written in a series.
- Separating two or more coordinate adjectives that describe the same noun. Be sure never to add an extra comma between the final adjective and the noun itself or to use commas with non-coordinated adjectives.
- Separating contrasted coordinate elements or to indicate a distinct pause or shift near the end of a sentence.
- Setting off phrases at the end of the sentence that refers back to the beginning or middle of the sentence. Such phrases are free modifiers that can be placed anywhere in the sentence without causing confusion.
- Setting off all geographical names, items in dates (except the month and day), addresses (except the street number and name), and titles in names.
- Shifting between the main discourse and a quotation.
- Preventing possible confusion or misreading.
Figure 2: The colorful, easy-to-use interface of our annotation tool designed for the annotation of BehaviorPR. The color codes for comma, period, exclamation mark, and question mark are orange, sky blue, light green, and yellow, respectively.