Merging Data, The Essence Of Creation Of Multi-Layer Corpora
Creators: name: Zipser, Florian
name: Frank, Mario
name: Schmolling, Jakob
Publisher: name: Zenodo
Year: 2014
Subjects: Pepper
Salt
ANNIS
converter
conversion framework
linguistics
linguistic data
linguistic format
merging
corpora
multi-layer corpus

description: The last couple of years have shown an increasing number of multi layer corpora. Such<br>
corpora allow the analysis of phenomena spreading through multiple annotation layers, for<br>
example corpora like TueBaDZ (see: http://www.sfs.uni-tuebingen.de/ascl/ressourcen/corpora/tueba-dz.html), PCC (see: http://www.ling.uni-potsdam.de/acl-lab/Forsch/pcc/pcc.html), FALKO (see: https://u.hu-berlin.de/falko) and many other corpora contain annotations on syntactical, rhetorical, information structural and other layers. Often, annotations were created manually, or semi-automatically with different tools like EXMARaLDA (see: http://www.exmaralda.org/), TreeTagger (see: http://www.cis.uni-muenchen.de/~schmid/tools/TreeTagger/) and MMAX2 (see: http://mmax2.sourceforge.net/).<br>
These tools are powerful and usable but unfortunately only provide a minimum of<br>
interoperability, which impedes the creation of multi layer corpora. Thus, multiple layers of<br>
such corpora often had to be merged by hand or by very proprietary scripts implemented for<br>
just one use case and therefore could not be reused for other corpora easily.<br>
With this poster we present a tool which merges several layers of annotations into a single<br>
multi layer corpus. When creating a multi layer corpus, several analyses base on the same<br>
primary data and often also on the same tokenization.<br>
We started merging the data on the tokenization level and traversed bottom up, to merge even<br>
higher levels of annotation. This concept is implemented in a module for the converter<br>
framework Pepper (see: https://u.hu-berlin.de/saltnpepper) with use of the common meta-<br>
model Salt. By using Pepper, the merging module is able to handle all formats which can be<br>
imported by a Pepper module. Multi layered corpora then can be mapped into a multilayer<br>
formats like PAULA (Chiarcos et al. 2008), GrAF (Ide &amp; Suderman 2007) or can be imported<br>
into ANNIS.
description: {"references": ["Dipper S. (2005). XML-based Stand-off Representation and Exploitation of Multi-Level", "Linguistic Annotation. In: Eckstein R., Tolksdorf R. (eds.) Berliner XML Tage.", "Ide N.&amp; Suderman K.(2007). GrAF: A Graph-based Format for Linguistic Annotations. In:", "Proceedings of the Linguistic Annotation Workshop, Prague, Czech Republic.", "Schmid, H. (1995). Improvements in Part-of-Speech Tagging with an Application to German.", "Proceedings of the ACL SIGDAT-Workshop. Dublin, Ireland.", "Stede M. (2004). The Potsdam commentary corpus. In Proceedings of the 2004 ACL", "Workshop on Discourse Annotation (DiscAnnotation '04), Bonnie Webber and Donna Byron", "(Eds.). Association for Computational Linguistics, Stroudsburg, PA, USA, 96-102.", "Telljohann, H./Hinrichs, E. W./K\u00fcbler, S./Zinsmeister, H./Beck, K. (2009). Stylebook for the", "T\u00fcbingen Treebank of Written German (T\u00fcBa-D/Z). Universit\u00e4t T\u00fcbingen Seminar f\u00fcr", "Sprachwissenschaft.", "Reznicek, M.; L\u00fcdeling, A.; Krummes, C.; Schwantuschke, F.; Walter, M.; Schmidt, K.;", "Hirschmann, H.; Andreas, T. (2012). Das Falko-Handbuch. Korpusaufbau und Annotationen", "Version 2.01", "Zeldes, Amir, Ritz, Julia, L\u00fcdeling, Anke &amp; Chiarcos, Christian (2009). \"ANNIS: A Search", "Tool for Multi-Layer Annotated Corpora\". In: Proceedings of Corpus Linguistics 2009, July", "20-23, Liverpool, UK.", "Zipser F., Romary L. (2010). A model oriented approach to the mapping of annotation formats", "using standards In: Proceedings of the Workshop on Language Resource and Language", "Technology Standards, LREC 2010. Malta. URL:", "http://hal.archives-ouvertes.fr/inria-00527799/en/", "Zipser F., Zeldes A., Ritz J., Romary L. &amp; Leser U. (2011). Pepper: Handling a multiverse of", "formats 33. Jahrestagung der Deutschen Gesellschaft f\u00fcr Sprachwissenschaft. G\u00f6ttingen,", "23.- 25. Februar 2011"]}