nltk/nltk — Issue #137: Reading Negra corpus files with a NegraCorpusReader
Author: alexrudnick
Created: 2012-01-17 06:26:55+00:00
State: open

The NegraCorpusReader enables reading corpora files created by annotate, a tool for efficient semi-automatic annotation of corpus data. Available corpora in this format are the German NEGRA and German TIGER corpora. Further information on the corpora and where to obtain them can be found here:

http://www.coli.uni-saarland.de/projects/sfb378/negra-corpus/negra-corpus.html
http://www.ims.uni-stuttgart.de/projekte/TIGER/TIGERCorpus/

Information on the annotate tool can be found at

http://www.coli.uni-saarland.de/projects/sfb378/negra-corpus/annotate.html

A NEGRA corpus file consists out of annotated sentences separated by #BOS (beginning of sentence) and #EOS (end of sentence) markers on their own line. Each sentence consists of words and columns containing the lemma, tag, chunk and/or morphological tag. Each word and its tag information has its own line.

Here is an example of the Negra corpus and its format

#BOS 2 2 899973978 1
Sie	PPER	3.Pl.*.Nom	SB	504
gehen	VVFIN	3.Pl.Pres.Ind	HD	504
gewagte	ADJA	Pos.*.Akk.Pl.St	NK	500
Verbindungen	NN	Fem.Akk.Pl.*	NK	500
und	KON	--	CD	502
Risiken	NN	Neut.Akk.Pl.*	CJ	502
ein	PTKVZ	--	SVP	504
,	$,	--	--	0
versuchen	VVFIN	3.Pl.Pres.Ind	HD	505
ihre	PPOSAT	*.Akk.Pl	NK	501
Möglichkeiten	NN	Fem.Akk.Pl.*	NK	501
auszureizen	VVIZU	--	HD	503
.	$.	--	--	0
#500	NP	--	CJ	502
#501	NP	--	OA	503
#502	CNP	--	OA	504
#503	VP	--	OC	505
#504	S	--	CJ	506
#505	S	--	CJ	506
#506	CS	--	--	0
#EOS 2
(http://www.coli.uni-saarland.de/projects/sfb378/negra-corpus/corpus-sample.export)

Because of similar corpus structure, this reader is based on and very
similar to the ConllCorpusReader. Both corpora have their tokens structured
as grid. NEGRA corpus file provide more token information though.

The NegraCorpusReader can be found at

https://github.com/ptnplanet/NLTK-Contributions

It is licensed under the Apache 2.0 Licence to enable the use with and within the NLTK.

Migrated from http://code.google.com/p/nltk/issues/detail?id=623

earlier comments

andreas@unstable.nl said, at 2011-06-05T11:57:49.000Z:

I've also written a corpus reader for the Negra export format, with the important difference that mine also gives you the complete parse trees. Since these can contain crossing branches, the terminals are indicated by indices. The code is at:

https://github.com/andreasvc/disco-dop/blob/master/negra.py