kowey/educe — pull request #14 by moreymat (2014-10-31 16:03:31+00:00)
Title: Clean PTB trees: remove empty nodes, strip functional tags

The Penn Treebank contains information that parsers do not use for training nor output:
* empty nodes (for traces),
* grammatical functions.

This PR provides the infrastructure to discard such information so that trees read from the Penn Treebank contain the same information as trees output by any parser, e.g. the Stanford Parser.

Trees without empty nodes are also easier to align with discourse annotations, cf. #13 .