Release 1.4.0
Added command=compare for pos-tagger output
Added LexiconChain, LexiconFile, LexiconSerializer and LexiconDeserializer for simple handling of multiple lexicons
Changed RegexMarkerFilter to apply to the entire regex by default (rather than to the last group - Not backwards compatible)
Added option evaluationPatternFile/evaluationPattern for command=compare, to allow different pattern in ref file and evaluation file
Added option labeledEvaluation to allow labaled/unlabeled evaluation of parse output
Bug: TalismaneConfig: Fixed several filter bugs
Bug: RollingSentenceProcessor: fixed bug when log-level = trace
Bug: ParseEvaluationFScoreCalculator: updated to compare indices instead of equality on pos-tagged tokens, for "noHead" and "wrongHead"
Bug: ParserEvaluatorImpl: remove root before evaluating token sequence
Bug: PosTagEvaluationFScoreCalculator: correctly compare empty/null tokens
Bug: AbstractTokenSequence: wasn't removing emtpy token from listWithWhiteSpace
Added PosTagSequenceFilters and posTaggerPostProcessingFilters for automatic removal of empty tokens
Moved sentenceCount to corpus readers and anlayze command, thus generalizing its applicability
ParserRegexBasedCorpusReader: implement PosTaggerAnnotatedCorpusReader and TokeniserAnnotatedCorpusReader to simplify reading of parsed corpora at all levels
ParserRegexBasedCorpusReaderImpl: added token filters, over-ridable readWord method for strange token encoding conventions
New tokeniser feature: LexiconPosTagForStringFeature (get lexicon pos-tags for any string)
Created separate lexicons for original Lefff, Talismane additions, and additions specific to the P+D tag
TalismaneFrench: allow option on whether to use compound postags (P+D, P+PRO) or separate tags with empty tokens
TalismaneFrench: trained new tokeniser, pos-tagger and parser models (latter two for P+D tagset)
talismane_ftb: allow conversion of empty tokens to P+D compound tags
talismane_ftbDep/FtbDepReader: possibility of keeping P+D compound tags
talismane_lefff: change LefffEntry to LexicalEntry wherever possible
Bug: talismane_machine_learning/FeatureResultImpl: only encode feature result when writing training name, otherwise return original result
talismane_trainer_fr: new posTagger features
talismane_trainer_fr: new "number in all letters" token filter
talismane_trainer_fr: when training, option to use multiple lexicons and compound postags (P+D)