joshua-decoder/joshua — pull request #157 by jganitkevitch (2014-01-19 04:28:32+00:00)
Title: Packer and tokenization updates

- Stole more non-breaking prefixes files from Moses
- Mysterious Perl one-liner temporary file fix (necessary for some file systems, it seems)
- Added odd UTF8 dashes to punctuation normalizer
- Packer supports mixed labeled and unlabeled features (will turn unlabeled into dense numerically labeled ones)
- Packer supports alignments in the 5th field (via -ga flag) and alignments in separate file (via -fa flag)
- PackedGrammar support access to alignments at runtime via getAlignments() method in Rule