flex first commit
flex tokenizer starting
Merge branch 'master' of github.com:ixa-ehu/ixa-pipe-tok into flex
working on it
adding flex file
working on jflex
working on jflex
token class and factory added
need to work on acronyms now
pending non breaking prefixes
working version but debugging needed
segmenter running
working on it
cli options new structure
segmenter and tokenizer working
english ready; spanish next
adding language support to flexer
adding timer
working on multilingualism
multilingual apostrophe support