Improved tokenization;
Improved handling of Unicode;
Added test of medium portuguese language text;
added missing file
Merge pull request #1 from fccoelho/master

Improved tokenization;