Tokenization improvements to improve learn quality. #2 & #3

(1) Ensure ordering of tokens is preserved as specified in the scheme
file.

(2) Introduces an optional 'priority' filed which can be used to change
the token ordering. High priority tokens comes left in the tokenization.

(3) Better priority based noice reduction while learning. In the case of
too much patterns to learn, noice reduction will kick in. In the first
phase, it will remove all low priority tokens. Second phase will remove
tokens from the largest array or the array which has comparatively low
priority tokens.

(4) Rewritten Malayalam scheme file to reflect new changes. This has
considerably reduced the number of tokens. It was 22,000+ tokens before
and 5000+ now!