Pluggable tokenizer

Loads and uses a pluggable tokenizer. Comes with a sane default tokenizer.

@willkg r?
I think I would have done the lower/blank pass in one list comprehension:

```
text = [token.lower() for token in text if token]
```