Added ScalaCodeTokenizer.tokenize which takes a String instead of an IDocument.

ScalaCodeTokenizer.tokenize asked for a IDocument only to right away get the IDocument's contents and then never use it again. It makes more sense to just take the contents as a string directly, which is what this change does.
Also deprecated ScalaCodeTokenizer.tokenize(IDocument, Int, Int).

(cherry picked from commit 2754e16c032e3dd70bbd09a20c36ae1f23322274)
Improved new ScalaCodeTokenizer.tokenize API to minimise allocation of string objects on caller side.

(cherry picked from commit 1c2db9513ffaea7c13b6260b87c6179c72e7895b)
Deprecated ScalaCodeScanner.tokenize(IDocument, Int, Int) incorrectly handled offset in generated tokens.

Also added a test.

Fixes bug introduced (into master) by SHA: e17b2ae97aaa5644d0cf1ad09ff6bc5ad039002a

(cherry picked from commit b9c41d8d2c5bddd88c471b297c53e2f3539014f2)
remove controversial & undocumented deprecation

(cherry picked from commit 9cd3d4f641aed2beab6d4133b8703c35893bd5ab)
Merge pull request #521 from huitseeker/issue/backport-tokenize-1001866

[Backport] Backport changes to the ScalaCodeScanner tokenizer