throw an exception if tokenization is failed
replace JUnit3 tests with JUnit4 and fix vector similarity tests
use expected instead of catching exception