reverted delete of the tokenizeString function, this is still needed for tests