master tokenizing method
remove unnecessary bits