editted tokenizing procedures in main thread