moving the convert_tokens function to a common file