updated the functions to get rid of invalid characters