post function for crawler