Send cURL requests in chunks and robustness against duplicate strings