changed looping to chunks of 500 instead of 1000 to avoid 502 errors