read requests chunk by chunk to reduce memory usage