implement --partial option, useful on flaky connections with big datasets