ran it succesfully for 3 million but the fraction of repeated queries is increasing monotonically with size; will need to do some verfication manually to see that the results are correct