can you see if there are better results with more units that still fit into memory? 1000 doesn't seem so much to me, but the dataset is quite big. Maybe there should be a batch-mechanism then?