Fix minor bug causing some gaps to be infilled twice