However, I have noticed that np.unique sometimes returns many duplicates of NAN. It's probably worth catching this case, perhaps by substituting something else for NAN before calling unique