This is scary, though I agree we don't want a bunch of warnings dumped on the user.

What is the recommended way of handling this in numpy? Is it possible to promote these warnings to errors and then explicitly catch them in the appropriate places in the code? Or maybe use a masked array instead of filling with nans?