It turned out not to be a big improvement when I did this (although it didn't slow it down either).  But generally if statements are relatively slow, so having them in the innermost loop is something to be avoided as much as possible.  

If we sometimes have highly masked images, we could have this function also return the smallest sub-image (as a new ImageView) that contains all the non-zero pixels in the masked image.  As long as the bounds were set correctly (which I think happens automatically when taking a sub-image), it shouldn't change any results.  

This might be pretty useful if a masked_image has only a small fraction near the middle with non-zero values.  I don't really know how common that use case is in practice.  I was mostly thinking in terms of masks for hot pixels, cosmic rays, etc.