TST: Old pandas can't assign nan to int series.
REF: Convert to array before checking finiteness.
ENH: Let non-ts pandas objects through. Need to set freq.