Fix CSV reading of mixed dtype columns

they might be cast to `float64` here if the column contains nothing but empty strings but if that's not the desired dtype they'll be cast to whatever the desired dtype is in the `astype(dtype)` call below.