BUG: Correct issue if patsy handles missing. Closes #2083.

Fixed in a fixup commit. Patsy's behavior makes this more difficult than it needs to be. Yes, adding back in the `_nan_rows(*combined)` if not empty handles the missing values in extra arrays.