BUG: Correct issue if patsy handles missing. Closes #2083.