Repository: alteryx/woodwork
Pull request #1467 (2022-07-06 21:41:48)

Null invalid PostalCode

codecov[bot]: # [Codecov](https://codecov.io/gh/alteryx/woodwork/pull/1467?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx) Report
> Merging [#1467](https://codecov.io/gh/alteryx/woodwork/pull/1467?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx) (58b3ec0) into [main](https://codecov.io/gh/alteryx/woodwork/commit/5614dfe5e0abc30819f2338828b4e1dec1f7197f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx) (5614dfe) will **decrease** coverage by `80.25%`.
> The diff coverage is `4.76%`.

> :exclamation: Current head 58b3ec0 differs from pull request most recent head fdbc383. Consider uploading reports for the commit fdbc383 to get more accurate results

```diff
@@             Coverage Diff             @@
##             main    #1467       +/-   ##
===========================================
- Coverage   98.68%   18.43%   -80.26%     
===========================================
  Files          95       95               
  Lines       10587    10607       +20     
===========================================
- Hits        10448     1955     -8493     
- Misses        139     8652     +8513     
```


| [Impacted Files](https://codecov.io/gh/alteryx/woodwork/pull/1467?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx) | Coverage Δ | |
|---|---|---|
| [woodwork/tests/logical\_types/test\_logical\_types.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvdGVzdHMvbG9naWNhbF90eXBlcy90ZXN0X2xvZ2ljYWxfdHlwZXMucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [woodwork/logical\_types.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvbG9naWNhbF90eXBlcy5weQ==) | `50.16% <14.28%> (-49.84%)` | :arrow_down: |
| [woodwork/tests/test\_config.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvdGVzdHMvdGVzdF9jb25maWcucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [woodwork/tests/test\_version.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvdGVzdHMvdGVzdF92ZXJzaW9uLnB5) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [woodwork/tests/utils/test\_utils.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvdGVzdHMvdXRpbHMvdGVzdF91dGlscy5weQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [woodwork/tests/utils/test\_concat.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvdGVzdHMvdXRpbHMvdGVzdF9jb25jYXQucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [woodwork/tests/type\_system/conftest.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvdGVzdHMvdHlwZV9zeXN0ZW0vY29uZnRlc3QucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [woodwork/tests/utils/test\_read\_file.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvdGVzdHMvdXRpbHMvdGVzdF9yZWFkX2ZpbGUucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [woodwork/tests/accessor/test\_indexers.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvdGVzdHMvYWNjZXNzb3IvdGVzdF9pbmRleGVycy5weQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [woodwork/tests/demo\_tests/test\_retail.py](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx#diff-d29vZHdvcmsvdGVzdHMvZGVtb190ZXN0cy90ZXN0X3JldGFpbC5weQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [58 more](https://codecov.io/gh/alteryx/woodwork/pull/1467/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/alteryx/woodwork/pull/1467?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/alteryx/woodwork/pull/1467?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx). Last update [5614dfe...fdbc383](https://codecov.io/gh/alteryx/woodwork/pull/1467?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=alteryx).


Patch:
diff --git a/docs/source/release_notes.rst b/docs/source/release_notes.rst
index a91b701..3f6b7f9 100644
--- a/docs/source/release_notes.rst
+++ b/docs/source/release_notes.rst
@@ -14,6 +14,7 @@ Future Release
         * Added ability to null invalid values for ``PhoneNumber`` logical type (:pr:`1460`)
         * Added ability to null invalid values for ``AgeFractional`` and ``AgeNullable`` logical types (:pr:`1462`)
         * Added ability to null invalid values for ``LatLong`` logical type (:pr:`1465`)
+        * Added ability to null invalid values for ``PostalCode`` logical type (US only) (:pr:`1467`)
     * Fixes
     * Changes
         * Set the minimum acceptable version of pandas to ``1.4.0`` for woodwork and ``1.4.3`` for spark add-on (:pr:`1461`)
diff --git a/woodwork/logical_types.py b/woodwork/logical_types.py
index 524b305..1d7741e 100644
--- a/woodwork/logical_types.py
+++ b/woodwork/logical_types.py
@@ -749,6 +749,9 @@ class PostalCode(LogicalType):
     standard_tags = {"category"}
 
     def transform(self, series, null_invalid_values=False):
+        if null_invalid_values:
+            series = _coerce_postal_code(series)
+
         if pd.api.types.is_numeric_dtype(series):
             try:
                 series = series.astype("Int64").astype("string")
@@ -868,7 +871,7 @@ def _coerce_string(series, regex=None):
         invalid = _get_index_invalid_string(series, regex)
 
         if invalid.any():
-            series[invalid] = None
+            series[invalid] = pd.NA
     return series
 
 
@@ -908,3 +911,9 @@ def _coerce_latlong(series):
     if invalid.any():
         series[invalid] = None
     return series
+
+
+def _coerce_postal_code(series):
+    if pd.api.types.is_numeric_dtype(series):
+        series = _coerce_integer(series).astype("Int64")
+    return _coerce_string(series, regex="postal_code_inference_regex")


Test patch:
diff --git a/woodwork/tests/logical_types/test_logical_types.py b/woodwork/tests/logical_types/test_logical_types.py
index b7e4190..994fd4a 100644
--- a/woodwork/tests/logical_types/test_logical_types.py
+++ b/woodwork/tests/logical_types/test_logical_types.py
@@ -620,3 +620,42 @@ def test_null_invalid_latlong():
     expected = pd.DataFrame({"data": data})
     df.ww.init(logical_types=types, null_invalid_values=True)
     pd.testing.assert_frame_equal(df, expected)
+
+
+def test_null_invalid_postal_code():
+    types = {"data": "PostalCode"}
+    invalid = ["text", 6.7, object, "123456"]
+    valid = [90210, "60018-0123", "10010"]
+    data = pd.Series(valid + invalid)
+    df = pd.DataFrame({"data": data})
+
+    df.ww.init(logical_types=types, null_invalid_values=False)
+    expected = pd.DataFrame({"data": data.astype("category")})
+    pd.testing.assert_frame_equal(df, expected)
+
+    nulls = [None] * len(invalid)
+    data = pd.Series(valid + nulls, dtype="string")
+    expected = pd.DataFrame({"data": data.astype("category")})
+    df.ww.init(logical_types=types, null_invalid_values=True)
+    pd.testing.assert_frame_equal(df, expected)
+
+
+def test_null_invalid_postal_code_numeric():
+    types = {"data": "PostalCode"}
+    invalid = [-6.7, 60018.0123, 123456.0]
+    valid = [90210, 60018.0, 10010.0]
+    data = pd.Series(valid + invalid)
+    df = pd.DataFrame({"data": data})
+
+    with pytest.raises(
+        TypeConversionError,
+        match="Please confirm the underlying data is consistent with logical type PostalCode",
+    ):
+        df.ww.init(logical_types=types, null_invalid_values=False)
+
+    nulls = [None] * len(invalid)
+    data = pd.Series(valid + nulls, dtype="Int64")
+    data = data.astype("string").astype("category")
+    expected = pd.DataFrame({"data": data})
+    df.ww.init(logical_types=types, null_invalid_values=True)
+    pd.testing.assert_frame_equal(df, expected)