Repository: pyjanitor-devs/pyjanitor
Pull request #960 (2021-12-02 02:28:45)

[ENH] Fix concatenate_columns for categories
<!-- Thank you for your PR!

BEFORE YOU CONTINUE! Please add the appropriate three-letter abbreviation to your title.

The abbreviations can be:
- [DOC]: Documentation fixes.
- [ENH]: Code contributions and new features.
- [TST]: Test-related contributions.
- [INF]: Infrastructure-related contributions.

Also, do not forget to tag the relevant issue here as well.

Finally, as commits come in, don't forget to regularly rebase!
-->

# PR Description

Solves this problem for pandas 1.3.4:

```python
df = (
    pd.DataFrame(dict(a=["asdf", "fdsa"], b=["thing1", "thing2"]))
    .astype(dict(a="category", b="category"))
    .concatenate_columns(["a", "b"], "new")
)
```

> ValueError: Cannot setitem on a Categorical with a new category, set the categories first

<!-- Doing so provides maintainers with context on what the PR is, and can help us more effectively review your PR. -->

<!-- Please also identify below which issue that has been raised that you are going to close. -->

<!-- As you go down the PR template, please feel free to delete sections that are irrelevant. -->

# PR Checklist

<!-- This checklist exists for newcomers who are not yet familiar with our requirements. If you are experienced with
the project, please feel free to delete this section. -->

Please ensure that you have done the following:

1. [x] PR in from a fork off your branch. Do not PR from `<your_username>`:`dev`, but rather from `<your_username>`:`<feature-branch_name>`.
<!-- Doing this helps us keep the commit history much cleaner than it would otherwise be. -->
2. [x] If you're not on the contributors list, add yourself to `AUTHORS.rst`.
<!-- We'd like to acknowledge your contributions! -->
3. [x] Add a line to `CHANGELOG.md` under the latest version header (i.e. the one that is "on deck") describing the contribution.
    - Do use some discretion here; if there are multiple PRs that are related, keep them in a single line.

# Automatic checks

There will be automatic checks run on the PR. These include:

- Building a preview of the docs on Netlify
- Automatically linting the code
- Making sure the code is documented
- Making sure that all tests are passed
- Making sure that code coverage doesn't go down.

# Relevant Reviewers

<!-- Finally, please tag relevant maintainers to review. -->

Please tag maintainers to review.

- @ericmjl


ericmjl: <!-- NETLIFY DEPLOY COMMENT GENERATED BY ACTIONS_NETLIFY -->
🚀 Deployed on https://deploy-preview-960--pyjanitor.netlify.app
ericmjl: @zbarry great to see you again! I'm going to take a look at the tests and see where things are failing.

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0a55586..57b1736 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -17,6 +17,7 @@
 -   [ENH] `As_Categorical` deprecated from `encode_categorical`; a tuple of `(categories, order)` suffices for \*\*kwargs. @samukweku
 -   [ENH] Deprecate `names_sort` from `pivot_wider`.@samukweku
 -   [ENH] Add `softmax` to `math` module. Issue #902. @loganthomas
+-   [ENH] Fix `concatenate_columns` not working on category inputs @zbarry
 
 ## [v0.21.2] - 2021-09-01
 
diff --git a/janitor/functions/concatenate_columns.py b/janitor/functions/concatenate_columns.py
index 5b4f14e..990079e 100644
--- a/janitor/functions/concatenate_columns.py
+++ b/janitor/functions/concatenate_columns.py
@@ -48,7 +48,7 @@ def concatenate_columns(
         raise JanitorError("At least two columns must be specified")
 
     df[new_column_name] = (
-        df[column_names].fillna("").astype(str).agg(sep.join, axis=1)
+        df[column_names].astype(str).fillna("").agg(sep.join, axis=1)
     )
 
     if ignore_empty:


Test patch:
diff --git a/tests/functions/test_concatenate_columns.py b/tests/functions/test_concatenate_columns.py
index dd17f06..25a7d7b 100644
--- a/tests/functions/test_concatenate_columns.py
+++ b/tests/functions/test_concatenate_columns.py
@@ -1,3 +1,4 @@
+"""Tests for concatenate_columns."""
 import pytest
 
 from janitor.errors import JanitorError
@@ -5,6 +6,7 @@ from janitor.errors import JanitorError
 
 @pytest.mark.functions
 def test_concatenate_columns(dataframe):
+    """Basic test for concatenate_columns."""
     df = dataframe.concatenate_columns(
         column_names=["a", "decorated-elephant"],
         sep="-",
@@ -15,19 +17,27 @@ def test_concatenate_columns(dataframe):
 
 @pytest.mark.functions
 def test_concatenate_columns_null_values(missingdata_df):
+    """Test for concatenating columns with null values.
+
+    Null values ought to show up as "nan" in strings
+    in the concatenated column.
+    """
     df = missingdata_df.concatenate_columns(
         column_names=["a", "decorated-elephant"],
         sep="-",
         new_column_name="index",
         ignore_empty=True,
     )
-    expected_values = ["1.0-1", "2.0-2", "3"] * 3
+    expected_values = ["1.0-1", "2.0-2", "nan-3"] * 3
     assert expected_values == df["index"].tolist()
 
 
 @pytest.mark.functions
 @pytest.mark.parametrize("column_names", [["a"], []])
 def test_concatenate_columns_errors(dataframe, column_names):
+    """
+    Test that an error is raised when less than two columns are specified.
+    """
     with pytest.raises(
         JanitorError, match="At least two columns must be specified"
     ):