Repository: pyjanitor-devs/pyjanitor (python)

Issue:
duplicated columns generate more rows than required in `pivot_longer`
# Brief Description

If the columns are duplicated, more rows are generated than necessary (duplicated) when melted.

# System Information

<!-- System information helps us. To keep things simple, just let us know the OS and Python version first.
You can provide the optional information later. -->

- Operating system: macOS
- Python version (required): 3.8

# Minimally Reproducible Code

copied from [SO](https://stackoverflow.com/q/72126060/7175713)
```
rows = [['credit', 1, 1, 2, 3]]
columns = ['Type', 'amount', 'active', 'amount', 'active']

df = pd.DataFrame(rows, columns=columns)
df = df.set_index('Type')


        amount  active  amount  active
Type                                   
credit       1       1       2       3


# output from pivot_longer
df.pivot_longer(names_to = ['amount', 'active'], names_pattern = ['amount', 'active'])
 
   amount  active
0       1       1
1       2       3
2       1       1
3       2       3

# expected output

        amount  active
Type
credit       1       1
credit       2       3
```




Pull request:
[ENH]fix for duplicate columns - pivot_longer
# PR Description

Please describe the changes proposed in the pull request:

- fix output for duplicate columns

**This PR resolves #1098.**

<!-- As you go down the PR template, please feel free to delete sections that are irrelevant. -->

# PR Checklist

<!-- This checklist exists for newcomers who are not yet familiar with our requirements. If you are experienced with
the project, please feel free to delete this section. -->

Please ensure that you have done the following:

1. [ ] PR in from a fork off your branch. Do not PR from `<your_username>`:`dev`, but rather from `<your_username>`:`<feature-branch_name>`.
<!-- Doing this helps us keep the commit history much cleaner than it would otherwise be. -->
2. [x] If you're not on the contributors list, add yourself to `AUTHORS.md`.
<!-- We'd like to acknowledge your contributions! -->
3. [x] Add a line to `CHANGELOG.md` under the latest version header (i.e. the one that is "on deck") describing the contribution.
    - Do use some discretion here; if there are multiple PRs that are related, keep them in a single line.

# Automatic checks

There will be automatic checks run on the PR. These include:

- Building a preview of the docs on Netlify
- Automatically linting the code
- Making sure the code is documented
- Making sure that all tests are passed
- Making sure that code coverage doesn't go down.

# Relevant Reviewers

<!-- Finally, please tag relevant maintainers to review. -->

Please tag maintainers to review.

- @ericmjl


Patch:
diff --git a/janitor/functions/pivot.py b/janitor/functions/pivot.py
index 69a7bee..fb7aaf5 100644
--- a/janitor/functions/pivot.py
+++ b/janitor/functions/pivot.py
@@ -606,6 +606,8 @@ def _computations_pivot_longer(
             name: extract_array(df[name], extract_numpy=True) for name in index
         }
 
+    if len(column_names) != len(set(column_names)):
+        column_names = pd.unique(column_names)
     df = df.loc[:, column_names]
 
     len_index = len(df)


Test patch:
diff --git a/tests/functions/test_pivot_longer.py b/tests/functions/test_pivot_longer.py
index a88f6fb..551acf7 100644
--- a/tests/functions/test_pivot_longer.py
+++ b/tests/functions/test_pivot_longer.py
@@ -1156,3 +1156,22 @@ def test_names_transform_numeric():
     ).reset_index()
 
     assert_frame_equal(actual, expected)
+
+
+def test_duplicated_columns():
+    """Test output for duplicated columns."""
+    rows = [["credit", 1, 1, 2, 3]]
+    columns = ["Type", "amount", "active", "amount", "active"]
+
+    df = pd.DataFrame(rows, columns=columns)
+    df = df.set_index("Type")
+
+    actual = pd.DataFrame(
+        {"amount": [1, 2], "active": [1, 3]},
+        index=pd.Index(["credit", "credit"], name="Type"),
+    )
+    expected = df.pivot_longer(
+        names_to=".value", names_pattern="(.+)", ignore_index=False
+    )
+
+    assert_frame_equal(actual, expected)