Repository: pyjanitor-devs/pyjanitor
Pull request #1369 (2024-06-01 16:00:49)

[ENH] allow pandas data structures in `complete`
# PR Description

Please describe the changes proposed in the pull request:

- Deprecate dictionary - be explicit and use a pandas object instead


**This PR improves the `complete` method.**

# PR Checklist

<!-- This checklist exists for newcomers who are not yet familiar with our requirements. If you are experienced with
the project, please feel free to delete this section. -->

Please ensure that you have done the following:

1. [x] PR in from a fork off your branch. Do not PR from `<your_username>`:`dev`, but rather from `<your_username>`:`<feature-branch_name>`.
<!-- Doing this helps us keep the commit history much cleaner than it would otherwise be. -->
2. [x] If you're not on the contributors list, add yourself to `AUTHORS.md`.
<!-- We'd like to acknowledge your contributions! -->
3. [x] Add a line to `CHANGELOG.md` under the latest version header (i.e. the one that is "on deck") describing the contribution.
    - Do use some discretion here; if there are multiple PRs that are related, keep them in a single line.

# Automatic checks

There will be automatic checks run on the PR. These include:

- Building a preview of the docs on Netlify
- Automatically linting the code
- Making sure the code is documented
- Making sure that all tests are passed
- Making sure that code coverage doesn't go down.

# Relevant Reviewers

<!-- Finally, please tag relevant maintainers to review. -->

Please tag maintainers to review.

- @ericmjl



Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f4ee02c..b4e1219 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -3,14 +3,13 @@
 ## [Unreleased]
 
 - [ENH] Added a `complete` method for polars. - Issue #1352 @samukweku
-- [ENH] Added a `row_to_names` method for polars. Issue #1352
 - [ENH] `read_commandline` function now supports polars - Issue #1352
-
 - [ENH] Improved performance for non-equi joins when using numba - @samukweku PR #1341
 - [ENH] Added a `clean_names` method for polars - it can be used to clean the column names, or clean column values . Issue #1343 @samukweku
 - [ENH] `xlsx_cells` function now supports polars - Issue #1352 @samukweku
 - [ENH] `xlsx_table` function now supports polars - Issue #1352 @samukweku
 - [ENH] Added a `pivot_longer` method, and a `pivot_longer_spec` function for polars - Issue #1352 @samukweku
+- [ENH] pandas Index,Series, DataFrame now supported in the `complete` method. - PR #1369 @samukweku
 
 ## [v0.27.0] - 2024-03-21
 
diff --git a/janitor/functions/complete.py b/janitor/functions/complete.py
index 5171ccc..c653ad7 100644
--- a/janitor/functions/complete.py
+++ b/janitor/functions/complete.py
@@ -1,6 +1,8 @@
-import functools
-from typing import Any, Dict, List, Optional, Tuple, Union
+from __future__ import annotations
 
+from typing import Any
+
+import numpy as np
 import pandas as pd
 import pandas_flavor as pf
 from pandas.api.types import is_list_like, is_scalar
@@ -15,22 +17,40 @@ def complete(
     df: pd.DataFrame,
     *columns: Any,
     sort: bool = False,
-    by: Optional[Union[list, str]] = None,
-    fill_value: Optional[Union[Dict, Any]] = None,
+    by: str | list = None,
+    fill_value: dict | Any = None,
     explicit: bool = True,
 ) -> pd.DataFrame:
-    """Complete a data frame with missing combinations of data.
+    """
+    Complete a data frame with missing combinations of data.
 
     It is modeled after tidyr's `complete` function, and is a wrapper around
     [`expand_grid`][janitor.functions.expand_grid.expand_grid], `pd.merge`
     and `pd.fillna`. In a way, it is the inverse of `pd.dropna`, as it exposes
     implicitly missing rows.
 
-    Combinations of column names or a list/tuple of column names, or even a
-    dictionary of column names and new values are possible.
-    If a dictionary is passed,
-    the user is required to ensure that the values are unique 1-D arrays.
-    The keys in a dictionary must be present in the dataframe.
+    The variable `columns` parameter can be a combination
+    of column names or a list/tuple of column names,
+    or a pandas Index, Series, or DataFrame.
+    If a pandas Index, Series, or DataFrame is passed, it should
+    have a name or names that exist in `df`.
+
+    A callable can also be passed - the callable should evaluate
+    to a pandas Index, Series, or DataFrame.
+
+    A dictionary can also be passed -
+    the values of the dictionary should be
+    either be a 1D array
+    or a callable that evaluates to a
+    1D array,
+    while the keys of the dictionary
+    should exist in `df`.
+
+    User should ensure that the pandas object is unique and/or sorted
+    - no checks are done to ensure uniqueness and/or sortedness.
+
+    If `by` is present, the DataFrame is *completed* per group.
+    `by` should be a column name, or a list of column names.
 
     Examples:
         >>> import pandas as pd
@@ -68,11 +88,25 @@ def complete(
         5  2004  Saccharina        2.0
 
         Expose missing years from 1999 to 2004:
-        >>> df.complete(
-        ...     {"Year": range(df.Year.min(), df.Year.max() + 1)},
-        ...     "Taxon",
-        ...     sort=True
-        ... )
+        >>> index = pd.Index(range(1999,2005),name='Year')
+        >>> df.complete(index, "Taxon", sort=True)
+            Year       Taxon  Abundance
+        0   1999      Agarum        1.0
+        1   1999  Saccharina        4.0
+        2   2000      Agarum        NaN
+        3   2000  Saccharina        5.0
+        4   2001      Agarum        NaN
+        5   2001  Saccharina        NaN
+        6   2002      Agarum        NaN
+        7   2002  Saccharina        NaN
+        8   2003      Agarum        NaN
+        9   2003  Saccharina        NaN
+        10  2004      Agarum        8.0
+        11  2004  Saccharina        2.0
+
+        A dictionary can be used as well:
+        >>> dictionary = {'Year':range(1999,2005)}
+        >>> df.complete(dictionary, "Taxon", sort=True)
             Year       Taxon  Abundance
         0   1999      Agarum        1.0
         1   1999  Saccharina        4.0
@@ -103,6 +137,7 @@ def complete(
         1      2        2         a     NaN       5
         2      1        2         b     3.0       6
         3      2        3         b     4.0       7
+
         >>> df.complete(
         ...     "group",
         ...     ("item_id", "item_name"),
@@ -138,12 +173,59 @@ def complete(
         6      2        2         b     0.0    99.0
         7      2        3         b     4.0     7.0
 
+        Expose missing rows per group, using a callable:
+        >>> df = pd.DataFrame(
+        ...     {
+        ...         "state": ["CA", "CA", "HI", "HI", "HI", "NY", "NY"],
+        ...         "year": [2010, 2013, 2010, 2012, 2016, 2009, 2013],
+        ...         "value": [1, 3, 1, 2, 3, 2, 5],
+        ...     }
+        ... )
+        >>> df
+          state  year  value
+        0    CA  2010      1
+        1    CA  2013      3
+        2    HI  2010      1
+        3    HI  2012      2
+        4    HI  2016      3
+        5    NY  2009      2
+        6    NY  2013      5
+
+        >>> def new_year_values(df):
+        ...     return pd.RangeIndex(start=df.year.min(), stop=df.year.max() + 1, name='year')
+        >>> df.complete(new_year_values, by='state',sort=True)
+            state  year  value
+        0     CA  2010    1.0
+        1     CA  2011    NaN
+        2     CA  2012    NaN
+        3     CA  2013    3.0
+        4     HI  2010    1.0
+        5     HI  2011    NaN
+        6     HI  2012    2.0
+        7     HI  2013    NaN
+        8     HI  2014    NaN
+        9     HI  2015    NaN
+        10    HI  2016    3.0
+        11    NY  2009    2.0
+        12    NY  2010    NaN
+        13    NY  2011    NaN
+        14    NY  2012    NaN
+        15    NY  2013    5.0
+
     Args:
         df: A pandas DataFrame.
         *columns: This refers to the columns to be completed.
-            It could be column labels (string type),
-            a list/tuple of column labels, or a dictionary that pairs
-            column labels with new values.
+            It could be column labels,
+            a list/tuple of column labels,
+            or a pandas Index, Series, or DataFrame.
+            It can also be a callable that gets evaluated
+            to a pandas Index, Series, or DataFrame.
+            It can also be a dictionay,
+            where the values are either a 1D array
+            or a callable that evaluates to a
+            1D array,
+            while the keys of the dictionary
+            should exist in `df`.
         sort: Sort DataFrame based on *columns.
         by: Label or list of labels to group by.
             The explicit missing rows are returned per group.
@@ -162,18 +244,31 @@ def complete(
     if not columns:
         return df
 
-    # no copy made of the original dataframe
+    # no copy is made of the original dataframe
     # since pd.merge (computed some lines below)
     # makes a new object - essentially a copy
     return _computations_complete(df, columns, sort, by, fill_value, explicit)
 
 
+def _create_cartesian_dataframe(df, columns, column_checker, sort):
+    """
+    Create a DataFrame from the
+    combination of all pandas objects
+    """
+    objects = _create_pandas_object(df, columns=columns, sort=sort)
+    objects = dict(zip(range(len(objects)), objects))
+    objects = _computations_expand_grid(objects)
+    objects = dict(zip(column_checker, objects.values()))
+    objects = pd.DataFrame(objects, copy=False)
+    return objects
+
+
 def _computations_complete(
     df: pd.DataFrame,
-    columns: List[Union[List, Tuple, Dict, str]],
+    columns: list | tuple | dict | str,
     sort: bool,
-    by: Optional[Union[list, str]],
-    fill_value: Optional[Union[Dict, Any]],
+    by: list | str,
+    fill_value: dict | Any,
     explicit: bool,
 ) -> pd.DataFrame:
     """
@@ -192,45 +287,28 @@ def _computations_complete(
         explicit,
     ) = _data_checks_complete(df, columns, sort, by, fill_value, explicit)
 
-    all_scalars = all(map(is_scalar, columns))
-
-    # nothing to 'complete' here
-    if (all_scalars and len(columns) == 1) or df.empty:
-        return df
-
-    # under the right conditions, stack/unstack can be faster
-    # plus it always returns a sorted DataFrame
-    # which does help in viewing the missing rows
-    # however, using a merge keeps things simple
-    # with a stack/unstack,
-    # the relevant columns combination should be unique
-    # and there should be no nulls
-    # trade-off for the simplicity of merge is not so bad
-    # of course there could be a better way ...
     if by is None:
-        uniques = _generic_complete(
-            df=df,
-            columns=columns,
-            all_scalars=all_scalars,
-            sort=sort,
+        uniques = _create_cartesian_dataframe(
+            df=df, column_checker=column_checker, columns=columns, sort=sort
         )
     else:
-        column_checker = by + column_checker
-        uniques = df.groupby(by, sort=False)
-        uniques = {
-            key: _generic_complete(
-                df=value,
+        grouped = df.groupby(by, sort=False)
+        uniques = {}
+        for group_name, frame in grouped:
+            _object = _create_cartesian_dataframe(
+                df=frame,
+                column_checker=column_checker,
                 columns=columns,
-                all_scalars=all_scalars,
                 sort=sort,
             )
-            for key, value in uniques
-        }
-        uniques = pd.concat(uniques, names=by, copy=False)
-        uniques = uniques.droplevel(-1, axis=0)
+            uniques[group_name] = _object
+        column_checker = by + column_checker
+        by.append("".join(column_checker))
+        uniques = pd.concat(uniques, names=by, copy=False, sort=False)
+        uniques = uniques.droplevel(axis=0, level=-1)
     columns = df.columns
     indicator = False
-    if fill_value is not None and not explicit:
+    if (fill_value is not None) and not explicit:
         # to get a name that does not exist in the columns
         indicator = "".join(columns)
     out = pd.merge(
@@ -242,189 +320,72 @@ def _computations_complete(
         sort=False,
         indicator=indicator,
     )
-
-    if fill_value is not None:
-        null_columns = [col for col in columns if out[col].hasnans]
-        if null_columns:
-            if is_scalar(fill_value):
-                # faster when fillna operates on a Series basis
-                fill_value = {col: fill_value for col in null_columns}
-            if explicit:
-                out = out.fillna(fill_value)
-            else:
-                # keep only columns that are not part of column_checker
-                # IOW, we are excluding columns that were not used
-                # to generate the combinations
-                fill_value = {
-                    col: value
-                    for col, value in fill_value.items()
-                    if col not in column_checker
-                }
-                if fill_value:
-                    # when explicit is False
-                    # use the indicator parameter to identify rows
-                    # for `left_only`, and fill the relevant columns
-                    # in fill_value with the associated value.
-                    boolean_filter = out.loc[:, indicator] == "left_only"
-                    out = out.drop(columns=indicator)
-                    # iteration used here,
-                    # instead of assign (which is also a for loop),
-                    # to cater for scenarios where the column_name
-                    # is not a string
-                    # assign only works with keys that are strings
-                    # Also, the output wil be floats (for numeric types),
-                    # even if all the columns could be integers
-                    # user can always convert to int if required
-                    for column_name, value in fill_value.items():
-                        # for categorical dtypes, set the categories first
-                        if isinstance(
-                            out[column_name].dtype, pd.CategoricalDtype
-                        ):
-                            out[column_name] = out[
-                                column_name
-                            ].cat.add_categories([value])
-                        out.loc[boolean_filter, column_name] = value
+    if indicator:
+        indicator = out.pop(indicator)
     if not out.columns.equals(columns):
-        return out.loc(axis=1)[columns]
-    return out
-
-
-def _generic_complete(
-    df: pd.DataFrame, columns: list, all_scalars: bool, sort: bool
-):
-    """Generate cartesian product for `_computations_complete`.
-
-    Returns a DataFrame, with no duplicates.
-    """
-    if all_scalars:
-        uniques = {
-            col: pd.factorize(df[col], sort=sort)[-1] for col in columns
+        out = out.loc[:, columns]
+    if fill_value is None:
+        return out
+    # keep only columns that are not part of column_checker
+    # IOW, we are excluding columns that were not used
+    # to generate the combinations
+    null_columns = [
+        col for col in out if out[col].hasnans and col not in column_checker
+    ]
+    if not null_columns:
+        return out
+    if is_scalar(fill_value):
+        # faster when fillna operates on a Series basis
+        fill_value = {col: fill_value for col in null_columns}
+    else:
+        fill_value = {
+            col: _fill_value
+            for col, _fill_value in fill_value.items()
+            if col in null_columns
         }
-        uniques = _computations_expand_grid(uniques)
-        uniques = pd.DataFrame(uniques, copy=False)
-        uniques.columns = columns
-        return uniques
-
-    uniques = {}
-    for ind, column in enumerate(columns):
-        if isinstance(column, dict):
-            len_columns = len(columns)
-            column = _complete_column(column, df=df, sort=sort)
-            # iteration here avoids any potential index collision
-            column = {
-                ind + len_columns + key: value for key, value in column.items()
-            }
-            uniques.update(column)
-        else:
-            uniques[ind] = _complete_column(column, df=df, sort=sort)
-    uniques = _computations_expand_grid(uniques)
-    uniques = pd.DataFrame(uniques, copy=False)
-    uniques.columns = uniques.columns.droplevel(0)
-    return uniques
-
-
-@functools.singledispatch
-def _complete_column(column, df, sort):
-    """
-    Args:
-        column: scalar/list/dict
-        df: Pandas DataFrame
-        sort: whether or not to sort the Series.
-
-    Returns:
-        A Pandas Series/DataFrame with no duplicates,
-        or a dictionary of unique Pandas Series.
-    """
-
-    _, arr = pd.factorize(df.loc(axis=1)[column], sort=sort)
-    return pd.Series(arr, name=column)
-
-
-@_complete_column.register(list)  # noqa: F811
-def _sub_complete_column(column, df, sort):  # noqa: F811
-    """
-    Args:
-        column: list
-        df: Pandas DataFrame
-        sort: whether or not to sort the DataFrame.
-
-    Returns:
-        Pandas DataFrame
-    """
-    outcome = df.loc(axis=1)[column]
-
-    exclude = outcome.isna().all(axis=1)
-
-    if exclude.any(axis=None):
-        outcome = outcome.loc[~exclude]
-
-    exclude = outcome.duplicated()
-
-    if exclude.any():
-        outcome = outcome.loc[~exclude]
-
-    if sort:
-        outcome = outcome.sort_values(by=column)
-
-    return outcome
-
-
-@_complete_column.register(dict)  # noqa: F811
-def _sub_complete_column(column, df, sort):  # noqa: F811
-    """
-    Args:
-        column: dictionary
-        df: Pandas DataFrame
-        sort: whether or not to sort the Series.
-
-    Returns:
-        A dictionary of unique pandas Series.
-    """
-
-    collection = {}
-    for ind, (key, value) in enumerate(column.items()):
-        arr = apply_if_callable(value, df)
-        if not is_list_like(arr):
-            raise ValueError(f"value for {key} should be a 1-D array.")
-        if not hasattr(arr, "shape"):
-            arr = pd.Series([*arr], name=key)
-
-        if not arr.size > 0:
-            raise ValueError(
-                f"Kindly ensure the provided array for {key} "
-                "has at least one value."
-            )
-
-        if isinstance(arr, pd.Index):
-            arr_ndim = arr.nlevels
-        else:
-            arr_ndim = arr.ndim
-
-        if arr_ndim != 1:
-            raise ValueError(f"Kindly provide a 1-D array for {key}.")
-
-        if sort:
-            _, arr = pd.factorize(arr, sort=sort)
-
-        if isinstance(key, tuple):  # handle a MultiIndex column
-            arr = pd.DataFrame(arr, columns=pd.MultiIndex.from_tuples([key]))
-
-        else:
-            arr = pd.Series(arr, name=key)
-
-        collection[ind] = arr
+    if not fill_value:
+        return out
+    if explicit:
+        return out.fillna(fill_value)
+    # when explicit is False
+    # use the indicator parameter to identify rows
+    # for `left_only`, and fill the relevant columns
+    # in fill_value with the associated value.
+    boolean_filter = indicator == "left_only"
+    # iteration used here,
+    # instead of assign (which is also a for loop),
+    # to cater for scenarios where the column_name
+    # is not a string
+    # assign only works with keys that are strings
+    # Also, the output wil be floats (for numeric types),
+    # even if all the columns could be integers
+    # user can always convert to int if required
+    for column_name, value in fill_value.items():
+        # for categorical dtypes, set the categories first
+        if isinstance(out[column_name].dtype, pd.CategoricalDtype):
+            out[column_name] = out[column_name].cat.add_categories([value])
+        out.loc[boolean_filter, column_name] = value
 
-    return collection
+    return out
 
 
 def _data_checks_complete(
     df: pd.DataFrame,
-    columns: List[Union[List, Tuple, Dict, str]],
-    sort: Optional[bool],
-    by: Optional[Union[list, str]],
-    fill_value: Optional[Union[Dict, Any]],
+    columns: (
+        list
+        | tuple
+        | dict
+        | str
+        | callable
+        | pd.Index
+        | pd.Series
+        | pd.DataFrame
+    ),
+    sort: bool,
+    by: list | str,
+    fill_value: dict | Any,
     explicit: bool,
-):
+) -> tuple:
     """
     Function to check parameters in the `complete` function.
     Checks the type of the `columns` parameter, as well as the
@@ -446,15 +407,42 @@ def _data_checks_complete(
         [*grouping] if isinstance(grouping, tuple) else grouping
         for grouping in columns
     ]
+
     column_checker = []
     for grouping in columns:
         if is_scalar(grouping):
             column_checker.append(grouping)
-        else:
-            check("grouping", grouping, [list, dict])
+        elif isinstance(grouping, (list, dict)):
             if not grouping:
                 raise ValueError("entry in columns argument cannot be empty")
             column_checker.extend(grouping)
+        elif callable(grouping):
+            grouping = apply_if_callable(
+                maybe_callable=grouping,
+                obj=df.iloc[:10],
+            )
+            if not isinstance(grouping, (pd.Index, pd.Series, pd.DataFrame)):
+                raise TypeError(
+                    "The callable should evaluate to either "
+                    "a pandas DataFrame, Index, or Series; "
+                    f"instead got {type(grouping)}."
+                )
+            column_checker = _check_pandas_object(
+                grouping=grouping, column_checker=column_checker
+            )
+
+        elif isinstance(grouping, (pd.DataFrame, pd.Index, pd.Series)):
+            column_checker = _check_pandas_object(
+                grouping=grouping, column_checker=column_checker
+            )
+        else:
+            raise TypeError(
+                "The argument to the variable columns parameter "
+                "in the complete function "
+                "should be a scalar, a list, dict, tuple, "
+                "pandas Index, Series, DataFrame, or callable; "
+                f"instead, got {type(grouping)}"
+            )
 
     # columns should not be duplicated across groups
     # nor should it exist in `by`
@@ -492,3 +480,67 @@ def _data_checks_complete(
                 )
 
     return columns, column_checker, sort, by, fill_value, explicit
+
+
+def _check_pandas_object(grouping: Any, column_checker: list):
+    """
+    Check if object is a pandas object.
+    """
+    if isinstance(grouping, pd.DataFrame):
+        column_checker.extend(grouping.columns)
+    elif isinstance(grouping, pd.MultiIndex):
+        if None in grouping.names:
+            raise ValueError("Ensure all labels in the MultiIndex are named.")
+        column_checker.extend(grouping.names)
+    elif isinstance(grouping, (pd.Series, pd.Index)):
+        if not grouping.name:
+            name_of_type = type(grouping).__name__
+            raise ValueError(f"Ensure the {name_of_type} has a name.")
+        column_checker.append(grouping.name)
+    return column_checker
+
+
+def _create_pandas_objects_from_dict(df, column):
+    """
+    Create pandas object if column is a dictionary
+    """
+    collection = []
+    for key, value in column.items():
+        arr = apply_if_callable(value, df)
+        if not is_list_like(arr):
+            raise TypeError(
+                f"Expected a list-like object for {key}; "
+                f"instead got {type(arr)}."
+            )
+        if not hasattr(arr, "shape"):
+            arr = np.asanyarray(arr)
+        if not isinstance(arr, (pd.Index, pd.Series)):
+            arr = pd.Series(arr)
+        arr.name = key
+        collection.append(arr)
+    return collection
+
+
+def _create_pandas_object(df, columns, sort):
+    """
+    Create pandas objects before building the cartesian DataFrame.
+    """
+    objects = []
+    for column in columns:
+        if is_scalar(column):
+            _object = df[column].drop_duplicates()
+            if sort:
+                _object = _object.sort_values()
+            objects.append(_object)
+        elif isinstance(column, list):
+            _object = df.loc[:, column].drop_duplicates()
+            if sort:
+                _object = _object.sort_values(column)
+            objects.append(_object)
+        elif isinstance(column, dict):
+            _object = _create_pandas_objects_from_dict(df=df, column=column)
+            objects.extend(_object)
+        else:
+            _object = apply_if_callable(maybe_callable=column, obj=df)
+            objects.append(_object)
+    return objects


Test patch:
diff --git a/tests/functions/test_complete.py b/tests/functions/test_complete.py
index 9808478..5d65bc8 100644
--- a/tests/functions/test_complete.py
+++ b/tests/functions/test_complete.py
@@ -88,7 +88,7 @@ def test_column_None(fill_df):
     assert_frame_equal(fill_df.complete(), fill_df)
 
 
-def test_empty_groups(fill_df):
+def test_empty_groups_dict(fill_df):
     """Raise ValueError if any of the groups is empty."""
     with pytest.raises(
         ValueError, match="entry in columns argument cannot be empty"
@@ -96,16 +96,34 @@ def test_empty_groups(fill_df):
         fill_df.complete("group", {})
 
 
+def test_empty_groups_list(fill_df):
+    """Raise ValueError if any of the groups is empty."""
+    with pytest.raises(
+        ValueError, match="entry in columns argument cannot be empty"
+    ):
+        fill_df.complete("group", [])
+
+
+def test_callable(fill_df):
+    """Raise ValueError if the callable does not evaluate to a pandas object."""
+    with pytest.raises(
+        TypeError,
+        match=r"The callable should evaluate to either a pandas.+",
+    ):
+        fill_df.complete("group", lambda f: 1)
+
+
 def test_dict_not_list_like(fill_df):
     """
     Raise ValueError if `*columns`
     is a dictionary, and the value
     is not list-like.
     """
-    with pytest.raises(ValueError):
+    with pytest.raises(TypeError, match="Expected a list-like object.+"):
         fill_df.complete("group", {"item_id": "cities"})
 
 
+@pytest.mark.xfail(reason="dictionary now accepts a MultiIndex")
 def test_dict_not_1D(fill_df):
     """
     Raise ValueError if `*columns`
@@ -118,6 +136,7 @@ def test_dict_not_1D(fill_df):
         fill_df.complete("group", {"item_id": fill_df})
 
 
+@pytest.mark.xfail(reason="no checks on pandas object's contents")
 def test_dict_empty(fill_df):
     """
     Raise ValueError if `*columns`
@@ -138,6 +157,34 @@ def test_by_not_found(fill_df):
         fill_df.complete("group", "item_id", by="name")
 
 
+def test_mi_none(fill_df):
+    """
+    Raise if MultiIndex has None as one of its names
+    """
+    mi = fill_df.loc[:, ["item_id", "item_name"]].drop_duplicates()
+    mi = pd.MultiIndex.from_frame(mi, names=[None, "item_name"])
+    with pytest.raises(
+        ValueError, match="Ensure all labels in the MultiIndex.+"
+    ):
+        fill_df.complete("group", mi)
+
+
+def test_index_none(fill_df):
+    """
+    Raise if Index has None as its name
+    """
+    with pytest.raises(ValueError, match="Ensure the Index has a name."):
+        fill_df.complete("group", pd.Index([0, 1, 2]))
+
+
+def test_series_none(fill_df):
+    """
+    Raise if Series has None as its name
+    """
+    with pytest.raises(ValueError, match="Ensure the Series has a name."):
+        fill_df.complete("group", pd.Series([0, 1, 2]))
+
+
 def test_group_None(fill_df):
     """Raise ValueError if entry is None."""
     with pytest.raises(
@@ -284,15 +331,17 @@ def test_dict_extension_array(df):
 
 @given(df=categoricaldf_strategy())
 @settings(deadline=None, max_examples=10)
-def test_dict_numpy(df):
+def test_dict_Index(df):
     """
     Test `complete` output when *columns
     is a dictionary, and value is
-    a numpy array.
+    a pandas array.
     """
     cols = ["names", "numbers"]
     df = df.assign(names=[*ascii_lowercase[: len(df)]])
-    new_numbers = np.arange(df.numbers.min(), df.numbers.max() + 1)
+    new_numbers = pd.RangeIndex(
+        start=df.numbers.min(), stop=df.numbers.max() + 1, name="numbers"
+    )
     new_numbers = {"numbers": new_numbers}
     cols = ["numbers", "names"]
     result = df.complete(new_numbers, "names", sort=True)
@@ -311,18 +360,16 @@ def test_dict_numpy(df):
 
 @given(df=categoricaldf_strategy())
 @settings(deadline=None, max_examples=10)
-def test_dict_Index(df):
+def test_complete_Index(df):
     """
     Test `complete` output when *columns
-    is a dictionary, and value is
-    a pandas array.
+    is an Index.
     """
     cols = ["names", "numbers"]
     df = df.assign(names=[*ascii_lowercase[: len(df)]])
     new_numbers = pd.RangeIndex(
-        start=df.numbers.min(), stop=df.numbers.max() + 1
+        start=df.numbers.min(), stop=df.numbers.max() + 1, name="numbers"
     )
-    new_numbers = {"numbers": new_numbers}
     cols = ["numbers", "names"]
     result = df.complete(new_numbers, "names", sort=True)
     columns = df.columns
@@ -340,19 +387,46 @@ def test_dict_Index(df):
 
 @given(df=categoricaldf_strategy())
 @settings(deadline=None, max_examples=10)
-def test_dict_duplicated(df):
+def test_complete_Series(df):
     """
     Test `complete` output when *columns
-    is a dictionary, and value is
-    duplicated.
+    is a Series.
     """
     cols = ["names", "numbers"]
     df = df.assign(names=[*ascii_lowercase[: len(df)]])
-    new_numbers = pd.RangeIndex(
-        start=df.numbers.min(), stop=df.numbers.max() + 1
+    new_numbers = range(df.numbers.min(), df.numbers.max() + 1)
+    new_numbers = pd.Series(new_numbers, name="numbers")
+    cols = ["numbers", "names"]
+    result = df.complete(new_numbers, "names", sort=True)
+    columns = df.columns
+    new_index = range(df.numbers.min(), df.numbers.max() + 1)
+    new_index = pd.MultiIndex.from_product([new_index, df.names], names=cols)
+    expected = (
+        df.set_index(cols)
+        .reindex(new_index)
+        .reset_index()
+        .reindex(columns=columns)
     )
-    new_numbers = new_numbers.append(new_numbers)
-    new_numbers = {"numbers": new_numbers}
+
+    assert_frame_equal(result, expected)
+
+
+@given(df=categoricaldf_strategy())
+@settings(deadline=None, max_examples=10)
+def test_complete_callable(df):
+    """
+    Test `complete` output when *columns
+    is a callable.
+    """
+    cols = ["names", "numbers"]
+    df = df.assign(names=[*ascii_lowercase[: len(df)]])
+
+    def _index(df):
+        return pd.RangeIndex(
+            start=df.numbers.min(), stop=df.numbers.max() + 1, name="numbers"
+        )
+
+    new_numbers = _index(df=df)
     cols = ["numbers", "names"]
     result = df.complete(new_numbers, "names", sort=True)
     columns = df.columns
@@ -372,7 +446,10 @@ def test_dict_duplicated(df):
 def test_single_column(df):
     """Test `complete` output if a single column is provided."""
     result = df.complete("names")
-    assert_frame_equal(result, df)
+    assert_frame_equal(
+        result.sort_values(df.columns.tolist(), ignore_index=True),
+        df.sort_values(df.columns.tolist(), ignore_index=True),
+    )
 
 
 def test_tuple_column():
@@ -401,6 +478,67 @@ def test_tuple_column():
     assert_frame_equal(result, expected)
 
 
+def test_pandas_dataframe():
+    """Test `complete` output if a DataFrame is provided."""
+    df = pd.DataFrame(
+        {
+            "group": [2, 1, 1],
+            "item_id": [2, 1, 2],
+            "item_name": ["b", "a", "b"],
+            "value1": [2, 1, 3],
+            "value2": [5, 4, 6],
+        }
+    )
+
+    result = df.complete(
+        "group",
+        df.loc[:, ["item_id", "item_name"]].drop_duplicates(),
+        sort=True,
+    )
+
+    columns = ["group", "item_id", "item_name"]
+    expected = (
+        df.set_index(columns)
+        .unstack("group")
+        .stack(future_stack=True)
+        .reset_index()
+        .reindex(columns=df.columns)
+        .sort_values(columns, ignore_index=True)
+    )
+    assert_frame_equal(result, expected)
+
+
+def test_pandas_multiindex():
+    """Test `complete` output if a MultiIndex is provided."""
+    df = pd.DataFrame(
+        {
+            "group": [2, 1, 1],
+            "item_id": [2, 1, 2],
+            "item_name": ["b", "a", "b"],
+            "value1": [2, 1, 3],
+            "value2": [5, 4, 6],
+        }
+    )
+    index = df.loc[:, ["item_id", "item_name"]].drop_duplicates()
+    index = pd.MultiIndex.from_frame(index)
+    result = df.complete(
+        "group",
+        index,
+        sort=True,
+    )
+
+    columns = ["group", "item_id", "item_name"]
+    expected = (
+        df.set_index(columns)
+        .unstack("group")
+        .stack(future_stack=True)
+        .reset_index()
+        .reindex(columns=df.columns)
+        .sort_values(columns, ignore_index=True)
+    )
+    assert_frame_equal(result, expected)
+
+
 # https://stackoverflow.com/questions/48914323/tidyr-complete-cases-nesting-misunderstanding
 def test_complete_multiple_groupings():
     """Test that `complete` gets the correct output for multiple groupings."""
@@ -460,7 +598,7 @@ def test_dict_tuple_callable(taxonomy_df):
     """
 
     result = taxonomy_df.complete(
-        {"Year": lambda x: range(x.Year.min(), x.Year.max() + 1)},
+        {"Year": lambda x: pd.RangeIndex(x.Year.min(), x.Year.max() + 1)},
         ("Taxon", "Abundance"),
         sort=True,
     )
@@ -488,7 +626,7 @@ def test_dict_tuple(taxonomy_df):
     """
 
     result = taxonomy_df.complete(
-        {"Year": [2000, 1999, 2001, 2002, 2003, 2004]},
+        {"Year": pd.Series([2000, 1999, 2001, 2002, 2003, 2004])},
         ("Taxon", "Abundance"),
         sort=True,
     )
@@ -520,7 +658,7 @@ def test_complete_groupby():
     )
 
     result = df.complete(
-        {"year": lambda x: range(x.year.min(), x.year.max() + 1)},
+        {"year": lambda x: pd.Series(range(x.year.min(), x.year.max() + 1))},
         by="state",
         sort=True,
     )
@@ -697,88 +835,72 @@ def test_nulls(fill_df):
     """
     Test output if nulls are present
     """
-    actual = fill_df.complete(["value1"], "value2", sort=True)
-    ind = [fill_df.value1.dropna().unique(), fill_df.value2.unique()]
-    ind = pd.MultiIndex.from_product(ind, names=["value1", "value2"])
-    ind = pd.DataFrame([], index=ind)
-    expected = fill_df.merge(
-        ind, on=["value1", "value2"], how="outer", sort=True
+    columns = fill_df.columns.tolist()
+    actual = (
+        fill_df.complete(["value1"], "value2", sort=True)
+        .loc[:, columns]
+        .sort_values(columns, ignore_index=True)
+    )
+    expected = (
+        fill_df.set_index(["value1", "value2"])
+        .unstack("value2")
+        .stack("value2", future_stack=True)
+        .reset_index()
+        .loc[:, columns]
+        .sort_values(columns, ignore_index=True)
     )
+
     assert_frame_equal(actual, expected)
 
 
 def test_groupby_tuple():
     """Test output for groupby on a tuple of columns."""
     # https://stackoverflow.com/q/77123843/7175713
-    data_dict = {
-        "Grid Cell": [1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2],
-        "Site": [
-            "A",
-            "A",
-            "A",
-            "A",
-            "B",
-            "B",
-            "B",
-            "C",
-            "C",
-            "C",
-            "D",
-            "D",
-            "D",
-            "D",
-        ],
-        "Date": [
-            "1999-01-01",
-            "1999-02-01",
-            "1999-03-01",
-            "1999-04-01",
-            "1999-01-01",
-            "1999-02-01",
-            "1999-03-01",
-            "2000-01-01",
-            "2000-02-01",
-            "2000-03-01",
-            "2000-01-01",
-            "2000-02-01",
-            "2000-03-01",
-            "2000-04-01",
-        ],
-        "Value": [
-            -2.45,
-            -3.72,
-            1.34,
-            4.56,
-            0.23,
-            3.26,
-            6.76,
-            -7.45,
-            -6.43,
-            -2.18,
-            -10.72,
-            -8.97,
-            -5.32,
-            -1.73,
-        ],
-    }
+    data_dict = [
+        {"Grid Cell": 1, "Site": "A", "Date": "1999-01-01", "Value": -2.45},
+        {"Grid Cell": 1, "Site": "A", "Date": "1999-02-01", "Value": -3.72},
+        {"Grid Cell": 1, "Site": "A", "Date": "1999-03-01", "Value": 1.34},
+        {"Grid Cell": 1, "Site": "A", "Date": "1999-04-01", "Value": 4.56},
+        {"Grid Cell": 1, "Site": "B", "Date": "1999-01-01", "Value": 0.23},
+        {"Grid Cell": 1, "Site": "B", "Date": "1999-02-01", "Value": 3.26},
+        {"Grid Cell": 1, "Site": "B", "Date": "1999-03-01", "Value": 6.76},
+        {"Grid Cell": 2, "Site": "C", "Date": "2000-01-01", "Value": -7.45},
+        {"Grid Cell": 2, "Site": "C", "Date": "2000-02-01", "Value": -6.43},
+        {"Grid Cell": 2, "Site": "C", "Date": "2000-03-01", "Value": -2.18},
+        {"Grid Cell": 2, "Site": "D", "Date": "2000-01-01", "Value": -10.72},
+        {"Grid Cell": 2, "Site": "D", "Date": "2000-02-01", "Value": -8.97},
+        {"Grid Cell": 2, "Site": "D", "Date": "2000-03-01", "Value": -5.32},
+        {"Grid Cell": 2, "Site": "D", "Date": "2000-04-01", "Value": -1.73},
+    ]
     df = pd.DataFrame.from_dict(data_dict)
-    expected = df.complete("Date", "Site", by="Grid Cell").sort_values(
-        ["Grid Cell", "Site", "Date"], ignore_index=True
+    expected = (
+        df.complete("Date", "Site", by="Grid Cell")
+        .sort_values(["Grid Cell", "Site", "Date"], ignore_index=True)
+        .loc[:, ["Grid Cell", "Site", "Date", "Value"]]
     )
 
     # https://stackoverflow.com/a/77123963/7175713
-    def reindex(g):
-        idx = pd.MultiIndex.from_product(
-            [g["Grid Cell"].unique(), g["Site"].unique(), g["Date"].unique()],
-            names=["Grid Cell", "Site", "Date"],
-        )
-        return g.set_index(["Grid Cell", "Site", "Date"]).reindex(
-            idx, fill_value=np.nan
-        )
+    data = [
+        {"Grid Cell": 1, "Site": "A", "Date": "1999-01-01", "Value": -2.45},
+        {"Grid Cell": 1, "Site": "A", "Date": "1999-02-01", "Value": -3.72},
+        {"Grid Cell": 1, "Site": "A", "Date": "1999-03-01", "Value": 1.34},
+        {"Grid Cell": 1, "Site": "A", "Date": "1999-04-01", "Value": 4.56},
+        {"Grid Cell": 1, "Site": "B", "Date": "1999-01-01", "Value": 0.23},
+        {"Grid Cell": 1, "Site": "B", "Date": "1999-02-01", "Value": 3.26},
+        {"Grid Cell": 1, "Site": "B", "Date": "1999-03-01", "Value": 6.76},
+        {"Grid Cell": 1, "Site": "B", "Date": "1999-04-01", "Value": np.nan},
+        {"Grid Cell": 2, "Site": "C", "Date": "2000-01-01", "Value": -7.45},
+        {"Grid Cell": 2, "Site": "C", "Date": "2000-02-01", "Value": -6.43},
+        {"Grid Cell": 2, "Site": "C", "Date": "2000-03-01", "Value": -2.18},
+        {"Grid Cell": 2, "Site": "C", "Date": "2000-04-01", "Value": np.nan},
+        {"Grid Cell": 2, "Site": "D", "Date": "2000-01-01", "Value": -10.72},
+        {"Grid Cell": 2, "Site": "D", "Date": "2000-02-01", "Value": -8.97},
+        {"Grid Cell": 2, "Site": "D", "Date": "2000-03-01", "Value": -5.32},
+        {"Grid Cell": 2, "Site": "D", "Date": "2000-04-01", "Value": -1.73},
+    ]
+
+    actual = pd.DataFrame(data)
 
-    actual = (
-        df.groupby("Grid Cell", group_keys=False).apply(reindex).reset_index()
-    )
     assert_frame_equal(expected, actual)
 
 
@@ -793,5 +915,5 @@ def test_MI_1(MI):
         how="outer",
         sort=True,
     ).rename_axis(columns=[None, None])
-    actual = MI.iloc[:2].complete({("a", "bar"): range(1, 5)})
+    actual = MI.iloc[:2].complete({("a", "bar"): pd.Series(range(1, 5))})
     assert_frame_equal(actual, expected)