TYP: lib.no_default
State: closed

@Dr-Irv in common with other public api typing discussions, the lib.NoDefault type annotations serves no purpose for the user facing api.

It is beneficial tough for ensuring that internal code handles it correctly. However, this type is actually wrong since we should not handle any NoDefault subclass/instance, only the singleton.

Unfortunately, it appears that recent code additions have coded for any subclass/instance instead.

As common with typing PRs there has been scope creep on this and ended up making code changes. Have marked as draft for now and will probably break off the code changes.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking
name: Stale

Comments:
> @Dr-Irv in common with other public api typing discussions, the lib.NoDefault type annotations serves no purpose for the user facing api.

I agree.  The current MS stubs do not use `lib.NoDefault` at all.

There is a PEP that is proposed to handle sentinel values like this:  https://peps.python.org/pep-0661/

So the proposed change here would make it easier for us to switch over to this (and eliminate `lib.no_default`) in the future.

> There is a PEP that is proposed to handle sentinel values like this: https://peps.python.org/pep-0661/

yes,

```
Type annotations for sentinel values will use [typing.Literal](https://docs.python.org/3/library/typing.html#typing.Literal). For example:

def foo(value: int | Literal[NotGiven]) -> None:
    ...
```

is what we eventually want for the internal code, to make sure we handle lib.no_default if passed onto the lower level functions but i'm not sure this is good for the public facing api where the sentinel is the default keyword argument. Yes it should allow us to ensure that the public facing function handles lib.no_default correctly without the false positives we get with `is` not narrowing the type but it also exposes it to the user. maybe we will need very thin functions when using lib.no_default that directly call lower level functions so that we don't expose these annotations to users.

Maybe we could also do that now (instead of this PR), but we would still have the false positives from mypy not knowing that we have narrowed from `is`, until the Literal form works.
> maybe we will need very thin functions when using lib.no_default that directly call lower level functions so that we don't expose these annotations to users.

I think this would duplicate many overloads. [It's a shame that there is no way to use a TypedDict for keywords that are not overload-related: would reduce many overloads to only a couple of lines.]
> I think this would duplicate many overloads.

and the other downside is that we would need the type of lib.no_default defined so would need to add ignores to the public functions if we wanted to omit it.

overall I think this is a case where for now, we accept that using type checking is more of a disadvantage than an advantage at the moment and that Any is actually the better solution overall.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.