add typing_extensions to dependencies
State: closed

- [ ] closes #25974 
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Dependencies
description: Required and optional dependencies
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
@WillAyd I have added `typing_extensions`(without freezing the version) to both `environment.yml` and `requirements-dev.txt` manually because I am not using anaconda(not a big fan of anaconda). Take a look if everything looks alright.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=h1) Report
> Merging [#25975](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/4814a2875d078d6e9e9300bd1f77af15ef4f0f0a?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25975/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25975      +/-   ##
==========================================
- Coverage   91.84%   91.84%   -0.01%     
==========================================
  Files         175      175              
  Lines       52550    52550              
==========================================
- Hits        48266    48262       -4     
- Misses       4284     4288       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.39% <ø> (ø)` | :arrow_up: |
| #single | `41.89% <ø> (-0.08%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/25975/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `75% <0%> (-12.5%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/25975/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.79% <0%> (-0.12%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=footer). Last update [4814a28...1807a23](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=h1) Report
> Merging [#25975](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/d86553b00fe263d264bd0b2ce691724591dc7b01?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25975/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25975      +/-   ##
==========================================
- Coverage   91.96%   91.95%   -0.01%     
==========================================
  Files         175      175              
  Lines       52405    52408       +3     
==========================================
- Hits        48193    48191       -2     
- Misses       4212     4217       +5
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.51% <100%> (ø)` | :arrow_up: |
| #single | `40.73% <100%> (-0.13%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/dtypes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/25975/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Jhc2UucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [pandas/core/arrays/datetimelike.py](https://codecov.io/gh/pandas-dev/pandas/pull/25975/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2RhdGV0aW1lbGlrZS5weQ==) | `97.69% <100%> (ø)` | :arrow_up: |
| [pandas/core/groupby/groupby.py](https://codecov.io/gh/pandas-dev/pandas/pull/25975/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9ncm91cGJ5LnB5) | `97.23% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/25975/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `75% <0%> (-12.5%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/25975/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.9% <0%> (-0.12%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25975/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `90.61% <0%> (-0.11%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=footer). Last update [d86553b...4abfc68](https://codecov.io/gh/pandas-dev/pandas/pull/25975?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Can you check master in a 3.5.1 environment? Might need to import Type from this library as part of this

Sent from my iPhone

> On Apr 3, 2019, at 6:58 AM, codecov[bot] <notifications@github.com> wrote:
> 
> Codecov Report
> 
> Merging #25975 into master will decrease coverage by <.01%.
> The diff coverage is n/a.
> 
> 
> @@            Coverage Diff             @@
> ##           master   #25975      +/-   ##
> ==========================================
> - Coverage   91.84%   91.84%   -0.01%     
> ==========================================
>   Files         175      175              
>   Lines       52550    52550              
> ==========================================
> - Hits        48266    48262       -4     
> - Misses       4284     4288       +4
> Flag	Coverage Δ	
> #multiple	90.39% <ø> (ø)	⬆️
> #single	41.89% <ø> (-0.08%)	⬇️
> Impacted Files	Coverage Δ	
> pandas/io/gbq.py	75% <0%> (-12.5%)	⬇️
> pandas/core/frame.py	96.79% <0%> (-0.12%)	⬇️
> Continue to review full report at Codecov.
> 
> Legend - Click here to learn more
> Δ = absolute <relative> (impact), ø = not affected, ? = missing data
> Powered by Codecov. Last update 4814a28...1807a23. Read the comment docs.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

@WillAyd  I am not sure what you mean.
Should I import `Type` from `typing_extensions` and then run tests in python 3.5.1 locally?
Can you check if current master runs on 3.5.1 first? If not then yes might need to import from type_extensions to resolve 

Sent from my iPhone

> On Apr 3, 2019, at 7:21 AM, Vaibhav Vishal <notifications@github.com> wrote:
> 
> @WillAyd I am not sure what you mean.
> Should I import Type from typing_extensions and then run tests in python 3.5.1 locally?
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

we should pin one of the builds to 3.5.1

separately not averse to bumping to 3.5.3 (or whatever we need)
Makes sense. I think we could pin 3.5.1 as part of this PR and tackle maybe bumping to 3.5.3 separately if we wanted to
We provide instructions for conda but you could get to the same place using pip or pipenv for sure 

Sent from my iPhone

> On Apr 3, 2019, at 1:59 PM, Vaibhav Vishal <notifications@github.com> wrote:
> 
> @vaibhavhrt commented on this pull request.
> 
> In environment.yml:
> 
> > @@ -25,6 +25,7 @@ dependencies:
>    - pytest-mock
>    - sphinx
>    - numpydoc
> +  - typing_extensions
> @WillAyd does it matter how I create virtualenv(anaconda, pipenv) when running tests locally. My system is low on space and anaconda takes up a lot of space. So is it ok if I use pipenv to create env and then run pytest pandas or will that make any difference.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

@WillAyd I finally managed to get MS Build Tools 14 working and build pandas. I should be able to run tests locally now. But I am too busy at work today. Tomorrow I will run tests on python 3.5.1 and let you know the results. I will take care of changes requested by @jreback too tomorrow.
What exactly are we using from this? If we’re only using one or two definitions, I’d rather just include those in our typing compat module. Adding a new dependency shouldn’t be done lightly. 
@TomAugspurger what's the concern with third party library? There's a lot of active development with types so I think it would be more work to vendor pieces of it and keep in sync rather than to include as dependency
@WillAyd on python 3.5.1 even master will fail. I can't even import pandas from local build. Here is relevant part of traceback:

    ...
      File "C:\Users\Harish\Documents\GitHub\pandas\pandas\core\dtypes\base.py", line 2, in <module>
      from typing import List, Optional, Type
    ImportError: cannot import name 'Type'

[typing.Type](https://docs.python.org/3/library/typing.html#typing.Type) was added in 3.5.2

I will fix this.
@WillAyd After adding typing_extensions to dependency and importing `Type` from `typing_extensions` 
I got the following test results:

     22 failed, 47816 passed, 1368 skipped, 1028 xfailed, 15 xpassed, 3469 warnings in 4985.95 seconds

I can't see how any of the changes I made can make tests fail. I have pushed the changes, you can see tests results on travis and let me know how to fix it.
@WillAyd tests are passing on travis but not locally, don't know why. Also some azure tests for mac and linux are failing. Do I remove `typing-extensions` from yml of those tests or should I install them from pip instead of conda?
You’d need to be explicit about which failures you are getting locally

Sent from my iPhone

> On Apr 5, 2019, at 4:59 AM, Vaibhav Vishal <notifications@github.com> wrote:
> 
> @WillAyd tests are passing on travis but not locally, don't know why. Also some azure tests for mac and linux are failing. Do I remove typing-extensions from yml of those tests or should I install them from pip instead of conda?
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

@WillAyd running tests locally takes a long time. I will worry about that some other time. Travis tests(for my fork too) is passing, so there is something wrong with my system. For now please let me know to fix the Azure tests(some of them are failing). They all have similar issue:

    conda env create -q --file=ci/deps/azure-macos-35.yaml
    Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.
    Collecting package metadata: ...working... done
    Solving environment: ...working... failed
    WARNING: The conda.compat module is deprecated and will be removed in a future release.

    ResolvePackageNotFound: 
      - typing-extensions

    ##[error]Bash exited with code '1'.

I think I need to change to something in the the `yaml` files of these tests but not sure what.
conda-forge doesn't build packages for 3.5 (which demonstrates one issue
with taking on dependencies @WillAyd).

On Fri, Apr 5, 2019 at 10:11 AM Vaibhav Vishal <notifications@github.com>
wrote:

> @WillAyd <https://github.com/WillAyd> running tests locally takes a long
> time. I will worry about that some other time. Travis tests(for my fork
> too) is passing, so there is something wrong with my system. For now please
> let me know to fix the Azure tests(some of them are failing). They all have
> similar issue:
>
> conda env create -q --file=ci/deps/azure-macos-35.yaml
> Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.
> Collecting package metadata: ...working... done
> Solving environment: ...working... failed
> WARNING: The conda.compat module is deprecated and will be removed in a future release.
>
> ResolvePackageNotFound:
>   - typing-extensions
>
> ##[error]Bash exited with code '1'.
>
> I think I need to change to something in the the yaml files of these
> tests but not sure what.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/25975#issuecomment-480312333>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIj1lR-8YeAAcyyrD31XCGeSvN5o9ks5vd2edgaJpZM4cagFJ>
> .
>

Hmm ok. So you think better to just add Type and Final to our internal typing package? Not opposed if that’s easier

Sent from my iPhone

> On Apr 5, 2019, at 8:17 AM, Tom Augspurger <notifications@github.com> wrote:
> 
> conda-forge doesn't build packages for 3.5 (which demonstrates one issue
> with taking on dependencies @WillAyd).
> 
> On Fri, Apr 5, 2019 at 10:11 AM Vaibhav Vishal <notifications@github.com>
> wrote:
> 
> > @WillAyd <https://github.com/WillAyd> running tests locally takes a long
> > time. I will worry about that some other time. Travis tests(for my fork
> > too) is passing, so there is something wrong with my system. For now please
> > let me know to fix the Azure tests(some of them are failing). They all have
> > similar issue:
> >
> > conda env create -q --file=ci/deps/azure-macos-35.yaml
> > Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies. Conda may not use the correct pip to install your packages, and they may end up in the wrong place. Please add an explicit pip dependency. I'm adding one for you, but still nagging you.
> > Collecting package metadata: ...working... done
> > Solving environment: ...working... failed
> > WARNING: The conda.compat module is deprecated and will be removed in a future release.
> >
> > ResolvePackageNotFound:
> > - typing-extensions
> >
> > ##[error]Bash exited with code '1'.
> >
> > I think I need to change to something in the the yaml files of these
> > tests but not sure what.
> >
> > —
> > You are receiving this because you were mentioned.
> > Reply to this email directly, view it on GitHub
> > <https://github.com/pandas-dev/pandas/pull/25975#issuecomment-480312333>,
> > or mute the thread
> > <https://github.com/notifications/unsubscribe-auth/ABQHIj1lR-8YeAAcyyrD31XCGeSvN5o9ks5vd2edgaJpZM4cagFJ>
> > .
> >
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

> Hmm ok. So you think better to just add Type and Final to our internal typing package? Not opposed if that’s easier

yes let's just add what we need to _typing, rather that this dep.
@jreback if no one is already working on adding `Type` and `Final` to `pandas.typing` I can do that. Let me know if I should open a new issue and PR for that. I am closing this PR.