CLN: replace usage internally of .iteritems with .items
State: closed

- [x] xref #25725
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Now that Python2 is being removed, ``.iteritems`` should be deprecated, and ``.items`` should be used instead.

EDIT: This PR ended up not deprecating .iteritems, but only change its usage internally with using .items

Labels:
name: Clean

Comments:
The error is unrelated to this PR:

```
CondaHTTPError: HTTP 504 GATEWAY_TIMEOUT for url <https://conda.anaconda.org/c3i_test/noarch/repodata.json>
Elapsed: 00:59.894676
CF-RAY: 4c8a8f7d1f385558-ORD
A remote server error occurred when trying to retrieve this URL.
A 500-type error (e.g. 500, 501, 502, 503, etc.) indicates the server failed to
fulfill a valid request.  The problem may be spurious, and will resolve itself if you
try your request again.  If the problem persists, consider notifying the maintainer
of the remote server.
```
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=h1) Report
> Merging [#26114](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/68b1da7f53f043b21e3f722e8872e19941a10250?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `90%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26114/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26114      +/-   ##
==========================================
- Coverage   91.99%   91.97%   -0.02%     
==========================================
  Files         175      175              
  Lines       52384    52392       +8     
==========================================
+ Hits        48189    48190       +1     
- Misses       4195     4202       +7
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.53% <90%> (-0.01%)` | :arrow_down: |
| #single | `40.73% <30%> (-0.14%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/panel.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcGFuZWwucHk=) | `35.43% <0%> (ø)` | :arrow_up: |
| [pandas/core/reshape/pivot.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9waXZvdC5weQ==) | `96.54% <100%> (ø)` | :arrow_up: |
| [pandas/io/json/table\_schema.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2pzb24vdGFibGVfc2NoZW1hLnB5) | `98.3% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexes/multi.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9tdWx0aS5weQ==) | `95.61% <100%> (ø)` | :arrow_up: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `83.76% <100%> (ø)` | :arrow_up: |
| [pandas/core/util/hashing.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdXRpbC9oYXNoaW5nLnB5) | `98.4% <100%> (ø)` | :arrow_up: |
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `99.56% <100%> (ø)` | :arrow_up: |
| [pandas/core/strings.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3RyaW5ncy5weQ==) | `98.86% <100%> (ø)` | :arrow_up: |
| [pandas/core/sparse/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL2ZyYW1lLnB5) | `95.49% <100%> (ø)` | :arrow_up: |
| [pandas/io/formats/style.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvc3R5bGUucHk=) | `96.68% <100%> (ø)` | :arrow_up: |
| ... and [7 more](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=footer). Last update [68b1da7...397bf66](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=h1) Report
> Merging [#26114](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/c407b7308cefdc8b66e25c61f1ad21f1f70ca121?src=pr&el=desc) will **increase** coverage by `0.07%`.
> The diff coverage is `90%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26114/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26114      +/-   ##
==========================================
+ Coverage   91.79%   91.86%   +0.07%     
==========================================
  Files         180      174       -6     
  Lines       50934    50700     -234     
==========================================
- Hits        46756    46577     -179     
+ Misses       4178     4123      -55
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.4% <90%> (-0.08%)` | :arrow_down: |
| #single | `41.78% <30%> (-0.32%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/panel.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcGFuZWwucHk=) | `17.61% <0%> (+1.74%)` | :arrow_up: |
| [pandas/core/reshape/pivot.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9waXZvdC5weQ==) | `96.52% <100%> (ø)` | :arrow_up: |
| [pandas/io/json/table\_schema.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2pzb24vdGFibGVfc2NoZW1hLnB5) | `98.3% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexes/multi.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9tdWx0aS5weQ==) | `95.67% <100%> (+0.81%)` | :arrow_up: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `83.77% <100%> (-5.61%)` | :arrow_down: |
| [pandas/core/util/hashing.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdXRpbC9oYXNoaW5nLnB5) | `98.41% <100%> (ø)` | :arrow_up: |
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `99.56% <100%> (ø)` | :arrow_up: |
| [pandas/core/strings.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3RyaW5ncy5weQ==) | `98.92% <100%> (-0.02%)` | :arrow_down: |
| [pandas/core/sparse/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL2ZyYW1lLnB5) | `95.63% <100%> (+2.06%)` | :arrow_up: |
| [pandas/io/formats/style.py](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvc3R5bGUucHk=) | `96.68% <100%> (-0.47%)` | :arrow_down: |
| ... and [115 more](https://codecov.io/gh/pandas-dev/pandas/pull/26114/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=footer). Last update [c407b73...4fec810](https://codecov.io/gh/pandas-dev/pandas/pull/26114?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

yeah I guess we need to deprecate this. let me have a look.
Is it needed to deprecate this?

I understand it that from the stdlib python 2/3 point of view, it makes sense: `items` + `iteritems` were originally modelled after dict's `items` and `iteritems` methods, and now with python 3 only, we only need to keep `items`.

However, leaving that aspect aside, from a consistency within pandas, I think having the methods `iteritems` / `itertuples` / `iterrows` is also nice. And I personally prefer to keep this consistency compared to the one with dict.


@jorisvandenbossche 

``iterrows`` and ``itertuples`` are code smells really and if encountered in a code base, the instinct should be to try to refactor into something vectorized.

``iteritems``/``items`` OTOH is idiomatic and useful for a lot of problem types. So I don't have a problem with distancing these methods from each other by just deprecating ``iteritems``. The grouping of these methods is deceptional and if you've learned that ``iteritems`` is a good method you may be lead to believe that ``iterrows`` and ``itertuples`` can be used in a similar manner, but really you shouldn't in 99% of the cases.

The dict argument is also important IMO, as having DataFrames/Series be dict-like is a nice pedagogical concept, and keeping``iteritems`` will require more explanation for the user (it's a python2 concept used to avoid returning lists, but that we chose to keep in order to keep similarity with ``iterrows`` and ``itertuples`` (but don't use those, because...), but it's really the same as ``items``, so just use ``items`` ...).

So, I think removing ``iteritems`` will make the basic pandas concepts a bit easier to learn/use and will encourage healthier habits for non-expert users.
I actually like this deprecation; it clearly delineates that that we are iterating over columns, rather than the row-iterators; plus it does make sense post python 2 for more compatibility in Series with dict.

@jorisvandenbossche ?
My main problem with deprecating this is that we have *consistently* documented to use `iteritems()` instead of `items()` (in the user guide, in the docstrings, in see also's, ... even the docstring of `items()` uses `iteritems()` in the example. `items()` is basically not mentioned a single time). So ideally, we should have thought about this much earlier to update that ... 
Because now we would want to deprecate that method that is documented in favor of an alias probably almost no one uses. 
A solution to your concern could be to keep the deprecations in 0.25 in v1.0 (+ maybe 2 minor versions)? That would not be unreasonable anyway for changes so close to 1.0.

Another alternative could be to make ``items`` the "advertised" method in 0.25, and just have a warning text in the ``iteritems`` doc string about a future deprecation, without actually deprecating it? This would be kind of like we did with ``tolist`` when ``to_list`` was added.

Alternatively IMO advertising DataFrames/Seried having a dict-like interface should probably be de-emphasised. E.g. [the phrase here](http://pandas-docs.github.io/pandas-docs-travis/getting_started/basics.html#iteritems)

> Consistent with the dict-like interface, iteritems() iterates through key-value pairs

isn't really true for Pandas on Python3 (while it was true for Pandas on Python2).

@jorisvandenbossche AFAICT there's only a handful of references to `iteritems` in the user guide (specifically four in the basics file) so I would think this is a pretty easy deprecation? Bundling it with dropping Python2 also seems to make a lot of sense from an API perspective.

If you feel strongly against this then something we can discuss at the sprint, but just wanted to offer that counterpoint of this I think not being too hard of a deprecation
@topper-123 can you merge master.

Conflicting files
doc/source/whatsnew/v0.25.0.rst
pandas/core/generic.py
pandas/core/indexes/multi.py
@jorisvandenbossche are you still -1 on this? 
@topper-123 can you merge master?

@jorisvandenbossche still objecting to this?
@TomAugspurger what do you think about this? I am +0 on this, @jorisvandenbossche ?
Maybe -0. 

I don’t have a strong preference either :)

> On Jul 3, 2019, at 07:16, Jeff Reback <notifications@github.com> wrote:
> 
> @TomAugspurger what do you think about this? I am +0 on this, @jorisvandenbossche ?
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

Sorry for the slow reply here. I don't feel stronly about it, so you can count me as a -0 as well (or -0.2 :-)) 

Basically I feel still the way as I said above (https://github.com/pandas-dev/pandas/pull/26114#issuecomment-488952625), that it feels "strange" to deprecate the method that we have always documented in favor of something that was basically not mentioned in the docs.

(BTW, apart from actually deprecation, we could also already start with changing the docs)

I've taken out the proposed deprecation.

I would like to get the rest of the PR in though, so internally we use ``items`` rather than ``iteritems``. If no one objects, I'll merge the PR in a few days (without deprecating iteritems).
Doc-only for now sounds good.

On Wed, Jul 10, 2019 at 6:39 AM Terji Petersen <notifications@github.com>
wrote:

> I've taken out the proposed deprecation.
>
> I would like to get the rest of the PR in though, so internally we use
> items rather than iteritems. If no one objects, I'll merge the PR in a
> few days (without deprecating iteritems).
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/26114?email_source=notifications&email_token=AAKAOIXFJTMFXRMXSHAJ2N3P6XJ7HA5CNFSM4HGPQJ52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZTKKCQ#issuecomment-510043402>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAKAOIW6U52OV3HEQQ2XMQTP6XJ7HANCNFSM4HGPQJ5Q>
> .
>

cool thanks, ping on green.
> 
> 
>     * do we need a note in whatsnew? (just saying this is now not recommended)?
> 
>     * can we add a code_check to prohibit this?

I can look into these two in a follow-up.
thanks @topper-123