BrandonFors/pandas-github-issues huggingface dataset:
DOC: clarify docs related to deprecation of indexing DataFrame with single partial datetime-string
State: closed

A follow-up on https://github.com/pandas-dev/pandas/pull/36179#discussion_r494830867 (cc @jbrockmendel). Attempt to clarify what is actually deprecated, remove the `:okwarning:` for several cases in the docs that were not actually deprecated, and updated to use the preferred `.loc` for some others.

Labels:
name: Docs
name: Datetime
description: Datetime data dtype
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Deprecate
description: Functionality to remove in pandas

Comments:
thanks @jorisvandenbossche 
nice
BLD: Only enable -Werror in the CI jobs
State: closed

closes #33315
closes #33314
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Build
description: Library building on various platforms

Comments:
In which `whatsnew` section should this go? Bug fixes - other?
Thanks! "Bug fixes - other" is fine for me 
(in pandas 1.0.0, we had a "build changes" section (https://pandas.pydata.org/docs/whatsnew/v1.0.0.html#build-changes), but that was a more relevant change, related to pyproject.toml)
also not averse to do this for 1.1.5 as well.
> also not averse to do this for 1.1.5 as well.

sure. however not an issue on MacPython for 1.1.5. Is causing issues with 1.2 though. see https://github.com/pandas-dev/pandas/issues/37784#issuecomment-732911912

https://github.com/MacPython/pandas-wheels/pull/112 my not be needed with this change.
> > also not averse to do this for 1.1.5 as well.
> 
> sure. however not an issue on MacPython for 1.1.5. Is causing issues with 1.2 though. see [#37784 (comment)](https://github.com/pandas-dev/pandas/issues/37784#issuecomment-732911912)
> 
> [MacPython/pandas-wheels#112](https://github.com/MacPython/pandas-wheels/pull/112) my not be needed with this change.

interesting ok, then fine not to backport.
> interesting ok, then fine not to backport.

it's not causing us issues, but i'm ok with backporting if it could be beneficial to others
> > interesting ok, then fine not to backport.
> 
> it's not causing us issues, but i'm ok with backporting if it could be beneficial to others

It would be helpful for the conda packages on macOS
all green 
this needs the note moved to 1.1.5
test failure unrelated

```
=========================== short test summary info ============================
FAILED pandas/tests/plotting/test_misc.py::TestDataFramePlots::test_scatter_matrix_axis
= 1 failed, 143660 passed, 3518 skipped, 1408 xfailed, 14 warnings in 1846.77s (0:30:46) =
```
Thanks @xhochy !
@meeseeksdev backport 1.1.x
TST : Categorical DataFrame.at overwritting row
State: closed

- [ ] closes #37763
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Categorical
description: Categorical Data Type

Comments:
thanks @ma3da
BLD: Use conda(-forge) compilers
State: closed

Use the compiler setup from conda-forge inside a conda environment. The compilers in conda-forge can be different and slightly incompatible with the system ones causing some headaches. Also this hopefully helps with initial setup of new contributors.

Is this worth a whatsnew entry?

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Build
description: Library building on various platforms

Comments:
I don't think a whatsnew is needed. But maybe we should update the contributing docs related to installing a compiler? https://pandas.pydata.org/docs/dev/development/contributing.html#installing-a-c-compiler (at mention that that manual step is not needed when installing the env using conda)

Does this also work on Windows? (I thought there you still need to manually install MS build tools in addition?)
> Does this also work on Windows? (I thought there you still need to manually install MS build tools in addition?)

You still need the build tools but I have mentioned the exact command to get the minimally necessary VS2017 components for that. This should also be sufficient for solely-`pip`-based setups.
TST: mark all plotting tests as slow
State: closed

- [ ] xref https://github.com/pandas-dev/pandas/pull/37735#issuecomment-732530791
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Marked all plotting tests as slow globally at the top of each module.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Visualization
description: plotting

Comments:
One test fails in travis.

```
    def test_area_lim(self):
        df = DataFrame(np.random.rand(6, 4), columns=["x", "y", "z", "four"])

        neg_df = -df

        for stacked in [True, False]:
            ax = _check_plot_works(df.plot.area, stacked=stacked)
            xmin, xmax = ax.get_xlim()
            ymin, ymax = ax.get_ylim()
            lines = ax.get_lines()
            assert xmin <= lines[0].get_data()[0][0]
            assert xmax >= lines[0].get_data()[0][-1]
>           assert ymin == 0
E           assert -0.11510497676407896 == 0
```

My changes presumably did not affect it.
Locally this test passes, the same matplotlib version as in travis build, 3.3.2.
I am not sure this had an effect, i think these are actually already skipped, maybe with some exceptions). However the builds are still extremely slow (30+min for tests). let me make an issue about this.
xref https://github.com/pandas-dev/pandas/issues/38091
can you merge master and let's see this run again.
thanks @ivanovmg 

can you update pandas/tests/plotting/common.py with some comments to indicate all test files are running as slow.
BUG: raise consistent exception on slicing failure
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
looks good. can you merge master as I think the recent style change may impact this. (e.g. the dict/tuple literals)
rebased+green
Do not commit any change! look at the lines 560-570, I have the timestamp resolution error because somebody repeated 'minutes' and 'seconds'!!!!!
CLN: inconsistent namespace usage in tests.indexes
State: closed

Finding these annoying while doing other work in this directory.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Code Style
description: Code style, linting, code_checks
name: Index
description: Related to the Index class or subclasses

Comments:
> can u update the precommit rule

short answer: not really.  this only gets a handful of things in a handful of files.

i pinged @MarcoGorelli on the script; there are things I think it should catch that it isn't.  i dont want to tinker with the script until im confident it is working correctly.
The original script only looks for class names - for methods, I think we could change `[A-Z]` to `\w` - I can test it out later

> i pinged @MarcoGorelli on the script

Where did you ping me? If it was in another issue then I may have missed it, sorry
> Where did you ping me? If it was in another issue then I may have missed it, sorry

Huh I'm not finding it, my bad.  I was asking why CLASS_NAMES had been removed, since ive noticed that both MultiIndex and pd.MultiIndex are present in some of the indexes test files.
It might be that they appear if they're present with starting brackets, currently we just check for cases of e.g. `MultiIndex(` and `pd.MultiIndex(` being in the same file. pandas/tests/indexes/test_common.py contains `pd.MultiIndex` and `MultiIndex`, but neither are called, they're just used in `isinstance` checks.

Maybe it's OK to remove the paren and extend the check beyond classes now
Makes sense.  For now I'll be happy with just fixing the handful in this PR so I don't keep stumbling over them in other branches.
sure - I've opened #38093 as a placeholder
> sure - I've opened #38093 as a placeholder

great yeah we ought to become strict on these
BUG: Series.where casting dt64 to int64
State: closed

- [x] closes #37682
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This sits on top of #38021

Labels:
name: Datetime
description: Datetime data dtype
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
@jreback just rebased with a fix for the unused import in groupby.py
@jreback gentle ping; in addition to being a bug this is a blocker for further simplification in blocks
merge master on this
rebased+green
rebased+green
let's rebase one more time after your _where change was merged; ping on green
ping
BUG: DataFrame.loc returning empty result with negative stepsize for MultiIndex
State: closed

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This was easier than thought. Should I add a whatsnew note for this?

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
yes pls add a whatsnew note (1.2. bugs for MI / indexing), merge master and ping on green.
@jreback green
BUG: Index.drop raising Error when Index has duplicates
State: closed

- [x] closes #38051
- [x] closes #33494
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

@jbrockmendel  This should deal with duplicates. For MultiIndex sometimes a slice with stepzize greater than zero was given, which dropped to many elements

Labels:
name: MultiIndex
name: Index
description: Related to the Index class or subclasses

Comments:
Exited for the MultiIndex cases, because would have to catch a PerformanceWarning there if it is not sorted and we are testing this at another place too.
looks good, @phofl can you merge master and ping on green.
Thanks for merging. Found that warnings call in another Test. Will look through as a follow up and clean this up
@phofl test_drop_with_duplicates_in_index failing on 32bit (not all tests completed on other envs yet)
Dtype casting issue, will look into this later.
I think this is a configuration which can not work on 32 bit for this one test, so I skipped it
@jreback green
thanks @phofl
ENH: NDArrayBackedExtensionArray.__array_function__
State: closed

motivated by getting np.delete and np.repeat working, had to implement a few others to get the tests passing.

If we go down this path, I plan to incrementally add support for others, some of which (e.g. vstack) will avoid some object-dtype casting.

cc @TomAugspurger @shoyer @seberg any suggestions to clean the implementation, particularly the fallback and the unknown-args comments?

Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
@TomAugspurger meant to cc you in the OP but made a typo
Closing to clear the queue; won't be super-useful until our min numpy version is 1.17
DOC: tidy 1.1.5 release notes
State: closed


Labels:
name: Docs

Comments:
test failure unrelated

```
=========================== short test summary info ============================
FAILED pandas/tests/plotting/frame/test_frame.py::TestDataFramePlots::test_area_lim
= 1 failed, 143863 passed, 3150 skipped, 1421 xfailed, 31 warnings in 2993.64s (0:49:53) =
```
CLN: remove ABCIndex
State: closed

It's a footgun.

BTW I expect the CI to fail until #38020 goes in, which will [demonstrate](https://github.com/pandas-dev/pandas/pull/38020#discussion_r529159757) that there is a test case that uses MultiIndex.

Labels:
name: Clean
name: Index
description: Related to the Index class or subclasses

Comments:
need a rebase.

should consider renaming ABCIndexClass -> ABCIndex
lgtm ping on green.
ping
nice. also helps for consistency when casting for mypy

> should consider renaming ABCIndexClass -> ABCIndex

+1
TST: tighten assert_index_equal calls
State: closed


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Index
description: Related to the Index class or subclasses

Comments:
> Unless you want to change `equiv=True` as the default

That is what I want, but last time I changed a default kwarg in one of these it caused a backlash, so im taking it slow.  This is basically all the changes that we would have to make to continue passing if we did change the default (along with #38034, #38028, and some other RangeIndex tinkering that hasn't been pushed yet)
> > Unless you want to change `equiv=True` as the default
> 
> That is what I want, but last time I changed a default kwarg in one of these it caused a backlash, so im taking it slow. This is basically all the changes that we would have to make to continue passing if we did change the default (along with #38034, #38028, and some other RangeIndex tinkering that hasn't been pushed yet)

ok fair, though again i still wouldn't actually specify the default and rather the exception no? we will like have to deprecate this change in any event (maybe that's just easier)? then specifying this is fine.
> i still wouldn't actually specify the default and rather the exception no?

if/when we do change the default we'll need to go through and change these all then.  in the interim this is just being explicit about which places it makes difference
> > i still wouldn't actually specify the default and rather the exception no?
> 
> if/when we do change the default we'll need to go through and change these all then. in the interim this is just being explicit about which places it makes difference

ok, problem is that new tests won't have this, i guess its not a big deal, but going to have a consistency nightmare, why are you against leaving off the `exact='equiv'` entirely?
> why are you against leaving off the exact='equiv' entirely?

not looking forward to re-doing the process of tracking down which ones need it when the default is changed.  not a big deal.  will revert.
> > why are you against leaving off the exact='equiv' entirely?
> 
> not looking forward to re-doing the process of tracking down which ones need it when the default is changed. not a big deal. will revert.

as an alternative, can you add as a comment after the call? 
just reverted them
ENH: adding support for Py3.6+ memory tracing for khash-maps
State: closed

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Tracemalloc is  a standard tool for tracing of memory consumption in Python. Until now khash-maps were invisible for this tool.

This PR follows more or less numpy's approach (see https://github.com/numpy/numpy/commit/03534ec90dba2bcdcd649be64be57939dde4c6f5) for adding  tracemalloc-support.

This PR fixes also silly mistakes from #37920 (see https://github.com/pandas-dev/pandas/pull/38048/commits/3291ed198432a80eb559382fbf9a35cb6a8ad749) and the somewhat imprecise implementation of `sizeof` for `XXXHashTable`-classes.



Labels:
name: Performance
description: Memory or execution speed performance
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
thanks @realead very nice
CLN: Remove .values from groupby.sem
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Closure of #22046 allows removing the workaround.

Labels:
name: Groupby
name: Clean

Comments:
thanks @rhshadrach
TYP: Add cast to ABC Index-like types
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

cc @simonjayhawkins 

For the added type-ignore, the attribute `inferred_freq` is added dynamically onto the index instance.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
can you merge master
@jreback merged and green.
thanks @rhshadrach
BUG: pytables in py39
State: closed

There are failures I'm getting locally since homebrew bumped up to py39.  I'm not seeing them on the CI, but they _did_ appear around the same time as these [different](https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=48423&view=logs&j=2d7fb38a-2053-50f3-a67c-09f6e91d3121&t=449937cc-3d50-56b5-5662-e489f41f1268) unexplained pytables failures started showing up.

Labels:
name: IO HDF5
description: read_hdf, HDFStore

Comments:
can you add pytables to the 3.9 build (I guess its built now?) https://github.com/pandas-dev/pandas/blob/master/ci/deps/azure-39.yaml
> can you add pytables to the 3.9 build (I guess its built now?) https://github.com/pandas-dev/pandas/blob/master/ci/deps/azure-39.yaml

it doesnt have a wheel, so i think the build would take too long
it needs to be installed via conda forge

i am sure it's there
> it needs to be installed via conda forge

sounds good, just pushed
i think that worked, great. There is an issue to add more deps to this build let me bump it.
I guess should put a release note?
> I guess should put a release note?

for 1.1.x?
> > I guess should put a release note?
> 
> for 1.1.x?

hmm yeah

while you are at it, can you add scipy and pyarrow==1.0 (though not sure they support 3.9 with that version)
scipy/pyarrow added and green
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.1.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 336cc6cba3188beb731b79ad55b1d86ddd0e25f0
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #38041: BUG: pytables in py39'
```

4. Push to a named branch :

```
git push YOURFORK 1.1.x:auto-backport-of-pr-38041-on-1.1.x
```

5. Create a PR against branch 1.1.x, I would have named this PR:

> "Backport PR #38041 on branch 1.1.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
BUG: Series constructor drops nanoseconds of Timedelta scalar
State: closed

- [x] closes #38032
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Timedelta
description: Timedelta data type
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
Well, actually, do this changes in cast.py make sense?
cc @jbrockmendel 
is #38405 doing the same thing as this?
> is #38405 doing the same thing as this?

Yes, I'll close.
Backport PR #37986 on branch 1.1.x: REGR: fix inplace operations for EAs with non-EA arg
State: closed

Backport PR #37986

Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
not sure what's going on here. but test_td_div_nan was `strict=False` in https://github.com/pandas-dev/pandas/pull/33680
ENH: preserve RangeIndex in factorize
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Performance
description: Memory or execution speed performance
name: Index
description: Related to the Index class or subclasses

Comments:
minor comment, any perf implications?
> any perf implications?

```
In [2]: ri = pd.Index(range(10**5))

In [3]: %timeit ri.factorize()
2.28 ms ± 120 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)   # <-- master
33.2 µs ± 199 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)  # <-- PR

In [5]: %timeit mi = pd.MultiIndex.from_arrays([ri, ri, ri])
18.2 ms ± 1.26 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- master
308 µs ± 5.21 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <-- PR

In [6]: mi = pd.MultiIndex.from_arrays([ri, ri, ri])

In [7]: %timeit mi.get_loc((10, 10, 10))
13.1 µs ± 65.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)  # <-- master
11 µs ± 116 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)  # <-- PR
```

> > any perf implications?
> 
> ```
> In [2]: ri = pd.Index(range(10**5))
> 
> In [3]: %timeit ri.factorize()
> 2.28 ms ± 120 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)   # <-- master
> 33.2 µs ± 199 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)  # <-- PR
> 
> In [5]: %timeit mi = pd.MultiIndex.from_arrays([ri, ri, ri])
> 18.2 ms ± 1.26 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- master
> 308 µs ± 5.21 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <-- PR
> 
> In [6]: mi = pd.MultiIndex.from_arrays([ri, ri, ri])
> 
> In [7]: %timeit mi.get_loc((10, 10, 10))
> 13.1 µs ± 65.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)  # <-- master
> 11 µs ± 116 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)  # <-- PR
> ```

nice ok, imagine these progate to other things in a + way.
rebased+green
BUG: unstack with missing levels results in incorrect index names
State: closed

- [x] closes #37510
- [x] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex

Comments:
Hello @GYHHAHA! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-12-31 07:14:20 UTC
frame hardcoded, cc @jreback 
thanks @GYHHAHA
BUG: RangeIndex.difference with mismatched step
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Index
description: Related to the Index class or subclasses

Comments:
i guess technically should have a whatsnew. otherwise lgtm. ping on green.
ping
Bug in loc raising KeyError when MultiIndex columns has only one level
State: closed

- [x] closes #29749
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

@jbrockmendel I think we should check for MultiIndex instead of level number here

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
comment on test placement, otherwise LGTM
thanks @phofl very nice
keep em coming!