BrandonFors/pandas-github-issues huggingface dataset:
DOC: 1.0.4 whatsnew
State: closed

Brings 1.0.4.rst to master & sets the release date (for tomorrow)

cc @simonjayhawkins.

Labels:
name: Docs

Comments:
just updated 1.0.4 release notes on 1.0.4. was planning this next.
I've set the release date to today. was planning to tag (locally) now and hopefully if conda and pip tests pass (they did on a practice run) tag today. The pdf docs seem problematic though but don't want to hold things up.
Ah great. I've restored from 1.0.x, so things should match.
Was also planning to remove the notes from 1.1 whatsnew for things backported. will open pr shortly.
I think will also need to update 1.0.3 contributors on master. #33970
Thanks. Feel free to push those changes to this branch, or as a followup. Whatever is easiest for you.
> Thanks. Feel free to push those changes to this branch, or as a followup. Whatever is easiest for you.

#34430 for the removals.
Can't Dooooo.... It seem like this is already backported (commit is empty).I won't do anything. MrMeeseeks out.
BUG: Fix failing MacPython 32bit wheels for groupby rolling
State: closed

- [x] closes #34410
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

What would be the best way to validate that this fixes the MacPython/pandas-wheels @TomAugspurger 

Labels:
name: Bug
name: 32bit
description: 32-bit systems
name: Window
description: rolling, ewma, expanding

Comments:
If you really wanted to, you could build a docker image with these changes.

```Dockerfile
FROM quay.io/pypa/manylinux1_i686:latest  # verify this is the right image.

RUN git clone --depth=1 https://github.com/mroeschke/pandas/mac32_rolling \
 && cd pandas \
 && /opt/python/cp37-cp37/bin/python -m pip install numpy cython python-dateutil pytz
WORKDIR pandas

RUN /opt/python/cp38-cp38/bin/python setup.py build_ext -i
```

We really need to get this hooked up to our CI here, but it isn't 100% straightforward.
Wondering if something like this would work?

https://docs.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops
> Wondering if something like this would work?

Yeah that's 100% what we need. Just integrating it into the current layout isn't straightforward.

There's also the fact that Anaconda stopped building 32-bit linux package, making this somewhat hard to test.
Here's a working Dockerfile if it helps. This is just Py37, could extend to others simply using something like this:

https://github.com/pypa/python-manylinux-demo/blob/84dc72039a0c85c1c960e024f024fd3d37387634/travis/build-wheels.sh#L18

```sh
FROM quay.io/pypa/manylinux1_i686:latest

RUN git clone -b mac32_rolling https://github.com/mroeschke/pandas.git \
 && cd pandas \
 && /opt/python/cp37-cp37m/bin/python -m pip install numpy cython python-dateutil pytz
WORKDIR pandas

RUN /opt/python/cp37-cp37m/bin/python setup.py build_ext -i -j4
```

FWIW still fails to build but I think errors are just what @TomAugspurger is fixing in #34416 

For this change in particular (being python-level only) we probably need an extra step to actually execute the tests if we wanted to verify quickly that way
we should drop 32 bit support (for 1.1)
I think this and the other 32 bit fixup are fine to merge if it gets the wheel building moving along @TomAugspurger
BUG: taking slices in _slice_take_blocks_ax0
State: closed

- [x] closes #34367
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
thanks
Thanks Brock!
DEPR: deprecate non keyword arguments in read_excel
State: closed

Follow-up to #27573.

Allows two non-keyword arguments, ``io`` and ``sheet_name``. I think ``sheet_name`` is quite often (e.g. Interactively) supplied without being a keyword argument and requiring it will just be needlessly annoying.

Also some clean-up in pandas/tests/io/excel.

Labels:
name: IO Excel
description: read_excel, to_excel
name: Deprecate
description: Functionality to remove in pandas
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
Great PR @topper-123
TST closes #28980 Certain comparison operations misbehaving for period dtype
State: closed

I've added test to compare `pd.Series([...], dtype="period[A-DEC]")`  to dtype `str`, `bool`, `int`, `float` and `None`.

- [ x ] closes #28980
- [ 5 ] tests added / passed
- [ x ] passes `black pandas`
- [ x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Period
description: Period data type

Comments:
Looks pretty good, can you modify the description with "closes" in front of the issue number so that it'll be closed automatically when this is merged?
> can you modify the description with "closes" in front of the issue number

@OlivierLuG I meant the description of the PR (where you have the bulleted list) rather than the PR title
Hello, If you think this is correct, I can pull a new request to close #28981, which test about the same tests ?
thanks
Macpython 32 bit build fixup
State: closed

Closes #34114 

cc @jbrockmendel and @WillAyd. I have no idea if this is correct, but it builds.

Labels:
name: Build
description: Library building on various platforms
name: 32bit
description: 32-bit systems

Comments:
I looked briefly at adding these builds to our CI, but didn't make any progress. I think it comes down to including a

```
container: quay.io/pypa/manylinux1_i686:latest
```

At the same level as `job` and `pool` in the yaml file.
the tokenizer.c edit looks good and the period.pyx edit looks harmless.

But `Py_ssize_t` -> `khiter_t` looks sketchy in a subset of the places the change is made; ill comment in-line.
I'm OK with merging this as-is if this is a blocker for wheel building or 1.0.4 or something
BUG: merge between partial index and index fails when result is empty
State: closed

- [x] closes #33814
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

The Issue was caused when setting the name of an numpy array.

Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Done
@jreback green
thanks @phofl
Initialize variables in pqyear, pquarter
State: closed

This resolves the `-Wmaybe-uninitialized` warning observed when building with the manylinux1 docker image

```
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DNPY_NO_DEPRECATED_API=0 -Ipandas/_libs/tslibs -I./pandas/_libs/tslibs -I/opt/python/cp38-cp38/lib/python3.8/site-packages/numpy/core/include -I/opt/python/cp38-cp38/include/python3.8 -c pandas/_libs/tslibs/period.c -o build/temp.linux-x86_64-3.8/pandas/_libs/tslibs/period.o
pandas/_libs/tslibs/period.c: In function ‘__pyx_f_6pandas_5_libs_6tslibs_6period_pqyear’:
pandas/_libs/tslibs/period.c:12464:3: warning: ‘__pyx_v_year’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   return __pyx_r;
   ^
pandas/_libs/tslibs/period.c: In function ‘__pyx_f_6pandas_5_libs_6tslibs_6period_pquarter’:
pandas/_libs/tslibs/period.c:12512:3: warning: ‘__pyx_v_quarter’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   return __pyx_r;
```

Hopefully this doesn't break anything. IIUC, these pointers are just used to set the result, so it shouldn't matter what the value is going in? 

Closes https://github.com/pandas-dev/pandas/issues/34114


Labels:
name: Build
description: Library building on various platforms

Comments:
cc @WillAyd and @jbrockmendel for checking the C code.
Not necessary to backport, since this only affects the builds after `-Werror` was added (1.1)
Hmm strange that this is required but also pretty harmless. A zero versus a junk value getting returned is probably moot :-)

lgtm
Thanks for confirming.
DOC: add link to benchmarks page in developer docs
State: closed

- [x] closes #34405
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew  

@jreback 



Labels:
name: Docs

Comments:
@jorisvandenbossche happy with this location?
thanks @SwordKnight6216
CLN: GH29547 Replace old string formatting with f-strings
State: closed

Is it normal for all my old commits on other branches to appear here? Am I doing something wrong?

Labels:
name: Clean

Comments:
Hello @matteosantama! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:









There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-05-26 22:32:42 UTC
> Is it normal for all my old commits on other branches to appear here? Am I doing something wrong?

Are you making sure to branch off master when creating new branches (and not committing directly to master)?
DOC: added D-Tale to the visualizations section of Ecosystem page
State: closed

- [x] closes #34376

Added [D-Tale](https://github.com/man-group/dtale) to the visualizations section of [Pandas Ecosystem](https://pandas.pydata.org/docs/ecosystem.html) doc


Labels:
name: Docs
name: Visualization
description: plotting

Comments:
lgtm. @datapythonista @jorisvandenbossche 
Thanks!
DOC: add a semicolon to hide long matplotlib axes return value
State: closed

add a semicolon to hide debug messages like:
Out[85]: 
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0ad6e3d0>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0ad4a350>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0aa10250>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0a6367d0>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7f3d088c2d50>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d089ea310>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d08d89890>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0a76ae10>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0a756090>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0a8dd750>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d096aaed0>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0937d490>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7f3d09f49a10>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0a0551d0>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0a418550>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f3d0a6f0ad0>]],
      dtype=object)
which you can see here:

https://pandas.pydata.org/docs/user_guide/visualization.html

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Thanks @partev
ENH: Set index of returned dataframe in DatetimeIndex.isocalendar
State: closed

This is a followup on #33595, (specifically [here](https://github.com/pandas-dev/pandas/pull/33595#issuecomment-614888200) and [here](https://github.com/pandas-dev/pandas/pull/33595#issuecomment-614897268)), where it was noted that `DatetimeIndex.isocalendar` should set the index of the
returned dataframe.

Labels:
name: Datetime
description: Datetime data dtype

Comments:
@jreback, thanks, what's new entry updated.  Back to green.
thanks @mgmarino 
btw I think we should a mention of using isocalendar in the docs proper, maybe around here: https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#time-date-components

if you can do a PR would be great.
I am very happy to expand the docs on this, but this is already a blurb on this right after this table:

https://github.com/pandas-dev/pandas/blame/master/doc/source/user_guide/timeseries.rst#L789
ahh i c, yep forgot about that. lgtm.
CLN: _consolidate_inplace less
State: closed


Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Clean

Comments:
great. any perf impact? (i know this means running a big asv.....)
> i know this means running a big asv

im dealing with hardware issues that keep me from running this for a while
> > i know this means running a big asv
> 
> im dealing with hardware issues that keep me from running this for a while

ok happy to merge this now then
I don't think this should have been merged without any discussion or performance check? 
sure the nightly regression should show this in any event: https://pandas.pydata.org/speed/pandas/#regressions?sort=1&dir=desc

but maybe note updating recently? @TomAugspurger 
Seems to have segfaulted when compiling pandas a couple weeks back and
never recovered.

I won't have time to debug this for a little while.

On Wed, May 27, 2020 at 7:56 AM Jeff Reback <notifications@github.com>
wrote:

> sure the nightly regression should show this in any event:
> https://pandas.pydata.org/speed/pandas/#regressions?sort=1&dir=desc
>
> but maybe note updating recently? @TomAugspurger
> <https://github.com/TomAugspurger>
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/34389#issuecomment-634641337>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKAOIQOPI7RNAYRXFZFH73RTUEZHANCNFSM4NKRKJXQ>
> .
>

>  the nightly regression should show this in any event

Not necessarily, we don't have benchmarks for all functionality and use cases.

The current BlockManager is optimized to deal with consolidated blocks. At once not consolidating anymore before certain operations, has the *potential* to cause a slowdown for those operations (where the slowdown could in principle be more than the gain of not consolidating). It might very well be that none of the changes has any impact. But I can't tell that without checking. And I don't think we should do that without checking. 
So my proposal is to revert the PR for now, until someone has time to check it.




(and note that this is not in conflict with what I am saying on the mailing list. It's my opinion that we should optimize the case of "many 1D blocks" in the internals, but that doesn't mean that right now "many 2D blocks" is optimized).
its fine to revert if you want @jbrockmendel can you put up a PR. but going to continue optimizing generally anyhow. 
I already made the branch before (using github's "revert" button), so -> https://github.com/pandas-dev/pandas/pull/34407
REF: is_subperiod, is_superperiod out of libfreqs
State: closed

When we first moved these functions from tseries.frequencies to libfrequencies a couple years ago there was a question of it was really necessary, and the answer is now an unambiguous "no"

Labels:
name: Frequency
description: DateOffsets

Comments:
seems fine, yeah if its not needed in cython then better in python space.
green
use ensure_clean rather than explicit os.remove #34384
State: closed

closes #34384 
I have made a start at this issue #34384. Please let me know if I am along the right lines (beginner contributor).

I've left a space in code_checks.sh where I expect to implement the check for instances of 'os.remove' throughout the code.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
This PR seems to have include all commits from a pull-request of mine which has been merged previously. Not sure why, if anybody thinks they know please let me know.
Hello @jnecus! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-10-12 09:36:39 UTC
> This PR seems to have include all commits from a pull-request of mine which has been merged previously. Not sure why, if anybody thinks they know please let me know.

It looks like you opened your pull request from `master`. It's advisable to start each new pull request from a new branch, and that (when you start working on it) that branch be synced up with `upstream master`.

See [creating a branch](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#creating-a-branch)

With how your branch is now, if you don't want your old commits, I think you'd have to do something like (assuming `upstream` is `https://github.com/pandas-dev/pandas`)
```bash
git fetch upstream master
git rebase -i upstream/master
```
and then drop the commits from your previous pull request. Once that's done, you'd have to do a forced push.
Thanks @MarcoGorelli I will make sure to start a new branch next time before submitting a PR.
if you can merge master. i think we may have to remove the code check for this (and just push the patches) as need os.remove in some places (though maybe can check other files and just skip the pytables ones)
LGTM pending green; @jnecus can you merge master
can you merge master again; also ideally would have a check for using these in code_checks.sh (but can be a followon is ok too)
thanks @jnecus 
@jreback Happy to do a followon code check, what are you suggesting to check for? Instances of os.remove? If so, this was tried originally (the following quote).

> i think we may have to remove the code check for this (and just push the patches) as need os.remove in some places (though maybe can check other files and just skip the pytables ones)


yes exactly i think we don't allow os.remove except in the 1 place that we allow it now (so will have to filter the grep a bit)

iow we have a set of cases that we allow it (one for now)
I've opened #37095 as a placeholder for this, @jnecus please reference it if you open a PR for this (and let us know if you want/need any help)
BUG/API: to_timedelta unit-argument ignored for string input
State: closed

Supersedes #23025
- [x] closes #12136
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry
- [x] merge master


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Timedelta
description: Timedelta data type

Comments:
Hello @pdnm! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:























There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-06-01 09:03:19 UTC
Closing in favor of #34634
BUG: pd.DataFrame.transform recursively loops in some cases #34224
State: closed

- [x] closes #34224 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Error handling within try/except in aggregate function of frame.py file


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
> can you add the test which hits this

Sure! Do You mean singling out a test that already exists and hits this, or creating a new test for this case?
> Sure! Do You mean singling out a test that already exists and hits this, or creating a new test for this case?

You'll want to create a new test for a case that was raising the RecursionError before, but now raises a more appropriate error / message. You could use the example from the original issue.
Hello @pedrooa! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-06-03 02:13:44 UTC
lgtm ping on green.
thanks @pedrooa
BUG:#29928 Fix to_json output 'table' orient for single level MultiIndex.
State: closed

`dataframe.to_json()` was writing incorrect index field name, so applying read_json resulted in NaN index values. 
`dataframe.to_json()` now converts single level MultiIndex into single Index before encoding.


- [x] closes #29928 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: IO JSON
description: read_json, to_json, json_normalize
name: MultiIndex
name: Stale

Comments:
is there an associated issue?
> 
> 
> is there an associated issue?

Yes, #29928
Considering I have no news on this PR, is it suitable, or is there anything I can do to improve it ?
@LucasG0 can you fix merge conflicts and move note to 1.2? Someone will take a look again after that
@WillAyd Done, I can restore test for timedeltas if needed. 
lgtm @jreback 
@jreback maybe you have an opinion on this ? :)
Thanks I will check it.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale. if you want to continue, pls ping and can re-open.
CLN Replace old string formatting syntax with f-strings #29547: 2 files w…
State: closed

Hello, i'm new to github / opensource. Let me know if I'm doing things correctly

pandas/_libs/tslibs/timedeltas.pyx
pandas/_libs/tslibs/timestamps.pyx


Replace old string formatting syntax with f-strings #29547

- [ x ] passes `black pandas`
- [  ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Clean

Comments:
Hello, I’ve fixed the 2 issues. All test were passed successfully.
thanks @OlivierLuG
BUG: new method for join with multiindex and single index (#34292)
State: closed

- [x] closes #34292
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Tests still to be added.
I replaced the method to join a multiindex & a single index because outer joins didn't include data from the single index. The new method is similar to a join between 2 multiindexes.
I'm not sure what the keep_order parameter was, but it was never used and in other _join-methods it didn't exist either. So I suggest removing it all together.

Labels:
name: MultiIndex

Comments:
Hello @CuylenE! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-05-25 23:04:33 UTC
> please always add test first

Test added. Though I'm not sure why non-related test are failing.
Closing for now as stale. @CuylenE ping us if you'd like to pick this up and we'll reopen
TST: category isin on frame
State: closed

- [x] closes #34256
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Categorical
description: Categorical Data Type

Comments:
@vampypandya can you update with the requested changes?
thanks @vampypandya