BrandonFors/pandas-github-issues huggingface dataset:
ENH: maybe_convert_objects corner cases
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry



Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
> does this have any user facing change?

not that im aware of.

motivating goal is to eventually kill off maybe_infer_to_datetimelike (along with the PR deprecating the string inference) and just use `lib.maybe_convert_objects(values, convert_datetime=True, convert_timedelta=True)`
can you rebase.

maybe_convert_objects is getting pretty long......
> maybe_convert_objects is getting pretty long......

on the plus side its mostly flat (at least the recent additions are).  im optimistic we can get rid of some other conversion functions/special-casing and it'll be extremely worth it (xref #40489 could use your input)
TST: More old issues
State: closed

- [x] closes #18561
- [x] closes #22060
- [x] closes #22449
- [x] closes #22512
- [x] closes #22719
- [x] closes #23542
- [x] closes #26859
- [x] closes #28697
- [x] closes #28928
- [x] closes #29687
- [x] closes #30263
- [x] closes #29699
- [x] closes #31861
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @mroeschke
REF: avoid maybe_convert_platform
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

After this it is exclusively used in core.construction and core.internals.construction

Labels:
name: Refactor
description: Internal refactoring of code
name: Interval
description: Interval data type
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
i think this had a failure on the 32bit builds, just needed to change the regex being tested.  will make a follow up now
BUG: MultiIndex.reindex with non-MultiIndex; Series constructor
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

preliminary to de-duplicating some code in internals.construction

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex

Comments:
question & can you add a whatsnew (this is catching another invalid case)? rebase & ping on green.
REF: simplify _try_cast
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
lgtm ping on green.
REF: simplify Index.astype
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Index
description: Related to the Index class or subclasses

Comments:
looks fine, can you rebase
rebased + greenish
REGR: fix DataFrame sum and prod with min_count and numeric_only
State: closed

- [ ] closes #41074
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

it appears that the clean-up in #35899 to simplify _reduce may have been a bit premature, have reinstated a few lines of the removed code for backport purposes.

The longer term fix would probably be to support min_count in _mgr.reduce, https://github.com/pandas-dev/pandas/pull/40143#issuecomment-787945291

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Reduction Operations
description: sum, mean, min, max, etc.
name: Nuisance Columns
description: Identifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.apply

Comments:
this overlaps with https://github.com/pandas-dev/pandas/pull/41706 ?
ok so this is an alt to https://github.com/pandas-dev/pandas/pull/41711 ?
> this overlaps with #41706 ?

#41711? yes and no. we could just do #41711 and not backport. do this, backport this and then merge #41711. or merge #41711 and I reopen this directly against 1.2.x

those options don't include do #41711, backport it and close this one.
> ok so this is an alt to #41711 ?

no really, this intended as a backport? #41711 was opened after is the alt. 
> > this overlaps with #41706 ?
> 
> #41711? yes and no. we could just do #41711 and not backport. do this, backport this and then merge #41711. or merge #41711 and I reopen this directly against 1.2.x
> 
> those options don't include do #41711, backport it and close this one.

#41711 and I reopen this directly against 1.2.x

why is this not the preferred option?
> #41711 and I reopen this directly against 1.2.x
> 
> why is this not the preferred option?

because we normally merge to master and backport. If #41711 gets merged first. then our options reduce.
closing in favor of https://github.com/pandas-dev/pandas/pull/41711#issuecomment-851591292

The change here was effectively only 

```
        else:
            if numeric_only:
                data = _get_data()
                labels = data._get_agg_axis(axis)
                values = data.values
            result = func(values)
```

when min_count and numeric_only is given, without touching any other library code
BUG: Ignore chartsheets
State: closed

- [x] closes #41448
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Issue #41448 relates specifically to our openpyxl engine, and is resolved by this commit. However, the bug also exists in our pyxlsb engine and cannot be resolved until the upstream https://github.com/willtrnr/pyxlsb/issues/33 is addressed. I propose closing #41448 and leaving the pyxlsb case as an xfail for now. Let me know if there's a better way to track this.

Labels:
name: Bug
name: IO Excel
description: read_excel, to_excel

Comments:
@datapythonista I think #41665 tries to address the CI failure on the numpydev build
@datapythonista CI is all green now. Thanks for taking a look at this.
Hi @rhshadrach, thanks for the questions. I've attempted to address them in the docstring without getting too wordy. String and integer inputs are interpreted with respect to `ExcelFile.sheet_names`, which now excludes chart sheets, so the accepted inputs should be internally consistent and the default `sheet_name=0` won't crash if there's a chart sheet in the first position.
Thanks @ahawryluk - makes sense. One last question - what is the error raised if the sheet name (as a string) is a chart sheet? If it's sensible, can you add a test for this.
@rhshadrach Good idea. It raises the same ValueError you would get for an invalid sheet name, so I've added the tests for loading by str or by an int out of range.
Thanks @ahawryluk!
TST: More old issues
State: closed

- [x] closes #19354
- [x] closes #19517
- [x] closes #20414
- [x] closes #20420
- [x] closes #20520
- [x] closes #20734
- [x] closes #20951
- [x] closes #21651
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @mroeschke
BUG: Empty DataFrame(..., dtype='datetime64[ns]').quantile(..., axis=1) returns float64 dtype
State: closed

- [ ] xref #41544 
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Bug
name: quantile
description: quantile method
name: Stale

Comments:
@mroeschke

`quantile` passes the test mentioned in #41544 with the code in 9317cfd. 

There may be other problematic cases not covered by the test:

1. Should the dtype of the resulting series ideally be 'object' here (due to mixed dtypes in the DataFrame)?
```
In [1]: df = DataFrame(columns=["a", "b"], dtype="datetime64[ns]")
In [2]: df["b"] = df["b"].view("int64")
In [3]: df.quantile(0.5, axis=1, numeric_only=False).dtype
Out [1]: dtype('float64')
```

2. Should the dtype of the result be datetime64[ns] here?
```
In [1]: df = DataFrame(columns=["a", "b"], dtype="datetime64[ns]")
In [2]: df.quantile([0.1, 0.5], axis=1, numeric_only=False).dtypes
Out [1]: Series([], dtype: object)
```
I can address both 'problems' if so.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the PR, but I think since there are some edge cases not entirely figured out yet, I think its best that those issues are first discussed in the original issue first before proceeding here. Happy to have you contribute to that discussion or work on other issues.
Min max sparse fillna
State: closed

- [x] closes #41552
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Sparse
description: Sparse Data Type
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
I am not quite sure what this test failure is pointing towards. It is the same exact failure with my other pull request #41162
@jbrockmendel , do I need to change any configuration in order for it to pass? Cause I am not sure why is it so
> do I need to change any configuration in order for it to pass? Cause I am not sure why is it so

no, this looks like the CI being flaky for unrelated reasons.  you can ignore it
thanks @taytzehao
DOC: added explanation of each of the documentation sections
State: closed

This is meant to help contributors understand what information belongs where.

- [ ] ~~closes #xxxx~~
- [ ] tests ~~added /~~ passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] ~~whatsnew entry~~


Labels:
name: Docs
name: Stale

Comments:
@afeld do you want to merge master here?
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
> @afeld do you want to merge master here?

@afeld can you do this and respond to https://github.com/pandas-dev/pandas/pull/41689#discussion_r641739368
Thanks for the PR, but it appears that this PR has gone stale. Close due to inactivity but if you're interested in continuing let us know and we can reopen.
CI: add sdist release workflow
State: closed

- [x] closes #39417



Labels:
name: CI
description: Continuous Integration

Comments:
The version that Simon added (temporarily) in #38852 (https://github.com/pandas-dev/pandas/pull/38852/commits/2840c65e94b68ae6eab6bf154a13163a2cb87cef) used a different environment to install the sdist. I think this is important to do here as well?
@fangchenli if you have time to update would be good to get this working
See also my comment above https://github.com/pandas-dev/pandas/pull/41685#issuecomment-861438693 about Simon's version that uses a separate environment to install the sdist
Build sdist without building pandas causes a

```
ModuleNotFoundError: No module named 'pandas._libs.interval'
```

while importing pandas
probably not using the installed pandas if /pandas directory still exists from where running the tests
thanks @fangchenli 

failures are unrelated (and i think on master)
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
DOC: Add woodwork to ecosystem docs
State: closed

- [x] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Docs

Comments:
thanks @gsheni
Add test for fixed regression in concat with empty DataFrames
State: closed

- [ ✓ ] closes #39037 
- [ ✓ ] tests added / passed
- [ ✓ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them



Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Hello @weikhor! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-05-28 15:24:46 UTC
@weikhor thanks for the PR. lint issue to resolve.
Hi I have make change to code. Thank 
Hi Thank for review. Have making changes to codes.
looks fine, let's see what the CI says
thanks @weikhor
TST: Old Issues
State: closed

- [x] closes #16710
- [x] closes #16719
- [x] closes #16867
- [x] closes #17091
- [x] closes #17286
- [x] closes #18549
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @mroeschke
BUG: PeriodIndex.get_loc with mismatched freq
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Bug
name: Frequency
description: DateOffsets
name: Period
description: Period data type
name: Index
description: Related to the Index class or subclasses

Comments:
is there an issue for this, does it need a release note?
lgtm ping with a whatsnew note
`idx.get_loc(datetime.datetime(2000, 1, 2, 1, 0), "pad")`, `idx.get_loc(pd.Timestamp("2000-01-02 01:00:00"), "pad")` and `idx.get_loc('2000-01-02 00:00', "pad")` still return 1.  why do we need to disallow a mismatched Period object? 
i dont think there is a user-facing change bc `__getitem__` goes through `engine.get_loc` where this needs to be addressed separately
Index.get_loc is public? so is a user-facing change? maybe should raise `ValueError` with appropriate message so it's more obvious when things fail. Raising a KeyError for a method that is supposed to return a value even when an exact match is not found seems counterintuitive.
> Index.get_loc is public? so is a user-facing change?

get_loc is in that weird space where it isn't really intended for users, but it is called from outside `self`, so it would be a code smell to make it private

I'll add a whatsnew note to be on the safe side

> maybe should raise ValueError with appropriate message so it's more obvious when things fail. Raising a KeyError for a method that is supposed to return a value even when an exact match is not found seems counterintuitive

get_loc is pretty consistent about raising KeyError IIRC
> > maybe should raise ValueError with appropriate message so it's more obvious when things fail. Raising a KeyError for a method that is supposed to return a value even when an exact match is not found seems counterintuitive
> 
> get_loc is pretty consistent about raising KeyError IIRC

seems reasonable, my concern is that we are changing tested behavior without a deprecation and therefore it could break users code, so that break should be more obvious.

maybe add a warning or deprecate this behavior instead.
> > Index.get_loc is public? so is a user-facing change?
> 
> get_loc is in that weird space where it isn't really intended for users, but it is called from outside `self`, so it would be a code smell to make it private

nothing in the api to suggest to the user that this method should not be used https://pandas.pydata.org/pandas-docs/dev/reference/api/pandas.Index.get_loc.html
> nothing in the api to suggest to the user that this method should not be used

OK.  I added a whatsnew note.
thanks @jbrockmendel 
This seems to have caused slowdowns in several of the Period indexing benchmarks: see https://pandas.pydata.org/speed/pandas/#regressions?sort=1&dir=desc, scrolldown to "2021-05-28 17:58"
I opened https://github.com/pandas-dev/pandas/issues/42247 to keep track of this regression.
ENH: add abiility for json.dumps to parse and add timezones
State: closed

- [x] closes #12997
- [x] tests modofied
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Ok, so this is a doozy AND it's not done, but it can be merged as is.
What do we do:
We add timezones to json output!
Before:
```
ts = pd.Timestamp('2020-10-5 4:00:05', tz='US/Pacific')
print(pd.io.json.dumps(ts, iso_dates=True))
print(ts.isoformat())
```
would provide
`"2020-10-05T11:00:05.000Z"`
and
`"2020-10-05T04:00:05.000-07:00"`
respectively.

Now they both produce
``"2020-10-05T04:00:05.000-07:00"``

So what do I mean by not complete?
This still doesn't work for arrays that have datetime values. This only works for single datetime values.
So:
`pd.DataFrame({'foo': range(5)}, index=pd.date_range('2020-10-12', freq='H', periods=5, tz='US/Eastern')).to_json(date_format='iso')`

will still produce:
`'{"foo":{"2020-10-12T04:00:00.000Z":0,"2020-10-12T05:00:00.000Z":1,"2020-10-12T06:00:00.000Z":2,"2020-10-12T07:00:00.000Z":3,"2020-10-12T08:00:00.000Z":4}}'`

and not 
`'{"foo":{"2020-10-12T00:00:00.000-04:00":0,"2020-10-12T01:00:00.000-04:00":1,"2020-10-12T02:00:00.000-04:00":2,"2020-10-12T03:00:00.000-04:00":3,"2020-10-12T04:00:00.000-04:00":4}}'`

Working on it currently. Open to recommendations!

Labels:

Comments:
Hello @jeliashi! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-05-26 15:36:58 UTC
> @jeliashi can you close this one and carry on with #41573 if possible, much easier to track.

sure thing
CI: fix npdev build
State: closed

- [ ] xref #41632
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

hopefully numpy will provide an alternative that makes this unnecessary

Labels:
name: CI
description: Continuous Integration
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
@jbrockmendel you are changing the code here for unreleased compat issues. IIUC numpy may make some further changes. would it be better to add something to the 54 failing tests for now?
I'm holding out hope that numpy will handle this upstream, xref https://github.com/numpy/numpy/issues/19078
@seberg should we just suppress these warnings for the time being in expectation of a fix/change upstream?
Sorry about being away for the long weekend.  I will look into this again, but I got the feeling that you will likely need some type of helper like `create_ndarray` in any case.  Hopefully, we can avoid the warning context at least, though.
We just discussed this in the numpy call, and I think the most likely solution will be to (unfortunately) delay the deprecation, to give you more time and hopefully a better solution to this.

That would mean you don't have to worry about string promotion changes for this release.
OK.  Is there anything I/we can do short(ish)-term to make the process easier on your end? (e.g. reopen this PR)
@jorisvandenbossche not sure.  Maybe we can meet up briefly and discuss how well `np.discover_array_params` would help you (I am very flexible)?  And also what pandas needs right now in terms of dtype/descriptor discovery from NumPy?
(Plus, I guess what pandas actually does itself, since you currently need those multiple passes before and after the NumPy pass.)

I have some hopes that we could both clean up pandas and make NumPy and pandas work better together in one go with just small modifications to NumPy (and exposure of the new API).  And that would just solve this issue incidentally.
> Maybe we can meet up briefly and discuss

Sure.  I'm in UTC-7, can do any AM between now and Tuesday, inclusive.
Great! Send you an invite, in case anyone is curious, put the details here also: https://hackmd.io/fe_VHCBjSwutSwhpPP8KSg
REF: move _str_extract function in accessor.py to array method
State: closed

perf neutral refactor, another step towards #41372

after this...
can use pyarrow native functions
for expand case use _wrap_result and _str_map (without a perf impact)

Labels:
name: Refactor
description: Internal refactoring of code
name: Strings
description: String extension data type and string data

Comments:
thanks @simonjayhawkins
DEPR: Series(dt64naive, dtype=dt64tz) -> will match DatetimeIndex
State: closed

- [ ] xref #33401
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

I think this is the last deprecation needed for the constructor/astype consistency.

Labels:
name: Timezones
description: Timezone data dtype
name: Deprecate
description: Functionality to remove in pandas
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
thanks @jbrockmendel
DOC: add `Styler.to_latex` info in `io.rst` doc page
State: closed

follow-on from adding `Styler.to_latex`.

This is basic addition that primarily points to the core method docs (which have detailed descriptions and examples)


Labels:
name: Docs
name: IO LaTeX
description: to_latex
name: Styler
description: conditional formatting using DataFrame.style

Comments:
thanks @attack68