BrandonFors/pandas-github-issues huggingface dataset:
TST: cleanup of any_* test fixtures GH38017
State: closed

- [X] closes #38017 
- [ ] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Hello @pratyushsharan! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



























































































There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-08-18 03:20:12 UTC
> * precommit is failing

Fixed.

> * can you show the number of tests on master vs number of tests here (should be the same unless there is an error somewhere)

I've checked the number of tests using `pytest pandas --collect-only` and both master and this branch have the same number of tests - 181359. The results on Azure pipelines also give the same result. 
This branch: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=64499&view=ms.vss-test-web.build-test-results-tab
Master branch: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=64511&view=ms.vss-test-web.build-test-results-tab
thanks @pratyushsharan very nice!
BUG: JoinUnit._is_valid_na_for
State: closed

Fixing this unblocks a refactor to share some code with the ArrayManager logic which I _think_ will fix some of the broken ArrayManager cases.

Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Thanks @jbrockmendel does this need a release note?
No, shouldn’t be user facing
Fix len one list dataframe constructor bug for datetime
State: closed

- [x] closes #42810
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: DataFrame
description: DataFrame data structure
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
moving to 1.3.3
Will move whatsnew later 
moved whatsnew
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.3.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 f5d1cacb384534b59461c36a55e39e2188e7ba53
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #43041: Fix len one list dataframe constructor bug for datetime'
```

4. Push to a named branch :

```
git push YOURFORK 1.3.x:auto-backport-of-pr-43041-on-1.3.x
```

5. Create a PR against branch 1.3.x, I would have named this PR:

> "Backport PR #43041 on branch 1.3.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
@meeseeksdev backport 1.3.x
BUG: interpolation with duplicate index values fails (#42585)
State: closed

- [ ] closes #42585
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Stale

Comments:
working on adding test cases here, and suspicious I may have missed a couple of details.  No rush to review until I get tests added.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the PR, but it appears it has gone stale. Closing for now, but if you could merge master and try to get the tests passing locally we can reopen.
REF: address the floating "_" in `styler.uuid`
State: closed

addresses the issue whereby the set uuid at initialisation has a floating underscore:

```python
>>> styler = Styler(df, uuid="abc")
>>> print(styler.uuid)
abc_
```

Note: This is v minor but was discovered, and is useful to fix, due to an upcoming PR.


Labels:
name: Styler
description: conditional formatting using DataFrame.style

Comments:
sure
DOC: Additional Items for the Cheat Sheet
State: closed

- https://github.com/pandas-dev/pandas/issues/40680

- Added Plotting Functions 
- Added a 3rd page consisting of various plots
- Adjusted the 2nd page and added a snippet to 'Drop a Column'

Labels:
name: Docs

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Will be reviewing soon.  Sorry for the delay
@rishitbhojak can you merge master
> @rishitbhojak can you merge master

Can you please elaborate @jreback  sir?
There is some nice stuff in the proposed changes, but in general I'm -1 on this. At least in its current version. I add the proposed changes here, to give context to my comments:

![Screenshot at 2022-04-05 00-16-28](https://user-images.githubusercontent.com/10058240/161647883-59f45b22-2681-48f0-b424-e88dee21405f.png)

The dropping a column part seems quite unreadable. Not sure if `pop` is the method we want to show in the cheatsheet for dropping a column, but regardless, we don't build the dataframe in any other function, I wouldn't do it here.

For the plots, more than a cheatsheet it looks like a gallery. Which is nice, but not sure if the cheatsheet is the right place for this. For a cheatsheet, it could be convenient to list the kind of plots, and the briefly explain what they do. More similar to the two plotting examples were there. Not just showing plot calls with the results, some times showing the data, some times without it.

Unless this can be improved significantly, present the information in a more consistent way, and showing the most relevant data (kde is probably among the least used functions, and there are lots of examples with it)., I'd rather stay with the current version.
Thanks for the pull request, but it appears to have gone stale and doesn't look likely to move forward in it's current state. If interested in continuing, please merge in main and address the review, and this can be reopened
DOC: arguments options correction
State: closed

oversight in recent method addition for 1.4.0 / 1.3.2. backport.

no release note necessary. Can also be backported to 1.3.2.


Labels:
name: Styler
description: conditional formatting using DataFrame.style

Comments:
thanks @attack68 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
Add example
State: closed

Add example to `reorder_levels` function of Pandas DataFrame

Labels:

Comments:
Hello @madhuri-15! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/frame.py`](https://github.com/pandas-dev/pandas/blob/11a0043ea7d3406e9f14b71ae2e0a2454755c70a/pandas/core/frame.py):

> [Line 6818:14](https://github.com/pandas-dev/pandas/blob/11a0043ea7d3406e9f14b71ae2e0a2454755c70a/pandas/core/frame.py#L6818): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 6828:14](https://github.com/pandas-dev/pandas/blob/11a0043ea7d3406e9f14b71ae2e0a2454755c70a/pandas/core/frame.py#L6828): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 6840:79](https://github.com/pandas-dev/pandas/blob/11a0043ea7d3406e9f14b71ae2e0a2454755c70a/pandas/core/frame.py#L6840): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 6848:75](https://github.com/pandas-dev/pandas/blob/11a0043ea7d3406e9f14b71ae2e0a2454755c70a/pandas/core/frame.py#L6848): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 6854:14](https://github.com/pandas-dev/pandas/blob/11a0043ea7d3406e9f14b71ae2e0a2454755c70a/pandas/core/frame.py#L6854): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 6868:14](https://github.com/pandas-dev/pandas/blob/11a0043ea7d3406e9f14b71ae2e0a2454755c70a/pandas/core/frame.py#L6868): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 6877:14](https://github.com/pandas-dev/pandas/blob/11a0043ea7d3406e9f14b71ae2e0a2454755c70a/pandas/core/frame.py#L6877): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 6906:84](https://github.com/pandas-dev/pandas/blob/11a0043ea7d3406e9f14b71ae2e0a2454755c70a/pandas/core/frame.py#L6906): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace



##### Comment last updated at 2021-08-15 11:29:15 UTC
`reorder_levels` seems to break a lexsort. Can it be documented how to reorder and maintain the lexsorted property, i.e. by calling a `sort` method.

As a follow-on it might be worth proposing an issue to add a `sort` keyword arg to the method so that it can sort if necessary.
This looks duplicative of https://github.com/pandas-dev/pandas/pull/42935 , which was opened first - let's close in favour of that one for the time being, we can come back to this if that one goes stale - thanks anyway for your PR, and sorry to do this
DOC: Ensure pandas warnings & exceptions are always documented in API docs
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

Also adds a pre-commit check that errors in `doc/source/reference/general_utility_functions.rst` are synced with `pandas/errors/__init__.py`

Labels:
name: Docs

Comments:
lgtm. @MarcoGorelli if any comments.
Yeah I suspect these types of checks could be done elsewhere. It might be trickier for other API sections depending on the import structure.
DEPR: Int64Index, UInt64Index & Float64Index
State: closed

* [x] xref #42717
* [x] xref #41153 & #42706

Deprecates Int64Index, UInt64Index & Float64Index.

I don't think it will be sensible to emit a deprecation warning for these because they have a lot of internal usage as is, so I've just added texts explaining that these are deprecated and will be removed in Pandas 2.0.

EDIT: it does now emit an deprecation warning when accessed from the main namespace. See changes to `pandas.__init__.py` for implementation.

```python
>>> pd.Int64Index
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.NumericIndex with the appropriate dtype instead.
```


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Index
description: Related to the Index class or subclasses

Comments:
xref #43002
IMHO, we probably should emit some sort of warning to users. I'm a little worried since 2.0 should be the next release after 1.4, and this is a somewhat big change. We can always suppress the deprecation warnings internally.
I've made a new version where accessing ``Int64Index`` etc. in the main namespace emits a FutureWarning. So these give warnings now:

```python
>>> import pandas as pd
>>> idx = pd.Int64Index([8, 9])
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.NumericIndex with the appropriate dtype instead.
>>> isinstance(idx, pd.Int64Index)
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.NumericIndex with the appropriate dtype instead.
True
```

but this does not:

```python
>>> idx = pd.Index([8, 9])
>>> idx
Int64Index([8, 9], dtype='int64')
>>> isinstance(idx, pd.NumericIndex)
True
```

This last example doesn’t give warnings, because this access pattern is not deprecated, but will only change meaning in Pandas 2.0 (will return a NumericIndex rather than a Int64Index, + Int64Index is a subclass of NumericIndex).

These warnings should  IMO give users a good hint on where they need to update their code to stay compatible in Pandas 2.0.


Got some failures still, but will work on them. The idea is completed, so commentd welcome.
Updated.
Gentle ping…
thanks @topper-123 very  nice!
@topper-123 want to enforce this deprecation and rip out some code?
Bug in Series constructor returning wrong missing values
State: closed

- [x] closes #43017
- [x] xref #43018 (if the reindex case is ok, then this is done too)
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

This ensures consistency with the DataFrame constructor

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Series
description: Series data structure
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
thanks @phofl 
@phofl we have had reports regarding both these cases. Is the new behavior now consistent?

```python
import pandas as pd

print(pd.__version__)
s = pd.Series(dtype="int", index=[0])
print(s)
s2 = pd.Series(dtype="bool", index=[0])
print(s2)
```
```
1.3.5
0    0
dtype: int64
0    False
dtype: bool
```
```
1.4.1
0   NaN
dtype: float64
0    True
dtype: bool
```

It appears to me (and maybe others from the issues) that the changes are confusing.

We have changed the `int` case and and instead create a `float` array saying that the missing value cannot be held in a integer array and yet for the bool case we continue to keep the bool dtype even though we cannot represent a missing value in a boolean array?

I appreciate that `int(np.nan)` raises `ValueError: cannot convert float NaN to integer` and `bool(np.nan)` is `True` but the users are not specifying `np.nan`, they are not supplying data. I wonder whether we ought to revert this until we change the constructors to use nullable dtypes?
>  I wonder whether we ought to revert this until we change the constructors to use nullable dtypes?

no opinion on the reversion, but i advise against the implicit assumption that constructors are going to default to nullable dtypes.  support has improved, but its still a mess.
PERF: unstack
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Avoids expensive get_indexer calls.

```
from asv_bench.benchmarks.reshape import *

self = Unstack()
self.setup("category")

%timeit self.time_full_product("category")
58.8 ms ± 1.52 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- master
31.6 ms ± 1.98 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
does this allow us to get rid of some cython code (that i wrote a long time ago) to do unstacking?
> does this allow us to get rid of some cython code (that i wrote a long time ago) to do unstacking?

no
Yes, just fewer times

On Wed, Aug 18, 2021 at 8:20 AM Jeff Reback ***@***.***>
wrote:

> ***@***.**** requested changes on this pull request.
> ------------------------------
>
> In pandas/core/reshape/reshape.py
> <https://github.com/pandas-dev/pandas/pull/43025#discussion_r691343014>:
>
> > @@ -287,7 +287,7 @@ def get_new_values(self, values, fill_value=None):
>
>          return new_values, new_mask
>
> -    def get_new_columns(self, value_columns):
> +    def get_new_columns(self, value_columns: Index | None):
>
> is this still called?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/43025#pullrequestreview-733035402>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB5UM6E6WA7525MXYRE3WV3T5PFVRANCNFSM5CEFXWSQ>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
> .
>
Replace str::replace hack for getting module name from ImportError in __init__.py
State: closed


Labels:
name: Refactor
description: Internal refactoring of code
name: Build
description: Library building on various platforms

Comments:
this looks fine, i doubt we actually test this line, can you confirm that this prints a nice message
Like this?
```
Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     import mymoduleisntloaded
... except ImportError as e:
...     module = e.name
...     raise ImportError(
...             f"C extension: {module} not built. If you want to import "
...             "pandas from the source directory, you may need to run "
...             "'python setup.py build_ext --force' to build the C extensions first."
...     ) from e
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ModuleNotFoundError: No module named 'mymoduleisntloaded'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
ImportError: C extension: mymoduleisntloaded not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --force' to build the C extensions first.
```
can you merge master just to make this this didn't affect anything.
thanks @2435191
PERF: Series.mad
State: closed

```
from asv_bench.benchmarks.stat_ops import *
self = SeriesOps()
self.setup("mad", "int")

%timeit self.time_op("mad", "int")
1.45 ms ± 22.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <- master
474 µs ± 2.77 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <- PR
```

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
sure
POC: use github issue form for usage questions
State: closed

#42951 has the context - figured the usage question form would be a good starting point as the least crucial issue template, but a common source of issues not being completely filled in.

Wasn't sure about the best way to test this out, so I just have a repo https://github.com/mzeitlin11/issue-forms which I'll try to keep up to date with the state of the PR. Anyone should feel free to open issues there and play around with it.

Labels:
name: Enhancement

Comments:
Instead of having an issue template for usage questions (which we tend to redirect to stackoverflow and close the issue anyways), maybe we can redirect users to StackOverflow or our gitter like numpy or numba do:

https://github.com/numba/numba/issues/new/choose
https://github.com/numpy/numpy/issues/new/choose
> Instead of having an issue template for usage questions (which we tend to redirect to stackoverflow and close the issue anyways), maybe we can redirect users to StackOverflow or our gitter like numpy or numba do:
>

Yep I'd be fine with that - this was more a testing ground to see what people think about the new github form in general and whether it's an improvement over the existing template.


> Yep I'd be fine with that - this was more a testing ground to see what people think about the new github form in general and whether it's an improvement over the existing template.

The new forms are definitely better IMO. I especially like the required fields and being able to add `ENH|BUG|...: <some title>` to the issues titles.
thanks @mzeitlin11 can always tweek going forwards if needed.
Testing to_datetime, converting none to NaT
State: closed

Addition of a test. Bug was fixed for #35888 somewhere in the main branch, this test just adds a unit test. 

- [x] closes #35888
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry (not sure if applicable here, can someone advise?) 


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Datetime
description: Datetime data dtype
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Stale

Comments:
Looks like there are linting errors. Please ensure `precommit` is installed and run on your commits to catch these: https://pandas.pydata.org/pandas-docs/stable/development/contributing_codebase.html#pre-commit
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
Thanks for the PR, but appears this PR has gone stale. Closing for now, but if interested in continuing please merge master and ping and we can reopen.
CI: Run python dev with numpy python 3.10 wheels
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Just testing the numpy wheels in preparation for making our own.


Labels:
name: CI
description: Continuous Integration
name: Python 3.10

Comments:
Also shaves like 4 min which is like 10% of time from the build. Verified that it runs from this commit https://github.com/numpy/numpy/commit/334428c4d10b778fd74fc3919f3dc9a3b1959875, which seems right.
nice! @lithomas1 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
DOC: Update param doc of to_parquet about compression
State: closed

Users are misled by current `compression` param doc. Actually more compressions like "ZSTD" are supported by pyarrow engine. This patch updates the param doc to reflect the supported compressions.

Labels:
name: Docs
name: IO Parquet
description: parquet, feather

Comments:
thanks @viirya 
Thanks @jreback @mroeschke
DOC: Fixed all remaining GL02 documentation errors and added appropriate check in CI
State: closed

- Closes #42809
- Adds check for GL02 in CI



Labels:
name: Docs

Comments:
there is a precommit failure
The pre-commit error is now fixed. The remaining errors stem from test_nat_doc_strings, which tests overlapping methods for matching docstrings. 

In order to fix the remaining GL02 docstring errors, I had to overwrite the docstring for two methods from the parent class. In order to fix these two errors, the docstrings will not match. How should I proceed?
right you will have to add these same methods to NaT (possibly adjusting the text)
thanks @fshi01 very nice!
TST: Fix test related to reverting fastparquet nullable support
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

xref https://github.com/pandas-dev/pandas/pull/42987#discussion_r687209605

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO Parquet
description: parquet, feather

Comments:
thanks @lithomas1 there is a doc warning but that looks unrelated
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.