BrandonFors/pandas-github-issues huggingface dataset:
CLN use default_index more
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Index
description: Related to the Index class or subclasses

Comments:
why?  no strong opinion, just curious
If `default_index` was to change (e.g. under an option), then this would align the default index across these different methods

The only one here where it changes the type is in `merge` not on index. Currently, pandas returns an `Int64Index` - would it be better to return `RangeIndex`?

upstream/main:
```
In [6]: df = pd.DataFrame({'a': [1,2,3]})

In [7]: df.merge(df, on='a').index
Out[7]: Int64Index([0, 1, 2], dtype='int64')
```

here:
```
In [1]: df = pd.DataFrame({'a': [1,2,3]})

In [2]: df.merge(df, on='a').index
Out[2]: RangeIndex(start=0, stop=3, step=1)
```
Would `default_index` help prime optional Index work as well?

The only one here where it changes the type is in merge not on index. Currently, pandas returns an Int64Index - would it be better to return RangeIndex?

+1 to generally return RangeIndex when possible
> Would default_index help prime optional Index work as well?

yup (that's how I came across this)
Could use a whatsnew (perf maybe) regarding the merge `Int64Index` -> `RangeIndex` change otherwise LGTM
A-OK
Thanks @MarcoGorelli
wheel CI: Always use latest minor version of cibuildwheel
State: closed

By removing the exact minor and patch version of the cibuildwheel action, it will always use the latest minor/patch version.

This ensures the latest Python versions are always used to build wheels.

Major version updates will be handled by #49475 for GitHub Actions.

Labels:
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
My mistake, cibuildwheel (currently) doesn't allow this and requires pinned `major.minor.patch` versions specified. So closing this PR.
CI maybe fix arm test
State: closed

- [ ] closes #49468  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration
name: ARM
description: aarch64 architecture

Comments:
Thanks for the quick fix @MarcoGorelli
Add Dependabot configuration for GitHub Actions updates
State: closed

Add a Dependabot configuration that checks once a week if the GitHub Actions are still using the latest version. If not, it opens a PR to update them.

It will actually open very few PRs, since we only have major versions specified (like v3), so only on a major v4 release it will update and open a PR.

See [Keeping your actions up to date with Dependabot](https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot).

Labels:
name: CI
description: Continuous Integration

Comments:
Thanks for the pull request, but I also think this needs some discussion in an issue first to see whether the maintainers want this and which dependency updater we want (for example there's also renovate which Ibis uses: https://github.com/marketplace/renovate)

Closing for now, but happy to have an issue to discuss
TST: avoid chained assignment in tests outside of specific tests on chaining
State: closed

Similarly as https://github.com/pandas-dev/pandas/pull/46980, now splitting off a part of the test changes in https://github.com/pandas-dev/pandas/pull/49467 that can be done separately

We have specific tests about chained indexing (in `pandas/tests/indexing`, eg `test_chaining_and_caching.py`), so outside those specific indexing tests, I can think we can avoid using chained indexing (regardless of the new CoW implementation, this would follow our own recommendation on best indexing practices, although it also shows that some cases of mixed positional/label based setting is somewhat convoluted ..)

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
There are in general two ways to avoid chained assignment if you have mixed label / positional based indexing. Considering the example of `frame["A"][:10]` (where `frame` has a non-integer index, so `:10` is positional), you can use either `loc` or `iloc` instead:

```
frame.loc[: frame.index[9], "A"] = ...
# or
frame.iloc[:10, frame.columns.get_loc("A")]
```

Do we have a preference for which to use in general? 
I started using the first, but this becomes inconvenient when slicing (label based is inclusive, so you need to decrement the slice start/stop for getting the equivalent index label; although for many tests this wouldn't actually matter for what is being tested). So later on I switched to `iloc` seeing this is a bit easier in that case. I can update the PR to more consistently use one of both methods.
Great to clean these up!

> Do we have a preference for which to use in general?

Personally I like the `iloc` version as it's one less mental step for me to convert a label to position to see what's being sliced or selected
@phofl mentioned another alternative: `frame.loc[frame.index[:10], "A"]`. This is less efficient (the slices gets converted into a list-like indexer), but for tests that doesn't matter and readability counts more. 
Thanks @jorisvandenbossche (yeah the `.loc[frame.index...]` looks good as well)
DEPR: enforce passing non boolean sort in concat
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.

introduced in #44629

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @lukemanley
STYLE Suppressing two instances of pylint W0703 broad-except. 
State: closed

- [x] partially addresses ability to re-enable pylint checks for `broad-except` from #48855 

34 instances of W0703 were found, this should address 2.    
- Suppressed one warning where there is existing logic to parse returned errors
- Suppressed another warning because downstream constructors could raise any error

Working on appropriate fixes for the rest and the eventual update to `pyproject.toml` -- will submit separate PRs as I go along.

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @roadswitcher looking forward to the follow up :)
BUG / CoW: also return new object in case of null slice for both rows and columsn (.(i)loc[:, :])
State: closed

Related to https://github.com/pandas-dev/pandas/pull/49450 (also ensuring we always return new objects from indexing operations, but for a different case). 

Currently, there is the specific corner case of `df.(i)loc[:, :]` that just returns `df` (this doesn't happen for `df.(i)loc[:]` or `df[:]`, where only a single dimension gets indexed)

Given the current inconsistency with `df.loc[:, :]` vs `df.loc[:]`, we could also consider changing this for 2.0 for the default behaviour as well, and not just for when CoW is enabled?

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

xref https://github.com/pandas-dev/pandas/issues/48998

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Copy / view semantics

Comments:
> Given the current inconsistency with df.loc[:, :] vs df.loc[:], we could also consider changing this for 2.0 for the default behaviour as well, and not just for when CoW is enabled?

Yeah it would be nice if there was a consistency in the returned object by whatever method if the entire object is sliced
So shall I change this for 2.0 also in the default mode? (so as small breaking change) @jbrockmendel thoughts?
> So shall I change this for 2.0 also in the default mode? (so as small breaking change)

I like this idea.
OK, then I updated this PR to have this new behaviour for all cases (removed the check for CoW enabled), and added a whatsnew note mentioning it as a small API change.


pandas/tests/indexing/test_loc.py::TestLocBaseIndependent::test_identity_slice_returns_new_object failing
Ah, that was the edit I made to the test when assuming this is only changed for CoW, now it is always a new object. Should be fixed now.
Fix memory leak with ujson module
State: closed

- [X] closes #49222 (Replace xxxx with the GitHub issue number)

This is mostly vendored from ujson upstream

https://github.com/ultrajson/ultrajson/blob/main/python/ujson.c

Using a static PyObject the way we did before is strongly discouraged. This is more verbose but should be more correct, although interestingly pushes us back to the legacy module initialization instead of PEP 489 multi phase initialization



Labels:
name: Performance
description: Memory or execution speed performance
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
@JelleZijlstra if you have the ability to confirm this patch fixes your issue would be appreciated
As an aside, has Pandas considered https://github.com/ijl/orjson (a Rust-based JSON (de)serializer)?
Its a pretty small memory leak. I don't think it would even register on a peakmem ASV. Let's see if it helps @jellezijlstra and can go from there
Looks good @WillAyd. Would be good to have a whatsnew note for this then LGTM 
Thanks @WillAyd 
This patch may have induced a potential regression. Please check the links below. If any ASVs are parameterized, the combinations of parameters that a regression has been detected appear as subbullets. This is a partially automated message.

 - https://asv-runner.github.io/asv-collection/pandas/#io.json.ToJSON.time_to_json
   - orient='split'; frame='df_int_float_str'
   - orient='split'; frame='df_int_floats'
   - orient='split'; frame='df_td_int_ts'
   - orient='values'; frame='df_int_float_str'
   - orient='values'; frame='df_int_floats'
   - orient='values'; frame='df_td_int_ts'
 - https://asv-runner.github.io/asv-collection/pandas/#io.json.ToJSONWide.time_to_json
   - orient='split'; frame='df_int_floats'
   - orient='split'; frame='df_td_int_ts'
   - orient='values'; frame='df_int_float_str'
   - orient='values'; frame='df_int_floats'
   - orient='values'; frame='df_td_int_ts'
STYLE: fix pylint no-else-continue warnings
State: closed

Related to https://github.com/pandas-dev/pandas/issues/48855
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @Moisan
DEPR: Enforce melt(value_name) behavior
State: closed

Introduced in https://github.com/pandas-dev/pandas/pull/35003

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
@mroeschke an observation here - I believe the error message should be that `value_name` should not exist in the `id_vars` or `var_name`. An example from the melt docs: 

```py
df = pd.DataFrame({'A': {0: 'a', 1: 'b', 2: 'c'},
                   'B': {0: 1, 1: 3, 2: 5},
                   'C': {0: 2, 1: 4, 2: 6}})

df
   A  B  C
0  a  1  2
1  b  3  4
2  c  5  6
```

This should not raise, since there is no name conflict with `id_vars` or `var_name`: 

```py
df.melt(id_vars='A', value_vars='B', value_name='C')
<ipython-input-5-6a973a750e09>:1: FutureWarning: This dataframe has a column name that matches the 'value_name' column name of the resulting Dataframe. In the future this will raise an error, please set the 'value_name' parameter of DataFrame.melt to a unique name.
  df.melt(id_vars='A', value_vars='B', value_name='C')
Out[5]: 
   A variable  C
0  a        B  1
1  b        B  3
2  c        B  5
```

This however should raise : 

```py
df.melt(id_vars='A', value_vars='B', value_name='variable')
Out[8]: 
   A variable variable
0  a        B        B
1  b        B        B
2  c        B        B
```


Thanks for the response @samukweku 

> I believe the error message should be that value_name should not exist in the id_vars or var_name

I think this is an _additional_ check that would be useful (from https://github.com/pandas-dev/pandas/issues/34731#issuecomment-643490012 generally we should be checking for conflicts with all combinations of `vars`/`values`  names & compared to existing column names). But this PR looks to be enforcing https://github.com/pandas-dev/pandas/pull/35003 which disallows `value_name` in the `df.columns`
(🎁) add python 3.11 to sdist.yml
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
thanks @KotlinIsland.
DEPR: inplace argument in set_axis
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.

introduced in #48130

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
thanks @lukemanley
DEPR: Index(ndarray[object]) inferring numeric dtype
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
DEPR: Index methods, to_time, Categorical constructor
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
DEPR: DatetimeIndex setops with mismatched tzs
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
DEPR: __setitem__ on dt64tz with mixed timezones
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
Backport PR #49377 on branch 1.5.x (BUG: Fix passing Colormap instance to plot methods with mpl >= 3.6)
State: closed

Backport of #49377 to 1.5.x

- [x] closes #49374
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Looks like https://github.com/pandas-dev/pandas/pull/48695 which added some additional mpl 3.6+ cmap logic hasn't been backported to 1.5.x (this was the cause of the merge conflict)
Can you link the original pr?
> Can you link the original pr?

In the top comment?
Yes, thx
thx @zmoon
enable pylint: invalid-sequence-index
State: closed

Issue #48855. This PR enables pylint type "R" warning: `invalid-sequence-index`.
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
The PR is closed. Changes are added to PR #49447.
enable pylint: no-self-use
State: closed

Issue #48855. This PR enables pylint type "R" warning: `no-self-use`.
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
The PR is closed. Changes are added to PR #49447.
enable pylint: literal-comparison
State: closed

Issue #48855. This PR enables pylint type "R" warning: `literal-comparison`.
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Hi @natmokval - if there's multiple codes which already pass, could you enable them all in the same PR please?
Hi @MarcoGorelli.  Sure, next time I will make a single PR with all codes which passed.
Earlier today I created three separate PRs. Might be better to update this PR and include PR #49448 and PR #49449 in it?
yeah that'd be good, thanks!
All changes are added to this PR.
API: Series([pydate, pydatetime])
State: closed

- [x] closes #49341 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype
name: Series
description: Series data structure
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors
name: datetime.date
description: stdlib datetime.date support

Comments:
Thanks @jbrockmendel
STYLE: enable pylint warnings for `cell-var-from-loop`
State: closed

- [x] addresses pylint issues with `cell-var-from-loop` as part of general pylint efforts in  #48855 

- Found one instance of the error that was an easy lift of function outside a loop
- Refactored two other locations for error based on tips from @mroeschke 

TIL `functools.partial` is a thing.



Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
> almost there 💪

The reward for a steep climb is the view from the top.