BrandonFors/pandas-github-issues huggingface dataset:
Suggested Improvement in how=left/inner parameter explanation
State: closed

- [x] Address Issue #53157
-- [ ] Pre commit checks passed 
- [x]  Contribution is made to pandas/core/frame.py
Suggestion to improve the wording for merge(), specifically the "how" parameter. I only made the changes to the left and inner arguments and will make changes to the other bullets if suggestion is worth pursuing. 

Labels:

Comments:
Currently working on failed tests. 
Thanks for the pull request, but based on the discussion in the issue, it doesn't appear that this is just a documentation issue and there might be bug. I think we should wait until the discussion is resolved there. Going to close this PR for now but happy to have your contribution on other issues labeled `good first issue`
BUG: convert_dtypes(dtype_backend="pyarrow") losing tz for tz-aware dtypes
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.2.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Arrow
description: pyarrow functionality

Comments:
Thanks @lukemanley
TST: adds test for Series and argwhere interaction
State: closed

- [x] closes #35331 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] ~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~ N/A
- [ ] ~Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~ N/A

### Details:
- Adds missing tests mentioned by @mroeschke last week. 
- Dug around briefly in other issues / prs to link the fix to this bug however nothing jumped out. 

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @mcgeestocks
BUG: Period.now not taking kwargs
State: closed

- [ ] closes #53369  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Confirmed working with both setuptools/meson.

Labels:
name: Bug
name: Build
description: Library building on various platforms
name: Period
description: Period data type

Comments:
Not sure if this build related error is due to this PR

```
   /home/runner/micromamba/envs/test/lib/python3.10/site-packages/Cython/Compiler/Main.py:370: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /home/runner/work/pandas/pandas/pandas/_libs/tslibs/offsets.pxd
    tree = Parsing.p_module(s, pxd, full_module_name)

  Error compiling Cython file:
  ------------------------------------------------------------
  ...

      def get_rule_code_suffix(self) -> str:
          prefix = self._get_suffix_prefix()
          month = MONTH_ALIASES[self.startingMonth]
          weekday = int_to_weekday[self.weekday]
          return f"{prefix}-{month}-{weekday}"
                 ^
  ------------------------------------------------------------

  /home/runner/work/pandas/pandas/pandas/_libs/tslibs/offsets.pyx:3324:15: Cannot convert Unicode string to 'str' implicitly. This is not portable and requires explicit encoding.
  [35/164] Compiling C object pandas/_libs/tslibs/offsets.cpython-310-x86_64-linux-gnu.so.p/src_datetime_np_datetime.c.o
  [36/164] Compiling C object pandas/_libs/tslibs/np_datetime.cpython-310-x86_64-linux-gnu.so.p/meson-generated_pandas__libs_tslibs_np_datetime.pyx.c.o
```
Marking as draft pending resolution of a Cython bug I think.
https://github.com/cython/cython/issues/5484
Does this change impact all of the Cython functions? From the docs makes it seems like it could be a performance hit for cdef functions we call in tight loops
Only affects single-arg functions.
(see https://github.com/cython/cython/issues/3090).

The affected functions are ``def`` functions I think as well. I don't think we really do any loops in Python so perf impact shouldn't be too horrible (the Cython functions should use the cdef variants).
Cython cut 3.0rc1 a little while ago, so this should be ready in a couple of days once the Github cache catches up.
Thanks @lithomas1
DOC: Fixing EX01 - Added examples
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Towards https://github.com/pandas-dev/pandas/issues/37875

@MarcoGorelli - we talked about this one (I was supposed to do a draft PR and tag you). 
It passes the local tests, but I'm not sure if the way I did it is fine.

Labels:
name: Docs

Comments:
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/53374/
DOC: Fixing EX01 - Added examples
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Towards https://github.com/pandas-dev/pandas/issues/37875

- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @DeaMariaLeon
PERF: Indexing with pyarrow timestamp & duration dtypes
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


pyarrow timestamps:

```
import pandas as pd

N = 100_000

idx = pd.Index(range(N), dtype="timestamp[s][pyarrow]")
idx2 = idx[::2]

%timeit idx.get_indexer_for(idx2)

# 264 ms ± 13 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)      -> main
# 1.1 ms ± 168 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  -> PR
```

pyarrow durations:

```
import pandas as pd

N = 100_000

idx = pd.Index(range(N), dtype="duration[s][pyarrow]")
idx2 = idx[::2]

%timeit idx.get_indexer_for(idx2)

# 705 ms ± 21.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)        -> main
# 1.35 ms ± 83.9 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)  -> PR
```



Labels:
name: Datetime
description: Datetime data dtype
name: Performance
description: Memory or execution speed performance
name: Arrow
description: pyarrow functionality

Comments:
Thanks @lukemanley
BUG: DataFrame bugs in ndarray assignments
State: closed

- [x] closes #51925 and #53366
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Stale

Comments:
Any updates on this?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
@mroeschke 
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: Fix Arrow CSV Parser erroring when specifying a dtype for index …
State: closed

…cols

- [ ] closes #53229  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: IO CSV
description: read_csv, to_csv
name: Arrow
description: pyarrow functionality

Comments:
Thanks @lithomas1
DEPR: infer bytes to bytes[pyarrow]
State: closed

Sits on top of #53025 and #53165.

Inferring bytes causes a few failures that need to be worked out before this is merged.

- [ ] in the string accessors there is a workaround for _str_map, used for .decode
- [ ] test_to_hdf_errors afils with "UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 0: surrogates not allowed"
- [ ] 3 of the SAS tests fail bc "expected" is constructed with `objects.astype('string[pyarrow]')` which isn't what we get with `bytes_pyarrow.str.decode(...)`
- [ ] similar failure for test_decode_errors_kwarg
- [ ] test_astype_unicode fails with UnicodeDecodeError


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Deprecate
description: Functionality to remove in pandas
name: Mothballed
description: Temporarily-closed PR the author plans to return to
name: Arrow
description: pyarrow functionality

Comments:
mothballing until after (at least) #53165
Refactor Extension Modules
State: closed

This puts all source files in `pandas/_libs/src` and all header files in `pandas/_libs/include` This simplifies our includes, letting us do them in an "absolute" way, while also making it clear which headers/sources were vendored

Labels:
name: Build
description: Library building on various platforms

Comments:
Thanks @WillAyd 
Just merged main and now im getting build errors

```
  pandas/_libs/hashtable.cpython-311-darwin.so.p/pandas/_libs/hashtable.pyx.c:754:10: fatal error: 'khash_python.h' file not found
  #include "khash_python.h"
           ^~~~~~~~~~~~~~~~
  1 error generated.
```

(my builds in general have gotten a lot more troublesome since the meson change, dont know if this is related)
Try to wipe your build folder and start from scratch with meson. I don't know that meson handles changes in the file system that gracefully compared to our old methodology
BUG: Correct .type for pyarrow.map_ and pyarrow.struct types
State: closed

- [ ] closes #53328 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Arrow
description: pyarrow functionality

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 b0d0e4e34c93e402308f6d302b7ffbd8b198815f
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #53344: BUG: Correct .type for pyarrow.map_ and pyarrow.struct types'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-53344-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #53344 on branch 2.0.x (BUG: Correct .type for pyarrow.map_ and pyarrow.struct types)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
REF: Use np.result_type instead of np.find_common_type
State: closed

- [ ] closes #53236 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions
name: Dependencies
description: Required and optional dependencies

Comments:
cc @jbrockmendel Could I get a review to get the CI back to green?
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 71cfd3a2d56ff81404e13b7e5c6a9720d52a3a81
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #53343: REF: Use np.result_type instead of np.find_common_type'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-53343-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #53343 on branch 2.0.x (REF: Use np.result_type instead of np.find_common_type)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DEPR: int slicing always positional
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

We've gone back and forth on this a couple times (#45324, #49869 would have done this as a breaking change in 2.0).

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Deprecate
description: Functionality to remove in pandas

Comments:
Do you remember why this was reverted before?
> Do you remember why this was reverted before?

There was a deprecation #45324 that was basically the opposite of this (make slice-with-integer-index label-based) but i messed up the deprecation and didn't warn in enough cases (discussed #49612), so actually enforcing it would have broken a bunch more stuff.

Also you pointed out (https://github.com/pandas-dev/pandas/issues/49612#issuecomment-1322569915) a different inconsistency, and I guess in the Dec dev call we decided to punt on this.


+1 for still moving forward with this deprecation
@phofl @marcogorelli OK here?
Looks good just needs a rebase
Thanks @jbrockmendel
DOC: fix docstring for pct_change
State: closed

- [x] closes #20752
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @gustavovargas
Update README.md
State: closed

I corrected some typo errors and added missing punctuation

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
thanks but no thanks, please see https://pandas.pydata.org/docs/dev/development/contributing.html for how to contribute
Update default value for `copy` parameter of `DataFrame.tz_convert` method
State: closed

The default value is in accordance with the method's [documentation](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.tz_convert.html).

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Yes agreed, this is supposed to be None. This defaults to True in the non CoW case. 
@jbrockmendel @phofl Is it worth updating the documentation for these methods then? As a user, this behavior of the functions is not obvious to me.

<img width="525" alt="image" src="https://github.com/pandas-dev/pandas/assets/45976948/bf899860-844e-40fb-a2eb-81d0accc584e">
Update default value for `fill_value` parameter of `Series.reindex` method
State: closed

The default value is in accordance with the method's [documentation](https://pandas.pydata.org/docs/reference/api/pandas.Series.reindex.html).

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
The fill_value is chosen to be compatible with the dtype. It makes more sense to update the docs
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
ENH: Deprecate literal json string input to read_json
State: closed

- [X] closes #52271 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
@mroeschke This looks like its going to be a pretty bulky PR after all of the unit tests are updated. 

Was hoping that someone could make sure that I'm on the right track before I finish updating the remainder of the unit tests?
Do we want to achieve the consistency as mentioned in https://github.com/pandas-dev/pandas/issues/52271#issuecomment-1490202911 in this PR? If not, maybe open a follow-up issue with the xref.
BUG: Rows are not inserted into DataFrame from lists of one element i…
State: closed

…n v2.0 #52825

- [x] closes #52825 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Hello @jbrockmendel,

Do you have any comments?
Thank you.
Will comment in #52825, it is not clear to me that the behavior here is correct
Looks like the issue needs more discussion before proceeding so closing for now. Can reopen if this is the agreed upon solution