BrandonFors/pandas-github-issues huggingface dataset:
Revert "Re-enable parallel builds in CI"
State: closed

Reverts pandas-dev/pandas#51902

Depends on #51525 which is reverted by #51951

Labels:

Comments:
@phofl ping on merging this and #51951
Didn't follow this last week. This breaks other stuff that you want to fix, that's why we are reverting?
Yes

On Tue, Mar 14, 2023 at 3:59 PM Patrick Hoefler ***@***.***>
wrote:

> Didn't follow this last week. This breaks other stuff that you want to
> fix, that's why we are reverting?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/51952#issuecomment-1468978629>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB5UM6EBY5WV6ZWSPYYWHQTW4DZ6ZANCNFSM6AAAAAAVZW7TYM>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>

ATM I can't maintain the C parts of the datetime code, which is unacceptable.
Sorry that it took me so long, I was going through the initial PR. If I understand it correctly it did not fix anything user visible and might get simpler if we can upstream stuff to NumPy? So should be good to revert, hopefully will get simpler in the future. 
This shouldn't be reverted. The whole point of this and the capsule PR is to clean up our symbol management and make an internal API for our extensions. Reverting this puts us back into a place of hacks and guesses until things work. Can we revert this revert?
Fine
Revert "Use PyCapsule for internal datetime functions"
State: closed

Reverts pandas-dev/pandas#51525

This is making #50960 a PITA.

Labels:

Comments:
Was there any discussion behind reverting this?

As far as I can tell, reverting this wasn't necessary for addressing #51961, and https://github.com/pandas-dev/pandas/pull/50960 wasn't urgent
I jumped the gun.
BUG: Series.getitem not respecting CoW with MultiIndex
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
Ah, this needs a merge of main now
ENH include the first incompatible key in error message when merging
State: closed

- [x] Closes #51861
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)..
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.

This PR intends to improve error message of `DataFrame.merge` when having incompatible keys, as mentioned in Issue #51861. In the current PR, the error message includes only the first incompatible key found for simplicity. Let me know if I should in addition show all incompatible keys or a certain number of them.

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Thanks @Charlie-XIAO
BUG: swapaxes creating result with read_only array
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
And on another note, we should also just deprecate this method IMO: https://github.com/pandas-dev/pandas/issues/51946
I stumbled upon this in #51944.

Checked all usages afterwards, so should be good when all three of them are in
```
mypy.....................................................................Failed
- hook id: mypy
- duration: 201.84s
- exit code: 1

pandas/core/generic.py:753: error: Item "ExtensionArray" of "Union[ExtensionArray, ndarray[Any, Any]]" has no attribute "swapaxes"  [union-attr]
```
BUG: CoW not tracking references when indexing midx with slice
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
I think https://github.com/pandas-dev/pandas/pull/51895 will (as a side affect) also fix this. But of course should merge at least the test anyway.
I guess we should merge this anyway since we'll backport CoW fixes?
Given that the other PR is fixing an annoying bug for geopandas, I was kind of planning to backport that as well (but yeah, the order of which gets merged first doesn't matter, fine to already merge and backport this)
Lets get this into the rc to be save
TST: Avoid datetime.now in tests for DST dependence
State: closed

Looks like this test failed over the weekend due to DST: https://github.com/pandas-dev/pandas/actions/runs/4395351454/jobs/7697080729

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Timezones
description: Timezone data dtype
name: Unreliable Test
description: Unit tests that occasionally fail

Comments:
thx @mroeschke
BUG/API: DTI/TDI/PI.map na_action="ignore"
State: closed

- [x] closes #51644
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Builds on #51934, so it would make sense to review that PR first.

Labels:
name: Datetime
description: Datetime data dtype
name: Timedelta
description: Timedelta data type
name: Period
description: Period data type

Comments:
Thanks @topper-123
REF: move ExtensionIndex.map to be part of DatetimeLikeArrayMixin.map
State: closed

Follow-up to #51809.

`ExtensionIndex.map` is only used by the datetime-like indexes (`DatetimeIndex.map` etc.) and contains no index-specific functionality, so shouldn't be an index method.

By moving this method to `DatetimeLikeArrayMixin.map` we simplify the code paths quite a bit when dealing with `DatetimeArray.map` etc. In the next PR I will make  `na_action="ignore"` work as expected and close #51644. 

Labels:
name: Refactor
description: Internal refactoring of code

Comments:
Thanks @topper-123
DOC: add examples to offsets CustomBusinessDay, Business/MonthBegin
State: closed

This pr is related to pr #50182, #49958 and #50027.
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Updated docs for `BusinessMonthBegin`, `CustomBusinessDay` and `MonthBegin`. Added more examples to highlight “the first day of the month” behavior. Gave examples that use the parameter n equals 0. Provided more examples for CustomBusinessDay parameters weekmask and calendar.

Labels:
name: Docs
name: Frequency
description: DateOffsets

Comments:
Hi @MarcoGorelli. I updated documentation for `CustomBusinessDay`, `BusinessMonthBegin` and `MonthBegin.` Corrected lists of parameters and provided more examples.
Could you, please, take a look at this PR?
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/51932/
Looks like ci failures are related to my new example for offset. 
```
D:\a\pandas\pandas\pandas\_libs\tslibs\src\datetime\np_datetime.c : fatal error C1083: Cannot open compiler generated file: 'D:\a\pandas\pandas\build\temp.win-amd64-cpython-39\Release\pandas\_libs\tslibs\src\datetime\np_datetime.obj': Permission denied
```
[link](https://github.com/pandas-dev/pandas/actions/runs/4421396269/jobs/7752145291#step:4:1267)

I use Linux and can’t catch errors raised for windows platform.

There is a Timedelta function in pandas, which shifts days as well and I can use this one.
`pd.Timestamp(2022, 8, 5, 16) + pd.offsets.CustomBusinessDay(1, offset=pd.Timedelta(days=1))`
Will it work for `windows?`

> Website preview of this PR available at: https://pandas.pydata.org/preview/51932/

Nice, thank you! 
> Looks like ci failures are related to my new example for offset.

it's unrelated, it happens before even having finished compiling pandas (which isn't affected by docstrings)
BLD: Try strip-all instead of strip-debug
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms

Comments:
Seems to save about 0.5 MB, looking at the artifacts page.

@jameslamb 
Can you try the wheel inside the zip file here https://github.com/pandas-dev/pandas/suites/11527579227/artifacts/596379601 and let me know if you still see debug symbols?
Amazing, thanks for looking into it so quickly!

I just pulled the wheel from that link you shared (`pandas-2.1.0.dev0+189.gd3f4d638df-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`) and can confirm I don't see any debug symbols (using several different approaches for finding them) 🎉
BUG: Incorrect parsing of ISO 8601 durations strings in Timedelta constructor
State: closed

- [x] Closes #51882
- [x] Closes #48122
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] ~~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Timedelta
description: Timedelta data type
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
@mroeschke I would like to reopen this pull request, I have updated it to the main branch. Issues #51882 and #48122 are still not resolved.
thanks for your PR

I do feel a bit nervous about this (as I do about the original implementation, tbh...), and think this might be a good case for a parametric test

would it be possible to add some reference implementation, such as [isoduration](https://github.com/bolsote/isoduration), and then to use [hypothesis](https://github.com/HypothesisWorks/hypothesis) to generate random tests in which you assert that the output matches that of the reference implementation?
> would it be possible to add some reference implementation, such as [isoduration](https://github.com/bolsote/isoduration), and then to use [hypothesis](https://github.com/HypothesisWorks/hypothesis) to generate random tests in which you assert that the output matches that of the reference implementation?

@MarcoGorelli I agree that is a good idea.
I am struggling with isoduration because [it creates duration strings where the number parts can be in scientific notation ](https://github.com/bolsote/isoduration/blob/ae0bd61507aa04e032402566a2c7c01a0402929c/src/isoduration/formatter/formatting.py#L34), this does not seem to be part of ISO8601

thanks - maybe there's something else we can use then? will take a look
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
CoW: change ChainedAssignmentError exception to a warning
State: closed

See https://github.com/pandas-dev/pandas/issues/51315 for context. Currently the detection gives false positives if the changed assignment comes from cython code. Therefore, changing this to a warning to still allow more broadly testing this without causing failures. 

Labels:
name: Copy / view semantics

Comments:
Yes, good catch, updated
Thanks @jorisvandenbossche
PERF: Reject non-string object arrays faster in factorize
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

A little more than 10% improvement here. Not posting asv's, since the other benchmarks are really flaky.

Idea here is that we can reject on the first non-string entry, but infer_dtype will go through the entire array, wasting comparisons.

Labels:
name: Performance
description: Memory or execution speed performance
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
Thanks @lithomas1
ENH: Adding implementation for issue #51748
State: closed

- [X] closes #51748 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

* Adding implementation for issue #51748 51748


Labels:

Comments:
Hi  @MarcoGorelli, this PR should be good to go now since I have taken care of all 5 tasks. I'm not sure if I should reach out to someone to inspect this PR.
I'm not sure we should be expanding the API here; I've put some comments in the linked issue.
@rhshadrach I saw the comments. Should I cancel the PR? 
In this instance, I would recommend closing to take it off the queue until there is some consensus for this feature. We can reopen if we decide to go forward with it. In the meantime, any feedback in the linked issue on why this is worth supporting would help in pushing the discussion forward.
@rhshadrach In the future, should I avoid tackling enhancement requests with the `Needs Triage` tag? It seems like I jumped the gun on this one
Yes - that would be my recommendation. And I would also extend that to enhancements that do not have a consensus among core developers as to whether they should be added. The pandas API is very large, and we are cautious about making potentially unnecessary additions.
Replace deprecated BadZipfile with BadZipFile
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

---

`BadZipfile` (with a small `f`) has been deprecated since Python 3.2, use `BadZipFile` (big `F`) instead, added in 3.2.

* https://docs.python.org/3/library/zipfile.html#zipfile.BadZipfile
* https://github.com/python/cpython/issues/86437

Does this need a news entry?

Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
This should be fine, since it's just in the tests.

Thanks @hugovk.
Series.astype(np.integer) doesn't show numpy warning
State: closed

- [X] closes #51523
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
Thanks @dannyi96
TYP: misc return types
State: closed

xref #47521

Most changes are from renaming `T` to `_<class>T`.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
`typing_extensions` has just been merged into main in #51480. So you can update this  to use `from _typing import Self` instead of using `typevar`.
Thanks @twoertwein
REF: Refactor CI env file creation and reduce pyarrow testing
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Curious, what is the benefit for having a script generate the dependency files at this point? Personally I slightly prefer the explicitness of just having the yamls.

I could see the benefit of a script/job that would generate conda lock files
> Curious, what is the benefit for having a script generate the dependency files at this point? Personally I slightly prefer the explicitness of just having the yamls.
> 
> I could see the benefit of a script/job that would generate conda lock files

It's hard to keep everything in sync with the action-3x yamls. 

They are all essentially the same, except for the Python version, and when a new one is added, it is easy for dependencies to disappear if they didn't work (you have to be careful that the dependencies match when reviewing the PR, which is burdensome).
conda lock seems cool, but I don't think it would be able to handle generate an environment file for different Python versions.

Might be worth it anyways, since pinning dependencies would make CI a lot more reliable.
Agreed the file are mostly the same besides python versions, but IMO the maintenance of them hasn't been too burdensome since the dependencies don't change often. We would be trading maintenance of these files with maintenance of this script.

> conda lock seems cool, but I don't think it would be able to handle generate an environment file for different Python versions.

I was working on a POC of generating conda-lock files for a platform + python version on my repo: https://github.com/mroeschke/pandas/pull/13/files. conda-lock would also be nice for the benefit of eliminating environment solving time too
The pain is mostly when supporting the newest Python, say Python 3.11.

A lot of packages like numba are slower to adopt the newest Python, and sometimes people forget to comment those dependencies out instead of deleting them. This is hard to spot when reviewing.

In general, I'd say that the script is pretty low maintenance, since the conda yaml format doesn't change.

It's also annoying, when it comes time to bump things like Cython versions (since CI won't fail, since we specify a lower bound, and CI will still pull the latest version even though the lower bound isn't correct).

I guess the main burden would be if you wanted to parametrize over the version of other packages in addition to pyarrow.
Fine with closing, didn't realize people were creating envs from this.
Update integer_na.rst with multiple improvements.
State: closed

updating and improving multiple problems in the documentation.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
I have made the necessary changes as suggested by you. Please do check.
Thanks @Aryasarkar008
Refactor C Extension Layout
State: closed

Simplifies includes for the build script and I think is easier to follow

khash doesn't pass cpplint at the moment. Need to look further if its even worth doing that or if we can get it excluded from cpplint

Labels:
name: Build
description: Library building on various platforms
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
Not immediately obvious from the diff, but this makes it so we have pandas/_libs/include for headers and pandas/_libs/src for modules. Within each, there is also a `vendored` sub-directory

This allows all of the header imports to follow the format `#include "pandas/<optional_subdir>/header.h"` . Today's world is a mixed bag for includes
@lithomas1