BrandonFors/pandas-github-issues huggingface dataset:
GHA: Use bash -el
State: closed

Part of https://github.com/pandas-dev/pandas/pull/46493#issuecomment-1079541210

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 75e604df3d4085c1d74f4705123f27bb5785eb11
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #46541: GHA: Use bash -el'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-46541-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #46541 on branch 1.4.x (GHA: Use bash -el)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
@meeseeksdev backport 1.4.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 75e604df3d4085c1d74f4705123f27bb5785eb11
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #46541: GHA: Use bash -el'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-46541-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #46541 on branch 1.4.x (GHA: Use bash -el)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
Upgrade GitHub Actions versions
State: closed

Part of https://github.com/pandas-dev/pandas/pull/46493#issuecomment-1079541210

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
@meeseeksdev backport 1.4.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 1b3bb88008011f8ce7e3ec65fcc0c31fb838352b
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #46540: Upgrade GitHub Actions versions'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-46540-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #46540 on branch 1.4.x (Upgrade GitHub Actions versions)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 1b3bb88008011f8ce7e3ec65fcc0c31fb838352b
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #46540: Upgrade GitHub Actions versions'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-46540-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #46540 on branch 1.4.x (Upgrade GitHub Actions versions)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
CI: Refactor sdist workflow and add setup-python action
State: closed

Smaller version of https://github.com/pandas-dev/pandas/pull/46493 according to suggestion in https://github.com/pandas-dev/pandas/pull/46493#issuecomment-1079541210

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration
name: Stale

Comments:
Sorry my comment wasn't clear - I was hoping that _each_ bullet point would be a separate PR.
@mroeschke recycled this PR for sdist refactoring. The action code also covers some more cases not relevant to sdist but relevant to future PRs.
Is the main sdist refactor to add the setup python action or are there independent cleanups that can be separate from the setup action?
Good point, I've pushed a reduced version.
@mroeschke let me know if I can make this PR smaller to ease reviewing.
Thanks for all the nice work with the CI @jonashaag. I see you've got several PRs open, and looks like most of them too big to be merged. Before things start to become too messy, I would suggest if you can create an issue for the refactoring of the CI, with the tasks that you've got in mind, or the final status you're aiming to achieve. And then you can open atomic PRs (without mixing things that can be addressed independently). Also, would be nice to keep closing PRs that aren't relevant anymore (so reviewers don't need to navigate and find the ones that are ready mixed with some that don't require attention anymore).

I understand it may be a bit frustrating to work this way. But in a project as big as pandas, we need to make sure that:
- Everybody, not only you, can properly understand what you're trying to achieve (in a single PR, and in the whole refactoring)
- PRs are easy to review (and it doesn't take a huge amount of time to review, since every feedback iteration would be very time consuming)
- When things go wrong in the future because of changes, it's easy to understand what happened, there are issues explaining what was the general idea, and PRs are atomic, so they can be understood better, and reverted if needed, without having to revert unrelated changes.

Thanks!
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Is this still relevant @jonashaag or has this been superseded by your other PRs?
Superseded.
CI: xfail geopandas downstream test on Windows due to fiona install
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Similar to https://github.com/pandas-dev/pandas/pull/46296 but now happening on Windows

```
    def _check_fiona(func):
        if fiona is None:
>           raise ImportError(
                f"the {func} requires the 'fiona' package, but it is not installed or does "
                f"not import correctly.\nImporting fiona resulted in: {fiona_import_error}"
            )
E           ImportError: the 'read_file' function requires the 'fiona' package, but it is not installed or does not import correctly.
E           Importing fiona resulted in: DLL load failed while importing ogrext: The specified procedure could not be found.
```

cc @jorisvandenbossche can this downstream test be changed to not go through fiona?



Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies

Comments:
@simonjayhawkins possibliy this is needed for 1.4.x but can wait and see if the package is fixed
> can this downstream test be changed to not go through fiona?

It's a bit unfortunate that such a breakage happened twice in a really short time. Generally it doesn't happen that often, but of course will also never be guaranteed to not happen ..

For the test itself, we could indeed change it to not use fiona (eg use pyarrow instead, which is already be installed anyway in that env).

Another option could also be to simply remove the test. In geopandas, we have a build that tests against pandas main, so that should also cover this (although that doesn't test exactly the same, since our downstream test is testing with released geopandas)

Will do a PR to remove the usage of fiona.
TST: Add test with large shape to check_below_min_count 
State: closed

The function in question didn't have any tests in the first place so I thought that I will add them and simply use the problematic tuple for `shape` argument.

- [x] closes #35227
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
pls rebase, this is failing on windows
on 32-bit (some other builds as well i think)
```
> =================================== FAILURES ===================================
_________ test_check_below_min_count__positive_min_count[1-False-None] _________

mask = None, min_count = 1, expected_result = False

    @pytest.mark.parametrize(
        "mask", [None, np.array([False, False, True]), np.array([True] + 2244366 * [False])]
    )
    @pytest.mark.parametrize("min_count, expected_result", [(1, False), (2812191852, True)])
    def test_check_below_min_count__positive_min_count(mask, min_count, expected_result):
        # shape taken to show that the issue with GH35227 is fixed
        shape = (2244367, 1253)
        result = nanops.check_below_min_count(shape, mask, min_count)
>       assert result == expected_result
E       assert True == False

```
Right, didn't test for 32-bit, my bad. I will work on it later this week. BTW, I am still new to the pandas testing process, is there any way to check 32-bit builds apart from configuring continuous integration tests as specified in https://pandas.pydata.org/docs/development/contributing_codebase.html#testing-with-continuous-integration?
All should be good now - it took me a while to figure out that `np.prod` will evaluate the dtype to 32 bits on all Windows machines (even 64 bit ones). In the latest version we have three - 2 running on all platforms and 1 running on 64 bit non-windows ones (it is the one with large shape that was initially causing the issue)
> All should be good now - it took me a while to figure out that `np.prod` will evaluate the dtype to 32 bits on all Windows machines (even 64 bit ones). In the latest version we have three - 2 running on all platforms and 1 running on 64 bit non-windows ones (it is the one with large shape that was initially causing the issue)

hmm yeah you can specify `np.prod(..., dtype='float64')` but directly yeah on windows this would easily overflow.

thanks
thanks @FactorizeD
change default value (for index) of func named to_csv
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

# Objective
We find the extra index output annoying, and to remedy that.

# Changes
```python
...
def to_csv(
        self,
        path_or_buf: FilePath | WriteBuffer[bytes] | WriteBuffer[str] | None = None,
        sep: str = ",",
        na_rep: str = "",
        float_format: str | None = None,
        columns: Sequence[Hashable] | None = None,
        header: bool_t | list[str] = True,
#       index: bool_t = True,
        index: bool_t = False,  # My Suggestion
        index_label: IndexLabel | None = None,
        mode: str = "w",
        encoding: str | None = None,
        compression: CompressionOptions = "infer",
        quoting: int | None = None,
        quotechar: str = '"',
        lineterminator: str | None = None,
        chunksize: int | None = None,
        date_format: str | None = None,
        doublequote: bool_t = True,
        escapechar: str | None = None,
        decimal: str = ".",
        errors: str = "strict",
        storage_options: StorageOptions = None,
    ) -> str | None:
...
```

Labels:

Comments:
I have checked and it doesn't seem to be there.

Should an Issue be created?
this default cannot be changed w/ discussion (but low likelihood in any event); you are welcome to create an issue though
Thanks for the PR, but agreed for deprecating behavior we prefer discussion in an issue first (I remember something similar being discussed a while back and the conclusion was not to change the output index default because it had been the output for years now and easily changed by toggling the keyword). Closing awaiting a format discussion in an issue
Revert "TYP: Many typing constructs are invariant"
State: closed

I accidentally pushed to pandas instead of my fork :(

Is it possible in github to disallow direct pushes to the repository (only allow PRs)?

edit: It seems that this is possible https://stackoverflow.com/a/57685576

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
@jreback @simonjayhawkins 
go ahead and merge when ready

i think can prevent merging directly to master
I protected the main branch from direct pushes.
Some windows tests fail with:

> ImportError: the 'read_file' function requires the 'fiona' package, but it is not installed or does not import correctly.
> Importing fiona resulted in: DLL load failed while importing ogrext: The specified procedure could not be found.

Seems unrelated, but I am not going to merge this myself - messed up enough for today :(
yep yep totally fine @twoertwein 
> I protected the main branch from direct pushes.

@jreback the release scripts do a direct push. for main this is only an issue for rc. Is 1.4.x protected?
nope other branches are not protected
STYLE: fix PDF026 issues
State: closed

- [x] closes #46528
- [x] closes #42359
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
A few checks a failing, you probably need to add the other new error codes to setup.cfg, and they can gradually be removed
> /home/runner/work/pandas/pandas/doc/source/whatsnew/v0.13.1.rst:33: WARNING: undefined label: tutorials
> > /home/runner/work/pandas/pandas/doc/source/whatsnew/v0.13.1.rst:33: WARNING: undefined label: tutorials

looks like this was unrelated - merging then, thanks @Moisan
REF: re-use tz_convert_from_utc_single in _localize_tso
State: closed

Largely sits on top of #46516 

The observation is that what _localize_tso is doing is similar to what tz_convert_from_utc_single is doing if we could just get a couple more pieces of information back from the latter call.   Getting `fold` back is easy to do by passing via pointer (which we do elsewhere).  Getting `new_tz` back is uglier but its the best we got.  (im assuming that returning a tuple`(int64_t, bint, Py_ssize_t)` would incur a perf penalty but ive bothered the cython folks enough already recently)

May be able to get some extra de-duplication in _create_tsobject_tz_using_offset and/or ints_to_pydatetime.  But I'm skittish perf-wise.

#46516 should definitely be merged.  This I won't be offended if reviewers ask for %timeits out the wazoo.

At some point, someone with just the right amount of adderall drip should figure out the optimal way to check for `typ=="dateutil" etc.

Labels:
name: Timezones
description: Timezone data dtype

Comments:
rebased?
> rebased?

has been rebased following #46516, im not aware of anything relevant since then
https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=75951&view=logs&j=915dbb23-6529-54ae-059d-ec0b59ba10e8&t=86b091c2-17fe-52fe-2454-5f56a371e155

32-bit failures :-<
looks like those were xpasses, should now be unfixed
green
Add a new material in Community tutorials page 
State: closed

Currently, [Joyful pandas](https://github.com/datawhalechina/joyful-pandas) is one of the popular tutorials for pandas on GitHub, which ranks after guipsamora/pandas_exercises and Julia Evans' pandas cookbook by stars. This tutorial is based on pandas with
a newer version and includes most of the essential parts in pandas. The webpage for the tutorial follows pandas's sphinx template. I believe this can be helpful for the community. Thanks.

Labels:
name: Docs

Comments:
Thanks @GYHHAHA 
Thanks for merging! Since Joyful pandas will be published in a paper version with 3 extra chapters (about visualization, feature engeering and performance optimization) in the next month, and I'm looking for potential recommenders currently. If @rhshadrach @mroeschke or anyone from pandas members is willing to make the recommendation, feel free to let me know and contact me through geng29@wisc.edu. Finally, I sincerely appreciate all efforts from pandas core teams and look forward for greater pandas package and better community.
ENH: consistency of input args for boundaries - Interval
State: closed

- [ ] xref #40245
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Interval
description: Interval data type

Comments:
Hello @weikhor! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





















































































































There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-05-30 12:21:22 UTC
Hi There is still issues inside code PR submit. Will tag reviewers once I finish. Thank 
Hi @jreback   This PR is for ENH: consistency of input args for boundaries - 'inclusive' in {"both", "neither", "left", "right"} for six classes 
  - Interval
  - IntervalIndex
  - IntervalDtype
  - IntervalArray
  - IntervalTree
  - ArrowIntervalType

I not able to submit one PR for each class because the six classes are related.  
 Thank
can you rebase this (as the pickle fix was merged)
can you rebase once more
the doc build is failing, which is working on master; its possible its superfluous, so if it builds locally can you rebase again.
@jreback The pipeline in greeen
TYP: Fix core/groupby/generic.py type ignores
State: closed

xref #37715

Two things to mention:

- not a huge fan of casts but couldn't find another way to fix the issues;
- added `Index()` in line 277 because `columns` attribute of a dataframe is supposed to be of Index type.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
After reviewing all the comments it seems that the only change that can be applied at the moment is `cast` from line 648. Given that in general it is better to omit such, would you prefer me to still go with this one change, or should I close this PR and come back to it after e.g. stubs are merged?
Thanks @FactorizeD can you get this PR into a mergeable state (and not marked as draft) even if only the one change.
can you merge master and fix the precommit
TYP: Index.join
State: closed

Overloads for Index.join

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
thanks @twoertwein
REF: _infer_tsobject_fold to infer_datetuil_fold
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Trying to de-duplicate a bunch of the timezone-handling code, splitting this off as theres no clear way it could affect perf

Labels:
name: Refactor
description: Internal refactoring of code
name: Datetime
description: Datetime data dtype

Comments:
Thanks @jbrockmendel