BrandonFors/pandas-github-issues huggingface dataset:
DOC: examples for Styler.pipe
State: closed


Addresses a recent issue raised in #45686

![Screen Shot 2022-02-08 at 19 18 03](https://user-images.githubusercontent.com/24256554/153050869-5e92947f-b548-43cc-bc4c-f3015c45dc7a.png)
![Screen Shot 2022-02-08 at 19 18 10](https://user-images.githubusercontent.com/24256554/153050880-8ac9bb91-854a-44b7-884b-76357c37c0de.png)



Labels:
name: Docs
name: Styler
description: conditional formatting using DataFrame.style

Comments:
> Some small comments otherwise LGTM.
> 
> Also off topic question, the docstring examples could (in theory) eventually generate different results from the pngs if future changes are made right?

yes they could, not sure how to get around this though given this is a visualisation device that is better shown? 
> > Some small comments otherwise LGTM.
> > Also off topic question, the docstring examples could (in theory) eventually generate different results from the pngs if future changes are made right?
> 
> yes they could, not sure how to get around this though given this is a visualisation device that is better shown?

Just curious. Agreed pngs are the better visualization method for these examples. 
nice @attack68 

should this close the original issue?
Backport PR #45453 on branch 1.4.x (REF: move np_can_hold_element to mgr method)
State: closed

Backport PR #45453

#45744 includes a partial backport of #45453. xref https://github.com/pandas-dev/pandas/pull/45744#issuecomment-1032546988

This PR is for completeness or alternatively we could remove the parts of #45453 included in #45744 added in https://github.com/pandas-dev/pandas/pull/45744/commits/0464ea8f0f5ec40cfd02394c1a848d1b9e278417 from 1.4.x

cc @jbrockmendel 

Labels:
name: Refactor
description: Internal refactoring of code
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
not yet changed milestone on https://github.com/pandas-dev/pandas/pull/45453 pending outcome of discussion here
No objections here (assuming CI failures are unrelated)
> No objections here (assuming CI failures are unrelated)

ok. let's merge this. I'll assume ci failures are unrelated for now. 1.4.1 release is scheduled for the end of the week, so will need to see a green again before then.
CI/TST: add is_ci_environment to test_close_file_handle_on_invalid_usecols skip
State: closed

follow-up to #45507 and #45771, xref https://github.com/pandas-dev/pandas/pull/45786#discussion_r798163348

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: CI
description: Continuous Integration

Comments:
Thanks @simonjayhawkins for catching this.
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
> Thanks @simonjayhawkins for catching this.

should maybe do this for `check_for_file_leaks` and `file_leak_context` in #45702?
> > Thanks @simonjayhawkins for catching this.
> 
> should maybe do this for `check_for_file_leaks` and `file_leak_context` in #45702?

Good point. Yeah those changes should be CI specific too.
TST: add test Index.where infers dtype instead of converting string to float (#32413)
State: closed

- [x] closes #32413
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This PR provides a test case for the already fixed problem #32413.

If the test name is not suitable, it would be good if you can suggest a better one.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @rendner
CI: Always run clipboard tests on GHA Linux
State: closed

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature



Labels:
name: CI
description: Continuous Integration

Comments:
backport? as its changing the CI config
> backport? as its changing the CI config

Yup, that would be a good idea.
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
ENH: port time.pxd from cython
State: closed

Until cython3 is out, we port it here.  This will allow a cython implementation of dateutil's tzlocal, which will [yak-shaving] for non-nano dt64/td64.

Labels:
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
> Actually, looks like this doesn't build on windows?

Well that doesn't look good.  @scoder area the analogous tests not run on any windows builds in the cython CI?
> are the analogous tests not run on any windows builds in the cython CI?

`libc/time.pxd` is different in 0.29.x. It still contains two struct fields that were commented out in 3.0 because they are GNU specific. Removing them from 0.29.x would be a backwards incompatible change, though.

https://github.com/cython/cython/blob/f0ccd6851e80654585fa6acb7a0626bf615857fe/Cython/Includes/libc/time.pxd
https://github.com/cython/cython/commit/1a6bba912fee840fc32b6a4d3288d57a12369820

Note that the field declarations shouldn't hurt as long as you're not trying to (auto-)convert the struct into a Python dict. So you can probably get by with adapting the ~tests~ Python exposed functions.
> pandas/tests/tslibs/test_ctime.py:10: error: Module "pandas._libs.tslibs" has no attribute "ctime"  [attr-defined]
Thanks @jbrockmendel
CI: Align GHA and Windows Azure dependencies
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Broken off from https://github.com/pandas-dev/pandas/pull/45478, Windows builds have been relatively stable in those PR 

Labels:
name: CI
description: Continuous Integration

Comments:
cool thanks @mroeschke 
@meeeseeksdev backport 1.4.x
Thanks @mroeschke

Not sure how important it is now, but we may want to revisit our pyarrow compatibility testing as pyarrow 2 (and 6) were removed here. Our pyarrow backed string array originally (and still does for now) use a fallback for unsupported operations added in pyarrow 2.0 and later. xref #42613
Ah gotcha. I'll try to get to https://github.com/pandas-dev/pandas/issues/45705 this week so we can add and `include` build for various pyarrow versions 

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-including-additional-values-in-combinations
PERF: faster constructors from ea scalars
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Perf improvement when constructing a DataFrame/Series from a scalar EA value.

Examples:

```
import pandas as pd

N = 1_000_000


%timeit pd.DataFrame({"A": pd.NA, "B": 1.0}, index=range(N), dtype=pd.Float64Dtype())
2.18 s ± 214 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)       <- main
22.4 ms ± 2.81 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)   <- PR


%timeit pd.Series(pd.NA, index=range(N), dtype=pd.Float64Dtype())
1.62 s ± 50.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)      <- main
7.33 ms ± 1.05 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)  <- PR


%timeit pd.Series(1, index=range(N), dtype='Int64')
242 ms ± 21.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)      <- main
11.6 ms ± 231 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)   <- PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
very nice @lukemanley keep em coming!
Fixing documentation format in read_csv
State: closed

add a mising ` in on_bad_lines option description in " New in version 1.3.0: " section , that was causing to make couple of lines red

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @eshirvana 
@meeseeksdev backport 1.4.x
backporting so that the stable docs are updated on the next release
PERF: use fast version of 1d-only checks
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Performance
description: Memory or execution speed performance

Comments:
> pandas/core/internals/concat.py:490: error: Unused "type: ignore" comment
pandas/core/internals/concat.py:491: error: No overload variant of "__getitem__" of "ExtensionArray" matches argument type "Tuple[int, slice]"  [call-overload]
pandas/core/internals/concat.py:491: note: Possible overload variants:
pandas/core/internals/concat.py:491: note:     def __getitem__(self, Union[int, integer[Any]]) -> Any
pandas/core/internals/concat.py:491: note:     def __getitem__(self, Union[slice, List[int], ndarray[Any, Any]]) -> Union[ExtensionArray, ndarray[Any, Any]]

related?
> related?

yah, will update
ENH: Index[complex]
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Complex
description: Complex Numbers

Comments:
Merge main one more time?
Thanks @jbrockmendel
TST: Remove unused fixtures in tests
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @mroeschke
DOC: Update import statement (GH21749)
State: closed

- [x] Addresses a task in #21749
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Docs

Comments:
Thanks @mariana-LJ 
Thanks a lot for the approval and merge @mroeschke! Looking forward to contributing more!
PERF: Merge empty frame
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Faster merge when left or right is empty. 

One ASV added:
```
       before           after         ratio
     [7651c082]       [4f451520]
     <main>           <merge-empty-frame>
-      12.0±0.3ms      1.42±0.09ms     0.12  join_merge.Merge.time_merge_dataframe_empty(False)
-        24.3±2ms      1.29±0.01ms     0.05  join_merge.Merge.time_merge_dataframe_empty(True)
```

Some additional examples:

```
N = 10_000_000

df1 = pd.DataFrame(
    np.random.randint(0, 100, (N, 2)),
    columns=['A', 'B'],
)

df2 = df1.set_index('A')

df_empty = pd.DataFrame(columns=['A', 'C'], dtype='int64')
```

```
%timeit df1.merge(df_empty, how="right")  
242 ms ± 5.32 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)        <-- main
1.12 ms ± 50.5 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)  <-- PR
```

```
%timeit df2.merge(df_empty, how="left", left_index=True, right_on='A')    
1.07 s ± 47.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  <-- main
241 ms ± 21.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  <-- PR
```



Labels:
name: Performance
description: Memory or execution speed performance
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
lgtm will merge on green.

tip to avoid conflicts in the release notes; add a note *not* at the bottom, add in middle somewhere
Greenish 
Thx @lukemanley
BUG: DataFrame[dt64].where downcasting
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
hmm this didn't seem to run all of the checks. can you rebase
can you rebase
rebased + greenish
Thanks @jbrockmendel
TST: Pass use_threads=False to pyarrow csv tests
State: closed

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Avoid hanging threads / tests from pyarrow csv.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Looks like this isn't working as intended. Closing
Test case for concat inconsistent multiIndex
State: closed

- [✔] closes #44786
- [✔ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [✔] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex

Comments:
Hello @weikhor! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-02-05 10:43:26 UTC
thanks @weikhor
PERF: groupby.transform skip fast path when only a single group exists
State: closed

- [x] closes #44977
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


groupby.transform can skip the fast path evaluation when only a single group exists. See additional comments in #44977. 

Note: one test had to be updated as it applied a groupby on a single group and expected a warning that is only generated by the fast path. The test was updated to include multiple groups so that the fast path would get evaluated. 


Labels:
name: Groupby
name: Performance
description: Memory or execution speed performance

Comments:
thanks @lukemanley
REF: avoid ravel/reshape in astype_nansafe, ndarray_to_mgr
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
> pandas/core/internals/blocks.py:1142: error: Argument 1 to "np_can_hold_element" has incompatible type "Union[dtype[Any], ExtensionDtype]"; expected "dtype[Any]"  [arg-type]

i think this is on main right?
> i think this is on main right?

correct.  #45796 includes a fix
asv failure may be real, need to look into it
TST: Create is_ci_environment helper
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
Test series replace different numpy int
State: closed

- [✔ ] closes #45311
- [✔] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [✔] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
thanks @weikhor very nice.

would take a follow up PR extending this test to use the extension dtypes as well (as not sure if we have full test coverage here in a similar manner)
BUG: Series[Interval].fillna(mismatched) coerce instead of raise
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Achieves consistency across methods in coercion behavior for IntervalDtype.

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
@jreback most recent commit fixes a mypy complaint on main
got it minor comment u can do in a follow up?
> got it minor comment u can do in a follow up?

yes
DEP: Protect some ExcelWriter attributes
State: closed

- [x] closes #45572 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Built on top of #45794

Any attributes we're looking to make protected are deprecated here, allowing users to use them until the next major release. Not deprecating any immutable attributes - users may find these useful and they do not require any extra care.

Docs for ExcelWriter are attached.
[excel_writer.pdf](https://github.com/pandas-dev/pandas/files/7991646/excel_writer.pdf)


Labels:
name: IO Excel
description: read_excel, to_excel
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @rhshadrach
BUG: Error writing an empty DataFrame to an ods file
State: closed

- [x] closes #45793 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: IO Excel
description: read_excel, to_excel

Comments:
Failure is unrelated

> pandas/core/internals/blocks.py:1142: error: Argument 1 to "np_can_hold_element" has incompatible type "Union[dtype[Any], ExtensionDtype]"; expected "dtype[Any]"  [arg-type]

cool
BUG: loc raising uncontrolled error when more indexers than levels
State: closed

- [x] closes #45762 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Stale

Comments:
@phofl status here?
Will have to look deeper. This should raise in general, but we have one or two tests which test this behavior explicitly. 
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. Feel free to update and reopen when you have the time, but closing to clear out the queue