BrandonFors/pandas-github-issues huggingface dataset:
TST: Add test to verify DataFrame's constructor doesn't convert Nones to strings on string columns
State: closed

- [x] closes #32218
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

### whatsnew
Adds a very simple test to verify that `None`s are not casted to strings as `"None"` inside the DataFrame's constructor for string columns.

Questions:
* Should the test be more general?
* What is the proper location for the test?


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@cgarciae can you merge master to resolve conflicts and can take a look
closing as stale
Add type to arg of series/drop-duplicates
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

I noticed in https://github.com/pandas-dev/pandas/pull/40867 that keep didn't specify a type, so adding it here.


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
@MarcoGorelli if you can have a look, I don't think I can assign the PR :)
> Nice, thanks @olgarithms !
> 
> Can we be more precise? How about
> 
> ```python
> keep: Literal['first', 'last', False]
> ```
> 
> where `Literal` will have to be imported as
> 
> ```python
> if TYPE_CHECKING:
>     from typing import Literal
> ```
> 
> (because it's not available in py37)

you can define this in pandas._typing as we use keep in a number of places
@olgarithms can you address comments 
Hi, yes! I am going to work on it this weekend!
@olgarithms closing to clear queue. ping if you would like to continue working in this, and i'll reopen.
REF: re-use maybe_cast_result in Series.combine
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
thanks, this is a nice cleanup
BUG: to_string truncation column with index=False
State: closed

- [x] closes #40904
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
Hello @mikeronayne! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-04-16 00:29:08 UTC
thanks @mikeronayne very nice!
DOC: Fixed documentation for few files
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them


Labels:
name: Docs

Comments:
dont' we have a master issue for this? (e.g. getting all doc-tests running)? cc @MarcoGorelli 
> dont' we have a master issue for this? (e.g. getting all doc-tests running)? cc @MarcoGorelli

I wasn't aware that not all doctests were running - anyway, there's #22459 still open
small comment, otherwise lgtm
lgtm @phofl over to you
thanks @ShaharNaveh
CLN: refactor `Styler._translate` into composite translate functions
State: closed

This composites `Styler._translate()` into:

```
def _translate():
    ...
    self._translate_header()
    ...
    self._translate_body()
    ...
```

where the `_translate_header()` and `_translate_body()` methods are given some documentation explaining what they are building. The code is not fundamentally changed, but variable names are renamed to add clarity to match the documentation and loop comprehensions replace `for loops` where possible.

Some tests are minimally altered since the generic `_element` method now returns more dict keys for some elements.

Labels:
name: Refactor
description: Internal refactoring of code
name: Styler
description: conditional formatting using DataFrame.style

Comments:
is there a `_translate_footer()` (maybe currently unused)?
> is there a `_translate_footer()` (maybe currently unused)?

No because the html element `<tfoot>` is not used in the jinja2 templates. 

It might be useful for *easily* adding styling of a `totals` row for example in the future, albeit can be done currently, its just not that *easy*.
thanks @attack68
Raising ValueError when xlim and ylim contain non-int and non-float dtype elements
State: closed

- [x] closes #40781 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Visualization
description: plotting
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
> pushed new changes addressing previous comments

Can you please add tests which hit the added code?
> > pushed new changes addressing previous comments
> 
> Can you please add tests which hit the added code?

I am not sure how to do that, I tested using an arbitrary dataset I created. Could you link me a documentation to pandas standard?
> > > pushed new changes addressing previous comments
> > 
> > 
> > Can you please add tests which hit the added code?
> 
> I am not sure how to do that, I tested using an arbitrary dataset I created. Could you link me a documentation to pandas standard?

This guide is extremely useful, please feel free to ask ask any questions that come up when reading it! https://pandas.pydata.org/docs/development/contributing.html

Also the precommit section will be useful, since that is currently failing on this pr.
Outside of the guide, looking at some tests in `pandas/tests/plotting` can give an idea of testing style
> Outside of the guide, looking at some tests in `pandas/tests/plotting` can give an idea of testing style

I did my best effort on writing the tests for individual cases. But it seems to have failed on CI, I would love a recommendation on this part too. 
@regmibijay can you merge master to resolve conflicts
@regmibijay if you want to merge master can revive this
I will revive and work on this thread again, which release should I target it for 1.3.x or 1.4.x?
1.4.0. 1.3 has already been released. 
if you can merge master and update to comments
moved to #43932 because of merge complications
TST: add test for Index.where when needing to cast to object dtype
State: closed

- [x] closes #32413
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry: Adds a basic test for #32413.

Hey! This is my first PR with actual code: added a very basic test to validate #32413. 

Questions:
* What is the best place for this test? (currently put it in `pandas/tests/indexes/test_indexing.py`)
* Should I try to create another test that uses the `index` input argument? 

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @jorisvandenbossche and @mroeschke for the feedback! I'll clean the PR a bit :)

@jorisvandenbossche moved the test there!
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
the underlying problem in #32413 was at a lower level IIRC.  should we have a test on astype or a constructor?
@cgarciae can you address @jreback comments.
closing as stale
Backport PR #40877 on branch 1.2.x (Skipt failing tests for numpy dev)
State: closed

Backport PR #40877: Skipt failing tests for numpy dev

Labels:

Comments:
@simonjayhawkins #40882, this one won't work because of implicit conflicts
> @simonjayhawkins #40882, this one won't work because of implicit conflicts

should be able to checkout, fix and push back to this branch.
Can't we use #40882? Backported there which is finished
didn't see you had already done it manually. Thanks @phofl
Backport PR #40877 on branch 1.2.x (Skipt failing tests for numpy dev)
State: closed

Backport PR #40877

Had to add the is_numpy_dev import, we could otherwise import from other directory if this would be preferrable

Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies
name: Upstream issue
description: Issue related to pandas dependency

Comments:
@phofl have added Backport PR #40877 to OP. the ref in the title donsn't create a link to the original PR.
> Had to add the is_numpy_dev import, we could otherwise import from other directory if this would be preferrable

maybe since we likely are doing a release between now and the revert, let's not change the code only the tests.
Seems reasonable,

Done
REGR: ufunc with DataFrame input not passing all kwargs
State: closed

- [x] closes #40662
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Credit to @attack68 for the fix, just figured we should try to get this is in for 1.2.4 if possible since we know the fix.

Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
lgtm Ci failure is because of numpy dev issue
Thanks @mzeitlin11 !
@meeseeksdev backport 1.2.x
Skipt failing tests for numpy dev
State: closed

- [x] xref #40874
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them


Skipping for now.


Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies
name: Upstream issue
description: Issue related to pandas dependency

Comments:
@simonjayhawkins if an issue with 1.2.4 release we can backport here
I've milestoned 1.2.4 but may need manual backport (the number of tests failing is different)
The test not failing on 1.2.4 is skipped there, will probably cause a merge conflict anyway
@meeseeksdev backport 1.2.x
Something went wrong ... Please have  a look at my logs.
Merged, looks like numpy identified the source already, can probably revert next sunday
> Merged, looks like numpy identified the source already, can probably revert next sunday

ok will backport to keep ci green on 1.2.x in the meantime.
@meeseeksdev backport 1.2.x
Í have added #40882 manually
Add test to check check_like with different indexes
State: closed

- [x] closes #39739
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Failure because of #40874
Two missing spaces in argmax, argmin docs
State: closed

I fixed the missing space that made the documentation render "isthe" instead of "is the".

Before the change:
- This **isthe** equivalent of the numpy.ndarray method argmax.
- This **isthe** equivalent of the numpy.ndarray method argmin.

After the change:
- This **is the** equivalent of the numpy.ndarray method argmax.
- This **is the** equivalent of the numpy.ndarray method argmin.


- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Labels:
name: Docs

Comments:
merge on green
BUG: DataFrame.from_records doesn't raise proper error message when Series passed
State: closed

- [x] closes #40429 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors
name: Stale

Comments:
Since this was tested, should we deprecate these cases first?
The docs include `sequence of tuples or dicts` as an input option, so continuing to support that seems fine (or at least should be deprecated). So those tests probably shouldn't be deleted.
Passing series will give an error and according to docs Series are not supported  (#40429 ).
With the PR, pandas will continue to support `sequence of tuples or dicts` and it will give more appropriate error message when Series passed.
By the way, any suggestion to fix failing test(I think related to mypy.)?
Thanks for the contribution @hasan-yaman 

I think it makes more sense if `DataFrame.from_records(data=Series([1, 2, 3]))` just returns `data.to_frame()`. The code here looks cumbersome, and likely to cause problems in the future. @jreback what do you think?

One question @hasan-yaman, why are you removing tests? I guess I'm missing something, but existing tests should be kept, and a new one passing a `Series` should be added. Removing or editing tests while changing existing code is dangerous.
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@hasan-yaman can you address @datapythonista comments
Thanks for the PR, but it appears to have gone stale and needs follow up from the other reviewers. Closing, but happy to open if you're interested on continuing.
Deprecate level keyword for dataframe and series aggregations
State: closed

- [x] closes #39983
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

This would deprecate the level keyword if we want to move forward with this

Labels:
name: Groupby
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Deprecate
description: Functionality to remove in pandas

Comments:
> wouldn't be averse to change some of the tests to use the groupby instead of the reduction method to remove / avoid the warnings (some, not all)

Most of the tests are pretty basic and already covered in groupby, the more complicated ones are mostly checking if the aggregation with level equals a groupby, so don't know if this makes sense here?
> > wouldn't be averse to change some of the tests to use the groupby instead of the reduction method to remove / avoid the warnings (some, not all)
> 
> Most of the tests are pretty basic and already covered in groupby, the more complicated ones are mostly checking if the aggregation with level equals a groupby, so don't know if this makes sense here?

kk sounds good
thanks @phofl
ARM64 parallel builds
State: closed

- [x] closes #36719 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Seems to have gotten build from > 1 hr to ~ 38 min. 

Labels:
name: CI
description: Continuous Integration
name: ARM
description: aarch64 architecture

Comments:
thanks @lithomas1 yes this looks great!


Hi!
Actually this PR changed from ARM64 to AMD64. See https://travis-ci.com/github/pandas-dev/pandas and expand `Build system information` . There you will see `OS/Arch:           linux/amd64`. The label "arm64-graviton2" is a known bug but Travis people don't want to fix it.

https://blog.travis-ci.com/2020-09-11-arm-on-aws, section "Quick tips" says: `You have to explicitly set virt: and group: keys to invoke the AWS Graviton2 CPU. Otherwise, your builds will be routed to the Travis CI default amd64 environment.`
wow that's a crazy bug

cc @lithomas1 
A fix is proposed with https://github.com/pandas-dev/pandas/pull/40959
TST Add test for iloc on interval index
State: closed

- [ ] closes #17130
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

Adds tests for using iloc on an interval as index.


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Interval
description: Interval data type

Comments:
thanks @EricLeer
BUG: Fix dtypes change using replace with nullable dtypes (#40732)
State: closed

- [x] closes #40732 
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry



Labels:
name: Bug
name: replace
description: replace method
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays
name: Stale

Comments:
Hello @hasan-yaman! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-04-12 18:30:53 UTC
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@hasan-yaman can you add a release note
Thanks @hasan-yaman for the PR. closing as stale. ping if you want to continue.
Fix bug in index.union with duplicates and not a subset of each other
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

The previous fix missed one case where other has dups which are unique in self


Labels:
name: Bug
name: Index
description: Related to the Index class or subclasses

Comments:
@jreback greenish, failure unrelated
very nice @phofl
Fix fillna accepting downcast as dict
State: closed

- [x] closes #40809
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

This fix makes sense in this case I think, but this does not fix the case for non-list like values, which raises a NotImplementedError

The docstring for Series.fillna says, that downcast may be a dict too, but I don't think that this makes sense, so we should maybe raise an error there?

Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
> This fix makes sense in this case I think, but this does not fix the case for non-list like values, which raises a NotImplementedError

by coincidence i've been looking at Block.downcast and found that in the 1D case all test cases have `dtypes in [None, "infer"]` (and the None case defaults to "infer"), and as you noted the 2D case only accepts "infer".  Anything we can do to be stricter and/or get downcast out of Block would be a plus by my lights
Could deprecate the downcast keyword?
> Could deprecate the downcast keyword?

id be on board for this, no idea about others.
thanks @phofl 
> > Could deprecate the downcast keyword?
> 
> id be on board for this, no idea about others.

yeah +1 here as well. this was an artifact from a while back when this was expensive. we should always just downcast (as that is what we do everywhere else)
Open an issue about this or directly a pr?
> Open an issue about this or directly a pr?

probably merits an issue, since im not sure @jreback and i are on the same page.  my immediate interest is in getting the downcast keyword out of Block.fillna (and ideally Block.interpolate), which i think is distinct from what he has in mind
opened #40988
REF: NDFrame dont mixin SelectionMixin
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code

Comments:
Ping
TYP: implement typing.Manager2D
State: closed

Makes it feasible to annotate a Manager as specifically-2D, which I find makes it easier to reason about some of the groupby code.

@jorisvandenbossche any objection to a refactor of ArrayManager to make SingleArrayManager share a parent class with ArrayManager, but not subclass it?  See https://github.com/pandas-dev/pandas/compare/master...jbrockmendel:typ-mgr-2d?expand=1#diff-80af4c7d6ece8cfeec6401a5f5babe27f19a7fc7476ad9148dca02f58a07a8dcR191

@simonjayhawkins the annotation for obj_with_exclusions https://github.com/pandas-dev/pandas/compare/master...jbrockmendel:typ-mgr-2d?expand=1#diff-f1ec980d06b0b54c8263f663f767636c3f5921f04f8a7ee91dfc71a2100b05cdR934 is a bit sketchy since it is a cache_readonly, not an attribute.  Thoughts on how best to handle this?  (Also, annotating it incorrectly as DataFrame in SeriesGroupBy doesn't get mypy to complain, which seems like it should?)

Labels:
name: Refactor
description: Internal refactoring of code
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
> any objection to a refactor of ArrayManager to make SingleArrayManager share a parent class with ArrayManager, but not subclass it

That sounds fine to me
thanks!
PERF: use ndarray.take instead of algos.take
State: closed

Un-revert part of #40510 in the hopes of tracking down where the perf impact was.

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
ok is this WIP?
> ok is this WIP?

no, this is ready
Can you run some benchmarks to verify this time?
Eg the one I mentioned in https://github.com/pandas-dev/pandas/pull/40818#issuecomment-816463888. I didn't list the exact impacted benchmarks in https://github.com/pandas-dev/pandas/pull/40510#issuecomment-808496069, but eg the select_dtypes ones were also affected.
> Eg the one I mentioned in #40818 (comment). I didn't list the exact impacted benchmarks in #40510 (comment), but eg the select_dtypes ones were also affected.

Just ran the time_frame_agg one, no change.  The non-asv motivator remains

```
In [1]: from pandas.core.algorithms import *
In [2]: arr = np.arange(10000)
In [3]: taker = np.random.randint(0, 10000, 1000)
In [4]: res = arr.take(taker)
In [5]: res2 = take_nd(arr, taker, allow_fill=False)

In [6]: %timeit res = arr.take(taker)
   ...: 
4.21 µs ± 93.8 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [7]: %timeit res2 = take_nd(arr, taker, allow_fill=False)
   ...: 
8.9 µs ± 670 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

```
`-b select_dtypes` also no change
gentle ping; this should be an unambiguous perf win
And this time https://pandas.pydata.org/speed/pandas/#/ doesn't show any regressions!