BrandonFors/pandas-github-issues huggingface dataset:
BUG: Patch maybe_convert_objects uint64 handling
State: closed

Makes method robust against known `numpy` bug that you can't compare `uint64` against `int64`
because they are casted to `float64` during the comparison, causing truncation.

xref #14937.
Follow-up to #14916.

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14951?src=pr) is 84.64% (diff: 100%)
> Merging [#14951](https://codecov.io/gh/pandas-dev/pandas/pull/14951?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #14951   diff @@
==========================================
  Files           144        144          
  Lines         51021      51021          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43188      43188          
  Misses         7833       7833          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [f79bc7a...987ae5c](https://codecov.io/gh/pandas-dev/pandas/compare/f79bc7a9d128187f3a93a3dae84ff03f4f4a62f4...987ae5c484d04e55704a9bbdebbfd96787b0a4f8?src=pr)
thanks!
added 'separator' argument to json_normalize
State: closed

 - [x] closes #14883
 - [ ] tests added / passed (added 3 tests: `test_simple_normalize_with_{default, user_specified, user_specified_unicode}_separator`)
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry (v0.20.0)

However, this doesn't work, even after making the fixes suggested in https://github.com/pandas-dev/pandas/pull/14891. I thought replacing `'.'` in `meta_keys = ['.'.join(val) for val in meta]` with `sep` would do the trick. It doesn't, so I'm a little puzzled. Happy to take a suggestion.

Labels:

Comments:
Gonna try again from scratch.
https://github.com/pandas-dev/pandas/pull/14950
BUG: sorting with large float and multiple columns incorrect
State: closed

Fixes https://github.com/pandas-dev/pandas/issues/14922
    
Having the `int` equivalent of `NaT` in an `int64` column caused wrong sorting because this special value was considered as "missing value".

 - [ ] closes #xxxx
 - [X] tests added / passed
 - [X] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
@jreback please review https://github.com/pandas-dev/pandas/pull/14944/commits/03699c67a5953882e025ea6eb2607ea3053c9337, I used rebase and now the changes do not show up in github as expected.
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14944?src=pr) is 84.65% (diff: 100%)
> Merging [#14944](https://codecov.io/gh/pandas-dev/pandas/pull/14944?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14944   diff @@
==========================================
  Files           144        144          
  Lines         51021      51031    +10   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43188      43199    +11   
+ Misses         7833       7832     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [f79bc7a...c244438](https://codecov.io/gh/pandas-dev/pandas/compare/f79bc7a9d128187f3a93a3dae84ff03f4f4a62f4...c244438e487d8fbfb3b055fdd7a29dbc8edbdf98?src=pr)
can you add a similar tests which uses datetimes and a NaT as well (just to verify that it doesn't break).

further pls check both ascending and descending.
can you add a whatsnew note (0.20.0, bug fixes)
Ok now ?
thanks!
DOC: Pandas Cheat Sheet (GH13202)
State: closed

 - [x ] closes #13202 

Pandas Cheat Sheet.  Someone needs to figure out (or tell me the best way to do it!) how the PDF version gets integrated into the documentation tree.


Labels:
name: Docs

Comments:
I see!.

can you add a whatsnew note (put in a  path, I can fix if its not exactly right).
@jreback Pushed out a whatsnew.  Not sure why there is a conflict.  The other thing I'm not sure of is whether the main documentation tree should have a link to the cheat sheet, and if so, how to make it so that the whole documentation build process isn't messed up by a link to something outside the tree.
thanks!

I merged this, and so should show up on the doc-build.! (the whatsnew reference).
https://github.com/pandas-dev/pandas/blob/master/doc/cheatsheet/Pandas_Cheat_Sheet.pdf

!
once we upload this to pandas.pydata.org will need to fix the link
cc @jorisvandenbossche
TST: Concat MultiIndex dataframes with deepcopy (#9967)
State: closed

 - [x] closes #9967
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``

Potentially related to #10638? Otherwise, don't think this was fixed recently

Labels:
name: Bug
name: MultiIndex

Comments:
yeah not really sure when this was fixed, if you *really* want, you can ``git bisect`` to narrow it down ....
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14936?src=pr) is 84.65% (diff: 100%)
> Merging [#14936](https://codecov.io/gh/pandas-dev/pandas/pull/14936?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14936   diff @@
==========================================
  Files           144        144          
  Lines         51021      51021          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43188      43191     +3   
+ Misses         7833       7830     -3   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [f79bc7a...55303de](https://codecov.io/gh/pandas-dev/pandas/compare/f79bc7a9d128187f3a93a3dae84ff03f4f4a62f4...55303de6c61fba632c178c04d9b126bf8f49117d?src=pr)
thanks!
BUG: Patch rank() uint64 behavior
State: closed

Adds `uint64` ranking functions to `algos.pyx` to allow for proper ranking with `uint64`.

Also introduces partial patch for `factorize()` by adding `uint64` hashtables and vectors for
usage. However, this patch is only partial because the larger bug of non-support for `uint64`
in `Index` has not been fixed (**UPDATE**: tackled in #14937):
~~~python
>>> from pandas import Index, np
>>> Index(np.array([2**63], dtype=np.uint64))
Int64Index([-9223372036854775808], dtype='int64')
~~~

Also patches a bug in `UInt64HashTable` from #14915 that had an erroneous null condition that was caught during testing and was hence removed.

Note there is overlap with #14934 with the implementation of `is_signed_integer_dtype` and `is_unsigned_integer_dtype`.  That PR should be merged before this one.

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14935?src=pr) is 84.66% (diff: 100%)
> Merging [#14935](https://codecov.io/gh/pandas-dev/pandas/pull/14935?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14935   diff @@
==========================================
  Files           144        144          
  Lines         51047      51056     +9   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43216      43225     +9   
  Misses         7831       7831          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [d7e8f31...2598cea](https://codecov.io/gh/pandas-dev/pandas/compare/d7e8f316dc7155252fa47ea23a3074d7d0197e54...2598ceaa753a146d23899d9e1ed48e7557ffce2d?src=pr)
@jreback : With #14934 merged, this is ready to be merged now.
@jreback : Addressed all comments, and both Appveyor and Travis are passing.  Ready to merge if there are no other concerns.
lgtm. ping on green.
@jreback : Everything is green now.  Ready to merge if there are no other concerns.
thanks!
BUG, TST: Check uint64 behaviour in algorithms.py
State: closed

First of a series of PR's to patch and test `uint64` behaviour in `core/algorithms.py`.  In this PR, the following functions are checked:

1. `duplicated()` : robust but now has test to confirm
2. `mode()` : robust but now has test to confirm
3. `unique()` : non-robust but patched and tested

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14934?src=pr) is 84.65% (diff: 100%)
> Merging [#14934](https://codecov.io/gh/pandas-dev/pandas/pull/14934?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14934   diff @@
==========================================
  Files           144        144          
  Lines         51030      51043    +13   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43199      43213    +14   
+ Misses         7831       7830     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [74de478...6d31057](https://codecov.io/gh/pandas-dev/pandas/compare/74de478392e09cf938d244f5990da4e001afc84c...6d31057d0b1b9bb6f2840771a9aff3860f2b9b00?src=pr)
@jreback : Tests are green.  Ready to merge if there is nothing else.
@jreback : Addressed all comments, and everything is green and passing.  Ready to merge if there are no other concerns.
few comments
thanks!
TST: Groupby.filter dropna=False with empty group (#10780)
State: closed

 - [x] closes #10780
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``

Didn't find a PR in 0.20 or 0.19.2 that closed this, but works on master 0.19.1

Labels:
name: Bug
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14926?src=pr) is 84.65% (diff: 100%)
> Merging [#14926](https://codecov.io/gh/pandas-dev/pandas/pull/14926?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14926   diff @@
==========================================
  Files           144        144          
  Lines         51023      51023          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43190      43191     +1   
+ Misses         7833       7832     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3ab0e55...ee4ce0f](https://codecov.io/gh/pandas-dev/pandas/compare/3ab0e552c9bac85048dc001d764bb7b903f73a36...ee4ce0f8087436e903497076ea6137521d1e177d?src=pr)
Added test for `dropna=True` and all green. 
thanks!
PERF: make all inference routines cpdef bint
State: closed

 going to rebase after #14827 

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14925?src=pr) is 84.64% (diff: 100%)


> No coverage report found for **master** at 5faf32a.

> Powered by [Codecov](https://codecov.io?src=pr). Last update [5faf32a...eb07104](https://codecov.io/gh/pandas-dev/pandas/compare/5faf32a62b52912bc0c4a26622bfc3d72b5121ff...eb0710469ab035ec81f9b5a07506d3604a665b41?src=pr)
MAINT: Only output errors in C style check
State: closed

Title is self-explanatory.  Follow-up to #14814.


Labels:
name: Build
description: Library building on various platforms

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14924?src=pr) is 84.64% (diff: 100%)
> Merging [#14924](https://codecov.io/gh/pandas-dev/pandas/pull/14924?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #14924   diff @@
==========================================
  Files           144        144          
  Lines         51019      51019          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43188      43187     -1   
- Misses         7831       7832     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [39efbbc...40e6e06](https://codecov.io/gh/pandas-dev/pandas/compare/39efbbce6e20a48b37809393714788bf67db8067...40e6e06c258013e5ccfabfade77895696ee4b4d0?src=pr)
lgtm. can you also move the pip install cpplint to the top of lint.sh (so it doesn't mess up the output).

ping when pushed.
@jreback : Moved `cpplint` install to right after the `if` statement (no need to install if not linting).
TST: Test empty input for read_csv (#14867)
State: closed

 - [x ] closes #14867
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [ ] whatsnew entry

LMK if it would be more suitable in a different test file.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO CSV
description: read_csv, to_csv

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14920?src=pr) is 84.64% (diff: 100%)
> Merging [#14920](https://codecov.io/gh/pandas-dev/pandas/pull/14920?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14920   diff @@
==========================================
  Files           144        144          
  Lines         51016      51016          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43184      43185     +1   
+ Misses         7832       7831     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3ccb501...46ad2a2](https://codecov.io/gh/pandas-dev/pandas/compare/3ccb50131b698bccee21780c660a70bed87396d0...46ad2a2b325132234221c1bb57b6d421850a00f8?src=pr)
Removed the extra test, all good now.
thanks!
BUG: Don't convert uint64 to object in DataFrame init
State: closed

The hack used to resolve #2355 is no longer needed. Removes the hack and patches several tests that relied on this hacky (and buggy) behavior.

Closes #14881.

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14917?src=pr) is 84.57% (diff: 100%)
> Merging [#14917](https://codecov.io/gh/pandas-dev/pandas/pull/14917?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14917   diff @@
==========================================
  Files           144        144          
  Lines         51057      51053     -4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43180      43177     -3   
+ Misses         7877       7876     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [f1cfe5b...9fe2199](https://codecov.io/gh/pandas-dev/pandas/compare/f1cfe5b4eb614f9b4e201e764e9bd25f2389b355...9fe2199ab85ac21c88f1d0fa6e1dfe48258ec5df?src=pr)
can you add the actual given test from the issue as well.
@jreback : It's no different from the first test of `test_constructor_overflow_int64`.
ok then.
BUG: Convert uint64 in maybe_convert_objects
State: closed

Adds handling for `uint64` objects during conversion.  When negative numbers and `uint64` are detected, we then convert the result to `object`.

Picks up where #4845 left off. Closes #4471.


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14916?src=pr) is 84.64% (diff: 100%)
> Merging [#14916](https://codecov.io/gh/pandas-dev/pandas/pull/14916?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14916   diff @@
==========================================
  Files           144        144          
  Lines         51016      51016          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43184      43185     +1   
+ Misses         7832       7831     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3ccb501...ed325cd](https://codecov.io/gh/pandas-dev/pandas/compare/3ccb50131b698bccee21780c660a70bed87396d0...ed325cdf184d4b4e28b43aeae3aeb7574a66c571?src=pr)
thanks!
BUG: Prevent uint64 overflow in Series.unique
State: closed

Introduces a `UInt64HashTable` class to hash `uint64` elements and prevent overflow in functions like `Series.unique`.

Closes #14721.

Labels:
name: Bug
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14915?src=pr) is 84.65% (diff: 100%)
> Merging [#14915](https://codecov.io/gh/pandas-dev/pandas/pull/14915?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14915   diff @@
==========================================
  Files           144        144          
  Lines         51016      51020     +4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43184      43189     +5   
+ Misses         7832       7831     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3ccb501...380c580](https://codecov.io/gh/pandas-dev/pandas/compare/3ccb50131b698bccee21780c660a70bed87396d0...380c58041a2cf327b1cc2de71586707dbdf16129?src=pr)
you will need to rebase after #14919. I moved a couple of things, and added a HashTable testing section.

Though what I would really like is to systematically add unit64 to as many places as we test other dtypes and see what breaks / falls out (can do in another PR). Then create issues for incompat places.
what was the bug (from before I moved unique)?
@jreback : The <a href="https://github.com/pandas-dev/pandas/pull/14915/files#diff-fafd7700903bc35b7e73b5e1fa65b6daR133">diff</a> explains it all.  We were smashing all integer types together into one hashtable, which does not work if you have `uint`.
ok, lgtm. ping on green.
@jreback : Sounds good.  I changed my mind about changing `pd.unique` in this PR.  `core.algorithms` is riddled with `uint64` bugs that need to be checked.  I'll tackle in a follow-up.
> @jreback : Sounds good. I changed my mind about changing pd.unique in this PR. core.algorithms is riddled with uint64 bugs that need to be checked. I'll tackle in a follow-up.

totally fine. we can make a master issue of these if that is helpful.
@jreback : Everything is green, so ready to merge if there are no other concerns.
thanks!

as I said, ideally have an issue which greatly expands test coverage for uint64 (and just mark the tests that are failing (e.g. assert that they are failing or comment out)).

Then can come back around and fix.
BUG: Fixed DataFrame.describe percentiles are ndarray w/ no median
State: closed

Explicit conversion to list for `percentiles`. Fixes the case where `percentiles` is passed as a numpy with no median (0.5) present. Closes #14908.


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
Thx, can u add tests and whatsnew?
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14914?src=pr) is 84.64% (diff: 100%)
> Merging [#14914](https://codecov.io/gh/pandas-dev/pandas/pull/14914?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **0.07%**

```diff
@@             master     #14914   diff @@
==========================================
  Files           144        144          
  Lines         51057      51016    -41   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43180      43185     +5   
+ Misses         7877       7831    -46   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [f1cfe5b...5c8199b](https://codecov.io/gh/pandas-dev/pandas/compare/f1cfe5b4eb614f9b4e201e764e9bd25f2389b355...5c8199b89490c26e980c473219b1c4b75d4cf057?src=pr)
small changes. ping on green.
lgtm. ping on green.
thanks!
TST:Test to_sparse with nan dataframe (#10079)
State: closed

 - [x] closes #10079
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``

Possibly fixed with a PR that fixed one of #10627. Look like `to_dense` is fixed for 0.20.0, but should be unrelated to this. 

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Sparse
description: Sparse Data Type
name: MultiIndex

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14913?src=pr) is 84.57% (diff: 100%)
> Merging [#14913](https://codecov.io/gh/pandas-dev/pandas/pull/14913?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14913   diff @@
==========================================
  Files           144        144          
  Lines         51057      51057          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43180      43181     +1   
+ Misses         7877       7876     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [f1cfe5b...48cd7e2](https://codecov.io/gh/pandas-dev/pandas/compare/f1cfe5b4eb614f9b4e201e764e9bd25f2389b355...48cd7e2388471e9229fb32e504b0abb121de8f28?src=pr)
thanks @mroeschke !

finding and testing these orphan issues is really great!
No problem! Making a push to clean up stale issues that have been solved, so more PRs to come.