BrandonFors/pandas-github-issues huggingface dataset:
DOC: add data_columns to doc string of to_hdf and HDFStore.append
State: closed

Follow-up of #13951, rebase of PR #13065


Labels:
name: Docs

Comments:
I combined the changes of #13065 with the changes merged in #13951 (found the explanation a bit clearer in #13065), and fixed the formatting

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14046?src=pr) is 85.25% (diff: 100%)

> Merging [#14046](https://codecov.io/gh/pydata/pandas/pull/14046?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #14046   diff @@
==========================================
  Files           139        139          
  Lines         50380      50380          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42953      42953          
  Misses         7427       7427          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [453bc26...142fb78](https://codecov.io/gh/pydata/pandas/compare/453bc26e07ebbfea33dda9ff2f4efab683718506...142fb782ecf9e4e5384af9261ee1c1feb3799c59?src=pr)

@jreback the way to prevent the need for `#noqa` in docstrings for long lines (as that is rather strange in the html docs), is to ensure that the too long line consists of only one word/url (so no spaces), then the linter does not complaining.

So in this example, I changed

```
...
`here <http://pandas.pydata.org/pandas-docs/stable/io.html#query-via-data-columns>`__ # noqa
...
```

with

```
... `here 
<http://pandas.pydata.org/pandas-docs/stable/io.html#query-via-data-columns>`__
...
```

(just letting you know for reviewing purposes :-)) Maybe we should document that somewhere ..

@jorisvandenbossche ahh good to know
unique docstring extend #13565 unique datetime tz issue
State: closed

 closes  #13565
whatsnew
extended doc string for unique addressing the issue with timezone aware datetimes ( see #13565 )


Labels:

Comments:
not this not correct. documenting this is silly when it can easily be fixed.

I thought it can't be fixed easily / its the expected behavior (your own words) so i added a the docstring. 

well its the 'expected behavior' because of numpy silliness

@paulgueltekin Thanks for the PR! I think it would be good to document this if we leave it as is (even when it is due to limitations of numpy). However, we are now thinking that we will return an object array of Timestamps (which can still hold the timezone information), and in that case this PR is not needed (see PR https://github.com/pydata/pandas/pull/13979#issuecomment-241200579)
BUG: DataFrame.to_records() bug in converting datetime64 index
State: closed

BUG: DataFrame.to_records() bug in converting datetime64 index with timezone #13937


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Timezones
description: Timezone data dtype

Comments:
needs tests and a whats new note.

@jreback  Tests for this issue is already present in file pandas/pandas/tests/frame/test_convert_to.py. 

if there were tests then your change would fail. pls add them 

there are very minimal tests in any event
and certainly nothing related to conversions of tz

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14044?src=pr) is 85.26% (diff: 100%)

> Merging [#14044](https://codecov.io/gh/pydata/pandas/pull/14044?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #14044   diff @@
==========================================
  Files           140        140          
  Lines         50634      50634          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43173      43173          
  Misses         7461       7461          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [d98e982...2460a7d](https://codecov.io/gh/pydata/pandas/compare/d98e982ecb0f6b7f3478a63caac34e0be238895c...2460a7d3af4681c5923ebc70ffc3753fc81374fe?src=pr)

@jreback This build with python2.7 failed. But there is not an error with the tests, there is an error with the Travis CI. It shows error like "The command "ci/lint.sh" exited with 1." 

you need to lint the code as indicated.

@jreback Build is successful. Please take a look.
CLN: Drop lags parameter from Panel.shift
State: closed

Title is self-explanatory.

xref #6928


Labels:
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14041?src=pr) is 85.25% (diff: 100%)

> Merging [#14041](https://codecov.io/gh/pydata/pandas/pull/14041?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #14041   diff @@
==========================================
  Files           139        139          
  Lines         50381      50380     -1   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42953      42953          
+ Misses         7428       7427     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [a01e58f...8cf5557](https://codecov.io/gh/pydata/pandas/compare/a01e58fa780ed28f983e1a022b07e196e037d42a...8cf5557e83fe9c0b828ccd37b42a9a63bb8898f7?src=pr)

@gfyoung Thanks!
CLN: Removed DataFrame.to_wide
State: closed

`to_wide` was deprecated since 2011 (https://github.com/pydata/pandas/commit/1134c9f6c40398d8eba37c312c88ce294e31c6c5)

xref https://github.com/pydata/pandas/issues/13777

No tests for this deprecation beforehand to remove :disappointed: 


Labels:
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14039?src=pr) is 85.25% (diff: 100%)

> Merging [#14039](https://codecov.io/gh/pydata/pandas/pull/14039?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #14039   diff @@
==========================================
  Files           139        139          
  Lines         50380      50379     -1   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42953      42952     -1   
  Misses         7427       7427          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [453bc26...7879d54](https://codecov.io/gh/pydata/pandas/compare/453bc26e07ebbfea33dda9ff2f4efab683718506...7879d549601f2f7b1b3f37dd96d612f296f3577e?src=pr)

lgtm. ping on green.

Let's leave that for another PR :-) (I/we will do a full read through of the whatsnew file anyway)

@gfyoung Thanks!
BUG: iloc fails with non lex-sorted MultiIndex #13797
State: closed

- [x] closes #13797
- [x] tests added / passed pandas/tests/test_multilevel.py
- [x] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry

corrected how iloc handles tuple-keys for multiindex


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14038?src=pr) is 85.23% (diff: 100%)

> Merging [#14038](https://codecov.io/gh/pydata/pandas/pull/14038?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #14038   diff @@
==========================================
  Files           140        140          
  Lines         50563      50563          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43101      43099     -2   
- Misses         7462       7464     +2   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [6c73e76...1cf86a1](https://codecov.io/gh/pydata/pandas/compare/6c73e765a5f1191c954c7789791818d2685c3c2c...1cf86a184739191af54a28c6a2664b2166c49c78?src=pr)

Yikes! This is a testament to how convoluted the indexing code currently is. If we did this right, it would not be possible for code using iloc to get anywhere near checks for multi-indexes. 

> If we did this right, it would not be possible for code using iloc to get anywhere near checks for multi-indexes. 

But should it? For `iloc` it shouldn't matter if you have a multi-index or not how the indexer is interpreted

> Yikes! This is a testament to how convoluted the indexing code currently is. If we did this right, it would not be possible for code using iloc to get anywhere near checks for multi-indexes.

This is not a helpful comment. This is just an implemention detail of the indexer. It could be a sub-class but its not, otherwise this would just be an overriden method.

> This is not a helpful comment. This is just an implemention detail of the indexer. It could be a sub-class but its not, otherwise this would just be an overriden method.

This fix looks fragile to me. That said, it's probably the right choice for now. It fixes the problem at hand, but its not obvious that it's fully correct, because too many different code paths reuse the same logic in different ways.

can you rebase / update?

Yes. I can probably find a time tomorrow.

@jreback 
Sorry for taking this long time. I added some comments, and modified the test code so that the test purpose became clear (i.e. iloc handles tuple as an integer locator instead of trying to interpret as multiindex representation.)

thanks!
DOC: unpin IPython version (GH13639)
State: closed


Labels:
name: Docs

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14037?src=pr) is 85.25% (diff: 100%)

> No coverage report found for **master** at 453bc26.
> 
> Powered by [Codecov](https://codecov.io?src=pr). Last update [453bc26...76594f5](https://codecov.io/gh/pydata/pandas/compare/453bc26e07ebbfea33dda9ff2f4efab683718506...76594f56655694a0a0bcb542d7c9b06c46e83118?src=pr)
BUG: #14016. Preserve categorical keys through concat
State: closed

- [x] closes #14016 
- [x] tests added / passed
- [ ] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Categorical
description: Categorical Data Type

Comments:
can you rebase / update?

can you rebase / update

can you rebase?
rebase this, some other changes may have fixed / made this easier.
can you rebase / update?
can you rebase
closing as stale
Require a problem description in issues
State: closed

Currently some issues are just code examples without a description **why** a change should be done. This leads to problems when (years later) the current behaviour is questioned and no one can remember why it was changed.


Labels:

Comments:
I think it may better to be added:
- checkbox whether the reporter uses the latest version
  - reason otherwise

needs discussion though

In Addition to the reason why the thing is actually a problem?

@jorisvandenbossche @TomAugspurger 

See also https://github.com/pydata/pandas/pull/14031 for basically the same thing for PRs. That one also has a small example why I think such a description is needed.

The addition seems fine to me.

ok, let's see how this works.

thanks @janschulz
BUG: read_fwf inference should respect skiprows (#11256)
State: closed

Fix the fact that we don't skip the rows when inferring colspecs by passing skiprows down the chain until it's needed. 
- [X] closes #11256
- [X] 3 tests added / passed
- [X] passes `git diff upstream/master | flake8 --diff`
- [X] whatsnew entry


Labels:
name: API Design
name: IO CSV
description: read_csv, to_csv

Comments:
can you rebase / update?

@dsm054 Do you have time to update this?
can you rebase / update

pls move release note to 0.20.0
Okay, will have another look at this now while rooting for Clemson.
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14028?src=pr) is 84.77% (diff: 95.00%)
> Merging [#14028](https://codecov.io/gh/pandas-dev/pandas/pull/14028?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14028   diff @@
==========================================
  Files           145        145          
  Lines         51212      51220     +8   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43413      43421     +8   
  Misses         7799       7799          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [e1a4144...b5b3e66](https://codecov.io/gh/pandas-dev/pandas/compare/e1a41446484c1c7284d37575dc2d271efd62f890...b5b3e667885492aec8898c6fb5940ad83586f0cd?src=pr)
thanks @dsm054 

if you want to do a followup to handle ``comment=`` in the same way would be great!

and I guess ``skip_blank_lines`` as well?
TST: Fix test_coercion for period dtype
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`

fix test to meet period dtype #13941.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14025?src=pr) is 85.30% (diff: 100%)

> No coverage report found for **master** at 6fa2b03.
> 
> Powered by [Codecov](https://codecov.io?src=pr). Last update [6fa2b03...b873683](https://codecov.io/gh/pydata/pandas/compare/6fa2b03d78a8a6e94de66a01852a01a5574af2a9...b873683797b8346b1c747a6ab8866580748d97f0?src=pr)

ty sir!
BUG: Parse NULL char as null value
State: closed

Fixes bug in C parser in which the `NULL` character (`'\x00'`) was being interpreted as a true line terminator, escape character, or comment character because it was used to indicate that a user had not specified these values. As a result, if the data contains this value, it was being incorrectly parsed. It should be parsed as `NULL`.

Closes #14012.


Labels:
name: Bug
name: IO CSV
description: read_csv, to_csv

Comments:
Points of clarification to make here:

1) The reason why we still default `lineterminator` to `'\0'` instead of say `'\n'` is to allow for proper carriage character (`'\r'`) detection, as can be seen <a href="https://github.com/pydata/pandas/blob/5d791cc7d955c0b074ad602eb03fa32bd3e17503/pandas/src/parser/tokenizer.c#L693">here</a>.

2) The test I added will fail if it weren't any of the modified `lineterminator`, `escapechar`, or `commentchar` checks.  They all used to compare against `'\0'` if none was specified, causing that stray `'\x00'` character to be incorrectly parsed in some way, shape, or form.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14019?src=pr) is 85.29% (diff: 100%)

> Merging [#14019](https://codecov.io/gh/pydata/pandas/pull/14019?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #14019   diff @@
==========================================
  Files           139        139          
  Lines         50241      50241          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42851      42851          
  Misses         7390       7390          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [5d791cc...5d39744](https://codecov.io/gh/pydata/pandas/compare/5d791cc7d955c0b074ad602eb03fa32bd3e17503...5d39744ef0cbc9db4e0d6c2c712eda2fe81482aa?src=pr)

thanks!
TST: add comprehensive coercion tests
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`

1st attempt to make dtype related tests more comprehensive. 

Validate whether tests classes have test methods covering dtype and Series/Index. Some of them are empty yet, going to add it gradually. 


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14010?src=pr) is 85.29% (diff: 100%)

> Merging [#14010](https://codecov.io/gh/pydata/pandas/pull/14010?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #14010   diff @@
==========================================
  Files           139        139          
  Lines         50241      50241          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42849      42851     +2   
+ Misses         7392       7390     -2   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [6b7857b...de3eb22](https://codecov.io/gh/pydata/pandas/compare/6b7857b0154353f3fc1e80110876371cb10b2136...de3eb22b8ccbf720f9d56891250ffda2b0ca5d34?src=pr)

thanks @sinhrks you are great on these comprehensive tests!
DEPR: Deprecate SparseList
State: closed

Title is self-explanatory.  Closes #13784.


Labels:
name: Sparse
description: Sparse Data Type
name: Deprecate
description: Functionality to remove in pandas

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14007?src=pr) is 85.28% (diff: 100%)

> Merging [#14007](https://codecov.io/gh/pydata/pandas/pull/14007?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #14007   diff @@
==========================================
  Files           139        139          
  Lines         50229      50231     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42837      42840     +3   
+ Misses         7392       7391     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [8b50d8c...f1ce0e9](https://codecov.io/gh/pydata/pandas/compare/8b50d8c854196e3025c9a881cafeedc5f509aaef...f1ce0e926394eaf5c416de61e9078aaf2f1cd554?src=pr)

thanks!
BUG: Avoid sentinel-infinity comparison problems (#13445)
State: closed

The problem causing #13445 ultimately traces to the fact that our Infinity/NegInfinity objects were greater than/less than themselves, which violates an assumption numpy makes when sorting.  This was separately reported as https://github.com/numpy/numpy/issues/7934, but we can fix and test downstream as well.
- [X] closes #13445
- [X] tests added / passed
- [X] passes `git diff upstream/master | flake8 --diff`
- [X] whatsnew entry


Labels:
name: Bug
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14006?src=pr) is 85.28% (diff: 100%)

> Merging [#14006](https://codecov.io/gh/pydata/pandas/pull/14006?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #14006   diff @@
==========================================
  Files           139        139          
  Lines         50229      50229          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42837      42838     +1   
+ Misses         7392       7391     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [8b50d8c...7d79370](https://codecov.io/gh/pydata/pandas/compare/8b50d8c854196e3025c9a881cafeedc5f509aaef...7d79370153954d9159e4258909c9e8685568ee1d?src=pr)

thanks @dsm054 

always nice to see PR's from you! keep em coming!
BUG: No affect both index and column(GH 13522)
State: closed

- [x] closes #13522 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14005?src=pr) is 85.29% (diff: 100%)

> Merging [#14005](https://codecov.io/gh/pydata/pandas/pull/14005?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #14005   diff @@
==========================================
  Files           139        139          
  Lines         50245      50247     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42857      42859     +2   
  Misses         7388       7388          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [7c0b742...6dfd2bc](https://codecov.io/gh/pydata/pandas/compare/7c0b74287408a2dcff4cbcafbbebbf9847a6905a...6dfd2bcfe6b3902c8d3774b4d0b3b07f0d611c71?src=pr)

can you run an asv and see if anything changed signficantly. lgtm otherwise.

@jreback all benchmarks not significantly changed beside groupby.groupby_frame_singlekey_integer.time_sum. It ranges from 1 to 1.2

thanks!
DOC: fixed using IP.prompt_manager which is removed from IPython 5.x
State: closed

- [x] closes #14003, #13639
- [ ] tests added / passed (there's no test on building docs)
- [x] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry (it's not user facing bug fix)

this PR was made during PyCon APAC, pandas sprint

thanks


Labels:
name: Build
description: Library building on various platforms
name: Docs

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14004?src=pr) is 85.28% (diff: 100%)

> Merging [#14004](https://codecov.io/gh/pydata/pandas/pull/14004?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #14004   diff @@
==========================================
  Files           139        139          
  Lines         50241      50241          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42849      42849          
  Misses         7392       7392          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [6b7857b...cfd9499](https://codecov.io/gh/pydata/pandas/compare/6b7857b0154353f3fc1e80110876371cb10b2136...cfd94992134852d36877d5dc45f26c3960a838ba?src=pr)

Thanks! 
Added a small comment. I also think a whatsnew notice is not needed, as it is not a user facing change/bug fix

Thanks! Done what you said.
- revert to # comment
- no whatsnew notice

@znmean Thanks!