BrandonFors/pandas-github-issues huggingface dataset:
DOC: contributing: explain how to set the asv reporting threshold
State: closed

The default `asv continuous` reporting threshold is 100% change (2x), which probably is far too large for pandas use case, so instruct how to set it lower. (Probably the asv default should be changed, but that's a different issue.)


Labels:
name: Docs

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13807?src=pr) is 85.23% (diff: 100%)

> Merging [#13807](https://codecov.io/gh/pydata/pandas/pull/13807?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13807   diff @@
==========================================
  Files           140        140          
  Lines         50415      50415          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42970      42970          
  Misses         7445       7445          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [8f64ad7...4014ebb](https://codecov.io/gh/pydata/pandas/compare/8f64ad700bb254e17fdb006b08199a2cc6ee5f36...4014ebbf7e381f28839814d2ddd0036759284b7f?src=pr)

Yes, thanks! That's a better default
ASV: fix params to be strings (for better repr)
State: closed

Small fix to using params in asv (otherwise the name was huge, list of 500000 elements)

cc @gfyoung (just so you know I changed this for future reference)


Labels:
name: Performance
description: Memory or execution speed performance

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13805?src=pr) is 85.23% (diff: 100%)

> Merging [#13805](https://codecov.io/gh/pydata/pandas/pull/13805?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13805   diff @@
==========================================
  Files           140        140          
  Lines         50415      50415          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42970      42970          
  Misses         7445       7445          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [8f64ad7...06a5fa4](https://codecov.io/gh/pydata/pandas/compare/8f64ad700bb254e17fdb006b08199a2cc6ee5f36...06a5fa4123e15340a9b9917d1fb339b5e8503e45?src=pr)
CLN: Removed the kwds param in to_csv
State: closed

We aren't using it anymore with the removal of parameters like `engine`.  Closes #8206.


Labels:
name: IO CSV
description: read_csv, to_csv

Comments:
Boo...tests and the `Series` object were abusing the `**kwds` parameter.  Though slightly awkward since I was the one who removed the `engine` parameter in `to_csv` and forgot about these tests. :cry:

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13804?src=pr) is 85.23% (diff: 100%)

> No coverage report found for **master** at 2c55f28.
> 
> Powered by [Codecov](https://codecov.io?src=pr). Last update [2c55f28...f096812](https://codecov.io/gh/pydata/pandas/compare/2c55f28fa7e6468d347894f26d6ecddcfada9cd9...f096812a2f25eee117f73057d198d458103892d4?src=pr)

So we did actually already remove `nanRep` in 0.14 (https://github.com/pydata/pandas/pull/6813, it was deprecated before), but it seems that it was forgotten to remove it in `Series.to_csv` as well. And also it is still listed in the docstring of `DataFrame.to_csv`. Can you remove that as well?

@jreback , @jorisvandenbossche : made the requested doc changes (hence the `[ci skip]`).  Ready to merge if there are no other concerns.

@gfyoung Thanks!
BUG: RangeIndex can be created without args
State: closed

- [x] closes #13793
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
lgtm.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13803?src=pr) is 85.23% (diff: 100%)

> Merging [#13803](https://codecov.io/gh/pydata/pandas/pull/13803?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13803   diff @@
==========================================
  Files           140        140          
  Lines         50415      50420     +5   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42970      42974     +4   
- Misses         7445       7446     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [98c5b88...aab6ae6](https://codecov.io/gh/pydata/pandas/compare/98c5b88d6f9f7bb0afa4fbae49a045d93a1cb33f...aab6ae6bc39f881dcaf93ed5df2d3513c3168880?src=pr)

thanks @sinhrks
BUG: TypeError in merge with timedelta64 column
State: closed

- [x] closes #13389
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

A root cause is `isnull` raises `TypeError` if datetime-like has other time unit than `ns`.

```
pd.isnull(np.array([1, 2], dtype='datetime64[D]'))
# TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
```


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Timedelta
description: Timedelta data type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13802?src=pr) is 85.23% (diff: 100%)

> Merging [#13802](https://codecov.io/gh/pydata/pandas/pull/13802?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13802   diff @@
==========================================
  Files           140        140          
  Lines         50415      50415          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42970      42969     -1   
- Misses         7445       7446     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [98c5b88...8dbfde2](https://codecov.io/gh/pydata/pandas/compare/98c5b88d6f9f7bb0afa4fbae49a045d93a1cb33f...8dbfde2204590455fc7e306ccdc405f91418d9eb?src=pr)
BUG: handle outofbounds datetimes in DatetimeConverter
State: closed

xref #2579

This at least solves the direct negative consequence (erroring code by importing pandas) of registering our converters by default.


Labels:
name: Bug
name: Visualization
description: plotting

Comments:
cc @ocehugo @tacaswell @TomAugspurger 

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13801?src=pr) is 85.23% (diff: 100%)

> Merging [#13801](https://codecov.io/gh/pydata/pandas/pull/13801?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13801   diff @@
==========================================
  Files           140        140          
  Lines         50415      50415          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42970      42972     +2   
+ Misses         7445       7443     -2   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [98c5b88...6b6b08e](https://codecov.io/gh/pydata/pandas/compare/98c5b88d6f9f7bb0afa4fbae49a045d93a1cb33f...6b6b08e4fe13f1de310cd25a184625aed114a178?src=pr)

working fine ;). No conflict with matplotlib so far in my usage. thanks for that!

@sinhrks @TomAugspurger OK to merge?

@sinhrks further comments? If not, be my guest to merge :-)

thx!
TST: Add test for skipfooter + decimal in read_csv
State: closed

Title is self-explanatory.  Only for Python engine because C engine doesn't support `skipfooter` yet.

Closes #6971.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO CSV
description: read_csv, to_csv

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13800?src=pr) is 85.23% (diff: 100%)

> Merging [#13800](https://codecov.io/gh/pydata/pandas/pull/13800?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13800   diff @@
==========================================
  Files           140        140          
  Lines         50415      50415          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42970      42970          
  Misses         7445       7445          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [98c5b88...f1ded8c](https://codecov.io/gh/pydata/pandas/compare/98c5b88d6f9f7bb0afa4fbae49a045d93a1cb33f...f1ded8c26ac0c62e44c44e48de3ffb6784e4526c?src=pr)

can you replicate the test in the associated issue as well.

@jreback : Ah, yes, good point.  Done.

thanks @gfyoung 

Also got a nice, round number for this PR as a bonus! :smile:
DOC: Added example to NDFrame.where
State: closed


Labels:
name: Docs

Comments:
I think adding how this can emulate np.where would also be useful (add in a Note section for that) 

from your expl on SO

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13798?src=pr) is 85.28% (diff: 100%)

> Merging [#13798](https://codecov.io/gh/pydata/pandas/pull/13798?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13798   diff @@
==========================================
  Files           139        139          
  Lines         50214      50214          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42822      42823     +1   
+ Misses         7392       7391     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [a0d05db...6ae0e79](https://codecov.io/gh/pydata/pandas/compare/a0d05dbf7477df863486d82a0cdd4e3e93023864...6ae0e79a63978513664340e54309b9c426e4e3b7?src=pr)

1. Added DataFrame.where/numpy.where example
2. Updated note according to @jorisvandenbossche and @jreback comments

Thanks for feedback!

can you update

Yep, will do.

On Tue, Aug 9, 2016 at 7:23 PM Jeff Reback notifications@github.com wrote:

> can you update
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> https://github.com/pydata/pandas/pull/13798#issuecomment-238722585, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/AMGNkRIvNC29BQrB3AwVQdWQd7xWOpDyks5qeQwDgaJpZM4JUwF8
> .

I noticed that most examples on the method's page use the `>>>` syntax. Is it ok to use the `.. ipython:: python` rst directive to generate them?

@jalexvig no, use the doc-string syntax. These are viewable from the command line, not generated docs (they are generated too, but that's not the primary use)

@jalexvig Thanks a lot!
BUG: int dtype for get_dummies
State: closed

closes #8725

Changes `get_dummies` return columns with `uint8` dtypes instead of coercing to floats if they were alongside other float columns.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Sparse
description: Sparse Data Type

Comments:
pls give a test on windows, as int dtype comparisons are wonky.

actually this may not work w/o #667 

@sinhrks would know more

Right I should have put a WIP in the title, sorry. That seems like a bit of a rabbit hole, but I'll give it a bit more time this week if I can.

> On Jul 27, 2016, at 05:42, Jeff Reback notifications@github.com wrote:
> 
> actually this may not work w/o #667
> 
> @sinhrks would know more
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or mute the thread.

I'm just working on #667, hopefully can send something this week... :)

Awesome, thanks :)

this has to be after #13849 

@TomAugspurger I merged the sparse PR of @sinhrks, so if you have the time, you can update this now

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13796?src=pr) is 85.27% (diff: 100%)

> Merging [#13796](https://codecov.io/gh/pydata/pandas/pull/13796?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13796   diff @@
==========================================
  Files           139        139          
  Lines         50553      50554     +1   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43110      43111     +1   
  Misses         7443       7443          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [58199c5...cace0f7](https://codecov.io/gh/pydata/pandas/compare/58199c5da263491a26fa6cb26d2ff9d38c4b5dac...cace0f7c550dea5197b7aa5eb0cbd49b357ee8b7?src=pr)

Green now. Lots of (small) changes to the tests, so a second pair of eyes would be grateful.

Looks good to me

thanks!

this makes this very memory efficient now! (and of course sparse is even better)
DOC: contributing: fix asv usage instructions
State: closed

The current asv usage instructions in contributing.rst may be unclear for PRs with multiple commits.

By default `asv continuous` uses head^ as the commit to compare to, whereas for PRs you usually want to compare to master. Write the instructions so that the comparison point is clear.


Labels:
name: Docs

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13794?src=pr) is 85.33% (diff: 100%)

> Merging [#13794](https://codecov.io/gh/pydata/pandas/pull/13794?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13794   diff @@
==========================================
  Files           141        141          
  Lines         50711      50711          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43275      43275          
  Misses         7436       7436          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3fdcea6...e624f9b](https://codecov.io/gh/pydata/pandas/compare/3fdcea6f89a5a9f9fdb98452343a21e576430bd9...e624f9beba65fc9690dc2f8e501814c83a712130?src=pr)

@pv Thanks a lot for clarifying this!

Yes, you can replace HEAD with a branch name, but is not necessary.

@pv I have actually another questions (was using the asv benchmark suite the last days): is there a way to run the suite for a specific commit? As with `asv run` I think you need to specify a range? 
The situation I wanted this was to compare the benchmarks of two commits (last release vs master), for which I wanted to use `asv compare`, but did not yet run benchmarks for only one of the two commits

asv run commit^!     (git single-commit range spec)

@pv Aha, thanks!
BUG: fix categories in HDFStore not filtering correctly (#13322)
State: closed

- [x] closes #13322
- [x] tests passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Fixed bug in pytables.py where series with categories were not being filtered properly. 


Labels:
name: Bug
name: IO HDF5
description: read_hdf, HDFStore

Comments:
always start with the tests

Sorry Jeff, I'm not sure what you mean. Are you saying I should add a unit test for this?

ANY change requires a test.

at a minimum, tests that reproduce the original issue. You FIRST right the tests, make sure they replicate the failure, THEN write code to fix. You have fixed it when your tests then pass and nothing else breaks.

Thanks for filling me in Jeff. Sorry for having to spell it out for me, but this is the first non-documentation PR I've submitted for pandas. I couldn't find any tests related to pytables but it seems like tests/frame/test_misc_api might be a good spot. Any insight would be appreciated. Thanks. 

This is the code I wrote to test the fix before I submitted the pull:

``` python
obsids = ['ESP_012345_6789', 'ESP_987654_3210']
imgids = ['APF00006np', 'APF0001imm']
data = [4.3, 9.8]

df = pd.DataFrame(dict(obsids=obsids, imgids=imgids, data=data))
df.to_hdf('testdf_no_cats.hdf', 'df',format='t', data_columns=True)

df.obsids = df.obsids.astype('category')
df.imgids = df.imgids.astype('category')
df.to_hdf('testdf_with_cats.hdf', 'df',format='t', data_columns=True)

# df without categories
df2 = pd.read_hdf('testdf_no_cats.hdf', 'df', where='obsids=B')
assert(len(df2) == 0)

# df with categories
df3 = pd.read_hdf('testdf_with_cats.hdf', 'df', where='obsids=B')
assert(len(df3) == 0)
```

https://github.com/pydata/pandas/blob/master/pandas/io/tests/test_pytables.py

Thanks Jeff, I completely missed that, I incorrectly assumed everything was in the one tests folder. Just submitted a new commit after confirming the test passed with my changes.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13792?src=pr) is 85.23% (diff: 100%)

> Merging [#13792](https://codecov.io/gh/pydata/pandas/pull/13792?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13792   diff @@
==========================================
  Files           140        140          
  Lines         50420      50422     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42975      42977     +2   
  Misses         7445       7445          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [cc216ad...a461208](https://codecov.io/gh/pydata/pandas/compare/cc216ada82a2b06a52524da7b1559a7f5e31b1c1...a4612085e40dfded55cf00a05b01f789a8bcf784?src=pr)

pls add a whatsnew in bug fix section.

@jreback Thanks for all of the feedback. I really appreciate you showing me how to do things the right way. I made the changes you requested and rebased the commit. 

thanks @shawnheide
ENH: add Index.dropna
State: closed

- [x] closes #6194
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

after this PR:

```
idx = pd.Index([1, 2, np.nan, 4])
idx.dropna()
# Float64Index([1.0, 2.0, 4.0], dtype='float64')
```

MI has option `how` to drop if any/all levels are NA (the same as `DataFrame`)

```
midx = pd.MultiIndex.from_arrays([[1, 2, np.nan, 4], [1, 2, np.nan, np.nan]])
midx.dropna()
# MultiIndex(levels=[[1, 2, 4], [1, 2]],
#            labels=[[0, 1], [0, 1]])

midx.dropna(how='all')
# MultiIndex(levels=[[1, 2, 4], [1, 2]],
#            labels=[[0, 1, 2], [0, 1, -1]])
```


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13791?src=pr) is 85.24% (diff: 100%)

> Merging [#13791](https://codecov.io/gh/pydata/pandas/pull/13791?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **0.01%**

``` diff
@@             master     #13791   diff @@
==========================================
  Files           140        140          
  Lines         50420      50438    +18   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42975      42998    +23   
+ Misses         7445       7440     -5   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [cc216ad...1672f26](https://codecov.io/gh/pydata/pandas/compare/cc216ada82a2b06a52524da7b1559a7f5e31b1c1...1672f26919cd08bcc08b267723d965b4b47c321f?src=pr)

Two minor comments, looks good to me for the rest!

Thanks,  made changes for that were pointed out. Added `New Index method` section under `what's new -> enhancement` to organize relevant fixes.

thanks @sinhrks
DOC: Added note to io.rst regarding reading in mixed dtypes
State: closed

- [x] closes #13746 
- [ ] tests added / passed
- [ ] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry

This is my first time contributing to the docs, I will gladly take any criticism.

Thanks


Labels:
name: Docs

Comments:
@jreback Updated to ipython blocks. I accidentally amended my commit instead of rebasing, but it looks like it doesn't really matter.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13782?src=pr) is 85.23% (diff: 100%)

> Merging [#13782](https://codecov.io/gh/pydata/pandas/pull/13782?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13782   diff @@
==========================================
  Files           140        140          
  Lines         50415      50415          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42971      42971          
  Misses         7444       7444          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [a3cddfa...7400607](https://codecov.io/gh/pydata/pandas/compare/a3cddfa9c4302c13747509caef7a056159010adf...7400607fa5d9ccef19a826d4581c65d393b4e237?src=pr)

> I noticed that there are two Travis CI builds for your commits, so you might want to rebase next time if you can

I don't think that does matter, if you rebase (or amend, which is in effect quite the same: you change the commit) and force push again, you will also get an additional travis build.

>  accidentally amended my commit instead of rebasing, but it looks like it doesn't really matter.

That does indeed not matter. Amending is perfectly fine (making a separate commit and pushing that is also fine), rebasing is only strictly necessary if there are merge conflicts to solve (or to incorporate changes from master, but that is not essential here)

@wcwagner I added some comments, but it is a really nice addition to the docs!

@jorisvandenbossche Thanks for the help!

@wcwagner looks really good. just some more links and think will be good.

I would also add a note in whatsnew (maybe enhancements) with a link to the new documentation (eg saying that its updated).

thanks @wcwagner 
great PR!

check out http://pandas-docs.github.io/pandas-docs-travis/ in a while to see the built docs. If things don't look exactly right, pls issue a followup PR.