BrandonFors/pandas-github-issues huggingface dataset:
ENH: Accept callable for skiprows in read_csv
State: closed

 Title is self-explanatory.

xref #10882.

Labels:
name: Enhancement
name: IO CSV
description: read_csv, to_csv

Comments:
Right now, I have added `[ci skip]` because I cannot get the C engine implementation to work yet.  Currently, if you pass in a `callable`, I get a segfault.  However, I'm not sure how to fix it.  Suggestions / help on what to do?  Is it some reference thing that I am missing?
One possibility - is the GIL released when you're trying to call that function in the C code?
@chris-b1 : Can't check at the moment, but could you explain why that would cause a segfault?
To be honest, not sure if/how it would cause a segfault, but the general rule is you can't do anything involving a `PyObject` without the GIL.
https://docs.python.org/2/c-api/init.html#thread-state-and-the-global-interpreter-lock

Might be of interest or relevance: <a href="https://github.com/scipy/scipy/pull/6509">scipy/scipy #6509</a>
@chris-b1 : You were right!  I needed to ensure that we were in a state to call `PyObject_CallFunction`.  The C engine is now happy.  PR should be ready to be reviewed soon.
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15059?src=pr) is 84.75% (diff: 100%)
> Merging [#15059](https://codecov.io/gh/pandas-dev/pandas/pull/15059?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #15059   diff @@
==========================================
  Files           145        145          
  Lines         51220      51238    +18   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43414      43426    +12   
- Misses         7806       7812     +6   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [b8e7c34...d15e3a3](https://codecov.io/gh/pandas-dev/pandas/compare/b8e7c340fcf08b03ffdc8891fdc5ce7510d825b2...d15e3a3317d2406830f058abcd1a1c8be54ea559?src=pr)
@jreback @jorisvandenbossche @chris-b1 : Thoughts on this?
@jreback : @jorisvandenbossche and @chris-b1 have approved these changes.  I'm not sure if they're waiting for you to have a look, but if there are no other concerns, this should be good to merge I think.
give a perf test on this, e.g. when the skiprows is NOT a callable perf should be roughly unchanged.
@jreback : Ran benchmarks, and no major change in performance when using `skiprows`.
thanks!
DOC: Add example of skipcols in read_csv
State: closed

Illustrate how we can use the `usecols` argument to skip particular columns.

Closes #10882.

Labels:
name: Docs
name: IO CSV
description: read_csv, to_csv

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15052?src=pr) is 84.75% (diff: 100%)
> Merging [#15052](https://codecov.io/gh/pandas-dev/pandas/pull/15052?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #15052   diff @@
==========================================
  Files           145        145          
  Lines         51146      51146          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43351      43351          
  Misses         7795       7795          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [018414e...ea3279f](https://codecov.io/gh/pandas-dev/pandas/compare/018414e704670d2d727b9624089caae6f7d12ff1...ea3279fef0d35475f4b256a0745a0b8e38bfcfc2?src=pr)
@gfyoung Thanks!
TST: Add new test for flaky usecols
State: closed

Title is self-explanatory.

xref #14984.
Closes #13604.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO CSV
description: read_csv, to_csv

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15051?src=pr) is 84.75% (diff: 100%)
> Merging [#15051](https://codecov.io/gh/pandas-dev/pandas/pull/15051?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #15051   diff @@
==========================================
  Files           145        145          
  Lines         51146      51146          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43350      43350          
  Misses         7796       7796          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [6eb705f...f59565c](https://codecov.io/gh/pandas-dev/pandas/compare/6eb705f5350a0eeffd9e7663d7b4bf54acb4d6af...f59565c3daaf2e34d1b4e6d6e17bbbaf998678ca?src=pr)
ERR: Disallow multi-char quotechar for C engine
State: closed

Raise `ValueError` or issue `ParserWarning` (as we do with other unsupported features) when a multi-char `quotechar` is passed in, and the C engine is used.

Closes #11592.

Labels:
name: Bug
name: IO CSV
description: read_csv, to_csv

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15050?src=pr) is 84.76% (diff: 100%)
> Merging [#15050](https://codecov.io/gh/pandas-dev/pandas/pull/15050?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #15050   diff @@
==========================================
  Files           145        145          
  Lines         51146      51151     +5   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43350      43356     +6   
+ Misses         7796       7795     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [de09c98...68d69fe](https://codecov.io/gh/pandas-dev/pandas/compare/de09c988dddb519ac59fde1556a6d9d35f43e702...68d69fe9b3c1bdf7715b395d3ac84d9bb25b385d?src=pr)
@jreback : Any comments on this PR?  Otherwise, everything is green and ready to go.
@sinhrks : Addressed all of your comments, and everything is still green.
BF: define expected Series of correct for arch (eg i386) int
State: closed

 - [x] Closes  #14866
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry (I believe it is nothing new ;) )


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15044?src=pr) is 84.75% (diff: 100%)
> Merging [#15044](https://codecov.io/gh/pandas-dev/pandas/pull/15044?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #15044   diff @@
==========================================
  Files           145        145          
  Lines         51146      51146          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43351      43350     -1   
- Misses         7795       7796     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [4de5cdc...11dccee](https://codecov.io/gh/pandas-dev/pandas/compare/4de5cdcb900ad365d7ed5352106a47d86156ea17...11dccee05d3bfee2fceae109940b9944525ab04f?src=pr)
can you rebase / update
this was closed by / superseded https://github.com/pandas-dev/pandas/pull/15766
TST: Test empty method and simplify logic
State: closed

 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``

Since the docstring of `.empty` states

>  True if NDFrame is entirely empty [no items], meaning any of the axes are of length 0.

I was wondering if the logic could be changed accordingly for code clarity. Unless it was set as `not all(len(self._get_axis(a)) > 0 for a in self._AXIS_ORDERS)` for a particular reason.

Also this method did not appear to have any test. 

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15043?src=pr) is 84.75% (diff: 100%)
> Merging [#15043](https://codecov.io/gh/pandas-dev/pandas/pull/15043?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #15043   diff @@
==========================================
  Files           145        145          
  Lines         51146      51146          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43350      43351     +1   
+ Misses         7796       7795     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [6eb705f...9a70034](https://codecov.io/gh/pandas-dev/pandas/compare/6eb705f5350a0eeffd9e7663d7b4bf54acb4d6af...9a7003439bfe21b807b25493d7c38f98414cc184?src=pr)
Tests moved and all green. 
@Thanks!
BUG: Indexing MultiIndex with Series failed.
State: closed

Previously, accessing elements of a MultiIndex-indexed DataFrame with a Series
failed. This changes that behavior so that it is possible to use a Series to
access elements from a MultiIndex-indexed DataFrame, just as one would use
a list.

 - [x] closes #14730
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry

Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15041?src=pr) is 84.75% (diff: 100%)
> Merging [#15041](https://codecov.io/gh/pandas-dev/pandas/pull/15041?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **0.01%**

```diff
@@             master     #15041   diff @@
==========================================
  Files           145        145          
  Lines         51129      51141    +12   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43343      43345     +2   
- Misses         7786       7796    +10   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [74e20a0...3698087](https://codecov.io/gh/pandas-dev/pandas/compare/74e20a0c4146919a82312c1027dcb57b48b613e3...369808734faf9ce542863121cd01320225095c05?src=pr)
thanks @bkandel (I moved the tests to the appropriate location)
DOC: Typing error in ExcelFile class example
State: closed

 - [x] closes #15037
 - [ ] tests added / passed
 - [ ] passes ``git diff upstream/master | flake8 --diff``
 - [ ] whatsnew entry

- Tests with local build of documentation passed
- Flake8 seems to be relevant for source code changes, only
- Thought that a whatsnew entry is not relevant in this case

Labels:
name: Docs
name: IO Excel
description: read_excel, to_excel

Comments:
thanks!
[WIP]: Add quantile shortcut
State: closed

 - [x] closes #15023 
 - [ ] tests added / passed
 - [ ] passes ``git diff upstream/master | flake8 --diff``
 - [ ] whatsnew entry


Labels:
name: API Design
name: Resample
description: resample method

Comments:
tests!
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15032?src=pr) is 84.76% (diff: 33.33%)
> Merging [#15032](https://codecov.io/gh/pandas-dev/pandas/pull/15032?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #15032   diff @@
==========================================
  Files           145        145          
  Lines         51131      51138     +7   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43346      43347     +1   
- Misses         7785       7791     +6   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [6141754...a3b713b](https://codecov.io/gh/pandas-dev/pandas/compare/6141754e3ef0574fda35a49925aa629655fd16a0...a3b713beffd1d78b7a3b80c3275410654cce30ca?src=pr)
can you rebase / update
if you'd like to add tests and finish this up, pls ping.
Ping @srinivasreddy. Would have used this :)
COMPAT: py2.7 compat for Timestamp.replace
State: closed

compat in Timestamp.replace when passing longs (and not ints)

xref https://github.com/statsmodels/statsmodels/issues/3349

Labels:
name: Datetime
description: Datetime data dtype
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15030?src=pr) is 84.77% (diff: 100%)
> Merging [#15030](https://codecov.io/gh/pandas-dev/pandas/pull/15030?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #15030   diff @@
==========================================
  Files           145        145          
  Lines         51131      51131          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43345      43345          
  Misses         7786       7786          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3662413...380ace1](https://codecov.io/gh/pandas-dev/pandas/compare/366241386282272328333256b6e267a68f8133a4...380ace170de3672bd0f7e26678ec227ae406a772?src=pr)
API: allow list-like to DataFrame rename
State: closed

 - [x] closes #14829
 - [ ] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: API Design

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15029?src=pr) is 84.77% (diff: 100%)
> Merging [#15029](https://codecov.io/gh/pandas-dev/pandas/pull/15029?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #15029   diff @@
==========================================
  Files           145        145          
  Lines         51131      51133     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43345      43348     +3   
+ Misses         7786       7785     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3662413...5e3848c](https://codecov.io/gh/pandas-dev/pandas/compare/366241386282272328333256b6e267a68f8133a4...5e3848c23bb42b6377239a83c5db3663e5721f5a?src=pr)
this is also now an alternative to the ``.index =`` API (which is fine I think). So can you put some references in the doc-string and update the docs (where .set_index is).
I added a couple doc references, but I might be missing something - wasn't sure if there is a canonical spot for the `.index/.columns=` api?
http://pandas.pydata.org/pandas-docs/stable/indexing.html#set-reset-index

ad-hoc indexing. This would be expanded in any event (and *maybe* moved to http://pandas.pydata.org/pandas-docs/stable/basics.html#renaming-mapping-labels) as these are the same idea.
Closing in favor of #15104
DOC: Added examples to pandas.concat documentation
State: closed

I've added two simple examples to the ``pd.concat`` documentation. I've also done a very small amount of clean up on the rest of the docstring.

 - [x] passes ``git diff upstream/master | flake8 --diff``

Labels:
name: Docs

Comments:
That's fine by me. I'll add a link to that walkthrough and a wider array of
examples. I'll aim for the new examples to be more discrete and less
holistic than the walkthrough.

On Sun, Jan 1, 2017, 10:46 AM Jeff Reback <notifications@github.com> wrote:

> *@jreback* commented on this pull request.
> ------------------------------
>
> In pandas/tools/merge.py
> <https://github.com/pandas-dev/pandas/pull/15028#pullrequestreview-14816434>
> :
>
> >      Returns
>      -------
>      concatenated : type of objects
> +
> +    Notes
> +    -----
>
> if you are going to add examples, then this needs to be more complete,
> using examples from here:
> http://pandas.pydata.org/pandas-docs/stable/merging.html#concatenating-objects
>
> certainly not all of these, but a quick selection (as well as having this
> link would be fine).
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/15028#pullrequestreview-14816434>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAAnCTODtT334eXdmY1T9r-nH2TZjBXOks5rN_SGgaJpZM4LYpwL>
> .
>

## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15028?src=pr) is 84.75% (diff: 100%)
> Merging [#15028](https://codecov.io/gh/pandas-dev/pandas/pull/15028?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **0.01%**

```diff
@@             master     #15028   diff @@
==========================================
  Files           145        145          
  Lines         51131      51146    +15   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43345      43350     +5   
- Misses         7786       7796    +10   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3662413...c58f9f8](https://codecov.io/gh/pandas-dev/pandas/compare/366241386282272328333256b6e267a68f8133a4...c58f9f8c40ed525f05de2cb124ab7e5f1dbe7b9d?src=pr)
@jorisvandenbossche and @jreback, I believe I've made the changes you've asked for. Please let me know if there's anything more I can do. 
@jorisvandenbossche I believe I have made the changes you've requested. Please let me know if there's more I can do. 
@palewire Thanks!
docs cheat sheet pandas Pd to pd
State: closed

Change pandas syntax from Pd to pd

Closes [issue 15017](https://github.com/pandas-dev/pandas/issues/15017#issuecomment-269871137)

Labels:
name: Docs

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15026?src=pr) is 84.77% (diff: 100%)
> Merging [#15026](https://codecov.io/gh/pandas-dev/pandas/pull/15026?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #15026   diff @@
==========================================
  Files           145        145          
  Lines         51131      51131          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43345      43345          
  Misses         7786       7786          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3662413...3cca5d2](https://codecov.io/gh/pandas-dev/pandas/compare/366241386282272328333256b6e267a68f8133a4...3cca5d28c5bebdb0e86e63af8423e4a119e9f5b9?src=pr)
thanks!
BUG: Patch float and uint handling in to_numeric
State: closed

1) Patches `float` handling by reducing the "closeness" level when checking conversions.
2) Patches `uint` handling by allowing casts to `uint` dtypes of equal or lesser size to `int64` (when values are less than `INT64_MAX`

Closes #14941.
Follow-up to #15005.

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15024?src=pr) is 84.77% (diff: 100%)
> Merging [#15024](https://codecov.io/gh/pandas-dev/pandas/pull/15024?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #15024   diff @@
==========================================
  Files           145        145          
  Lines         51123      51123          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43337      43337          
  Misses         7786       7786          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [0252385...9e35819](https://codecov.io/gh/pandas-dev/pandas/compare/0252385a71f8b8738c3223dcc44af001baa79b10...9e35819b8efaf6e4f66579f9285c33645ab5ade7?src=pr)
thanks!
BUG: Parse uint64 in read_csv
State: closed

Adds behavior to allow for parsing of `uint64` data in `read_csv`. Also ensures
that they are properly handled along with `NaN` and negative values.

Closes #14983.

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: IO CSV
description: read_csv, to_csv

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/15020?src=pr) is 84.77% (diff: 100%)
> Merging [#15020](https://codecov.io/gh/pandas-dev/pandas/pull/15020?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #15020   diff @@
==========================================
  Files           145        145          
  Lines         51131      51131          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43345      43345          
  Misses         7786       7786          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3662413...d018e38](https://codecov.io/gh/pandas-dev/pandas/compare/366241386282272328333256b6e267a68f8133a4...d018e386034e1c6bc04227cf5c5857e871ffde42?src=pr)
pls run a perf check (*maybe* need some additonal asvs)
@jreback : Added one benchmark, and performance does not seem impacted significantly (if at all).
@jreback : Any other comments about this PR?  Otherwise, seems ready to merge.
@gfyoung haven't looked yet. soon.
thanks @gfyoung 

my comments are for future code generation.
@jreback : Yep, got it.  Thanks!