BrandonFors/pandas-github-issues huggingface dataset:
BUG: Fix justification for DataFrame.to_string(index=False)
State: closed

I believe this fixes #13032.  I'm a new contributor, so please let me know if I've missed any steps.

Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
did the soln in #14196 not work?

## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14569?src=pr) is 85.27% (diff: 100%)

> Merging [#14569](https://codecov.io/gh/pandas-dev/pandas/pull/14569?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #14569   diff @@
==========================================
  Files           140        140          
  Lines         50693      50691     -2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43230      43227     -3   
- Misses         7463       7464     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [7a2bcb6...278c700](https://codecov.io/gh/pandas-dev/pandas/compare/7a2bcb6605bacea858ec14cfac424898deb568b3...278c7003e520c8ccf226f31fb2a2c479f913f371?src=pr)

I tried to make some updates to that other PR over at [this branch](https://github.com/buggi22/pandas/tree/update_other_pull_request), but there are a large number of test failures due to the change in functionality.  I haven't had a chance to figure out which failures are simply due to out-of-date test cases, as opposed to newly-introduced bugs.

On another note, I realized that my changes in this PR should only apply to cases where the first column is numeric.  I've added some new commits to this branch to try to address this, but unfortunately this seems significantly more fragile.  I can see why a more comprehensive fix like PR #14196 would be preferable.

@buggi22 can you rebase / update?
closing as stale
DEPR: add deprecation warning for com.array_equivalent
State: closed

xref https://github.com/pandas-dev/pandas/issues/14555

pandas.core.common.array_equivalent was removed without deprecation warning.
This commits adds it back to the core.common namespace with deprecation warning

We should still decide on the usefulness to add to the public API (in this commit I didn't add any pointer to alternative import location)

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14567?src=pr) is 85.27% (diff: 100%)

> Merging [#14567](https://codecov.io/gh/pandas-dev/pandas/pull/14567?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #14567   diff @@
==========================================
  Files           140        140          
  Lines         50689      50693     +4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43225      43229     +4   
  Misses         7464       7464          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [1d95179...d5e5173](https://codecov.io/gh/pandas-dev/pandas/compare/1d951794187b72841d59b3dfde0a98309a64dec7...d5e51735853ef58a3514431ad960c9f9c69058ec?src=pr)

lgtm. I would add a note in the release notes as well.
PERF: casting loc to labels dtype before searchsorted
State: closed

Intrigued by the profiling results of the below example (Multi-index `loc` indexing, based on the example in https://github.com/pandas-dev/pandas/issues/14549), where `searchsorted` seemed to take the majority of the computation time. 
And it seems that `searchsorted` casts both inputs (in this case `labels` and `loc`) to a common dtype, and the `labels` of the MultiIndex were in this case `int16`, while `loc` (output from `Index.get_loc`) is a python int.

By casting `loc` to the dtype of `labels`, the specific example gets a ca 20 x speed improvement

```
df = pd.DataFrame({'a': np.random.randn(500*5000)}, index=pd.MultiIndex.from_product([date_range("2014-01-01", periods=500), range(5000)]))
dt = pd.Timestamp('2015-01-01')
%timeit df.loc[dt]
```

On master:
```
In [3]: %timeit df.loc[dt]
The slowest run took 5.70 times longer than the fastest. This could mean that an intermediate result is being cached.
100 loops, best of 3: 9.39 ms per loop
```

with this PR:
```
In [3]: %timeit df.loc[dt]
The slowest run took 122.51 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 3: 422 µs per loop
```

Just putting it here as a showcase. 
Actual PR probably needs some more work (other places where this can be done, can loc ever be out of bound for that dtype?, benchmarks, ..)


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Performance
description: Memory or execution speed performance

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14551?src=pr) is 85.26% (diff: 100%)

> Merging [#14551](https://codecov.io/gh/pandas-dev/pandas/pull/14551?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #14551   diff @@
==========================================
  Files           140        140          
  Lines         50672      50676     +4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43207      43211     +4   
  Misses         7465       7465          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [60a335e...6447e4c](https://codecov.io/gh/pandas-dev/pandas/compare/60a335e457bc40d009d5be99b384a62fa34ba3fc...6447e4c6c561fab2b249f2323dc368c0da13c97a?src=pr)

lgtm. just release note. I think this can only help :>

OK, added whatsnew notice. Will merge, but will open a new issue for follow-up on this (other cases, benchmarks that capture this (at the moment there are none)).
BUG: fix categorical searchsorted bug (#14522)
State: closed

 - [ ] closes #14522 
 - [ ] tests added / passed
 - [ ] passes ``git diff upstream/master | flake8 --diff``
 - [ ] whatsnew entry

the bug is cause by `x.categories.values.searchsorted`, where the search should not be performed.

Labels:
name: Bug
name: Categorical
description: Categorical Data Type

Comments:
@chiangqiqi Can you add tests for this?

I think I miss understood the meaning, will try to fix this soon.
DOC: Simplify the gbq integration testing procedure for contributors
State: closed

I've updated the contributing documentation to simplify the steps required to run Google BigQuery integration tests on a forked repository of pandas.


Labels:
name: Docs
name: CI
description: Continuous Integration

Comments:
I followed the updated instructions for GBQ integration testing. All gbq tests passed. Here is a link to the Travis build:
https://travis-ci.org/parthea/pandas/builds/171878521

thanks!
COMPAT: Updated method call for _Openpyxl22Write.write_cells() for Openpyxl =>2.4.0 #14519
State: closed

- [X] closes #14519
- [X] tests added / passed
- [X] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry

Updated _Openpyxl22Writer.write_cells() to use new coordinate call method wks['coordinate']
instead of wks.cell() in accordance with https://openpyxl.readthedocs.io/en/default/tutorial.html#accessing-one-cell
in the newest release of Openpyxl (2.4.0 and later).

I did not know how to write a true test for this fix since it involves Excel file I/O and pandas does not currently supporting reading in .xlsx as far as I know. The best I could do is use it with openpyxl 2.4.0 install on my machine and passing a small dataframe into an .xlsx. Please let me know if you have any suggestions on how to better test this. 


Labels:
name: IO Excel
description: read_excel, to_excel
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
@jorisvandenbossche if you could point me to where you think the UserWarning might be stemming from in pandas codebase then I'd be more than happy to take a look at it. I just went the most obvious edit that I could find and only that used this wks.cell(coordinate) method call. If you know of any other sources for the issue please let me know.

@ditenberg for finding the UserWarning, you could try sticking

```
import warnings
warnings.simplefilter('error', UserWarning)
```

at the top of the excel file tests. That would turn any UserWarnings into exceptions. Glancing through the test file, it looks like we use `.cell` in a few places: https://github.com/pandas-dev/pandas/blob/7f5a45c1b388b3f7f309f82bfa0733b7b9980c3a/pandas/io/tests/test_excel.py#L1970

@TomAugspurger I put your suggested code into the pandas/io/tests/test_excel.py file and changed the cell value retrieval syntax in your mentioned location and a few others throughout the file. After running `nosetests pandas/io/tests/test_excel.py` I got back a clean report with no errors or UserWarnings.

@TomAugspurger There are deeper issues with the _OpenpyxlXXWriter since they are deemed incompatible in the test_excel.py comments to be skipped during tests. Regardless, with commits the UserWarning no longer appears after test.

@ditenberg can you rebase / update.

this needs to be back compat (easy enough, you simply have a version check on the import and branch based on that).
@ditenberg can you update this?
closing as stale, but would like you to update if possible.
DOC: add sep argument to read_clipboard signature
State: closed

- [x] closes #13726



Labels:
name: Docs
name: IO Data
description: IO issues that don't fit into a more specific label

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14537?src=pr) is 85.26% (diff: 100%)

> Merging [#14537](https://codecov.io/gh/pandas-dev/pandas/pull/14537?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #14537   diff @@
==========================================
  Files           140        140          
  Lines         50693      50672    -21   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43229      43208    -21   
  Misses         7464       7464          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [e1cdc4b...18c30eb](https://codecov.io/gh/pandas-dev/pandas/compare/e1cdc4b83d7e970e9683056be722d5fa2a00fa70...18c30ebe0169e8e52d8da60a3da0d44bdd6d6167?src=pr)

I would not list `delim_whitespace` specifically, as this is actually a equivalent for `sep='\s+'` (that is the reason it is checked for in the code, as it is an alternative way to specify the `sep`).

So would only list `sep` of the two, and explain clearly what the default `sep` values means (you can even say it is equivalent to `delim_whitespace=True`).

@jorisvandenbossche Thanks for your review. While that makes sense, as issue https://github.com/pandas-dev/pandas/issues/13726 mentions, with both the arguments added, it'll be helpful for argument completion in interactive use. Thoughts?

> with both the arguments added, it'll be helpful for argument completion in interactive use

The point I was trying to make is that in the current way they are specified, having both arguments is not useful: `sep='\s+'` is the default, and setting `delim_whitespace` to True is equivalent with this, so in practice you never need this. 

Towards the users, I would maybe rather make `delim_whitespace=True` the default, as that is easier to understand what it means compared to `sep='\s+'`. But that is maybe a bit more complex to implement. 
So to conclude, I would just document `sep`, and leave `delim_whitespace` out of the signature.

@jorisvandenbossche ah, got it. Will remove `delim_whitespace` from the signature.

@TomAugspurger Can this be merged?

Looks like the build fails due to this? https://github.com/pypa/pip/issues/4059

@keshavramaswamy Thanks a lot! (I made a small adjustment before merging, and made the docstring a raw string, which is needed for sphinx to interpret the `\` literally)
BUG: DataFrame.quantile with NaNs (GH14357)
State: closed

Closes #14357

I first used `nanpercentile` when available (np >= 1.9), but this only can handle numeric data, not datetimes. So therefore included a custom `_nanpercentile`

I also added some tests regarding empty dataframes/series in comments, as these are still failing (but are also failing on 0.19.0)


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14536?src=pr) is 85.26% (diff: 86.95%)

> Merging [#14536](https://codecov.io/gh/pandas-dev/pandas/pull/14536?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #14536   diff @@
==========================================
  Files           140        140          
  Lines         50672      50685    +13   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43208      43219    +11   
- Misses         7464       7466     +2   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [7f5a45c...cdd247b](https://codecov.io/gh/pandas-dev/pandas/compare/7f5a45c1b388b3f7f309f82bfa0733b7b9980c3a...cdd247b49d5a99f148f2b9cd45af270d17d21c7f?src=pr)
BUG: Index.insert may not coerced to correct dtype
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry
### on current master

```
# NG, must be float dtype
pd.Index([1]).insert(1, 1.0)
# Int64Index([1, 1], dtype='int64')

# NG, must be object dtype
pd.DatetimeIndex(['2011-01-01']).insert(1, 1)
# TypeError: cannot insert DatetimeIndex with incompatible label
```
## after the PR

```
pd.Index([1]).insert(1, 1.0)
# Float64Index([1.0, 1.0], dtype='float64')

pd.DatetimeIndex(['2011-01-01']).insert(1, 1)
# Index([2011-01-01 00:00:00, 1], dtype='object')
```


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14535?src=pr) is 85.25% (diff: 94.23%)

> Merging [#14535](https://codecov.io/gh/pandas-dev/pandas/pull/14535?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **0.04%**

``` diff
@@             master     #14535   diff @@
==========================================
  Files           143        140     -3   
  Lines         50787      50666   -121   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43273      43194    -79   
+ Misses         7514       7472    -42   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [f26b049...6c0ab48](https://codecov.io/gh/pandas-dev/pandas/compare/f26b049786624ed983f2718687c23e3f1adbb670...6c0ab489f8605fd130a2c41b813d1e6218e7a82d?src=pr)

I would keep this for 0.20, as it are some slight changes without a pressing bug fix (unless you have an example case?)

Further, are we sure that we want to upcast the Int64Index to floats, even when the value set is coerceable to a integer? 
Using another (series and indexing, not index and `insert`), but similar case:

```
In [145]: s = pd.Series([2,2,3,4])

In [146]: s[2] = 5.0

In [147]: s
Out[147]: 
0    2
1    2
2    5
3    4
dtype: int64
```

Here the integer dtype is preserved.

can you update?
@sinhrks status?
can you rebase / update
issue now in #16277
BUG: pivot_table sometimes returns Series (#4386)
State: closed

- [x] closes #4386
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

The [`pivot_table` docs](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.pivot_table.html) mentions that a `DataFrame` should be returned. In some cases, a `Series` is returned. I had to also alter some existing tests that expected `pivot_table` to return a `Series`

Any feedback welcomed, thanks!


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14534?src=pr) is 85.28% (diff: 100%)

> Merging [#14534](https://codecov.io/gh/pandas-dev/pandas/pull/14534?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #14534   diff @@
==========================================
  Files           140        140          
  Lines         50693      50695     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43235      43236     +1   
- Misses         7458       7459     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [726efc7...02ef0a8](https://codecov.io/gh/pandas-dev/pandas/compare/726efc7cd55744eb88636dd6f11293f18355a10a...02ef0a8108b04019e8d9ed64855a147677cca1ea?src=pr)

Hi @jreback:

I added a fix to address the original bug. However, I believe `to_frame()` will be necessary because two scenarios ([pivoting with only the columns argument](https://github.com/pandas-dev/pandas/blob/master/pandas/tools/tests/test_pivot.py#L351) and [pivoting with no values or cols passed](https://github.com/pandas-dev/pandas/blob/master/pandas/tools/tests/test_pivot.py#L486)) will always return `Series` from the `groupby` and `unstack` operations respectively. 

this looks like its duplicating #13554. Can you see if you tests are any different (maybe there is another case).

Ah yes, I wanted to move #13554 along since it looked inactive for a while (but it looks active now). I can look into adding more tests and building upon #13554. 

can you rebase
Closing since this PR duplicated #13554 when it was inactive. Will reopen if #13554 stalls.
BUG: CategoricalDtype could not be sorted
State: closed

- [x] closes #14368
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14533?src=pr) is 85.27% (diff: 100%)

> No coverage report found for **master** at 096d886.
> 
> Powered by [Codecov](https://codecov.io?src=pr). Last update [096d886...6d5d80e](https://codecov.io/gh/pandas-dev/pandas/compare/096d8866a90c8cbb44ab8243320e811fc24190bd...6d5d80ef4d71a382319fd837da526143e5554367?src=pr)

I don't see a way forward here, so I'm going to close to avoid polluting the PR list. See full discussion on https://github.com/pandas-dev/pandas/issues/14368. 

thanks for the effort @bkandel
ENH: merge_asof() has left_index/right_index and left_by/right_by (#14253)
State: closed

- [x] closes #14253 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: API Design

Comments:
I'll be the first to admit that there is some kludge in here, identified by comments in the source code. There are some code paths that can only be hit during a `pd.merge_asof()` when `left_index` or `right_index` is set, so no existing functionality will be affected.

I think for pandas 2.0 we'll need to take a [more holistic view of how to do merges](https://github.com/pandas-dev/pandas2/issues/31).

## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14531?src=pr) is 85.20% (diff: 87.23%)


> No coverage report found for **master** at 3552dc0.

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3552dc0...acad843](https://codecov.io/gh/pandas-dev/pandas/compare/3552dc0c4533a5eafafe859f5afd29a7ce063e03...acad843dcdf717eea2418a22390d36f84398f3a7?src=pr)
Hmm, AppVeyor got a syntax error for a line that isn't mine:

```
from pip._vendor. import string_types
```

Seems like everyone is failing with this. Filed issue #14603.

@jreback Would this change be more appropriate for 0.19.2 or 0.20.0?

> Would this change be more appropriate for 0.19.2 or 0.20.0?

There is a tiny API change (in that some of the positional args to `pd.merge_asof` changed) because adding new ones, but think that is ok as we introduced this in 0.19.0, though to be conservative we can defer to 0.20.0

@jorisvandenbossche 

@jreback @jorisvandenbossche It's all green finally.
@chrisaycock Thanks!

> think that is ok as we introduced this in 0.19.0, though to be conservative we can defer to 0.20.0

I personally have a slight preference to keep this for 0.20.0, but as this was only introduced in 0.19.0, it's not a breakpoint for me
GH #14499 Panel.ffill ignores axis parameter and fill along axis=1
State: closed

- [x] closes #14499, closes #8251
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
handle case where `axis=0`, tests added for every case and passed, whatsnew added

## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14528?src=pr) is 84.65% (diff: 100%)
> Merging [#14528](https://codecov.io/gh/pandas-dev/pandas/pull/14528?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14528   diff @@
==========================================
  Files           144        144          
  Lines         51030      51039     +9   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43198      43208    +10   
+ Misses         7832       7831     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [8b497e4...5fdf296](https://codecov.io/gh/pandas-dev/pandas/compare/8b497e49387ad167acb1c8ae31d67676e30add5e...5fdf2960db1efb8f41ba0df9630038e6cf27f847?src=pr)
@jreback you're okay with this pr ?

@Tux1 can you rebase / update
Done

> On 22 Dec 2016, at 07:09, Jeff Reback <notifications@github.com> wrote:
> 
> @Tux1 <https://github.com/Tux1> can you rebase / update
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub <https://github.com/pandas-dev/pandas/pull/14528#issuecomment-268668173>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGWku4E3Ck1ncZf0bkgZaE4sGggfblb0ks5rKbGpgaJpZM4KjmbC>.
> 


pls comment if youd like to update / rebase and continue