BrandonFors/pandas-github-issues huggingface dataset:
DOC: Fix additional join examples in "10 Minutes to pandas" #13029
State: closed

- [x] closes #13029 


Labels:
name: Docs

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13171?src=pr) is **84.12%**

> Merging [#13171](https://codecov.io/gh/pydata/pandas/pull/13171?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13171   diff @@
==========================================
  Files           138        138          
  Lines         50384      50384          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42382      42382          
  Misses         8002       8002          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [82f54bd...633c7ff](https://codecov.io/gh/pydata/pandas/compare/82f54bd1dd53cb031e5d801405b34f062155d823...633c7ff3d1d15b5759db94bf97593583f9f2a300)

thanks @Xndr7
BUG: preserve join keys dtype
State: closed

- closes #8596, preserve join keys dtype
- adds `Index.where` method for all Index types (like `np.where/Series.where`), but preserves dtypes


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
ok, removed the dep on my other PR.

@sinhrks @jorisvandenbossche @TomAugspurger comments?

@sinhrks any comments on this?
DOC: Fix delim_whitespace regex typo.
State: closed

Minor typo in the explanation of delim_whitespace which tripped up a user on SO (although the user should probably have been using `delim_whitespace=True` directly anyhow.)


Labels:
name: Docs

Comments:
@dsm054 I see that the same typo has been made in the read_csv docstring. Can you fix it there as well?

@jorisvandenbossche: added the read_csv fix and we're green.

Thanks!
BUG: Misc fixes for SparseSeries indexing with MI
State: closed

- [x] closes #13144
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Fixed following bugs. These makes `MultiIndex` repr work.

```
orig = pd.Series([1, np.nan, np.nan, 3, np.nan], index=idx)
sparse = orig.to_sparse()
sparse['A']    
# IndexError: Out of bounds access
```

```
# the result must have MultiIndex
sparse.loc['A':]
# (A, 0)    1.0
# (A, 1)    NaN
# (B, 0)    NaN
# (C, 0)    3.0
# (C, 1)    NaN
# dtype: float64
# BlockIndex
# Block locations: array([0, 3], dtype=int32)
# Block lengths: array([1, 1], dtype=int32)
```


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Sparse
description: Sparse Data Type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13163?src=pr) is **84.12%**

> Merging [#13163](https://codecov.io/gh/pydata/pandas/pull/13163?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **-<.01%**

``` diff
@@             master     #13163   diff @@
==========================================
  Files           138        138          
  Lines         50380      50380          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42380      42378     -2   
- Misses         8000       8002     +2   
  Partials          0          0          
```
1. File `...das/sparse/series.py` was modified. [more](https://codecov.io/gh/pydata/pandas/commit/eb241028936aa72e21d664c00a802a357c65d0fc/changes?src=pr#70616E6461732F7370617273652F7365726965732E7079) 
   - Misses `+1` 
   - Partials `0` 
   - Hits `-1`

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [4de83d2...eb24102](https://codecov.io/gh/pydata/pandas/compare/4de83d25d751d8ca102867b2d46a5547c01d7248...eb241028936aa72e21d664c00a802a357c65d0fc)

thanks @sinhrks
ENH/BUG: str.extractall doesn't support index
State: closed

- [x] closes #10008
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

```
# same output as pd.Series(["a1a2", "b1", "c1"]).str.extractall("[ab](?P<digit>\d)")
idx = pd.Index(["a1a2", "b1", "c1"])
idx.str.extractall("[ab](?P<digit>\d)")
#         digit
#   match      
#0 0         1
#   1         2
#1 0         1
```

**NOTE** Also fixed a bug `Series.str.extractall` raises `ValueError` if its `Index` is `str` with 2 or more characters.

```
s = pd.Series(["a1a2", "b1", "c1"], index=pd.Index(["xx", "yy", "zz"]))
s.str.extractall("[ab](?P<digit>\d)")
# ValueError: Length of names must match number of levels in MultiIndex.
```


Labels:
name: Bug
name: Enhancement
name: Strings
description: String extension data type and string data

Comments:
CC: @tdhock

thanks @sinhrks very nice!
Fine-grained errstate handling
State: closed

- [x] closes #13109 
- [x] closes #13135
- [x] tests passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

The precise strategy to be taken here is open for discussion. I tried to be reasonably fine-grained rather than slap a generic decorator over everything because it's easier to go that direction than the reverse. The `errstate()` blocks in the tests were added _after_ fixing all of the library code. Unfortunately, these are less fine-grained than I would like because some of the tests have many lines of the form `assert_array_equal(pandas_expression_to_test, expected_raw_numpy_expression)` where `expected_raw_numpy_expression` is what is triggering the warning. It was tedious to try to rewrite all of that to wrap just `expected_raw_numpy_expression`.

I think I got everything exercised by the test suite except for parts of the test suite that are skipped on my machine due to dependencies. We'll see how things go in the CI.

I haven't added any new tests yet. Could do if requested.


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
@rkern ohh that was easy 😄 

can we have a test which asserts the actual error state (is the default). Put in `pandas/tests/test_util.py`.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13145?src=pr) is 85.27% (diff: 100%)

> Merging [#13145](https://codecov.io/gh/pydata/pandas/pull/13145?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **0.01%**

``` diff
@@             master     #13145   diff @@
==========================================
  Files           139        139          
  Lines         50380      50439    +59   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42953      43012    +59   
  Misses         7427       7427          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [453bc26...ef9c001](https://codecov.io/gh/pydata/pandas/compare/453bc26e07ebbfea33dda9ff2f4efab683718506...ef9c00130c5353b72a19e4357b7926ab4a6a0290?src=pr)

Please excuse the CI churn. The jobs for the older commits could be cancelled.

no prob. thats an annoyance about Travis. 

@rkern I think we can now do this for 0.19.0

can you rebase / update?

@rkern can you rebase

@rkern, you now have another watcher on this... :-) Do you have an ETA when you could rebase?

@jreback Rebased.

this looks pretty good. just some minor comments. can you add in the whatsnew a sub-section, then add a warning (so its a big red box!) near the top.

can you add docs and rebase. ping when green. this touches lots of code so I will merge then.

@jreback rebased and docced.

looks fine. though failing on all sparse comparisons....pls ping when green

@jreback Fixed thinko. All green.

@rkern Thanks a lot!
COMPAT: boto import issues
State: closed


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
@TomAugspurger any progress on #11915 

for `boto` to release a broken version is really annoying.

@TomAugspurger I just moved this all to a better location which at worse case would only raise on actual s3 import, rather than at pandas import time. I think this will also make integrating via s3fs slightly easier.

lmk.

K, rebasing on those changes is `test_parser.py` right now. I think I was pretty close to having it working. Will put up a PR this morning.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13136?src=pr) is **84.14%**

> Merging [#13136](https://codecov.io/gh/pydata/pandas/pull/13136?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **+<.01%**

``` diff
@@             master     #13136   diff @@
==========================================
  Files           137        137          
  Lines         50287      50285     -2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42312      42311     -1   
+ Misses         7975       7974     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [43989fd...75e3ef1](https://codecov.io/gh/pydata/pandas/compare/43989fd7cb9917e885c2b55a172c4f9f3838d59d...75e3ef15e144a6cf2dd7dbe83a24311a96d7a081)

I have revised a bit. certainly ok with you making whatever changes, just want to get this out of the main import path.

@TomAugspurger any issue with this reorg? (I think your PR should easily go on top of this). hopefully can add more test coverage :)

Yeah, +1 on this I think.

@TomAugspurger gr8.
FIX: revert part of #11770
State: closed

Reverts part of:

https://github.com/pydata/pandas/pull/11770

Closes:

https://github.com/matplotlib/matplotlib/issues/6365

The mistake in #11770 was missing that pandas had a 1/us not 1/s
scaled bucket.


Labels:
name: Bug
name: Datetime
description: Datetime data dtype
name: Visualization
description: plotting

Comments:
@tacaswell any tests?

can you add a whatsnew note? (can be something as simple as compat for .....issue because of incorrect tick display with mpl < 2.0....)

Only took 4 force-pushes to get there.

@jorisvandenbossche Added back the minimal set of constants to compute what was needed.

What sort of test do you want for this?  You merged the last one with out a test :imp: 

can u create something that failed on the previous and now works? just maybe a check on axis ticks or something simple

thanks @tacaswell pls verify with mpl >= 2.0 when you have a chance.

did we have an issue where this came up on pnadas side? or was only on mpl side?

@TomAugspurger @jorisvandenbossche 

Only on the MPL tracker.

Sorry i took to long to get a test in.

On Fri, May 20, 2016, 10:37 Tom Augspurger notifications@github.com wrote:

> Only on the MPL tracker.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly or view it on GitHub
> https://github.com/pydata/pandas/pull/13131#issuecomment-220623417

np

ideally if u can post a couple of assertions we can add something to prevente future regressions
Make os.linesep default line_terminator in to_clipboard
State: closed

closes #11720
- Make os.linesep default line_terminator in to_clipboard
- Add line_terminator kw to Series.to_csv
- Refactor to_clipboard

Observed/fixed problems in to_clipboard:
- try..except: pass
- excel=None, though it defaults to True
- several code paths with return
- sep argument could be passed via kwargs
- "other keywords are passed to to_csv" only if excel=True
- line_terminator kw isn't supported by Series to_csv

Is it ok to to change Series.to_csv api?
Also we need tests, right? But what does `# pragma: no cover` mean?


Labels:
name: Windows
description: Windows OS
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
tests are in the obvious location: https://github.com/pydata/pandas/blob/master/pandas/io/tests/test_clipboard.py

pls add tests.

Have no time today. will add asap

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13125?src=pr) is 84.11% (diff: 80.00%)

> Merging [#13125](https://codecov.io/gh/pydata/pandas/pull/13125?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **1.12%**

``` diff
@@             master     #13125   diff @@
==========================================
  Files           140        138     -2   
  Lines         50563      50388   -175   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43103      42386   -717   
- Misses         7460       8002   +542   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [37f95ce...255895f](https://codecov.io/gh/pydata/pandas/compare/37f95cef85834207db0930e863341efb285e38a2...255895fe35afad2a249637c526fea92db1b40fac?src=pr)

- [x] compatibility fixes
- [x] tests

can you rebase / update?

I'll look into it on monday.

Rebased.

same comment as on the issue. https://github.com/pandas-dev/pandas/commit/4a1a330343f31fc8ca62b98a84978475968cba9c updated the vendored pyperclip.

is this PR still relevant?
closing, but if you want to update / respond we can reopen
BUG: Added checks for NaN in __call__ of EngFormatter
State: closed

- [x] closes #11981 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Closes #11981

Updated the v0.18.2 document

In the function `__call__` of EngFormatter check if decimal input is NaN.


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
once you have made the changes and pushed, ping on green.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13124?src=pr) is **84.14%**

> Merging [#13124](https://codecov.io/gh/pydata/pandas/pull/13124?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **-<.01%**

``` diff
@@             master     #13124   diff @@
==========================================
  Files           137        137          
  Lines         50287      50289     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42312      42313     +1   
- Misses         7975       7976     +1   
  Partials          0          0          
```
1. File `...das/tseries/index.py` (not in diff) was modified. [more](https://codecov.io/gh/pydata/pandas/commit/d7315dd64adfd577abe0dccf2f371aa2d5b6d80c/changes?src=pr#70616E6461732F747365726965732F696E6465782E7079) 
   - Misses `+1` 
   - Partials `0` 
   - Hits `-1`

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [43989fd...d7315dd](https://codecov.io/gh/pydata/pandas/compare/43989fd7cb9917e885c2b55a172c4f9f3838d59d...d7315dd64adfd577abe0dccf2f371aa2d5b6d80c)

@jreback the Travis-ci build job 19538.5 has failed for Python 3.5, as except with comma is not valid in Python 3. But, surprisingly the tests have passed for Python 3.4.

```
File "/usr/share/google/boto/boto_plugins/compute_auth.py", line 64

    except (urllib2.URLError, urllib2.HTTPError, IOError), e:

                                                         ^
SyntaxError: invalid syntax
```

Is this Pandas error or testing framework error. Should I run the Travis again.

thanks @yaduart 

As I am a newbie here, I am not sure of the processes. It would help me understand how to close the pull request, if you could tell how you decided to close the pull request as the Travis-ci checks were failing, or did you run them again, or was the Travis-ci bypassed.

@yaduart no, I merged it via https://github.com/pydata/pandas/commit/d0734ba4d0f4c228110dc3974943ce4ec2adeea4

it is a slightly edited commit. When I merged it closed the PR and the issue.

thanks for the contribution.

If I had closed it for another reason (e.g. not accepting) or something would always write something.

Why was the travis-ci check failing earlier?

unrelated, we picked up a broken `boto` version (not sure why it was released), and were importing it on main pandas import (which is wrong, but that's how it was). 

Thanks for clarifying. Cheers
PERF: quantile now operates per block boosting perf / fix quantile with nan
State: closed

closes #11623
closes #13098
There is a slight API change in that the returned Series are now named by the quantile.

this PR

```
In [1]: df = DataFrame(np.random.randn(1000, 3), columns=list('ABC'))

In [2]: %timeit df.quantile([0.1, 0.5], axis=1)
1000 loops, best of 3: 830 µs per loop
```

0.18.1

```
In [3]: df.quantile([0.1, 0.5], axis=1)
In [1]: df = DataFrame(np.random.randn(1000, 3), columns=list('ABC'))

In [2]: %timeit df.quantile([0.1, 0.5], axis=1)
1 loop, best of 3: 317 ms per loop
```


Labels:
name: Performance
description: Memory or execution speed performance
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Clean

Comments:
cc @sinhrks 

@sinhrks this also resolves some thorny issues with `.replace` and datetime w/tz (now the conversion happens at a much lower level).

Thanks. I've at a loss how to achieve the behavior you did in `reduction`. 

BTW, we don't seem to test `.quantile(axis=1)` with mixed dtype currently. It's nice to be added.

ok added test for mixed, though not a whole lot you can do with it.
BUG: GH12896 where extra elements are returned in MultiIndex slicing
State: closed

- [x] closes #12896
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Is this the right way to do this?

cc @MaximilianR 


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
pls update when you can.

Will do tonight!

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13117?src=pr) is **84.14%**

> Merging [#13117](https://codecov.io/gh/pydata/pandas/pull/13117?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **+<.01%**

``` diff
@@             master     #13117   diff @@
==========================================
  Files           138        137     -1   
  Lines         50384      50287    -97   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42382      42312    -70   
+ Misses         8002       7975    -27   
  Partials          0          0          
```
1. 2 files (not in diff) in `pandas/io` were modified. [more](https://codecov.io/gh/pydata/pandas/commit/94a6beda94cbb38505f0a892fa58a0e21bd3e660/changes?src=pr#70616E6461732F696F) 
   - Misses `+57` 
   - Hits `+3`
2. 5 files (not in diff) in `pandas/core` were modified. [more](https://codecov.io/gh/pydata/pandas/commit/94a6beda94cbb38505f0a892fa58a0e21bd3e660/changes?src=pr#70616E6461732F636F7265) 
   - Misses `-16` 
   - Hits `-69`
3. 4 files (not in diff) in `pandas` were modified. [more](https://codecov.io/gh/pydata/pandas/commit/94a6beda94cbb38505f0a892fa58a0e21bd3e660/changes?src=pr#70616E646173) 
   - Misses `-2` 
   - Hits `-5`
4. File `...das/indexes/multi.py` was modified. [more](https://codecov.io/gh/pydata/pandas/commit/94a6beda94cbb38505f0a892fa58a0e21bd3e660/changes?src=pr#70616E6461732F696E64657865732F6D756C74692E7079) 
   - Misses `0` 
   - Partials `0` 
   - Hits `+1`
5. File `pandas/io/s3.py` (not in diff) was deleted. [more](https://codecov.io/gh/pydata/pandas/commit/94a6beda94cbb38505f0a892fa58a0e21bd3e660/changes?src=pr#70616E6461732F696F2F73332E7079)

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [01dd111...94a6bed](https://codecov.io/gh/pydata/pandas/compare/01dd11109a0d1def8bc3b03d06c533817cc273f2...94a6beda94cbb38505f0a892fa58a0e21bd3e660)

ty sir!
Validate Timestamp with invalid offset raises ValueError
State: closed

- [x] closes #11630
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Datetime
description: Datetime data dtype
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13116?src=pr) is **84.14%**

> Merging [#13116](https://codecov.io/gh/pydata/pandas/pull/13116?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **+<.01%**

``` diff
@@             master     #13116   diff @@
==========================================
  Files           137        137          
  Lines         50287      50287          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42311      42312     +1   
+ Misses         7976       7975     -1   
  Partials          0          0          
```
1. File `...das/tseries/index.py` (not in diff) was modified. [more](https://codecov.io/gh/pydata/pandas/commit/eef9274d0f9cbe9b7e720a0cc42d6cae4d4caa97/changes?src=pr#70616E6461732F747365726965732F696E6465782E7079) 
   - Misses `-1` 
   - Partials `0` 
   - Hits `+1`

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [af7bdd3...eef9274](https://codecov.io/gh/pydata/pandas/compare/af7bdd3883c8d61e9d9388d3aa699930eee7fff8...eef9274d0f9cbe9b7e720a0cc42d6cae4d4caa97)

@gliptak The test you added is failing (see the travis logs)

BTW, you have a large diff, but I don't think you actually changed much? Please only change code that is actually related to the issue, and do not move around other code (difficult to review the diff)

I think #12482 is in a better place to address that issue.