BrandonFors/pandas-github-issues huggingface dataset:
Implement Akima1DInterpolator
State: closed

- [x] closes #7588
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Enhancement
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
This build failed https://travis-ci.org/pydata/pandas/jobs/121843938 likely because of the `Scipy` version.

`axis` was added to `Akima1DInterpolator` at `0.17.0` as per https://github.com/scipy/scipy/commit/b0395b40c577042436511ce13b9bc067508956db

Is there a minimum version of `Scipy` for `pandas`? What would be a good way to handle lower versions?

Thanks

you can you call this in a try: except to handle the scipy signature differences. (then if it doesn't work in the excpet, raise again). we don't have a min version. 

The two test failures seem to be unrelated.

The two test failures seem to be unrelated.

pls add a whatsnew as well

Added whatsnew

pls squash

add a small note when to use this in : http://pandas-docs.github.io/pandas-docs-travis/missing_data.html#interpolation (you might want to break up that section a bit and make it into a table)

add to the doc-string for `.interpolate` with a `versionadded` tag

Added notes and versionadded tag

Updated per comments.

ok, looks good. ping when green (and pls verify that tests are skipped as appropriate)

@gliptak I just added some more doc comments, looks very good for the rest!

@jreback Build is green. Tests skipped https://s3.amazonaws.com/archive.travis-ci.org/jobs/122101334/log.txt

```
#268 pandas.tests.test_generic.TestSeries.test_interpolate_akima: no scipy.stats module
#269 pandas.tests.test_generic.TestSeries.test_interpolate_corners: no scipy.stats module
#270 pandas.tests.test_generic.TestSeries.test_interpolate_pchip: no scipy.stats module
```

I also opened https://github.com/pydata/pandas/issues/12856 (consistently fails the same way)

@gliptak thanks!

pls review the built docs for accuracy (and good looking ness) and issue a follow up if necessary. (prob will take a few hours before they are built)

@jreback @jorisvandenbossche Thank you for guiding this through

Docs updated good.
BUG: SparseSeries.reindex with fill_value
State: closed

- [x] closes #12797
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Sparse
description: Sparse Data Type

Comments:
thanks! (sorry commented on other one!)
CLN: cleaning core/common.py
State: closed

partial on #12503
- create `pandas/types`
- moved _some_ dtype to `pandas/types`
- move _some_ missing value utils to `pandas/core/missing`
- move `.take` and `.diff` to `core/algorithms`
- create `formats` with `core/formats,style` moved and `printing.py` (form `core/common.py`)
- removed some non-used functions


Labels:
name: Clean

Comments:
cc @wesm 

still more to do, but this is a start.

@shoyer @jorisvandenbossche @sinhrks @TomAugspurger 

any comments. don't want to leave this out too long as it touches lots of things.

Not objections to your description.

Not a big deal, but does `formatting/` make more sense than `formats/`?

its slightly longer, but no big deal. 

assuming you're just moving stuff around, this seems sane to me

I'll look later this AM (pacific time)

This all looks fine to me. 

Aside: I'm on vacation most of this month but planning to spend some time in May / June / July on refactoring to look toward retooling DataFrame internals (it would be nice to plot a course to removing the BlockManager, deprecating Panel, and some of the other under-the-hood internals work we've been discussing). In the meantime, anything that can be done to get organized and compartmentalize non-user code that will be affected will help this proceed more quickly. I think this patch helps us along this path. 

np on schedule. will be a bit more internal refactoring soon. 

@TomAugspurger do you feel strongly on the `formats` -> `formatting`?

Not at all, especially since this is all internal and not user facing.

@jreback : Are you missing something in `setup.py`?  When I try installing the `master` branch, the `formats` folder does not get installed, leading to the error `ImportError: No module named formats.printing`

I pushed a change.

though I installed fine from a fresh clone (this had to do with package setup)

@gfyoung you prob got somethign like this: https://ci.appveyor.com/project/jreback/pandas-465/build/1.0.409/job/fnp6uhksqcp039g4

@jreback : Indeed, I did, and I'm happy to say that with your most recent commit, everything installs just fine now!  Thanks!

great not sure why our install test on Travis didn't fail 
but I guess it's not actually testing the built version but just creating it

in any event appveyor catches it because of the way it builds (it's a full conda build) not just an inplace build 

@TomAugspurger I think the doc build broke: lots of Styler warnings now. https://travis-ci.org/pydata/pandas/jobs/121316764.

building locally so far so good though (not done yet)

Yes, just wanted to mention the same. It's the same error again as before. I can't remember anymore what solved it before

Mmm, I think it magically fixed itself last time. Will look later.

The `.. currentmodule:: pandas.core.style` in api.rst should probably updated?

yep, just pushed it

https://github.com/pydata/pandas/commit/8250d7c7ddfd321f91cf7ccd44cb906cdc63c0a0

ok, this now built ok for me locally.

Still these but IIRC they were there before

```
None:None: WARNING: toctree contains reference to nonexisting document u'generated/template:'
```
ENH: GH11128 add weekday_name to DatetimeIndex and .dt
State: closed

closes #11128 
replaces #11813
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Completed merge of Bahrunnur's PR 'weekday-name'
    Moved 'enhancements' to v0.18.1.txt
    Added tests to pandas/tests/series/test_datetime_values.py
    Added weekday_name property producing 'NaT' to NaTType class


Labels:
name: Enhancement
name: Datetime
description: Datetime data dtype

Comments:
looks good. just some minor points.

come to think of it, prob should add `month_name` as well I guess. Then of course there is the locale (IOW, we really should return a locale based string for the weekdays). issue. Can you make an issue for these 2 things?

I _think_ we can just instantiate the `LocaleTime()` object when this module is imported. Not really sure why that wouldn't work, you can try it if you want. This should really be a singleton type object, e.g.

```
locale_time = LocalTime()
```

then use `locale_time`

ok this looks good. I'd like to use the `LocaleTime()` object as I described (should be cached) to return the locale specific names here. 

xref #12806 

can you try adding this with the `LocaleTime`support?

Yes, just need to find the time, maybe later this week.
On Apr 17, 2016 8:46 AM, "Jeff Reback" notifications@github.com wrote:

> can you try adding this with the LocaleTimesupport?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly or view it on GitHub
> https://github.com/pydata/pandas/pull/12803#issuecomment-211045629

@BastiaanBergman ok, great thanks

@jorisvandenbossche comments?

I guess we can address using `LocaleTime()` later.

Sorry, didn't get to it, will try again late May.

Looks good

thanks!
ENH/PERF SparseArray.take indexing
State: closed

- [x] related to #4400 (not close yet as `SparseDataFrame` indexing test is not sufficient)
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Added more tests for sparse indexing. Fixed followings:
- `SparseArray.take` has optimized logic to omit dense `np.ndarray` creation.
- `SparseSeires.iloc` can work with negative indices. Made `SparseArray.take` to handle negative indices as the same rule as `Index` (#12676)


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Performance
description: Memory or execution speed performance
name: Sparse
description: Sparse Data Type

Comments:
@jreback rebased, and now green.

thanks @sinhrks
Fix for #12723: Unexpected behavior with binary operators and fill…
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

closes #12723 


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
If it is OK, I'd like to add a whatsnew and rebase.

pls test for the sparse cases as well

The work is done if all tests are passed. 
Thanks, everyone.

If it is ok, 
might we close #12723 ?

issues are closed when the PRs are merged - I'll have a look soon

thanks for the fix!
TST: Make sparse test more strict
State: closed

- [x] tests added / passed
- [x] passes ``git diff upstream/master | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Sparse
description: Sparse Data Type

Comments:
sure!

btw. if you need these merged in a particular order, lmk.

Pls merge this PR prior to #12787. I'll rebase #12787 once.

ty!
BUG: SparseDataFrame indexing may return normal Series
State: closed

- [x] related to #4400 (not close yet)
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Found a below bug in `SparseDataFrame` indexing.

```
# NG, must be SparseSeries
df = pd.DataFrame([[1, 2], [np.nan, 4]]).to_sparse()
type(df.loc[0])  
# pandas.core.series.Series
```


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Sparse
description: Sparse Data Type

Comments:
Rebased, and now green.

thanks!
BUG: DataFrame.drop() does nothing for non-unique Datetime MultiIndex
State: closed

- [x] closes #12701
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Closes #12701

Follows the suggested fix in the comments to the bug report.
Also added a line in whatsnew and a test.

Regarding the tests it passes the one I added. But I just noticed that plenty others fail and I have yet to find out whether it's caused by this (who knows, it might)
Update: It's the same tests failing on master branch as on this one (phew..). I guess the errors deserve bug reports but that's for another time.


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
@jonaslb nothing fails on master main build. There is a single failure on 2 of the compat builds, but these are known.

are you talking about something specific?

@jreback [Here's a gist](https://gist.github.com/jonaslb/b8ad05c4ed04b11fe9cfe633e09ca070) (7 errors / 2 failures) of my test-run. Yes I guess it's specific since it's something with clipboard and then there are some index and slice problems apparently. But lets wait and see what travis says - I'm pretty certain I didn't mess it up with this commit :) Sometime later I'll look more into it and maybe make bug reports.

thanks @jonaslb
DOC: Clarify when csv separator is being parsed as regex. Resolves #10208
State: closed

- [x] closes #10208
- [ ] tests added / passed -- only docs, not needed
- [x] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry -- neccessary for docs?


Labels:
name: Docs
name: IO CSV
description: read_csv, to_csv

Comments:
also update the docs in `io.rst` (with the same), though you can be 

Updated!

thanks!
BUG: replace coerces incorrect dtype
State: closed

- [x] closes #12747
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: API Design

Comments:
Found this affects more functions than I thought. Let me do it in 4 separate PRs.

1 Add numeric related tests to clarify the current behaviour (and future change)
2. Fix numeric coercion.
3. Add datetime-like tests.
4. Fix datetime-like coercion.

## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/12780?src=pr) is 85.97% (diff: 89.65%)
> Merging [#12780](https://codecov.io/gh/pandas-dev/pandas/pull/12780?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #12780   diff @@
==========================================
  Files           140        140          
  Lines         51254      51275    +21   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          44068      44085    +17   
- Misses         7186       7190     +4   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [a62fdf8...f9154e8](https://codecov.io/gh/pandas-dev/pandas/compare/a62fdf8b85bcb0fcedbe9b5269625d50f4b2cdd7...f9154e8a70e128cb0b07edcc325b4444930adcf5?src=pr)
after #13147 this should be slightly more standard

@sinhrks looks pretty good, just a couple of small comments.
@sinhrks status? (lgtm so far)
Updated to fix the above points, and now ready for review.
looks fine. still some internal things ideally should clean up. but can do later.

though couple of comments if you can address.
@sinhrks I am rebasing this and will merge after testing.
BUG: Sparse misc fixes including __repr__
State: closed

- [x] closes #10560
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Fixes following issues.

```
# NG, should be SparseArray (categorized as API change)
a = pd.SparseArray([1, np.nan, np.nan, 3, np.nan])
type(a.take([1, 2]))
# numpy.ndarray

# NG
s = pd.Series([1, np.nan, np.nan, 3, np.nan]).to_sparse()
s.loc[[1, 3]]
# TypeError: reindex() got an unexpected keyword argument 'level'

# NG
s.iloc[2]
# IndexError: index out of bounds

# NG, must be SparseSeries (root cause of 10560)
type(s.iloc[2:])
# pandas.sparse.array.SparseArray
```


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Sparse
description: Sparse Data Type

Comments:
xref: `SparseDataFrame` indexing #4400

@sinhrks awesome!

I repurposed #4400 to just be some addl tests.

never saw this one before: https://travis-ci.org/pydata/pandas/jobs/120431675
BUG: to_dense does not preserve dtype in SparseArray
State: closed

- [x] closes #10648
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Sparse
description: Sparse Data Type

Comments:
thanks @sinhrks awesome!
PERF: to_numeric for numeric dtypes
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Skip `object` conversion if input is numeric already.

```
-  146.41ms    26.45μs      0.00  miscellaneous.to_numeric.time_from_float
```


Labels:
name: Performance
description: Memory or execution speed performance
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
lgtm. ping on green.

```
In [1]: pd.date_range('20130101',periods=3,tz='US/Eastern')
Out[1]: DatetimeIndex(['2013-01-01 00:00:00-05:00', '2013-01-02 00:00:00-05:00', '2013-01-03 00:00:00-05:00'], dtype='datetime64[ns, US/Eastern]', freq='D')

In [2]: pd.to_numeric(pd.date_range('20130101',periods=3,tz='US/Eastern'))
TypeError: 'Index' does not have the buffer interface

In [3]: pd.to_numeric(pd.date_range('20130101',periods=3,tz='US/Eastern').to_series())
Out[3]: 
2013-01-01 00:00:00-05:00    1357016400000000000
2013-01-02 00:00:00-05:00    1357102800000000000
2013-01-03 00:00:00-05:00    1357189200000000000
Freq: D, dtype: int64
```

@sinhrks this looked fine. can you just rebase  (I think this was sent when codecov was misbehaving).

we can push this as well, lmk.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/12777?src=pr) is **83.91%**

> Merging [#12777](https://codecov.io/gh/pydata/pandas/pull/12777?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **+<.01%**

``` diff
@@             master     #12777   diff @@
==========================================
  Files           136        136          
  Lines         49918      49931    +13   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          41885      41899    +14   
+ Misses         8033       8032     -1   
  Partials          0          0          
```
1. File `...das/tseries/index.py` (not in diff) was modified. [more](https://codecov.io/gh/pydata/pandas/commit/2d2b45a5db7325603eec9c219df99788fae12c46/changes?src=pr#70616E6461732F747365726965732F696E6465782E7079) 
   - Misses `-1` 
   - Partials `0` 
   - Hits `+1`

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by 2d2b45a

@jreback Current master has following issues. I've updated the PR to included the fixes.

#### 1. `to_numeric(Index)` returns `np.ndarray`, not `Index`

To be compat with `to_datetime`, it must be `Index`.

```
pd.to_numeric(pd.Index([1, 2, 3]))
# array([1, 2, 3])
```

#### 2. datetime-likes are not actually supported

Based on your examples, it should be `asi8` repr.

```
pd.to_numeric(pd.date_range('2011-01-01', freq='M', periods=3))
# TypeError: 'Index' does not have the buffer interface

pd.to_numeric(pd.Series(pd.date_range('2011-01-01', freq='M', periods=3)))
# TypeError: Invalid object type
```

Added scalar impl and tests (`test_scalar`). Now green except for `codecov/changes`.

thanks!
BUG: filter (with dropna=False) when there are no groups fulfilling the condition
State: closed

- [x] closes #12768
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Implemented the fix for #12768 which was proposed in the bug report. 


Labels:
name: Bug
name: Groupby

Comments:
run: `git diff master | flake8 --diff` to see PEP issues

thanks @jonaslb
Retain name in PeriodIndex resample
State: closed

- [x] closes #12769
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Period
description: Period data type
name: Resample
description: resample method

Comments:
pls don't change stuff just because py charm does
use targeted changes

I would remove it if I didn't think it made it better / cleaner. Happy to if you don't think it does...

run `git diff master | flake8 --diff` to see PEP issues

@jreback green

Although codecov states that this line isn't tested: 
![image](https://cloud.githubusercontent.com/assets/5635139/14229714/cea0c69a-f908-11e5-9d8c-063aa683fb8f.png)

Unless I'm mistaken, I added a test for this + a note that the test needed to be expanded to test for the index, rather than only its length. But that requires another PR

no I just fixed something on codecov

looks good. ping on green.

@jreback green, apart from the codecov anomaly 

thanks @MaximilianR 

I moved the tests to the proper location (and some other ones, not sure why there were there). `test_resample`.

now, if in `test_resample` you do this (and do it with `_ts` as well).

```
def _simple_pts(start, end, freq='D'):
    rng = period_range(start, end, freq=freq, name='foo')
    return Series(np.random.randn(len(rng)), index=rng)
```

you will get some failures, could just be some test failures though (e.g. the expected need to change).
BUG: loffset argument not applied for resample().count() on timeseries
State: closed

- [x] closes #12725
- [ ] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

The code to do this already existed in the `_downsample` method, which is called when using functions like `mean`. `max`, etc., but there was nothing in the `_groupby_and_aggregate` method used for `count` and `size`. If we pull out the offset code from the `_downsample` method into a separate method, we can reuse it without duplicating it.


Labels:
name: Bug
name: Resample
description: resample method

Comments:
@onesandzeroes perfect!
Manhattan plot for bioinformatitians
State: closed

Hi everyone,
I'm a pandas user and just wanted to contribute but it was very complicated to commit as a developer...
BTW, I added Manhattan plot which is very popular in genetics and biology
Here is the R equivalent
https://cran.r-project.org/web/packages/qqman/qqman.pdf
## Cheers,

ARYA
- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
@airanmehr Thanks for your interest in contributing!

However, I think this function is too specific to be included in pandas. It seems a rather domain-specific function, is that correct?

But I would certainly want to encourage you to share it in other ways than including it in pandas itself. This can be by starting to simply put it on github, eg in a notebook with an example. So people can find this and use it. 
A next step would be to make a small package from it that people can install and use. If you would be interested in something like that, I would suggest to look at https://github.com/uwescience/shablona ("A template for small scientific python projects")

closing as out of scope for pandas
CLN: Move boxing logic to BlockManager
State: closed

- [x] closes #12741 closes #12772 closes #12469
- [x] tests added / passed
  - [x] `map` related boxing
  - [x] `quantile`
- [x] passes `git diff upstream/master | flake8 --diff`
- added `Series.asobject` to return boxed `object` values
- remove `common/i8_boxer()`, `Series._maybe_box()` (only used by `.quantile()`)

If base direction looks OK, going to add some tests based on the spec.


Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Clean

Comments:
@sinhrks this is awesome!!!!!

2 small comments.

further pls review: #12469 which I think can be closed upon merging this? and #10207 the underlying issue.

https://github.com/jreback/pandas/commit/b7ec8432f586783bea1ef6cf9b7197c3d5250111
cleans up the datetime logic & coercion. and makes it more block friendly.

going to merge this, then can come back and fix the frame to use blocks. This is a bit of a special internal method as it is a reduction which we normally don't pass thru to the block manager (so using this `raw` kw). I have to think about how this works, but this is ok for now. Can clean up in a later PR.

thanks @sinhrks 

I may do a PR to make the reductions be a little nicer.
TST: Add period and other dtype related tests
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- xref #12549 
- added more `Period` related tests in preparation for period dtype.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Timezones
description: Timezone data dtype
name: Period
description: Period data type

Comments:
thanks!