BrandonFors/pandas-github-issues huggingface dataset:
BUG: Index does not inherit existing Index or DatatetimeIndex object …
State: closed

This PR is meant to fix issue #11193 
The issue prevent, when creating a new `Index` object based on a passed `Index` or `DatetimeIndex` object, to inherit the existing `Index` name if a new name is not provided. 

I've also created two unit test which test the new behaviour on both cases (passing an `Index` or `DatetimeIndex` object) and added a comment in the whatsnew documentation.


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Great!
- This could test other classes (`PeriodIndex`, etc), both when they're called with their own constructor (`pd.PeriodIndex(period_index)`, and with different constructors (`pd.Index(period_index)`).
- This could also update all the fields in `_attributes`, rather than just `name` (although it'd have to protect against differently typed Indexes having different attributes. In many cases it would be identical; a standard `Index` only has `name` in `_attributes`.)

I wonder if you could just call `_infer_new` from this (unmerged) PR: https://github.com/pydata/pandas/pull/11497, and that would take care of all this.

I'll rebase on master and squash my commits. I should update my PR shortly.

minor correction, pls rebase as I just merged #11497 which _might_ impact this. ping when green.

I've just rebased @jreback I'll add a second comment when the Travis CI build is completed. 

@jreback let me know if this needs further changes.

@IamGianluca can you update

I've updated the unit test @jreback 

merged via https://github.com/pydata/pandas/commit/fc40dcce34874c8399474b0a3b8119ea32207055

thanks!
Use tz.gettz() instead of zoneinfo.gettz()
State: closed

zoneinfo.gettz() seems to have problems (1 & 2) on system which do not install
the zoninfo tarball (e.g. Debian, Gentoo and Fedora) but rely on the system
zoneinfo files. This results in test failures (3 & 4)
tz.gettz() doesn't suffer from this problem.

1 https://github.com/dateutil/dateutil/issues/8
2 https://github.com/dateutil/dateutil/issues/11
3 https://github.com/pydata/pandas/issues/9059
4 https://github.com/pydata/pandas/issues/8639

Signed-off-by: Justin Lecher jlec@gentoo.org


Labels:

Comments:
looks like some older comments here (and commit message) can u fix

Sorry for the noise
API: DataFrame.to_csv formatting parameters for float indexes
State: closed

Fix issue #11553 

Two things:

1) I've created a `Float64Index._format_native_types` method which is a copy-paste of  `FloatBlock.to_native_types`. I would have preferred to call the latter directly, but I'm not sure what the `placement` parameter of the `FloatBlock` constructor means. I guess I doesn't really matters, since I could put whatever value and it should work (I think), and my hesitation a bit unfounded, but I don't know if it would be really a clean solution. Maybe someone can think of a more elegant way?

2) Since a `Float64Index` containing only NaNs collapses when part of a multi-index, its NaNs values would not be converted using `na_rep`, so I had to hack a solution. I put a comment in the relevant part. I'm not quite convinced myself of the elegance of the solution, though.

What do you think?


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: API Design

Comments:
And I've corrected the fact that the `decimal` option was not taken into account for `0.0`, by replacing the use of `%g` formatting with a call to `str`

@jreback all set

ok, looks pretty good. I think we can take this opportunity to re-factor a bit as I have noted above.

@nbonnotte if you can update / refactor as above would be great

I will, just had not the time yet. If you'd like this to be done quickly, because of the schedule for the 0.18.0 release, let me know.

no, just pinging :)

lmk when you can update

@jreback I just pushed the changes. Let me know if other changes are needed.

Happy Holidays!

lgtm. some very minor formatting changes. in general like to have blank lines between different sub-tests and to format code nicely.

ping when pushed and green.

If you and Travis agree, that should be it.

I have added some comments as you suggested, to make the code clearer. But to be honest I'm not quite sure about how some bits of the code fit with the rest, for instance how and where the `quoting` parameter is handled. I mostly moved stuff around, and it's difficult to explain things in comments when one does not understand what is going on. I hope the comments I wrote are OK, though.

Also, I suppose `FloatArrayFormatter` could be refactorized a bit, because right now it looks like more a potpourri, a mixture of different bits of code that are related but do not have much code in common. But I'm not sure it is worth the effort.

@nbonnotte code looks great. just see if we have some tests for `quoting` if not, pls add them.

the explanations are fine. Just want to have a note instead of a mass of code to explain a bit what is happening.

as far as refactoring, you got what I was looking for (integration between `Index` and data formatting). if you see additional thinks, don't hesitate with another PR!

ping when pushed / green (or if tests ok, lmk)

Yeah, the `quoting` parameter is used in `pandas/tests/test_format.py`, at least in `test_to_csv_quotechar()`, `test_to_csv_doublequote()`, and `test_to_csv_escapechar()`.

thanks!
BUG: style.bar results in incorrect gradient using specific browser
State: closed

It's not a bug on pandas side, but CSS `linear-gradient` with `0%` width seems not be rendered properly using specific browsers.

Below is the output using Mac OSX 10.10.5 / Firefox. 

![2015-11-23 10 53 47](https://cloud.githubusercontent.com/assets/1696302/11329349/58e78d80-91de-11e5-82e6-6c90b80d93fb.png)

The fix simply exclude `linear-gradient` if its width is 0.

After the fix (same environment)

![2015-11-23 10 54 18](https://cloud.githubusercontent.com/assets/1696302/11329369/9d37dce2-91de-11e5-9636-29d2b2a843cd.png)


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
@TomAugspurger ?

@sinhrks lgtm. (prob a whatsnew conflict)

thank you sir!
ENH: Read from compressed data sources
State: closed

This is the start of an effort to address compression in `read_*` methods (at the moment, just `read_pickle`, as per #11666. So far, I've factored out the compression handling in the `_read` function of `pandas.io.parsers` and used that in the pickling read routine.

Tests on the way, along with actual testing that it works, docs and (if we want), I can also put something similar in the other `read_*` methods too.


Labels:
name: Enhancement
name: IO Data
description: IO issues that don't fit into a more specific label
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
needs some tests

can you add some tests and upate

@khs26 can you update

@khs26 can you update

closing but pls reopen if you wish to continue working
BUG: GH11616 fixes timezone selection error
State: closed

closes #11616 
closes #11682 


Labels:
name: Bug
name: Groupby
name: Timezones
description: Timezone data dtype

Comments:
pls add a note in 0.18.0 bug fixes as well.

pls add a test for #11682 which I believe this should fix as well

These errors are occurring for different reasons. First error #11616 is in _possibly_downcast_to_dtype. It is not able to retrieve correct date from int64 object. hence throwing an exception that I have fixed by checking for timezone info and then converting to correct timezone.
#11682 is in _infer_dtype_from_scalar function as correctly pointed out by ajenkins-cargometrics . We are checking for 'tz' attribute . I have fixed that by replacing 'tzinfo' and added test case for that. 
Please review. 

looks good. pls add 2 entries in bug fixes in whatsnew for 0.18.0

Done. Please review.

@jreback  One of the five [Build job](https://travis-ci.org/pydata/pandas/builds/93192293) is failing . I am not able to figure out the reason . Please help.

merged via https://github.com/pydata/pandas/commit/e8382665d12dcf270f587cf76cd0e6d5a1dac184

thanks!
BLD: Change pytables to tables in the asv dependencies.
State: closed

The asv benchmarks fails to build due to [PyTables PyPI package name being `tables` and not `pytables`](https://pypi.python.org/pypi/tables). This PR fixes the issue and the asv benchmarks now builds correctly.


Labels:

Comments:
this is meant for conda not thru pip
so I don't think this will work 

I see. I think noting in [CONTRIBUTING.md](https://github.com/pydata/pandas/blob/master/CONTRIBUTING.md) that conda is the default would help. I was confused by the remark to use either `conda` or `virtualenv` with asv.

yeh glad to take a PR to clarify this a bit

and/or alternate asv config file that only uses pip
DOC: a few touchups
State: closed

Comments in https://github.com/pydata/pandas/pull/11660#issuecomment-158427446

@jorisvandenbossche 
@jreback 

I think we're ok to merge.


Labels:
name: Docs

Comments:
looks good (for a follow-up, you should also use a space before the `:` in the Parameters section)

@TomAugspurger ready on this?
DOC: warnings and remove HTML
State: closed

Couple comments in https://github.com/pydata/pandas/issues/11610#issuecomment-158411503
- warning in the whatsnew
- warning in the Styler docstring
- removes generated HTML version of the notebook, that's built as part of the doc build.

Building the docs now to double check, I'll merge  when those are good. 


Labels:
name: Docs

Comments:
Looks good!

@jorisvandenbossche IIRC we have `ipython` as a requirement of the doc-builds, should also add `ipython-notebook` (or `jupyter`)

in principle, it's not `ipython notebook`, but just ipython for <4 (nbconvert it included, so that is OK) or `nbconvert` for >= 4

But maybe safer to use `ipython nbconvert` instead of `jupyter nbcnvert` in make.py (as the first one works for all versions, the jupyter one only for the more recent)

@TomAugspurger did you also check the api docs for the new styler if they render work correctly?

Yep, the warning shows up there, things look good. I can upload the generated version somewhere if you want.

Ahh, shoot found one mistake.  I forgot to change `Styler.set` to `Styler.use` in `api.rst`. Is it easiest for you to change that?

This is line 1713 in the current version of master, not sure how to link to it since GH renders the rst. `Styler.set` -> `Styler.use`

no, you can go ahead and change it (I still have to switch from windows to linux for the final build)
DOC: sponsor notice
State: closed

cc @jorisvandenbossche 
cc @cpcloud 
cc @shoyer 
cc @wesm 


Labels:
name: Docs

Comments:
Seems like a good idea to me

@jorisvandenbossche @TomAugspurger 

anyone see what's wrong with this link? at the top
http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html

`:: _numfocus organization:` should be
`.. _numfocus organization:` I think.

@jreback I already fixed it, but it is still in the build queue I suppose, so it is not yet fixed online

and indeed, @TomAugspurger is correct

@jorisvandenbossche oh, I just pushed the fix :)

but in the wrong direction! :-) (you made it incorrect again)

@jorisvandenbossche oh really? shoot. ok how about your turn :)

Reverting on master is bad practice I suppose?

no it's fine go ahead
Bug#11637 fix to_csv
State: closed

This is a fix to Bug #11637. 
Error Description : incorrect output when a mix of integer and string column names passed as columns parameter in to_csv(). 
[format.py](https://github.com/pydata/pandas/blob/master/pandas/core/format.py#L1330) is the cause of this discrepancy. 

Please review .


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: IO CSV
description: read_csv, to_csv

Comments:
looks good. pls add a whatsnew new for bug fixes for 0.17.1

ping when green.

Done .

merged via https://github.com/pydata/pandas/commit/784445cbe81cc8b2da4e2324689f175cb2a2edbf

thanks!
BUG #11638 return correct dtype for int and float
State: closed

closes #11638
 Please review. 


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
tests!
use the example from the issue 

Added test case .
Squashed commits.
Please review.

@varun-kr looks good. can you add a release not to whatsnew/0.17.0 in bug fixes.

It should be 0.17.1.  Right ?

yep

ping when green.

Done.

merged via https://github.com/pydata/pandas/commit/24cdf45423237bbcf290bc249e93814bfca71c8e

I expanded the tests a bit, thanks for the push!
PERF: improves performance in remove_unused_categories
State: closed

on master:

``` python
In [1]: np.random.seed(2718281)

In [2]: n = 1 << 20

In [3]: ix = tm.makeCategoricalIndex(n, n // 10)

In [4]: %timeit ix.remove_unused_categories()
1 loops, best of 3: 527 ms per loop
```

on branch:

``` python
In [4]: %timeit ix.remove_unused_categories()
1 loops, best of 3: 216 ms per loop
```


Labels:
name: Performance
description: Memory or execution speed performance
name: Categorical
description: Categorical Data Type

Comments:
merged via https://github.com/pydata/pandas/commit/7031e5f11155f6e8f6a8f04afb6159455bd62036

thanks you sir!
TST: Fix two failing tests on Windows #10631
State: closed

I spotted two failing tests with errors similar to what described @chris-b1 in #10631:

```
Attribute "dtype" are different
[left]:  int32
[right]: int64
```

This PR fixes this on windows.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
thanks!
ENH: Implement export of datetime64[ns, tz] dtypes with a fixed HDF5 #11411
State: closed

closes #11411


Labels:
name: Enhancement
name: IO HDF5
description: read_hdf, HDFStore
name: Timezones
description: Timezone data dtype

Comments:
@jorisvandenbossche easiest just to implement! xref #11281 

Awesome!
CLN/COMPAT: Remove raise StopIteration syntax in SparseArray.__iter__
State: closed

closes #11108 


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
why is this bad? e.g. if you are iterating with a generator with works just fine

this causes a `PendingDeprecationWarning` in 3.5 and will be deprecated in 3.7 ([PEP479](http://legacy.python.org/dev/peps/pep-0479/))
this syntax doesn't cause a problem here, but it could if combined with a context manager, where the `StopIteration` exception will be caught in `__exit__()` and potentially causes the context manager to take a different path (depending on the presence of try-except constructs in the context manager)

is this an API change though?

No.  Using `return` (or just leaving it out, since there is no more code) and `raise StopIteration` are the same in this function (except for the warnings).  They are different only when there is something inside the generator that could interfere with exception propagation.

ok seems reasonable. add a whatsnew note.

which section?

I guess API changes; this is mainly to have a record of this.

green

thanks!
Roll apply nonfloat dtypes
State: closed

Fixed issue [#4964](https://github.com/pydata/pandas/issues/4964) so rolling_apply works on non-numeric columns as well.

I followed instructions [here](https://github.com/pydata/pandas/blob/master/CONTRIBUTING.md) but I'm not sure where to add documentation in the [release notes](https://github.com/pydata/pandas/blob/master/CONTRIBUTING.md#documenting-your-code). 

Also would like to get feedback on the code and complete any code changes before adding to documentation.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
this would be after #11603 

but you don't actually want to modify `roll_generic`, rather you want to have different functions for different dtypes based on there input. So would need to modify the templates to generate `object` ones (which is prob useful), otherwise coerce to float.

Are the #11603 changes already in master?

I thought about creating a different rolling_\* function for each dtype but the logic would be almost identical to rolling_generic() with changes in the dtype of input and a couple of dtype changes in the function.

It seemed cleaner (and DRY) to always give it a float as 'input' and give it another numpy array called array_to_roll when we have a non-float type we wish to roll over. Are you concerned about performance? I was unsure if coercion should be exposed to user and if it should default to True, but optimized for speed - anyhow, moot points if that's not the route we wish to go.

But curious if you prefer different but almost identical functions to `roll_generic` for each non float dtype. I can certainly give this a go in another branch, and we can see how similar the functions are then discuss further. 

@sandhujasmine we use template generation to handle the differnt dtypes, see `src/generate_code.pyx`. you would end up with `roll_generic_float64` and `roll_generic_object`. These refer to the input types (you can also use a fused type for this, look for the `numeric` type). but we usually have a separate routine for `object` as well, as it sometimes needs customization.
further its possible to have a matrixed set of input/output types (but I don't really like this), and would instead assert that the input type produces the same output type (and if it doesn't then it would raise and we would just use the `roll_generic_object` anyhow.
BUG: fix col iteration in DataFrame.round, #11611
State: closed

Fixes #11611. The iterator now uses DataFrame.iteritems instead of direct indexing.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
need some tests

pls add the test example as well (you can just smoke tests it I think; e.g. assert it runs).

add a tests for passing a non-unique decimal.

put this in the bug-fix section for a whatsnew entry.

Agreed.

merged via https://github.com/pydata/pandas/commit/80a2d530bae32ef7ecf93aa275d17659ee8e8ac1

thanks!
BUG: loc against CategoricalIndex may results in normal Index
State: closed

Closes #11586.

After the PR:

```
import pandas as pd
index = pd.CategoricalIndex(list('aabbca'), categories=list('cabe'))
df = pd.DataFrame({'A' : np.arange(6,dtype='int64')}, index=index)

# OK (not changed)
df.loc[['a', 'b']].index
# CategoricalIndex([u'a', u'a', u'a', u'b', u'b'], categories=[u'c', u'a', u'b', u'e'], ordered=False, dtype='category')

# Fixed to return Categorical Index if value exists in categories
df.loc[['a', 'b', 'e']].index
# CategoricalIndex([u'a', u'a', u'a', u'b', u'b', u'e'], categories=[u'a', u'b', u'e'], ordered=False, dtype='category')

# raise KeyError otherwise (not changed)
df.loc[['a', 'b', 'x']].index
# KeyError: 'a list-indexer must only include values that are in the categories'
```

There are separate paths when `CategoricalIndex` values (codes) are unique and not-unique and both fixed.

Also, `.reindex` intentionally returns normal `Index` when passed values are not `Categorical`, I kept the behavior as it is. If `.reindex` can always return `CategoricalIndex`, above 2 separate fixes are not required (fixing `.reindex` to return `CategoricalIndex` should work both paths).

```
# return normal Index(not changed)
df.index.reindex(['a', 'b'])
# (Index([u'a', u'a', u'a', u'b', u'b'], dtype='object'), array([0, 1, 5, 2, 3]))

# return CategoricalIndex(not changed)
df.index.reindex(pd.Categorical(['a', 'b']))
# (CategoricalIndex([u'a', u'a', u'a', u'b', u'b'], categories=[u'a', u'b'], ordered=False, dtype='category'), array([0, 1, 5, 2, 3]))
```


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Categorical
description: Categorical Data Type

Comments:
s reindex should give back a like index to the target / meaning if u put in a Categorical we try to give u a Categorical; if u out in an Index u get an Index

note that what u get is not dependent on what the actual index is! this is on purpose 

this is essentially the difference between .loc and .reindex

Ah yes, thus the behavior is unchanged (as in the example at the bottom of the body). The fix should only applied to `.loc`.

thanks!
BUG GH11600  - MultiIndex column level names lost when to_sparse() called
State: closed

closes #11600 

Fixed problem with multi-index column level names not propagating into sparse frames or back out to dense on a round trip through sparse. Includes 4 new tests to cover some relevant scenarios. Problem fixed for the conventions to_sparse path, I'm not sure about other paths where something else is passed to SparseDataSeries directly, those would be outside scope of bug.


Labels:
name: Bug
name: Sparse
description: Sparse Data Type
name: MultiIndex

Comments:
Hi Jeff- Hopefully I understood what you were looking for in the tests, it now has one roundtrip and one check after sparse construction, both using the pandas-specific asserts. If this was not what you meant just let me know how it could be better. Cheers, Zeke

couple of points; pls add a whatsnew note (bug fixes) for 0.17.1.

ping when green.

pls squash as well

Code all fixed up but squashing still left me two commits for some reason. I'll look into the Git stuff but it might take a couple days as I have some other stuff to do.

Squashed all my stuff down to one commit and updated pull request. Looks like you have a commit that's mixed in there to adjust a warning but github says its okay. Let me know if anything can be better, I think I hit everything you asked for.

merged via https://github.com/pydata/pandas/commit/207e0ce811451280e835475b7b8b2c34b3818d4e

thanks!

Great! I updated on stackoverflow with a link to the issue here just in case somebody (possibly who hasn't updated) should ever run into it. Seems perhaps unlikely for such an edge case but better that than somebody finding the SO question and not knowing its fixed if they update!
CI: testing newer matplotlib with jinja
State: closed


Labels:

Comments:
I don't think this should work at all w/o jinja2

just raise a nice error 

Are you talking about the warnings? those were just so I could see them in the Travis logs (wasn't sure when setUpModule gets called). It errors as soon as you `df.style` if you don't have Jinja.

forgot to mention, my last run succeeded. I'm building from my real branch [here](https://travis-ci.org/pydata/pandas/jobs/91149197). It would be nice to remove the globals and stuff in `test_style` though.

I am a tad confused about the link between mpl and jinja...

Well I'm extremely confused by it :). I haven't been able reproduce it locally, but from my other PR I found out that _trying_ to import Jinja, whether it's installed or not, was was what caused the failures, which makes no sense at all. It's [these](https://github.com/pydata/pandas/blob/master/ci/requirements-2.7_LOCALE.run) requirements if your curious (that's matplotlib 1.2.1, so you might not be).

@TomAugspurger keep open?
PERF: Faster Series construction with no data and DatetimeIndex.
State: closed

closes pydata/pandas#11433

Code. taken from @jreback comment on pydata/pandas#11433

asv benchmark:

```
    before     after       ratio
  [6c4d2c7d] [6439bd94]
-     1.68s   364.81μs      0.00  series_methods.series_constructor_no_data_datetime_index.time_series_constructor_no_data_datetime_index
SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
```


Labels:
name: Datetime
description: Datetime data dtype
name: Performance
description: Memory or execution speed performance

Comments:
lgtm. add a whatsnew note (perf section).
of course if you can make the code I did better / simplier, pls do so.

ping when note added, ready, and green.

done, done, and done ;)

minor comments, ping when green.

changes made, waiting for Travis again

green. ping @jreback 

thanks!