BrandonFors/pandas-github-issues huggingface dataset:
Add override_header_style option to data_frame.to_excel()
State: closed

[Pandas v0.16.0]

The Excel export (`data_frame.to_excel`) currently hard codes cell styles for all header frames.  This pull request allows this style to be overridden by the user.

Don't override the default header style (Default)
`data_frame.to_excel(..... "override_header_style":None)`

Don't style the header at all
`data_frame.to_excel(..... "override_header_style": {} )`

Pass in a custom style
`data_frame.to_excel(..... "override_header_style": { "font" : {"font_color" : "#990000", "bold" : False }, "align" : {"horizontal" : "right"} } )`

There are currently no tests for the to_excel function mentioned here.  (All the other, existing tests pass.)  I'd be happy to write some tests, though it might mean refactoring some functionality out of constructors as **init** is notoriously difficult to mock.  (mock.assert_called_with ?)


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: IO Excel
description: read_excel, to_excel

Comments:
does this work for all engines? can you post the effect for overriding the styles? (e.g. show an example and the excel output as a screenshot)

closing as stale, but if you would like to reopen and fixup pls do.
Series string  operators
State: closed

Makes the string operators available to the `Series.str` object

``` python
In [1]: a = pd.Series(['%s', '%s'])

In [2]: a.str % 2
Out[2]: array(['2', '2'], dtype=object)

In [3]: a = pd.Series(['a', 'b'])

In [4]: a.str * 2
Out[4]: array(['aa', 'bb'], dtype=object)

In [5]: a.str + 'test'
Out[5]: array(['atest', 'btest'], dtype=object)
```


Labels:

Comments:
This already works on the series itself, eg 

```
In [22]: a = pd.Series(['a', 'b'])

In [23]: a * 2
Out[23]:
0    aa
1    bb
dtype: object
```

so why would you also want it on `s.str` ?

Oh man, sorry.

No problem!
Change of index values dtype should be reflected in index type
State: closed

This PR should eventually address issue #9966. I will look at testing first to make sure I have understood how pandas ought to work in these situations.


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
Tests should fail for now.

@jreback Do you agree that the inplace multiplication in 6279449 should raise an Exception? I guess what I'm really asking is if there is a way an object can change its type in response to an inplace operation? There might be, because this works for simple ints and floats.

in place are disable as Index are disabled anyhow
so u can test for this but it shouldn't work (eg test that it raises)

want to update for 0.17.0?

can you update?

Sorry I haven't gotten back to you before. I'm afraid I'll be to busy to be
able to finish the remaining parts here.

2015-06-27 1:34 GMT+02:00 jreback notifications@github.com:

> can you update?
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/pydata/pandas/pull/9974#issuecomment-115917725.

closing in favor of #10638
TST: Check index names
State: closed

Derived from #9862. Added `check_names` kw to followings testing functions:
- `assert_index_equal` (default=True).
- `assert_series_equal` (default=True). Maybe the same option can be used to check `Series.name` in the future, otherwise it may better to choose separate name such as `check_index_name`. 
- `assert_panelnd_equal` (default=False). We have to check following ToDo to specify default=True.
- https://github.com/pydata/pandas/blob/master/pandas/util/testing.py#L771
- `assert_frame_equal` is untouched as it already has the kw. Not refactored the logic because it has a path which doesn't use `assert_index_equal` internally. 

Looked through test cases which have to be `check_names=False`, and added name check tests.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
@sinhrks thanks!
TST: adding sdist and install tests to the 2.6 regression - closes #9878
State: closed

closes #9878

Adds a test for packaging and installing pandas from a tarball, inside of the 2.6 regression.

After regular nosetests complete, cython is removed from the venv, pandas is packaged with cdist command, and the package is installed via pip. Limited nosetests run once installed.

The output of this test in the log is now 'folded' via http://blog.travis-ci.com/2013-05-22-improving-build-visibility-log-folds/ by putting install_test.sh into the 'after_script' section of the travis config.

This is a new PR with squashed commits to replace the original closed PR: https://github.com/pydata/pandas/pull/9922


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Build
description: Library building on various platforms

Comments:
@ajamian awesome thanks!
BUG: DataFrame constructor fails when columns is set and data=[]
State: closed

From this StackOverflow question: http://stackoverflow.com/questions/29730488/setting-columns-for-an-empty-pandas-dataframe


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
pls run a perf check on this and report any anomalies.

None of the tests have a ratio reproducibly above 1 (i.e., increasing repeats on the worst performers gets rid of them all).

merged via https://github.com/pydata/pandas/commit/514fe2d052feb06287e451488ca35e44ca12d423

thanks!

note that you can generally avoid rebase conflicts in the release notes by putting the note NOT at the bottom, e.g. just put it in the Bug Fix list somewhere where there is space left
add support for quandl historical financial data
State: closed

| Q | A |
| --- | --- |
| Bug fix? | no |
| New feature? | yes |
| BC breaks? | no |
| Deprecations? | no |
| Tests pass? | yes |
| Fixed tickets | - |
| License | BSD |
| Doc PR | - |


Labels:

Comments:
Getting a few Travis build errors trying to install dependencies that don't seem related to this PR?

```
Error: HTTPError: 403 Client Error: Forbidden: http://conda.binstar.org/pandas/linux-64/beautiful-soup-4.2.1-py27_0.tar.bz2
```

```
Error: HTTPError: 403 Client Error: Forbidden: http://conda.binstar.org/pandas/linux-64/bottleneck-0.8.0-np18py33_0.tar.bz2
```

Sumit this PR to https://github.com/pydata/pandas-datareader/issues

once that is merged we could consider porting back to main pandas 

in 0.17.0 all of the datareader is going to be ripped out and pandas-dataeader  will be the source for things like this
Update excel.py to output OrderedDict instead of dict
State: closed

It should fix https://github.com/pydata/pandas/issues/9930


Labels:

Comments:
This does seem like a good idea. Can you write a test to verify that it works?

Sorry but I don't understand why CI raises this error for Python 2.x

```
Error: HTTPError: 403 Client Error: Forbidden: http://conda.binstar.org/pandas/linux-64/bigquery-2.0.17-py27_0.tar.bz2
```

I also don't understand why "OrderedDict should be imported from `pandas.compat` instead to ensure backwards" because according https://docs.python.org/2/library/collections.html collections exists since Python 2.4 "New in version 2.4." but maybe collections is existing since 2.4 but not OrderedDict.

I noticed that a pip package exists https://pypi.python.org/pypi/ordereddict but I don't know how to automatically install it only for Python < 2.7 (so 2.6 in our case)

I think that CI failure is something spurious related to binstar. I just retriggered the Travis tests for this PR -- hopefully we won't see that error message again.

As for OrderedDict, indeed it first appeared in Python 2.7. We already have a copy of OrderedDict bundled with pandas for exactly this reason. To use it, just use `from pandas.compat import OrderedDict`.

Follow-up: #9944
Allow add_categories() to accept Series/np.array and make raising on dup...
State: closed

...licates optional, fixes #9927


Labels:
name: Bug
name: API Design
name: Categorical
description: Categorical Data Type

Comments:
@mortada ok, for now pls take out the `raise_on_duplicates` kw and leave the rest.

@jreback  it's updated 

@mortada thanks!
BUG: need better inference for path in Series construction (GH9456)
State: closed

closes  #9456.


Labels:
name: Bug
name: Datetime
description: Datetime data dtype
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
I rewrote the fix to be more generic. If the type of Index elements is the same as the type of data elements, we use the Index to access and copy the dict value. If not, we use the original index. See comments in diff.

this overlaps with #10269 (your PR attempts to fix Series the other DataFrame)

superseded by #10269 

thanks!
Added grouby apply notebook
State: closed

Here is a notebook I created with help at the pandas dev sprint in Montreal.  

Its meant to help document the behavior in issue 9867: It's difficult to predict what DataFrame.groupby().apply() will return

https://github.com/pydata/pandas/issues/9867


Labels:

Comments:
@mbirdi could you put this notebook in a [gist](https://gist.github.com/) share a link to the preview on [nbviewer](http://nbviewer.ipython.org/)? I don't think we actually want to merge this into the pandas repo :).

@shoyer 

Thanks for the tips.  I used gist and and nbviewer.  It looks like it worked from my side.  I think this link should be viewable by others.

http://nbviewer.ipython.org/gist/mbirdi/05f8a83d340476e5f03a

@mbirdi OK, thanks! I added a link to that notebook from the original issue.
TST: adding sdist and install tests to the 2.6 regression - closes #9878
State: closed

closes #9878 

This PR adds a test for packaging and installing pandas from a tarball, inside of the 2.6 regression.

After regular nosetests complete, cython is removed from the venv, pandas is packaged with cdist command, and the package is installed via pip. Limited nosetests run once installed.

The travis log output is a little messy with the pip command output; ideally I wanted to 'fold'/collapse the output unless there was a failure but I don't think you can do that from within the 'script' portion of the regression, only the before and after scripts, see:
http://blog.travis-ci.com/2013-05-22-improving-build-visibility-log-folds/
If there is a way to do this maybe we can add it later, or move where this INSTALL_TEST is happening to a before or after script.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Build
description: Library building on various platforms

Comments:
I _think_ that you can add another entry after the script (and put your change in), maybe something like. `install_test.sh` which will 'fold' it (I think if you run separate scripts it folds that output

This is in `.travis.yml`

```
script:
  - echo "script"
  - ci/script.sh
# nothing here, or failed tests won't fail travis
```

@jrebeck I gave this a try; it looks like log output will only fold if it is outside the 'script' section of the travis job (here is the log output a separate install_test.sh script: https://travis-ci.org/ajamian/pandas/jobs/58972085)

Is it ok if we put the install_test in say the after_scripts section? I think this would give the folding.

yep that's ok too

@jreback looks like folding works for stuff in the after_script section of travis - i put it before everything else in that section.
Add time_format argument to ExcelWriter.
State: closed

Currently, converting to Excel a DataFrame containing `datetime.time` instances leads to unformated float values such as `0.666667` instead of `16:00:00`. This pull request fixes this by adding a default time format and adds an argument to specify your own time format.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: API Design
name: IO Excel
description: read_excel, to_excel

Comments:
- needs tests

does this have an issue number?

@jreback OK for the tests. I guess I only need to modify [these lines](https://github.com/pydata/pandas/blob/7e3212edd642adfd916baf65691dbef5f7b03d0b/pandas/io/tests/test_excel.py#L843-880) to add a time series to `df` & `df_expected`, then add a `time_format='HH-MM-SS'` argument to the writer2 constructor?

There’s no issue related to this pull request. Do I need to create one? That seemed quite obvious to me that `time_format` has only been forgotten.

ok on the issue number, use this PR number as the issue number in the release note.

Would prefer you add a new set of tests (like those though)

@BertrandBordage can you update?

closing as stale, but if you would like to reopen and fixup pls do.
BUG: barplot with log=True not working for values smaller than 1
State: closed

Closes #9905. Also include a fix that `log` kw is ignored in horizontal bar plot. Test cases are based on standard mpl result:

```
plt.bar([0, 1, 2], [0.1, 0.01, 0.001], log=True)

plt.gca().get_ylim()
# (0.001, 0.10000000000000001)

plt.gca().yaxis.get_ticklocs()
# [  1.00000000e-04   1.00000000e-03   1.00000000e-02   1.00000000e-01
#    1.00000000e+00   1.00000000e+01]
```

Maybe `log` in bar plot and `logx` and `logy` kw should be merged in future...


Labels:
name: Visualization
description: plotting

Comments:
Looks good. Thanks.

@sinhrks Thanks a lot! Can confirm this fixes my issue!
DOC: add redirects to ensure old links instead of building the docstring page
State: closed

We still have a lot of API pages with 'old' namespace usage (eg `pandas.io.parsers.read_csv` instead of `pandas.read_csv` as how it should show up in the documentation). Now we build the pages as well, this PR changes this in adding redirect pages.

(The reason I care about this is that googling eg `read_csv` still often turns up the old page (although I don't know how fast this disappears when we would just remove it) and these are therefore used a lot on eg SO)


Labels:
name: Docs

Comments:
Does Google pick this up as a permanent redirect? In theory, I think that sort of thing is possible (redirecting the links in search results before users even click on them).

@shoyer Not really an idea, no expert at that. If there is a better solution, I am all ears (certainly as I would like to restructure the tutorial docs as well later on, which would make a lot of links invalid). 

I quickly did some google search, and you probably mean a 301 redirect (instead of the meta refresh I did here)?

Yeah, in theory I think a 301 redirect would be preferable. But it looks like that's only possible to do with server-side code, not inside an HTML page.

Depending on how the pydata server is hosted, I suppose we could add a .htaccess file with this: http://sphinx-doc.org/latest/config.html#confval-html_extra_path
add/delete str/dt/cat dynamically from __dir__ (fix for #9627)
State: closed

@jreback this PR makes it easier to add/delete items from `__dir__`. The `.str/.dt/.cat` are now only present in `__dir__` when they are appropriate types.

However, the IPython tab completion does not seem to only source the list from `__dir__`, and therefore even if an item is removed from `__dir__` it is still showing up in tab completion for me.

@shoyer I'd probably need some help/feedback with this. Here's a related ticket https://github.com/pydata/pandas/pull/9617 

Thanks.


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
Some thoughts on the design here: we could add a class attribute `_possible_accessors` which corresponds to a set of all possibly accessors (e.g., `Series._possible_acccessors = set(['cat', 'str', dt'])`). Then we can have something like the following generic method defined on `PandasObject`:

``` python
def _dir_deletions(self):
    deletions = set()
    for accessor in self._possible_accessors:
        try:
            getattr(self, accessor)
        except AttributeError:
            deletions.add(accessor)
    return deletions
```

@shoyer I like this idea! I noticed that there's already a `_accessors` attribute under `Series` though

```
_accessors = frozenset(['dt', 'cat', 'str'])
```

but it's not obvious to me how this is being used. 

With regards to IPython auto-completion, this does seem trickier than I thought. Take a look at this example:

``` python
class A(object):
    something = 'asdf'

    def __dir__(self):
        return ['other']


class B(object):
    def __init__(self):
        self.something = 'asdf'

    def __dir__(self):
        return ['other']

a = A()
b = B()
```

Autocomplete on `a` still picks up `something`, even though we excluded it from `__dir__` (on `b`, it's excluded). I'm guessing IPython uses `dir(A)` somehow.... might be worth asking over there, since I'm pretty puzzled at this point :).

Yes, looks like `_accessors` is probably the appropriate attribute to use then :). You'll have to do some searching to figure out how it's currently used.

@shoyer updated to incorporate your suggestions, now using `self._accessors` 
@jreback updated to make use of the fact that `.str/.dt/.cat` are mutually exclusive

lgtm

pls add a release note (use the pr number as the issue)

LGTM, though you might test the built-in function `dir` directly instead of using `__dir__`

@shoyer that makes sense, it's updated 

OK, looks good to me! Please ping when it Travis is green.

Too bad this won't work for IPython auto-complete yet... but at least they're working on it.

travis is green now

yeah I hope the IPython auto-completion will work soon

thanks!

great news - the issue has been closed on the IPython side, we should soon have the tab completion working
Fixed latex output for multi-indexed dataframes - GH9778
State: closed

Proposed fix for https://github.com/pydata/pandas/issues/9778

The formatting issue was caused by an incorrect number of elements in the (first) index columns of `strcols`. The length of reinserted columns was based on the number of elements per index level, but should have relied on the number of rows/occurrences of such elements.


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: MultiIndex

Comments:
This looks great to me. Can you please:
1. [add a note to what's new](http://pandas-docs.github.io/pandas-docs-travis/contributing.html#documenting-your-code)
2. [squash your changes into one commit](http://pandas-docs.github.io/pandas-docs-travis/contributing.html#committing-your-code) (we prefer not to have commits for which Travis tests fail merged into master, because it can complicate things like `git bisect`)

@shoyer: Thanks a lot for your feedback.

I've just added a note to the next release, and squashed all the commits. Do let me know if there's anything else that should be added/updated.

This looks to good to go, once the tests on Travis pass. I'll try to check later, but please feel free to ping me if you notice that's happened.

@shoyer: I'm not sure how Travis is set up on this repo, but [tests are passing](https://travis-ci.org/yred/pandas) on my fork.

Do we handle if there are `df.index.names` ?

Thanks for fixing this!

@hayd: That's a good point.

It currently doesn't print out index names. Would that be expected with this fix or should I add it in a separate PR?

At this point probably better to make a separate PR. But I do recall being able to export index names....

On Wed, Apr 15, 2015 at 6:11 PM, Yasin A. notifications@github.com
wrote:

> @hayd: That's a good point.
> 
> ## It currently doesn't print out index names. Would that be expected with this fix or should I add it in a separate PR?
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/pydata/pandas/pull/9908#issuecomment-93610178

@shoyer: it seems that regression happened a little earlier as well.

I'll be updating the code for that (separately), and potentially cleaning up `to_latex()` so that things are a bit clearer/cleaner.

@yred Fantastic, thanks!

thanks @yred !
str'ified column names in DataFrame.to_hdf(); Fixes #9057
State: closed

closes #9057

Converted column names to strings before creating pytables metadata object

This brute force `str()`ing method _might_ not be the best approach, but AFAIK it hasn't broken any of the tests.


Labels:
name: Bug
name: IO HDF5
description: read_hdf, HDFStore

Comments:
Yeah, my bad, that makes sense. I moved them over.

the reason for not allowing this is that queries become non-obvious and the data does not correctly round trip

so I suppoose this is ok but needs a warning to show that this is losing data

pls do the tests like the others, e.g. use `ensure_clean_path` which handles the cleanup in the correct way

pls add a warning when things are stringified as well (as this is generally something which is not obvious and makes querying different).

Do you think that it's better to just straight-up disallow `to_hdf()` on `DataFrame`s with integer column names? If there are numerous places in the code where it's assumed that a column name can be mapped to a Python identifier, then simply working around the problem (as in this PR) may just be asking for a world of hurt. Thoughts?

I think it might be better to raise a more informative error message

the point of the Table format is to query and so have integer names though can work is a bit odd
(the actual exception comes from PyTables though as it cannot hold integer names) 

@jameshiebert I think let's just raise an error message. Pls rebase and update.

Sounds good. Closing this in favor of #10098
Use 'python-dateutil' instead of 'dateutil' for requirement files
State: closed

The former also is known to the 'pip install' command, and still works with conda install. Making requirements files compatible with 'pip install' is more relevant now that packages like numpy have a (binary) wheel available.


Labels:
name: Enhancement
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
these are tailored to conda installs, so this will not work

This _will_ work, because `conda` accepts `python-dateutil`.

@chrish42 pls ping when this is green (I restarted the jobs).

hmm, sees `python-dateutil` not defined in the default conda channel....?

Works now with conda:

```
$ conda create -n test python-dateutil
Fetching package metadata: ....
Solving package specifications: .
Package plan for installation in environment /Users/chrish/anaconda/envs/test:

The following NEW packages will be INSTALLED:

    openssl:         1.0.1k-1
    pip:             6.1.1-py27_0
    python:          2.7.9-1
    python-dateutil: 2.4.2-py27_0
    readline:        6.2-2
    setuptools:      15.2-py27_0
    six:             1.9.0-py27_0
    sqlite:          3.8.4.1-1
    tk:              8.5.18-0
    zlib:            1.2.8-0

Proceed ([y]/n)?
```

hmm, can you rebase on master and push again?

Rebased, but two of the builds are still failing.

@chrish42 so the older versions of dateutil fail here because they don't exist on conda.

They _can_ be built and posted to the `pandas` binstar channel. But would need to do for all arches. Alternative is to not change those for those builds in the ci file.

Change for versions where python-dateutils is not available undone. Also rebased.

@chrish42 thanks!
ENH: repr now shows index name #6482
State: closed

Let me know if that is not good enough, I played around with indenting it but it doesn't look good with big indexes.

This, current:
![screen shot 2015-04-14 at 11 06 30 am](https://cloud.githubusercontent.com/assets/1778723/7139869/836ff68e-e296-11e4-89d9-9512df67b150.png)
vs with indention
![screen shot 2015-04-14 at 11 07 18 am](https://cloud.githubusercontent.com/assets/1778723/7139870/8370d48c-e296-11e4-8fc6-65d5b9fe725d.png)


Labels:
name: Enhancement
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
closing in favor of #9901