BrandonFors/pandas-github-issues huggingface dataset:
BUG: fixed df.info() output missing new line
State: closed

Had this minor bug working with pandas from the command line

```
>>> df.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 891 entries, 0 to 890
Data columns (total 12 columns):
PassengerId    891 non-null int64
# ... etc
Embarked       889 non-null object
dtypes: float64(2), int64(5), object(5)>>>
```

(The interpreter prompt is not on a new line at the end.) 

This most likely went unnoticed because pandas users tend to work in the ipython interpreter.

I had to change 2 tests, because they were not expecting the extra newline (using split('\n') and counting strings to determine if output was valid) and I don't know if that is a big issue or not.


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
can you squash these commits together pls
even though very minor, can you put a release note in v0.15.0

minor comment, pls rebase and ping when green

ping

this looks correct to me

@jorisvandenbossche @TomAugspurger 
cc @sinhrks

Yep; I should have noticed this when I was messing with the info repr.

Thanks @BorisVerk
DOC: setuptools is a required runtime dependency
State: closed

Without setuptools pandas fails to import at https://github.com/pydata/pandas/blob/master/pandas/io/gbq.py#L9


Labels:

Comments:
hmm I think easier to simply eliminate the need for pkg_resources - not sure why this is in their in the first place 

cc @jacobschaer 

can u just do gflags.**version** (or similar)?

cc @jacobschaer ?

@jreback I seem to recall Google was doing something weird with their version strings before. I'd have to look to see if that's still the case. Isn't setuptools a requirement for pandas anyways? @sean-schaefer

@jacobschaer `setuptools` is only a requirement of building from source. not a run-time (on windows). 

Any way to eliminate its usage (e.g. using LooseVersion)?

@jacobschaer ?

Was looking into this - I'm having some account issues with GBQ for integration testing, but it looks like using:

```
if LooseVersion(apiclient.__version__ >= '1.2.0'):
```

works. However, that's only part of the `google-api-python-client` package. As for gflags, a quick look shows no easy way to identify it's version programatically. However, in light of https://github.com/pydata/pandas/issues/8327, it may no longer be needed...

@jacobschaer 

I don't mind that setup tools is require for gbq
just don't want it necessary on main importing of pandas
can u move this detection so that it would fail w/ o setuptools ?
and then add setuptools to install.rst

@jacobschaer status?

During testing, I exceed a rate limiter in BQ (my bad - had two tests running at once on accident). I'll re-run the integration test later. I haven't tried removing setuptools from the virtual enviroment, but I can't see why this wouldn't work.  https://github.com/pydata/pandas/pull/8409

Integration test finally passed locally. Still need to validate that it actually solved the problem... anyone want to try it? :D

closing in favor of #8409
CLN: Partially add future division imports
State: closed

Adds division import to cases where something is already imported from **future** . #5445


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
It seems some trailing whitespace also got trimmed.

this seems right but did anything fail?

Passes locally (2.7.8), and Travis just came back green.

what I mean is this doesn't make a difference 
why are we doing this?

maybe just close the issue?

I haven't looked at all of the potentially-affected files (a quick, crude grep shows about 70 other files would need to be looked into). I was just trying to do something quick that would help with the referenced issue. No protests from me, if you would rather close. 

my point is what does importing division do?
as everything is truediv at this point anyhow
what does it change? is it necessary?

Ah. I did not know everything was truediv. In that case, probably best to close. My apologies.

If you can produce a situation where this import actually changes things I think would be good to know. IIRC the purpose of this was to make the _tests_ always use true-div. can you reread the original issue and see. thxs

@jtratner can you remind again what the necessity for importing `division` in each file is?

not totally convinced this actually matters.

It's not necessary, we fixed it to do true division where required ages ago
and now it would have zero impact.

On Thu, Mar 5, 2015 at 3:44 PM jreback notifications@github.com wrote:

> not totally convinced this actually matters.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/pydata/pandas/pull/8106#issuecomment-77477544.
TST: Moving testing method outside of TestCase
State: closed

Make assert_numpy_array_equal available without subclassing TestCase; see #8023


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
If this is not along the lines of what you have in mind for #8023, please feel free to close this.

this is fine

the idea is that all module level functions should be available on TestCase (and vice versa)

you can prob do this automatically 

by creating methods for all assert_\* in the module on TestCase

@jreback is this OK to merge?

I think the automatic adding of methods to TestCase can be done later in another PR, this is just adding the two test functions to the main testing namespace

@bwignall Can you squash the two commits?

yep this is ok

@bwignall sorry, I forgot this PR, and it is now already merged as part of another one, see  #8923 
But thanks for your work!
Added warning about attribute access on 'reserved' words
State: closed

closes https://github.com/pydata/pandas/issues/8082


Labels:
name: Docs

Comments:
@FragLegs can you update?

Sorry for the delay - been on vacation.

Should we also have a warning on creation (or a different Attribute Error)?

maybe this should be a warning on creation?  I.e.:

``` python
>>> DataFrame(some_data, columns=['labels', 'val', 'fun'])
UserWarning: Using reserved column name `labels` will be inaccessible by `getattr` calls - you must use `[]` instead.
```

or maybe we could raise a differently phrased attribute error:

``` python
>>> df = pandas.DataFrame(columns=['labels'])
>>> df['labels']
Series([], name: labels, dtype: object)
>>> df.labels
AttributeError: 'DataFrame' object has no indexer 'labels'.
```

though I'm not sure that's clearer :P .  (see jtratner/pandas@c1cee7daad838aa434a69450f032846288514eef for the potential diff)

Would that error message have helped @FragLegs ?

@jtratner I'm not sure. If I had gotten that error message, I would have still been left wondering, "Why doesn't it have a labels attribute? I know it's there." The warning on creation would be more helpful, or perhaps an error message that has some of that warning's flavor:

```
AttributeError: 'DataFrame' object has no indexer 'labels' - use `[]` instead.
```

Of course that would require checking to see if the attribute does exist as a column during the error handling, which feels a little hacky. I do agree that it would be helpful to have something more in the error or warning messages though, especially since searching the documentation for these keywords will probably not get you to the needed information very easily.

@FragLegs can  you squash into a single pls. ping when green.

well, whatever error message we pick, we can put the exact text in the docs
so it comes up if you search on Google.

merged via https://github.com/pydata/pandas/commit/9b12ccbcf2bc5893dcca262c81ac5dc28096c682

thanks!
TST: Fix timezone test to avoit dateutil issue
State: closed

Closes #7993.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Windows
description: Windows OS
name: Timezones
description: Timezone data dtype

Comments:
this doesn't seem to fix the test on windows though.....

```

C:\Users\Jeff Reback\Documents\GitHub\pandas>git checkout sinhrks-gh7993
Switched to branch 'sinhrks-gh7993'

C:\Users\Jeff Reback\Documents\GitHub\pandas>git diff master
diff --git a/pandas/tseries/tests/test_tslib.py b/pandas/tseries/tests/test_tslib.py
index 2cd8539..cdd5170 100644
--- a/pandas/tseries/tests/test_tslib.py
+++ b/pandas/tseries/tests/test_tslib.py
@@ -37,7 +37,7 @@ class TestTimestamp(tm.TestCase):
         import pytz
         import dateutil
         timezones = [(None, 0), ('UTC', 0), (pytz.utc, 0),
-                     ('Asia/Tokyo', 9), ('US/Eastern', -4), ('dateutil/US/Pacific', -7),
+                     ('Asia/Tokyo', 9), ('US/Eastern', -4), ('dateutil/America/Los_Angeles', -7),
                      (pytz.FixedOffset(-180), -3), (dateutil.tz.tzoffset(None, 18000), 5)]

         for date_str, date, expected in tests:

C:\Users\Jeff Reback\Documents\GitHub\pandas>c:\python27-64\python.exe setup.py build
running build
running build_py
copying pandas\version.py -> build\lib.win-amd64-2.7\pandas
copying pandas\tseries\tests\test_tslib.py -> build\lib.win-amd64-2.7\pandas\tseries\tests
running build_ext
skipping 'pandas\index.c' Cython extension (up-to-date)
skipping 'pandas\algos.c' Cython extension (up-to-date)
skipping 'pandas\lib.c' Cython extension (up-to-date)
skipping 'pandas\tslib.c' Cython extension (up-to-date)
skipping 'pandas\parser.c' Cython extension (up-to-date)
skipping 'pandas\hashtable.c' Cython extension (up-to-date)
skipping 'pandas\src\sparse.c' Cython extension (up-to-date)
skipping 'pandas\src\testing.c' Cython extension (up-to-date)
skipping 'pandas\msgpack.cpp' Cython extension (up-to-date)

C:\Users\Jeff Reback\Documents\GitHub\pandas>c:\python27-64\Scripts\nosetests.exe build\lib.win-amd64-2.7\pandas\tseries\tests\test_tslib.py
..........F....................
======================================================================
FAIL: test_repr (pandas.tseries.tests.test_tslib.TestTimestamp)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeff Reback\Documents\GitHub\pandas\build\lib.win-amd64-2.7\pandas\tseries\tests\test_tslib.py", line 162, in test_repr
    self.assertIn(tz_repr, repr(date_tz))
  File "C:\Users\Jeff Reback\Documents\GitHub\pandas\build\lib.win-amd64-2.7\pandas\util\testing.py", line 143, in assertIn
    assert a in b, "%s: %r is not in %r" % (msg.format(a,b), a, b)
AssertionError: : '/US/Pacific' is not in "Timestamp('2014-03-07 00:00:00-0800', tz='dateutil/America/Los_Angeles')"

----------------------------------------------------------------------
Ran 31 tests in 1.407s

FAILED (failures=1)

C:\Users\Jeff Reback\Documents\GitHub\pandas>
```

Sorry, made mistake. Fixed.

thanks!
ENH: add support for datetime.date/time in to_sql (GH6932)
State: closed

Closes #6932

Support for writing `datetime.date` and `datetime.time` object columns with `to_sql`.

Works nicely for the sqlalchemy mode, for sqlite fallback it writes OK for `datetime.date`, but comes back as object strings (this is also the case for datetime)

Remaining question:
- when reading the data back in `TIME` columns are converted back to `datetime.time`, but `DATE` columns are converted to `datetime64` and not `datetime.date`. Is this OK? Or better try to convert to `datetime.date`?
- `datetime.time` does not work with sqlite fallback for some reason (`datetime.date` does)


Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
@danielballan @mangecoeur @artemyk any comments?

@jorisvandenbossche Only thing is that this will probably need a manual merge with PR #8083 (which changes _get_column_names_and_types to return an array of tuples like `[(colname, coltype, is_index), ...]` , not just `[(colname, coltype), ...]` .  Or I can try to rebase PR #8083 off this.

@artemyk you are correct, but we will just see which gets merged first, and then the other has to rebase. No need to do in advance.

Looks good to me
ENH: add support dtype='category' in Series constructor
State: closed

xref #8074
closes #8076

```
In [1]: Series([1,1,2,2,3,4,5],dtype='category')
Out[1]: 
0    1
1    1
2    2
3    2
4    3
5    4
6    5
dtype: category
Levels (5, int64): [1 < 2 < 3 < 4 < 5]
```


Labels:
name: Enhancement
name: Categorical
description: Categorical Data Type

Comments:
Apart from the naming LGTM

@JanSchulz any other comments? (better to merge smaller fixes that large ones!)

@JanSchulz though in doc revisit, maybe add about creating categorical Series with
`Series(.....,dtype='categorical')` somewhere

no, this is ONLY called internally, going to ignore the passed in fill_value etc, which don't matter. These routines are pretty generic and meant for generic fill value substitution (but here we are doing it only on the codes which are by definition numeric), so only -1 matters.

I think we end up with a complete rewrite/revise of the docs, so I wouldn't touch that until after. If you have something specific, just add a line in #8078

Then an `assert not kwargs`, so we see that in the unittests?

fixed up todo list....lt me see about the kwargs (they are passed on thru anyhow)

Looks good.
Added chunksize argument to to_sql
State: closed

Large dataframes may fail to write to a database in one go due to packet size errors .  This add a `chunksize` parameter which will write the dataframe to the database in batches.  See https://github.com/pydata/pandas/issues/7347 .

Notice that there is quite a bit of code duplication between the SQLAlchemy and legacy APIs. This could be factored out if desired at the cost of a few private methods.

Closes #7347


Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
Thanks for the PR! Looking good at first sight, will try to do a more detailed review tomorrow.

Just something to consider --- it might be better to strip out the transaction handling code and just document the fact that batch writing is non-atomic.  It seems like sqlite, at least, does not support transactions for DDL statements like `CREATE TABLE ...` , so the transaction code adds cruft but does not fulfill its intended purpose.

@jorisvandenbossche thanks for the comments, I implemented the suggested fixes.  The reason for using Session() was some weirdness with sqlalchemy 0.7 (specifically, Engine does not support contextmanager), but now I got around it using only low-level elements.

About the version, if you need a newer version for something, we always can consider setting a higher minimum required version. In any case, we will have to raise the bar to 0.7.4 to support schema's.

For the PR, added two comments. Looking good for the rest! And further:
- needs a whatsnew entry (in enhancements)
- can you add also a small section in the docs? (in the "Writing DataFrames" section (http://pandas.pydata.org/pandas-docs/stable/io.html#writing-dataframes)) For example add in short something like "If you the connection times out, you can solve this by using the chunksize .."
- squash the commits

Thanks a lot!

Let me know regarding changing sqlalchemy required version vs. leaving transaction code as-is and I will make changes if needed and then squash the commits.  Thanks greatly!

apart form the formatting comments, and the one comment above on `keys` (https://github.com/pydata/pandas/pull/8062#discussion_r16560233), I think this is ready to squash and merge

@danielballan @hayd @mangecoeur of someone else wants to give this a look, certainly welcome

Removed keys (good catch), fixed doc changes, and squashed the commits.

Thanks a lot!
ENH: improve datetime string parsing with CustomBusinessDay
State: closed

Currently CustomBusinessDay requires a strict format for holiday definition

```
CustomBusinessDay(holidays=['2014-05-12'])
```

but the following would raise a ValueError:

```
CustomBusinessDay(holidays=['5/12/2014'])
```

this PR makes the date parsing easier and adds a unit test


Labels:
name: Frequency
description: DateOffsets

Comments:
hmm, this is going to be a perf issue I think maybe. Not sure if their are perf tests already for this. Can you investigate and/or add?

 This takes a `calendar` option for a created set of the `Holiday`. Is that not sufficient?

I'm not sure how the `calendar` option works when I'd like to define my own custom list of holidays. The example I see in the docs is with a predefined `USFederalHolidayCalendar`. That's why I'm passing an explicit list. Also in general it seems a bit weird that the `holidays` option requires a specific datetime format.

I can look into the performance impact, will keep you posted. Thanks. 

Take a look at the [documentation for holidays](http://pandas.pydata.org/pandas-docs/stable/timeseries.html#holidays-holiday-calendars).  The predefined calendar is merely there as an example.

@mortada can you revisit this? I would rather see this in the contstruction methods of the holidays than here.

I think this is covered by the `calendar` option, would accept an example / docs that clarify things though

@jreback I agree, this should be closed, thanks
BUG: rolling_count() and expanding_*() with zero-length args; rolling/expanding_apply with min_periods=0
State: closed

Closes https://github.com/pydata/pandas/issues/8056
Closes https://github.com/pydata/pandas/issues/8080


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
rebase this

Rebased.

this is fine,

can you fix the docs in whatsnew here:

http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html
(the change about the `center`)

you need to adjust the `` somewhere

Fixed, I think. I guess it doesn't like non-whitespace following closing double quotes.

ok, ping on green

Green.

Updated to fix #8080.

thanks, merged
TST: Fix tseries.converter test for MPL1.4
State: closed

Closes #7233.

Other tests have no problem with MPL1.4.dev:
- tests\test_graphics.py
- tseries\tests\test_plotting.py


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Visualization
description: plotting

Comments:
@cpcloud can we add mpl 1.4rc to the conda build? (not sure if you can do that)

@sinhrks this is good thanks

@sinhrks pls rebase and lmk when green (as I fixed the options error issue)

I'm looking into getting a way for us to test pandas against dev versions of anything we want (they'll have to be conda packages but that not a big deal)

@jreback Rebased, and now green.
BUG: stack with datetimes
State: closed

closes https://github.com/pydata/pandas/issues/8039

The only change I made was to select from the MultiIndex itself, rather than the underlying `.values` attribute. The test set passed but I want to look at this a bit more closely to make sure everything is ok.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex

Comments:
@TomAugspurger need some tests
Fix testing multiindex dtypes in assert_frame_equal and assert_series_equal
State: closed

When using `assert_frame_equal` or `assert_series_equal` in the presence of a multiindex, the dtype of the index as a whole is `object`.  Hence, there is no checking that the dtypes of the individual levels match.  We now iterate through the levels.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
let's see if anything breaks (on the regular tests!)

Sounds good (at least locally on my end all the tests passed)

nice addition! thanks!
BUG: Don't call np.roll on empty arrays.
State: closed

On some versions of numpy (such as 1.7, but not 1.8), this throws a
ZeroDivisionError.

Fixes #8019.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
add a note in v0.15.0 to bug fixes (with the issue number)

Updated.

perfect, ping on green.

OK, this should be good to go. I'm seeing consistent failures of TestYahooOptions, an external network test. I don't think this is related to these changes.

ok, pls rebase on master and ping when green (the other options issue is fixed)

@jreback it's green!

@Wilfred gr8 thanks!

someone pointed me to your package: https://github.com/ahlmss/mdf

neat!
DOC/CLN: Cleanups plotting.py
State: closed

- Some refactoring which should not affects to output.
- Made `DataFrame.plot` and `Series.plot` should have consistent docstring except differences.


Labels:
name: Docs
name: Visualization
description: plotting

Comments:
@sinhrks Docstrings are looking nice! Made some comments (and note that a lot of them are comments on the existing docstrings, but thought was a good opportunity to look at that too)

I did not yet really look at the actual code refactor, but the only think I was worrying a bit is: you changed the order of keyword arguments, that is right? Now you put `kind` second (which is very logical), before `x` and `y`. But I was wondering how many people would rely on the position of `x` and `y`? (for the others that doesn't matter I think, but for those first keyword, it does maybe)

Thanks, fixed the points. And agreed to hold `x` and `y` in the current arg order.

@sinhrks Can you rebase this? @TomAugspurger Can you also have a look?

Yep, rebased.

This cleanup really needed to be done, thanks @sinhrks. Just had a few inline comments. There's also a few lines in the docstring that are longer that 80 characters, but if this is a formatting thing so that it looks nice in the REPL, then no problem.

@sinhrks pls rebase

@sinhrks I can merge once you get a chance to rebase. Let me know.

@sinhrks status?

@sinhrks will you have a chance to do this in the next couple days? If not I can cherry pick things. I'd like to get all the vis PRs done by tomorrow.

@TomAugspurger let's either cherry-pick or push to 0.15.1 whatever you can do in next couple of days
DOC: Fixing 'chunksize' parameter name typo io.rst
State: closed

Fixing an error in the 'chunksize' parameter name for the io doc.  With the typo (parameter called 'chunsize'), the sample code from the documentation fails silently / confusingly - returns an iterator over individual dataframe elements rather than dataframe chunks.


Labels:
name: Docs
name: IO HDF5
description: read_hdf, HDFStore

Comments:
@nesanders good point (and not why it was in a code-block originally, don't remember). (that of course is not caught in the doc build!)
BUG: fix py3 read_html bytes input
State: closed

closes #7927


Labels:
name: Bug
name: Testing
description: pandas testing functions or related to the test suite
name: IO Data
description: IO issues that don't fit into a more specific label
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
@jreback any comments?

ok