BrandonFors/pandas-github-issues huggingface dataset:
TST/CLN: remove np.assert_equal
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`

Not to use `np.testing.assert_equal`, as `assert_numpy_array_equal` is more strict to check dtypes.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
I think we need to add something to maybe the lint.sh script to prevent anything np.testing from slipping back in

maybe a recursive grep would work?

I think we could do something like this; if these are fixed (then just test if this grep outputs any lines, if so fail the lint.sh)

```
[Mon May 23 19:33:26 ~/pandas]$ grep -r --include '*.py' --exclude nosetester.py 'from numpy.testing import' pandas 
pandas/computation/tests/test_eval.py:from numpy.testing import assert_allclose
pandas/io/tests/json/test_ujson.py:from numpy.testing import (assert_array_almost_equal_nulp,
pandas/stats/tests/test_var.py:from numpy.testing import run_module_suite, TestCase
pandas/tests/test_graphics.py:from numpy.testing import assert_allclose
pandas/tseries/tests/test_converter.py:from numpy.testing import assert_almost_equal as np_assert_almost_equal
pandas/tseries/tests/test_timedeltas.py:from numpy.testing import assert_allclose
```

thanks @sinhrks nice cleanup
BUG: remove_unused_categories dtype coerces to int64
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

```
c = pd.Categorical(['a', 'b'], categories=['a', 'b', 'c'])
c.codes
# array([0, 1], dtype=int8)

# NG, must be int8 dtype
c = c.remove_unused_categories()
c.codes
# array([0, 1])
```

It is because `np.unique` uses platform int for `unique_inverse`

```
np.unique(np.array([0, 3, 2, 3], dtype=np.int8), return_inverse=True)
(array([0, 2, 3], dtype=int8), array([0, 2, 1, 2]))
```


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Categorical
description: Categorical Data Type

Comments:
thank you sir!
ENH: Allow to_sql to recognize single sql type #11886
State: closed

Follow-up in https://github.com/pydata/pandas/pull/13614

---

This solves #11886 
It checks whether the passed dtype variable is a dictionary.
If not, it creates a new dictionary with keys as the columns of the dataframe.
It then passes this dictionary to the pandasSQL_builder.


Labels:
name: Enhancement
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
Could someone please tell me why this build failed? (https://travis-ci.org/pydata/pandas/jobs/132018332)
I can't find any error in the logs.

you have pep issues

https://travis-ci.org/pydata/pandas/jobs/132018332

`pandas/io/tests/test_sql.py:2050:80: E501 line too long (80 > 79 characters)
pandas/io/tests/test_sql.py:2051:53: E127 continuation line over-indented for visual indent
`
How should I fix these lines?
https://github.com/pydata/pandas/pull/13252/files#diff-052a926a9b2c33a84e27d51c8fad3466R2050

@RahulHP Is there a reason you closed this?

To clean up the unwanted commits here, you can do:

```
git fetch upstream/master
git rebase upstream/master
git push -f origin/master
```

I thought someone else would be better equipped to handle this. I have tried it but I seem to have trouble with the rebase related commands.

I'll try it again now.

@RahulHP You still have some commits that should not be in here. Normally, by doing exactly:

```
git fetch upstream/master
git rebase upstream/master
git push -f origin/master
```

it should be OK. If this does not work, can you show the output of those commands?

@RahulHP  Do you have time to update this?

No I don't, sorry for the trouble.

@RahulHP No problem, I will rebase this for you then, as it was a good enhancement!

Rebased in https://github.com/pydata/pandas/pull/13614
TST/ERR: Add Period ops tests / fix error message
State: closed

- [x] related to #13242

closes #13251
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`

This must be after #13079.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Period
description: Period data type

Comments:
merged #13079 
see my comments on #13251 (may already be covered here)

thanks @sinhrks all of these tests are really helpful!
Fixes resampler for grouping kw bug #13235
State: closed

- [x] closes #13235
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Resample
description: resample method

Comments:
- needs tests
- add whatsnew entry (bug fix)

test & whatsnew pls

thanks @roycoding
Correct ValueError invalid type promotion exception
State: closed

- [x] closes #12599
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
Please review

add a whatsnew. you should always add a whatsnew and tests on the very first push. that way I can look at an already done PR and just comment on changes, rather than having to go back and forth so much.

@jreback The reason I'm not adding a whatnews early, as it conflicts (very) frequently ... I will rebase/squash in a minute.
Maybe whatsnew could be generated from the commit logs (instead of constantly rebasing on the same section of a single file)?

I thought I explained this

first it doesn't really matter if he whatsnew conflicts - usually these go thru several iterations

next you simply put them NOT AT THE END then you won't have conflicts - this is the reason for the blank lines in the bug fixes section

Added whatsnew

thanks @gliptak
BUG: DataFrame.align bug with join keyword (GH13914)
State: closed

- [x] closes #13194
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: API Design

Comments:
@jreback Is this okay?

in the future, simply push to the SAME PR, don't open new ones for the same thing. ok for now.

@jreback Implemented all of your comments. Hope it's fine now. 
Regarding to the PR push activities, I'll keep it in mind. 

@MaximilianR It was a typo. Fixed it.

@jreback I think I have it this time. Is this method right?

can you rebase / update

@jreback 
You mean 
`git checkout bug13207`
`git rebase master`
`git push origin bug13207`

Right?

yes, see docs [here](http://pandas.pydata.org/pandas-docs/stable/contributing.html#combining-commits)

@jreback Done. I hope this is alright.

I don't see any changes, you need to `git push yourbranchname -f`

I did a `git push origin bug13207 -f`

If I do a `git push bug13207 -f` it says
 Please make sure you have the correct access rights
and the repository exists

you cannot push to origin you need to push to your fork

My origin is set as my forked repo.

`git remote -v`

Output:  
origin  git@github.com:pfrcks/pandas.git (fetch)
origin  git@github.com:pfrcks/pandas.git (push)

@pfrcks : You can set your local branch to track your origin version by doing the following:

```
git push -u origin <branch_name>
```

From then on, you can just do `git push -f` or `git push`

@gfyoung I have followed the comments you gave.

It gives "Everything up-to-date" as output.

@pfrcks: that's perfectly fine!  That's because your local branch is sync'ed with your remote branch with the same name.  You can now see that by typing `git status`

can you rebase / update?

You should `git merge --squash`, commit that, `git reset --hard upstream/master`, then cherry pick the merged commit. That will clean up the merge commit etc. 

I am currently on bug13207 branch that I have created.
Issuing `git merge --squash` gives `(nothing to squash)Already up-to-date.`

I followed the rest of the steps and have again pushed the branch using `git push -f origin bug13207`

@pfrcks Possibly your upstream master was not updated? (in any case, although you say you force pushed, no new/rebased commits are in this PR and there are still conflicts, so something went wrong). Can you do 

```
git fetch upstream
```

before rebasing/merging? And then force push again?

can you rebase this on master
I rebased.

`git fetch origin`
`git rebase origin/master`
can you rebase, move release note to 0.20.0
closing as stale
Bug Fix for #13207. Added empty attribute for Index
State: closed

- [x] closes #13207 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: API Design

Comments:
you need to do this on a new and separate branch. you are including  commits from another PR.

pls see [docs](http://pandas.pydata.org/pandas-docs/stable/contributing.html)

@jreback I don't understand. I have worked for this fix and committed on a seperate branch named : my_branch2, like the docs say,
and then executed git push origin my_branch2.
I am not able to identify what is causing this problem.
Just below the heading on this page it is also showing
pydata:master from pfrcks:my_branch2

you branched off of the prior branch rather than master, rebase

`git rebase -i origin/master` and force push
BUG: Fix #13213 json_normalize() and non-ascii characters in keys
State: closed

- [ ] closes #13213 
- [ ] tests added / passed
- [ ] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Bug
name: Unicode
description: Unicode strings
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
pls add a whatsnew note

move the test to where I indicated

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13214?src=pr) is **84.15%**

> Merging [#13214](https://codecov.io/gh/pydata/pandas/pull/13214?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13214   diff @@
==========================================
  Files           138        138          
  Lines         50388      50394     +6   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42386      42408    +22   
+ Misses         8002       7986    -16   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [2429ec5...44745ca](https://codecov.io/gh/pydata/pandas/compare/2429ec5a3be278a2fb444eb2aae0c9b05b05cb62...44745ca323ee73924c6413eb4bec3dbe660a2155)

thanks @fmarczin !

Thank you for taking care of this so fast!
BUG: Sparse creation with object dtype may raise TypeError
State: closed

closes #11633 
closes #11856

Added basic tests for `groupby` and `pivot_table`.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Sparse
description: Sparse Data Type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13201?src=pr) is **84.15%**

> Merging [#13201](https://codecov.io/gh/pydata/pandas/pull/13201?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13201   diff @@
==========================================
  Files           138        138          
  Lines         50388      50394     +6   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42386      42407    +21   
+ Misses         8002       7987    -15   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [4e4a7d9...443b47e](https://codecov.io/gh/pydata/pandas/compare/4e4a7d9ae5b3c05b21dc0a0f3c12648236ec586e...443b47e83be2191477f6a8530e06001f5da69ce0)

thanks @sinhrks very nice as usual

side-note (nothing to do with this PR I think, but should investigate at some point).

```
nosetests pandas/sparse
....
/Users/jreback/pandas/pandas/sparse/array.py:52: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  return _wrap_result(name, op(self.sp_values, other),
```
BUG: Period and Series/Index comparison raises TypeError
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry
  `Period` and `Series/Index` comparison raises `TypeError` if `Period` is on left hand side.

```
# OK
pd.Series([pd.Period('2011-01-01', freq='D')]) > pd.Period('2011-01-01', freq='D')
#0    False
# dtype: bool

# NG
pd.Period('2011-01-01', freq='D') > pd.Series([pd.Period('2011-01-01', freq='D')])
# TypeError: Cannot compare type 'Period' with type 'Series'
```


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Period
description: Period data type

Comments:
lgtm. merge on green (or you can ping and i can).

thanks!
CLN: no return on init
State: closed

`__init__` shouldn't return anything


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
its odd that nothing borks at this, e.g. python itself, linter. what showed this issue ?

@jreback yeah, I _think_ it's just like returning `None`. 
No tool caught it actually - just saw it as I was updating PRs. Surprised lint didn't.
PERF: DataFrame transform
State: closed

- [x] closes #12737, 
  closes #13191
- [ ] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

```
    before     after       ratio
  [2de2884 ] [4b352d9 ]
-  164.84ms     1.73ms      0.01  groupby.groupby_transform_dataframe.time_groupby_transform_dataframe
     4.44ms     3.77ms      0.85  groupby.groupby_transform_series.time_groupby_transform_series
     6.40ms     4.76ms      0.74  groupby.groupby_transform_series2.time_groupby_transform_series2
```


Labels:
name: Bug
name: Groupby
name: Performance
description: Memory or execution speed performance
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
This also fixes (unless it was deliberate?) the issue of `SeriesGroupBy.transform` losing the Series's name, xref #9862

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13192?src=pr) is **84.12%**

> Merging [#13192](https://codecov.io/gh/pydata/pandas/pull/13192?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13192   diff @@
==========================================
  Files           138        138          
  Lines         50385      50382     -3   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42383      42381     -2   
+ Misses         8002       8001     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [2de2884...9d78f65](https://codecov.io/gh/pydata/pandas/compare/2de2884a7e7abf64f9967f6d8bc05a2d45f59bb4...9d78f659f55128e55d0b062f30fb04ae68994d3e)

thanks @chris-b1 very nice PR as usual!
TST: Use compatible time zones (#13186)
State: closed

- resolves #13186, allowing `pandas.tests.series.test_timeseries.TestSeriesTimeSeries.test_getitem_setitem_datetime_tz_dateutil` to pass on platforms that do not supply zoneinfo for, e.g., "US/Central".

The test failures in Travis are all unrelated to the code affected by this change.


Labels:
name: Timezones
description: Timezone data dtype
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
pls fix the linting

thanks @neirbowj
ENH: Improved error message (issue 13084)
State: closed

- [ ] closes #13084 
- [ ] tests passed
- [ ] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew:

As per issue 13084, made it clearer for the user to understand why a concat failed. No additional code was needed. This is because the _verify_integrity method inside the BlockManager class of pandas/core/internals.py already checked that the length of the concat data and index are the same. An alternative option was to pass the error in the e argument of the construction_error method of the SingleBlockManager. The alternative was not performed as there appeared to be no other reason why the shape of the dataframes would be different, as a result using the e argument would result in duplicating code. The method was also not extended to identify the dataframes with duplicate index values to avoid overcomplicating the code. The end user should be able to easily identify the dataframe/s causing the errors


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
if you'd like to update, pls reopen the issue.
BUG: Bug in .to_datetime() when passing integers or floats, no unit and errors=coerce
State: closed

closes #13180


Labels:
name: Bug
name: Datetime
description: Datetime data dtype

Comments:
@jorisvandenbossche 

note that prior to 0.18.1 we would handle

`pd.to_datetime([1])` with `unit='ns'` as the default. This PR addresses this, but after looking again, should we _make_ the user specify it here (when it IS ns), e.g. as in the accompanied issue.

any comments @jorisvandenbossche 

I have pandas version 0.19.2 and i still get this error when using to_datetime() method. My column type is int64 and here is the error message when i use the following:
![image](https://user-images.githubusercontent.com/11993393/32751562-6095ec36-c894-11e7-86ad-c1b86c2bc517.png)

1)  pd.to_datetime(ds['createdTime'], unit='s')   -->  OutOfBoundsDatetime: cannot convert input with unit 's'

2) pd.to_datetime(ds['createdTime']) --> no conversion

![image](https://user-images.githubusercontent.com/11993393/32751625-96334e74-c894-11e7-8578-2e3c2db66888.png)

Any idea how to fix this? Thanks a lot.

@melzoghbi if you still have an issue, please open a *new* issue (as your example is not related to the above issue). However, please first make sure to test on the latest pandas (0.21.0) and make sure to have an actual reproducible examples (so copy-pasteable code, not as an image). 

Further, for me this actually converts to a datetime:

```
In [121]: s = pd.Series([1508258340299])

In [122]: pd.to_datetime(s)
Out[122]: 
0   1970-01-01 00:25:08.258340299
dtype: datetime64[ns]
```
Thanks for your help and this actually shows the datetime but not the actual date that i am looking for. 

For example:
import pandas as pd
s = pd.Series([1510675225])
pd.to_datetime(s)

Should show a date as 11/14/27 but it actually shows epoch time as of 1970 datetime.

Any way i will open a new issue for this.
issue persists in pd 0.23.3
converting the unixtime to date time. 
1970-01-01 00:00:01.529009633
it returns 1970
this is still an issue in the Pandas `0.23.4`, is there any workaround?
This is a closed issue that is fixed. If you think there is still a bug, please open a *new* issue with a reproducible example.

If it is about the year 1970, that is not a bug but because `to_datetime` interprets integers as nanoseconds by default:

```
In [11]: s = pd.Series([1510675225])

In [12]: pd.to_datetime(s)
Out[12]: 
0   1970-01-01 00:00:01.510675225
dtype: datetime64[ns]

In [13]: pd.to_datetime(s, unit='s')
Out[13]: 
0   2017-11-14 16:00:25
dtype: datetime64[ns]
```
BUG: fix to_records confict with unicode_literals #13172
State: closed

- [ ] closes #13172 
- [ ] tests added / passed
- [ ] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Unicode
description: Unicode strings
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
pls add the test from the issue & a whats new (bug fix)

lgtm. minor test change; ping when green.

done.
原始邮件
发件人:Jeff Rebacknotifications@github.com
收件人:pydata/pandaspandas@noreply.github.com
抄送:starplanetzhangjinjie@yimian.com.cn; Authorauthor@noreply.github.com
发送时间:2016年5月16日(周一) 20:15
主题:Re: [pydata/pandas] BUG: fix to_records confict with unicode_literals#13172 (#13178)

lgtm. minor test change; ping when green.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub

thanks @starplanet nice patch!
DOC: Clarify Categorical Crosstab Behaviour
State: closed

Follow-on to #13073 by explaining the `Categorical` behaviour in the documentation.


Labels:
name: Docs
name: Categorical
description: Categorical Data Type

Comments:
can u add a similar example to the docs as well (in crosstab section)

Example added to doc.

@jreback : Documentation has been updated, and Travis is happy.  Ready to merge if there is nothing else.

ty sir!
Correct bool to datetime conversion (behaviour based on int/float processing)
State: closed

- [x] closes #11853
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Datetime
description: Datetime data dtype
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13176?src=pr) is **84.12%**

> Merging [#13176](https://codecov.io/gh/pydata/pandas/pull/13176?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13176   diff @@
==========================================
  Files           138        138          
  Lines         50587      50385   -202   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42593      42383   -210   
- Misses         7994       8002     +8   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [b4e2d34...e27a444](https://codecov.io/gh/pydata/pandas/compare/b4e2d34edcbc404f6c90f76b67bcc5fe26f0945f...e27a444001cfcfe0760ecba1160bab8fe950d620)

add a whatsnew entry

added whatsnew

build is green

Would you like to see any other changes? Thanks

Would you like to see any other changes? Thanks

@gliptak actually I think I steered you wrong here. These _should_ raise if `errors='raise'` (more informative message though), pass thru for `errors='ignore'`, and become `NaT` for `errors='coerce'`.

They _should not_ convert to 0/1 datetimes.

What is the informative message when `raise`? Also what does pass through mean when `ignore`? Thanks

```
# this is what an unconvertible string does
In [1]: pd.to_datetime('foo', errors='raise')
ValueError: Unknown string format

# ignore just passes thru the values
In [2]: pd.to_datetime('foo', errors='ignore')
Out[2]: 'foo'
```

so a messge like `booleans are not convertible to datetimes` is prob enough.

@jreback Is this the expected processing?

ok we have another case.

I accidently gave it this (from the original issue)

```
In [5]: pd.to_datetime(df.bool)
TypeError: object of type 'instancemethod' has no len()
```

`bool` is a method! so this hit the `else` clause. I think need to make that an `elif` and `is_string_object`, then have the `else` fall thru and raise/coerce as needed. (you could even remove the `is_bool_object` and just do that in the else and in the exception do `type(v)`

updated

build is green

thanks @gliptak