internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pandas-dev/pandas

Issue:
BUG: common.is_hashable returns False for np.float64 on Python 3
xref #8929
xref #9193

Here are [some test results](https://travis-ci.org/shoyer/pandas/builds/47184451) on Travis-CI illustrating the issue and my [commit](https://github.com/shoyer/pandas/commit/f8eb1c687164274a6b9a9640e565de60293ad2c1)

Any ideas for the fix? This is quite likely a numpy bug, but it's one we need to work around. The simplest thing would be revert back to just calling `hash`, but I expect that would cause some other test cases to fail.

CC @aevri



Discussion:

Resolving patch:
diff --git a/pandas/core/common.py b/pandas/core/common.py
--- a/pandas/core/common.py
+++ b/pandas/core/common.py
@@ -2537,13 +2537,13 @@ def is_hashable(arg):
     >>> is_hashable(a)
     False
     """
-    # don't consider anything not collections.Hashable, so as not to broaden
-    # the definition of hashable beyond that. For example, old-style classes
-    # are not collections.Hashable but they won't fail hash().
-    if not isinstance(arg, collections.Hashable):
-        return False
+    # unfortunately, we can't use isinstance(arg, collections.Hashable), which
+    # can be faster than calling hash, because numpy scalars on Python 3 fail
+    # this test
+
+    # reconsider this decision once this numpy bug is fixed:
+    # https://github.com/numpy/numpy/issues/5562
 
-    # narrow the definition of hashable if hash(arg) fails in practice
     try:
         hash(arg)
     except TypeError:


Test patch:
diff --git a/pandas/tests/test_common.py b/pandas/tests/test_common.py
--- a/pandas/tests/test_common.py
+++ b/pandas/tests/test_common.py
@@ -424,7 +424,7 @@ def __hash__(self):
             raise TypeError("Not hashable")
 
     hashable = (
-        1, 'a', tuple(), (1,), HashableClass(),
+        1, 3.14, np.float64(3.14), 'a', tuple(), (1,), HashableClass(),
     )
     not_hashable = (
         [], UnhashableClass1(),
@@ -434,13 +434,10 @@ def __hash__(self):
     )
 
     for i in hashable:
-        assert isinstance(i, collections.Hashable)
         assert com.is_hashable(i)
     for i in not_hashable:
-        assert not isinstance(i, collections.Hashable)
         assert not com.is_hashable(i)
     for i in abc_hashable_not_really_hashable:
-        assert isinstance(i, collections.Hashable)
         assert not com.is_hashable(i)
 
     # numpy.array is no longer collections.Hashable as of
@@ -455,7 +452,7 @@ class OldStyleClass():
             pass
         c = OldStyleClass()
         assert not isinstance(c, collections.Hashable)
-        assert not com.is_hashable(c)
+        assert com.is_hashable(c)
         hash(c)  # this will not raise
Repository: pandas-dev/pandas

Issue:
BUG: DataFrame doesn't hold _metadata when sliced by slice instance
```
import pandas as pd


class TestDataFrame(pd.DataFrame):

    _metadata = ['testattr']

    @property
    def _constructor(self):
        return TestDataFrame

df = TestDataFrame({0: [1, 2, 3], 1: [1, 2, 3]})
df.testattr = 'XXX'
df.testattr
# XXX

# Using list is OK
df.iloc[[0, 1], :].testattr
# XXX

# Using slice is NG
df.iloc[0:1, :].testattr
# AttributeError: 'TestDataFrame' object has no attribute 'testattr'
```



Discussion:
should prob work though vey few tests ATM for using _metadata at all (it's not really a user feature)

Understood. Because I'm trying to subclass `DataFrame`, I'll dig in when I have a time.


Resolving patch:
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -1179,6 +1179,7 @@ def _slice(self, slobj, axis=0, kind=None):
         """
         axis = self._get_block_manager_axis(axis)
         result = self._constructor(self._data.get_slice(slobj, axis=axis))
+        result = result.__finalize__(self)
 
         # this could be a view
         # but only in a single-dtyped view slicable case


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -14057,6 +14057,28 @@ def test_assign_bad(self):
         with tm.assertRaises(KeyError):
             df.assign(C=df.A, D=lambda x: x['A'] + x['C'])
 
+    def test_dataframe_metadata(self):
+
+        class TestDataFrame(DataFrame):
+            _metadata = ['testattr']
+
+            @property
+            def _constructor(self):
+                return TestDataFrame
+
+
+        df = TestDataFrame({'X': [1, 2, 3], 'Y': [1, 2, 3]},
+                           index=['a', 'b', 'c'])
+        df.testattr = 'XXX'
+
+        self.assertEqual(df.testattr, 'XXX')
+        self.assertEqual(df[['X']].testattr, 'XXX')
+        self.assertEqual(df.loc[['a', 'b'], :].testattr, 'XXX')
+        self.assertEqual(df.iloc[[0, 1], :].testattr, 'XXX')
+        # GH9776
+        self.assertEqual(df.iloc[0:1, :].testattr, 'XXX')
+
+
 def skip_if_no_ne(engine='numexpr'):
     if engine == 'numexpr':
         try:
Repository: pandas-dev/pandas

Issue:
read_csv() interprets index_col="True" as "1"
epi.txt

```
OBS ID PERIOD TMT Y0 AGE COUNT
1 1 1 0 11 31 5
2 1 2 0 11 31 3
3 1 3 0 11 31 3
4 1 4 0 11 31 3
5 2 1 0 11 30 3
6 2 2 0 11 30 5
7 2 3 0 11 30 3
8 2 4 0 11 30 3
9 3 1 0 6 25 2
10 3 2 0 6 25 4
```

```
epi = pandas.read_csv("./epi.datn.txt", sep=" ", index_col=True)
print(epi)
```

```
    OBS  PERIOD  TMT  Y0  AGE  COUNT
ID                                  
1     1       1    0  11   31      5
1     2       2    0  11   31      3
1     3       3    0  11   31      3
1     4       4    0  11   31      3
2     5       1    0  11   30      3
2     6       2    0  11   30      5
2     7       3    0  11   30      3
2     8       4    0  11   30      3
3     9       1    0   6   25      2
3    10       2    0   6   25      4
```

I had some code that used `False` to disable reading indexes from the data, and was expecting `True` to be the opposite. Instead it read the **second** column, because True == 1 in python.

I think that `True` should be explicitly handled, because this behaviour is counterintuitive given that `False` is meaningful. `False` is not the same as `index_col=0`, after all.



Discussion:
I would not do this, rather it should raise if its `True` as its not an allowed value.


Resolving patch:
diff --git a/pandas/io/parsers.py b/pandas/io/parsers.py
--- a/pandas/io/parsers.py
+++ b/pandas/io/parsers.py
@@ -652,6 +652,8 @@ def _clean_options(self, options, engine):
         # really delete this one
         keep_default_na = result.pop('keep_default_na')
 
+        if index_col is True:
+            raise ValueError("The value of index_col couldn't be 'True'")
         if _is_index_col(index_col):
             if not isinstance(index_col, (list, tuple, np.ndarray)):
                 index_col = [index_col]


Test patch:
diff --git a/pandas/io/tests/test_parsers.py b/pandas/io/tests/test_parsers.py
--- a/pandas/io/tests/test_parsers.py
+++ b/pandas/io/tests/test_parsers.py
@@ -520,6 +520,11 @@ def test_usecols_index_col_False(self):
         df = self.read_csv(StringIO(s_malformed), usecols=cols, index_col=False)
         tm.assert_frame_equal(expected, df)
 
+    def test_index_col_is_True(self):
+        # Issue 9798
+        self.assertRaises(ValueError, self.read_csv, StringIO(self.ts_data),
+                          index_col=True)
+
     def test_converter_index_col_bug(self):
         # 1835
         data = "A;B\n1;2\n3;4"
Repository: pandas-dev/pandas

Issue:
BUG: unstack on unicode name level breaks
xref #9650

```
In [62]: i = pd.MultiIndex(
    levels=[[u'foo', u'bar'], [u'one', u'two'], [u'a', u'b']],
    labels=[[0, 0, 1, 1], [0, 1, 0, 1], [1, 0, 1, 0]],
    names=[u'first', u'second', u'third'])

In [63]: s = pd.Series(0, index=i)

In [64]: s.unstack([1,2]).stack(0)
KeyError: 'Level t not found'
```



Discussion:
Working on this.


Resolving patch:
diff --git a/pandas/core/index.py b/pandas/core/index.py
--- a/pandas/core/index.py
+++ b/pandas/core/index.py
@@ -4023,7 +4023,7 @@ def sortlevel(self, level=0, ascending=True, sort_remaining=True):
         labels = list(self.labels)
         shape = list(self.levshape)
 
-        if isinstance(level, (str, int)):
+        if isinstance(level, (compat.string_types, int)):
             level = [level]
         level = [self._get_level_number(lev) for lev in level]
 


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -16,7 +16,7 @@
 
 from pandas.compat import(
     map, zip, range, long, lrange, lmap, lzip,
-    OrderedDict, u, StringIO
+    OrderedDict, u, StringIO, string_types
 )
 from pandas import compat
 
@@ -12428,6 +12428,30 @@ def test_unstack_bool(self):
                                                        ['c', 'l']]))
         assert_frame_equal(rs, xp)
 
+    def test_unstack_level_binding(self):
+        # GH9856
+        mi = pd.MultiIndex(
+                levels=[[u'foo', u'bar'], [u'one', u'two'], [u'a', u'b']],
+                labels=[[0, 0, 1, 1], [0, 1, 0, 1], [1, 0, 1, 0]],
+                names=[u'first', u'second', u'third'])
+        s = pd.Series(0, index=mi)
+        result = s.unstack([1, 2]).stack(0)
+
+        expected_mi = pd.MultiIndex(
+                        levels=[['foo', 'bar'], ['one', 'two']],
+                        labels=[[0, 0, 1, 1], [0, 1, 0, 1]],
+                        names=['first', 'second'])
+
+        expected = pd.DataFrame(np.array([[np.nan, 0],
+                                          [0, np.nan],
+                                          [np.nan, 0],
+                                          [0, np.nan]],
+                                         dtype=np.float64),
+                                index=expected_mi,
+                                columns=pd.Index(['a', 'b'], name='third'))
+
+        self.assert_frame_equal(result, expected)
+
     def test_unstack_to_series(self):
         # check reversibility
         data = self.frame.unstack()
Repository: pandas-dev/pandas

Issue:
API/BUG: awkward syntax to add categories to a Categorical
from [SO](http://stackoverflow.com/questions/29709918/pandas-and-category-replacement/29712287#29712287)
- the `.add_categories` should be able to take an `Index`/ndarray, ATM it must be converted to a list
- should be a keyword for something like: add any additional categories that I am passing to you (even if the current ones are duplicates, just add these to the end). e.g.

```
In [147]: s = pd.Categorical(list('aabbcd'))

In [148]: s2 = list('aabbcdefg')

In [149]: s
Out[149]: 
[a, a, b, b, c, d]
Categories (4, object): [a, b, c, d]

In [150]: s.add_categories(s.categories.sym_diff(Index(s2)).tolist())
Out[150]: 
[a, a, b, b, c, d]
Categories (7, object): [a, b, c, d, e, f, g]
```

I would ideally just like to say:

```
s.add_categories(s2, take_new=True)
```

(maybe not the best keyword, but something like this)



Discussion:
cc @JanSchulz


Resolving patch:
diff --git a/pandas/core/categorical.py b/pandas/core/categorical.py
--- a/pandas/core/categorical.py
+++ b/pandas/core/categorical.py
@@ -687,7 +687,7 @@ def add_categories(self, new_categories, inplace=False):
         if len(already_included) != 0:
             msg = "new categories must not include old categories: %s" % str(already_included)
             raise ValueError(msg)
-        new_categories = list(self._categories) + (new_categories)
+        new_categories = list(self._categories) + list(new_categories)
         new_categories = self._validate_categories(new_categories)
         cat = self if inplace else self.copy()
         cat._categories = new_categories


Test patch:
diff --git a/pandas/tests/test_categorical.py b/pandas/tests/test_categorical.py
--- a/pandas/tests/test_categorical.py
+++ b/pandas/tests/test_categorical.py
@@ -727,6 +727,19 @@ def f():
             cat.add_categories(["d"])
         self.assertRaises(ValueError, f)
 
+        # GH 9927
+        cat = Categorical(list("abc"), ordered=True)
+        expected = Categorical(list("abc"), categories=list("abcde"), ordered=True)
+        # test with Series, np.array, index, list
+        res = cat.add_categories(Series(["d", "e"]))
+        self.assert_categorical_equal(res, expected)
+        res = cat.add_categories(np.array(["d", "e"]))
+        self.assert_categorical_equal(res, expected)
+        res = cat.add_categories(Index(["d", "e"]))
+        self.assert_categorical_equal(res, expected)
+        res = cat.add_categories(["d", "e"])
+        self.assert_categorical_equal(res, expected)
+
     def test_remove_categories(self):
         cat = Categorical(["a","b","c","a"], ordered=True)
         old = cat.copy()
Repository: pandas-dev/pandas

Issue:
BUG: Index constructor does not set name when given a TimedeltaIndex as data
```
In [46]: tdi = pd.timedelta_range(start=0, periods=4, freq='D')

In [47]: pd.Index(tdi, name='foo').name is None
Out[47]: True

In [48]: pd.Index(tdi.values, name='foo').name is None
Out[48]: False
```

Other index types seem to work fine:

```
In [51]: dti = pd.date_range(start='2000-01-01', periods=4, freq='D')

In [52]: pd.Index(dti, name='foo').name is None
Out[52]: False
```



Discussion:

Resolving patch:
diff --git a/pandas/tseries/tdi.py b/pandas/tseries/tdi.py
--- a/pandas/tseries/tdi.py
+++ b/pandas/tseries/tdi.py
@@ -140,7 +140,7 @@ def __new__(cls, data=None, unit=None,
                 copy=False, name=None,
                 closed=None, verify_integrity=True, **kwargs):
 
-        if isinstance(data, TimedeltaIndex) and freq is None:
+        if isinstance(data, TimedeltaIndex) and freq is None and name is None:
             if copy:
                 data = data.copy()
             return data


Test patch:
diff --git a/pandas/tseries/tests/test_timedeltas.py b/pandas/tseries/tests/test_timedeltas.py
--- a/pandas/tseries/tests/test_timedeltas.py
+++ b/pandas/tseries/tests/test_timedeltas.py
@@ -949,6 +949,10 @@ def test_constructor_name(self):
                             name='TEST')
         self.assertEqual(idx.name, 'TEST')
 
+        # GH10025
+        idx2 = TimedeltaIndex(idx, name='something else')
+        self.assertEqual(idx2.name, 'something else')
+
     def test_freq_conversion(self):
 
         # doc example
Repository: pandas-dev/pandas

Issue:
Series.resample('1S', how='last') on series with dtype=datetime64[ns] is very slow
example test:

``` python
intSeries = pd.Series(5, pd.date_range(start='2000-01-01', end='2000-01-05', freq='555000U'), dtype='int64')
timeSeries = intSeries.astype('datetime64[ns]')

time0 = time.time()
intSeries.resample('1S', how='last')
print 'resampling the int series took ', str(timedelta(seconds = time.time()-time0)), '(H:M:S)'

time0 = time.time()
timeSeries.resample('1S', how='last')
print 'resampling the datetime64[ns] series took ', str(timedelta(seconds = time.time()-time0)), '(H:M:S)'
```

output shows 14sec vs 0.5sec:
resampling the int series took  **0:00:00.054000** (H:M:S)
resampling the datetime64[ns] series took  **0:00:14.350000** (H:M:S)



Discussion:
yep...seems to be not taking a fast path. FYI, pls always post `pd.show_versions()`. welcome a pull-request to fix.

## INSTALLED VERSIONS

commit: None
python: 2.7.5.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.14.0
nose: 1.3.0
Cython: 0.19.1
numpy: 1.8.1
scipy: 0.14.0
statsmodels: 0.5.0
IPython: 1.0.0
sphinx: 1.1.3
patsy: 0.2.1
scikits.timeseries: None
dateutil: 2.2
pytz: 2014.4
bottleneck: 0.8.0
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.3.1
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.5.5
lxml: 3.3.5
bs4: 4.3.1
html5lib: None
bq: None
apiclient: None
rpy2: None
sqlalchemy: 0.9.6
pymysql: None
psycopg2: None


Resolving patch:
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -1496,6 +1496,8 @@ def aggregate(self, values, how, axis=0):
 
         if is_datetime_or_timedelta_dtype(values.dtype):
             values = values.view('int64')
+            # GH 7754
+            is_numeric = True
         elif is_bool_dtype(values.dtype):
             values = _algos.ensure_float64(values)
         elif com.is_integer_dtype(values):
diff --git a/vb_suite/timeseries.py b/vb_suite/timeseries.py
--- a/vb_suite/timeseries.py
+++ b/vb_suite/timeseries.py
@@ -135,6 +135,16 @@ def date_range(start=None, end=None, periods=None, freq=None):
     Benchmark("ts.resample('D', how='mean')", setup,
               start_date=datetime(2012, 4, 25))
 
+# GH 7754
+setup = common_setup + """
+rng = date_range(start='2000-01-01 00:00:00',
+                    end='2000-01-01 10:00:00', freq='555000U')
+int_ts = Series(5, rng, dtype='int64')
+ts = int_ts.astype('datetime64[ns]')
+"""
+
+timeseries_resample_datetime64 = Benchmark("ts.resample('1S', how='last')", setup)
+
 #----------------------------------------------------------------------
 # to_datetime
 


Test patch:
Repository: pandas-dev/pandas

Issue:
BUG: select_column does not preserve the column name
```
In [4]: df = DataFrame({'A' : np.random.randn(10), 'B' : 'foo'})

In [5]: df.to_hdf('test.h5','df',mode='w',format='table',data_columns=True)

# this should have a name of B
In [7]: with pd.HDFStore('test.h5') as store:
    print store.select_column('df','B')
   ...:     
0    foo
1    foo
2    foo
3    foo
4    foo
5    foo
6    foo
7    foo
8    foo
9    foo
dtype: object
```



Discussion:

Resolving patch:
diff --git a/pandas/io/pytables.py b/pandas/io/pytables.py
--- a/pandas/io/pytables.py
+++ b/pandas/io/pytables.py
@@ -3608,7 +3608,7 @@ def read_column(self, column, where=None, start=None, stop=None, **kwargs):
                                                 nan_rep=self.nan_rep,
                                                 encoding=self.encoding
                                                 ).take_data(),
-                                      a.tz, True))
+                                      a.tz, True), name=column)
 
         raise KeyError("column [%s] not found in the table" % column)
 


Test patch:
diff --git a/pandas/io/tests/test_pytables.py b/pandas/io/tests/test_pytables.py
--- a/pandas/io/tests/test_pytables.py
+++ b/pandas/io/tests/test_pytables.py
@@ -3766,7 +3766,7 @@ def f():
             # valid
             result = store.select_column('df', 'index')
             tm.assert_almost_equal(result.values, Series(df.index).values)
-            self.assertIsInstance(result,Series)
+            self.assertIsInstance(result, Series)
 
             # not a data indexable column
             self.assertRaises(
@@ -3806,6 +3806,14 @@ def f():
             result = store.select_column('df3', 'string', start=-2, stop=2)
             tm.assert_almost_equal(result.values, df3['string'].values[-2:2])
 
+            # GH 10392 - make sure column name is preserved
+            df4 = DataFrame({'A': np.random.randn(10), 'B': 'foo'})
+            store.append('df4', df4, data_columns=True)
+            expected = df4['B']
+            result = store.select_column('df4', 'B')
+            tm.assert_series_equal(result, expected)
+
+
     def test_coordinates(self):
         df = tm.makeTimeDataFrame()
Repository: pandas-dev/pandas

Issue:
Improve Documentation for pandas.Series.interpolate as to how index is used
From the docs:

> ‘linear’: ignore the index and treat the values as equally spaced. default
> ‘index’, ‘values’: use the actual numerical values of the index

but the remaining types, for example ‘quadratic’, ‘cubic’, ‘barycentric’, `krogh`, don't specify whether they assume equal spacing or use the index values. 

I believe that they DO use the index values.



Discussion:

Resolving patch:
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -2872,18 +2872,21 @@ def interpolate(self, method='linear', axis=0, limit=None, inplace=False,
                   'polynomial', 'spline' 'piecewise_polynomial', 'pchip'}
 
             * 'linear': ignore the index and treat the values as equally
-              spaced. default
+              spaced. This is the only method supported on MultiIndexes.
+              default
             * 'time': interpolation works on daily and higher resolution
               data to interpolate given length of interval
             * 'index', 'values': use the actual numerical values of the index
             * 'nearest', 'zero', 'slinear', 'quadratic', 'cubic',
               'barycentric', 'polynomial' is passed to
-              `scipy.interpolate.interp1d` with the order given both
+              `scipy.interpolate.interp1d` with the order given. Both
               'polynomial' and 'spline' requre that you also specify and order
-              (int) e.g. df.interpolate(method='polynomial', order=4)
+              (int) e.g. df.interpolate(method='polynomial', order=4). These
+              use the actual numerical values of the index
             * 'krogh', 'piecewise_polynomial', 'spline', and 'pchip' are all
               wrappers around the scipy interpolation methods of similar
-              names. See the scipy documentation for more on their behavior:
+              names. These use the actual numerical values of the index. See 
+              the scipy documentation for more on their behavior:
               http://docs.scipy.org/doc/scipy/reference/interpolate.html#univariate-interpolation
               http://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html
 


Test patch:
Repository: pandas-dev/pandas

Issue:
BUILD: 2.7 with numpy_dev currently broken
seems a recent change in numpy master: https://github.com/numpy/numpy/issues/6014

I think might be a bug on numpy's part.



Discussion:

Resolving patch:
diff --git a/pandas/core/categorical.py b/pandas/core/categorical.py
--- a/pandas/core/categorical.py
+++ b/pandas/core/categorical.py
@@ -1722,6 +1722,7 @@ def _concat_compat(to_concat, axis=0):
     ----------
     to_concat : array of arrays
     axis : axis to provide concatenation
+        in the current impl this is always 0, e.g. we only have 1-d categoricals
 
     Returns
     -------
@@ -1744,7 +1745,7 @@ def convert_categorical(x):
 
         # convert to object type and perform a regular concat
         from pandas.core.common import _concat_compat
-        return _concat_compat([ np.array(x,copy=False).astype('object') for x in to_concat ],axis=axis)
+        return _concat_compat([ np.array(x,copy=False).astype('object') for x in to_concat ],axis=0)
 
     # we could have object blocks and categorical's here
     # if we only have a single cateogoricals then combine everything
@@ -1761,4 +1762,4 @@ def convert_categorical(x):
             raise ValueError("incompatible categories in categorical concat")
 
     # concat them
-    return Categorical(np.concatenate([ convert_categorical(x) for x in to_concat ],axis=axis), categories=categories)
+    return Categorical(np.concatenate([ convert_categorical(x) for x in to_concat ],axis=0), categories=categories)


Test patch: