internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pandas-dev/pandas

Issue:
BUG: Series.float divided by Series.interger (with 0's)
# problem when denominator is interger type and has 0 element
# no such problem with pandas <= 0.13.1

print pd.**version**
print np.**version**

testdata=pd.DataFrame({'TL': (1,0), 'PI': (-0.01,-0.02)})
print "--- direct pd.Series division (wrong) : \n", (testdata.PI)/testdata.TL
print "--- np.where filtered (still wrong!!): ",np.where(testdata.TL>0.001,testdata.PI/testdata.TL,0 )
print "--- direct pd.Series division (OK for float type) : \n", testdata.PI/(testdata.TL.astype(float))
print "--- values division (correct)", testdata.PI.values/testdata.TL.values

-------------- Output ---------
0.14.0
1.7.1
--- direct pd.Series division (wrong) : 
0   -inf
1   -inf
dtype: float64
--- np.where filtered (still wrong!!):  [-inf   0.]
--- direct pd.Series division (OK for float type) : 
0   -0.010000
1        -inf
dtype: float64
--- values division (correct) [-0.01  -inf]



Discussion:
hmm, this does look like its doing the wrong thing. care to do a pull-request to fix ?

I am new to development and probably will take too long


Resolving patch:
diff --git a/pandas/core/common.py b/pandas/core/common.py
--- a/pandas/core/common.py
+++ b/pandas/core/common.py
@@ -1328,7 +1328,7 @@ def _fill_zeros(result, x, y, name, fill):
                 # correctly
                 # GH 6178
                 if np.isinf(fill):
-                    np.putmask(result,signs<0 & mask, -fill)
+                    np.putmask(result,(signs<0) & mask, -fill)
 
                 result = result.reshape(shape)
 


Test patch:
diff --git a/pandas/tests/test_series.py b/pandas/tests/test_series.py
--- a/pandas/tests/test_series.py
+++ b/pandas/tests/test_series.py
@@ -2362,6 +2362,17 @@ def test_div(self):
         expected = Series([np.nan,np.inf,-np.inf])
         assert_series_equal(result, expected)
 
+        # float/integer issue
+        # GH 7785
+        p = DataFrame({'first': (1,0), 'second': (-0.01,-0.02)})
+        expected = Series([-0.01,-np.inf])
+
+        result = p['second'].div(p['first'])
+        assert_series_equal(result, expected)
+
+        result = p['second'] / p['first']
+        assert_series_equal(result, expected)
+
     def test_operators(self):
 
         def _check_op(series, other, op, pos_only=False):
@@ -4865,12 +4876,12 @@ def test_astype_unicode(self):
         test_series = [
             Series([digits * 10, tm.rands(63), tm.rands(64), tm.rands(1000)]),
             Series([u"データーサイエンス、お前はもう死んでいる"]),
-            
+
         ]
-        
+
         former_encoding = None
         if not compat.PY3:
-            # in python we can force the default encoding 
+            # in python we can force the default encoding
             # for this test
             former_encoding = sys.getdefaultencoding()
             reload(sys)
Repository: pandas-dev/pandas

Issue:
Non-unique column names in fixed format HDF5
This leads to an error on read, writing is possible however. Since the format has a fundamental problem with this case if there are columns with the same name but different data types this is not fixable.

However, `df.to_hdf` should just not write the respective dataframe if the column index is not unique.



Discussion:
can u put up an example for the tests

```
df = pd.DataFrame(columns=["a"])
df2 = pd.concat([df, df], axis=1)
df2.to_hdf("test.h5", "test")
pd.read_hdf("test.h5", "test")
```

Only the last call fails with an `InvalidIndexError: Reindexing only valid with uniquely valued Index objects` in `Index.get_indexer`. You could silence the error by using `get_indexer_non_unique` instead but as described I think the reading will be broken if you have different datatypes in the columns.

It can be fixed by adding a check for the uniqueness of the column index in the fixed format. Note that this problem does not occur for the table format.

perfect. want to do a PR to validate and raise ValueError (could also be NotImplementedError) on writing?

I'll build one with tests, yes. Is it enough to fix this in `BlockManagerFixed.write`?

@filmor I believe so. Put in a test with `table` format as well (if you can't find a test with that is similar; if you do then put the tests together)


Resolving patch:
diff --git a/pandas/io/pytables.py b/pandas/io/pytables.py
--- a/pandas/io/pytables.py
+++ b/pandas/io/pytables.py
@@ -2680,6 +2680,9 @@ def write(self, obj, **kwargs):
 
         self.attrs.ndim = data.ndim
         for i, ax in enumerate(data.axes):
+            if i == 0:
+                if not ax.is_unique:
+                    raise ValueError("Columns index has to be unique for fixed format")
             self.write_index('axis%d' % i, ax)
 
         # Supporting mixed-type DataFrame objects...nontrivial


Test patch:
diff --git a/pandas/io/tests/test_pytables.py b/pandas/io/tests/test_pytables.py
--- a/pandas/io/tests/test_pytables.py
+++ b/pandas/io/tests/test_pytables.py
@@ -4370,6 +4370,17 @@ def test_categorical(self):
             # FIXME: TypeError: cannot pass a where specification when reading from a Fixed format store. this store must be selected in its entirety
             #result = store.select('df', where = ['index>2'])
             #tm.assert_frame_equal(df[df.index>2],result)
+    
+    def test_duplicate_column_name(self):
+        df = DataFrame(columns=["a", "a"], data=[[0, 0]])
+        
+        with ensure_clean_path(self.path) as path:
+            self.assertRaises(ValueError, df.to_hdf, path, 'df', format='fixed')
+            
+            df.to_hdf(path, 'df', format='table')
+            other = read_hdf(path, 'df')
+            tm.assert_frame_equal(df, other)
+        
 
 def _test_sort(obj):
     if isinstance(obj, DataFrame):
Repository: pandas-dev/pandas

Issue:
itertuples does not work with categorical column in dataframe
Code Snippet

``` python
df = pd.DataFrame({"id":[1,2,3,4,5,6], "raw_grade":['a', 'b', 'b', 'a', 'a', 'e']})
df['grade'] = pd.Categorical(df['raw_grade'])
for t in df.itertuples(index=False):
    print(t)
```

Error

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-45-182b29164b1a> in <module>()
      1 df = pd.DataFrame({"id":[1,2,3,4,5,6], "raw_grade":['a', 'b', 'b', 'a', 'a', 'e']})
      2 df['grade'] = pd.Categorical(df['raw_grade'])
----> 3 for t in df.itertuples(index=False):
      4     print(t)

/home/has2k1/scm/pandas/pandas/core/frame.pyc in itertuples(self, index)
    549         # use integer indexing because of possible duplicate column names
    550         arrays.extend(self.iloc[:, k] for k in range(len(self.columns)))
--> 551         return zip(*arrays)
    552 
    553     if compat.PY3:  # pragma: no cover

TypeError: izip argument #3 must support iteration
```

This is on master at commit 24b309f.

Edit:
Version string - pandas: 0.14.1-78-g24b309f



Discussion:

Resolving patch:
diff --git a/pandas/core/series.py b/pandas/core/series.py
--- a/pandas/core/series.py
+++ b/pandas/core/series.py
@@ -973,7 +973,9 @@ def _get_repr(
         return result
 
     def __iter__(self):
-        if np.issubdtype(self.dtype, np.datetime64):
+        if  com.is_categorical_dtype(self.dtype):
+            return iter(self.values)
+        elif np.issubdtype(self.dtype, np.datetime64):
             return (lib.Timestamp(x) for x in self.values)
         else:
             return iter(self.values)


Test patch:
diff --git a/pandas/tests/test_categorical.py b/pandas/tests/test_categorical.py
--- a/pandas/tests/test_categorical.py
+++ b/pandas/tests/test_categorical.py
@@ -647,6 +647,27 @@ def test_nan_handling(self):
                                       np.array(["a","b",np.nan], dtype=np.object_))
         self.assert_numpy_array_equal(s3.cat._codes, np.array([0,1,2,0]))
 
+    def test_sequence_like(self):
+
+        # GH 7839
+        # make sure can iterate
+        df = DataFrame({"id":[1,2,3,4,5,6], "raw_grade":['a', 'b', 'b', 'a', 'a', 'e']})
+        df['grade'] = Categorical(df['raw_grade'])
+
+        # basic sequencing testing
+        result = list(df.grade.cat)
+        expected = np.array(df.grade.cat).tolist()
+        tm.assert_almost_equal(result,expected)
+
+        # iteration
+        for t in df.itertuples(index=False):
+            str(t)
+
+        for row, s in df.iterrows():
+            str(s)
+
+        for c, col in df.iteritems():
+            str(s)
 
     def test_series_delegations(self):
Repository: pandas-dev/pandas

Issue:
BUG/API: bug in multi-index slicing with missing indexers
I think the following _could_ work and not raise `KeyError` because of a missing indexer. This would follow the API of loc which will be effectively a reindex as long as you have at least 1 found value.

```
s = pd.Series(np.arange(9),index=pd.MultiIndex.from_product([['A','B','C'],['foo','bar','baz']],names=['one','two'])).sortlevel()

s.loc[['A','D']]

idx = pd.IndexSlice
s.loc[idx[:,['foo','bah']]]
```



Discussion:

Resolving patch:
diff --git a/pandas/core/index.py b/pandas/core/index.py
--- a/pandas/core/index.py
+++ b/pandas/core/index.py
@@ -3638,9 +3638,17 @@ def _convert_indexer(r):
                 ranges.append(k)
             elif com.is_list_like(k):
                 # a collection of labels to include from this level (these are or'd)
-                ranges.append(reduce(
-                    np.logical_or,[ _convert_indexer(self._get_level_indexer(x, level=i)
-                                                     ) for x in k ]))
+                indexers = []
+                for x in k:
+                    try:
+                        indexers.append(_convert_indexer(self._get_level_indexer(x, level=i)))
+                    except (KeyError):
+
+                        # ignore not founds
+                        continue
+
+                ranges.append(reduce(np.logical_or,indexers))
+
             elif _is_null_slice(k):
                 # empty slice
                 pass


Test patch:
diff --git a/pandas/tests/test_indexing.py b/pandas/tests/test_indexing.py
--- a/pandas/tests/test_indexing.py
+++ b/pandas/tests/test_indexing.py
@@ -1941,6 +1941,39 @@ def f():
             df.val['X']
         self.assertRaises(KeyError, f)
 
+
+        # GH 7866
+        # multi-index slicing with missing indexers
+        s = pd.Series(np.arange(9),
+                      index=pd.MultiIndex.from_product([['A','B','C'],['foo','bar','baz']],
+                                                       names=['one','two'])
+                      ).sortlevel()
+
+        expected = pd.Series(np.arange(3),
+                             index=pd.MultiIndex.from_product([['A'],['foo','bar','baz']],
+                                                              names=['one','two'])
+                             ).sortlevel()
+
+        result = s.loc[['A']]
+        assert_series_equal(result,expected)
+        result = s.loc[['A','D']]
+        assert_series_equal(result,expected)
+
+        # empty series
+        result = s.loc[['D']]
+        expected = s.loc[[]]
+        assert_series_equal(result,expected)
+
+        idx = pd.IndexSlice
+        expected = pd.Series([0,3,6],
+                             index=pd.MultiIndex.from_product([['A','B','C'],['foo']],
+                                                              names=['one','two'])
+                             ).sortlevel()
+        result = s.loc[idx[:,['foo']]]
+        assert_series_equal(result,expected)
+        result = s.loc[idx[:,['foo','bah']]]
+        assert_series_equal(result,expected)
+
     def test_setitem_dtype_upcast(self):
 
         # GH3216
Repository: pandas-dev/pandas

Issue:
BUG: warnings import missing for panel
It appears the `warnings` import is missing in `panel.py`

```
In [1]: import numpy as np

In [2]: import pandas as pd

In [3]: pd.__version__
Out[3]: '0.14.1'

In [4]: p = pd.Panel(np.random.rand(3, 3, 3))

In [5]: p.major_xs(1, copy=False)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-5-0b8025289379> in <module>()
----> 1 p.major_xs(1, copy=False)

/home/aido/.env/local/lib/python2.7/site-packages/pandas/core/panel.pyc in major_xs(self, key, copy)
    715         """
    716         if copy is not None:
--> 717             warnings.warn("copy keyword is deprecated, "
    718                           "default is to return a copy or a view if possible")
    719 

NameError: global name 'warnings' is not defined
```



Discussion:
I would like to turn this into a PR if I may.

sure

pls add a test as well (which should fail w/o tht patch)

use assert_produces_warning


Resolving patch:
diff --git a/pandas/core/panel.py b/pandas/core/panel.py
--- a/pandas/core/panel.py
+++ b/pandas/core/panel.py
@@ -7,6 +7,7 @@
                            OrderedDefaultdict)
 from pandas import compat
 import sys
+import warnings
 import numpy as np
 from pandas.core.common import (PandasError, _try_sort, _default_index,
                                 _infer_dtype_from_scalar, notnull)


Test patch:
diff --git a/pandas/tests/test_panel.py b/pandas/tests/test_panel.py
--- a/pandas/tests/test_panel.py
+++ b/pandas/tests/test_panel.py
@@ -18,6 +18,7 @@
                                  assert_frame_equal,
                                  assert_series_equal,
                                  assert_almost_equal,
+                                 assert_produces_warning,
                                  ensure_clean,
                                  assertRaisesRegexp,
                                  makeCustomDataframe as mkdf,
@@ -2324,6 +2325,13 @@ def test_panel_index():
                                        np.repeat([1, 2, 3], 4)])
     assert(index.equals(expected))
 
+
+def test_import_warnings():
+    # GH8152
+    panel = Panel(np.random.rand(3, 3, 3))
+    with assert_produces_warning():
+        panel.major_xs(1, copy=False)
+
 if __name__ == '__main__':
     import nose
     nose.runmodule(argv=[__file__, '-vvs', '-x', '--pdb', '--pdb-failure'],
Repository: pandas-dev/pandas

Issue:
read_sql function eats informative error messages.
Consider the following code. 

```
import pandas as pd
import psycopg2 

bad_query = 'select bad_column from my_table;'
conn = psycopg2.connect(...)
pd.read_sql(bad_query, conn)
```

Running the above in my environment produces a DatabaseError exception with the following message: 
"Execution failed on sql: select bad_column from my_table;"

However, when I run the below code to query the database: 

```
import psycopg2

bad_query = 'select bad_column from my_table;'
conn = psycopg2.connect(...)
conn.cursor().execute(bad_query)
```

The following ProgrammingError message is returned:
"column "bad_column" does not exist in my_table"

This information is helpful for debugging, especially as queries become more complicated, and I believe it should be propagated through to the user.



Discussion:
Thanks for the report, and I agree it would be helpful to propagate this.

Could you also try it with a sqlalchemy engine instead of a raw psycopg2 connection? (to see of the error message is also suppressed in that mode)

BTW, I also recommend to use an sqlalchemy engine instead of a raw connection, as postgresql is not really supported without sqlalchemy (for reading a query, it will work (but not guaranteed), but for reading tables or writing to the database, it will not work anymore)

I just checked. The informative error message is successfully propagated when an sqlalchemy engine is used as opposed to the psycopg2 connection object. 

Thanks for checking! Then it is only an issue with the fallback mode


Resolving patch:
diff --git a/pandas/io/sql.py b/pandas/io/sql.py
--- a/pandas/io/sql.py
+++ b/pandas/io/sql.py
@@ -1104,7 +1104,7 @@ def execute(self, *args, **kwargs):
                     "Execution failed on sql: %s\n%s\nunable to rollback" % (args[0], e))
                 raise_with_traceback(ex)
 
-            ex = DatabaseError("Execution failed on sql: %s" % args[0])
+            ex = DatabaseError("Execution failed on sql '%s': %s" % (args[0], e))
             raise_with_traceback(ex)
 
     def read_sql(self, sql, index_col=None, coerce_float=True, params=None,


Test patch:
Repository: pandas-dev/pandas

Issue:
DOC: mention syntax for using parameters in read_sql in Notes in docstring
See http://stackoverflow.com/questions/24408557/pandas-read-sql-with-parameters.

Add a note mentioning this is driver-dependent which works, and they have to look at their drivers documentation.



Discussion:

Resolving patch:
diff --git a/pandas/io/sql.py b/pandas/io/sql.py
--- a/pandas/io/sql.py
+++ b/pandas/io/sql.py
@@ -341,7 +341,11 @@ def read_sql_query(sql, con, index_col=None, coerce_float=True, params=None,
         Attempt to convert values to non-string, non-numeric objects (like
         decimal.Decimal) to floating point, useful for SQL result sets
     params : list, tuple or dict, optional
-        List of parameters to pass to execute method.
+        List of parameters to pass to execute method.  The syntax used
+        to pass parameters is database driver dependent. Check your 
+        database driver documentation for which of the five syntax styles,
+        described in PEP 249's paramstyle, is supported.
+        Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'}
     parse_dates : list or dict
         - List of column names to parse as dates
         - Dict of ``{column_name: format string}`` where format string is
@@ -387,7 +391,11 @@ def read_sql(sql, con, index_col=None, coerce_float=True, params=None,
         Attempt to convert values to non-string, non-numeric objects (like
         decimal.Decimal) to floating point, useful for SQL result sets
     params : list, tuple or dict, optional
-        List of parameters to pass to execute method.
+        List of parameters to pass to execute method.  The syntax used
+        to pass parameters is database driver dependent. Check your 
+        database driver documentation for which of the five syntax styles,
+        described in PEP 249's paramstyle, is supported.
+        Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'}
     parse_dates : list or dict
         - List of column names to parse as dates
         - Dict of ``{column_name: format string}`` where format string is


Test patch:
Repository: pandas-dev/pandas

Issue:
BUG: at allowing fallback type indexing (which is not correct, its a restricted version of loc)
xref #7651

```
In [19]: s = pd.Series([1,2,3], index=list('abc'))

In [20]: s.at['a']
Out[20]: 1

In [23]: s.at[0]
Out[23]: 1
```



Discussion:

Resolving patch:
diff --git a/pandas/core/indexing.py b/pandas/core/indexing.py
--- a/pandas/core/indexing.py
+++ b/pandas/core/indexing.py
@@ -1510,6 +1510,18 @@ class _AtIndexer(_ScalarAccessIndexer):
     """ label based scalar accessor """
     _takeable = False
 
+    def _convert_key(self, key):
+        """ require they keys to be the same type as the index (so we don't fallback) """
+        for ax, i in zip(self.obj.axes, key):
+            if ax.is_integer():
+                if not com.is_integer(i):
+                    raise ValueError("At based indexing on an integer index can only have integer "
+                                     "indexers")
+            else:
+                if com.is_integer(i):
+                    raise ValueError("At based indexing on an non-integer index can only have non-integer "
+                                     "indexers")
+        return key
 
 class _iAtIndexer(_ScalarAccessIndexer):
 


Test patch:
diff --git a/pandas/tests/test_indexing.py b/pandas/tests/test_indexing.py
--- a/pandas/tests/test_indexing.py
+++ b/pandas/tests/test_indexing.py
@@ -866,6 +866,28 @@ def f():
             df.loc[[3]]
         self.assertRaises(KeyError, f)
 
+        # at should not fallback
+        # GH 7814
+        s = Series([1,2,3], index=list('abc'))
+        result = s.at['a']
+        self.assertEquals(result, 1)
+        self.assertRaises(ValueError, lambda : s.at[0])
+
+        df = DataFrame({'A' : [1,2,3]},index=list('abc'))
+        result = df.at['a','A']
+        self.assertEquals(result, 1)
+        self.assertRaises(ValueError, lambda : df.at['a',0])
+
+        s = Series([1,2,3], index=[3,2,1])
+        result = s.at[1]
+        self.assertEquals(result, 3)
+        self.assertRaises(ValueError, lambda : s.at['a'])
+
+        df = DataFrame({0 : [1,2,3]},index=[3,2,1])
+        result = df.at[1,0]
+        self.assertEquals(result, 3)
+        self.assertRaises(ValueError, lambda : df.at['a',0])
+
     def test_loc_getitem_label_slice(self):
 
         # label slices (with ints)
Repository: pandas-dev/pandas

Issue:
BUG: invalid dtype conversion when setting string column
http://stackoverflow.com/questions/24650117/how-can-i-select-all-dataframe-rows-that-are-within-a-certain-distance-of-a-give/24653406#24653406

```
In [32]: df = pd.DataFrame(np.random.rand(30, 3), columns=tuple('ABC'))

In [33]: df['event'] = np.nan

In [34]: df.dtypes
Out[34]: 
A        float64
B        float64
C        float64
event    float64
dtype: object

In [35]: df.loc[10,'event'] = 'foo'

In [37]: df.dtypes
Out[37]: 
A        object
B        object
C        object
event    object
dtype: object
```



Discussion:

Resolving patch:
diff --git a/pandas/core/internals.py b/pandas/core/internals.py
--- a/pandas/core/internals.py
+++ b/pandas/core/internals.py
@@ -556,9 +556,15 @@ def setitem(self, indexer, value):
             else:
                 dtype = 'infer'
             values = self._try_coerce_and_cast_result(values, dtype)
-            return [make_block(transf(values),
+            block = make_block(transf(values),
                                ndim=self.ndim, placement=self.mgr_locs,
-                               fastpath=True)]
+                               fastpath=True)
+
+            # may have to soft convert_objects here
+            if block.is_object and not self.is_object:
+                block = block.convert(convert_numeric=False)
+
+            return block
         except (ValueError, TypeError) as detail:
             raise
         except Exception as detail:


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -534,6 +534,15 @@ def test_setitem_cast(self):
         self.frame['something'] = 2.5
         self.assertEqual(self.frame['something'].dtype, np.float64)
 
+        # GH 7704
+        # dtype conversion on setting
+        df = DataFrame(np.random.rand(30, 3), columns=tuple('ABC'))
+        df['event'] = np.nan
+        df.loc[10,'event'] = 'foo'
+        result = df.get_dtype_counts().order()
+        expected = Series({'float64' : 3, 'object' : 1 }).order()
+        assert_series_equal(result, expected)
+
     def test_setitem_boolean_column(self):
         expected = self.frame.copy()
         mask = self.frame['A'] > 0