internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pandas-dev/pandas

Issue:
BUG: _apply_empty_result should take function args
See https://github.com/pydata/pandas/issues/4196#issuecomment-41275055



Discussion:

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -3313,7 +3313,7 @@ def apply(self, func, axis=0, broadcast=False, raw=False, reduce=None,
             f = func
 
         if len(self.columns) == 0 and len(self.index) == 0:
-            return self._apply_empty_result(func, axis, reduce)
+            return self._apply_empty_result(func, axis, reduce, *args, **kwds)
 
         if isinstance(f, np.ufunc):
             results = f(self.values)
@@ -3322,7 +3322,8 @@ def apply(self, func, axis=0, broadcast=False, raw=False, reduce=None,
         else:
             if not broadcast:
                 if not all(self.shape):
-                    return self._apply_empty_result(func, axis, reduce)
+                    return self._apply_empty_result(func, axis, reduce, *args,
+                                                    **kwds)
 
                 if raw and not self._is_mixed_type:
                     return self._apply_raw(f, axis)
@@ -3333,11 +3334,12 @@ def apply(self, func, axis=0, broadcast=False, raw=False, reduce=None,
             else:
                 return self._apply_broadcast(f, axis)
 
-    def _apply_empty_result(self, func, axis, reduce):
+    def _apply_empty_result(self, func, axis, reduce, *args, **kwds):
         if reduce is None:
             reduce = False
             try:
-                reduce = not isinstance(func(_EMPTY_SERIES), Series)
+                reduce = not isinstance(func(_EMPTY_SERIES, *args, **kwds),
+                                        Series)
             except Exception:
                 pass
 


Test patch:
Repository: pandas-dev/pandas

Issue:
ERR: Poor error message passing iterator to DataFrame constructor.
E.g.:

``` python
from itertools import izip
df = DataFrame(izip(range(3), range(-3, 1), list('abc')), columns=list('abc'))
---------------------------------------------------------------------------
PandasError                               Traceback (most recent call last)
<ipython-input-3-15234514160e> in <module>()
      1 from itertools import izip
      2
----> 3 df = DataFrame(izip(range(3), range(-3, 1), list('abc')), columns=list('abc'))

../pandas/core/frame.pyc in __init__(self, data, index, columns, dtype, copy)
    270                                          copy=False)
    271             else:
--> 272                 raise PandasError('DataFrame constructor not properly called!')
    273
    274         NDFrame.__init__(self, mgr, fastpath=True)

PandasError: DataFrame constructor not properly called!
```



Discussion:
a list of iterators is ok (and tested already)

```
DataFrame([ range(10), range(10) ])
```

I understand, just saying we should consider improving the error message...

Once you hit that else clause, you aren't going to get back what you want anyways, so we may as well give a better error message (like "Hey, you can't pass an iterator here" or something like that).


Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -253,6 +253,8 @@ def __init__(self, data=None, index=None, columns=None, dtype=None,
             else:
                 mgr = self._init_ndarray(data, index, columns, dtype=dtype,
                                          copy=copy)
+        elif isinstance(data, collections.Iterator):
+            raise TypeError("data argument can't be an iterator")
         else:
             try:
                 arr = np.array(data, dtype=dtype, copy=copy)


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -3135,6 +3135,10 @@ def test_constructor_miscast_na_int_dtype(self):
         expected = DataFrame([[np.nan, 1], [1, 0]])
         assert_frame_equal(df, expected)
 
+    def test_constructor_iterator_failure(self):
+        with assertRaisesRegexp(TypeError, 'iterator'):
+            df = DataFrame(iter([1, 2, 3]))
+
     def test_constructor_column_duplicates(self):
         # it works! #2079
         df = DataFrame([[8, 5]], columns=['a', 'a'])
Repository: pandas-dev/pandas

Issue:
PERF: use fastpath=True in Index methods (delete/drop/insert/etc)?
Been hit by this when optimizing index-oblivious Blocks. In my case,

```
is_deleted = np.zeros(len(index), dtype=np.bool_)
is_deleted[deleted_loc] = True
index = index[~is_deleted]
```

was still a lot faster then than `index.delete(deleted_loc)`.

It appears, that delete doesn't add `fastpath=True` to ctor and that triggers type inference for string (object) indices. There seems to be plenty of methods that do it the same way and hence are slow too, can we do something about that?



Discussion:
sure
want to do a quick pr ?

vbenches a plus but not that necessary
maybe spot check ok

so what's the downside here?

@jtratner it's the same as one of my earlier PRs about `__getitem__`: we drop type inference that might've changed index type after deletion of non-matching elements. That is if we go with "trivial" solution.

If trivial is not necessary, then dtype inference may be preserved without the necessity to traverse each element of dtype=O array. If one made an int8 array of object classes (or int 16 if 255 classes is not enough) to accompany the main one, then instead of enumerating all entries each time it would suffice to do single `np.bincount(x.ravel()).nonzero()` to find out which classes are present. Memory overhead for that int8 array should be acceptable, but it may require some non-trivial effort to make sure they're in sync.


Resolving patch:
diff --git a/pandas/core/index.py b/pandas/core/index.py
--- a/pandas/core/index.py
+++ b/pandas/core/index.py
@@ -1760,14 +1760,13 @@ def slice_locs(self, start=None, end=None):
 
     def delete(self, loc):
         """
-        Make new Index with passed location deleted
+        Make new Index with passed location(-s) deleted
 
         Returns
         -------
         new_index : Index
         """
-        arr = np.delete(self.values, loc)
-        return Index(arr)
+        return np.delete(self, loc)
 
     def insert(self, loc, item):
         """


Test patch:
Repository: pandas-dev/pandas

Issue:
read_csv: chunksize clashes with nrows
If nrows are determined, the use of chunksize option does not create a TextFileReader object.

``` python
reader = pd.read_table(fname, sep=',',chunksize=4, na_values=['null'],
                       nrows=20)
type(reader)
```

``` bash
pandas.core.frame.DataFrame
```

My suggestion:
- Either put into docs that they are not to be used concurrently
- or it's a feature request that they can.

I would find it useful to get chunks of x size, but only for the first n row of a huge file.



Discussion:
I think easiest ATM to simply raise NotImplementedError if both `nrows` and `chunksize` are specified.

Implemented is a bit non-trivial but would be useful I agree.

You want to do a PR for the NotImplementedError? Then we'll create an issue to implement this at some point.

I would like to but find io/parsers.py quite confusing.

`read_csv` is 'declared' here:

``` python
449 read_csv = _make_parser_function('read_csv', sep=',')
450 read_csv = Appender(_read_csv_doc)(read_csv)
```

`_make_parser_function` defines a `parser_f` on line 311, and that definition has both the `chunksize` and `nrows` option, but is it true, that they are in no case, so for none of the generated parsers, work together? In that case should I implemented right there, at `parser_f` definition?

look here: https://github.com/pydata/pandas/blob/master/pandas/io/parsers.py#L230


Resolving patch:
diff --git a/pandas/io/parsers.py b/pandas/io/parsers.py
--- a/pandas/io/parsers.py
+++ b/pandas/io/parsers.py
@@ -227,7 +227,10 @@ def _read(filepath_or_buffer, kwds):
     # Create the parser.
     parser = TextFileReader(filepath_or_buffer, **kwds)
 
-    if nrows is not None:
+    if (nrows is not None) and (chunksize is not None):
+        raise NotImplementedError("'nrows' and 'chunksize' can not be used"
+                                  " together yet.")
+    elif nrows is not None:
         return parser.read(nrows)
     elif chunksize or iterator:
         return parser


Test patch:
diff --git a/pandas/io/tests/test_parsers.py b/pandas/io/tests/test_parsers.py
--- a/pandas/io/tests/test_parsers.py
+++ b/pandas/io/tests/test_parsers.py
@@ -1973,13 +1973,6 @@ def test_header_names_backward_compat(self):
                                header=0)
         tm.assert_frame_equal(result, expected)
 
-    def test_integer_overflow_bug(self):
-        # #2601
-        data = "65248E10 11\n55555E55 22\n"
-
-        result = self.read_csv(StringIO(data), header=None, sep=' ')
-        self.assertTrue(result[0].dtype == np.float64)
-
     def test_int64_min_issues(self):
         # #2599
         data = 'A,B\n0,0\n0,'
@@ -2141,6 +2134,10 @@ def test_ignore_leading_whitespace(self):
         expected = DataFrame({'a':[1,4,7], 'b':[2,5,8], 'c': [3,6,9]})
         tm.assert_frame_equal(result, expected)
 
+    def test_nrows_and_chunksize_raises_notimplemented(self):
+        data = 'a b c'
+        self.assertRaises(NotImplementedError, self.read_csv, StringIO(data),
+                     nrows=10, chunksize=5)
 
 
 class TestPythonParser(ParserTests, tm.TestCase):
Repository: pandas-dev/pandas

Issue:
BUG: period indexing via [] for label indexingshould return partial ranges
http://stackoverflow.com/questions/19730653/slicing-on-a-perod-index-in-pandas-when-slice-start-and-end-may-be-out-of-bounds/19731145#19731145



Discussion:

Resolving patch:
diff --git a/pandas/tseries/period.py b/pandas/tseries/period.py
--- a/pandas/tseries/period.py
+++ b/pandas/tseries/period.py
@@ -730,6 +730,12 @@ def __iter__(self):
         for val in self.values:
             yield Period(ordinal=val, freq=self.freq)
 
+    def searchsorted(self, key, side='left'):
+        if isinstance(key, compat.string_types):
+            key = Period(key, freq=self.freq).ordinal
+
+        return self.values.searchsorted(key, side=side)
+
     @property
     def is_all_dates(self):
         return True


Test patch:
diff --git a/pandas/tseries/tests/test_period.py b/pandas/tseries/tests/test_period.py
--- a/pandas/tseries/tests/test_period.py
+++ b/pandas/tseries/tests/test_period.py
@@ -1959,6 +1959,23 @@ def test_range_slice_seconds(self):
             for d in ['2013/01/01', '2013/01', '2013']:
                 assert_series_equal(s[d:], s)
 
+    def test_range_slice_outofbounds(self):
+        # GH 5407
+        didx = DatetimeIndex(start='2013/10/01', freq='D', periods=10)
+        pidx = PeriodIndex(start='2013/10/01', freq='D', periods=10)
+
+        for idx in [didx, pidx]:
+            df = DataFrame(dict(units=[100 + i for i in range(10)]), index=idx)
+            empty = DataFrame(index=DatetimeIndex([], freq='D'), columns=['units'])
+
+            tm.assert_frame_equal(df['2013/09/01':'2013/09/30'], empty)
+            tm.assert_frame_equal(df['2013/09/30':'2013/10/02'], df.iloc[:2])
+            tm.assert_frame_equal(df['2013/10/01':'2013/10/02'], df.iloc[:2])
+            tm.assert_frame_equal(df['2013/10/02':'2013/09/30'], empty)
+            tm.assert_frame_equal(df['2013/10/15':'2013/10/17'], empty)
+            tm.assert_frame_equal(df['2013/06':'2013/09'], empty)
+            tm.assert_frame_equal(df['2013/11':'2013/12'], empty)
+
     def test_pindex_qaccess(self):
         pi = PeriodIndex(['2Q05', '3Q05', '4Q05', '1Q06', '2Q06'], freq='Q')
         s = Series(np.random.rand(len(pi)), index=pi).cumsum()
Repository: pandas-dev/pandas

Issue:
BUG: setitem with a multi-index and integer indices
http://stackoverflow.com/questions/23764306/multi-indexed-dataframe-setting-values/23764822?noredirect=1#comment36573548_23764822



Discussion:

Resolving patch:
diff --git a/pandas/core/indexing.py b/pandas/core/indexing.py
--- a/pandas/core/indexing.py
+++ b/pandas/core/indexing.py
@@ -333,11 +333,7 @@ def _setitem_with_indexer(self, indexer, value):
                 obj = self.obj[item]
                 index = obj.index
                 idx = indexer[:info_axis][0]
-                try:
-                    if idx in index:
-                        idx = index.get_loc(idx)
-                except:
-                    pass
+
                 plane_indexer = tuple([idx]) + indexer[info_axis + 1:]
                 lplane_indexer = _length_of_indexer(plane_indexer[0], index)
 


Test patch:
diff --git a/pandas/tests/test_indexing.py b/pandas/tests/test_indexing.py
--- a/pandas/tests/test_indexing.py
+++ b/pandas/tests/test_indexing.py
@@ -607,6 +607,17 @@ def test_loc_setitem(self):
         expected = DataFrame({'a' : [0.5,-0.5,-1.5], 'b' : [0,1,2] })
         assert_frame_equal(df,expected)
 
+    def test_loc_setitem_multiindex(self):
+
+        # GH7190
+        index = pd.MultiIndex.from_product([np.arange(0,100), np.arange(0, 80)], names=['time', 'firm'])
+        df = DataFrame(np.nan,columns=['A', 'w', 'l', 'a', 'x', 'X', 'd', 'profit'], index=index)
+        t, n = 0, 2
+
+        df.loc[(t,n),'X'] = 0
+        result = df.loc[(t,n),'X']
+        self.assertEqual(result, 0)
+
     def test_loc_setitem_dups(self):
 
         # GH 6541
Repository: pandas-dev/pandas

Issue:
BUG: assignment with 0-dim arrays and a multi-index is buggy
This works in the non-multi-index case, so is a bug
http://stackoverflow.com/questions/23830248/potential-bug-pandas-multiindex-and-0-d-sized-values

```
np.random.seed(1234)
T = 100 # time
N = 80 # firms
TIndex = np.arange(0, T)
FIndex = np.arange(0, N)

index = pd.MultiIndex.from_product([TIndex, FIndex], names=['time', 'firm'])
df = pd.DataFrame(-999, columns=['A', 'w', 'l', 'a', 'x', 'X', 'd', 'profit'], index=index)
t, n = 0, 2
df.loc[(t,n), 'X'] = np.array(0)
```



Discussion:

Resolving patch:
diff --git a/pandas/core/indexing.py b/pandas/core/indexing.py
--- a/pandas/core/indexing.py
+++ b/pandas/core/indexing.py
@@ -339,7 +339,7 @@ def _setitem_with_indexer(self, indexer, value):
 
                 # require that we are setting the right number of values that
                 # we are indexing
-                if is_list_like(value) and lplane_indexer != len(value):
+                if is_list_like(value) and np.iterable(value) and lplane_indexer != len(value):
 
                     if len(obj[idx]) != len(value):
                         raise ValueError(
@@ -386,7 +386,7 @@ def setter(item, v):
 
             def can_do_equal_len():
                 """ return True if we have an equal len settable """
-                if not len(labels) == 1:
+                if not len(labels) == 1 or not np.iterable(value):
                     return False
 
                 l = len(value)


Test patch:
diff --git a/pandas/tests/test_indexing.py b/pandas/tests/test_indexing.py
--- a/pandas/tests/test_indexing.py
+++ b/pandas/tests/test_indexing.py
@@ -619,14 +619,20 @@ def test_loc_setitem_multiindex(self):
         self.assertEqual(result, 0)
 
         df = DataFrame(-999,columns=['A', 'w', 'l', 'a', 'x', 'X', 'd', 'profit'], index=index)
-        df.loc[(t,n),'X'] = 0
+        df.loc[(t,n),'X'] = 1
         result = df.loc[(t,n),'X']
-        self.assertEqual(result, 0)
+        self.assertEqual(result, 1)
 
         df = DataFrame(columns=['A', 'w', 'l', 'a', 'x', 'X', 'd', 'profit'], index=index)
-        df.loc[(t,n),'X'] = 0
+        df.loc[(t,n),'X'] = 2
         result = df.loc[(t,n),'X']
-        self.assertEqual(result, 0)
+        self.assertEqual(result, 2)
+
+        # GH 7218, assinging with 0-dim arrays
+        df = DataFrame(-999,columns=['A', 'w', 'l', 'a', 'x', 'X', 'd', 'profit'], index=index)
+        df.loc[(t,n), 'X'] = np.array(3)
+        result = df.loc[(t,n),'X']
+        self.assertEqual(result,3)
 
     def test_loc_setitem_dups(self):
Repository: pandas-dev/pandas

Issue:
BUG: inferred_freq results in None for eastern hemispheres timezones
Found during #7299.

```
dates = ['2000-01-31', '2000-02-29', '2000-03-31', '2000-04-30']
pd.DatetimeIndex(dates).inferred_freq
# M

pd.DatetimeIndex(dates, tz='US/Eastern').inferred_freq
# M

pd.DatetimeIndex(dates, tz='Mexico/General').inferred_freq
# M

pd.DatetimeIndex(dates, tz='Asia/Tokyo').inferred_freq
# None

pd.DatetimeIndex(dates, tz='Europe/Paris').inferred_freq
# None

pd.DatetimeIndex(dates, tz='Australia/Sydney').inferred_freq
# None
```



Discussion:
this should use localize the timestamps before returning i8: https://github.com/pydata/pandas/blob/master/pandas/tseries/frequencies.py#L649


Resolving patch:
diff --git a/pandas/tseries/frequencies.py b/pandas/tseries/frequencies.py
--- a/pandas/tseries/frequencies.py
+++ b/pandas/tseries/frequencies.py
@@ -668,6 +668,9 @@ class _FrequencyInferer(object):
     def __init__(self, index, warn=True):
         self.index = index
         self.values = np.asarray(index).view('i8')
+        if index.tz is not None:
+            self.values = tslib.date_normalize(self.values, index.tz)
+
         self.warn = warn
 
         if len(index) < 3:


Test patch:
diff --git a/pandas/tseries/tests/test_frequencies.py b/pandas/tseries/tests/test_frequencies.py
--- a/pandas/tseries/tests/test_frequencies.py
+++ b/pandas/tseries/tests/test_frequencies.py
@@ -256,6 +256,22 @@ def test_infer_freq(self):
         rng = Index(rng.to_timestamp('D', how='e').asobject)
         self.assertEqual(rng.inferred_freq, 'Q-OCT')
 
+    def test_infer_freq_tz(self):
+
+        # GH 7310
+        for tz in [None, 'Asia/Tokyo', 'US/Pacific', 'Europe/Paris']:
+            dates = ['2010-11-30', '2010-12-31', '2011-01-31', '2011-02-28']
+            idx = DatetimeIndex(dates)
+            self.assertEqual(idx.inferred_freq, 'M')
+
+            dates = ['2011-01-01', '2011-01-02', '2011-01-03', '2011-01-04']
+            idx = DatetimeIndex(dates)
+            self.assertEqual(idx.inferred_freq, 'D')
+
+            dates = ['2011-12-31 22:00', '2011-12-31 23:00', '2012-01-01 00:00', '2012-01-01 01:00']
+            idx = DatetimeIndex(dates)
+            self.assertEqual(idx.inferred_freq, 'H')
+
     def test_not_monotonic(self):
         rng = _dti(['1/31/2000', '1/31/2001', '1/31/2002'])
         rng = rng[::-1]
Repository: pandas-dev/pandas

Issue:
BUG: Series.map(dict) doesn't work when dict keys are tuples of length 1?
I ran across an odd issue today where some analysis code that was working pre-pandas 0.14.0 started failing. The issue seems to be specific to trying to use `Series.map(some_dict)` when the dict keys (and values in the series) are single item tuples. Code to demonstrate the issue:

``` python
import pandas as pd
pd.version.version
Out[10]: '0.14.0'

df = pd.DataFrame({'a': [(1,), (2,), (3, 4), (5, 6)]})
label_mappings = {
    (1,): 'A',
    (2,): 'B',
    (3, 4): 'A',
    (5, 6): 'B'
}
df['labels'] = df['a'].map(label_mappings)
df
Out[7]: 
        a labels
0    (1,)    NaN
1    (2,)    NaN
2  (3, 4)      A
3  (5, 6)      B
```

The expected output here is:

``` python
df
Out[7]: 
        a labels
0    (1,)      A
1    (2,)      B
2  (3, 4)      A
3  (5, 6)      B
```



Discussion:
ok...want to take a stab at fixing?


Resolving patch:
diff --git a/pandas/core/series.py b/pandas/core/series.py
--- a/pandas/core/series.py
+++ b/pandas/core/series.py
@@ -1940,7 +1940,7 @@ def map_f(values, f):
 
         if isinstance(arg, (dict, Series)):
             if isinstance(arg, dict):
-                arg = self._constructor(arg)
+                arg = self._constructor(arg, index=arg.keys())
 
             indexer = arg.index.get_indexer(values)
             new_values = com.take_1d(arg.values, indexer)


Test patch:
diff --git a/pandas/tests/test_series.py b/pandas/tests/test_series.py
--- a/pandas/tests/test_series.py
+++ b/pandas/tests/test_series.py
@@ -4860,6 +4860,25 @@ def test_map_na_exclusion(self):
         exp = s * 2
         assert_series_equal(result, exp)
 
+    def test_map_dict_with_tuple_keys(self):
+        '''
+        Due to new MultiIndex-ing behaviour in v0.14.0,
+        dicts with tuple keys passed to map were being
+        converted to a multi-index, preventing tuple values
+        from being mapped properly.
+        '''
+        df = pd.DataFrame({'a': [(1,), (2,), (3, 4), (5, 6)]})
+        label_mappings = {
+            (1,): 'A',
+            (2,): 'B',
+            (3, 4): 'A',
+            (5, 6): 'B'
+        }
+        df['labels'] = df['a'].map(label_mappings)
+        df['expected_labels'] = pd.Series(['A', 'B', 'A', 'B'], index=df.index)
+        # All labels should be filled now
+        tm.assert_series_equal(df['labels'], df['expected_labels'])
+
     def test_apply(self):
         assert_series_equal(self.ts.apply(np.sqrt), np.sqrt(self.ts))
Repository: pandas-dev/pandas

Issue:
ENH: add nlargest/nsmallest to Series groupby
see #5534



Discussion:
@cpcloud ?

didn't know this was here...nice.


Resolving patch:
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -78,7 +78,8 @@
 
 _series_apply_whitelist = \
     (_common_apply_whitelist - set(['boxplot'])) | \
-    frozenset(['dtype', 'value_counts', 'unique', 'nunique'])
+    frozenset(['dtype', 'value_counts', 'unique', 'nunique',
+               'nlargest', 'nsmallest'])
 
 _dataframe_apply_whitelist = \
     _common_apply_whitelist | frozenset(['dtypes', 'corrwith'])


Test patch:
diff --git a/pandas/tests/test_groupby.py b/pandas/tests/test_groupby.py
--- a/pandas/tests/test_groupby.py
+++ b/pandas/tests/test_groupby.py
@@ -4047,6 +4047,7 @@ def test_groupby_whitelist(self):
             'value_counts',
             'diff',
             'unique', 'nunique',
+            'nlargest', 'nsmallest',
         ])
 
         for obj, whitelist in zip((df, s),
@@ -4381,6 +4382,27 @@ def test_max_nan_bug(self):
         tm.assert_frame_equal(r, e)
         self.assertFalse(r['File'].isnull().any())
 
+    def test_nlargest(self):
+        a = Series([1, 3, 5, 7, 2, 9, 0, 4, 6, 10])
+        b = Series(list('a' * 5 + 'b' * 5))
+        gb = a.groupby(b)
+        r = gb.nlargest(3)
+        e = Series([7, 5, 3, 10, 9, 6],
+                   index=MultiIndex.from_arrays([list('aaabbb'),
+                                                 [3, 2, 1, 9, 5, 8]]))
+        tm.assert_series_equal(r, e)
+
+    def test_nsmallest(self):
+        a = Series([1, 3, 5, 7, 2, 9, 0, 4, 6, 10])
+        b = Series(list('a' * 5 + 'b' * 5))
+        gb = a.groupby(b)
+        r = gb.nsmallest(3)
+        e = Series([1, 2, 3, 0, 4, 6],
+                   index=MultiIndex.from_arrays([list('aaabbb'),
+                                                 [0, 4, 1, 6, 7, 8]]))
+        tm.assert_series_equal(r, e)
+
+
 def assert_fp_equal(a, b):
     assert (np.abs(a - b) < 1e-12).all()