internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pandas-dev/pandas

Issue:
DataFrame.append() does not create missing columns
Appending a Series to a DataFrame does not create new columns in the DataFrame for unmatched labels in the Series.

```
df = pd.DataFrame({'a': {'x': 1, 'y': 2}, 'b': {'x': 3, 'y': 4}})
row = pd.Series([5, 6, 7], index=['a', 'b', 'c'], name='z')

df = df.append(row)
print(df.to_string())
```

Result:

```
   a  b
x  1  3
y  2  4
z  5  6
```

This behavior is contrary to the documentation for append().
## INSTALLED VERSIONS

Python: 3.3.0.final.0
OS: Windows
Release: 7
Processor: x86 Family 6 Model 37 Stepping 5, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.13.0
Cython: Not installed
Numpy: 1.7.0rc1
Scipy: Not installed
statsmodels: Not installed
    patsy: Not installed
scikits.timeseries: Not installed
dateutil: 2.2
pytz: 2013.9
bottleneck: Not installed
PyTables: Not Installed
    numexpr: Not Installed
matplotlib: 1.3.0
openpyxl: Not installed
xlrd: Not installed
xlwt: Not installed
xlsxwriter: Not installed
sqlalchemy: Not installed
lxml: Not installed
bs4: Not installed
html5lib: Not installed
bigquery: Not installed
apiclient: Not installed



Discussion:

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -3513,9 +3513,12 @@ def append(self, other, ignore_index=False, verify_integrity=False):
                                 'ignore_index=True')
 
             index = None if other.name is None else [other.name]
-            other = other.reindex(self.columns, copy=False)
+            combined_columns = self.columns.tolist() + ((self.columns | other.index) - self.columns).tolist()
+            other = other.reindex(combined_columns, copy=False)
             other = DataFrame(other.values.reshape((1, len(other))),
-                              index=index, columns=self.columns).convert_objects()
+                              index=index, columns=combined_columns).convert_objects()
+            if not self.columns.equals(combined_columns):
+                self = self.reindex(columns=combined_columns)
         elif isinstance(other, list) and not isinstance(other[0], DataFrame):
             other = DataFrame(other)
             if (self.columns.get_indexer(other.columns) >= 0).all():


Test patch:
diff --git a/pandas/tools/tests/test_merge.py b/pandas/tools/tests/test_merge.py
--- a/pandas/tools/tests/test_merge.py
+++ b/pandas/tools/tests/test_merge.py
@@ -1163,6 +1163,14 @@ def test_append(self):
         self.assertRaises(ValueError, self.frame.append, self.frame,
                           verify_integrity=True)
 
+        # new columns
+        # GH 6129
+        df = DataFrame({'a': {'x': 1, 'y': 2}, 'b': {'x': 3, 'y': 4}})
+        row = Series([5, 6, 7], index=['a', 'b', 'c'], name='z')
+        expected = DataFrame({'a': {'x': 1, 'y': 2, 'z': 5}, 'b': {'x': 3, 'y': 4, 'z': 6}, 'c' : {'z' : 7}})
+        result = df.append(row)
+        assert_frame_equal(result, expected)
+
     def test_append_length0_frame(self):
         df = DataFrame(columns=['A', 'B', 'C'])
         df3 = DataFrame(index=[0, 1], columns=['A', 'B'])
Repository: pandas-dev/pandas

Issue:
BUG: boolean query evaluation
http://stackoverflow.com/questions/21414959/querying-a-single-row-dataframe-with-anded-conditionals/21415119#21415119

cc @cpcloud 



Discussion:
I think the OP is trying to use implict boolean conditionals....

I think it's because it's a 1-element `Series` which I think I turn into a scalar. Not sure though. I can check tonight.

k....(you examples in the docs are longer than 1 element) and see to be ok.....

alright i see where the issue is

i'm binging on pandas tonight :cookie: 


Resolving patch:
diff --git a/pandas/computation/align.py b/pandas/computation/align.py
--- a/pandas/computation/align.py
+++ b/pandas/computation/align.py
@@ -81,16 +81,12 @@ def wrapper(terms):
             return _align_core_single_unary_op(terms[0])
 
         term_values = (term.value for term in terms)
+
         # only scalars or indexes
         if all(isinstance(term.value, pd.Index) or term.isscalar for term in
                terms):
             return np.result_type(*term_values), None
 
-        # single element ndarrays
-        all_has_size = all(hasattr(term.value, 'size') for term in terms)
-        if all_has_size and all(term.value.size == 1 for term in terms):
-            return np.result_type(*term_values), None
-
         # no pandas objects
         if not _any_pandas_objects(terms):
             return np.result_type(*term_values), None


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -12860,6 +12860,20 @@ def test_query_lex_compare_strings(self):
         for parser, engine in product(PARSERS, ENGINES):
             yield self.check_query_lex_compare_strings, parser, engine
 
+    def check_query_single_element_booleans(self, parser, engine):
+        tm.skip_if_no_ne(engine)
+        columns = 'bid', 'bidsize', 'ask', 'asksize'
+        data = np.random.randint(2, size=(1, len(columns))).astype(bool)
+        df = DataFrame(data, columns=columns)
+        res = df.query('bid & ask', engine=engine, parser=parser)
+        expected = df[df.bid & df.ask]
+        assert_frame_equal(res, expected)
+
+    def test_query_single_element_booleans(self):
+        for parser, engine in product(PARSERS, ENGINES):
+            yield self.check_query_single_element_booleans, parser, engine
+
+
 class TestDataFrameEvalNumExprPandas(tm.TestCase):
 
     @classmethod
Repository: pandas-dev/pandas

Issue:
assigning using df.loc and unsorted labels doesn't put values in the right places
Using 0.13.1 (gohlke binaries, Win64, using numpy 1.8):

When using .loc, when passing unsorted labels, pandas doesn't assign the values in the designed labels' locations. Example:
`df = pandas.DataFrame(index=[3, 5, 4], columns=['A'])`
`df.loc[[4, 3, 5], 'A'] = [1, 2, 3]`
Result is:

```
Out[30]: 
   A
3  1
5  2
4  3
```

One would have expected:

```
Out[30]: 
   A
3  2
5  3
4  1
```



Discussion:

Resolving patch:
diff --git a/pandas/core/internals.py b/pandas/core/internals.py
--- a/pandas/core/internals.py
+++ b/pandas/core/internals.py
@@ -633,7 +633,8 @@ def setitem(self, indexer, value):
             # if we are an exact match (ex-broadcasting),
             # then use the resultant dtype
             elif len(arr_value.shape) and arr_value.shape[0] == values.shape[0] and np.prod(arr_value.shape) == np.prod(values.shape):
-                values = arr_value.reshape(values.shape)
+                values[indexer] = value
+                values = values.astype(arr_value.dtype)
 
             # set
             else:


Test patch:
diff --git a/pandas/tests/test_indexing.py b/pandas/tests/test_indexing.py
--- a/pandas/tests/test_indexing.py
+++ b/pandas/tests/test_indexing.py
@@ -646,6 +646,32 @@ def test_loc_setitem_frame(self):
         result = df.ix[:,1:]
         assert_frame_equal(result, expected)
 
+        # GH 6254
+        # setting issue
+        df = DataFrame(index=[3, 5, 4], columns=['A'])
+        df.loc[[4, 3, 5], 'A'] = [1, 2, 3]
+        expected = DataFrame(dict(A = Series([1,2,3],index=[4, 3, 5]))).reindex(index=[3,5,4])
+        assert_frame_equal(df, expected)
+
+        # GH 6252
+        # setting with an empty frame
+        keys1 = ['@' + str(i) for i in range(5)]
+        val1 = np.arange(5)
+
+        keys2 = ['@' + str(i) for i in range(4)]
+        val2 = np.arange(4)
+
+        index = list(set(keys1).union(keys2))
+        df = DataFrame(index = index)
+        df['A'] = nan
+        df.loc[keys1, 'A'] = val1
+
+        df['B'] = nan
+        df.loc[keys2, 'B'] = val2
+
+        expected = DataFrame(dict(A = Series(val1,index=keys1), B = Series(val2,index=keys2))).reindex(index=index)
+        assert_frame_equal(df, expected)
+
     def test_loc_setitem_frame_multiples(self):
 
         # multiple setting
Repository: pandas-dev/pandas

Issue:
infer_freq on a series
Somewhat related to #6273 . I'm fairly certain though not positive that this also used to work on 0.12.x.

```
df = pd.DataFrame(np.random.rand(5,3))
df['date'] = ['1-1-1990', '2-1-1990', '3-1-1990', '4-1-1990', '5-1-1990']
pd.infer_freq(df['date'])
```

Raises Series has no attribute `is_monotonic`. I guess `infer_freq` now expects either an Index or ....?

```
[22]: pd.version.version
[22]: '0.13.0-472-g9109812'
```



Discussion:

Resolving patch:
diff --git a/pandas/tseries/index.py b/pandas/tseries/index.py
--- a/pandas/tseries/index.py
+++ b/pandas/tseries/index.py
@@ -254,6 +254,11 @@ def __new__(cls, data=None,
         else:
             try:
                 subarr = tools.to_datetime(data, box=False)
+
+                # make sure that we have a index/ndarray like (and not a Series)
+                if isinstance(subarr, ABCSeries):
+                    subarr = subarr.values
+
             except ValueError:
                 # tz aware
                 subarr = tools.to_datetime(data, box=False, utc=True)


Test patch:
diff --git a/pandas/tests/test_index.py b/pandas/tests/test_index.py
--- a/pandas/tests/test_index.py
+++ b/pandas/tests/test_index.py
@@ -146,6 +146,31 @@ def test_constructor_corner(self):
         # corner case
         self.assertRaises(TypeError, Index, 0)
 
+    def test_constructor_from_series(self):
+
+        expected = DatetimeIndex([Timestamp('20110101'),Timestamp('20120101'),Timestamp('20130101')])
+        s = Series([Timestamp('20110101'),Timestamp('20120101'),Timestamp('20130101')])
+        result = Index(s)
+        self.assertTrue(result.equals(expected))
+        result = DatetimeIndex(s)
+        self.assertTrue(result.equals(expected))
+
+        # GH 6273
+        # create from a series, passing a freq
+        s = Series(pd.to_datetime(['1-1-1990', '2-1-1990', '3-1-1990', '4-1-1990', '5-1-1990']))
+        result = DatetimeIndex(s, freq='MS')
+        expected = DatetimeIndex(['1-1-1990', '2-1-1990', '3-1-1990', '4-1-1990', '5-1-1990'],freq='MS')
+        self.assertTrue(result.equals(expected))
+
+        df = pd.DataFrame(np.random.rand(5,3))
+        df['date'] = ['1-1-1990', '2-1-1990', '3-1-1990', '4-1-1990', '5-1-1990']
+        result = DatetimeIndex(df['date'], freq='MS')
+
+        # GH 6274
+        # infer freq of same
+        result = pd.infer_freq(df['date'])
+        self.assertEqual(result,'MS')
+
     def test_index_ctor_infer_periodindex(self):
         from pandas import period_range, PeriodIndex
         xp = period_range('2012-1-1', freq='M', periods=3)
Repository: pandas-dev/pandas

Issue:
BUG: groupby agg with a single function and mixed-types raises
http://stackoverflow.com/questions/21706030/pandas-groupby-agg-function-column-dtype-error



Discussion:

Resolving patch:
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -2123,6 +2123,7 @@ def _aggregate_item_by_item(self, func, *args, **kwargs):
         obj = self._obj_with_exclusions
         result = {}
         cannot_agg = []
+        errors=None
         for item in obj:
             try:
                 data = obj[item]
@@ -2133,11 +2134,19 @@ def _aggregate_item_by_item(self, func, *args, **kwargs):
             except ValueError:
                 cannot_agg.append(item)
                 continue
+            except TypeError as e:
+                cannot_agg.append(item)
+                errors=e
+                continue
 
         result_columns = obj.columns
         if cannot_agg:
             result_columns = result_columns.drop(cannot_agg)
 
+            # GH6337
+            if not len(result_columns) and errors is not None:
+                raise errors
+
         return DataFrame(result, columns=result_columns)
 
     def _decide_output_index(self, output, labels):


Test patch:
diff --git a/pandas/tests/test_groupby.py b/pandas/tests/test_groupby.py
--- a/pandas/tests/test_groupby.py
+++ b/pandas/tests/test_groupby.py
@@ -370,6 +370,26 @@ def f(grp):
         e.name = None
         assert_series_equal(result,e)
 
+    def test_agg_api(self):
+
+        # GH 6337
+        # http://stackoverflow.com/questions/21706030/pandas-groupby-agg-function-column-dtype-error
+        # different api for agg when passed custom function with mixed frame
+
+        df = DataFrame({'data1':np.random.randn(5),
+                        'data2':np.random.randn(5),
+                        'key1':['a','a','b','b','a'],
+                        'key2':['one','two','one','two','one']})
+        grouped = df.groupby('key1')
+
+        def peak_to_peak(arr):
+            return arr.max() - arr.min()
+
+        expected = grouped.agg([peak_to_peak])
+        expected.columns=['data1','data2']
+        result = grouped.agg(peak_to_peak)
+        assert_frame_equal(result,expected)
+
     def test_agg_regression1(self):
         grouped = self.tsframe.groupby([lambda x: x.year, lambda x: x.month])
         result = grouped.agg(np.mean)
Repository: pandas-dev/pandas

Issue:
inplace where with complex column changes dtype to float
http://stackoverflow.com/questions/21766182/replacing-out-of-bounds-complex-values-in-a-pandas-dataframe/21766586#21766586



Discussion:
@hayd 

`df3.where(df3<5000,np.nan,inplace=True)`

is the same (and is implemented) like this:

`df3[df3<5000] = np.nan`

their certianly could be a coercion bug in this (as this boils down to `Block.where` to actually do this and it has to coerce inputs and outputs (or attempt to anyhow)

@hayd if you have the test frame for this example would be great (or a simple one too that reproduces)

Sure thing:

```
In [11]: df = DataFrame([[1+1j, 2], [5+1j, 4+1j]], columns=['a', 'b'])

In [12]: df[df.abs() >= 5] = np.nan

In [13]: df
Out[13]:
    a       b
0   1  (2+0j)
1 NaN  (4+1j)

In [14]: df.a.dtype  # expected complex128 (that of df.b.dtype)
Out[14]: dtype('float64')

expected =  DataFrame([[1+1j, 2], [np.nan, 4+1j]], columns=['a', 'b'])
```

this demonstrates for where too:

```
In [15]: df.where(df <= 5)  # works correctly
Out[15]:
        a       b
0  (1+1j)  (2+0j)
1     NaN  (4+1j)

In [16]: df.where(df <= 5, inplace=True)

In [17]: df
Out[17]:
    a       b
0   1  (2+0j)
1 NaN  (4+1j)
```


Resolving patch:
diff --git a/pandas/core/internals.py b/pandas/core/internals.py
--- a/pandas/core/internals.py
+++ b/pandas/core/internals.py
@@ -1221,7 +1221,11 @@ class ComplexBlock(FloatOrComplexBlock):
     is_complex = True
 
     def _can_hold_element(self, element):
-        return isinstance(element, complex)
+        if is_list_like(element):
+            element = np.array(element)
+            return issubclass(element.dtype.type, (np.floating, np.integer, np.complexfloating))
+        return (isinstance(element, (float, int, complex, np.float_, np.int_)) and
+                not isinstance(bool, np.bool_))
 
     def _try_cast(self, element):
         try:


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -8736,6 +8736,13 @@ def create():
         result = df.where(pd.notnull(df),DataFrame(1,index=df.index,columns=df.columns))
         assert_frame_equal(result, expected)
 
+    def test_where_complex(self):
+        # GH 6345
+        expected = DataFrame([[1+1j, 2], [np.nan, 4+1j]], columns=['a', 'b'])
+        df = DataFrame([[1+1j, 2], [5+1j, 4+1j]], columns=['a', 'b'])
+        df[df.abs() >= 5] = np.nan
+        assert_frame_equal(df,expected)
+
     def test_mask(self):
         df = DataFrame(np.random.randn(5, 3))
         cond = df > 0
Repository: pandas-dev/pandas

Issue:
BUG: duplicate columns and drop buggy
```
In [21]: df = DataFrame({'A' : np.random.randn(5),
 'B' : np.random.randn(5),'C' : np.random.randn(5), 
  'D':['a','b','c','d','e'] })

In [22]: df
Out[22]: 
          A         B         C  D
0 -0.941264  0.272726 -0.547948  a
1  0.069432  1.398414  0.039093  b
2 -0.073638 -1.264676 -1.339994  c
3  1.520017 -0.562979 -0.739326  d
4 -0.395157  0.542807  0.766582  e

[5 rows x 4 columns]

In [23]: df.take([2,0,1,2,3], axis=1).drop('C',axis=1)
Out[23]: 
          A         B  D
0 -0.941264  0.272726  a
1  0.069432  1.398414  b
2 -0.073638 -1.264676  c
3  1.520017 -0.562979  d
4 -0.395157  0.542807  e

[5 rows x 3 columns]

In [24]: df.take([2,0,1,2,1], axis=1).drop('C',axis=1)
IndexError: index 3 is out of bounds for axis 0 with size 3
```



Discussion:

Resolving patch:
diff --git a/pandas/core/internals.py b/pandas/core/internals.py
--- a/pandas/core/internals.py
+++ b/pandas/core/internals.py
@@ -3399,6 +3399,11 @@ def reindex_items(self, new_items, indexer=None, copy=True,
                 new_blocks.append(na_block)
                 new_blocks = _consolidate(new_blocks, new_items)
 
+            # consolidate
+            # import for non-unique which creates a block for each item
+            # and they must be consolidated before passing on
+            new_blocks = _consolidate(new_blocks, new_items)
+
         return self.__class__(new_blocks, new_axes)
 
     def _make_na_block(self, items, ref_items, placement=None,


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -3243,6 +3243,19 @@ def check(result, expected=None):
         df['that'] = 1
         check(df, expected)
 
+    def test_column_dups2(self):
+
+        # drop buggy GH 6240
+        df = DataFrame({'A' : np.random.randn(5),
+                        'B' : np.random.randn(5),
+                        'C' : np.random.randn(5),
+                        'D' : ['a','b','c','d','e'] })
+
+        expected = df.take([0,1,1], axis=1)
+        df2 = df.take([2,0,1,2,1], axis=1)
+        result = df2.drop('C',axis=1)
+        assert_frame_equal(result, expected)
+
     def test_column_dups_indexing(self):
 
         def check(result, expected=None):
Repository: pandas-dev/pandas

Issue:
isnull() and notnull() - documentation
Hi,

It will be nice if someone can add that
`isnull()` function have a contrary which is called `notnull()`
and
`notnull()` function have a contrary which is called `isnull()`
into documentation

http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.notnull.html
http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.isnull.html

That's probably stupid for you but I was searching a long time to find that
(I was looking for `isnotnull()`)

Maybe this documentation could also tell that

`df[df['col'].notnull()]` is same as ``df[~df['col'].isnull()]`

Kind regards



Discussion:
care to do a pull-request for this?

you can do just add a `see also section`

You should add something like this to the docstring:

```
See also
--------
isnull : short explanation
```

I'm sorry but I don't speak enough English to help documentation. My explanations could be confusing.


Resolving patch:
diff --git a/pandas/core/common.py b/pandas/core/common.py
--- a/pandas/core/common.py
+++ b/pandas/core/common.py
@@ -121,6 +121,10 @@ def isnull(obj):
     isnulled : array-like of bool or bool
         Array or bool indicating whether an object is null or if an array is
         given which of the element is null.
+
+    See also
+    --------
+    pandas.notnull: boolean inverse of pandas.isnull    
     """
     return _isnull(obj)
 
@@ -268,6 +272,10 @@ def notnull(obj):
     isnulled : array-like of bool or bool
         Array or bool indicating whether an object is *not* null or if an array
         is given which of the element is *not* null.
+    
+    See also
+    --------
+    pandas.isnull : boolean inverse of pandas.notnull
     """
     res = isnull(obj)
     if np.isscalar(res):
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -2545,12 +2545,20 @@ def interpolate(self, method='linear', axis=0, limit=None, inplace=False,
     def isnull(self):
         """
         Return a boolean same-sized object indicating if the values are null
+
+        See also
+        --------
+        notnull : boolean inverse of isnull
         """
         return isnull(self).__finalize__(self)
 
     def notnull(self):
         """Return a boolean same-sized object indicating if the values are
         not null
+
+        See also
+        --------
+        isnull : boolean inverse of notnull
         """
         return notnull(self).__finalize__(self)
 


Test patch:
Repository: pandas-dev/pandas

Issue:
infer_freq fails on Index
Continuing with the my projects are the pandas test suite theme of issue reports. This broke for me on a recent upgrade. 

Ok.

```
pd.infer_freq(['2004-01', '2004-02', '2004-03', '2004-04'])
```

No longer works.

```
pd.infer_freq(pd.Index(['2004-01', '2004-02', '2004-03', '2004-04']))
```

Seems like this should still work.

```
[12]: pd.version.version
[12]: '0.13.1-254-g150f323'
```



Discussion:
hmm...ok. I guess that should work


Resolving patch:
diff --git a/pandas/tseries/frequencies.py b/pandas/tseries/frequencies.py
--- a/pandas/tseries/frequencies.py
+++ b/pandas/tseries/frequencies.py
@@ -655,8 +655,11 @@ def infer_freq(index, warn=True):
     if isinstance(index, pd.PeriodIndex):
         raise TypeError("PeriodIndex given. Check the `freq` attribute "
                          "instead of using infer_freq.")
-    if not isinstance(index, pd.DatetimeIndex) and isinstance(index, pd.Index):
-        raise TypeError("cannot infer freq from a non-convertible index type {0}".format(type(index)))
+    if isinstance(index, pd.Index) and not isinstance(index, pd.DatetimeIndex):
+        if isinstance(index, (pd.Int64Index, pd.Float64Index)):
+            raise TypeError("cannot infer freq from a non-convertible index type {0}".format(type(index)))
+        index = index.values
+
     index = pd.DatetimeIndex(index)
     inferer = _FrequencyInferer(index, warn=warn)
     return inferer.get_freq()


Test patch:
diff --git a/pandas/tseries/tests/test_frequencies.py b/pandas/tseries/tests/test_frequencies.py
--- a/pandas/tseries/tests/test_frequencies.py
+++ b/pandas/tseries/tests/test_frequencies.py
@@ -274,11 +274,20 @@ def test_invalid_index_types(self):
         # test all index types
         for i in [ tm.makeIntIndex(10),
                    tm.makeFloatIndex(10),
-                   tm.makeStringIndex(10),
-                   tm.makeUnicodeIndex(10),
                    tm.makePeriodIndex(10) ]:
             self.assertRaises(TypeError, lambda : infer_freq(i))
 
+        for i in [ tm.makeStringIndex(10),
+                   tm.makeUnicodeIndex(10) ]:
+            self.assertRaises(ValueError, lambda : infer_freq(i))
+
+    def test_string_datetimelike_compat(self):
+
+        # GH 6463
+        expected = infer_freq(['2004-01', '2004-02', '2004-03', '2004-04'])
+        result = infer_freq(Index(['2004-01', '2004-02', '2004-03', '2004-04']))
+        self.assertEqual(result,expected)
+
     def test_series(self):
 
         # GH6407