internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pandas-dev/pandas

Issue:
CLN: Fixups to exceptions in generic
Improved message and better exception types.



Discussion:
@jtratner this still relevant? close?

I'll either close or fix this tonight.
On Oct 8, 2013 7:47 AM, "jreback" notifications@github.com wrote:

> @jtratner https://github.com/jtratner this still relevant? close?
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/5051#issuecomment-25883023
> .

this looks fine....merge at your leisure

@jtratner closing this?

I'll push it to 0.14 - I think there's one item from this I want to include.

np.


Resolving patch:
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -233,7 +233,7 @@ def _construct_axes_from_arguments(self, args, kwargs, require_all=False):
             if alias is not None:
                 if a in kwargs:
                     if alias in kwargs:
-                        raise Exception(
+                        raise TypeError(
                             "arguments are multually exclusive for [%s,%s]" % (a, alias))
                     continue
                 if alias in kwargs:
@@ -246,8 +246,8 @@ def _construct_axes_from_arguments(self, args, kwargs, require_all=False):
                     kwargs[a] = args.pop(0)
                 except (IndexError):
                     if require_all:
-                        raise AssertionError(
-                            "not enough arguments specified!")
+                        raise TypeError(
+                            "not enough/duplicate arguments specified!")
 
         axes = dict([(a, kwargs.get(a)) for a in self._AXIS_ORDERS])
         return axes, kwargs


Test patch:
diff --git a/pandas/tests/test_panel.py b/pandas/tests/test_panel.py
--- a/pandas/tests/test_panel.py
+++ b/pandas/tests/test_panel.py
@@ -1248,14 +1248,12 @@ def test_transpose(self):
         expected = self.panel.swapaxes('items', 'minor')
         assert_panel_equal(result, expected)
 
-        ## test bad aliases
-        # test ambiguous aliases
-        self.assertRaises(AssertionError, self.panel.transpose, 'minor',
-                          maj='major', majo='items')
-
-        # test invalid kwargs
-        self.assertRaises(AssertionError, self.panel.transpose, 'minor',
-                          maj='major', minor='items')
+        # duplicate axes
+        with tm.assertRaisesRegexp(TypeError, 'not enough/duplicate arguments'):
+            self.panel.transpose('minor', maj='major', minor='items')
+
+        with tm.assertRaisesRegexp(ValueError, 'repeated axis in transpose'):
+            self.panel.transpose('minor', 'major', major='minor', minor='items')
 
         result = self.panel.transpose(2, 1, 0)
         assert_panel_equal(result, expected)
Repository: pandas-dev/pandas

Issue:
PERF: slow assignement of an index to a frame
http://stackoverflow.com/questions/19578736/timespan-of-groups-in-pandas-timeseries



Discussion:

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -1914,7 +1914,11 @@ def insert(self, loc, column, value, allow_duplicates=False):
     def _sanitize_column(self, key, value):
         # Need to make sure new columns (which go into the BlockManager as new
         # blocks) are always copied
-        if _is_sequence(value):
+
+        # dont' need further processing on an equal index
+        if isinstance(value, Index) and (not len(self.index) or value.equals(self.index)):
+                value = value.values.copy()
+        elif isinstance(value, Series) or _is_sequence(value):
             is_frame = isinstance(value, DataFrame)
             if isinstance(value, Series) or is_frame:
                 if value.index.equals(self.index) or not len(self.index):
diff --git a/vb_suite/frame_methods.py b/vb_suite/frame_methods.py
--- a/vb_suite/frame_methods.py
+++ b/vb_suite/frame_methods.py
@@ -120,6 +120,21 @@ def j():
 frame_getitem_single_column2 = Benchmark('j()', setup,
                                          start_date=datetime(2010, 6, 1))
 
+#----------------------------------------------------------------------
+# assignment
+
+setup = common_setup + """
+idx = date_range('1/1/2000', periods=100000, freq='D')
+df = DataFrame(randn(100000, 1),columns=['A'],index=idx)
+def f(x):
+    x = x.copy()
+    x['date'] = x.index
+"""
+
+frame_assign_timeseries_index = Benchmark('f(df)', setup,
+                                          start_date=datetime(2013, 10, 1))
+
+
 #----------------------------------------------------------------------
 # to_string
 
diff --git a/vb_suite/timeseries.py b/vb_suite/timeseries.py
--- a/vb_suite/timeseries.py
+++ b/vb_suite/timeseries.py
@@ -229,12 +229,12 @@ def date_range(start=None, end=None, periods=None, freq=None):
 # tz_localize with infer argument.  This is an attempt to emulate the results
 # of read_csv with duplicated data.  Not passing infer_dst will fail
 setup = common_setup + """
-dst_rng = date_range('10/29/2000 1:00:00', 
+dst_rng = date_range('10/29/2000 1:00:00',
                      '10/29/2000 1:59:59', freq='S')
 index = date_range('10/29/2000', '10/29/2000 00:59:59', freq='S')
 index = index.append(dst_rng)
 index = index.append(dst_rng)
-index = index.append(date_range('10/29/2000 2:00:00', 
+index = index.append(date_range('10/29/2000 2:00:00',
                                 '10/29/2000 3:00:00', freq='S'))
 """
 


Test patch:
Repository: pandas-dev/pandas

Issue:
BUG: embedded pandas object cause formatting exceptions
http://stackoverflow.com/questions/19594003/should-it-be-supported-to-store-a-pandas-dataframe-in-another-pandas-dataframe



Discussion:

Resolving patch:
diff --git a/pandas/core/format.py b/pandas/core/format.py
--- a/pandas/core/format.py
+++ b/pandas/core/format.py
@@ -3,6 +3,7 @@
 
 import sys
 
+from pandas.core.base import PandasObject
 from pandas.core.common import adjoin, isnull, notnull
 from pandas.core.index import Index, MultiIndex, _ensure_index
 from pandas import compat
@@ -1486,6 +1487,8 @@ def _format(x):
                 if x is None:
                     return 'None'
                 return self.na_rep
+            elif isinstance(x, PandasObject):
+                return '%s' % x
             else:
                 # object dtype
                 return '%s' % formatter(x)


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -2356,6 +2356,21 @@ def test_constructor_error_msgs(self):
         with assertRaisesRegexp(ValueError, 'If using all scalar values, you must must pass an index'):
             DataFrame({'a': False, 'b': True})
 
+    def test_constructor_with_embedded_frames(self):
+
+        # embedded data frames
+        df1 = DataFrame({'a':[1, 2, 3], 'b':[3, 4, 5]})
+        df2 = DataFrame([df1, df1+10])
+
+        df2.dtypes
+        str(df2)
+
+        result = df2.loc[0,0]
+        assert_frame_equal(result,df1)
+
+        result = df2.loc[1,0]
+        assert_frame_equal(result,df1+10)
+
     def test_insert_error_msmgs(self):
 
         # GH 4107, more descriptive error message
Repository: pandas-dev/pandas

Issue:
BUG: set_index not resetting ref_locs
http://stackoverflow.com/questions/19612650/pandas-multiindex-names-not-working



Discussion:

Resolving patch:
diff --git a/pandas/core/internals.py b/pandas/core/internals.py
--- a/pandas/core/internals.py
+++ b/pandas/core/internals.py
@@ -1992,6 +1992,9 @@ def _set_ref_locs(self, labels=None, do_refs=False):
             self._ref_locs = rl
             return rl
 
+        elif do_refs:
+            self._reset_ref_locs()
+
         # return our cached _ref_locs (or will compute again
         # when we recreate the block manager if needed
         return getattr(self, '_ref_locs', None)


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -8551,6 +8551,19 @@ def test_rename_inplace(self):
         self.assert_('foo' in frame)
         self.assert_(id(frame['foo']) != c_id)
 
+    def test_rename_bug(self):
+        # GH 5344
+        # rename set ref_locs, and set_index was not resetting
+        df = DataFrame({ 0 : ['foo','bar'], 1 : ['bah','bas'], 2 : [1,2]})
+        df = df.rename(columns={0 : 'a'})
+        df = df.rename(columns={1 : 'b'})
+        df = df.set_index(['a','b'])
+        df.columns = ['2001-01-01']
+        expected = DataFrame([[1],[2]],index=MultiIndex.from_tuples([('foo','bah'),('bar','bas')],
+                                                                    names=['a','b']),
+                             columns=['2001-01-01'])
+        assert_frame_equal(df,expected)
+
     #----------------------------------------------------------------------
     # Time series related
     def test_diff(self):
Repository: pandas-dev/pandas

Issue:
DOC/BUG: fix details for 'quoting' parser parameter
``` python
Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

Imported NumPy 1.7.1, SciPy 0.12.0, Matplotlib 1.3.0 + guidata 1.6.1, guiqwt 2.3.1
Type "scientific" for more details.
>>> from csv import QUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONNUMERIC, QUOTE_NONE
>>> QUOTE_MINIMAL
0
>>> QUOTE_ALL
1
>>> QUOTE_NONNUMERIC
2
>>> QUOTE_NONE
3
>>> 
```



Discussion:

Resolving patch:
diff --git a/pandas/io/parsers.py b/pandas/io/parsers.py
--- a/pandas/io/parsers.py
+++ b/pandas/io/parsers.py
@@ -36,13 +36,13 @@
 %s
 lineterminator : string (length 1), default None
     Character to break file into lines. Only valid with C parser
-quotechar : string
-    The character to used to denote the start and end of a quoted item. Quoted
+quotechar : string (length 1)
+    The character used to denote the start and end of a quoted item. Quoted
     items can include the delimiter and it will be ignored.
-quoting : int
-    Controls whether quotes should be recognized. Values are taken from
-    `csv.QUOTE_*` values. Acceptable values are 0, 1, 2, and 3 for
-    QUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONE, and QUOTE_NONNUMERIC, respectively.
+quoting : int or csv.QUOTE_* instance, default None
+    Control field quoting behavior per ``csv.QUOTE_*`` constants. Use one of
+    QUOTE_MINIMAL (0), QUOTE_ALL (1), QUOTE_NONNUMERIC (2) or QUOTE_NONE (3).
+    Default (None) results in QUOTE_MINIMAL behavior.
 skipinitialspace : boolean, default False
     Skip spaces after delimiter
 escapechar : string


Test patch:
Repository: pandas-dev/pandas

Issue:
BLD: plot failures in master
This started after I merged #5375 (which passed cleanly before merging)

https://travis-ci.org/pydata/pandas/jobs/13376953



Discussion:
seems dateutil 2.2 was released 11/1/13...and broke things...and I can't find the release notes.....


Resolving patch:
diff --git a/pandas/tseries/tools.py b/pandas/tseries/tools.py
--- a/pandas/tseries/tools.py
+++ b/pandas/tseries/tools.py
@@ -305,6 +305,10 @@ def dateutil_parse(timestr, default,
 
     res = DEFAULTPARSER._parse(fobj, **kwargs)
 
+    # dateutil 2.2 compat
+    if isinstance(res, tuple):
+        res, _ = res
+
     if res is None:
         raise ValueError("unknown string format")
 


Test patch:
Repository: pandas-dev/pandas

Issue:
HDFStore.select slowed by decode even when using columns=
I realized when profiling a slow select (200% more wall-time as direct pytables call and high memory usage)  that most of the time is spend inside bytes.decode called by [_unconvert_strings_array](https://github.com/pydata/pandas/blob/master/pandas/io/pytables.py#L3966), even when selecting only int64 columns. It seems spend time and memory to decode string that are never returned.

I'm using python 3.3 and latest pandas (commit 2d2e8b5146024a90001f96862cdd0172adb4d1b8).

I gladly get back with more details if needed.



Discussion:
can you provide a reproducable example?

try making the columns that you are using `date_columns=list_of_columns_I_want` when you append; that way ONLY those will be selected, see [here](http://pandas.pydata.org/pandas-docs/dev/io.html#query-via-data-columns)

Thanks for your reply!

I tested it with and without data_columns, nothing changes. Here's an example:

```
h5 = pd.HDFStore('test.hdf5')
data = pd.DataFrame(np.random.rand(1000000,2), columns=['a', 'b'])
data['b'] = data['b'].apply(str)
del h5['test']
h5.append('test', data)
%prun h5.select('test', columns=['a'])
```

just the existence of a string column will make it slow because of the decode, even if the decoded data is not needed. Also using `data_columns = ['a']` and with a query on a, nothing changes about the 100000 calls to decode :/


Resolving patch:
diff --git a/pandas/io/pytables.py b/pandas/io/pytables.py
--- a/pandas/io/pytables.py
+++ b/pandas/io/pytables.py
@@ -3972,8 +3972,11 @@ def _unconvert_string_array(data, nan_rep=None, encoding=None):
     # where the passed encoding is actually None)
     encoding = _ensure_encoding(encoding)
     if encoding is not None and len(data):
-        f = np.vectorize(lambda x: x.decode(encoding), otypes=[np.object])
-        data = f(data)
+        try:
+            data = data.astype(str).astype(object)
+        except:
+            f = np.vectorize(lambda x: x.decode(encoding), otypes=[np.object])
+            data = f(data)
 
     if nan_rep is None:
         nan_rep = 'nan'


Test patch:
Repository: pandas-dev/pandas

Issue:
missing _is_copy causes strangeness
Note the duplication of the assignment is not a copy/paste error-- the first one works, the second one fails, presumably because a different path is taken if the column already exists than if it doesn't.

```
>>> pd.__version__
'0.12.0-1081-ge684bdc'
>>> df = pd.DataFrame({"A": [1,2]})
>>> df._is_copy
False
>>> df.to_pickle("tmp.pk")
>>> df2 = pd.read_pickle("tmp.pk")
>>> hasattr(df2, "_is_copy")
False
>>> df2["B"] = df2["A"]
>>> df2["B"] = df2["A"]
Traceback (most recent call last):
  File "<ipython-input-155-e1fb2db534a8>", line 1, in <module>
    df2["B"] = df2["A"]
  File "/usr/local/lib/python2.7/dist-packages/pandas-0.12.0_1081_ge684bdc-py2.7-linux-i686.egg/pandas/core/frame.py", line 1841, in __setitem__
    self._set_item(key, value)
  File "/usr/local/lib/python2.7/dist-packages/pandas-0.12.0_1081_ge684bdc-py2.7-linux-i686.egg/pandas/core/frame.py", line 1907, in _set_item
    self._check_setitem_copy()
  File "/usr/local/lib/python2.7/dist-packages/pandas-0.12.0_1081_ge684bdc-py2.7-linux-i686.egg/pandas/core/generic.py", line 1001, in _check_setitem_copy
    if self._is_copy:
  File "/usr/local/lib/python2.7/dist-packages/pandas-0.12.0_1081_ge684bdc-py2.7-linux-i686.egg/pandas/core/generic.py", line 1525, in __getattr__
    (type(self).__name__, name))
AttributeError: 'DataFrame' object has no attribute '_is_copy'
```



Discussion:
easy fix

unpicking doesn't hit the normal creation machinery 

thanks for the report 

Why not just define '_is_copy=None' on NDFrame?
On Nov 8, 2013 4:45 PM, "jreback" notifications@github.com wrote:

> easy fix
> 
> unpicking doesn't hit the normal creation machinery
> 
> thanks for the report
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/issues/5475#issuecomment-28100586
> .

That way, worst that happens is that we accidentally don't name something a
copy


Resolving patch:
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -74,6 +74,7 @@ class NDFrame(PandasObject):
         '_data', 'name', '_cacher', '_is_copy', '_subtyp', '_index', '_default_kind', '_default_fill_value']
     _internal_names_set = set(_internal_names)
     _metadata = []
+    _is_copy = None
 
     def __init__(self, data, axes=None, copy=False, dtype=None, fastpath=False):
 


Test patch:
diff --git a/pandas/tests/test_indexing.py b/pandas/tests/test_indexing.py
--- a/pandas/tests/test_indexing.py
+++ b/pandas/tests/test_indexing.py
@@ -1736,6 +1736,16 @@ def f():
         df = DataFrame({'A':['aaa','bbb','ccc'],'B':[1,2,3]})
         df.loc[0]['A'] = 111
 
+        # make sure that _is_copy is picked up reconstruction
+        # GH5475
+        df = DataFrame({"A": [1,2]})
+        self.assert_(df._is_copy is False)
+        with tm.ensure_clean('__tmp__pickle') as path:
+            df.to_pickle(path)
+            df2 = pd.read_pickle(path)
+            df2["B"] = df2["A"]
+            df2["B"] = df2["A"]
+
     def test_floating_index_doc_example(self):
 
         index = Index([1.5, 2, 3, 4.5, 5])
Repository: pandas-dev/pandas

Issue:
CLN: Move print_versions into pandas proper.
Moves print_versions under util so that it can be called from buildbot
(and so we can tell people to run `from pandas.util.print_versions import show_versions; show_versions()`
when trying to debug)

cc @yarikoptic



Discussion:
Any objections? This is minor...

fine by me

I checked that running `python print_versions.py` works and this runs appropriately on Travis [i.e. it prints versions]

@yarikoptic - ask and ye shall receive, you can use:

``` python
from pandas.util.print_versions import show_versions
show_versions()
```


Resolving patch:
diff --git a/ci/print_versions.py b/ci/print_versions.py
--- a/ci/print_versions.py
+++ b/ci/print_versions.py
@@ -2,15 +2,15 @@
 
 
 def show_versions():
-    import subprocess
+    import imp
     import os
     fn = __file__
     this_dir = os.path.dirname(fn)
-    pandas_dir = os.path.dirname(this_dir)
-    sv_path = os.path.join(pandas_dir, 'pandas', 'util',
-                            'print_versions.py')
-    return subprocess.check_call(['python', sv_path])
+    pandas_dir = os.path.abspath(os.path.join(this_dir,".."))
+    sv_path = os.path.join(pandas_dir, 'pandas','util')
+    mod = imp.load_module('pvmod', *imp.find_module('print_versions', [sv_path]))
+    return mod.show_versions()
 
 
 if __name__ == '__main__':
-    show_versions()
+    return show_versions()


Test patch:
Repository: pandas-dev/pandas

Issue:
df.set_index() broken
http://nbviewer.ipython.org/gist/michaelaye/7685834

Version: d057fc9  ('0.12.0-1178-gd057fc9')

I tried a sequence of "python setup.py clean && python setup.py install" without success.



Discussion:
are u trying to set a as a float index ?

set_index is not failing but rather the rendering of the returned Dataframe

the problem is the repr is not using iloc to select rows and is using getitem 
I'll fix in a bit

that was only for the example. In my case I have a TimeSeries I want to make an index and I get that error. Ok, sorry for jumping the gun to set_index, I was wondering how that could have been missed by the tests.

Display code, all you guys' favorite stuff it seems. ;)


Resolving patch:
diff --git a/pandas/core/format.py b/pandas/core/format.py
--- a/pandas/core/format.py
+++ b/pandas/core/format.py
@@ -284,6 +284,7 @@ def __init__(self, frame, buf=None, columns=None, col_space=None,
         self.line_width = line_width
         self.max_rows = max_rows
         self.max_cols = max_cols
+        self.max_rows_displayed = min(max_rows or len(self.frame),len(self.frame))
         self.show_dimensions = show_dimensions
 
         if justify is None:
@@ -483,7 +484,7 @@ def write(buf, frame, column_format, strcols):
 
     def _format_col(self, i):
         formatter = self._get_formatter(i)
-        return format_array(self.frame.icol(i)[:self.max_rows].get_values(),
+        return format_array((self.frame.iloc[:self.max_rows_displayed,i]).get_values(),
                             formatter, float_format=self.float_format,
                             na_rep=self.na_rep,
                             space=self.col_space)


Test patch:
diff --git a/pandas/tests/test_format.py b/pandas/tests/test_format.py
--- a/pandas/tests/test_format.py
+++ b/pandas/tests/test_format.py
@@ -1455,6 +1455,22 @@ def test_repr_html_long(self):
         assert u('%d rows ') % h in long_repr
         assert u('2 columns') in long_repr
 
+    def test_repr_html_float(self):
+        max_rows = get_option('display.max_rows')
+        h = max_rows - 1
+        df = pandas.DataFrame({'idx':np.linspace(-10,10,h), 'A':np.arange(1,1+h), 'B': np.arange(41, 41+h) }).set_index('idx')
+        reg_repr = df._repr_html_()
+        assert '...' not in reg_repr
+        assert str(40 + h) in reg_repr
+
+        h = max_rows + 1
+        df = pandas.DataFrame({'idx':np.linspace(-10,10,h), 'A':np.arange(1,1+h), 'B': np.arange(41, 41+h) }).set_index('idx')
+        long_repr = df._repr_html_()
+        assert '...' in long_repr
+        assert str(40 + h) not in long_repr
+        assert u('%d rows ') % h in long_repr
+        assert u('2 columns') in long_repr
+
     def test_repr_html_long_multiindex(self):
         max_rows = get_option('display.max_rows')
         max_L1 = max_rows//2
Repository: pandas-dev/pandas

Issue:
performance of DataFrame.apply between 0.12 and 0.13rc1
Here is a small example of a performance regression I've noticed between 0.12 and 0.13rc1:

``` python
s = pd.Series(np.arange(4096.))
df = pd.DataFrame({i:s for i in range(4096)})

# under 0.12
 %timeit df.apply((lambda x: np.corrcoef(x, s)[0, 1]), axis=1)
1 loops, best of 3: 792 ms per loop

# under 0.13rc1
%timeit df.apply((lambda x: np.corrcoef(x, s)[0, 1]), axis=1)
1 loops, best of 3: 1.7 s per loop
```

These are run on the same machine, with the following pip requirements consistent between the two setups besides pandas 0.12.0 vs 0.13.0rc1:
Cython==0.19.2
Jinja2==2.7.1
MarkupSafe==0.18
Pygments==1.6
Sphinx==1.1.3
argparse==1.2.1
docutils==0.11
ipython==1.0.0
matplotlib==1.3.0
nose==1.3.0
numpy==1.7.1
pyparsing==2.0.1
python-dateutil==2.2
pytz==2013.8
pyzmq==14.0.1
scipy==0.12.0
six==1.4.1
tornado==3.1.1
wsgiref==0.1.2



Discussion:

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -3434,7 +3434,7 @@ def _apply_standard(self, func, axis, ignore_failures=False, reduce=True):
 
                 if self._is_mixed_type:  # maybe a hack for now
                     raise AssertionError('Must be mixed type DataFrame')
-                values = self.values.ravel()
+                values = self.values
                 dummy = Series(NA, index=self._get_axis(axis),
                                dtype=values.dtype)
 
diff --git a/vb_suite/frame_methods.py b/vb_suite/frame_methods.py
--- a/vb_suite/frame_methods.py
+++ b/vb_suite/frame_methods.py
@@ -289,3 +289,12 @@ def f(K=100):
 frame_isnull  = Benchmark('isnull(df)', setup,
                            start_date=datetime(2012,1,1))
 
+#----------------------------------------------------------------------
+# apply
+
+setup = common_setup + """
+s = Series(np.arange(1028.))
+df = DataFrame({ i:s for i in range(1028) })
+"""
+frame_apply_user_func = Benchmark('df.apply(lambda x: np.corrcoef(x,s)[0,1])', setup,
+                           start_date=datetime(2012,1,1))


Test patch: