internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pandas-dev/pandas

Issue:
BUG: Categorical Series breaks when re-serializing with msgpack.
xref #8732 

```
df=pd.DataFrame([[1,'John P. Doe'],[2,'Jane Dove'],[1,'John P. Doe']])
df[1]=pd.Categorical(df[1])
```

DataFrames work fine
```
In [9]: pd.read_msgpack(df.to_msgpack()).dtypes
Out[9]: 
0       int64
1    category
dtype: object
```

Series are breaking on re-serialization
```
In [10]: pd.read_msgpack(df[1].to_msgpack())
ValueError: cannot specify a dtype with a Categorical unless dtype='category'
```


Discussion:
cc @iorilu 

Resolving patch:
diff --git a/pandas/io/packers.py b/pandas/io/packers.py
--- a/pandas/io/packers.py
+++ b/pandas/io/packers.py
@@ -593,17 +593,13 @@ def decode(obj):
     elif typ == u'series':
         dtype = dtype_for(obj[u'dtype'])
         pd_dtype = pandas_dtype(dtype)
-        np_dtype = pandas_dtype(dtype).base
 
         index = obj[u'index']
         result = globals()[obj[u'klass']](unconvert(obj[u'data'], dtype,
                                                     obj[u'compress']),
                                           index=index,
-                                          dtype=np_dtype,
+                                          dtype=pd_dtype,
                                           name=obj[u'name'])
-        tz = getattr(pd_dtype, 'tz', None)
-        if tz:
-            result = result.dt.tz_localize('UTC').dt.tz_convert(tz)
         return result
 
     elif typ == u'block_manager':


Test patch:
diff --git a/pandas/io/tests/test_packers.py b/pandas/io/tests/test_packers.py
--- a/pandas/io/tests/test_packers.py
+++ b/pandas/io/tests/test_packers.py
@@ -363,6 +363,8 @@ def setUp(self):
             'F': [Timestamp('20130102', tz='US/Eastern')] * 2 +
                  [Timestamp('20130603', tz='CET')] * 3,
             'G': [Timestamp('20130102', tz='US/Eastern')] * 5,
+            'H': Categorical([1, 2, 3, 4, 5]),
+            'I': Categorical([1, 2, 3, 4, 5], ordered=True),
         }
 
         self.d['float'] = Series(data['A'])
@@ -370,6 +372,8 @@ def setUp(self):
         self.d['mixed'] = Series(data['E'])
         self.d['dt_tz_mixed'] = Series(data['F'])
         self.d['dt_tz'] = Series(data['G'])
+        self.d['cat_ordered'] = Series(data['H'])
+        self.d['cat_unordered'] = Series(data['I'])
 
     def test_basic(self):
Repository: pandas-dev/pandas

Issue:
BUG: applymap on empty DataFrame returns Series
I'm unsure about a corner case I encountered:

``` python
>>> pd.DataFrame([]).applymap(round)
Series([], dtype: float64)
```

full example

```
In [1]: df = pandas.DataFrame({ 'x' : [], 'y' : [], 'z' : []})

In [2]: df
Out[2]: 
Empty DataFrame
Columns: [x, y, z]
Index: []

In [3]: df.applymap(lambda x: x)
Out[3]: 
x   NaN
y   NaN
z   NaN
dtype: float64

In [4]: df = pandas.DataFrame({ 'x' : [1], 'y' : [1], 'z' : [1]})

In [5]: df.applymap(lambda x: x)
Out[5]: 
   x  y  z
0  1  1  1
```

I would have expected to get the empty DataFrame back.



Discussion:
the test for reduction is on an empty Series (e.g. `round(Series([]))`)

so maybe not that sophisticated.

pull-request welcome to fix (though maybe tricky).

This is an API change from 0.12 and is not documented!

0.12:

```
In [1]: import pandas as pd

In [2]: df = pd.DataFrame()

In [3]: df
/Users/spike/.virtualenvs/pd_0_12/lib/python2.7/site-packages/pandas/core/config.py:570: DeprecationWarning: height has been deprecated.

  warnings.warn(d.msg, DeprecationWarning)

Out[3]:
Empty DataFrame
Columns: []
Index: []

In [4]: df.applymap(lambda x: 1)
Out[4]:
Empty DataFrame
Columns: []
Index: []
```

0.15:

```
In [1]: import pandas as pd

In [2]: df = pd.DataFrame()

In [3]: df
Out[3]:
Empty DataFrame
Columns: []
Index: []

In [4]: df.applymap(lambda x: 1)
Out[4]: Series([], dtype: float64)
```

@stringfellow well I suspect it actually changed in 0.13. Since their wasn't a test, a regression like this is very hard to detect. As I said above a pull-request to restore this behavior would be welcome.

I just checked this with 0.18.0 and I still get a Series back. Is this supposed to be fixed in the near future?

this issue is still open, pull-requests are welcome


Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -4288,6 +4288,8 @@ def applymap(self, func):
 
         # if we have a dtype == 'M8[ns]', provide boxed values
         def infer(x):
+            if x.empty:
+                return lib.map_infer(x, func)
             return lib.map_infer(x.asobject, func)
 
         return self.apply(infer)


Test patch:
diff --git a/pandas/tests/frame/test_apply.py b/pandas/tests/frame/test_apply.py
--- a/pandas/tests/frame/test_apply.py
+++ b/pandas/tests/frame/test_apply.py
@@ -405,6 +405,16 @@ def test_applymap(self):
         for f in ['datetime', 'timedelta']:
             self.assertEqual(result.loc[0, f], str(df.loc[0, f]))
 
+        # GH 8222
+        empty_frames = [pd.DataFrame(),
+                        pd.DataFrame(columns=list('ABC')),
+                        pd.DataFrame(index=list('ABC')),
+                        pd.DataFrame({'A': [], 'B': [], 'C': []})]
+        for frame in empty_frames:
+            for func in [round, lambda x: x]:
+                result = frame.applymap(func)
+                tm.assert_frame_equal(result, frame)
+
     def test_applymap_box(self):
         # ufunc will not be boxed. Same test cases as the test_map_box
         df = pd.DataFrame({'a': [pd.Timestamp('2011-01-01'),
Repository: pandas-dev/pandas

Issue:
DOC: docstrings of is_type_factory and is_instance_factory in config.py not accurate.
The validators returned by the functions don't return True, contrary to the docstrings' descriptions.
DOC: docstrings of is_type_factory and is_instance_factory in config.py not accurate.
The validators returned by the functions don't return True, contrary to the docstrings' descriptions.


Discussion:
@fumitoh exactly what are you referring? 

[here](https://github.com/pandas-dev/pandas/blob/b94186d4c58ee055656a84f55618be537db0095a/pandas/core/config.py#L750-L758) and [here](https://github.com/pandas-dev/pandas/blob/b94186d4c58ee055656a84f55618be537db0095a/pandas/core/config.py#L771-L772)

Both `inner` s return None.


but how is this incorrect?
the doc string says it's returning a function which it is; not the result of the function 
The doc string says:

    Returns
    -------
    validator - a function of a single argument x , which returns the
                True if type(x) is equal to `_type`

The validator does not return True. It's just for raising an error when invalid type objects are passed.
ok if you'd like to do a PR to fix would be fine


@fumitoh exactly what are you referring? 

[here](https://github.com/pandas-dev/pandas/blob/b94186d4c58ee055656a84f55618be537db0095a/pandas/core/config.py#L750-L758) and [here](https://github.com/pandas-dev/pandas/blob/b94186d4c58ee055656a84f55618be537db0095a/pandas/core/config.py#L771-L772)

Both `inner` s return None.


but how is this incorrect?
the doc string says it's returning a function which it is; not the result of the function 
The doc string says:

    Returns
    -------
    validator - a function of a single argument x , which returns the
                True if type(x) is equal to `_type`

The validator does not return True. It's just for raising an error when invalid type objects are passed.
ok if you'd like to do a PR to fix would be fine



Resolving patch:
diff --git a/pandas/core/config.py b/pandas/core/config.py
--- a/pandas/core/config.py
+++ b/pandas/core/config.py
@@ -747,8 +747,8 @@ def is_type_factory(_type):
 
     Returns
     -------
-    validator - a function of a single argument x , which returns the
-                True if type(x) is equal to `_type`
+    validator - a function of a single argument x , which raises
+                ValueError if type(x) is not equal to `_type`
 
     """
 
@@ -768,8 +768,8 @@ def is_instance_factory(_type):
 
     Returns
     -------
-    validator - a function of a single argument x , which returns the
-                True if x is an instance of `_type`
+    validator - a function of a single argument x , which raises
+                ValueError if x is not an instance of `_type`
 
     """
     if isinstance(_type, (tuple, list)):


Test patch:
Repository: pandas-dev/pandas

Issue:
DOC: Exponentially Weighted Windows page not showing formulas
The Exponentially Weighted Windows page is not rendering the formulas in LaTeX format.
http://pandas.pydata.org/pandas-docs/stable/computation.html#exponentially-weighted-windows



Discussion:
Thanks for the report.  Seemed to work as of 0.18.1.  @jorisvandenbossche any ideas?
http://pandas.pydata.org/pandas-docs/version/0.18.1/computation.html#exponentially-weighted-windows

Resolving patch:
diff --git a/doc/source/conf.py b/doc/source/conf.py
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -46,7 +46,7 @@
               'ipython_sphinxext.ipython_console_highlighting',
               'sphinx.ext.intersphinx',
               'sphinx.ext.coverage',
-              'sphinx.ext.pngmath',
+              'sphinx.ext.mathjax',
               'sphinx.ext.ifconfig',
               'sphinx.ext.linkcode',
               ]


Test patch:
Repository: pandas-dev/pandas

Issue:
DOC: add IndexSlice docstring + add to api docs
Hello Developers,

I could not find any information about "Pandas.IndexSlice" function in API Reference.There is only one example  at this [link](http://pandas.pydata.org/pandas-docs/stable/advanced.html#advanced-mi-slicers) . I want to filter the pivot table  as in Excel.But i am confused to using between loc(slice(...)) and IndexSlice.Please could you show some practical easy methods  to filter the pivot table.

Thank you very much...



Discussion:
Can you give a very concrete example of a dataframe and what you want to achieve? (by the way, for usage questions, something like StackOverflow is often a easier platform to get help) 
Further, everything you can do with `IndexSlice`, you can also do without it using a more verbose syntax (eg `slice(None)` instead of `:`). 

But indeed, `IndexSlice` should be added to the api overview

I had a question on [this link](http://stackoverflow.com/questions/35734146/how-to-filter-values-with-the-spesific-index-in-pivot-table-multidimensional-da/35735000#35735000) in stackover forums.Also I want to filter the multiple index integer values .For example , on 1999 and 2000 years , A country 's Commodity - Transaction and  Quantity  values.


Resolving patch:
diff --git a/pandas/core/indexing.py b/pandas/core/indexing.py
--- a/pandas/core/indexing.py
+++ b/pandas/core/indexing.py
@@ -43,6 +43,36 @@ def get_indexers_list():
 
 # the public IndexSlicerMaker
 class _IndexSlice(object):
+    """
+    Create an object to more easily perform multi-index slicing
+
+    Examples
+    --------
+
+    >>> midx = pd.MultiIndex.from_product([['A0','A1'], ['B0','B1','B2','B3']])
+    >>> columns = ['foo', 'bar']
+    >>> dfmi = pd.DataFrame(np.arange(16).reshape((len(midx), len(columns))),
+                            index=midx, columns=columns)
+
+    Using the default slice command:
+
+    >>> dfmi.loc[(slice(None), slice('B0', 'B1')), :]
+               foo  bar
+        A0 B0    0    1
+           B1    2    3
+        A1 B0    8    9
+           B1   10   11
+
+    Using the IndexSlice class for a more intuitive command:
+
+    >>> idx = pd.IndexSlice
+    >>> dfmi.loc[idx[:, 'B0':'B1'], :]
+               foo  bar
+        A0 B0    0    1
+           B1    2    3
+        A1 B0    8    9
+           B1   10   11
+    """
 
     def __getitem__(self, arg):
         return arg


Test patch:
Repository: pandas-dev/pandas

Issue:
DOC: update api.rst for Categorical accessors / methods
we have a ``Categorical`` section, but this mainly describes ``.cat``. Maybe split this out into a separate section to document the actual ``Categorical`` methods / accessors (even though it is *slightly* duplicative.

http://pandas-docs.github.io/pandas-docs-travis/api.html#categorical

also need to add other public methods (that are not available on ``.cat``). This is actually a big list, things like:

``isnull``, ``unique``, ``map``, ``value_counts``, etc.

These are all Series like methods / accessors.


Discussion:
@TomAugspurger @jorisvandenbossche 
I pushed a tiny update: https://github.com/pandas-dev/pandas/commit/b4c6fc1dc4fac2951c7e3ee035968760b4f4adb9

but I think should create a separate sub-section for this
I think this absence was rather intentional. Because people should actually mostly use categorical Series, we didn't want to give too much attention to plain Categorical in the docs?

Quoting from the API docs:

> The following two Categorical constructors are considered API

which shows the Categorical / Cateogorical.from_codes constructors. This implies that the rest of the methods are not considered as public API. 
I actually think a Categorical should not have a `value_counts`, so IMO should not be added.
the problem is we cannot reference these from *other* doc-strings then. I suppose we could have a hidden section?
I mainly wanted to reference ``Categorical.unique`` (for the examples, even though they *are* in ``pd.unique``)
I think we should just not reference them then, as you say yourself, the examples are also in `pd.unique` (but the categorical unique is indeed a case where there is now additional information in that docstring that is useful).

People will actually typically look at Series.unique docstring when they work interactively and have a categorical series. And looking at that now, there are no examples included there, but all are in `pd.unique`. So maybe we should include them there as well, or explicitly state "see pd.unique docstring for full documentation" ?

Resolving patch:
diff --git a/pandas/core/base.py b/pandas/core/base.py
--- a/pandas/core/base.py
+++ b/pandas/core/base.py
@@ -871,8 +871,9 @@ def value_counts(self, normalize=False, sort=True, ascending=False,
 
         See Also
         --------
-        pandas.unique
-        pandas.Categorical.unique
+        unique
+        Index.unique
+        Series.unique
         """)
 
     @Appender(_shared_docs['unique'] % _indexops_doc_kwargs)


Test patch:
Repository: pandas-dev/pandas

Issue:
'n/a' is not among default list of missing value strings
`n/a` denotes missing values in the [Brain Imaging Data Structure (BIDS)](https://www.nature.com/articles/sdata201644) standard. Adding it to the list of strings denoting missing values would make using pandas on BIDS datasets easier.


Discussion:

Resolving patch:
diff --git a/pandas/io/common.py b/pandas/io/common.py
--- a/pandas/io/common.py
+++ b/pandas/io/common.py
@@ -31,7 +31,7 @@
 # '1.#INF','-1.#INF', '1.#INF000000',
 _NA_VALUES = set([
     '-1.#IND', '1.#QNAN', '1.#IND', '-1.#QNAN', '#N/A N/A', '#N/A',
-    'N/A', 'NA', '#NA', 'NULL', 'null', 'NaN', '-NaN', 'nan', '-nan', ''
+    'N/A', 'n/a', 'NA', '#NA', 'NULL', 'null', 'NaN', '-NaN', 'nan', '-nan', ''
 ])
 
 try:


Test patch:
diff --git a/pandas/tests/io/parser/na_values.py b/pandas/tests/io/parser/na_values.py
--- a/pandas/tests/io/parser/na_values.py
+++ b/pandas/tests/io/parser/na_values.py
@@ -70,8 +70,8 @@ def test_non_string_na_values(self):
 
     def test_default_na_values(self):
         _NA_VALUES = set(['-1.#IND', '1.#QNAN', '1.#IND', '-1.#QNAN',
-                          '#N/A', 'N/A', 'NA', '#NA', 'NULL', 'null', 'NaN',
-                          'nan', '-NaN', '-nan', '#N/A N/A', ''])
+                          '#N/A', 'N/A', 'n/a', 'NA', '#NA', 'NULL', 'null',
+                          'NaN', 'nan', '-NaN', '-nan', '#N/A N/A', ''])
         assert _NA_VALUES == parsers._NA_VALUES
         nv = len(_NA_VALUES)
Repository: pandas-dev/pandas

Issue:
REGR: pd.unique casts nan to string with other strings
#### Code Sample, a copy-pastable example if possible

```python
In [1]: import numpy as np; import pandas as pd
In [2]: pd.unique(['a', np.nan, 'c', 'c'])
Out[2]: array(['a', 'nan', 'c'], dtype=object)

In [3]: pd.__version__
Out[3]: '0.20.0rc1'

```
Notice the quotes around `'nan'`. This came up from the seaborn test suite

w#### Expected Output

```python
# 0.19.2
In [1]: import numpy as np; import pandas as pd
In [2]: pd.unique(['a', np.nan, 'c', 'c'])
Out[2]: array(['a', nan, 'c'], dtype=object)
```


Discussion:
hmm, unique is taking a more general path now, I think this is inferred to string (and not object)

Resolving patch:
diff --git a/pandas/core/algorithms.py b/pandas/core/algorithms.py
--- a/pandas/core/algorithms.py
+++ b/pandas/core/algorithms.py
@@ -161,7 +161,11 @@ def _ensure_arraylike(values):
     """
     if not isinstance(values, (np.ndarray, ABCCategorical,
                                ABCIndexClass, ABCSeries)):
-        values = np.array(values)
+        inferred = lib.infer_dtype(values)
+        if inferred in ['mixed', 'string', 'unicode']:
+            values = np.asarray(values, dtype=object)
+        else:
+            values = np.asarray(values)
     return values
 
 


Test patch:
diff --git a/pandas/tests/indexes/common.py b/pandas/tests/indexes/common.py
--- a/pandas/tests/indexes/common.py
+++ b/pandas/tests/indexes/common.py
@@ -323,6 +323,7 @@ def test_get_unique_index(self):
 
             idx = ind[[0] * 5]
             idx_unique = ind[[0]]
+
             # We test against `idx_unique`, so first we make sure it's unique
             # and doesn't contain nans.
             self.assertTrue(idx_unique.is_unique)
@@ -336,7 +337,6 @@ def test_get_unique_index(self):
                 tm.assert_index_equal(result, idx_unique)
 
             # nans:
-
             if not ind._can_hold_na:
                 continue
 
diff --git a/pandas/tests/test_algos.py b/pandas/tests/test_algos.py
--- a/pandas/tests/test_algos.py
+++ b/pandas/tests/test_algos.py
@@ -387,6 +387,12 @@ def test_uint64_overflow(self):
         exp = np.array([1, 2, 2**63], dtype=np.uint64)
         tm.assert_numpy_array_equal(algos.unique(s), exp)
 
+    def test_nan_in_object_array(self):
+        l = ['a', np.nan, 'c', 'c']
+        result = pd.unique(l)
+        expected = np.array(['a', np.nan, 'c'], dtype=object)
+        tm.assert_numpy_array_equal(result, expected)
+
     def test_categorical(self):
 
         # we are expecting to return in the order
@@ -1378,8 +1384,8 @@ def test_no_mode(self):
         exp = Series([], dtype=np.float64)
         tm.assert_series_equal(algos.mode([]), exp)
 
-    # GH 15714
     def test_mode_single(self):
+        # GH 15714
         exp_single = [1]
         data_single = [1]
Repository: pandas-dev/pandas

Issue:
ENH: Add support for "nthreads" option from Feather 0.4.0
I'm releasing Feather 0.4.0 soon, and it has the option for multithreaded reads (one of the benefits of moving to the Arrow codebase). We should add corresponding support in the pandas interface


Discussion:

Resolving patch:
diff --git a/pandas/io/feather_format.py b/pandas/io/feather_format.py
--- a/pandas/io/feather_format.py
+++ b/pandas/io/feather_format.py
@@ -43,6 +43,7 @@ def to_feather(df, path):
     df : DataFrame
     path : string
         File path
+
     """
     path = _stringify_path(path)
     if not isinstance(df, DataFrame):
@@ -83,7 +84,7 @@ def to_feather(df, path):
     feather.write_dataframe(df, path)
 
 
-def read_feather(path):
+def read_feather(path, nthreads=1):
     """
     Load a feather-format object from the file path
 
@@ -93,6 +94,10 @@ def read_feather(path):
     ----------
     path : string
         File path
+    nthreads : int, default 1
+        Number of CPU threads to use when reading to pandas.DataFrame
+
+       .. versionadded 0.21.0
 
     Returns
     -------
@@ -102,4 +107,8 @@ def read_feather(path):
 
     feather = _try_import()
     path = _stringify_path(path)
-    return feather.read_dataframe(path)
+
+    if feather.__version__ < LooseVersion('0.4.0'):
+        return feather.read_dataframe(path)
+
+    return feather.read_dataframe(path, nthreads=nthreads)


Test patch:
diff --git a/pandas/tests/io/test_feather.py b/pandas/tests/io/test_feather.py
--- a/pandas/tests/io/test_feather.py
+++ b/pandas/tests/io/test_feather.py
@@ -27,11 +27,11 @@ def check_error_on_write(self, df, exc):
             with ensure_clean() as path:
                 to_feather(df, path)
 
-    def check_round_trip(self, df):
+    def check_round_trip(self, df, **kwargs):
 
         with ensure_clean() as path:
             to_feather(df, path)
-            result = read_feather(path)
+            result = read_feather(path, **kwargs)
             assert_frame_equal(result, df)
 
     def test_error(self):
@@ -98,6 +98,12 @@ def test_unsupported_other(self):
         df = pd.DataFrame({'a': pd.period_range('2013', freq='M', periods=3)})
         self.check_error_on_write(df, ValueError)
 
+    @pytest.mark.skipif(fv < '0.4.0', reason='new in 0.4.0')
+    def test_rw_nthreads(self):
+
+        df = pd.DataFrame({'A': np.arange(100000)})
+        self.check_round_trip(df, nthreads=2)
+
     def test_write_with_index(self):
 
         df = pd.DataFrame({'A': [1, 2, 3]})
Repository: pandas-dev/pandas

Issue:
COMPAT: __Pyx_PyNumber_IntOrLong raises ValueError on PyPy
the function [``__Pyx_PyNumber_IntOrLong``](https://github.com/cython/cython/blob/master/Cython/Utility/TypeConversion.c#L268) from cython raises an exception when the conversion fails. On CPython, this will always be a ``TypeError`` from line 314, but on other implementations i.e. PyPy the call to ``PyNumber_Int`` on line 298 raises a ``ValueError`` upon failure.

Pandas catches only a ``TypeError``. The easiest fix would be for pandas to catch both ``TypeError`` and ``ValueError`` when conversion fails, a harder fix would be to change cython to raise a ValueError on CPython as well, and then fix all the downstream libraries that use it (Or to scope the call to ``PyNumber_Int`` and convert the exception, and then require a minimum cython version for pandas-on-PyPy).

I already have the fix for catching multiple exceptions, should I issue a pull request for that?


Discussion:
can you show an example of where this happens?

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -1919,7 +1919,7 @@ def get_value(self, index, col, takeable=False):
 
         try:
             return engine.get_value(series._values, index)
-        except TypeError:
+        except (TypeError, ValueError):
 
             # we cannot handle direct indexing
             # use positional
diff --git a/pandas/core/indexes/base.py b/pandas/core/indexes/base.py
--- a/pandas/core/indexes/base.py
+++ b/pandas/core/indexes/base.py
@@ -1590,7 +1590,7 @@ def __contains__(self, key):
         hash(key)
         try:
             return key in self._engine
-        except TypeError:
+        except (TypeError, ValueError):
             return False
 
     _index_shared_docs['contains'] = """
@@ -1610,7 +1610,7 @@ def contains(self, key):
         hash(key)
         try:
             return key in self._engine
-        except TypeError:
+        except (TypeError, ValueError):
             return False
 
     def __hash__(self):


Test patch: