internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pandas-dev/pandas

Issue:
BUG: .str.cat for all-na corner case
A corner case that wrongly casts to FloatIndex if the result is all-na (e.g. due to alignment)
```
import pandas as pd
>>> idx = pd.Index(['a', 'b', 'c'])
>>> idz = pd.Index(['d', 'e', 'f'])
>>> idx.str.cat(idz, join='left')
Float64Index([nan, nan, nan], dtype='float64')
```


Discussion:

Resolving patch:
diff --git a/pandas/core/strings.py b/pandas/core/strings.py
--- a/pandas/core/strings.py
+++ b/pandas/core/strings.py
@@ -2260,9 +2260,11 @@ def cat(self, others=None, sep=None, na_rep=None, join=None):
             result = cat_core(all_cols, sep)
 
         if isinstance(self._orig, Index):
-            result = Index(result, name=self._orig.name)
+            # add dtype for case that result is all-NA
+            result = Index(result, dtype=object, name=self._orig.name)
         else:  # Series
-            result = Series(result, index=data.index, name=self._orig.name)
+            result = Series(result, dtype=object, index=data.index,
+                            name=self._orig.name)
         return result
 
     _shared_docs['str_split'] = ("""


Test patch:
diff --git a/pandas/tests/test_strings.py b/pandas/tests/test_strings.py
--- a/pandas/tests/test_strings.py
+++ b/pandas/tests/test_strings.py
@@ -630,11 +630,31 @@ def test_str_cat_align_mixed_inputs(self, join):
         with pytest.raises(ValueError, match=rgx):
             s.str.cat([t, z], join=join)
 
-    def test_str_cat_raises(self):
-        # non-strings hiding behind object dtype
-        s = Series([1, 2, 3, 4], dtype='object')
-        with pytest.raises(TypeError, match="unsupported operand type.*"):
-            s.str.cat(s)
+    @pytest.mark.parametrize('box', [Series, Index])
+    @pytest.mark.parametrize('other', [Series, Index])
+    def test_str_cat_all_na(self, box, other):
+        # GH 24044
+
+        # check that all NaNs in caller / target work
+        s = Index(['a', 'b', 'c', 'd'])
+        s = s if box == Index else Series(s, index=s)
+        t = other([np.nan] * 4, dtype=object)
+        # add index of s for alignment
+        t = t if other == Index else Series(t, index=s)
+
+        # all-NA target
+        if box == Series:
+            expected = Series([np.nan] * 4, index=s.index, dtype=object)
+        else:  # box == Index
+            expected = Index([np.nan] * 4, dtype=object)
+        result = s.str.cat(t, join='left')
+        assert_series_or_index_equal(result, expected)
+
+        # all-NA caller (only for Series)
+        if other == Series:
+            expected = Series([np.nan] * 4, dtype=object, index=t.index)
+            result = t.str.cat(s, join='left')
+            tm.assert_series_equal(result, expected)
 
     def test_str_cat_special_cases(self):
         s = Series(['a', 'b', 'c', 'd'])
Repository: pandas-dev/pandas

Issue:
BUG: to_datetime(Timestamp) utc argument
```
In [7]: pd.to_datetime(pd.Timestamp(2018,1,1), utc=True)
Out[7]: Timestamp('2018-01-01 00:00:00')

# This should raise
In [7]: pd.to_datetime(pd.Timestamp(2018,1,1, tz='US/Pacific'), utc=True)
Out[7]: Timestamp('2018-01-01 00:00:00')
```


Discussion:
to be honest I would actually deprecate ``box``; its not very useful externally.
Sure thing. I'd agree that `to_datetime` should prioritize keeping values in pandas data structures. Let's create a new issue regarding the deprecation of `box` in `to_datetime` and `to_timedelta`

Resolving patch:
diff --git a/pandas/core/tools/datetimes.py b/pandas/core/tools/datetimes.py
--- a/pandas/core/tools/datetimes.py
+++ b/pandas/core/tools/datetimes.py
@@ -568,6 +568,11 @@ def to_datetime(arg, errors='raise', dayfirst=False, yearfirst=False,
 
     if isinstance(arg, Timestamp):
         result = arg
+        if tz is not None:
+            if arg.tz is not None:
+                result = result.tz_convert(tz)
+            else:
+                result = result.tz_localize(tz)
     elif isinstance(arg, ABCSeries):
         cache_array = _maybe_cache(arg, format, cache, convert_listlike)
         if not cache_array.empty:


Test patch:
diff --git a/pandas/tests/indexes/datetimes/test_tools.py b/pandas/tests/indexes/datetimes/test_tools.py
--- a/pandas/tests/indexes/datetimes/test_tools.py
+++ b/pandas/tests/indexes/datetimes/test_tools.py
@@ -655,6 +655,16 @@ def test_non_iso_strings_with_tz_offset(self):
                                            tzinfo=pytz.FixedOffset(240))] * 2)
         tm.assert_index_equal(result, expected)
 
+    @pytest.mark.parametrize('ts, expected', [
+        (Timestamp('2018-01-01'),
+         Timestamp('2018-01-01', tz='UTC')),
+        (Timestamp('2018-01-01', tz='US/Pacific'),
+         Timestamp('2018-01-01 08:00', tz='UTC'))])
+    def test_timestamp_utc_true(self, ts, expected):
+        # GH 24415
+        result = to_datetime(ts, utc=True)
+        assert result == expected
+
 
 class TestToDatetimeUnit(object):
     @pytest.mark.parametrize('cache', [True, False])
Repository: pandas-dev/pandas

Issue:
DOC: Add dateutil to intersphinx
Looks like we'ree using `dateutil` items in the `See Also` section, for example:
https://pandas-docs.github.io/pandas-docs-travis/generated/pandas.tseries.offsets.DateOffset.html

But, for sphinx to generate the links, we need `dateutil` configured as an intersphinx item in `doc/source/conf.py`.


Discussion:

Resolving patch:
diff --git a/doc/source/conf.py b/doc/source/conf.py
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -368,13 +368,14 @@
 
 
 intersphinx_mapping = {
-    'statsmodels': ('http://www.statsmodels.org/devel/', None),
+    'dateutil': ("https://dateutil.readthedocs.io/en/latest/", None),
     'matplotlib': ('https://matplotlib.org/', None),
+    'numpy': ('https://docs.scipy.org/doc/numpy/', None),
     'pandas-gbq': ('https://pandas-gbq.readthedocs.io/en/latest/', None),
+    'py': ('https://pylib.readthedocs.io/en/latest/', None),
     'python': ('https://docs.python.org/3/', None),
-    'numpy': ('https://docs.scipy.org/doc/numpy/', None),
     'scipy': ('https://docs.scipy.org/doc/scipy/reference/', None),
-    'py': ('https://pylib.readthedocs.io/en/latest/', None)
+    'statsmodels': ('http://www.statsmodels.org/devel/', None),
 }
 import glob
 autosummary_generate = glob.glob("*.rst")


Test patch:
Repository: pandas-dev/pandas

Issue:
DOC: Split api.rst in sections
The pandas API reference page is huge. Which makes it difficult for users to find information, and for us to edit.

Given the size of the page, I think it'd make more sense to split the current page `api.rst` into different pages, one per section:
- `api/series.rst`
- `api/functions.rst`
- ...

The document is already divided into sections, I think the current division would make sense with few changes:
- Input/Output
- General functions
- Series
- DataFrame
- Panel
- Index (will include all the Index classes/section in the top level of `api.rst`)
- Scalars
- Date Offsets and Frequencies
- Window
- GroupBy
- Resampling
- Style
- Plotting
- General utility functions
- Extensions

Does anybody think a single page is better? Any other division you think of?


Discussion:
I agree, think this would definitely more approachable split up into pages; it's far too monolithic as it stands. Makes sense to follow the sections already defined.
Makes sense to me. One small nit is that I would combine `Date Offsets` with `Frequencies` since they are essentially the same thing and `Frequencies` only has the `to_offset` function under that section.

Resolving patch:
diff --git a/doc/make.py b/doc/make.py
--- a/doc/make.py
+++ b/doc/make.py
@@ -183,7 +183,7 @@ def clean():
         Clean documentation generated files.
         """
         shutil.rmtree(BUILD_PATH, ignore_errors=True)
-        shutil.rmtree(os.path.join(SOURCE_PATH, 'generated'),
+        shutil.rmtree(os.path.join(SOURCE_PATH, 'api', 'generated'),
                       ignore_errors=True)
 
     def zip_html(self):
diff --git a/scripts/validate_docstrings.py b/scripts/validate_docstrings.py
--- a/scripts/validate_docstrings.py
+++ b/scripts/validate_docstrings.py
@@ -17,6 +17,7 @@
 import sys
 import json
 import re
+import glob
 import functools
 import collections
 import argparse
@@ -776,9 +777,11 @@ def validate_all(prefix, ignore_deprecated=False):
     seen = {}
 
     # functions from the API docs
-    api_doc_fname = os.path.join(BASE_PATH, 'doc', 'source', 'api.rst')
-    with open(api_doc_fname) as f:
-        api_items = list(get_api_items(f))
+    api_doc_fnames = os.path.join(BASE_PATH, 'doc', 'source', 'api', '*.rst')
+    api_items = []
+    for api_doc_fname in glob.glob(api_doc_fnames):
+        with open(api_doc_fname) as f:
+            api_items += list(get_api_items(f))
     for func_name, func_obj, section, subsection in api_items:
         if prefix and not func_name.startswith(prefix):
             continue


Test patch:
Repository: pandas-dev/pandas

Issue:
DOC: Weird documentation (or behavior) of pd.read_csv escapechar= arg
#### Problem description

Documentation of pandas.read_csv states:

```
escapechar : str (length 1), default None

    One-character string used to escape delimiter when quoting is QUOTE_NONE.
```

But in reality you can correctly parse following CSV:

```
id, name
1, "Hello, my name is \"John\""
```

using this code:

```python
import pandas as pd

df=pd.read_csv("john.csv", quotechar='"', escapechar="\\")
print(df.name) # prints Hello, my name is "John"
print(df.id) # prints 1
```

Also same doc states that default quoting is QUOTE_MINIMAL, not QUOTE_NONE in example above.

Trying same with quoting=csv.QUOTE_NONE will give you incorrect result:

```python
import pandas as pd
import csv

df=pd.read_csv("john.csv", quotechar='"', escapechar="\\", quoting=csv.QUOTE_NONE)
print(df.name) # prints my name is John""
print(df.id) # prints 1, "Hello
```

Trying same without escapechar will also give you incorrect result:

```python
import pandas as pd

df=pd.read_csv("john.csv", quotechar='"')
print(df.name) # prints Hello, my name is \John\""
print(df.id) # prints 1
```

Looks like doc is misleading here, incorrectly stating that escapechar is used only for QUOTE_NONE when in fact it can be used with default mode (maybe as result of some bugfixes?)


Discussion:
@frenzykryger : Thanks for reporting this!  Indeed, the documentation on `escapechar` can be much improved and should really be more generic in being just an "escape character" (and not as tied to the `quoting` parameter).  PR is more than welcome!

Resolving patch:
diff --git a/pandas/io/parsers.py b/pandas/io/parsers.py
--- a/pandas/io/parsers.py
+++ b/pandas/io/parsers.py
@@ -261,7 +261,7 @@
    whether or not to interpret two consecutive quotechar elements INSIDE a
    field as a single ``quotechar`` element.
 escapechar : str (length 1), optional
-    One-character string used to escape delimiter when quoting is QUOTE_NONE.
+    One-character string used to escape other characters.
 comment : str, optional
     Indicates remainder of line should not be parsed. If found at the beginning
     of a line, the line will be ignored altogether. This parameter must be a


Test patch:
Repository: pandas-dev/pandas

Issue:
CI: db option is skipping lots of tests
https://travis-ci.org/pandas-dev/pandas/jobs/474120016

there are NO tests running! @datapythonista 


Discussion:
let's revert all of this db bizness. this is causing too many issues. @datapythonista ASAP pls.

Resolving patch:
diff --git a/pandas/conftest.py b/pandas/conftest.py
--- a/pandas/conftest.py
+++ b/pandas/conftest.py
@@ -1,4 +1,3 @@
-import collections
 from datetime import date, time, timedelta
 from decimal import Decimal
 import importlib
@@ -55,24 +54,14 @@ def pytest_runtest_setup(item):
     if 'network' in item.keywords and item.config.getoption("--skip-network"):
         pytest.skip("skipping due to --skip-network")
 
+    if 'db' in item.keywords and item.config.getoption("--skip-db"):
+        pytest.skip("skipping due to --skip-db")
+
     if 'high_memory' in item.keywords and not item.config.getoption(
             "--run-high-memory"):
         pytest.skip(
             "skipping high memory test since --run-high-memory was not set")
 
-    # if "db" not explicitly set in the -m pattern, we skip the db tests
-    pattern = item.config.getoption('-m')
-    if 'db' in item.keywords and not pattern:
-        pytest.skip('skipping db unless -m "db" is specified')
-    elif 'db' in item.keywords and pattern:
-        markers = collections.defaultdict(bool)
-        for marker in item.iter_markers():
-            markers[marker.name] = True
-        markers['db'] = False
-        db_in_pattern = not eval(pattern, {}, markers)
-        if not db_in_pattern:
-            pytest.skip('skipping db unless -m "db" is specified')
-
 
 # Configurations for all tests and all test modules
 
diff --git a/pandas/util/_tester.py b/pandas/util/_tester.py
--- a/pandas/util/_tester.py
+++ b/pandas/util/_tester.py
@@ -16,7 +16,7 @@ def test(extra_args=None):
         import hypothesis  # noqa
     except ImportError:
         raise ImportError("Need hypothesis>=3.58 to run tests")
-    cmd = ['--skip-slow', '--skip-network']
+    cmd = ['--skip-slow', '--skip-network', '--skip-db']
     if extra_args:
         if not isinstance(extra_args, list):
             extra_args = [extra_args]


Test patch:
diff --git a/test_fast.bat b/test_fast.bat
--- a/test_fast.bat
+++ b/test_fast.bat
@@ -1,3 +1,3 @@
 :: test on windows
 set PYTHONHASHSEED=314159265
-pytest --skip-slow --skip-network -m "not single" -n 4 -r sXX --strict pandas
+pytest --skip-slow --skip-network --skip-db -m "not single" -n 4 -r sXX --strict pandas
diff --git a/test_fast.sh b/test_fast.sh
--- a/test_fast.sh
+++ b/test_fast.sh
@@ -5,4 +5,4 @@
 # https://github.com/pytest-dev/pytest/issues/1075
 export PYTHONHASHSEED=$(python -c 'import random; print(random.randint(1, 4294967295))')
 
-pytest pandas --skip-slow --skip-network -m "not single" -n 4 -r sxX --strict "$@"
+pytest pandas --skip-slow --skip-network --skip-db -m "not single" -n 4 -r sxX --strict "$@"
Repository: pandas-dev/pandas

Issue:
Cython language_level Warning - switch to Python3?
When compiling with Cython the following warning gets issued:

```bash
Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: clones/pandas/pandas/_libs/groupby.pxd
```

Haven't looked into the implications as of yet but probably worth switching to Python3, especially given the upcoming drop of Python2 support


Discussion:
Have you tried this locally?  For me it causes `python setup.py build_ext --inplace` to fail
Yea it failed pretty miserably. Just for the sake of explicitness I updated the setup.py to have the following line:

```python
directives = {'linetrace': False, 'language_level': 3}
```

Haven't looked in detail but assume it's an issue with how we are doing imports
what is next on the list for this?
I think mostly minor stuff around division differences. There’s also one fused type containing Unicode which was failing before for Py27 builds. 

Most of this was wrapped up in https://github.com/pandas-dev/pandas/commit/62bee57d0fc07c35bd4b89cb18a3dd99aec0dc3e - is the Py27 failure something we need to worry about after 0.24 release?
no issue for 0.24.0, just saw the warnings and led me here :>

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -451,7 +451,7 @@ def run(self):
 # pinning `ext.cython_directives = directives` to each ext in extensions.
 # github.com/cython/cython/wiki/enhancements-compilerdirectives#in-setuppy
 directives = {'linetrace': False,
-              'language_level': 2}
+              'language_level': 3}
 macros = []
 if linetrace:
     # https://pypkg.com/pypi/pytest-cython/f/tests/example-project/setup.py


Test patch:
Repository: pandas-dev/pandas

Issue:
docstring templates not being filled
```
>>> print(pd.DatetimeIndex.fillna.__doc__)

Fill NA/NaN values with the specified value

Parameters
----------
value : scalar
    Scalar value to use to fill holes (e.g. 0).
    This value cannot be a list-likes.
downcast : dict, default is None
    a dict of item->dtype of what to downcast if possible,
    or the string 'infer' which will try to downcast to an appropriate
    equal type (e.g. float64 to int64 if possible)

Returns
-------
filled : %(klass)s
```

I'm pretty sure that the `%(klass)s` is supposed to be `DatetimeIndex`.


Discussion:
Looks like a missing template parameter in <a href="https://github.com/pandas-dev/pandas/blob/e76c90e/pandas/core/indexes/base.py#L1910-L1926" target="_blank">pandas/core/indexes/base.py</a>.

@jbrockmendel : Perhaps it should just be `Index` ?

~~~python
print(pd.DatetimeIndex.dropna.__doc__)
~~~

Outputs:

~~~
Return Index without NA/NaN values

Parameters
----------
how :  {'any', 'all'}, default 'any'
    If the Index is a MultiIndex, drop the value when any or all levels
    are NaN.

Returns
-------
valid : Index
~~~
cc @jreback 

Resolving patch:
diff --git a/pandas/core/indexes/base.py b/pandas/core/indexes/base.py
--- a/pandas/core/indexes/base.py
+++ b/pandas/core/indexes/base.py
@@ -1920,7 +1920,7 @@ def notna(self):
 
         Returns
         -------
-        filled : %(klass)s
+        filled : Index
         """
 
     @Appender(_index_shared_docs['fillna'])


Test patch:
Repository: pandas-dev/pandas

Issue:
DOC: `applymap` calling on first column twice
#### Code

```python
import pandas as pd
df=pd.DataFrame([[1,2,3],[4,5,6]])
df.index=['a','b'] #I was suspicious of the index
def bla(x):
    print('Applying to:',x)
    return x+1

df.applymap(bla)
```

#### Output

```python
    Applying to: 1
    Applying to: 4
    Applying to: 1
    Applying to: 4
    Applying to: 2
    Applying to: 5
    Applying to: 3
    Applying to: 6
       0  1  2
    a  2  3  4
    b  5  6  7
```
#### Problem description

Seems `applymap` is calling on the first column twice. ~~There is no effect in the end result, just a run-time hit as far as I can tell.~~ This affects functions with side effects, as well as runtime.

#### Expected Output

```Python
    Applying to: 1
    Applying to: 4
    Applying to: 2
    Applying to: 5
    Applying to: 3
    Applying to: 6
       0  1  2
    a  2  3  4
    b  5  6  7
```

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-8-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8

pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 36.0.1
Cython: None
numpy: 1.12.1
scipy: 0.18.1
statsmodels: None
xarray: None
IPython: 6.1.0
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.3.0
numexpr: 2.6.1
matplotlib: 2.0.0
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.5.3
html5lib: 0.999999999
httplib2: 0.9.2
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
boto: None
pandas_datareader: None

</details>



Discussion:
See the note in the `apply` [docstring ](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.apply.html)- though applymap should also apparently be updated to include this note.


> Notes
> 
> In the current implementation apply calls func twice on the first column/row to decide whether it can ?> take a fast or slow code path. This can lead to unexpected behavior if func has side-effects, as they will > take effect twice for the first column/row.


Thanks, I did not think to look at the apply help, guess this answers that.

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -6501,6 +6501,14 @@ def applymap(self, func):
         --------
         DataFrame.apply : Apply a function along input axis of DataFrame.
 
+        Notes
+        -----
+        In the current implementation applymap calls `func` twice on the
+        first column/row to decide whether it can take a fast or slow
+        code path. This can lead to unexpected behavior if `func` has
+        side-effects, as they will take effect twice for the first
+        column/row.
+
         Examples
         --------
         >>> df = pd.DataFrame([[1, 2.12], [3.356, 4.567]])


Test patch:
Repository: pandas-dev/pandas

Issue:
Question regarding Series.argsort documentation
`Series.argsort` has three arguments, `axis`, `kind` and `order`, but only `kind` is actually used.  Should the other two be removed from the documentation? (I'm guessing they were once used and are left in the code for compatibility reasons?)

https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.argsort.html


Discussion:
I believe axis and order are needed to match the signature of `np.argsort`,
which dispatches to objects with a `.argsort` method,

On Thu, Jan 17, 2019 at 7:19 AM Daniel Saxton <notifications@github.com>
wrote:

> Series.argsort has three arguments, axis, kind and order, but only kind
> is actually used. Should the other two be removed from the documentation?
> (I'm guessing they were once used and are left in the code for
> compatibility reasons?)
>
>
> https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.argsort.html
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/issues/24816>, or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIsD8fJTSlj6d2y9CPEQ9qgJ50heAks5vEHh_gaJpZM4aFLm5>
> .
>

> I believe axis and order are needed to match the signature of `np.argsort`

Could probably update the documentation similar to [`Series.repeat`](https://pandas-docs.github.io/pandas-docs-travis/api/generated/pandas.Series.repeat.html) in that case to make it more explicit that it has no effect but is accepted for compatibility with numpy.
Happy to look at updating the documentation to reflect his.

Resolving patch:
diff --git a/pandas/core/series.py b/pandas/core/series.py
--- a/pandas/core/series.py
+++ b/pandas/core/series.py
@@ -2992,11 +2992,13 @@ def argsort(self, axis=0, kind='quicksort', order=None):
 
         Parameters
         ----------
-        axis : int (can only be zero)
+        axis : int
+            Has no effect but is accepted for compatibility with numpy.
         kind : {'mergesort', 'quicksort', 'heapsort'}, default 'quicksort'
             Choice of sorting algorithm. See np.sort for more
             information. 'mergesort' is the only stable algorithm
-        order : ignored
+        order : None
+            Has no effect but is accepted for compatibility with numpy.
 
         Returns
         -------


Test patch: