Repository: pandas-dev/pandas
Pull request: BUG: to_datetime(strs, utc=True) used previous UTC offset
- [x] closes #24992
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
to_datetime uses previous row's timezone when timezone not specified and utc=True
#### Code Sample, a copy-pastable example if possible

Without utc=True:
```python
pd.to_datetime([
    '2018-11-28T00:00:00', 
    '2018-11-28T00:00:00+12:00', 
    '2018-11-28T00:00:00', 
    '2018-11-28T00:00:00+06:00', 
    '2018-11-28T00:00:00'
])
```

Result:
```
Index([      2018-11-28 00:00:00, 2018-11-28 00:00:00+12:00,
             2018-11-28 00:00:00, 2018-11-28 00:00:00+06:00,
             2018-11-28 00:00:00],
      dtype='object')
```

With utc=True:
```python
pd.to_datetime([
    '2018-11-28T00:00:00', 
    '2018-11-28T00:00:00+12:00', 
    '2018-11-28T00:00:00', 
    '2018-11-28T00:00:00+06:00', 
    '2018-11-28T00:00:00'
], utc=True)
```

Result:
```
DatetimeIndex(['2018-11-28 00:00:00+00:00', '2018-11-27 12:00:00+00:00',
               '2018-11-27 12:00:00+00:00', '2018-11-27 18:00:00+00:00',
               '2018-11-27 18:00:00+00:00'],
              dtype='datetime64[ns, UTC]', freq=None)
```

#### Problem description

The behavior leads to unexpected and inconsistent treatment of datetimes where the timezone is not explicitly specified. Simply using the last timezone it saw from a datetime where the timezone was specified is pretty confusing.

#### Expected Output
I would be fine with it just assuming that any datetimes where the timezone is NOT specified are already in UTC. Another valid way of handling it would be to use os.environ['TZ']. Another valid way would be to add another input argument to to_datetime that allowed specifying which timezone to use when there is no timezone specified. Any of these would lead to conclusive, explainable results. This is what I would like to see:

```
DatetimeIndex(['2018-11-28 00:00:00+00:00', '2018-11-27 12:00:00+00:00',
               '2018-11-28 00:00:00+00:00', '2018-11-27 18:00:00+00:00',
               '2018-11-28 00:00:00+00:00'],
              dtype='datetime64[ns, UTC]', freq=None)
```

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.7.2.final.0
python-bits: 64
OS: Darwin
OS-release: 18.2.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.24.0
pytest: None
pip: 19.0.1
setuptools: 40.7.0
Cython: None
numpy: 1.16.0
scipy: 1.2.0
pyarrow: None
xarray: None
IPython: 7.2.0
sphinx: None
patsy: None
dateutil: 2.7.5
pytz: 2018.9
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: 4.3.0
bs4: None
html5lib: None
sqlalchemy: 1.2.17
pymysql: None
psycopg2: 2.7.7 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

</details>

Hints from issue discussion:
cc @jbrockmendel @mroeschke if you have thoughts on the expected. This should probably be all True (order shouldn't matter).

```python
In [21]: items = ['2018-11-28T00:00:00+12:00', '2018-11-28T00:00:00']

In [22]: a = pd.to_datetime(items, utc=True)

In [23]: b = pd.to_datetime(list(reversed(items)), utc=True)[::-1]

In [24]: a == b
Out[24]: array([ True, False])
```

The 0.24.0 behavior is the same as 0.23.4.
Agreed on "order shouldn't matter".  Something similar came up in #24006 with how we treat "now" and "today".

Commit message: BUG: to_datetime(..., utc=True) used previous UTC offset

Patch to review:
diff --git a/doc/source/whatsnew/v0.25.0.rst b/doc/source/whatsnew/v0.25.0.rst
index d0ddb6e09d555..1b46994334abf 100644
--- a/doc/source/whatsnew/v0.25.0.rst
+++ b/doc/source/whatsnew/v0.25.0.rst
@@ -96,7 +96,7 @@ Timedelta
 Timezones
 ^^^^^^^^^
 
--
+- Bug in :func:`to_datetime` with ``utc=True` and datetime strings that would apply previously parsed UTC offsets to subsequent arguments (:issue:`24992`)
 -
 -
 
diff --git a/pandas/_libs/tslib.pyx b/pandas/_libs/tslib.pyx
index 798e338d5581b..f932e236b5218 100644
--- a/pandas/_libs/tslib.pyx
+++ b/pandas/_libs/tslib.pyx
@@ -645,6 +645,8 @@ cpdef array_to_datetime(ndarray[object] values, str errors='raise',
                             out_tzoffset_vals.add(out_tzoffset * 60.)
                             tz = pytz.FixedOffset(out_tzoffset)
                             value = tz_convert_single(value, tz, UTC)
+                            out_local = 0
+                            out_tzoffset = 0
                         else:
                             # Add a marker for naive string, to track if we are
                             # parsing mixed naive and aware strings
diff --git a/pandas/tests/indexes/datetimes/test_tools.py b/pandas/tests/indexes/datetimes/test_tools.py
index 38f5eab15041f..b94935d2521eb 100644
--- a/pandas/tests/indexes/datetimes/test_tools.py
+++ b/pandas/tests/indexes/datetimes/test_tools.py
@@ -714,6 +714,29 @@ def test_iso_8601_strings_with_different_offsets(self):
                                   NaT], tz='UTC')
         tm.assert_index_equal(result, expected)
 
+    def test_iss8601_strings_mixed_offsets_with_naive(self):
+        # GH 24992
+        result = pd.to_datetime([
+            '2018-11-28T00:00:00',
+            '2018-11-28T00:00:00+12:00',
+            '2018-11-28T00:00:00',
+            '2018-11-28T00:00:00+06:00',
+            '2018-11-28T00:00:00'
+        ], utc=True)
+        expected = pd.to_datetime([
+            '2018-11-28T00:00:00',
+            '2018-11-27T12:00:00',
+            '2018-11-28T00:00:00',
+            '2018-11-27T18:00:00',
+            '2018-11-28T00:00:00'
+        ], utc=True)
+        tm.assert_index_equal(result, expected)
+
+        items = ['2018-11-28T00:00:00+12:00', '2018-11-28T00:00:00']
+        result = pd.to_datetime(items, utc=True)
+        expected = pd.to_datetime(list(reversed(items)), utc=True)[::-1]
+        tm.assert_index_equal(result, expected)
+
     def test_non_iso_strings_with_tz_offset(self):
         result = to_datetime(['March 1, 2018 12:00:00+0400'] * 2)
         expected = DatetimeIndex([datetime(2018, 3, 1, 12,


Code review comments:
diff_hunk: @@ -96,7 +96,7 @@ Timedelta
 Timezones
 ^^^^^^^^^
 
--
+- Bug in :func:`to_datetime` with ``utc=True` and datetime strings that would apply previously parsed UTC offsets to subsequent arguments (:issue:`24992`)
path: doc/source/whatsnew/v0.25.0.rst
text: @user1:
small nitpick: missing the second backtick at the end of <code>``utc=True`</code>

```suggestion
- Bug in :func:`to_datetime` with ``utc=True`` and datetime strings that would apply previously parsed UTC offsets to subsequent arguments (:issue:`24992`)
```