CI: Migrate to circleci 2.0
State: closed

closes https://github.com/pandas-dev/pandas/issues/19929


Labels:
name: CI
description: Continuous Integration

Comments:
this is more readable! hahah
Slightly! I haven't tried de-duplicating things between the jobs yet, but will once things are passing (hopefully this commit).
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21814?src=pr&el=h1) Report
> Merging [#21814](https://codecov.io/gh/pandas-dev/pandas/pull/21814?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/2a2d1cfb00d69e92192eeca8e11488f782f25337?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/21814/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&height=150&width=650)](https://codecov.io/gh/pandas-dev/pandas/pull/21814?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #21814   +/-   ##
=======================================
  Coverage   92.05%   92.05%           
=======================================
  Files         169      169           
  Lines       50709    50709           
=======================================
  Hits        46679    46679           
  Misses       4030     4030
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.46% <ø> (ø)` | :arrow_up: |
| #single | `42.25% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/21814?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/21814/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.24% <0%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21814?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21814?src=pr&el=footer). Last update [2a2d1cf...03e732f](https://codecov.io/gh/pandas-dev/pandas/pull/21814?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Still need to figure out the database tests.
@TomAugspurger 
Locale stuff will hopefully be working now.

FYI, the slow cases where flipped in https://github.com/pandas-dev/pandas/blob/a43947283dd6dc9f9dc2742700c7fa4677d7e27a/circle.yml#L37. Job 1 (36_LOCALE) was running slow only and Job 2 (36_LOCALE_SLOW) was skipping the slow ones. I've corrected that.

Working on the database now.
@TomAugspurger what is the status here?
Just down to locale failures, which https://github.com/pandas-dev/pandas/pull/22213 hopefully fixed.

Will merge master and try again.
Actually, looking more closely I don't think those locale failures will be resolved by #22213...

```pytb
value = '三, 08 8月 2018 17:42:53 GMT'

    def parse_timestamp(value):
        """Parse a timestamp into a datetime object.
    
        Supported formats:
    
            * iso8601
            * rfc822
            * epoch (value is an integer)
    
        This will return a ``datetime.datetime`` object.
    
        """
        if isinstance(value, (int, float)):
            # Possibly an epoch time.
            return datetime.datetime.fromtimestamp(value, tzlocal())
        else:
            try:
                return datetime.datetime.fromtimestamp(float(value), tzlocal())
            except (TypeError, ValueError):
                pass
        try:
            # In certain cases, a timestamp marked with GMT can be parsed into a
            # different time zone, so here we provide a context which will
            # enforce that GMT == UTC.
            return dateutil.parser.parse(value, tzinfos={'GMT': tzutc()})
        except (TypeError, ValueError) as e:
>           raise ValueError('Invalid timestamp "%s": %s' % (value, e))
E           ValueError: Invalid timestamp "三, 08 8月 2018 17:42:53 GMT": ('Unknown string format:', '三, 08 8月 2018 17:42:53 GMT')
```

I'll look more closely today.
I can reproduce some of the locale failures locally with

```diff
diff --git a/pandas/tests/io/parser/test_network.py b/pandas/tests/io/parser/test_network.py
index a7cc3ad98..2b728c03d 100644
--- a/pandas/tests/io/parser/test_network.py
+++ b/pandas/tests/io/parser/test_network.py
@@ -59,6 +59,9 @@ class TestS3(object):
 
     def test_parse_public_s3_bucket(self, tips_df):
         pytest.importorskip('s3fs')
+        import locale
+
+        locale.setlocale(locale.LC_ALL, "zh_CN.UTF-8")
         # more of an integration test due to the not-public contents portion
         # can probably mock this though.
         for ext, comp in [('', None), ('.gz', 'gzip'), ('.bz2', 'bz2')]
```

This is in `S3File.info`, but I'm not sure whether s3fs is to blame, or boto, or moto, or someone else.

Going to skip these if the `locale.LC_TIME` isn't `en_US.UTF-8`. Will open followup issues.
In theory, just `TestDatetime64.test_datetime_name_accessors[crh_UA.UTF-8]` should fail now. Looking into it now.
This is fun

```python
(Pdb) x
'İyun'
(Pdb) y
'İyun'
(Pdb) [(ord(a), ord(b)) for a, b in zip(x, y)]
[(73, 304), (775, 121), (121, 117), (117, 110)]
```

`x` came from us, `y` came from `calendar.month_name`. It seems like `calendar.month_name` normalizes the strings

```python
(Pdb) unicodedata.normalize("NFC", x) == x
False
(Pdb) unicodedata.normalize("NFC", y) == y
True
```

Not sure why this wasn't failing earlier though.
And it passes for me locally as well, even when I override LANG and LC_ALL. I'm only able to reproduce in the docker image from circle.
All green (finally).


lgtm! thanks @TomAugspurger herculean effort!
Opened https://github.com/pandas-dev/pandas/issues/22396 with a couple followup items. Will do that in a couple days in case we notice anything else.
Thanks a lot Tom!