POC: make core.config self-contained
State: closed

As discussed in #25162.  The idea from that issue would then be to move core.config to a location explicitly "upstream" from everything else in pandas.

Labels:
name: Refactor
description: Internal refactoring of code

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=h1) Report
> Merging [#25176](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/e3b095074c5c3b29ec25a994cd204349b872d961?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `88.6%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25176/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25176      +/-   ##
==========================================
- Coverage   92.37%   92.37%   -0.01%     
==========================================
  Files         166      166              
  Lines       52408    52416       +8     
==========================================
+ Hits        48412    48419       +7     
- Misses       3996     3997       +1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.79% <88.6%> (-0.01%)` | :arrow_down: |
| #single | `42.86% <59.49%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/printing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25176/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvcHJpbnRpbmcucHk=) | `85.07% <100%> (-0.35%)` | :arrow_down: |
| [pandas/core/config.py](https://codecov.io/gh/pandas-dev/pandas/pull/25176/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29uZmlnLnB5) | `86.92% <88.46%> (-0.13%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=footer). Last update [e3b0950...138c6c2](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=h1) Report
> Merging [#25176](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/cc4a7e510f4bbd5aeb8138de16b888f4ce1551da?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25176/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25176      +/-   ##
==========================================
- Coverage   91.26%   91.26%   -0.01%     
==========================================
  Files         173      173              
  Lines       52968    52970       +2     
==========================================
+ Hits        48340    48341       +1     
- Misses       4628     4629       +1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.83% <100%> (ø)` | :arrow_up: |
| #single | `41.71% <57.89%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/config\_init.py](https://codecov.io/gh/pandas-dev/pandas/pull/25176/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29uZmlnX2luaXQucHk=) | `99.21% <ø> (-0.03%)` | :arrow_down: |
| [pandas/core/config.py](https://codecov.io/gh/pandas-dev/pandas/pull/25176/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29uZmlnLnB5) | `87.29% <100%> (+0.25%)` | :arrow_up: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25176/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `87.57% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=footer). Last update [cc4a7e5...18d471f](https://codecov.io/gh/pandas-dev/pandas/pull/25176?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> all for the intent. but this is moving printing things into a very weird place.

Yah it does seem like a weird place for `pprint_thing`.  A few thoughts come to mind:

- it may be that `core.config` uses `pprint_thing` but could get away with something lower-tech.  I'll take a look at this.
- Part of the idea is that `core.config` would be moved to somewhere outside of core to make it explicitly "upstream".  That might make the scope feel less akward.
- pd.io's dependency graph is a mess.  In particular, DataFrame, Series, and many of the Index subclasses rely on io.formats.format via circular runtime imports for their `__repr__` methods.  This isn't a problem, but definitely a bit smelly.  This could be seen as a step towards clearing up the pd.io dependency graph.
FYI, I extracted pd.core.config from pandas a few months ago and made it into self-contained library. It at https://github.com/topper-123/optioneer. You`d be welcome to take a look if it can be used for this.

@topper-123 thanks, thats pretty similar to what I have in mind.  Did you find that pprint_thing was unnecessary?
@jbrockmendel, yes, basically. I wanted to avoid the dependency on pandas or other external dependencies, so I replaced pprint_thing with some custom formatting.
wouldn’t be averse to ripping out code and using this
> wouldn’t be averse to ripping out code and using this

I haven't looked at it closely enough to have an opinion, but assume that @topper-123 is on the ball.  I like the external-library idea in part because it would make it easier to share a config library with the rest of the ecosystem (mentioned in #25162).

Whether internal or external, I think parts of `config_init` need to go up into it, the display namespace.
Just using `str(...)` instead of `pprint_thing(...)` seems to work fine.
Updated to move the `date_dayfirst` and `date_yearfirst` options from `config_init` to `config`.  As a result, `config` could now (if moved outside of `core`) be imported at the module-level in `tslibs.parsing` instead of as a runtime import.

w/r/t the PY2 compat, the only compat code this module uses is `unicode` (or `text_type` in master), and that is only used for the `is_unicode` factory, which is never used.  We could just remove that object and with it the remaining compat code in this file.

Locally I've been toying with moving `config` to a `pandas._dag` directory with `__init__`-docstring
```
"""
_dag is an experimental directory for pandas code with:
    - no pandas dependencies
    - self-contained tests

A module or sub-package might be a good fit for _dag if it:
    - is a vendored version of outside code (e.g. tzlocal, Pyperclip)
    - it is needed by _libs and has no pandas dependencies
      (e.g. config, localization)
"""
```

Thoughts?
After a couple days of trying to wrap my head around the cython codebase, I'm extra-adamant about the benefits of a DAG-as-possible dependency structure for newcomers.
Closing in favor of #25613 (and follow-ups)