DOC: Final reorganization of documentation pages
State: closed

- [X] xref #24499

You can see the changes here: https://datapythonista.github.io/pandas-doc-preview/

Pages moved:
- comparison_with_whatever -> getting_started/comparison/comparison_with_whatever
- ecosystem -> getting_started/ecosystem
- cookbook -> user_guide/cookbook
- rpy2 has been removed

Changes to the documentation home page:
- Moved the long description to the Package overview page, and left a small paragraph, and a link to it
- Made the toctree show just 2 levels, so it's useful and it can be seen what it's there

May be useful to see how the master docs look like, to compare: https://pandas-docs.github.io/pandas-docs-travis/

I will follow up with:
- A change of the style (the one we saw in the Austin sprint)
- An integration with the pandas website (pydata.pandas.org). I'll create a issue for this before, as there is some discussion needed.

@TomAugspurger @jorisvandenbossche @jreback 




Labels:
name: Docs

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=h1) Report
> Merging [#24890](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/2fa0835737a0e3e111e893d30ed2f25b7249fd4b?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24890/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #24890      +/-   ##
==========================================
+ Coverage   92.38%   92.39%   +<.01%     
==========================================
  Files         166      166              
  Lines       52413    52413              
==========================================
+ Hits        48424    48425       +1     
+ Misses       3989     3988       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.81% <ø> (ø)` | :arrow_up: |
| #single | `42.89% <ø> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/24890/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `88.13% <0%> (+0.09%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=footer). Last update [2fa0835...5126e80](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=h1) Report
> Merging [#24890](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/2fa0835737a0e3e111e893d30ed2f25b7249fd4b?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24890/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #24890      +/-   ##
==========================================
- Coverage   92.38%   92.38%   -0.01%     
==========================================
  Files         166      166              
  Lines       52413    52389      -24     
==========================================
- Hits        48424    48398      -26     
- Misses       3989     3991       +2
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.8% <ø> (-0.01%)` | :arrow_down: |
| #single | `42.89% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/nanops.py](https://codecov.io/gh/pandas-dev/pandas/pull/24890/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvbmFub3BzLnB5) | `93.84% <0%> (-0.53%)` | :arrow_down: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/24890/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `83.58% <0%> (-0.18%)` | :arrow_down: |
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/24890/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `94.4% <0%> (-0.07%)` | :arrow_down: |
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/24890/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `97.98% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/arrays/integer.py](https://codecov.io/gh/pandas-dev/pandas/pull/24890/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2ludGVnZXIucHk=) | `96.32% <0%> (ø)` | :arrow_up: |
| [pandas/core/arrays/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/24890/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL3RpbWVkZWx0YXMucHk=) | `88.3% <0%> (ø)` | :arrow_up: |
| [pandas/core/arrays/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/24890/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2RhdGV0aW1lcy5weQ==) | `97.79% <0%> (ø)` | :arrow_up: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/24890/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `88.13% <0%> (+0.09%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=footer). Last update [2fa0835...045e155](https://codecov.io/gh/pandas-dev/pandas/pull/24890?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> rpy2 has been removed

Thoughts on making that a subsection of the "Comparison with R" page? It's not really a comparison, but I imagine that people coming from pandas to R would be interested in it.
I am not fully sure I find the ecosystem page fitting for the getting started section (it is also relevant for people familiar with pandas that would not look anymore at a getting started section), but +1 on the other changes!
I didn't feel like the cookbook and the ecosystem belong 100% to any of the sections, but I don't think they are important enough to be in the top level.

I'm also happy moving the ecosystem to the user guide, but won't be able to take care myself of it until late this evening (UK time), I need to go now.
> I didn't feel like the cookbook and the ecosystem belong 100% to any of the sections, but I don't think they are important enough to be in the top level.

Yep, agreed. I would maybe rather put it in the user guide then, but no strong feeling.
Actually, I think that a "more curated" ecosystem page would be valuable enough for top-level (with content like: want to scale your pandas -> see those packages, extend pandas data types -> look for those, ... and then only random list of packages). But that's for another issue :-)
See https://gitter.im/pydata/pandas?at=5c4afb5dba355012a496295e

@jorisvandenbossche and I both like the idea of moving ecosystem to the top level and then putting some care into it to make it actually nice. We'll followup with the improvements, but move it to the top level for now to avoid the multiple redirects.
The doc and lint jobs finished. Merging.
Cool!
Having the ecosystem in the top level, and improving it sounds good to me too.

If there is time, in couple of hours I could try to make the home page links a bit more compact:
```
PDF Version

Zipped HTML

Date: Jan 25, 2019 Version: 0.24.0rc1+73.gbb86a9d3

Binary Installers: https://pypi.org/project/pandas

Source Repository: https://github.com/pandas-dev/pandas

Issues & Ideas: https://github.com/pandas-dev/pandas/issues

Q&A Support: https://stackoverflow.com/questions/tagged/pandas

Developer Mailing List: https://groups.google.com/forum/#!forum/pydata
```
I think something like the next would be not only more compact, but also more clear:
```
Date: Jan 25, 2019 Version: 0.24.0rc1+73.gbb86a9d3

Download documentation: PDF Version | Zipped HTML

Useful links: Binary Installers | Source Repository | Issues & Ideas | Q&A Support | Mailing List
```
 What do you think?
> If there is time, in couple of hours I could try to make the home page links a bit more compact:

I'm probably tagging the release before then. But I'm happy to rebuild and re-upload the docs with any improvements.

---

A table-like structure for those links makes a lot of sense.

will see if it's fast enough so I can have it now then
I opened #24928
@datapythonista I just saw this and the change in the docs at https://pandas.pydata.org/pandas-docs/stable/reference/index.html.  There is one issue for me is that it is now MUCH harder to find the API specifics of an individual method.  In the previous docs with 0.23.4, I could go to that page and use the browser search to find a specific method (since I knew all the methods were listed) and then just click on the docs for that method to see the details of the signature.  

So having an index of ALL of the functions is really needed.  Or maybe it is there and I can't find it.

If this should be a new issue, let me know.
Joris raised this point earlier I think. You might try using the sphinx
search bar.

On Mon, Jan 28, 2019 at 8:36 AM Irv Lustig <notifications@github.com> wrote:

> @datapythonista <https://github.com/datapythonista> I just saw this and
> the change in the docs at
> https://pandas.pydata.org/pandas-docs/stable/reference/index.html. There
> is one issue for me is that it is now MUCH harder to find the API specifics
> of an individual method. In the previous docs with 0.23.4, I could go to
> that page and use the browser search to find a specific method (since I
> knew all the methods were listed) and then just click on the docs for that
> method to see the details of the signature.
>
> So having an index of ALL of the functions is really needed. Or maybe it
> is there and I can't find it.
>
> If this should be a new issue, let me know.
>
> —
> You are receiving this because you modified the open/close state.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/24890#issuecomment-458155683>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIlWHLQG8T1NEHSCyyVPoLh0RKPaMks5vHwr7gaJpZM4aP5Xs>
> .
>

@TomAugspurger Fair enough.  I still think having that one list was useful.  As I'm working on projects, I will see how it affects my workflow.

Maybe it would be possible to put that search on the top of the page, rather than at the bottom of the left-hand-side?  For many web sites, it is natural to look for a search at the top right.
The sphinx search option returned as the first option the correct method for any search like "Series.fillna", "Index.asof", "read_csv or pandas.read_csv"...

If you prefer searching in a huge document, you can also use the pdf, that would be somehow similar to what we had until now. (but I'm sure using the search option will be more convenient).
@datapythonista The other thing I've done is when I'm offline (e.g., on a plane), I can pull up the docs on my local machine, and search them using browser search, and I don't have the sphinx search there.
Sphinx search is client-side javascript only, if I'm not wrong. I think it should work locally.
@datapythonista You're right. Thanks for the pointer.  Still would like to see the search box at the top right of the page  :-)