Repository: networkx/networkx
Pull request: CI: Configure circleCI to deploy docs.
Closes #4118 - configure CircleCI to deploy the development documentation. Largely drawn from the doc deployment configuration for [scipy](https://github.com/scipy/scipy/blob/master/.circleci/config.yml).

Other tasks (beyond reviewing the proposed configuration) to be done before removing draft status:
 - [x] remove doc deployment from travis CI

Problem statement:
Viewable documentation builds for PRs
Travis CI supports uploading artifacts, but

> Note that the artifacts addon is not available for pull request builds.

So I think going with Circle CI to build the docs on PRs seems like a
good solution.  Since you are familiar with how NumPy does it, it should
be straightforward.  It will also speed up our Travis CI time.

It might be a good idea to add more checks if possible.
For instance, sphinx can check that links aren't broken.
There maybe other stuff that can be checked.
If possible it would be nice to somehow automatically check that
new functions have been added to the documentation and, if not,
to post a note on the PR.
It would be nice to get this in sooner, rather than later.
So you may want to do the easiest things in one PR and create
a separate PR for additional checks.

You should double check that I haven't missed anything, but here are
some pointers:

In ``.travisci``, we can remove
```
    - os: linux
      python: 3.7
      env:
      - OPTIONAL_DEPS=1
      - BUILD_DOCS=1
      - DEPLOY_DOCS=1
      addons:
        apt:
          packages:
          - libgdal-dev
          - graphviz
          - texlive
          - texlive-latex-extra
          - latexmk
```
and
```
script:
  - if [[ "${BUILD_DOCS}" == 1 ]]; then
      source tools/travis/build_docs.sh;
    else
      source tools/travis/script.sh;
    fi

after_success:
  - if [[ "${REPORT_COVERAGE}" == 1 ]]; then
      codecov;
    fi
  - if [[ "${BUILD_DOCS}" == 1 && "${DEPLOY_DOCS}" == 1 ]]; then
      source tools/travis/deploy_docs.sh;
    fi
```
can be replaced with
```
script:
  - source tools/travis/script.sh

after_success:
  - if [[ "${REPORT_COVERAGE}" == 1 ]]; then
      codecov;
    fi
```

You should also remove:
- tools/travis/build_docs.sh
- tools/travis/deploy_docs.sh

The basic recipe for building docs should be something like:
```
## Depending on what OS circle CI you will need somehow install
## libgdal-dev graphviz texlive texlive-latex-extra latexmk

# Install
python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements/optional.txt
pip install -r requirements/extras.txt
pip install -r requirements/doc.txt
pip install .

# Build docs
cd doc
make html
make latexpdf LATEXMKOPTS="-silent"
cd ..

# Deploy docs on PR
# I don't know how to do this

# Deploy docs on master
# somehow set up a deploy key (see tools/travis/deploy_docs.sh)

# Push the latest docs to the networkx/documentation repo (gh-pages branch)
GH_REF=git@github.com:networkx/documentation.git
echo "-- pushing docs --"

# The next two lines should be updated ...
git config --global user.email "travis@travis-ci.com"
git config --global user.name "NetworkX Travis Bot"

cd doc
git clone --quiet --branch=gh-pages --depth=1 ${GH_REF} ghpages_build
cd ghpages_build

# Overwrite previous commit
git rm -r latest
cp -a ../build/html latest
cp -a ../build/latex/networkx_reference.pdf latest/_downloads/.
git add latest
git commit -m "Deploy GitHub Pages"

git push --force --quiet "${GH_REF}" gh-pages > /dev/null 2>&1
```

Hints from issue discussion:
You will likely have to fuss around to get gdal installed.  Every time I try to install it on a different OS, I have to follow different steps to get it to work.  Here are the most likely things you may need to do:
```
  export CPLUS_INCLUDE_PATH=/usr/include/gdal
  export C_INCLUDE_PATH=/usr/include/gdal
```
You may need to double check that it is installed in ``/usr/include/gdal``, although that is where I usually find it.  I also normally install the version of gdal with the same number as the installed version of GDAL.  I normally use ``sed`` or a related tool to change ``requirements/extra.txt`` so that
```
gdal==1.10.0
```
is set to the version I need.  You can check what versions of gdal are available here:
- https://pypi.org/project/GDAL/#history
I am pretty certain you don't need to install ``gdal`` (now that you mention it).

There are two doctests (and one is missing ``>>> ``)
```
readwrite/nx_shp.py:    nx.write_shp(digraph, '/shapefiles') # doctest +SKIP
readwrite/nx_shp.py:    >>> G = nx.read_shp('test.shp') # doctest: +SKIP
```
Both are skipped and we don't run doctests during the build currently (we used to, but stopped eeef3c8df81dc045dd808af1ef837cd00175ce9d).

We autobuild the reference via `doc/reference/readwrite/nx_shp.rst`, but that doesn't need ``gdal``.  And there is no example in the gallery.



Yeah AFAICT the presence/absence of `gdal` doesn't have an impact on the doc build.

Re: running the doctests - this is something we could consider re-enabling in CircleCI workflow if there is interest.
I mainly turned off doctests via Sphinx b/c I was trying speed up Travis (and we are running them using ``pytest --doctest-modules`` anyway).  Now that we are using CircleCI that concern is removed.  It should be redundant, but you never know.  Perhaps Sphinx and pytest run them slightly differently or the are run using slightly different environments.  So as long as Circle CI is still finishing before Travis, let's enable them here as well.  Let's enable ``make linkcheck`` first, though.  We aren't currently checking for broken links.
>  Let's enable make linkcheck first, though. We aren't currently checking for broken links.

Agreed - I'd also like to fix the remaining sphinx warnings and afterwards enable the -e flag to escalate any new warnings to errors for the CI so that the doc build will fail if any new warnings are introduced.
CircleCI now builds the PDF (see #4123), which can be found in ``build/html/_downloads/networkx_reference.pdf``.

Commit message: Apply changes from code review

 * rm ghp-import
 * add step to rm current latest before cp

Patch to review:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 487ac9b82d3..8bc77c131f0 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -47,3 +47,48 @@ jobs:
 
       - store_artifacts:
           path: doc/build/html
+
+      - persist_to_workspace:
+          root: doc/build
+          paths:
+            - html
+
+  deploy:
+    working_directory: ~/repo
+    docker:
+      - image: circleci/python:3.8.3-buster
+
+    steps:
+      - attach_workspace:
+          at: /tmp/build
+
+      - add_ssh_keys:
+          fingerprints:
+            - "e5:04:e6:c4:d6:c3:34:b9:02:e6:9f:25:1c:01:9e:e9"
+
+      - run:
+          name: upload_devdocs
+          command: |
+            set -e
+            git clone git@github.com:networkx/documentation.git
+            cd documentation
+            rm -rf latest
+            cp -R /tmp/build/html latest
+            touch latest/.nojekyll
+            git config --global user.email "networkx-circleci-bot@nomail"
+            git config --global user.name "networkx-circleci-bot"
+            git config --global push.default simple;
+            git commit -am "Docs built from $CIRCLE_SHA1"
+            git push origin gh-pages
+
+workflows:
+  version: 2
+  default:
+    jobs:
+      - build
+      - deploy:
+          requires:
+            - build
+          filters:
+            branches:
+              only: master


Code review comments:
diff_hunk: @@ -47,3 +47,48 @@ jobs:
 
       - store_artifacts:
           path: doc/build/html
+
+      - persist_to_workspace:
+          root: doc/build
+          paths:
+            - html
+
+  deploy:
+    working_directory: ~/repo
+    docker:
+      - image: circleci/python:3.8.3-buster
+
+    steps:
+      - attach_workspace:
+          at: /tmp/build
+
+      - add_ssh_keys:
+          fingerprints:
+            - "e5:04:e6:c4:d6:c3:34:b9:02:e6:9f:25:1c:01:9e:e9"
+
+      - run:
+          name: upload_devdocs
+          command: |
+            set -e
+            git clone git@github.com:networkx/documentation.git
+            cd documentation
+            rm -rf latest
+            cp -R /tmp/build/html latest
+            touch latest/.nojekyll
path: .circleci/config.yml
text: @user1:
This isn't needed.  There is already a ``.nojekyll`` in the directory containing ``latest``.