Backport PR #5495: Fix various broken things in docs
Backport PR #5496: Exclude whatsnew/pr directory from docs builds

Silences a couple of annoying Sphinx warnings.