Updates to sphinx extensions

I split this off of #1355 to make it smaller. See that PR for full list of changes. Unfortunately, it's a bit difficult to split this one up any more, since they're a bit interdependent.
I take it that this file is also not/no longer being used..?