Updates to sphinx extensions

I split this off of #1355 to make it smaller. See that PR for full list of changes. Unfortunately, it's a bit difficult to split this one up any more, since they're a bit interdependent.
These two lines could be combined.