Cache external intersphinx inventories in the source directory so they don't have to be loaded for each module during the build process.