Fix race condition

It is possible that some other request created the cache-dir between the
is_dir() check and the mkdir() call. So only if the dir doesn't exist
after the mkdir() call we should throw an error