Rethrow exception if delete file fails for a different reason than 'file not found', add caching to dir creation