Also, what happens if `path` exists but is not a directory? Should we check for that and throw a clear exception?