Fix case where path exists but is not a dir