Fix setting permissions of directories as well