Fix possibility of non-removed symlinked directory.

Fixes #4080
no but on a symlink is_file will be false as far as I see from the docs.

>  Returns TRUE if the filename exists and is a regular file, FALSE otherwise. 

So removing symlinks that point to files will not work and recursion will be called.