Move file away if it blocks a directory creation.
Add option -i to allow more fine grained control which urls are crawled.
Special treatment for urls ending with  slash dot.