crawler fixes, root Makefile dir prune added.