Remove directories with binaries