Apply ignore list to remote dirs

Make sure that we don't warn the user about remote_dirs in directories
which we are ignoring anyway.