Add missing check.
Simplify checking of already visited URLs.
Remove redundant test
Rename parameter.
Update docstring.
Rename function, add docstring, add comments.