aitjcize/creepy — pull request #3 by AdotDdot (2014-06-01 15:39:42+00:00)
Title: added feature for max url dept

Using set_max_depth the user can choose the max depth of the url which will be used as targets by the crawler. Also added a comment under process_document in case the user wants to do something with depth. Hope it's useful.