This re-organizes a fair amount of the base PicoSpider code.
See squashed commits for details.

Squashed commit of the following:

commit 89c989592a966a2a5d48146f41e9c5334c2e513b
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 17:46:30 2014 -0500

    Minor fixes for new flow

commit ff0bb67f9f9d8f996e06de9cae790fc66134c05a
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 15:26:56 2014 -0500

    Breaking apart the functions further.

commit 8560f07df642cb86105945cab050853199c2666d
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 13:55:38 2014 -0500

    simple variable change

commit f7467dfb3c528b89ed34164bfc298a7e9fc5fb84
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 13:54:07 2014 -0500

    Generator version looks to work the same.

commit 706fe2b3840d852df68c11c41ba8c816c6a5cdde
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 13:34:34 2014 -0500

    This version uses map and generators.

commit 11498b871dae276ed9d55b2fe6012fbc002f2e2e
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 13:02:21 2014 -0500

    maybe yielding will do cool things ?

commit 105d8509b2e6ff67449e84b03227829c7520a9c4
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 06:17:26 2014 -0500

    Move toward configurability

commit d1f4a4b7e80f15aeb7f35da6b8db189756bdebf9
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 06:11:22 2014 -0500

    Renaming variables

commit 3464c78b5b167d7a97eb592c474a64740867b5e3
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 05:01:34 2014 -0500

    Give PicoSpider some default rules.

commit ed646047e5a54f2a2a743abce98f373411359037
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 04:29:38 2014 -0500

    Comment out logging to test performance of logging

commit a7e6103789c49812e055a8f86aae89ef8ea51ec1
Author: Stella Tigre <stella.cannefax@gmail.com>
Date:   Mon Mar 17 04:29:13 2014 -0500

    Adjust formatting and options for Rules / link extractors
Merge branch 'master' of github.com:StellaCannefax/pico-nova