Add data configurations

I have added the data configurations for several well-known sites.  These will be the configuration for the final automated system.
Added the beginning functionality of web site crawling

I have added the configuration and general basics for putting together a complete crawl of a website.  I have not put in the actual lists for processed and processing and the crawl, but I do have the looping ready and the process started.  Tomorrow I will add those peices to make it completely functional.
Finished the initial site crawling implementation

I have added the initial version of the website crawling functionality.  This does however cause vary long process strings.  I am going to have to add the ablility to stop after a certain amount of pages and also allow for pauses between downloads to keep from being blacklisted.
Added a max page crawl


I have added the max page crawl to the website source.  This will allow you to make sure that you hold to a managable size for travering the website.  I have also added "####" between the html for each page in the webData variable.  Thus, we can not determine the begining and end of each page stored in the variable.  Now I will have to add a backoff time and modify the config file reader to parse the new website variables
Added config reader support for the new options

I have added config reader support for the crawl, domain base and max page count options.  The only thing necessary for being a good citizen of the internet is the addition of a backoff algorithm, which will wait for the next code update.
Added message for printing out configs

We now list out the which config we are currently on.  This gives
us an indication of where errors are coming from.
Docstrings added

I have added the correct docstrings that can be used by epydoc to
produce HTML documenatation of the package.
Change to optparse

I have changed over to optparse which is a leaner way of handling option parsing
and error conditions.
Merge pull request #1 from janies/feature/fixop

convert to a more complete optionlist
Merge branch 'janies-docstring' into develop

Conflicts:
	.gitignore
	src/dataleach/leach/__init__.py
Adjust testing

Minor adjustment to the configreader testing.
Update auto-generate output

This creates a directory if the directory in a config does not already exist.
Merge pull request #3 from janies/feature/create-dir

Configuration does not auto create a directory
Better handling of RSS link extraction

I have slightly changed how I am handling Link extraction from
RSS feeds.
Change from URLLib2 to Requests

Changed to the better structured and more usable Requests from
urllib2.
Merge pull request #5 from janies/feature/rss_feed_fix

RSS feed parser returning empty url strings for processing
Remove Random Print

This is a one line change that removes a print statement.
Modify for tests

I made some minor test modifications.
Add unicode to the list KeeperReg inputs

I have aded unicode to the list of valid input types for KeeperReg.
Merge pull request #4 from janies/develop

Add Json processing to website