- Figured out problem with database, now correctly opens and closes.
- Added check for crawl delay, since btn.com only has a crawler delay
against a user agent, now checks to see if the user agent and url entry
exists to do the update, if not it uses the insert command. Really
should so that for visit time and request rate.