Rename the [crawl] section to [crawler] for consistency
Apply the same defaults in crawler-add as in crawld
Add a 'state' column to the queue database (NEW, FAILED, REJECTED, ACCEPTED, COMPLETE) for external tracking.
Be more efficient about adding items to the queue.