various changes

Signed-off-by: Tom Everett <tome@criticalmass.com>
single threaded crawl works again

Signed-off-by: Tom Everett <tome@criticalmass.com>
oops

Signed-off-by: Tom Everett <tome@criticalmass.com>