fix session usage
create inmemory sqlite db for testing.

its not great but it works
make the crawler do the right thing based on the db