this working with pickle on the crawler!
working but uses a lot of memory to popluate database
site updating quite well