iParadigms/walker — pull request #79 by pchristopher1275 (2014-11-13 18:03:58+00:00)
Title: Dan req1

Dan noticed some odd behavior with walker. See his note below. I was able to mitigate both bugs (or it appears so now) with this patch.

=====
Hey Pete, could you look at two more things real quick for me, things I found prepping for this walker presentation tomorrow.

1) Starting with an empty database, I did `walker seed -u http://turnitin.com` and then `walker dispatch`, and as expected it dispatched one link, which I see in the database. But, the console shows 0 links dispatched. I think it's because the dispatched link has no previous crawl time and is being counted in model.go as uncrawled, and we're leaving that number out.

2) Then I ran `walker fetch` to do a single fetch and got: "Failed storing fetch results: marshal bigint: value 14947298106726739655 out of range" You could probably reproduce it yourself; I'm not sure why it's an error because that number does seem to be within the range of a 64 bit integer.