Respider stability fix, attempt 1.

Recover from a panic during spidering, in the infinite loop that handles
respidering, so that if the spidering throwing a panic is the reason
that respidering stops, we'll now get a log entry with the reason, and
respidering should continue.

When I see such a log entry, I can then fix the underlying bug.

Also, record the timestamp at which a data set is persisted.  Er, this
lies to the user at present: it shows the timestamp at which the data
was set as the current persisted data structure, which might be "at
startup, from older data", while claiming that the time is the time of
scan.  Will think on a better phrasing for this (but need to get back to
work, so am committing as-is).