changed to download the web pages before doing any processing.
download to file system first. and then process each file. at least, that's the plan.
experiment to separate downloading and processing of stuff