sbenthall/bigbang — pull request #54 by npdoty (2014-09-07 03:11:26+00:00)
Title: initial infrastructure for crawling W3C web archives of mailing lists

This adds a w3crawl.py file which handles the very different collection for W3C mailing lists and modifies mailman.py to use that function when it looks like a w3c mailing list archive. That code may not be complete, but this is a starting point, related to issue #53. For now, it parses messages and stores them as .mbox archives in the appropriate archives directory.

A couple strange notes about the pull request, owing to my still amateur understanding of git histories:
* The `public-privacy.html` file has actually been removed (git rm) but is still included in the diff here I guess because it was added and committed on this branch before it was removed and I'm not sure how to rebase to get rid of it without losing other history which might be useful.
* Also, I don't think I actually changed `bigbang/process.py`, it's possible that I didn't correctly resolve the merge conflicts, but that's not code that I wrote or rely on for this pull request.