ArchiveTeam/twitpic-grab — pull request #4 by yipdw (2014-09-06 06:21:20+00:00)
Title: Track downloaded URLs in the Lua script

Wget seems to track downloaded URLs on a per-source URL basis, which means that we end up downloading e.g. the same stylesheets, Javascripts, etc. for each twitpic URL. We only need to download those once per
WARC, so we implement our own URL tracking machinery.