wummel/linkchecker — pull request #420 by chuckbjones (2013-08-06 19:13:29+00:00)
Title: Add maxfilesize config option

Replaces the MaxFilesizeBytes constant with a configurable setting, defaulting to the old value of 5MB.

I've had a site serving up a 4MB pdf file as text/html, causing linkchecker to spin for hours trying to parse it. I wish I knew how to detect that specific issue, but at least ignoring very large files allows linkchecker to complete in a reasonable amount of time.