We were downloading the entire contents of PDFs when we were trying to get their titles (which would always fail since we only look for titles as they'd appear in an html or xml doc). now, we get the first MB and if it's more than that, we just use the url as the title of the archive (and in the case of a PDF, pass off the PDF to the PDF downloading task).

This fixes #353 since that pdf was large and we were super slow when trying to get the title (I'm guessing the user game up and throught we weren't indexing it.

we need to revisit our entire flow for finding mime type/determinging file type of url. it's a sloppy now.