crypt3lx2k/Imageboard-Image-Scraper — pull request #2 by DanielOaks (2014-06-23 02:50:59+00:00)
Title: Downloading thread .html, lots of new stuff

This builds off the other Pull Request #1. Sorry about this monolithic request, but there was a whole lot I wanted to do with this.

1: Does the https stuff in PR #1

2: It changes the argument parsing to use [docopt](https://github.com/docopt/docopt) instead of argparse. Docopt's a whole bunch easier, in my opinion, a newer and nicer way to do it all. Also threw docopt in `Lib/ext/docopt.py` so people don't need to have it already installed.

3: Added a nicer number of bytes display, upon completion. Repurposed an old metric function to do bytes conversion, so it ends up like this:
```
Downloaded 30 file(s) (6.2MiB) in 3.343909 second(s) (1.9MiB per second).
```
(uses MiB, GiB, KiB where appropriate) instead of just spitting out the raw number of bytes.

4: Updated the `keep-names` regex, as well as the `keep-names` logic. They changed how this worked, so I had to update the regex, and I had to also account for when people had images with exactly the same names, or spoilered images which had 'Spoiler Image' as their name (so, every image had the same name). If two images have the same name with `keep-names` enabled, it just makes the name `35235235256.jpg` or whatever it is.

5: Lets you download the page `.html`. Throws all the static files (css, js, favicon) into a folder called `static/` in the thread dir, while using `Lib/StaticFiles.py` to control the caching and downloading of those files (caches them for three hours in `~/.config/imageboard-image-scraper/static`, after that it'll redownload them).

Doesn't bother downloading the thumbnails, instead just changes the thumbnail URLs to the URLs of the images we've already downloaded (so we don't need to download thumbnails). This approach might be super duper lazy, but we're just throwing it in as an add-on feature for now anyways, so I think we can dodgy it up for now.

So yeah, lots of changes and updates. Lemme know what you think, and keep in mind that if you accept this one, it'll pull the changes in for PR #1 as well.

Thanks!