Package: img-lurker
img-lurkerimg-lurker is a gallery downloader.img-lurker takes a URL of a (HTML) web page and downloads linked images on it.
If the page contains only thumbnails, linking to a the full size version of
the image, img-lurker will rather take the bigger one.
If there are links to other HTML pages (themselves containing a the full size
image), img-lurker will follow those links to find the bigger size.img-lurker has a "minimum image size" for considering an image is worthy of being downloaded and
isn't UI stuff like buttons/separators. img-lurker will not follow links if the link tag doesn't
contain an image tag (assumed to be the thumbnail).ExampleConsider a site with following HTML:<a href="fullimage1.jpg">
    <img src="thumbnail1.jpg" />
</a>
<a href="fullimage2.jpg">
    <img src="thumbnail2.jpg" />
</a>img-lurker would downloadfullimage1.jpgandfullimage2.jpg.
If instead the links point to other HTML pages containing the full size version
of the images (for examplefullimage1.htmlcontainingfullimage1.jpg),
img-lurker would still findfullimage1.jpgby following the page links.OptionsCookies--cookie KEY=VALUEInject a specific cookie, which might be required to visit some restricted
access pages. For example, some subreddits require you to pass the cookie "over18=1".The option can be passed several times to inject multiple cookies.Pagination--next-page-xpath HTML_XPATHimg-lurker can handle pagination for sites where a gallery contains so many
images that the site is split in numbered pages.HTML_XPATHshould be an XPath expression locating the HTML link to the "next
page".
If this argument is given, after downloading all images of a "page", img-lurker
will follow the link pointed to byHTML_XPATHand repeat on the next page.Warning: this can issue a lot of traffic for huge galleries. Be cautious or you
might get blocked by the website.Stop/resume--history-file FILEMark all downloaded images URLs in this file and avoid redownloading URLs
present in this file.
Useful when running img-lurker multiple times on the same gallery, typically if
the gallery has received fresh images. Also useful if you use--next-page-xpathoption and kill img-lurker to avoid flooding the site, make
a pause (minutes? hours? days?) then restart img-lurker: the history file will
help it resume where it was interrupted.This makes the assumption that:each image will always have the same URL, e.g. no varying tokens/timestamps in the URL, etc.conversely, an URL will always point to the same image, it will not point to another image at some point, e.g. the
images are NOT numbered in ascending order (else1.jpgwould point to
different images over time).Tell apart thumbnails from "big images" to download only the latter--min-thumb-size WIDTHxHEIGHT
--min-image-size WIDTHxHEIGHTMinimum size for an image to be considered a thumbnail worth following or an
image worth downloading. Useful not to download navigation buttons, logos, etc.
Default values are--min-thumb-size=128x128and--min-image-size=400x400.--max-aspect-ratio WIDTH:HEIGHTMaximum ratio between WIDTH and HEIGHT (or HEIGHT on WIDTH, img-lurker is smart
enough to figure out) to consider an image is worth downloading.For example, pass "16:9" and img-lurker will accept images with dimensions
1920x1080 or 1080x1920 as they are respectively 16:9 and 9:16 but also 1600x1200
or 1200x1600 because they are 4:3 (and 3:4) which is lower (more looking like
a square) than the max "16:9". Ratios of portrait and landscape are considered
equivalent.
However, passing "16:9" would discard a banner with dimensions 1200x300 because
its ratio is 4:1 which is way more distorted (very thin rectangle) than 16:9.
It would also reject a banner with dimensions 300x1200 because it is 1:4,
equivalent to 4:1.A photo is rarely square but is almost never thin like 4:1, except panoramas, so
configure this option if you intend to download panoramas for example.
The default value is--max-aspect-ratio=4:1.Debug--debugDebug log.Limitationsimg-lurker will not interpret javascript, though it has specific hints to detect
lazy-loaded images, so it might not work on sites like instagram.img-lurker will not open iframes, so it will fail to download a few images from
reddit.img-lurker does not crawl a site and does not support nested galleries, it only
takes one gallery and expects it to contain the images desired.