Package: prop-request
FirstIf you want to read README in Japanease, please readthis.What it can doYou can download files recursively, extract specific tags from html, etc.InstallIf you have pip$pipinstallprop-requestIf you don't have pip$sudowgethttps://github.com/mino-38/prop/releases/latest/download/prop-O/usr/local/bin/prop# or$sudocurl-Lhttps://github.com/mino-38/prop/releases/latest/download/prop-o/usr/local/bin/prop# After download binary filesudochmoda+rx/usr/local/bin/propUsage$prop[options]URLOptions.-o, --output [path]The -o, --output option allows you to specify the output file or directory.The -o and --output options can be used to specify the destination file or directory, and will save the file in the specified directory with the same name as the source file.Also, when using the -r option, please specify the destination directory with this option-OSave the file with the same name as the original download.If -o, --output, or this option is not specified, the file will be output to standard output.-a, --fake-user-agentFake the UserAgent value.-U, --upgradeUpdate prop.This is just running pip install --no-cache-dir --upgrade prop-request, so you can run this directly.-s, --search-words [search-words]You can search from the html code of the specified URL.The following is an example of what can be used as search words (you can also specify rel, alt, etc., but there are many, so only the main ones are used)queryvaluetagstag nameclassclass nameididtextvalue of taghrefreferencesrcreference to an image, etclimitnumber of retrievals (can also be specified with the -M option)If you want to specify multiple values, separate them with ',' (no spaces).Ex:$prop-stags=a,scriptlimit=5URL

->Get5tagsfromtheURLsourcecode.-M, --limit [limit]Specify the number of files to download recursively, or the number of results to retrieve with the -s, --search option.-R, --read-file [file]Read from a file with pre-defined URLs and options.Also, since the session is retained, it is possible to access the file after logging in.Ex:Contents of instruct.txt-a -n -d name=hoge password=hogehoge -o /dev/null https://www.example.com/login.php
-O https://www.example.com/page.html$prop-Rinstruct.txt
>>>Downloadhttps://www.example.com/page.htmlaspage.html-r, --recursive [number of levels to go down]Recursive download from the specified URL.If the number of descending levels is not specified, it is assumed to be 1.The target of the recursive download is the URL specified in the href attribute of the 'a' tag and the src attribute of the img.If the -nE option is not specified, it will automatically replace the reference to the local file after downloading.If you use this option, please specify the destination directory with -o, --output option.Options available only with the -r option-I, --interval [interval]Specify the interval of the download.Recursive downloads can overload the target site, so it is recommended to specify at least 5 seconds.If a shorter time than the robots.txt directive is specified, it will be replaced by the robots.txt value.-f, --format [format]Allows you to specify the format of the file name of the file to be downloaded.The special format is as follows.formatvalue to be assigned%(root)shostname of the download source%(file)sfile name of download source%(num)dsequential number starting from 0%(ext)sfile extensionEx:$prop-r-f"%(num)dtest-%(file)s"-ostore_directoryURL

->store_directory/0test-[filename],store_directory/1test-[filename]...Downloadwiththename

$prop-r-f"test-%(num)d.%(ext)s"-ostore_directoryURL

->store_directory/test-0.[ext],store_directory/test-1[ext]...DownloadasNote that if the format does not include %(num)d or %(file)s, it will not be reflected (because the store name does not change dynamically).Also, there are some restrictions: %(file)s and %(ext)s formats can only be used at the end, more than one %(num)d cannot be used, and special formats such as %(num)d%(file)s cannot be used consecutively (because it is impossible to generate an exact sequential number for %(num)d).Option to restrict (or extend) the download targetshort option namelong option nameprocessing-np--no-parentoption to ignore URLs in the hierarchy above the starting URL-nc--no-contentoption to target only URLs with the href attribute of the 'a' tag-nb--no-bodyoption to ignore URLs of src attribute of 'img' tag-nd--no-downloadedoption to ignore files written in histories-dx--download-externaloption to include URLs of external sites in the download-st--startoption to specify a file name to start downloading-n--download-filenameOnly download files include specified stringThe -nc and -nb options cannot be used together.For options not listed hereUse the -h and --help options to see help message.Please refer to the help page, which includes the options listed here.Where to save history, logs and cacheYou can see where the history is stored with --history-directory, where the logs are written with --log-file option, and where the cache is stored with --cache-directory.# A quick look at the logs$cat$(prop--log-file)# History list$ls$(prop--history-directory)# Where to store the cache$prop--cache-directoryAlso, have the option to remove them.# Remove log file$prop--purge-log# Remove histories$prop--purge-history# Remove caches$prop--purge-cacheWhere to setting fileBy changing content of setting file, you can set proxy, timeout, and so on. (This setting is globally.)The path of setting file depends, but filename is same: "config.json"If you installed prop with pip, the directory of setting file is the same place as the directory of prop-request.
You can see it by "pip show prop-request" command.If you installed prop with binary file, the path of setting file is "~/.prop-datas/config.json"New featureWhen the output will be printed to stdout and size of the output is large, ask whether you want to continue.LicenseMIT license.