Package: thetrove-downloader
thetrove-downloaderThis Python package allows to easily download any resource, be it folder or file,
fromThe Trove.is.Installation and RequirementsThe program requires Python 3.9 or above.To install the program it is sufficient to use Python pip and get the packagethetrove-downloader. The package will
add a new commandthetrove-downloader.python3-mpipinstallthetrove-downloaderUpdates should be performed via pip.python3-mpipinstall--upgradethetrove-downloaderUsagethetrove-downloader[-h][-t,--targetTARGET][--target-urlURL][-j,--jsonJSON][-f,--folderFOLDER][-o,--outputOUTPUT][-b,--blacklistBLACKLIST][-w,--whitelistWHITELIST][--no-download][--no-preserve-root]-h, --helpshow help message-t, --target <TARGET>download target (folder or file)To download a specific resource, start the program with the--targetcommand and supply a valid path (e.g., "
Books/White Star").--target-urldownload target (full URL to folder or file)Overrides--targetargument with a full URL and sets the download target to the path of the URL (
e.g., "https://thetrove.is/Open%20Source/BIND%20RPG" to "Open Source/BIND RPG").-j, --json <JSON>save/read instructions from a JSON fileThe--jsonoption allows to specify a file to save the target, folder, output, whitelist, and blacklist instructions
in JSON format. Launching the program with only the--jsonoption will load the instructions previously saved in the
file and perform all of them in succession. This can be used to keep a local copy of specific resources up to date.-f, --folder <FOLDER>destination folderThe--folderoption allows to specify a folder in which the targeted resource will be downloaded.-o, --output <OUTPUT>output name of download targetThe--outputoption allows to override the folder/file name of the resource. Note that if the targeted resource is a
folder, the option will only apply to the top-level folder and leave all contents therein unaltered.-b, --blacklist <BLACKLIST>regex blacklist for files/foldersThe--whitelistoption allows to specify a regex pattern that will be used to skip files and folder that do not
contain the supplied pattern. Overwrites blacklist options.-w, --whitelist <WHITELIST>regex whitelist for files/folders, overrides blacklistThe--whitelistoption allows to specify a regex pattern that will be used to skip files and folder that do not
contain the supplied pattern. Overwrites blacklist options.--no-downloadlist content without downloadingThe--no-downloadflag allows to list the content under the supplied target without performing any download. The
instructions will still be saved in a JSON file if the--jsonoptions is used, but not the flag.--no-preserve-rootallows the download of root foldersThe--no-preserve-rootflag allows downloading entire root folders such as "/Books/". This flag is not preserved
with the--jsonoption and must be passed manually every time.NOTE: the program requires to use either--target,--json, or both--targetand--json. Without one of
these combinations the program won't have a valid download instruction.Notes on CrawlingThe program performs all GET requests without throttling or delay and with no disallowed paths as per the instructions
atthetrove.is/robots.txt.However, please do take care to not overuse this utility and risk overwhelming their connection. Thank you.Root folders such as "/Books" are protected and cannot be downloaded without using the--no-preserve-rootflag.ContributingAll contributions and suggestions are welcome!If you have suggestions for fixes or improvements, you can open an issue with your idea, see#Issuesfor
details.IssuesIf you encounter any problem during usage of the program, an issue can be opened on the project's page
onGitLab.When opening an issue for a problem, please copy the error message – if any – and describe the operation in progress
when the error occurred.