Package: artvee-scraper
artvee-scraperartvee-scraperis an easy to use command line utility for fetching public domain artwork fromhttps://www.artvee.com.Artvee Web-scraperInstallationSynopsisExamplesAvailable CommandsInstallationUsing PyPI$python-mpipinstallartvee-scraperPython 3.8+ is officially supported.Synopsisartvee-scraper <command> [optional arguments] [positional arguments]ExamplesView help$artvee-scraper-husage: artvee-scraper [-h] {log-json,file-json,file-multi} ...Scrape artwork from https://www.artvee.compositional arguments:{log-json,file-json,file-multi}log-json            Artwork is output to the log as a JSON objectfile-json           Artwork is represented as a JSON object and written to a filefile-multi          Artwork image and metadata are written as separate filesoptional arguments:-h, --help            show this help message and exitView help for thefile-jsoncommand$artvee-scraperfile-json-husage: artvee-scraper file-json [-h] [-t [1-16]] [-l {DEBUG,INFO,WARNING,ERROR,CRITICAL}][-c {abstract,figurative,landscape,religion,mythology,posters,animals,illustration,still-life,botanical,drawings,asian-art}][--log-dir LOG_DIR] [--log-max-size [1-10240]] [--log-max-backups [0-100]][--space-level [2-6]] [--sort-keys] [--overwrite-existing]dir_pathpositional arguments:dir_path              JSON file output directoryoptional arguments:-h, --help            show this help message and exit-t [1-16], --worker-threads [1-16]Number of worker threads (1-16)-l {DEBUG,INFO,WARNING,ERROR,CRITICAL}, --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}Set the application log level-c {abstract,figurative,landscape,religion,mythology,posters,animals,illustration,still-life,botanical,drawings,asian-art}, --category {abstract,figurative,landscape,religion,mythology,posters,animals,illustration,still-life,botanical,drawings,asian-art}Category of artwork to scrape--space-level [2-6]   Enable pretty-printing; number of spaces to indent (2-6)--sort-keys           Sort JSON keys in alphabetical order--overwrite-existing  Overwrite existing filesoptional log file arguments:--log-dir LOG_DIR     Log file output directory--log-max-size [1-10240]Maximum log file size in MB (1-10,240)--log-max-backups [0-100]Maximum number of log files to keep (0-100)Download artwork fromartvee.comand save each as individal files (JSON format) in the directory~/artvee/downloads$artvee-scraperfile-json~/artvee/downloadsAvailable Commandslog-jsonfile-jsonfile-multilog-jsonDownload artwork and output each to the log as a JSON objects.
Note: This command is intended for development test usage; typically it is not desirable to dump the data to the log.$artvee-scraperlog-json[optionalarguments]Optional arguments-h|--help(boolean)Display help message.-t|--worker-threads(integer)The number of worker threads used for processing. Range of values is [1-16]. The default value is3.-l|--log-level(string)Application log level. One of:DEBUG, INFO, WARNING, ERROR, CRITICAL. The default value isINFO.-c|--category(string)Category of artwork to fetch. One of:abstract, figurative, landscape, religion, mythology, posters, animals, illustration, still-life,  botanical, drawings, asian-art. May be repeatedly used to specify multiple categories (-c animals, -c drawings). The default value isALLcategories.Optional log file arguments--log-dir(string)Path to existing directory used to storeartvee_scraper.loglog files. Disabled by default.--log-max-size(integer)Maximum size in MB the log file should reach before triggering a rollover. Only applies if--log-dirhas been specified. Range of values is [1-10240]. The default value is1024MB (1GB).--log-max-backups(integer)Maximum number of log file archives to keep. Only applies if--log-dirhas been specified. The actively written file isartvee_scraper.log. Backup files will have an incrementing numerical suffix;artvee_scraper.log.1 ... artvee_scraper.log.N. If this value is zero, rollovers will be disabled. Range of values is [0-100]. The default value is10.Optional writer arguments--space-level(integer)Pretty print JSON; number of spaces to indent. Range of values is [2-6]. Disabled by default.--sort-keys(boolean)Sort JSON keys in alphabetical order. Disabled by default.--include-image(boolean)Image will be included in output. Excessive output warning! Disabled by default.Basic Example$artvee-scraperlog-jsonOutput:...2038-01-19 03:14:07.988 DEBUG [ThreadPoolExecutor-0_0] scraper._image_link_from(120) | Retrieving image download link from URL https://artvee.com/dl/study-for-old-canal-red-green/2038-01-19 03:14:07.989 DEBUG [ThreadPoolExecutor-0_0] connectionpool._new_conn(1001) | Starting new HTTPS connection (1): artvee.com:4432038-01-19 03:14:07.999 INFO [ThreadPoolExecutor-0_0] log_writer.write(44) | {"url": "https://artvee.com/dl/study-for-old-canal-red-green/", "title": "Study for Old Canal (Red & Green)", "category": "Abstract", "artist": "Oscar Bluemner", "date": "1916", "origin": "American, 1867-1938"}...Advanced Example$artvee-scraperlog-json--worker-threads2--log-levelDEBUG--categoryabstract--log-dir/var/log/artvee--log-max-size2048--log-max-backups10--space-level2--sort-keys--include-imageOutput:$cat/var/log/artvee/artvee_scraper.log...2038-01-19 03:14:07.988 DEBUG [ThreadPoolExecutor-0_0] scraper._image_link_from(120) | Retrieving image download link from URL https://artvee.com/dl/study-for-old-canal-red-green/2038-01-19 03:14:07.989 DEBUG [ThreadPoolExecutor-0_0] connectionpool._new_conn(1001) | Starting new HTTPS connection (1): artvee.com:4432038-01-19 03:14:07.999 INFO [ThreadPoolExecutor-0_0] log_writer.write(44) | {"artist": "Oscar Bluemner","category": "Abstract","date": "1916","image": "/9j/4AAQSkZJRgABA ... o4xSSSVkumh//9k=""origin": "American, 1867-1938","title": "Study for Old Canal (Red & Green)","url": "https://artvee.com/dl/study-for-old-canal-red-green/"}...file-jsonDownload artwork and write each to the filesystem. Each artwork is stored as a JSON object.$artvee-scraperfile-json[optionalarguments]<dir_path>Positional argumentsdir_path(string) Position0.Path to existing directory used to store output files.Optional arguments-h|--help(boolean)Display help message.-t|--worker-threads(integer)The number of worker threads used for processing. Range of values is [1-16]. The default value is3.-l|--log-level(string)Application log level. One of:DEBUG, INFO, WARNING, ERROR, CRITICAL. The default value isINFO.-c|--category(string)Category of artwork to fetch. One of:abstract, figurative, landscape, religion, mythology, posters, animals, illustration, still-life,  botanical, drawings, asian-art. May be repeatedly used to specify multiple categories (-c animals, -c drawings). The default value isALLcategories.Optional log file arguments--log-dir(string)Path to existing directory used to storeartvee_scraper.loglog files. Disabled by default.--log-max-size(integer)Maximum size in MB the log file should reach before triggering a rollover. Only enabled if--log-dirhas been specified. Range of values is [1-10240]. The default value is1024MB (1GB).--log-max-backups(integer)Maximum number of log file archives to keep. Only enabled if--log-dirhas been specified. The actively written file isartvee_scraper.log. Backup files will have an incrementing numerical suffix;artvee_scraper.log.1 ... artvee_scraper.log.N. If this value is zero, rollovers will be disabled. Range of values is [0-100]. The default value is10.Optional writer arguments--space-level(integer)Pretty print JSON; number of spaces to indent. Range of values is [2-6]. Disabled by default.--sort-keys(boolean)Sort JSON keys in alphabetical order. Disabled by default.--overwrite-existing(boolean)Allow existing duplicate files to be overwritten. Disabled by default.Basic Example$artvee-scraperfile-json~/artvee/downloadsOutput:$cat~/artvee/downloads/peter-nicolai-arbo-the-valkyrie.json{"url": "https://artvee.com/dl/the-valkyrie-2/", "title": "The Valkyrie", "category": "Mythology", "artist": "Peter Nicolai Arbo", "date": "1869", "origin": "Norwegian, 1831–1892", "image": "/9j/4AAQSkZJRgABA ... o4xSSSVkumh//9k="}Advanced Example$artvee-scraperfile-json--worker-threads1--log-levelINFO--categorymythology--log-dir/var/log/artvee--log-max-size512--log-max-backups10--space-level4--sort-keys--overwrite-existing~/artvee/downloadsOutput:$cat~/artvee/downloads/peter-nicolai-arbo-the-valkyrie.json{"artist": "Peter Nicolai Arbo","category": "Mythology","date": "1869","image": "/9j/4AAQSkZJRgABA ... o4xSSSVkumh//9k=""origin": "Norwegian, 1831–1892","title": "The Valkyrie","url": "https://artvee.com/dl/the-valkyrie-2/"}file-multiDownload artwork and write each to the filesystem. Each artwork is stored as two files: metadata (JSON) & image (JPG).$artvee-scraperfile-multi[optionalarguments]<metadata_dir_path><image_dir_path>Positional argumentsmetadata_dir_path(string) Position0.Path to existing directory used to store output metadata files.image_dir_path(string) Position1.Path to existing directory used to store output image files.Optional arguments-h|--help(boolean)Display help message.-t|--worker-threads(integer)The number of worker threads used for processing. Range of values is [1-16]. The default value is3.-l|--log-level(string)Application log level. One of:DEBUG, INFO, WARNING, ERROR, CRITICAL. The default value isINFO.-c|--category(string)Category of artwork to fetch. One of:abstract, figurative, landscape, religion, mythology, posters, animals, illustration, still-life,  botanical, drawings, asian-art. May be repeatedly used to specify multiple categories (-c animals -c drawings). The default value isALLcategories.Optional log file arguments--log-dir(string)Path to existing directory used to storeartvee_scraper.loglog files. Disabled by default.--log-max-size(integer)Maximum size in MB the log file should reach before triggering a rollover. Only enabled if--log-dirhas been specified. Range of values is [1-10240]. The default value is1024MB (1GB).--log-max-backups(integer)Maximum number of log file archives to keep. Only enabled if--log-dirhas been specified. The actively written file isartvee_scraper.log. Backup files will have an incrementing numerical suffix;artvee_scraper.log.1 ... artvee_scraper.log.N. If this value is zero, rollovers will be disabled. Range of values is [0-100]. The default value is10.Optional writer arguments--space-level(integer)Pretty print JSON; number of spaces to indent. Range of values is [2-6]. Disabled by default.--sort-keys(boolean)Sort JSON keys in alphabetical order. Disabled by default.--overwrite-existing(boolean)Allow existing duplicate files to be overwritten. Disabled by default.Basic Example$artvee-scraperfile-multi~/artvee/downloads/metadata~/artvee/downloads/imagesOutput:$cat~/artvee/downloads/metadata/peter-nicolai-arbo-the-valkyrie.json{"url": "https://artvee.com/dl/the-valkyrie-2/", "title": "The Valkyrie", "category": "Mythology", "artist": "Peter Nicolai Arbo", "date": "1869", "origin": "Norwegian, 1831–1892"}$cat~/artvee/downloads/images/peter-nicolai-arbo-the-valkyrie.jpg<FF><D8><FF><E0>^@^PJFIF^@^A^A^A^A,^A,^@^@<FF><E1><D5>$Exif^@^@II*^@^...^<X-nA2_vއ%6gS`QErVOOqk;R,u{w9~onDbsEWQ㿟xyrAdvanced Example$artvee-scraperfile-multi--worker-threads1--log-levelINFO--categorymythology--log-dir/var/log/artvee--log-max-size512--log-max-backups10--space-level2--sort-keys--overwrite-existing~/artvee/downloads/metadata~/artvee/downloads/imagesOutput:$cat~/artvee/downloads/metadata/peter-nicolai-arbo-the-valkyrie.json{"artist": "Peter Nicolai Arbo","category": "Mythology","date": "1869","origin": "Norwegian, 1831–1892","title": "The Valkyrie","url": "https://artvee.com/dl/the-valkyrie-2/"}$cat~/artvee/downloads/images/peter-nicolai-arbo-the-valkyrie.jpg<FF><D8><FF><E0>^@^PJFIF^@^A^A^A^A,^A,^@^@<FF><E1><D5>$Exif^@^@II*^@^...^<X-nA2_vއ%6gS`QErVOOqk;R,u{w9~onDbsEWQ㿟xyr