Package: metatube
Download YouTube metadata for videos relating to a search queryThis is a Python script that can download metadata (including comments and likes) for YouTube videos relating to a search query. Uses theYouTube Data API v3. Metadata is saved in ansqlalchemycompatible database, for instance, PostgreSQL or SQLite.Metatubeis pauses retrieval once your daily quota is used up (the default as of this writing is 10,000 requests per day) and waits until quota refill. If interrupted,metatubewill, upon restart, first fill gaps in the download history, then continue downloading ‘into the future’. Once caught up to within ten minutes of the current time,metatubeexits.If you usemetatubefor scientific research, please cite it in your publication:Fink, C. (2020):metatube: Python script to download YouTube metadata.doi:10.5281/zenodo.3773302.InstallationpipinstallmetatubeConfigurationCopy the example configuration filemetatube.yml.exampleto a suitable location, depending on your operating system:on Linux systems:system-wide configuration:/etc/metatube.ymlper-user configuration:~/.config/metatube.ymlOR${XDG_CONFIG_HOME}/metatube.ymlon MacOS systems:per-user configuration:${XDG_CONFIG_HOME}/metatube.ymlon Microsoft Windows systems:per-user configuration:%APPDATA%\metatube.ymlAdapt the configuration:Configure a database connection string (connection_string), pointing to an existing database (the format is described in thesqlalchemy documentation.Configure an APIaccess keyto the YouTube Data API v3 (youtube_api_key).Define search terms (search_terms)All of these configuration options can alternatively be supplied as command line arguments tometatube(seeUsage) or as aconfigdictdirectly to the constructor ofYouTubeVideoMetadataDownloader. Command line options (seemetatube --help) orconfigdictboth override config file.UsageCommand line executablemetatube\--postgresql-connection-string"postgresql:///metatube"\--youtube-api-key"abcdefghijklmn"\"how to build a tallbike"PythonImport themetatubemodule. Instantiate aYouTubeVideoMetadataDownloader, optionally supply aconfigdictionary. Then run the instance’sdownload()method.importmetatube# config from config filedownloader=YouTubeVideoMetadataDownloader()downloader.download()# config from config file,# overriding `search_terms`downloader=YouTubeVideoMetadataDownloader({"search_terms":"Critical Mass Vladivostok"})downloader.download()# entire config from dictionarydownloader=YouTubeVideoMetadataDownloader({"youtube_api_key":"opqrstuvwxyz","connection_string":"postgresql://server1/bicyclelover123:supersecretpassword@metatube","search_terms":"dashcam bicycle commute albuquerque"})downloader.download()Data privacyBy default, metatube pseudonymises downloaded metadata, i.e. it replaces (direct) identifiers with randomised identifiers (generated using hashes, i.e. ‘one-way encryption’). This serves as one step of a responsible data processing workflow. However, the text and descriptions of videos and comments might nevertheless qualify asindirect identifiers, as they, combined or on their own, might allow re-identification of the commenter or uploader. If you want to use data downloaded using metatube in a GDPR-compliant fashion, you have to follow up the data collection stage withdata minimisationand further pseudonymisation or anonymisation efforts.Metatube can keep original identifiers (i.e. skip pseudonymisation). Set the according command line argument, configuration file orconfigdict(see thesample config fileand below). Ensure that you fulfil all legal and organisational requirements to handle personal information before you decide to collect non-pseudonyismed data.importmetatubedownloader=YouTubeVideoMetadataDownloader({"search_terms":"Winter Cycling Congress","pseudonymise":False# get legal/ethics advice before doing this})downloader.download()