Package: yt-scraper
YouTube ScraperA simple command utility to extract information from the YouTube API v3 for scientific purposes.AboutThis Python based command line utility enables the easy extraction of information from the YouTube API (Version 3). Currently, it supports only a small subset of the API interface and focuses on extracting related videos from given starting points.InstallationFirst, make sure you have a recent version ofPython 3installed.Next, installyt-scraperby usingpip:sudopipinstallyt-scraperUpdateby adding the--upgradeflag:sudopipinstall--upgradeyt-scraperWindowsusers may need to alter the command to:py-mpipinstall--upgradeyt-scraperMacusers may need to alter the command to:python3-mpipinstall--upgradeyt-scraperAPI KeyIn order to use this program, you will need an official YouTube API key.
You can obtain one fromthispage and
use it with the following examples by appending-k <KEY>to them.UsageCurrently,yt-scraperhas two commands:searchandconfigThe first command is used to query the YouTube API, whereas the second is used to
configure the default configuration.The search command starts a video search from one or multiple given starting points.
These could be multiple videos orginating from a search (usingterm),
or a provided list of video ids (usinginput),
or just one root video (usingidorurl).For example, the following command will return the first video when one searches forcat.$yt-scrapersearchterm'cat'[STATUS] Result:
 Depth: 0, Rank: 0, ID: hY7m5jjJ9mM
            Title: CATS will make you LAUGH YOUR HEAD OFF - Funny CAT compilation
            Related Videos: []One can also provide a video id or a video url as a starting point,
which is more interesting when combined with the--max-depthoption:$yt-scrapersearchid'0A2R27kCeD4'--max-depth2Depth: 0, Rank: 0, ID: 0A2R27kCeD4
            Title: 🤣 Funniest 🐶 Dogs and 😻 Cats - Awesome Funny Home Animal Videos 😇
            Related Videos: ['pc8-8KfIW5c']
     Depth: 1, Rank: 0, ID: pc8-8KfIW5c
                Title: 🦁 Funniest Animals 🐼 - Try Not To Laugh 🤣 - Funny Domestic And Wild Animals' Life
                Related Videos: ['OrJMUNEyZsE']
         Depth: 2, Rank: 0, ID: OrJMUNEyZsE
                    Title: Funniest Videos for Pets to Watch Compilation | Funny Pet Videos
                    Related Videos: []Additionally, one can specify the number of videos
that should be returned on each level by utilizing the--numberoption.
For instance, the following command returns two related videos
from a given video (specified by it's url) and
additionally one related video from each sibling:$yt-scrapersearchurl'https://www.youtube.com/watch?v=0A2R27kCeD4'--depth1--number2--number1[STATUS] Result:
  Depth: 0, Rank: 0, ID: 0A2R27kCeD4
             Title: 🤣 Funniest 🐶 Dogs and 😻 Cats - Awesome Funny Home Animal Videos 😇
             Related Videos: ['pc8-8KfIW5c', 'tbyAuT50eu4']
      Depth: 1, Rank: 0, ID: pc8-8KfIW5c
                 Title: 🦁 Funniest Animals 🐼 - Try Not To Laugh 🤣 - Funny Domestic And Wild Animals' Life
                 Related Videos: []
      Depth: 1, Rank: 1, ID: tbyAuT50eu4
                 Title: 😁 Funniest 😻 Cats and 🐶 Dogs - Awesome Funny Pet Animals 😇
                 Related Videos: []For the sake of brevity, you can shorten--numberto-nand--depthto-d.There are even global commands, too!
Global options are specified in front of the command and
alter the behavior of all commands.
This may not sound very meaningful to you given
that there only two commands right now and you are right!
But this is likely to change in the future.For example, to see more output during the program execution,
specify--verboseor-vright afteryt-scraper:$yt-scraper-vsearchid'0A2R27kCeD4'--max-depth2There are many more options that you can make use of.
All of them are described in the Options section.Sometimes you may find yourself struggling with all the possible options.
Fortunately, there is theconfigcommand for all the lazy typer out there.Setting a particular default option like the output directory to~/my_datais as easy
as typing$yt-scraperconfigsetencodingutf-8Forgetful? Just double-check by typinggetinstead ofset:$yt-scraperconfiggetencoding[STATUS] The value of 'encoding' is set to 'utf-8'.ConfigurationSearch optionsDefaultDescription-n,--number1Number of the videos fetched per level (can be specified several times)-d,--max-depth0Number of recursion steps to perform.-k,--api-keyRequiredThe API key that should be used to query the YouTube API v3.-o,--output-dirOptionalPath to the directory where output files are saved-f,--output-formatcsvSpecifies the file format of output files.-N,--output-nameOptionalSpecifies the file name or prefix of output files.-r,--region-codedeReturn only videos which are unrestricted in the given region.-l,--lang-codedeReturn videos mostly relevant to a specified language.-s,--safe-searchnoneFilter sensitive or restricted videos.-e,--encodingutf-8Transform fetched text to another encoding.-u,--uniqueFalseDo not process seen videos againGlobal optionsDefaultDescription-c,--config-pathSystem-specificSpecifies a configuration file. For details, seeconfiguration.-v,--verboseFalseShows more output during program execution.-V,--versionOptionalShows the current program version and exitsMore information can be found by adding the--helpoption to commands or
reading theYouTube API manual.Old-fashioned people, who do not like theconfigcommand,
can manually configure the program by editing theconfig.tomlfile.
It is secretly used and altered when using theconfigcommand.
Entered values are used in all future queries as long as
they are not overwritten by actual command line options.For example, to always use the API keyABCDEFGHand a search depth of 3,
where on each level one video less is returned,
just create following configuration file:config.tomlapi_key="ABCDEFGH"number=[4,3,2,1]depth=3verbose=trueAn example toml is included:config.tomlThen put this file in your standard configuration folder. Typically this folder can be found at the following location:Mac OS X:~/Library/Application Support/YouTube ScraperUnix:~/.config/youtube-scraperWindows:C:\Users\<user>\AppData\Roaming\YouTube ScraperIf the folder does not exist, you may need to create it first.Release History0.2.6AddedUNLICENSEto project0.3.0Uploaded toPyPI0.4.0New commandsearch0.5.0Option--depthrenamed to--max-depthVideo attributes, such as title, description, channel are fetchedMore consistent option handling0.6.0New export feature:csvNew command:configNew API options:region-code, lang-code and safe-search0.7.0New--versionoptionNew--encodingoptionNew--export-nameoptionNew--uniqueoptionNew input method by importing a file or reading from stdinAdded prompt when encountering an API error0.8.0New--format sqlSQLite exportNewconfig wherecommandMultiple API keys with automatic key switching is now possibleNew--includeoptionNew--excludeoptionRenamedinputargument tofilePiping of urls is now possibleRoadmapEvery of these features is going to be a minor patch:Add node video data attributes, such as title and description.Add possibility to specify more than one API key to switch seamlessly.Add possibility to query more than 50 videos on one level.Add youtube-dl integration for downloading subtitles.Add a testing suite.[o] Add export functionality to CSV, SQLlite or Pandas.Add more information about quota to READMEContributingIf you come across any bugs or have a suggestion,
please don't hesitate tofile an issue.Contributions in any form are welcomed.
I will accept pull-requests if they extentyt-scraper's functionality.To set up the development environment,
please installPoetryand runpoetry installinside the project.
A test suite will be added soon.In general, the contribution process is somewhat like this:Fork it ($ git clone https://github.com/rattletat/yt-scraper)Create your feature branch ($ git checkout -b feature/fooBar)Commit your changes ($ git commit -am 'Add some fooBar')Push to the branch ($ git push origin feature/fooBar)Create a new Pull RequestAuthorMichael BrauweilerTwitter:@rattletatEmail:rattletat@posteo.meLicenseThis plugin is free and unemcumbered software released into the public domain.For more information, see the includedUNLICENSEfile.