Package: youtube-discussion-tree-api
YouTube Discussion Tree APIThis is a python API that allows you to obtain the discussion
that occurs in the comments of a YouTube video as a tree structure.
It also controls the quota usage that consumes your implementation over
YouTube Data API through this library, and allows you to represent and
serialize the discussion tree.InstallIt is recommended to install it through pip:pipinstallyoutube_discussion_tree_apiAlso, if you want to use it from source you will have to install manually the dependencies:pipinstall-rrequeriments.txtAPINow we're going to dive into the features that the API provide and how to use them.Generating a treeThis is the main feature of the API. The easiest way of generating a tree is:fromyoutube_discussion_tree_apiimportYoutubeDiscussionTreeApiapi=YoutubeDiscussionTreeApi("<put your gcs api and services api key>")tree=api.generateTree("put the videoId")First, you create a YoutubeDiscussionTreeApi object with the API KEY of your GCP project. This object
is thought as the template object that holds all the interactions with the YouTube Data API.Then, we call the method generateTree, that give us as a response a YoutubeDiscussionTree object that
holds the tree structure.Each node of the tree is a Node object:classNode:id:intauthor_name:strauthor_id:inttext:strlike_count:intparent_id:intpublished_at:strTherootnode represents the video. In the text field of the root Node, a transcription of the whole video is provided.
The API can only generate trees of the videos that have manual or auto-generated English subtitles.The method generateTree has two optional parameters that are:sumarization -> bool: This parameter is for applying a summarization to the transcription of the video.conflict_solving_algorithm -> function: This parameter is a function that does the resolution of conflicts that we face when generating the tree. Will be explained later.We can get the list of all the nodes of our tree by doing:tree.get_nodes()Serializing a treeWe can serialize our tree into a xml file:tree.serialize("outuput_file.xml")This will construct a xml file with the following structure:<entailment-corpusnum_edges="162"num_nodes="163"><argument-list>...<argument-list><argument-pairs>...<argument-pairs></entailment-corpus>In argument-list component there will be all the comment nodes with some information of the Node objects.
In the argument-pairs component, there will be all the edges that communicate the nodes described in the previous component.A node will be represented as:<argauthor="Flash Man"author_id="UCeFi97LktRRtpCvi_vqEmfg"id="Ugh8N1Ch9gCr-HgCoAEC"likeCount="1145">Mydadisanexpertdragonslayer."eeerbutIdon'tseeanydragonsaround..."You'rewelcome.</arg>And a pair as:<pairid="1"><tid="Ugh8N1Ch9gCr-HgCoAEC"/><hid="LnX3B9oaKzw"/></pair>Where h is the destination and t the originAlso, the function serialize() has an optional argument calledaditional_atributes. You can pass a function that receives a Node object and outputs a {key : value} dictionarythat will represent the additional attributes that you would like to add to the xml tags that represent the Nodes. The key will be the attribute name and the value will be the value of the attribute.defmy_additional_atributes(node):return{"date":node.publishedAt"sentiment":sentiment_analysis(node.txt)#Imagine we have a function that does sentiment analysis from an input text}tree.serialize("outuput_file.xml",my_aditional_atributes)The nodes will have the following form:<argauthor="Flash Man"author_id="UCeFi97LktRRtpCvi_vqEmfg"id="Ugh8N1Ch9gCr-HgCoAEC"likeCount="1145"date="12-12-2012"sentiment="NEGATIVE">Mydadisanexpertdragonslayer."eeerbutIdon'tseeanydragonsaround..."You'rewelcome.</arg>Representing a treeIf you want to see a representation of the tree, you can do:tree.show()And this will output a tree like:LnX3B9oaKzw
├── Ugg06_f0qAVH6HgCoAEC
├── Ugg662Arr0neQXgCoAEC
│   ├── Ugg662Arr0neQXgCoAEC.8LvCxcl0tY18LvI0OyzxUW
│   ├── Ugg662Arr0neQXgCoAEC.8LvCxcl0tY18LvIRl9v4kU
│   ├── Ugg662Arr0neQXgCoAEC.8LvCxcl0tY18LvIlAkRi3e
                    .
                    .
                    .Conflict Solving AlgorithmIf you read the "Generating a tree" part, you may have seen that in the generateTree function there is an optional
argument for a conflict solving algorithm. Well, lets explain what we define as a conflict.We say that there is a conflict when a user wants to reply to another user's comment, but this second one
has made more than one contribution previously to the comment thread. This is because YouTube doesn't store
the reference to the comment of the comment thread that we want to reply. It automatically sets the parent id
of the reply to the top level comment of the comment thread, instead of the id of the comment that we are replying to.In order to solve that, in this library there is an implementation of an algorithm that solve this conflicts automatically.
The algorithm it's an implementation of the tf-idf algorithm that selects a candidate from a set of candidates for the
reply that we don't know to which of this candidate belongs.In order to make this library as flexible as possible, it's open to accept a function with an implementation of a
conflict solving  algorithm made by you through the optional argument of generateTree.This function will receive as parameter the reply Node and a set of Node candidates.Search videosThrough the YoutubeDiscussionTreeAPI object, you can also request for a set of videos that are found matching a given query:fromyoutube_discussion_tree_apiimportYoutubeDiscussionTreeApiapi=YoutubeDiscussionTreeApi("<put your gcs api and services api key>")videos=api.search_videos("Functional Programming")This will return a set with 5 Video objects:classVideo:id:inttitle:strdescription:strchannel_name:strchannel_id:strpublished_at:strYou can expand or diminish the number of results in the result set by passing another optional parameter:fromyoutube_discussion_tree_apiimportYoutubeDiscussionTreeApiapi=YoutubeDiscussionTreeApi("<put your gcs api and services api key>")videos=api.search_videos("Functional Programming",50)QuotaAdditionally, this library maintains the number of quota that your implementation is consuming against the
Google YouTube API service. This is because the free service is limited to a quota of 10000 daily. Each tree
generation consumes 2 quota, and each video search consumes 100  quota.A data object is serialized in a pickle file. You will see that it will appear in the path in which your script
that uses the library is. It automatically resets its value day after day.In order to get quota information, you can do:fromyoutube_discussion_tree_apiimportYoutubeDiscussionTreeApiapi=YoutubeDiscussionTreeApi("<put your gcs api and services api key>")api.quota_info()This will give you a dict like:{"limit":10000"spent":<num-expended-quota>}Author NoteHey, Quim here! :man_technologist:So, this is it for the YouTube Discussion Tree API, hope you enjoy the package! If you have any comment(xD)
on the implementation, or you want to share some features that can be added to de library, hit me up!
Any kind of feedback will be pleasantly accepted! :smile:Quim10^-12