Package: yt-fts
yt-fts - Youtube Full Text Searchyt-ftsis a command line program that usesyt-dlpto scrape all of a youtube channels subtitles and load them into an sqlite database that is searchable from the command line. It allows you to query a channel for specific key word or phrase and will generate time stamped youtube urls to
the video containing the keyword.It also supports semantic search via theOpenAI embeddings APIusingchromadb.Blog PostSemantic SearchCHANGELOGhttps://github.com/NotJoeMartinez/yt-fts/assets/39905973/6ffd8962-d060-490f-9e73-9ab179402f14Installationpipinstallyt-ftsDependencies:This project requiresyt-dlpinstalled globally. Platform specific installation instructions are available on theyt-dlp wiki.pippython3-mpipinstall-Uyt-dlpMacOS/Homebrewbrewinstallyt-dlpWindows/wingetwingetinstallyt-dlpdownloadDownload subtitles for a channel.Takes a channel url or id as an argument. Specify the number of jobs to parallelize the download with the--number-of-jobsoption.yt-ftsdownload--number-of-jobs5"https://www.youtube.com/@3blue1brown"listList saved channels.The (ss) next to the channel name indicates that the channel has semantic search enabled.yt-ftslist┏━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ID ┃ Name                  ┃ Count ┃ Channel ID               ┃
┡━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1  │ ChessPage1 (ss)       │ 19    │ UCO2QPmnJFjdvJ6ch-pe27dQ │
│ 2  │ 3Blue1Brown           │ 127   │ UCYO_jab_esuFRV4b17AJtAw │
│ 3  │ george hotz archive   │ 410   │ UCwgKmJM4ZJQRJ-U5NjvR2dg │
│ 4  │ The Tim Dillon Show   │ 288   │ UC4woSp8ITBoYDmjkukhEhxg │
│ 5  │ Academy of Ideas (ss) │ 190   │ UCiRiQGCHGjDLT9FQXFW0I3A │
└────┴───────────────────────┴───────┴──────────────────────────┘searchFull text search for string in saved channels.The search string does not have to be a word for word and matchSearch strings are limited to 40 characters.# search in all channelsyt-ftssearch"life in the big city"# search in specific channelyt-ftssearch"life in the big city"--channel"The Tim Dillon Show"# search in specific channel by idyt-ftssearch"life in the big city"-c4"Dennis would go hey life in the big city"

    Channel: The Tim Dillon Show
    Title: 154 - The 3 AM Episode - YouTube
    Time Stamp: 00:58:53.789
    Video ID: MhaG3Yfv1cU
    Link: https://youtu.be/MhaG3Yfv1cU?t=3530Search in videoyt-ftssearch"text to search"--video[VIDEO_ID]Advanced Search SyntaxThe search string supports sqliteEnhanced Query Syntax.
which includes things likeprefix querieswhich you can use to match parts of a word.yt-ftssearch"rea* kni* Mali*"--channel"The Tim Dillon Show"output:"real knife fight down here in Malibu I"

    Channel: The Tim Dillon Show
    Title: #200 - Knife Fights In Malibu | The Tim Dillon Show - YouTube
    Time Stamp: 00:45:39.420
    Video ID: e79H5nxS65Q
    Link: https://youtu.be/e79H5nxS65Q?t=2736vsearchVector search, requires that you enable semantic search for a channel withget-embeddings.
It has the same options assearchbut output will be sorted by similarity to the search string
and the return limit is 10.yt-fts vsearch "deep quote by russian author" --channel "Academy of Ideas""the great Russian author Fyodor Dostoevsky above all don't 
lie to yourself he wrote the man who lies to"

    Distance: 0.25210678577423096
    Channel: Academy of Ideas - (UCiRiQGCHGjDLT9FQXFW0I3A)
    Title: The Psychology of Self-Deception - YouTube
    Time Stamp: 00:10:01.749
    Video ID: Uig8Lw7ixI0
    Link: https://youtu.be/Uig8Lw7ixI0?t=598How ToExport search results:For both thesearchandvsearchcommands you can export the results to a csv file with
the--exportflag. and it will save the results to a csv file in the current directory.yt-ftssearch"life in the big city"--export
yt-ftsvsearch"existing in large metropolaten center"--exportDelete a channel:You can delete a channel with thedeletecommand.yt-ftsdelete--channel"3Blue1Brown"Update a channel:The update command currently only works for full text search and will not update the
semantic search embeddings.yt-ftsupdate--channel"3Blue1Brown"Semantic Search via OpenAI embeddings APIYou can enable semantic search for a channel by using theget-embeddingscommand.
This feature requires an OpenAI API key set in the environment variableOPENAI_API_KEY,
or you can pass the key with the--openai-api-keyflag.get-embedingsFetches OpenAI embeddings for specified channelyt-ftsget-embeddings--channel"3Blue1Brown"After the embeddings are saved you will see a(ss)next to the channel name when you
list channels and you will be able to use thevsearchcommand for that channel.