jamescalam/channel-metadata huggingface dataset:
Title: How to Build Python Packages for Pip
Category: Education
Published: 2021-04-02 15:19:32 UTC
The most powerful feature of Python is its community. Almost every use-case out there has a package built specifically for it.

Need to send mobile/email alerts? pip install knockknock  -  Build ML apps? pip install streamlit  -  Bored of your terminal? pip install colorama  -  It's too easy!

I know this is obvious, but those libraries didn't magically appear. For each package, there is a person, or many persons - that actively developed and deployed that package.

Every single one.

All 300K+ of them.

That is why Python is Python, the level of support is phenomenal - mindblowing.

In this video, we will learn how to build our own packages. And add them to the Python Package Index (PyPI). Afterward, we will be able to install our packages using pip install!

GitHub Repo:
https://github.com/jamescalam/aesthetic_ascii

Medium Article:
https://towardsdatascience.com/how-to-package-your-python-code-df5a7739ab2e

📖 Here's a free link:
https://towardsdatascience.com/how-to-package-your-python-code-df5a7739ab2e?sk=04d9f67c0654445bbcbbf6825f535900
Title: How-to Structure a Q&A ML App
Category: Education
Published: 2021-04-09 15:22:50 UTC
▶️ Stoic Q&A App Playlist: https://www.youtube.com/playlist?list=PLIUOU7oqGTLixb-CatMxNCO-mJioMmZEB

I'm planning on doing something different, a series of videos where we work through the steps - from start-to-finish - of (attempting) to build a Q&A web app that answers our questions with Stoic answers.

In this video, I'm outlining the idea and describing the high-level setup that I think we'll need to put together. It should be cool!

We'll be using the Haystack framework for 'Q&A at scale', which using HuggingFace transformers under-the-hood, and the Elasticsearch document store.

Find the repo here:
https://github.com/jamescalam/aurelius
Title: How to Index Q&A Data With Haystack and Elasticsearch
Category: Education
Published: 2021-04-12 15:00:11 UTC
▶️ Stoic Q&A App Playlist: https://www.youtube.com/playlist?list=PLIUOU7oqGTLixb-CatMxNCO-mJioMmZEB

The second video in 'Building a Stoic Q&A App' - here we're setting up Elasticsearch and Haystack to store the data (Meditations) ready for retrieval when we ask our app questions.

Find the code here:
https://github.com/jamescalam/aurelius/tree/main/code/labs
Title: Q&A Document Retrieval With DPR
Category: Education
Published: 2021-04-15 15:00:10 UTC
▶️ Stoic Q&A App Playlist: https://www.youtube.com/playlist?list=PLIUOU7oqGTLixb-CatMxNCO-mJioMmZEB

The third video in building our Stoic Q&A app.

In open-domain question answering, we typically design a model architecture that contains a data source, retriever, and reader/generator.

The first of these components is typically a document store. The two most popular stores we use here are Elasticsearch and FAISS.

Next up is our retriever — the topic of this video. The job of the retriever is to filter through our document store for relevant chunks of information (the documents) and pass them to the reader/generator model.

DPR (dense passage retriever) is a dense vector retriever that is trained on question-context pairs. Encoding both accordingly - enabling super accurate similarity indexing.

If you're interested in learning more about DPR, I wrote about it on Medium here:
https://towardsdatascience.com/how-to-create-an-answer-from-a-question-with-dpr-d76e29cc5d60

(Free link):
https://towardsdatascience.com/how-to-create-an-answer-from-a-question-with-dpr-d76e29cc5d60?sk=1bdd7c1bff80bf51410962691c690c69

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: How to Use Type Annotations in Python
Category: Education
Published: 2021-04-27 14:53:25 UTC
Type annotations - also known as type signatures - are used to indicate the datatypes of variables and input/outputs of functions and methods.

In many languages, datatypes are explicitly stated. In these languages, if you don't declare your datatype - the code will not run.

Type annotations have a long and convoluted history with Python, going all the way back to the first release of Python 3 with the initial implementation of function annotations.

Type annotations in Python are not make-or-break like in other languages (like C). They're optional chunks of syntax that we can add to make our code more explicit.

Erroneous type annotations will do nothing more than highlight the incorrect annotation in our code editor - no errors are ever raised due to annotations.

So, if type annotations are not enforced, why use them?

Well, as we touched upon already - declaring types makes our code more explicit, and if done well, easier to read - both for ourselves and others.

Read the Medium article here:
https://towardsdatascience.com/type-annotations-in-python-d90990b172dc

📖 Here's a free link:
https://towardsdatascience.com/type-annotations-in-python-d90990b172dc?sk=29bc29ab5478a842363963b421781b47

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff

00:00 Intro
00:55 Datatypes Example in C
2:53 Static and Dynamic Typed Languages
3:47 Type Annotations in Python
4:25 How to Define Simple Types
6:04 IDE Warnings
8:20 More Complex Types
9:53 dict[str, int]
11.07 Multiple Types
11:38 Union Operator (Py 3.9)
12:34 Union Operator (Py 3.10)
13:21 Optional Operator
Title: Extractive Q&A With Haystack and FastAPI in Python
Category: Education
Published: 2021-04-29 15:00:04 UTC
▶️ Stoic Q&A App Playlist: https://www.youtube.com/playlist?list=PLIUOU7oqGTLixb-CatMxNCO-mJioMmZEB

In this video we work through building an extractive Q&A stack using Haystack, and embedding it within a FastAPI instance in Python.

We use the BERT transformer for our reader model, alongside Elasticsearch and the BM25 retriever algorithm.

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: Sentence Similarity With Transformers and PyTorch (Python)
Category: Education
Published: 2021-05-05 15:00:20 UTC
Easy mode: https://youtu.be/Ey81KfQ3PQU

All we ever seem to talk about nowadays are BERT this, BERT that. I want to talk about something else, but BERT is just too good  -  so this video will be about BERT for sentence similarity.

A big part of NLP relies on similarity in highly-dimensional spaces. Typically an NLP solution will take some text, process it to create a big vector/array representing said text - then perform several transformations.

It's highly-dimensional magic.

Sentence similarity is one of the clearest examples of how powerful highly-dimensional magic can be.

The logic is this:
- Take a sentence, convert it into a vector.
- Take many other sentences, and convert them into vectors.
- Find sentences that have the smallest distance (Euclidean) or smallest angle (cosine similarity) between them - more on that here.
- We now have a measure of semantic similarity between sentences - easy!

At a high level, there's not much else to it. But of course, we want to understand what is happening in a little more detail and implement this in Python too.

Medium article:
https://towardsdatascience.com/bert-for-measuring-text-similarity-eec91c6bf9e1

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/bert-for-measuring-text-similarity-eec91c6bf9e1?sk=c0f2990b4660210b447e52d55bd0f4e5

👾 Discord
https://discord.gg/c5QtDB9RAP

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff

00:00 Intro
00:16 BERT Base Network
1:11 Sentence Vectors and Similarity
1:47 The Data and Model
3:01 Two Approaches
3:16 Tokenizing Sentences
9:11 Creating last_hidden_state Tensor
11:08 Creating Sentence Vectors
17:53 Cosine Similarity
Title: Sentence Similarity With Sentence-Transformers in Python
Category: Education
Published: 2021-05-05 15:00:09 UTC
🎁 Free NLP for Semantic Search Course:
https://www.pinecone.io/learn/nlp

Hard mode: https://youtu.be/jVPd7lEvjtg

All we ever seem to talk about nowadays are BERT this, BERT that. I want to talk about something else, but BERT is just too good  -  so this video will be about BERT for sentence similarity.

A big part of NLP relies on similarity in highly-dimensional spaces. Typically an NLP solution will take some text, process it to create a big vector/array representing said text - then perform several transformations.

It's highly-dimensional magic.

Sentence similarity is one of the clearest examples of how powerful highly-dimensional magic can be.

The logic is this:
- Take a sentence, convert it into a vector.
- Take many other sentences, and convert them into vectors.
- Find sentences that have the smallest distance (Euclidean) or smallest angle (cosine similarity) between them - more on that here.
- We now have a measure of semantic similarity between sentences - easy!

At a high level, there's not much else to it. But of course, we want to understand what is happening in a little more detail and implement this in Python too.

Medium article:
https://towardsdatascience.com/bert-for-measuring-text-similarity-eec91c6bf9e1

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/bert-for-measuring-text-similarity-eec91c6bf9e1?sk=c0f2990b4660210b447e52d55bd0f4e5

👾 Discord
https://discord.gg/c5QtDB9RAP

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: NER With Transformers and spaCy (Python)
Category: Education
Published: 2021-05-11 15:00:28 UTC
Named entity recognition (NER) consists of extracting 'entities' from text - what we mean by that is given the sentence:

"Apple reached an all-time high stock price of 143 dollars this January."

We might want to extract the key pieces of information - or 'entities' - and categorize each of those entities. Like so:

- Apple  : Organization
- 143 dollars :  Monetary Value
- this January :  Date

For us humans, this is easy. But how can we teach a machine to distinguish between a granny smith apple and the Apple we trade on NASDAQ?

(No, we can't rely on the 'A' being capitalized…)

This is where NER comes in - using NER, we can extract keywords like apple and identify that it is, in fact, an organization - not a fruit.

The go-to library for NER is spaCy, which is incredible. But what if we added transformers to spaCy? Even better - we'll cover exactly that in this video.
Title: Training BERT #1 - Masked-Language Modeling (MLM)
Category: Education
Published: 2021-05-19 14:51:39 UTC
🎁 Free NLP for Semantic Search Course:
https://www.pinecone.io/learn/nlp

BERT, everyone's favorite transformer costs Google ~$7K to train (and who knows how much in R&D costs). From there, we write a couple of lines of code to use the same model - all for free.

BERT has enjoyed unparalleled success in NLP thanks to two unique training approaches, masked-language modeling (MLM), and next sentence prediction (NSP).

MLM consists of giving BERT a sentence and optimizing the weights inside BERT to output the same sentence on the other side.

So we input a sentence and ask that BERT outputs the same sentence.

However, before we actually give BERT that input sentence - we mask a few tokens.

So we're actually inputting an incomplete sentence and asking BERT to complete it for us.

How to train BERT with MLM:
https://youtu.be/R6hcxMMOrPE

Medium article:
https://towardsdatascience.com/masked-language-modelling-with-bert-7d49793e5d2c

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/masked-language-modelling-with-bert-7d49793e5d2c?sk=17a19eca8dc8280bea4138802580ffe0

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: Training BERT #2 - Train With Masked-Language Modeling (MLM)
Category: Education
Published: 2021-05-19 14:51:49 UTC
🎁 Free NLP for Semantic Search Course:
https://www.pinecone.io/learn/nlp

BERT has enjoyed unparalleled success in NLP thanks to two unique training approaches, masked-language modeling (MLM), and next sentence prediction (NSP).

In many cases, we might be able to take the pre-trained BERT model out-of-the-box and apply it successfully to our own language tasks.

But often, we might need to pre-train the model for a specific use case even further.

Further training with MLM allows us to tune BERT to better understand the particular use of language in a more specific domain.

Out-of-the-box BERT - great for general purpose use. Fine-tuned with MLM BERT - great for domain-specific use.

In this video, we'll cover exactly how to fine-tune BERT models using MLM in PyTorch.

👾 Code:
https://github.com/jamescalam/transformers/blob/main/course/training/03_mlm_training.ipynb

Meditations data:
https://github.com/jamescalam/transformers/blob/main/data/text/meditations/clean.txt

Understanding MLM:
https://youtu.be/q9NS5WpfkrU

📙 Medium article:
https://towardsdatascience.com/masked-language-modelling-with-bert-7d49793e5d2c

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/masked-language-modelling-with-bert-7d49793e5d2c?sk=17a19eca8dc8280bea4138802580ffe0

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: Training BERT #3 - Next Sentence Prediction (NSP)
Category: Education
Published: 2021-05-25 14:56:47 UTC
Next sentence prediction (NSP) is one-half of the training process behind the BERT model (the other being masked-language modeling - MLM).

Where MLM teaches BERT to understand relationships between words - NSP teaches BERT to understand relationships between sentences.

In the original BERT paper, it was found that without NSP, BERT performed worse on every single metric -  so it's important.

Now, when we use a pre-trained BERT model, training with NSP and MLM has already been done, so why do we need to know about it?

Well, we can actually further pre-train these pre-trained BERT models so that they better understand the language used in our specific use-cases. To do that, we can use both MLM and NSP.

So, in this video, we'll go into depth on what NSP is, how it works, and how we can implement it in code.

Training with NSP:
https://youtu.be/x1lAcT3xl5M

📙 Medium article:
https://towardsdatascience.com/bert-for-next-sentence-prediction-466b67f8226f

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/bert-for-next-sentence-prediction-466b67f8226f?sk=3595968413abde1c5833e1a96e449673

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: Training BERT #4 - Train With Next Sentence Prediction (NSP)
Category: Education
Published: 2021-05-27 16:15:39 UTC
Next sentence prediction (NSP) is one-half of the training process behind the BERT model (the other being masked-language modeling - MLM).

Although NSP (and MLM) are used to pre-train BERT models, we can use these exact methods to further pre-train our models to better understand the specific style of language in our own use cases.

So, in this video, we'll cover exactly how we take an unstructured body of text, and use it to pre-train a BERT model using NSP.

Meditations data:
https://github.com/jamescalam/transformers/blob/main/data/text/meditations/clean.txt

Jupyter Notebook
https://github.com/jamescalam/transformers/blob/main/course/training/06_nsp_training.ipynb

📙 Medium article:
https://towardsdatascience.com/bert-for-next-sentence-prediction-466b67f8226f

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/bert-for-next-sentence-prediction-466b67f8226f?sk=3595968413abde1c5833e1a96e449673

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: New Features in Python 3.10
Category: Education
Published: 2021-06-08 15:00:02 UTC
The Python 3.10 release has several new features like structural pattern matching, a new typing Union operator, and parenthesized context managers!

Python 3.10 has now been released, here we test all of the best new features introduced.

We'll cover some of the most interesting additions to Python - structural pattern matching, parenthesized context managers, more typing, and the new and improved error messages.

Download the latest release:
https://www.python.org/downloads/release/python-3100/

📙 Medium article:
https://towardsdatascience.com/whats-new-in-python-3-10-a757c6c69342

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/whats-new-in-python-3-10-a757c6c69342?sk=648ae12c1025a83affba4eecec0d46c6

👾 Discord
https://discord.gg/c5QtDB9RAP

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff

00:00 Intro
00:45 Type Annotations in Python
01:10  Typing Union Operator
02:07 Parenthesized Context Managers
05:07 Structural Pattern Matching
09:31 Better Error Messages
Title: Training BERT #5 - Training With BertForPretraining
Category: Education
Published: 2021-06-15 15:00:19 UTC
NSP Logic
https://youtu.be/1gN1snKBLP0

MLM Logic
https://youtu.be/q9NS5WpfkrU

📙 Medium article:
https://towardsdatascience.com/how-to-train-bert-aaad00533168

📖 Here's a free link:
https://towardsdatascience.com/how-to-train-bert-aaad00533168?sk=5ad4e5e44a6c573b3be1967c9abdcc35

👾 Discord
https://discord.gg/c5QtDB9RAP

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: FREE 11 Hour NLP Transformers Course (Next 3 Days Only)
Category: Education
Published: 2021-06-04 13:00:19 UTC
The offer has now expired! You can find the final 70% discount here:
https://bit.ly/3DFvvY5

In total, 10823 people redeemed the code - which is incredible, I'm very happy so many of you were interested in the course and I hope it will help many of you in learning about transformers and NLP where it may have been too expensive to otherwise - so thank you all!

👾 Discord
https://discord.gg/c5QtDB9RAP

🕹️ Free AI-Powered Code Refactoring with Sourcery:
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: How-to Use HuggingFace's Datasets - Transformers From Scratch #1
Category: Education
Published: 2021-06-22 13:00:07 UTC
How can we build our own custom transformer models?

Maybe we'd like our model to understand a less common language, how many transformer models out there have been trained on Piemontese or the Nahuatl languages?

In that case, we need to do something different. We need to build our own model - from scratch.

In this video, we'll learn how to use HuggingFace's datasets library to download multilingual data and prepare it for training our custom transformer tokenizer and model.

---
Part 2: https://youtu.be/JIeAB8vvBQo
Part 3: https://youtu.be/heTYbpr9mD8
Part 4: https://youtu.be/35Pdoyi6ZoQ

📙 Medium article:
https://towardsdatascience.com/transformers-from-scratch-creating-a-tokenizer-7d7418adb403

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/transformers-from-scratch-creating-a-tokenizer-7d7418adb403?sk=aea909609f41be43bdb2dbbd75a801f2

👾 Discord
https://discord.gg/c5QtDB9RAP

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: Build a Custom Transformer Tokenizer - Transformers From Scratch #2
Category: Education
Published: 2021-06-24 14:00:06 UTC
How can we build our own custom transformer models?

Maybe we'd like our model to understand a less common language, how many transformer models out there have been trained on Piemontese or the Nahuatl languages?

In that case, we need to do something different. We need to build our own model - from scratch.

In this video, we'll learn how to use HuggingFace's tokenizers library to build our own custom transformer tokenizer.

Part 1: https://youtu.be/GhGUZrcB-WM
---
Part 3: https://youtu.be/heTYbpr9mD8
Part 4: https://youtu.be/35Pdoyi6ZoQ

📙 Medium article:
https://towardsdatascience.com/transformers-from-scratch-creating-a-tokenizer-7d7418adb403

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/transformers-from-scratch-creating-a-tokenizer-7d7418adb403?sk=aea909609f41be43bdb2dbbd75a801f2

👾 Discord
https://discord.gg/c5QtDB9RAP

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff
Title: 3 Vector-based Methods for Similarity Search (TF-IDF, BM25, SBERT)
Category: Education
Published: 2021-06-29 13:00:23 UTC
Vector similarity search is one of the fastest-growing domains in AI and machine learning. At its core, it is the process of matching relevant pieces of information together.

Similarity search is a complex topic and there are countless techniques for building effective search engines.

In this video, we'll cover three vector-based approaches for comparing languages and identifying similar 'documents', covering both vector similarity search and semantic search:

- TF-IDF
- BM25
- Sentence-BERT

📰 Original article:
https://www.pinecone.io/learn/semantic-search/

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

Mining Massive Datasets Book (Similarity Search):
📚 https://amzn.to/3CC0zrc (3rd ed)
📚 https://amzn.to/3AtHSnV (1st ed, cheaper)

👾 Discord
https://discord.gg/c5QtDB9RAP

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff

00:00 Intro
01:37 TF-IDF
11:44 BM25
20:30 SBERT
Title: 3 Traditional Methods for Similarity Search (Jaccard, w-shingling, Levenshtein)
Category: Education
Published: 2021-06-29 12:00:04 UTC
Similarity search is one of the fastest-growing domains in AI and machine learning. At its core, it is the process of matching relevant pieces of information together.

Similarity search is a complex topic and there are countless techniques for building effective search engines.

In this video, we'll cover three traditional approaches for comparing languages and identifying similar 'documents':

- Jaccard Similarity
- w-shingling
- Levenshtein distance

📰 Original article:
https://www.pinecone.io/learn/semantic-search/

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

Mining Massive Datasets Book (Similarity Search):
📚 https://amzn.to/3CC0zrc (3rd ed)
📚 https://amzn.to/3AtHSnV (1st ed, cheaper)

👾 Discord
https://discord.gg/c5QtDB9RAP

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff

00:00 Intro
00:23 Jaccard Similarity
02:39 w-shingling
07:17 Levenshtein Distance