jamescalam/channel-metadata huggingface dataset:
Title: API Series #2 - Building an API with Flask in Python
Category: Science & Technology
Published: 2021-10-07 14:52:32 UTC
Next video - how to deploy to the cloud: https://youtu.be/3fsIcMgUOY8

How can we set up a way to communicate from one software instance to another? It sounds simple, and — to be completely honest — it is.

All we need is an API.

An API (Application Programming Interface) is a simple interface that defines the types of requests (demands/questions, etc.) that can be made, how they are made, and how they are processed.

In our case, we will be building an API that allows us to send a range of GET/POST/PUT/PATCH/DELETE requests (more on this later), to different endpoints, and return or modify data connected to our API.

We will be using the Flask framework to create our API and Insomnia to test it.

🕹️ Medium article:
https://towardsdatascience.com/the-right-way-to-build-an-api-with-python-cd08ab285f8f

Free article link: 
https://towardsdatascience.com/the-right-way-to-build-an-api-with-python-cd08ab285f8f?sk=6e2dda4c8b6012767114e12ff34b1464

Download Insomnia:
https://insomnia.rest/download
Title: Intro to Sentence Embeddings with Transformers
Category: Science & Technology
Published: 2021-10-20 17:06:20 UTC
Transformers have wholly rebuilt the landscape of natural language processing (NLP). Before transformers, we had okay translation and language classification thanks to recurrent neural nets (RNNs) — their language comprehension was limited and led to many minor mistakes, and coherence over larger chunks of text was practically impossible.

Since the introduction of the first transformer model in the 2017 paper ‘Attention is all you need’, NLP has moved from RNNs to models like BERT and GPT. These new models can answer questions, write articles (maybe GPT-3 wrote this), enable incredibly intuitive semantic search — and much more.

In this video, we will explore how these embeddings have been adapted and applied to a range of semantic similarity applications by using a new breed of transformers called ‘sentence transformers’.

🌲 Pinecone article:
https://www.pinecone.io/learn/sentence-embeddings/

Vectors in ML:
https://www.youtube.com/playlist?list=PLIUOU7oqGTLgz-BI8bNMVGwQxIMuQddJO
Title: Fine-tune Sentence Transformers the OG Way (with NLI Softmax loss)
Category: Science & Technology
Published: 2021-10-22 14:39:46 UTC
Sentence embeddings with transformers can be used across a range of applications, such as semantic textual similarity (STS), semantic clustering, or information retrieval (IR) using concepts rather than words.

This video dives deeper into the training process of the first sentence transformer, sentence-BERT, or more commonly known as SBERT. We will explore the Natural Language Inference (NLI) training approach of softmax loss to fine-tune models for producing sentence embeddings.

Be aware that softmax loss is no longer the preferred approach to training sentence transformers and has been superseded by other methods such as MSE margin and multiple negatives ranking loss. But we’re covering this training method as an important milestone in the development of ever-improving sentence embeddings.

🌲 Pinecone article:
https://www.pinecone.io/learn/train-sentence-transformers-softmax/

Check out the Sentence Transformers library:
https://github.com/UKPLab/sentence-transformers

Talk by Nils Reimers (one of the SBERT creators) on training:
https://www.youtube.com/watch?v=RHXZKUr8qOY

He does more NLP vids too:
https://www.youtube.com/channel/UC1zCuTrfpjT6Sv2kJk-JkvA

00:00 Intro
00:42 NLI Fine-tuning
01:44 Softmax Loss Training Overview
05:47 Preprocessing NLI Data
12:48 PyTorch Process
19:48 Using Sentence-Transformers
30:45 Results
35:49 Outro
Title: Fine-tune High Performance Sentence Transformers (with Multiple Negatives Ranking)
Category: Science & Technology
Published: 2021-10-26 13:00:22 UTC
Transformer-produced sentence embeddings have come a long way in a very short time. Starting with the slow but accurate similarity prediction of BERT cross-encoders, the world of sentence embeddings was ignited with the introduction of SBERT in 2019. Since then, many more sentence transformers have been introduced. These models quickly made the original SBERT obsolete.

How did these newer sentence transformers manage to outperform SBERT so quickly? The answer is multiple negatives ranking (MNR) loss.

This video will cover what MNR loss is, the data it requires, and how to implement it to fine-tune our own high-quality sentence transformers.

Implementation will cover two approaches. The first is more involved, and outlines the exact steps to fine-tune the model (we'll just run over it quickly). The second approach makes use of the sentence-transformers library’s excellent utilities for fine-tuning.

🌲 Pinecone article:
https://www.pinecone.io/learn/fine-tune-sentence-transformers-mnr/

Check out the Sentence Transformers library:
https://github.com/UKPLab/sentence-transformers

Talk by Nils Reimers (one of the SBERT creators) on training:
https://www.youtube.com/watch?v=RHXZKUr8qOY

He does more NLP vids too:
https://www.youtube.com/channel/UC1zCuTrfpjT6Sv2kJk-JkvA

00:00 Intro
01:02 NLI Training Data
02:56 Preprocessing
10:11 SBERT Finetuning Visuals
14:14 MNR Loss Visual
16:37 MNR in PyTorch
23:04 MNR in Sentence Transformers
34:20 Results
36:14 Outro
Title: Hybrid Search Walkthrough in Pinecone
Category: Science & Technology
Published: 2021-10-29 15:05:00 UTC
Pinecone offers a production-ready vector database for high performance and reliable *semantic search* at scale. But did you know Pinecone's semantic search can be paired with the more traditional keyword search?

Semantic search is a compelling technology allowing us to search using abstract concepts and *meaning* rather than relying on specific words. However, sometimes a simple keyword search can be just as valuable — especially if we know the exact wording of what we're searching for.

In this video, we will explore these features through a start-to-finish example of basic keyword search in Pinecone.

🌲 Check the docs:
https://www.pinecone.io/docs/examples/basic-hybrid-search/

🔑 Free API key:
https://app.pinecone.io

00:52 How Hybrid Search Works
01:25 Preprocessing
03:01 Creating Keywords
05:34 Creating an Index
06:50 Data Upsert
08:33 Query Setup
10:52 Keyword Search
12:31 OR Logic
14:49 AND Logic
15:10 Negation
17:04 Outro
Title: API Series #3 - How to Deploy Flask APIs to the Cloud (GCP)
Category: Science & Technology
Published: 2021-11-02 14:30:00 UTC
Building that first API is for many of us, a significant step towards creating impactful tools that may one day be used by many developers. But often those APIs don't make it out of our local machines.

Fortunately, it's incredibly easy to deploy APIs. Assuming you have no idea what you're doing right now - you will probably be deploying your first API in around ten minutes.

I'm not joking, it's super easy. Let's get started.

📕 Article:
https://towardsdatascience.com/how-to-deploy-a-flask-api-8d54dd8d8b8a

📖 Free article link:
TO ADD
Title: All You Need to Know on Multilingual Sentence Vectors (1 Model, 50+ Languages)
Category: Science & Technology
Published: 2021-11-04 13:00:10 UTC
We’ve learned about how sentence transformers can be used to create high-quality vector representations of text. We can then use these vectors to find similar vectors, which can be used for many applications such as semantic search or topic modeling.

These models are very good at producing meaningful, information-dense vectors. But they don’t allow us to compare sentences across different languages.

Often this may not be a problem. However, the world is becoming increasingly interconnected, and many companies span across multiple borders and languages. Naturally, there is a need for sentence vectors that are language agnostic.

Unfortunately, very few textual similarity datasets span multiple languages, particularly for less common languages. And the standard training methods used for sentence transformers would require these types of datasets.

Different approaches need to be used. Fortunately, some techniques allow us to extend models to other languages using more easily obtained language translations.

In this video, we will cover how multilingual models work and are built. We’ll learn how to develop our own multilingual sentence transformers, the datasets to look for, and how to use high-performing pretrained multilingual models.

🌲 Pinecone article:
https://www.pinecone.io/learn/multilingual-transformers/

00:00 Intro
01:19 Multilingual Vectors
05:55 Multi-task Training (mUSE)
09:36 Multilingual Knowledge Distillation
11:13 Knowledge Distillation Training
13:43 Visual Walkthrough
14:53 Parallel Data Prep
20:23 Choosing a Student Model
24:55 Initializing the Models
30:05 ParallelSentencesDataset
33:54 Loss and Fine-tuning
36:59 Model Evaluation
39:23 Outro
Title: Question-Answering in NLP (Extractive QA and Abstractive QA)
Category: Science & Technology
Published: 2021-11-16 12:06:13 UTC
Search is a crucial functionality in many applications and companies globally. Whether in manufacturing, finance, healthcare, or *almost* any other industry, organizations have vast internal information and document repositories.

Unfortunately, the scale of many companies’ data means that the organization and accessibility of information can become incredibly inefficient. The problem is exacerbated for language-based information. Language is a tool for people to communicate often abstract ideas and concepts. Naturally, ideas and concepts are harder for a computer to comprehend and store in a meaningful way.

How do we minimize this problem? The answer lies with *semantic search*, specifically with the question-answering (QA) flavor of semantic search.

This article will introduce the different forms of QA, the components of these 'QA stacks', and where we might use them.

🌲 Pinecone article:
https://www.pinecone.io/learn/question-answering/

00:00 Meaningful Search
01:23 Use-case
02:22 Open Domain QA (ODQA)
06:41 SQuAD Format
10:45 Quick Preprocessing
15:18 Creating Context Vectors Database
23:24 Open-book Extractive QA
32:50 Open-book Abstractive QA
41:53 Closed-book Abstractive QA
47:27 Final Thoughts
Title: Today Unsupervised Sentence Transformers, Tomorrow Skynet (how TSDAE works)
Category: Science & Technology
Published: 2021-11-24 16:24:24 UTC
To adapt a pretrained transformer to produce meaningful sentence vectors, we typically need a more supervised fine-tuning approach. We can use datasets like natural language inference (NLI) pairs, labeled semantic textual similarity (STS) data, or parallel data (pairs of translations).

For some domains and languages, such as finance and English, this data is fairly easy to find or gather. But many domains and many languages have very little labeled data. If you can find semantic similarity pairs for the agriculture industry, please let me know. There are many languages, such as Dhivehi, where unlabelled data is hard to find and labelled data practically non-existent.

This means you either spend a very long time gathering tens of thousands of labeled samples or you can try an unsupervised fine-tuning approach.

Unsupervised training methods for sentence transformers are not as effective as their supervised counterparts, but they do work. And if you have no other choice, why not?

In this video, we will introduce the concept of unsupervised fine-tuning for sentence transformers. We will learn to train these models using the unsupervised Transformer-based Sequential Denoising Auto-Encoder (TSDAE) approach.

🌲 Pinecone article:
https://www.pinecone.io/learn/unsupervised-training-sentence-transformers/

00:00 Why Language Embedding Matters
05:12 Supervised Methods
05:29 Natural Language Inference
07:15 Semantic Textual Similarity
07:43 Multilingual Training
10:00 TSDAE (Unsupervised)
18:50 Data Preparation
29:05 Initialize Model
32:39 Model Training
36:25 NLTK Error
37:15 Evaluation
41:01 TSDAE vs Supervised Methods
42:42 Why TSDAE is Cool
Title: Making The Most of Data: Augmented SBERT
Category: Science & Technology
Published: 2021-12-17 14:24:40 UTC
🎁 Free NLP for Semantic Search Course:
https://www.pinecone.io/learn/nlp

ML models are data-hungry. They consume massive amounts of data to identify generalized patterns and apply those learned patterns to new data.

As models get bigger, so do datasets. And although we have seen an explosion of data in the past decade, it is often not accessible or in an ML-friendly format, especially in niche domains.

For many niche, low-resource domains, finding or annotating a substantial dataset manually is practically impossible.

Fortunately, we don't need to label (or even find) this new data. Instead, we can automatically generate or label data using one or more *data augmentation* techniques.

In this video, we will introduce data augmentation and its application to the field of NLP. We will focus on the 'in-domain' flavor of a particular data-augmentation strategy named augmented SBERT (AugSBERT).

🌲 Pinecone article:
https://www.pinecone.io/learn/data-augmentation/
Title: Building Transformer Tokenizers (Dhivehi NLP #1)
Category: Science & Technology
Published: 2021-12-28 15:45:03 UTC
🎁 Free NLP for Semantic Search Course:
https://www.pinecone.io/learn/nlp

Get in touch with Ashraq:
https://www.linkedin.com/in/ismailashraq/

The language of Dhivehi (or Maldivian) is fascinating. It uses a complex writing system known as Thaana, and I absolutely cannot comprehend any of it. It is so wildly different from anything I know - but, like the archipelago, it looks wonderful.

Ashraq described the difficulty of applying NLP to his native tongue of Dhivehi. There are several reasons for this, which we will explore in this video, and learn how to build an effective Dhivehi WordPiece tokenizer.

📕 Article:
https://towardsdatascience.com/designing-tokenizers-for-low-resource-languages-7faa4ab30ef4

📖 Article Friend Link (Free Access):
https://towardsdatascience.com/designing-tokenizers-for-low-resource-languages-7faa4ab30ef4?sk=c0c16de9eea7dbe1d2a9c106abf38e1a

00:00 Intro
01:06 Dhivehi Project
02:28 Hurdles for Low Resource Domains
04:21 Dhivehi Dataset
04:52 Download Dhivehi Corpus
08:25 Tokenizer Components
08:44 Normalizer Component
11:55 Pre-tokenization Component
14:59 Post-tokenization Component
16:26 Decoder Component
17:41 Tokenizer Implementation
21:04 Tokenizer Training
24:22 Post-processing Implementation
27:12 Decoder Implementation
28:07 Saving for Transformers
30:33 Tokenizer Test and Usage
31:36 Download Dhivehi Models
32:21 First Steps
Title: AugSBERT: Domain Transfer for Sentence Transformers
Category: Science & Technology
Published: 2022-01-04 14:59:50 UTC
🎁 Free NLP for Semantic Search Course:
https://www.pinecone.io/learn/nlp

When building language models, we can spend months optimizing training and model parameters, but it’s useless if we don't have the correct data.

The success of our language models relies first and foremost on data. The augmented SBERT training strategy can help us.

Given this scenario, we can transfer information from an out-of-domain (or *source*) dataset to our target domain. We will learn how to do this here. First, we will learn to assess which source datasets align best with our target domain quickly. Then we will explain and work through the AugSBERT domain-transfer training strategy.

🌲 Pinecone article:
https://www.pinecone.io/learn/augsbert-domain-transfer/

🔗 n-gram Similarity Script: https://gist.github.com/jamescalam/b73f37017ae32bd6094747c4b0fca94a
🔗 AugSBERT In-Domain Article: https://www.pinecone.io/learn/data-augmentation/

00:00 Why Use Domain Transfer
04:08 Strategy Outline
06:05 Train Source Cross-Encoder
12:44 Cross-Encoder Outcome
15:12 Labeling Target Data
20:31 Training Bi-encoder
23:58 Evaluator Bi-encoder Performance
28:08 Final Points
Title: How to build a Q&A AI in Python (Open-domain Question-Answering)
Category: Science & Technology
Published: 2022-01-11 14:00:20 UTC
🎁 Free NLP for Semantic Search Course:
https://www.pinecone.io/learn/nlp

How can we design these natural, human-like Q&A interfaces? The answer is open-domain question-answering (ODQA). ODQA allows us to use natural language to query a database.

That means that, given a dataset like a set of internal company documents, online documentation, or as is the case with Google, everything on the world’s internet, we can retrieve relevant information in a natural, more human way.

🌲 Pinecone article:
https://www.pinecone.io/learn/retriever-models/

🔗 Nils YT Talk: https://youtu.be/XNJThigyvos?t=118
🔗 MNR Loss Article: 
🔗 Free Pinecone API Key: https://app.pinecone.io/

00:00 Why QA
04:05 Open Domain QA
08:24 Do we need to fine-tune?
11:44 How Retriever Training Works
12:59 SQuAD Training Data
16:29 Retriever Fine-tuning
19:32 IR Evaluation
25:58 Vector Database Setup
33:42 Querying
37:41 Final Notes
Title: How to build a Q&A Reader Model in Python (Open-domain QA)
Category: Science & Technology
Published: 2022-01-18 16:37:37 UTC
🎁 Free NLP for Semantic Search Course:
https://www.pinecone.io/learn/nlp

Open-domain question-answering (ODQA) is a wildly popular *pipeline* of databases and language models that allow us to ask a machine human-like questions and return comprehensible and even intelligent answers.

Despite the outward guise of simplicity, ODQA requires a reasonably advanced set of components placed together to enable the *extractive* Q&A functionality.

We call this *extractive* Q&A because the models are not generating an answer. Instead, the answer already exists but is hidden somewhere within potentially thousands, millions, or even more data sources.

By enabling extractive Q&A, we enable a more *intelligent* and *efficient* way to retrieve information from what can be massive stores of data.

🌲 Pinecone article:
https://www.pinecone.io/learn/reader-models/

00:00 Intro
00:13 ODQA Components
03:09 Data Preprocessing
22:35 Fine-tuning
Title: Streamlit for ML #1 - Installation and API
Category: Science & Technology
Published: 2022-01-25 16:00:09 UTC
▶️ Streamlit for ML Part 2:
https://www.youtube.com/watch?v=U0EoaFFGyTg&list=PLIUOU7oqGTLg5ssYxPGWaci6695wtosGw&index=2

Streamlit has proven itself as an incredibly popular tool for quickly putting together high-quality ML-oriented web apps. More recently, it has seen wider adoption in production environments by ever-larger organizations.

All of this means that there is no better time to pick up some experience with Streamlit. Fortunately, the basics of Streamlit are incredibly easy to learn, and for most tools, this will be more than you need!

In this series, we will introduce Streamlit by building a general knowledge Q&A interface. We will learn about key Streamlit components like write, text_input, container. How to use external libraries like Bootstrap to quickly create new app components. And use caching to speed up our app.

📕 Article:
https://towardsdatascience.com/getting-started-with-streamlit-for-nlp-75fe463821ec

📖 Friend link to article:
https://towardsdatascience.com/getting-started-with-streamlit-for-nlp-75fe463821ec?sk=ac5e0b7c39938f52162862411a66a58b

00:00 Intro
00:39 App Outline
03:36 Streamlit Installation
06:15 Streamlit API Basics
Title: Streamlit for ML #2 - ML Models and APIs
Category: Science & Technology
Published: 2022-01-26 16:30:36 UTC
▶️ Streamlit for ML Part 3:
https://www.youtube.com/watch?v=lYDiSCDcxmc&list=PLIUOU7oqGTLg5ssYxPGWaci6695wtosGw&index=3

Streamlit has proven itself as an incredibly popular tool for quickly putting together high-quality ML-oriented web apps. More recently, it has seen wider adoption in production environments by ever-larger organizations.

All of this means that there is no better time to pick up some experience with Streamlit. Fortunately, the basics of Streamlit are incredibly easy to learn, and for most tools, this will be more than you need!

In this series, we will introduce Streamlit by building a general knowledge Q&A interface. We will learn about key Streamlit components like write, text_input, container. How to use external libraries like Bootstrap to quickly create new app components. And use caching to speed up our app.

🔗 Code to Create Index:
https://gist.github.com/jamescalam/2123ce0bb8a871f48a151a023a7ece67

📕 Article:
https://towardsdatascience.com/getting-started-with-streamlit-for-nlp-75fe463821ec

📖 Friend link to article:
https://towardsdatascience.com/getting-started-with-streamlit-for-nlp-75fe463821ec?sk=ac5e0b7c39938f52162862411a66a58b

00:00 Intro
00:47 Creating the Vector DB
08:56 Implementing Retrieval
Title: Streamlit for ML #3 - Make Apps Fast with Caching
Category: Science & Technology
Published: 2022-01-27 15:00:36 UTC
▶️ Streamlit for ML Part 4:
https://www.youtube.com/watch?v=XdxeKiY2UXg&list=PLIUOU7oqGTLg5ssYxPGWaci6695wtosGw&index=4

Streamlit has proven itself as an incredibly popular tool for quickly putting together high-quality ML-oriented web apps. More recently, it has seen wider adoption in production environments by ever-larger organizations.

All of this means that there is no better time to pick up some experience with Streamlit. Fortunately, the basics of Streamlit are incredibly easy to learn, and for most tools, this will be more than you need!

In this series, we will introduce Streamlit by building a general knowledge Q&A interface. We will learn about key Streamlit components like write, text_input, container. How to use external libraries like Bootstrap to quickly create new app components. And use caching to speed up our app.

▶️ Streamlit for ML Playlist:
https://www.youtube.com/watch?v=JLKUV-LiXjk&list=PLIUOU7oqGTLg5ssYxPGWaci6695wtosGw&index=1

📕 Article:
https://towardsdatascience.com/getting-started-with-streamlit-for-nlp-75fe463821ec

📖 Friend link to article:
https://towardsdatascience.com/getting-started-with-streamlit-for-nlp-75fe463821ec?sk=ac5e0b7c39938f52162862411a66a58b

00:00 Intro
02:35 Streamlit Caching
06:56 Experimental Caching Primitives
Title: Streamlit for ML #4 - Adding Bootstrap Components
Category: Science & Technology
Published: 2022-01-28 15:11:42 UTC
▶️ Streamlit for ML Part 5.1:
https://www.youtube.com/watch?v=SGazDb8o-to&list=PLIUOU7oqGTLg5ssYxPGWaci6695wtosGw&index=5

Streamlit has proven itself as an incredibly popular tool for quickly putting together high-quality ML-oriented web apps. More recently, it has seen wider adoption in production environments by ever-larger organizations.

All of this means that there is no better time to pick up some experience with Streamlit. Fortunately, the basics of Streamlit are incredibly easy to learn, and for most tools, this will be more than you need!

In this series, we will introduce Streamlit by building a general knowledge Q&A interface. We will learn about key Streamlit components like write, text_input, container. How to use external libraries like Bootstrap to quickly create new app components. And use caching to speed up our app.

▶️ Streamlit for ML Playlist:
https://www.youtube.com/watch?v=JLKUV-LiXjk&list=PLIUOU7oqGTLg5ssYxPGWaci6695wtosGw&index=1

📕 Article:
https://towardsdatascience.com/getting-started-with-streamlit-for-nlp-75fe463821ec

📖 Friend link to article:
https://towardsdatascience.com/getting-started-with-streamlit-for-nlp-75fe463821ec?sk=ac5e0b7c39938f52162862411a66a58b

00:00 Intro
02:35 Streamlit Caching
06:56 Experimental Caching Primitives
Title: Adding New Doc Stores to Haystack
Category: Science & Technology
Published: 2022-03-15 15:00:14 UTC
🥳  Released with Haystack v1.3! Install direct from PyPI with:

pip install 'farm-haystack[pinecone]'

PR:
https://github.com/deepset-ai/haystack/pull/2254

00:00 Intro
02:15 Contributing or Testing
03:31 ODQA
06:20 What is Haystack?
08:13 Haystack QA Workflow
14:52 Contributing to Open Source
22:54 Haystack Doc Stores
26:09 Doc Store Core Methods
29:31 Final Notes, Contribute/Test