BERTopic Explained

of particular areas in our data by searching through for each point in our data set.


BERTopic Explained

Searching through and finding the kNearestNeighbours.


BERTopic Explained

And that is what this nNeighbours parameter is setting.


BERTopic Explained

It's saying, okay, how many of the nearest neighbors are we going to have a look at?


BERTopic Explained

And as you can imagine, if you set an nNeighbours parameter to be very small,


BERTopic Explained

it is only really going to pick up on the local structure of your data.


BERTopic Explained

Whereas if you set it to be very large,


BERTopic Explained

it's going to focus more on the global structure.


BERTopic Explained

So what we can do is try and find something in the middle


BERTopic Explained

where we're focusing on both the global and local structure.


BERTopic Explained

And this can work incredibly well.


BERTopic Explained

So if we take a look at our reduced data set,


BERTopic Explained

we can see straight away that the continents are in the right places.


BERTopic Explained

They've kind of been switched over there.


BERTopic Explained

So we have north is in the south and south is in north.


BERTopic Explained

And in Oceania, we have a few islands that seem to...


BERTopic Explained

So New Zealand is over here rather than over here where I suppose it should be.


BERTopic Explained

But otherwise, it seems to have done relatively well.


BERTopic Explained

Now, a lot of these...


BERTopic Explained

So we have Japan over here.


BERTopic Explained

We have the Maldives and the Philippines.


BERTopic Explained

A lot of these kind of bits that go off of the main clusters are actually islands,


BERTopic Explained

which makes sense.


BERTopic Explained

They are not as close to the continents or anything.


BERTopic Explained

So it kind of makes sense that they are not within the same cluster.


BERTopic Explained

And we can even see to an extent the shape of some countries.


BERTopic Explained

So over here, as soon as I saw this sort of yellow bit sticking out on the left of Europe,


BERTopic Explained

I thought that looks a lot like the Spanish Peninsula.


BERTopic Explained

And if we go over there, we actually see that it is all these cities over here are Spain.


BERTopic Explained

And then we also have Europe, Portugal as well.


BERTopic Explained

Next to Spain over here, we have France.


BERTopic Explained

And you have the local structure of countries.


BERTopic Explained

It does seem to be preserved quite well.


BERTopic Explained

So that seems to work really well already.


BERTopic Explained

Now, there was just one other parameter that I added in order to get this sort of graph.


BERTopic Explained

And that was the minute-this parameter.


BERTopic Explained

Now, what this does is limits how closely UMAP can put two points together.


BERTopic Explained

So by default, this is, I believe, 0.1.


BERTopic Explained

And what I did was increase this up to 0.5 because with 0.1,


BERTopic Explained

a lot of these links tend to just be really tightly stringed together


BERTopic Explained

and you almost get like strings of countries.


BERTopic Explained

But increasing that, it kind of pushed the structure out a little bit to be less stringy


BERTopic Explained

and more kind of like continents.


BERTopic Explained

So that's another really cool parameter that you can sort of fine-tune


BERTopic Explained

and hopefully get some good results with.


BERTopic Explained

Now, let's take a look at the code that we apply to our Reddit Topics data.


BERTopic Explained

So come down here.


BERTopic Explained

And what I'm doing is reset.


BERTopic Explained

Or what I was doing here is taking a look at the different nearest neighbors values


BERTopic Explained

and seeing what seems to work well.


BERTopic Explained

And in the end, what I settled on is first compressing our data into a 3D space rather than 2D,


BERTopic Explained

which makes sense.


BERTopic Explained

It means we have more dimensionality to maintain more information.


BERTopic Explained

So nearest neighbors or n neighbors, sorry, to 3, number of components to 3,


BERTopic Explained

that's received the 3D aspect.


BERTopic Explained

You set that to 2 if you want to 2D.


BERTopic Explained

And then I also added in min-dis.


BERTopic Explained

But rather than increasing min-dis from 0.1, I actually decrease it so the points will be closer together.


BERTopic Explained

And what we got out from that is this.


BERTopic Explained

And you can see here we have these four clusters.


BERTopic Explained

And we have color over here.


BERTopic Explained

It's my bad.


BERTopic Explained

But essentially what we have is, I think, so red is language technology.


BERTopic Explained

Purple is investing.


BERTopic Explained

Blue is PyTorch.


BERTopic Explained

And green is Python.


BERTopic Explained

So we have all of these of language technology, PyTorch and Python all kind of together


BERTopic Explained

because they're more similar than investing, which is really far away.


BERTopic Explained

But we can clearly see that these look like pretty good clusters.


BERTopic Explained

There's some overlap.


BERTopic Explained

And that's to be expected.


BERTopic Explained

There are a lot of topics or threads that could be in both PyTorch and Python


BERTopic Explained

and language technology all at once.


BERTopic Explained

So there is, of course, some overlap.


BERTopic Explained

But for the most part, we can see some pretty clear clusters there.


BERTopic Explained

So that tells us, okay, that's great.


BERTopic Explained

Now we can move on to the next step, which is clustering with HDBSCAN.


BERTopic Explained

Now, what we've seen here, we have like color-coded clusters.


BERTopic Explained

But in reality, we're assuming that we don't actually have the subreddit titles already.


BERTopic Explained

We want to be trying to extract those clusters without knowing what the clusters already are.


BERTopic Explained

So we're going to do that.


BERTopic Explained

All we start with is actually a pip install of HDBSCAN.


BERTopic Explained

Super easy.


BERTopic Explained

And then we can import HDBSCAN.


BERTopic Explained

And we fit to our three-dimensional reddit topics data.


BERTopic Explained

And then we can use this condensed tree plot to view how our clusters are being built.


BERTopic Explained

But what we see at the moment on there is all of these red lines.


BERTopic Explained

Now, these are not actually lines.


BERTopic Explained

They're circles identifying the clusters that have been extracted.


BERTopic Explained

Now, obviously, it seems like there's a lot of clusters that have been extracted.


BERTopic Explained

We don't want to do that.


BERTopic Explained

So what we can do is make the criteria for selecting a cluster more strict.


BERTopic Explained

So at the moment, the number of points needed to create a cluster,


BERTopic Explained

I believe by default is something like five, which is very small when we have just over 3,000 vectors


BERTopic Explained

that we want to cluster into just four topics.


BERTopic Explained

So we can increase the minimum cluster size.


BERTopic Explained

And if we go with something like 80 to start with, we get this.


BERTopic Explained

So now we can actually see the condensed tree plot.


BERTopic Explained

And basically, it's moving through the lambda value here,


BERTopic Explained

which is just the parameters that HDBSCAN is using.


BERTopic Explained

And we can imagine this as a line starting from the bottom and going up.


BERTopic Explained

And as the line moves up, we are identifying the strongest, almost clusters within our data.


BERTopic Explained

Now, at the moment, we're just returning two.


BERTopic Explained

And realistically, we want to try and return like four clusters.


BERTopic Explained

And to me, it probably seems like we'd want this one on the left,


BERTopic Explained

but then we'd want this one, this one, and this one for our clusters.


BERTopic Explained

We're pulling in this big green blob instead,


BERTopic Explained

because it's being viewed algorithmically as a stronger cluster.


BERTopic Explained

So maybe we can try and reduce maybe our min cluster size.


BERTopic Explained

We say it to be too high.


BERTopic Explained

We can reduce that, just reducing by 20.


BERTopic Explained

And then we pull in this really long one on the left over here.


BERTopic Explained

So that also does not look very good.


BERTopic Explained

So clearly the issue is not the min cluster size.


BERTopic Explained

And maybe we need to change something else.


BERTopic Explained

So what we can do is we'll keep min cluster size at 80


BERTopic Explained

and we'll instead reduce the min samples,


BERTopic Explained

which by default is set to the same as the min cluster size.


BERTopic Explained

Now, min samples is basically how dense the core of a cluster needs to be


BERTopic Explained

in order to become a core of a cluster.


BERTopic Explained

So we reduce that, and if we reduce that, then we get the four clusters.


BERTopic Explained

That to me looks like we probably want.


BERTopic Explained

So we have these four.


BERTopic Explained

I imagine these three over here, which are connected for longer up here,


BERTopic Explained

are probably Python, PyTorch, and language technology.


BERTopic Explained

And then over here we probably have investing.


BERTopic Explained

So let's have a look at what we have.


BERTopic Explained

Okay, cool. So you see that it looks like we have similar clusters


BERTopic Explained

to what we got before, right?


BERTopic Explained

When we were pulling the colors in from the data.


BERTopic Explained

And if we have a look here, yes, green investing,


BERTopic Explained

red language technology, purple Python, and blue is PyTorch.


BERTopic Explained

And then we also have these orange points.


BERTopic Explained

Now, these are the outliers.


BERTopic Explained

I think this doesn't belong in any of these clusters.


BERTopic Explained

And if we have a look at these, come over to here,


BERTopic Explained

we see things like daily general discussion and advice thread.


BERTopic Explained

Okay. So to an extent, it kind of makes sense.


BERTopic Explained

Like this isn't really necessarily about investing.


BERTopic Explained

It's just like the general discussion.


BERTopic Explained

And then up here, we have some, maybe it would be better


BERTopic Explained

if these were included within the language technology cluster.


BERTopic Explained

And then over here, down in the bottom,


BERTopic Explained

we have similar to the investing bit,


BERTopic Explained

where we had the discussion thread.


BERTopic Explained

We have another discussion thread here, but from Python.


BERTopic Explained

So it's not perfect.


BERTopic Explained

There are some outliers that we would like to be in there,


BERTopic Explained

but quite a few of those outliers kind of make sense


BERTopic Explained

that they're not in there as a discussion threads


BERTopic Explained

or something else.


BERTopic Explained

So it's kind of, it's pretty cool to see that it can actually do that so well.


BERTopic Explained

Otherwise, I think that clustering looks pretty good.


BERTopic Explained

So let's move on to the next step of CTF IDF.


BERTopic Explained

Now we'll go through CTF IDF very quickly,


BERTopic Explained

but all this is really doing is it's looking at the frequency of words


BERTopic Explained

within a particular class or within a particular topic


BERTopic Explained

that we've built and saying, okay, are those words very common?


BERTopic Explained

So words like V, A would be very common


BERTopic Explained

and they would be scored lowly, thanks to the IDF part of the function,


BERTopic Explained

which is the inverse document frequency.


BERTopic Explained

Look at how common these words are.


BERTopic Explained

But if we have a rare word, something like Python,


BERTopic Explained

and it just keeps popping up in one of these topics all the time,


BERTopic Explained

then there's a good idea that maybe Python,


BERTopic Explained

the word Python is pretty relevant to that particular topic.


BERTopic Explained

Now I'm not going to go, not really going to go through the code on this.


BERTopic Explained

Just make it really quick.


BERTopic Explained

All I'm going to do is show you, we go through here,


BERTopic Explained

actually tokenizing using NLTK.


BERTopic Explained

Now with that topic, we don't really need to worry so much about this.


BERTopic Explained

It's going to be handled for us.


BERTopic Explained

So that's not so important.


BERTopic Explained

But we can see we create these tokens, which are the words


BERTopic Explained

and it's these tokens or terms that the CTF IDF algorithm


BERTopic Explained

is going to be looking at in terms of frequency


BERTopic Explained

and identifying the most relevant ones.


BERTopic Explained

So, for example, here we have PyTorch, model, gradients.


BERTopic Explained

They're probably pretty relevant to the PyTorch or Python


BERTopic Explained

or even natural language topic, but less so for investing.


BERTopic Explained

So come down here, we have this is a number of tokens


BERTopic Explained

within each topic, including our not relevant or outlier topic.


BERTopic Explained

And if we scroll down, these are our TF IDF scores for basically


BERTopic Explained

for every single possible word or token within each of our topics.


BERTopic Explained

So you see we have five rows here and then within each of those rows,


BERTopic Explained

we have a sparsely populated matrix where each entry in that matrix


BERTopic Explained

represents one token or one term by the word the,


BERTopic Explained

although we removed the so that wouldn't be in there.


BERTopic Explained

So like the word Python or PyTorch or investing or money


BERTopic Explained

or something along those lines.


BERTopic Explained

And then I'm just going to return the top end most common words per class.


BERTopic Explained

We come down here and we see.


BERTopic Explained

OK, cool. So this one is our irrelevant topic.


BERTopic Explained

And then up here, I mean, we can see straight away from these what


BERTopic Explained

hopefully what topics they are.


BERTopic Explained

We have market, stock, it's probably investing.


BERTopic Explained

Up here, we have project, Python code.


BERTopic Explained

I'm pretty sure it's Python.


BERTopic Explained

Loss, X model, PyTorch, pretty confident that's PyTorch.


BERTopic Explained

And then here we have NLP and text data model.


BERTopic Explained

It's got to be a language technology.


BERTopic Explained

So high level, that's more in detail, but so kind of high level.


BERTopic Explained

That's how it works.


BERTopic Explained

Now, how do we apply what we've just gone through to the topic library?


BERTopic Explained

Fortunately, it's incredibly easy.


BERTopic Explained

So again, I'm starting another notebook here.


BERTopic Explained

This is number six.


BERTopic Explained

Same again, download data, remove anything that's particularly long.


BERTopic Explained

This one's going to be really short, by the way, so no problem.


BERTopic Explained

And then what we do is as before, we initialize our UMAP and HEB scan components,


BERTopic Explained

but we're just initializing them, not fitting or anything right now.


BERTopic Explained

I've initialized these with the same parameters.


BERTopic Explained

I just add the minimum spanning tree here for the HEB scan model,


BERTopic Explained

which is basically just going to build like a linked tree through the data.


BERTopic Explained

And I found that improves the performance, so added that in there.


BERTopic Explained

And another thing that I've added in is this prediction data equals true.


BERTopic Explained

Now, when you're using HEB scan with that topic, you need to include that.


BERTopic Explained

Otherwise, you will get this attribute error that I mentioned here.


BERTopic Explained

So attribute error, no prediction data is generated.


BERTopic Explained

You will get that error if you don't include prediction data equals truth


BERTopic Explained

for your HEB scan component.


BERTopic Explained

Then other things that we need is our sentence transform, the embedding model.


BERTopic Explained

And in the TF-IDF or CTF-IDF set, I didn't really go through it,


BERTopic Explained

but I removed stop words like the A and so on.


BERTopic Explained

So again, as we use all right to start,


BERTopic Explained

I'm going to include the count vectorizer to remove those stop words.


BERTopic Explained

And I added a few more as well.


BERTopic Explained

So I took the NLTK stop words here,


BERTopic Explained

and then I also added these other words that kept popping up all the time


BERTopic Explained

and kind of polluting the topics.


BERTopic Explained

And yeah, initialize the model.


BERTopic Explained

And then we can initialize BERT topic as we did before,


BERTopic Explained

but then we add all of these different components into the definition.


BERTopic Explained

So it's literally what we've just been through.


BERTopic Explained

We do the same thing.


BERTopic Explained

We initialize all those components,


BERTopic Explained

but we don't run them in their respective libraries.


BERTopic Explained

We actually just pass them to BERT topic library,


BERTopic Explained

which makes it, I think, a lot nicer to work with


BERTopic Explained

because you just throw everything, you initialize everything,


BERTopic Explained

and then just throw everything into BERT topic,


BERTopic Explained

and then BERT topic will run through it all for you, which is really cool.


BERTopic Explained

So as before, we do the fit transform, which is like fit and predict.


BERTopic Explained

We'll get topics and probabilities from that.


BERTopic Explained

And we come down here and we have our topic.


BERTopic Explained

So topic zero, it looks like investing.


BERTopic Explained

Number one, PyTorch, two, language technology, and three, Python.


BERTopic Explained

So that's really cool.


BERTopic Explained

It has the topics.


BERTopic Explained

And you can also see the hierarchy of those topics as well.


BERTopic Explained

So we have Python, PyTorch, and language technology


BERTopic Explained

kind of grouped together earlier than the investing topic.


BERTopic Explained

So I think that is incredibly cool,


BERTopic Explained

how easily we can organize data using meaning


BERTopic Explained

rather than using keywords or some sort of complex rule-based method.


BERTopic Explained

With BERT topic, we literally just feed in our data.


BERTopic Explained

Obviously, it helps to have an understanding of UMAP,


BERTopic Explained

HTTP scan, and the transformer embedding models.


BERTopic Explained

But using all of what we've just learned,


BERTopic Explained

we're able to do this, which is very impressive,


BERTopic Explained

just automatically cluster all of our data in a meaningful way.


BERTopic Explained

Okay, so that's it for this video.


BERTopic Explained

I hope BERT topic and all these other components of BERT topic


BERTopic Explained

are as interesting to you as they are to me.


BERTopic Explained

But I'll leave it there for now.


BERTopic Explained

So thank you very much for watching,


BERTopic Explained


Streamlit for ML #5.2 - MUI Card Component Build

Okay, in the last video, we put together the default Streamlit component or custom component


and cell-py environment for actually building our own.


So what we have at the moment is pretty simple.


We haven't really done anything.


We just have this custom component Streamlit and we can see the code for this component


as well if we have a look over here.


So this is just a default component.


We're not actually doing anything here.


We're just actually not in that file.


It's in the init.py here.


We're just running this using nothing more than this code here.


Okay, so we have this My Component object.


But that's all that you can see while we're clicking and everything else.


So this here is Hello James, click me.


That's Hello World, click me.


That's all it is, it's just a little object.


So what we want to do is actually create a card component based on Material UI, which


is what you see here, like this soft card.


We're not going to make it that fancy, although it's really not difficult.


If you want to take it up further, you definitely can.


What we're going to do is we're going to take like a title, a subtitle.


We're going to include the body of text like this.


And we also want to include like a button, not like, well, kind of similar to this way


we have like this circle when we hover over it.


And if we click it, it does this nice little ripple effect.


So let's start putting that part of our component together, the actual code that creates this


card.


So most of this is going to come from this card component.tsx file.


Now in here at the moment, there's a lot of code, like far too much.


We don't need all of this.


We can delete a lot of it.


So let's do that first before we do anything else.


First I'm going to change this to any.


We're going to remove all of this.


We don't need the state variable.


We also don't need this name variable as well, but I'm going to keep it in here for now.


This is how we pass arguments to our card.


So we're going to title, subtitle, body, and link.


We're going to remove the theme.


We don't need this.


We do want to include something here.


So this is where we return the actual HTML that controls our look and actual structure


of our code.


So I'm going to remove these functions that we have here as well.


We don't need any of these.


These are all very specific to that default component that we saw.


And then the rest of this we do need to remove.


We don't need so many comments for sure.


That does basically everything we can remove.


I'm also going to just clean this up or just remove everything actually for now.


So we'll leave it like this.


So this here is very core what we need in our React component.


So if we switch over to the hosted version of our component now, we should see that's


pretty much it.


Well it should pretty much be empty.


So here, so now we just have this.


You click zero times, but that click me part of the component, that's not there anymore.


It doesn't exist.


I mean it is actually still there, but it's just some empty span tags.


There's nothing we can actually see.


So now what I want to do, let's modify this so that we can have a different app or a different


way of viewing our components.


So come over to init.py.


Okay, I'm just going to put this is like test one.


We have our components and I'm going to include this divider.


Test two.


Remove this.


Okay.


And then here I'm just going to put anything ready.


Oh, and also remove this.


Rerun.


Okay, now we just get this.


Now let's add some something to our components.


So if I go to here, let's go to the card component.


What I'm going to do is I'm going to copy this, paste it a few times and I'm going to


include the components or the parts of our card.


So I want to have a title, subtitle, a body and a link.


Okay and we'll just rename these as well.


Title.


Remove that as well now.


So now if I switch back over to the local host and rerun, we're probably going to get


an error.


I would expect to see some sort of error.


Okay so actually, okay that's fine.


So at the moment we have that name and everything in there but they're not being used and they're


also not being passed to our component.


If we look at our code in inet.py, we're just passing this one parameter and we're also


passing key.


Now our component doesn't include any of those, it actually includes these like titles, subtitle,


body and link.


So how do we add those in?


We head over to inet.py and scroll up a little bit and we see the definition of our component.


This acts as like the middleman between our component.


This actually should be called a component and later on we should be calling this card


component as well.


Do we have any other my components?


Okay, seems fine.


So in here we are accepting just name.


We don't want to accept just name, we want to accept all the other stuff as well.


So I'm going to remove most of this and I'm going to swap this out for title, we don't


need to do that.


Subtitle, body and also link.


So we pass those in and then in our component function we also need to pass those.


So title, subtitle, body and link.


Now we can remove these other parts for now.


So we don't need this key.


A key is useful if you have multiple components and they have some action on them so you press


to expand them.


You want to include key as you would pass just like a random number to each version


of that component and that allows Streamlit to identify each of those individual versions


of your component as actual independent components.


So when you're pressing action, expand one, it doesn't expand all of them.


But we don't have any actions I don't think so we don't need to worry about that.


Okay, we also don't need to return any component value.


So before where we saw it was saying like we click and it was increasing that number


that's what was going on there but we're not doing that for our component.


We're just displaying some information rather than modifying anything.


Okay now if we switch back to our local host we probably see an error.


So we rerun and we get this type error card component saying three required position arguments.


Now the reason for that is because when we're actually calling our component later on in


init.py we're only passing a single parameter to it which is hello and world.


So we need to pass at least four, one for each of the arguments that we have.


So let's have a look at that in here.


Okay so we have card component and we're passing at the moment this is just a title.


We want to pass okay let's say do title, subtitle and have some random text here for the body.


And we also want to have a link so I'm going to go to google.com.


Okay and then let's just copy this component.


This doesn't return anything so let's remove this as well.


Let's copy that and this is going to be number two so it's going to be title two.


Add some more text in here and yeah we'll just leave it for google.com.


Now let's have a look at our local host.


So rerun and now we have this so we still don't have anything in there.


Let's have a look at why let's try to figure that out.


So we have our card component creates this.


Let's have a look in here.


Okay so at the moment we just have span.


Okay we're not actually showing any of these any of these parameters that we've passed


to our component.


So let's go ahead and do that.


We have let's wrap everything in a div and then I'm going to go just page second heading


for the title.


So to pass actual variables into here we just use each curly braces.


So for subtitle and then we also want to include the body.


So body and also a link.


So for the that we'll just use a ref again using the curly braces in here as well.


So link, simple as link.


Now let's have a look at what we what we get.


Okay so we can see that we have our component now.


We have the title, subtitle, some random text for the body, link click that and it will


or it should take us to google doesn't seem to work at the moment.


I'm not going to worry about that now but we'll make sure we get it working for the


actual card.


Okay so that's we have the key components there but we need to actually format this


into a material UI card.


So let's let's go and figure that out.


When we come to here we'll let's copy we'll copy this one.


Maybe some earlier one.


Yeah let me I'll copy this one.


It's outline card here.


Okay so there's a few things in here we need to need to make sure we include.


So we have these imports.


We have the card, card actions that's where we have like the button or where we will place


a button.


So I'm going to import all those.


We do it here.


Save that and then we also want to include all of this stuff here.


So we have a card I'm going to down just copy all of that.


And let me paste after our link here so I can keep track of everything.


To typography.


Okay so we also need to add card around this.


I figure out in a minute.


So let's remove a few things.


So we have typography.


I want to sort of body I'm going to include in here.


Other typography to just remove I don't really want those.


Button.


We're going to modify all of this so let's just leave it like that.


Remove those and now we should see a card.


Let's open this.


So look okay so we kind of see we actually just have this the contents of the card.


We don't actually have the card.


That's because we also need to add the card tag in there as well.


So if we just do card.


The variant is outlined.


Is it in there?


Ah here.


So this bit here should be everything should outline.


Have this on the outside so basically all of this in here.


All this constant card is actually being inserted into this wrapped within card.


That's why we can't actually see it at the moment.


So add this and we should it should work now.


We should be able to see something.


Okay so now we have our card.


So we have the body of that but we need to include title, subtitle and this learn more


is a button at the moment but I want to make it like a like an icon button.


So we'll modify that as well.


So let's return to this.


I want to include the card header.


So if I go here card header and here we're going to have the title and also the subtitle.


So we have title and I think subheader.


Subtitle and we just close that tag like that.


So look.


Okay so cool we have our title with subtitle now.


It's looking better.


Another thing that I'd like to include is a bit of padding here.


So I'm going to write style and we add some padding.


It's going to be 0.5 rem.


So this just means if we have multiple cards and they choose each other they're not going


to be touching.


There's going to be a bit of space between them.


Okay so it looks pretty good I think.


We just need to include the actual button.


Okay so buttons are a little bit different.


I do have an example on a card.


Here we have these buttons.


That's basically what we want.


So let's have a look at how that is done.


So we see we have this icon button.


So we have card actions here and then we have icon button.


So I'm going to copy this.


Place it in card actions and then we place this one here.


We don't want that.


Okay icon button.


It's going to be like open URL.


And for share icon I'm not going to do that.


I'm going to expand icon.


It's the one I want.


I'm going to change this to expand icon.


And we want to modify this.


So oh sorry we need to import everything.


So we see at the moment it says cannot find name icon buttons because we haven't imported


it at the top.


So let's do that.


So let's have a look at our example again.


This example.


We have icon button here.


So and we also have share icon.


So let me copy that across.


Okay and it's launch icon I think.


I think I wrote expand icon.


Yep so launch icon instead.


And we import that.


So we write import launch icon from.


And here we're going to use the icons material that we installed in the previous video.


So icons material and it's launch.


Okay so let's have a look at what that looks like now.


Okay so we have this launch icon.


It doesn't do anything at the moment because we don't have any URL in there.


We haven't connected up our link yet so let's do that.


So we need to pass that into our icon button part here.


So we have prep as we did before.


And we're just passing our link.


And then another thing.


So okay let me save that.


That should run it.


Okay so now it's going blue.


That goes to another form.


Because it's an actual link.


We're still not opening anything.


We need to.


The reason for that is because Streamlet is displaying all this through a iframe which


is like an embedded web page within the web page.


So it's actually opening it within that iframe.


But at the moment it's just not working.


It's just not opening anything.


It's just because it's kind of being.


I think it's being blocked by Streamlet or something along those lines.


I'm not sure.


So what we want to say is we want to open it in the parent browser of that iframe to


target this actual window.


So to do that we need to add in here.


We need to modify the target.


So target is going to be equal to underscore parent.


Now save that and try again.


Okay now it's working again.


So there are our cards.


One other thing we need to just see here is the actual font is different to the MUI font.


Here it looks a lot nicer in here.


That's because they're using the Roboto font now to implement that we need to remember


where it is on here and maybe we can search at the top.


Okay yeah so we can we can just copy this.


So we need to npm install in our front end directory this.


So this is going to install the Roboto font for us.


I'm going to open another terminal window.


Okay navigate to the front end directory.


And we do npm install.


Once that has installed.


So switch back to here.


We just need to import it into our card component.tsx.


So we can go to the top and it's going to write a comment.


Import font.


Paste that and save.


And now this should look a lot nicer.


Okay so now it's definitely using the Roboto font.


It looks a little cleaner.


So that's that's everything for the component.


I don't think there's anything else.


Obviously in designing your own component you might want to make it more fancy and some


other things in there.


But for this I think that's everything that I'd like to cover.


In the next video what we're going to do is have a look at how to actually set this up


for distribution.


So other people can pick it up and install your component and actually use it in their


own streamer apps.


So that's also pretty interesting and pretty straightforward.


There's not too much to do there.


The bulk of the work is what we've just done.


So thank you very much for watching.


I hope that has been useful and I'll see you in the next video.


Python Environment Setup for Machine Learning

Hello and welcome to this video on how to set up a Python environment specifically for machine


learning. So this is often an overlooked part of machine learning and there's not that many


tutorials out there on how to do this properly. So I thought it'd be a good idea to just go through


this and explain exactly how I set up my environment. So you can see here we have Jupiter


and the first thing you might notice is that I have these different environments. You have the


default Python 3 base environment and then I also have this GCP which I use for the cloud and then


I also have this one which is my machine learning environment. Now the difference between each of


these is the machine learning environment specifically has packages in Python for machine


learning like TensorFlow, PyTorch, Transformers, Pandas, NumPy. It has all of those packages but


nothing else so there's no excess baggage if you like. So I wanted to just go through and explain


how to actually set this up. So we're going to close this Jupiter notebook.


We're going to open this new Anaconda prompt here. So I'm assuming that you have already


installed Python and that you are using the Anaconda distribution. So if you are not using


this you can download it over from here. You can head over to Anaconda.com and you just click on


products and individual edition over here. And just download. Okay so the installation for Anaconda


is pretty simple if you're on Windows. It's a little different if you're on Linux and I don't


know how it is on Mac but generally it's pretty straightforward and if you do need any help with


it you can just you can find out quite quickly. So once we have that installed we want to go over to


our Anaconda prompt and to make sure that we have installed it correctly we just want to type


python-v and this will show us the version of Python that we have. So I'm at the moment using


Python 3.8.3. Just make that a little bigger and okay if that works that's good. So at the moment


we're using the core base environment that you can see here and that is just the default


environment that gets installed whenever you install the Anaconda distribution.


But what we want to do is actually create a new environment which is our machine learning


environment. So to do that we use the syntax like this. So we conda create. Conda is just


referring to Anaconda or name. So you can also write this as name or again with n.


And then you want to enter your environment name here and then you would also


write python and your python version.


And at the end of that you would also type Anaconda. So for us I'm going to use an


environment name of mln like that and I also want to be installing python 3.8.


And that should be everything. So we'll just enter and now python will work through and actually


install that. So I've already installed a mln before but I uninstalled it. So it's coming up


this warning saying it already exists but I'm going to continue creating the environment because


I want to reinstall it. So put yes.


You shouldn't see that on yours. And then this will take a little bit of time just to


get everything together. Okay so now we are just showing a list of all the packages that will be


installed. So we just want to accept that. So press yes and enter.


Now we'll go ahead and insert the package.


Press yes and enter. Now we'll go ahead and install all of those.


Okay so everything is set up. Now we can switch over to our new environment. So at the moment


we're in base. We can switch over to our new environment with conda activate and the environment


name which in our case is mln. So let's go ahead and do that. And now you can see that the name


here is switched to mln which is our new environment. Now we just need to install our


machine learning packages. So we're going to go ahead and install the basics. So we have pandas


and matplotlib. We're going to install both of those with a conda install. So we have two options


here. We have conda or pip to install our packages. Generally conda will most likely


integrate with your environment better. So it's usually a good idea to try that first. If that


doesn't work then try pip install. So we'll go ahead and conda install. We're going to do pandas


and matplotlib. Then add anything else here that you feel that you might also need. But this is all


we're going to go with. So it's a good point knowing that we also need numpy but numpy is


included as a dependency of pandas. So we don't need to explicitly mention numpy here. And that


will go ahead and ask us for permission to install the packages that it finds. We click yes and then


we go ahead with the installation again. Now we can go ahead and install tensorflow, transformers


and pytorch which are all machine learning frameworks. So tensorflow we can install it


quite easily. All we need to do is conda install tensorflow. Okay so now we have the yes or no


from tensorflow. Okay so tensorflow is now installed so we can now go ahead and install


transformers. So transformers we are going to use pip because conda doesn't recognize the most


recent versions of the transformers library at the time of recording at least. So we have to use pip


to get the most recent versions. So we pip install transformers. There we go. And finally we have


pytorch which is slightly more complex but we make it quite easy by just taking a look at the


pytorch start locally guide which you can find here. So pytorch.org gets started locally. And


all we do is we come down to the start locally bit. We select our pytorch build. So this is the


stable release and this is like a beta release which gets released more often but it's more like


to have bugs and errors in. So I think most people probably want to avoid this. You can choose your OS


so from its windows package manager. So that is conda. You can also use pip but I would recommend


conda because it will install the dependencies we need as well. We're using python and then this


bottom one here refers to cuda. So we use cuda as the gpu acceleration library. So essentially with


this if you have a navidia gpu cuda lets you use it to speed up any machine learning tasks that you


have in either pytorch or tensorflow. So you can read tensorflow's gpu setup guide if you do have


a gpu. This is quite useful. So you just you head on down to the bottom here or if you're on linux


there's this guide as well which is quite useful. And then we have the windows setup here. So all


you need to do is install all of these which is reasonably straightforward but there are a lot of


good guides out there if you do need help with it. And then you just head on down and set your paths


so that sensorflow slash pytorch can actually see cuda. Another useful guide as well is this navidia


cuda installation guide which can be quite useful as well. Now I would recommend using cuda 10.2


at the time of recording. So unless you are using the latest rtx 30 series so that is the navidia


geforce rtx 3090 3080 and i think it's 3070. So the support flows is a little bit sketchy at the


moment and you actually need cuda 11 alongside the nightly builds of pytorch and tensorflow.


So this is what I mentioned over here. That's a little bit more difficult and I'm not going to be


covering that here but again there are a lot of good guides out there if you do need help with it.


So if you don't have a gpu or you just don't care about gpu acceleration you just click none and it


will change the command down here which we'll be using for our installation. So I'll be using this


command here. So we're doing a conda install and then we have a few packages not just pytorch here.


Pytorch, tortsvision,


torch audio,


cuda toolkit and first we are using 10.2 and then we set our channel to pytorch as well.


Now we can go ahead and install that.


So just select yes again and now that is our environment completely set up. So all we need


to do now is actually add this environment to jupyter. So remember at the start we had that


little box and we had python3, gcp and ml. So we're going to add a new one called ml environment.


So to do that we need to install ipykernel


and with that ipykernel we are going to install our new environment. So we do that by specifying


the name of it here mln and then we also want to set the display name. So this is the name


that we will see when we enter into jupyter lab and we can have that box. So this can be anything


you want. So for me I'm just going to put ml environment. Let me just run that again.


Okay so that is ready and now we can just go ahead switch back to our base environment which is our


default environment. And now we just open up jupyter lab.


And we can see here we now have this other ml environment and this is the one that we just


created. So that is it for this short video. I hope it's been useful and I will see you again


AugSBERT: Domain Transfer for Sentence Transformers

Today we're going to talk about how we can train language models across different domains.


Now to do this we're going to be using what is called the Augmented Esper Training Strategy


and we're going to be using the domain transfer flavor of that. So the training strategy is used


where you have some data but not enough or you in our case with domain transfer we have some data


in one domain so say maybe we have Quora question pairs but we need Sack Overflow question pairs and


all we have on the Sack Overflow side are unlabeled question pairs. So what we're essentially doing is


using that source data set the for example Quora question pairs to train a model that can then


transfer its knowledge and label the Sack Overflow question pairs. Now we don't need to stick with


question pairs and question pairs we can have and go from question pairs to semantic similarity


pairs or anything else but we do have to make sure that our source domain and target domain are at


least similar and we can think of this required similarity as a bridge across a gap between the


source domain and target domain. The greater that gap so the less overlap those two domains have


the greater this gap here is and if this gap is bigger we obviously need a bigger stronger bridge


and we can think of that bridge as our model we need a bigger stronger model to bridge these


longer or greater gaps between the two domains. So in this video we're going to be going through


one example but I have tested these five different data sets so what we can see is the Ngram


similarity. So this is how many Ngrams overlap between the two data sets as a whole so this is


not a perfect measure of similarity of course because it's only capturing the syntactical


overlap and it's not really capturing whether the two data sets have the same language structure


or talking about the same topics that much although it does help and this is just a very


simple measure to help us figure out whether the two domains that we're trying to transfer across


will be easy to bridge or not. So we would expect from this that the medical question pairs down


here because of this these lowest similarity scores it would probably not be the best domain


transfer to these different domains whereas these different domains STSBRTEMRPC and Quora


question pairs are all reasonably similar so we'd expect the performance or the transfer or ease of


transfer between those domains to be much smoother but like I said this isn't perfect and in fact


one of the best domain transfer performances that I saw was from the medical question pairs


to the Quora question pairs which obviously makes sense semantically because question pairs will


probably have more of a similar structure than entailment pairs which is what we have up here


with RT or just semantic textual similarity pairs as we have up here so that does make sense


logically but it's not really reflected in this chart here. So this is the overall strategy that


we're going to follow with domain transfer augmented STSBRTEMRPC. So the first step is to


get our data so we're going to have a labeled source data set which in the earlier example


would have been the Quora question pairs and we're also going to have the unlabeled target data set


which would be our stack overflow question pairs. We're not actually going to use those two examples


we're going to use from here we're going to go with MRPC as our source data set and STSB


as our target data set. So we'll go ahead and we'll get our labeled source data so that's going to be


MRPC and we'll train a cross encoder model with it. Now the reason I'll just quickly mention is


the reason we're training a cross encoder model rather than a buy encoder model which is Sentence


Transformer is because we need less data to train a good cross encoder and the cross encoder here


is only used to label the target data so we need less data to train a good cross encoder


but cross encoders are also slow which is why we don't use a cross encoder over here as our final


model. So we're basically using that greater performance with less data to build a data set


over here that can train a fast sentence transformer over here to a similar level or


similar performance to that cross encoder. That's basically what we're doing it's knowledge


distillation from our cross encoder all the way over to our sentence transformer that's all we're doing.


So let's jump over to the code and we'll have a look at how we do those or perform those first


two steps. Okay so we're starting with the source domain MRPC as I said and the target domain is STSB


so first thing we do is get that source data so we use Hug & Face data sets for this so you may


need to pip install that so just pip install data sets and from data sets we import load data sets


so this is just going to load the MRPC data set from Hug & Face which is stored in the larger


glue data set and MRPC is like a subset of that and then we're taking the train split from there


and we can see the data set features here so we have sentence one and two which are sentence


pairs that we're going to be learning from to in order to label the other STSB sentence pairs


and we also have the label as well and we only have 3.6 thousand samples here which is quite


a small number particularly for a sentence transformer so that's why we're going with the


cross encoder here so that we can build something that has good performance whereas a sentence


transformer might need a little more data although not necessarily so we need to first format that


data for training and to do that we just go through each row in the data set that we just


downloaded and sentence transformers always uses this input example object so all we're doing here


is creating an empty list and then within that empty list we're populating it with these input


examples which contain sentence pairs here and also a label and then behind the scenes we're


using PyTorch so we also need to initialize a PyTorch data loader with those input examples


and we're using batch size 16 and also shuffling that data as well so that's our data ready for


training the cross encoder and then we move on to if we'd like to we can move on to adding this


validation set as well which is useful because then we get a performance from our source validation


set if possible you really probably want to switch this out for the target data set validation set


so this would be in our case it would be STSB I'm not going to do that here just


to emulate in in like an actual project you may not have this validation data from your target


domain but I would definitely recommend if you do have the time to just manually label something so


that you can check that performance you will want to do that later anyway but for now we'll just use


source data set we're using this cross encoder because training cross encoder and it's coming


from this cross encoder section of sentence transformers library and it's a correlation


evaluator okay so that's going to check the correlation between the labels that we have here


so the true labels in our validation set and the predicted labels from the cross encoder


so cross encoder does not output sentence vectors like a sentence transformer it outputs a similarity


score and then we just initialize that correlation value we're using from input examples so we're


using input examples up here so we use that method there and yeah that's our training and validation


data ready to go so we can go ahead and start training the cross encoder so for training the


cross encoder we load this cross encoder object from sentence transformers again just using a


vert based encase so that's the initial model that we're using to initialize our cross encoder from


a search coming from the hooking face models and the number of labels so this you can change this


if you are training a cross encoder that you'd like to predict nli labels for example so you'd


have 012 which would be your contradiction neutral and entailment classes you would put something


like three because you have three labels then you want to output three labels but first we want to


add a similarity score so we just have one label then we have here we have a number of epochs now


anything more than one you're probably going to over fit so i would i think always stick with one


and unless you really have reason to do otherwise and here we have the percentage of warm-up steps


that we're going to use so this is the number of steps where we're not going to be training at the


full learning rate but we're going to be gradually increasing up to that full learning rate so it's


quite high 35 percent but it's what tends to work best i found find with cross encoders a higher


percentage of warm-up steps tends to work reasonably well and then over here we have our


optimizer parameters which is another training parameter that i modified a little bit so here


we're using slightly higher than the default two e to the minus five we're using five e to the minus


five now everything else is as you usually would with a cross encoder so we're using the fit method


we'll do that with a sentence transformer as well the train data loader this is slightly different to


the sentence transformer parameter but otherwise everything is exactly the same way so passing in


our source data which is the training data loader that we created earlier we have our evaluator


number of epochs warm-up steps learning rate and where we are going to save the model so that


source is going to be mrpc cross encoder so that trains our model it doesn't take long you can see


here it's it took on this computer 14 seconds it's very quick and then we can go ahead and label the


target data so that's on to the next step so let me actually go back to here so we've just been


kind of cross these off so we've labeled the source data and we trained our cross encoder


okay so this is our performance of the cross encoder models so that is the the source models


over here on different target data sets over here so the source model for mrpc on the sgsb target


the stsb target data over here so this is a the validation set of the stsb target data is what we


are aiming for so this 0.63 now we should be able to get better performance in this i think


uh but we'll have a look and we can also see i i said the stsb is quite easy as a data set to get


good results on and we can see that here because all of these do achieve better performance but


what we or what i really want to focus on here is the the BERT based on case performance as a


benchmark because this is what you would get if you didn't have your any any training data or which


we don't we don't have any target domain training data instead we're kind of using this


org expert training strategy to train from the source data set so these are our benchmarks


and we can see that the from from here so we have these red circles which represent that the


the model performance has decreased from that benchmark and then the blue ones over here


they represent that the model performance has increased from the benchmark and then we also


have grey which is to indicate that it was the same value or maybe a couple of points higher


but not that much higher so we can see straight away that the worse and we can see that it does


perform worse so all of these glue data sets don't really correlate so much with the medical


question pairs and we can see a lot more red in this column so that is to be expected in the other


glue data sets we see a lot more grey and blue so the results are either similar or better so


obviously the next bit is just targeting the main part we're not going to be doing this optional


augment data because it doesn't tend to at least for these data sets i didn't find this to help


much but if you if you do want to see how to do that the last video that we did on augmented expert


and last article does cover that augmentation of data using something called random sampling


so we're just taking random samples from our data set and creating new pairs with that and then


labeling them with our cross encoder that's all it all there is to it so but for this i didn't find


any significant performance increase it did increase the performance for a couple of the


models a little bit but not not a huge amount so i i think for the time it takes it's probably not


worth covering in some cases but if you think it might help especially if you have a small data set


i'll check out the other video and an article so we need to get out unlabeled target data and


i'm going to skip that step so we'll just just cross it out and we are going to label it with


our cross encoder so from up here we're going to take that and label this this target data


so let's have a look at how we do that so yeah the labeling that target data


first we need target data it's same again we're just going to download that from the


plugin face data sets library so we're coming from the glue like larger data sets again


but this time we're using stsb which is quite an easy data set to get good results on to be honest


so i wouldn't expect this sort of performance for everything but this is just the example


we're going to use here and so we obviously using the training as well and we can see here okay in


that data set we have the features sentence one sentence two and we also have the label so obviously


we don't actually need to use a cross encoder to create our labels but we are going to because


that's the whole point of this training strategy and i just couldn't find a reasonable reasonably


good data set that didn't have the labels already so we're just pretending they don't exist so you


didn't this here it's not there um if you if you're struggling to train a cross encoder or you can


just download it from here by the way um or if it's taking a long time i mean it shouldn't take that


long but i know on on some machines it can take a while so just if you're following along with this


in real time just i would do that having any issues so we have our target data and what we do


is we zip those sentence one sentence two together so i'll delete that let's say we have sentence one


and we have the first i don't know what it could be it could be i think it's something like a plane


a plane is taking off and we would have basically a tuple where we have the first sentence and we


also have the second related sentence which is something like an airplane


lands i don't think it's lands it is something to taking off as well but i don't remember what


exactly it is so you would get something like this and then the next one would be something else


like um the dog runs and i don't know the dog warps okay so something like that so we're getting


these like tuples of sentence pairs and we're feeding them into the predict method of our cross


encoder and that will be the first thing we do is we're going to do a couple of things


the predict method of our cross encoder and that will return a set of scores obviously these here


would be a reasonably high similarity because they're talking about a similar topic not


necessarily the same we've got warps and runs and lands and taking off but they would be


so reasonably high and then what i'm going to do so this just makes things easier in terms of


seeing what we actually what we're actually working with what we're looking at so i'm just


going to pass those sentence ones and sentence twos and then there's a new scores into a


pattern state frame and then we can see everything here so we can see a plane is taking off and


an airplane is taking off so that's that would be the first two i was talking about


and then we have all the labels that our cross encoder is predicting now all these are pretty high


because these are all very similar semantically similar pairs so that's what that's what we would


expect but later on there are other pairs which are less similar and then we we move on to train


the by encoder so let's quickly switch back to our visual again okay so now we have identified


or we've downloaded that on labeled target data cross that off and we've also labeled it using


our cross encoder model so the final step as we saw is training our by encoder with that


labeled target data so let's have a look at how we would do that so we have our labeled target data


in this pandered data frame here so we're just going to iterate through the rows in our data


frame and i'm going to append all those input examples as we did before to this data list here


okay and we have our sentence pairs and we have the predicted labels in there


so once we've created those that list of input examples again we're just pushing all of that to


a pyotorch data loader and from there we can go on and initialize our sentence transformer so


to do that we are using models and sentence transformer from the sentence transformers


library and we so the sentence transformer takes the typical transformer like bert


and what bert outputs is 512 word vectors and what we want is one single sentence vector for


from our sentence transformer so to do that we need a way of translating those 512 word vectors


into a single vector and we do that by pooling all of those 512 tokens into one token by taking


the average value across each dimension and that that's all we that's all we do there so that's why


we have this vert layer followed by a pooling layer which is taking the mean pooling and then


we combine both of those into a single sentence transformer object and that's our that's our


sentence transformer initialized but obviously we need to train it so we come down here to train it


on the data we have at the moment we have continuous values in our labels from zero to one


so what we can use is the cosine similarity loss so we initialize that and then again we don't want


to over fit so epochs is set to one and this time we're using a lower number of warm-up steps which


is 10 percent this time and we train with that so once that is done we are completely ready to train


and we just call model fit so train objectives is slightly different or it's sort of cross encoder


bit method but everything else is the same we're just using the default value here so 2e to minus


five so you don't really need you can remove that if you want but if you do want to change that


i put that in so you can see where you would change that and yeah we're ready to train we go


ahead and train and see how that performs so we evaluate it this time we're not using cross


encoder so the the evaluate is slightly different this time using a embedding similarity evaluator


which is going to take two sentence vectors and it's going to calculate the similarity between


them and then it's going to compare that to the that predicted similarity to the true similarity


as per our validation set here so in that validation set we do have these labels but in


stsp those labels are from the or in the range zero to five so we use this lambda function to


divide everything by five which brings us into a range of zero to one which is what we need for


this embedding similarity evaluator and then as we did before we're creating that list of input


examples and then we can initialize the evaluator so we're using the embedding similarity evaluator


from input examples again because we have input examples and we're passing in our input example


with data and right csv is false that just means i will print the score to the to our notebook


and then to actually evaluate all we do is pass the model to our evaluator and it will do everything


course and we get a similarity here or a correlation score of 0.76 which is is pretty good


you would you would think of you can think of something like 0.5 is kind of like your moderate


correlation and 0.8 is like high correlation so that's pretty good okay and if we for the final


quick part i just want to have a look at the other performances that i found because not


everything is going to be as good as that but we just got there so we can see so this is what we


just did over here we got 76 just now so we got slightly higher than what i got before


so we got 76 here but obviously like i said stsv is an easy data set to get good results on


and the others are more difficult so the rest of them are you're more in this moderate


um similarity range or correlation range so the medical question pairs that actually perform


better with the bioncoder training than the cross encoder training but nonetheless they're still


within the same sort of range that i would expect it's not it's not a massive improvement it's this


is a benchmark down here and there are a few percentage points better which is is probably


reasonable particularly because they will not be these blue data sets were not that similar to our


medical question pairs data set so that that makes sense and then the other ones yeah they're all


sort of within that moderate range the one that i switch did surprise me although it does make


sense i mean that question pairs um is this one here so the the transfer from from medical question


pairs to core question pairs was pretty good and i suppose if you have a look at here we from core


question pairs and medical question pairs it's not as good and maybe because the language in core


question pairs is simpler than the medical question pairs i'm not sure uh but from medical


question pairs to core question pairs it worked quite well so that i suppose points out where


that n-gram similarity doesn't always correlate exactly to what you would expect but anyway i


think the results from this are probably pretty typical from what you can expect now i think this


can be really useful if you if you really don't have any any labeled data within your target domain


to at least squeeze out a few percentage points of performance more than you would be able to


without this training strategy so for that reason i think this can be quite useful whether or not


it is the best approach to take will depend on your on your data um so i think it's useful not


always the best option but definitely something useful to know about and be able to apply if you


need it so yeah that's it um thank you very much for watching i hope it's been useful


Product Quantization for Vector Similarity Search (+ Python)

And welcome to the next video in our series on similarity search.


We're going to be covering product quantization,


which is a very effective method for reducing the memory usage of our vectors.


Now, I'll just show you this very quickly.


So we end it in the first column over here.


We have the recall speed of memory for a dataset of size 1 million.


So we're using the SIFT 1M dataset here.


And the dimensionality of the vectors in there is 128.


So very small compared to a lot of vector sets out there.


Now, the memory usage of that when we just saw them as they are


is a quarter of a gigabyte, just over, which is pretty big.


And you think this is a small dataset, small dimensionality.


So, you know, this is already pretty significant.


And when you start getting into high dimensionality or just larger datasets,


it quickly becomes very unmanageable.


So product quantization or PQ allows us to reduce that significantly.


So in this example, so the tests we run here,


we reduce the memory usage by 97%, which is pretty significant.


So from a quarter of a gigabyte to 6.5 megabytes, it's like nothing.


And then the speed increase is pretty significant as well.


So just under a six-fold speed increase there.


So pretty good.


Obviously, the recall also decreases a fair bit.


So that's just something that we need to be careful with.


But at the same time, there's different parameters


that we can use in order to fine tune that.


So what we're just going to cover in this video


is the logic behind product quantization.


There will also be another video, which will be following this one,


where we'll have a look at product quantization in FICE.


And we'll also have a look at a composite index of product quantization


and inverted files, so IVF.


And with that, we can actually make the speed even faster.


The memory goes up a little bit to, I think it's like nine megabytes.


So pretty minimal. But the speed increase is very significant.


It's like a 92 times speed increase over a flying deck.


So it's super fast.


But we'll be covering that in the next video.


For this one, I'm just going to work through what product quantization is


and how it works.


So we'll start with the example of dimensionality reduction,


which is what you can see here.


So we have this S value up here, which is kind of like the scope of possible vectors


that we can have. So all along here.


In the middle here, this is just a...


I've highlighted a single vector.


So this would be X, our vector.


And D over here is, of course, our dimensionality of that vector.


So typically, well, in this CIF 1M dataset, for example,


we'd use a dimensionality of 128.


A dimensionality reduction is something like PCA, for example,


where we would just reduce the dimensionality there.


And we will try to maintain the geometric or spatial properties of that vector,


even though it has less symmetries, we would still try and maintain that.


So vectors that were close to each other before with a high dimensionality


should be close to each other again in low dimensionality.


Now, what we have here is not then dimensionality reduction,


this is quantization.


Now, quantization is a very generic term,


and it focuses on any method where we're trying to reduce the scope of our vectors.


Now, in this case, so beforehand, the scope is typically very,


either a very large number or technically infinite.


So if we're using floats, for example,


the number of possible vectors that you can have within that space


is usually pretty big that you may as well say it's infinite.


Now, in this case, what we do is reduce that.


So we reduce the possible scope into a more finite set.


Then we may be going from something like a practically infinite scope


to something like 256 possible vectors.


That would be a very small number, but it's just an example.


Then notice that the dimensionality does not necessarily change.


It can change with quantization, but that's not the point of quantization.


The point is to reduce the scope of our possible vectors.


Now, a very high level, let's just kind of cover what product quantization is doing.


So what we have here, this big blue block at the top here,


this is our initial vector x.


Now we will say dimensionality, this is 128.


So d equals 128.


Now the number of bits required to saw this vector,


let's say it's a 32, every value is a 32 bit float.


So we have 32 bits multiplied by the dimensionality is 128.


So in total, that gives us 4096 bits that we need to saw this vector.


Now that is pretty significant.


So what we are going to do with PQ is try our best to reduce that.


First thing we do with product quantization is split our vector x into sub vectors.


Each one represented by u, so u to subscript of j,


where j is a value from 0 up to 7 in this case,


representing the position of that sub vector.


Now the next step is the quantization.


The product quantization.


So the next step, we process each one of these sub vectors through a clustering algorithm.


Each of these clustering algorithms is specific to each subspace.


So it's specific to j equals 0, 1, 2, 3, or so on.


And within each one of those clustering sets, we have a set of centroids.


Now the number of centroids, we set that beforehand.


So when we're building our index, we set the number of centroids that we want.


And the more centroids we have, the more memory we use,


but also the more accurate the index becomes.


Because you think you have, say you have two centroids in one vector subspace.


Your sub vector will have to be assigned to one of those two centroids.


And in the case that we only have two centroids,


neither of them could be particularly close to our vector or sub vector.


So that average distance between sub vectors and the clusters that they're assigned to


is called the quantization error.


And for good accuracy, good recall, good results,


we want to minimize that quantization error as much as possible.


And we do that by adding more centroids.


So if we compare our cluster of two centroids to a set of clusters


where we have, let's say, 200 centroids,


the chances are we're going to find a centroid that is more closely fitting to our sub vector


in the 200 centroid algorithm than the one with just two centroids.


So that's the logic behind it, sort of the memory usage and accuracy trade off there.


Obviously, the more centroids we have, the more data we store in our index


to cover the full scope of all those centroids.


Now, once we've assigned a centroid to our sub vector,


we map that centroid to its unique ID.


So every centroid within this index will have a unique ID.


That unique ID is a 8-bit integer, typically.


So we've now gone from...


So originally we had a vector x of dimension i.t. 128.


And each value was a 32-bit load.


So the number of bits we use there, 4096.


Now we only have a ID vector which contains 8 8-bit integers.


So that's 8 multiplied by 8.


In this case, we have 64 bits.


So we've compressed our 4096-bit vector into a 64-bit vector.


And that's what gets stored in our index.


Now, there's also something called a codebook,


which maps each one of those IDs back to the reconstruction or reproduction values,


which are those centroid vectors.


But of course, because we've minimised the scope there,


we don't need to store that many of those mappings.


So say our scope is 256, we only need to store 256 of those mappings.


So this is how product quantisation can be so effective.


Now, we'll go through some more visuals,


but I want to actually write this out in code as we go along.


So for me, at least writing out in code makes it much easier to,


at least logically understand the process.


So there are a few variables that we need to define here.


So we have this vector up here.


It's very small.


It's using integer values.


You know, the data types here don't matter.


We just want to build out the process of product quantisation.


This is not an effective implementation or anything like that.


We just want to understand the process and logic behind it.


So the first thing is how many sub vectors are we going to convert our X full vector into?


In this case, I want to create four sub vectors.


And we also need to get the dimensionality of X as well, which is just the length of X.


Now we have those two.


And the first thing we need to do is make sure that D is divisible by M,


because we need equally sized sub vectors.


So what we do is rewrite assert that D is in fact divisible by M.


So we just write this.


And if that is the case, we don't get an error.


So if I just add a one in there, I'm going to throw it off.


We get this assertion error.


So we just add that in there to say, OK, D is definitely divisible by M.


So that's good.


We can continue.


And what we want to do is say, OK, what is the length of each sub vector?


It's going to be equal to D divided by M.


And we'll have to convert that over to an int as well.


So just do that.


And let's see what we get.


So each sub vector will be of size three.


Now let's build that.


So our sub vector or our set of sub vectors will be assigned to U.


What we're going to do is go X from row to row plus D.


And the D underscore for row in range zero to D.


I'm going to take in steps of the D underscore.


OK, and then let's see what we get.


And see that we now have our sub vectors.


Now, what does that look like?


Let's visualize that.


So here we have that vector.


So D up here is our 12.


And we're splitting that vector X of dimension 12 into four sub vectors.


So M equals four down here.


We'll create four sub vectors.


Now D over M, which is what we calculated before, produces this D star.


Now in the code, we can't write the star.


So I've changed it to D underscore.


And the value of that is three.


So each of our sub vectors has a dimensionality of three.


So here we have that D star dimensionality of three here.


These are our sub vectors.


We have four in total.


So that is equal to M.


Now, next thing I want to do is I want to set K.


So K is the number of possible values that we're going to have in our data set.


So we are going to do two to the power of five.


And we print that out and we get 32.


So K is going to be equal to 32.


Now, the one thing we need to make sure of here is this K value.


So the number of possible centroids that is going to be shared across our entire set of


subspaces or sub vectors.


So we need to make sure that that is divisible by M again.


So we do the same thing again.


We say assert that K is in fact divisible by M.


If it is, we can go on to get K star or K underscore, which is going to be K divided by M.


Okay, and let's have a look at what that is.


So this means that with that, well, we should also make sure that's an integer.


That's an integer.


Like so.


Okay, so what that means is that we are going to have eight centroids per subspace.


So each one of these sub vectors here will have the possibility of being assigned


to one of the nearest eight centroids within its subspace.


And in total, across all of our sub vectors, of course, we have a total of 32 possible centroids.


Now, what that 32 there means is that we will have a codebook.


So codebook is simply a mapping of our centroid vectors or in fact,


let's do the other way around, is a mapping of centroid vector IDs.


So each one of the centroids that we haven't created yet, but we will create,


will be assigned a unique ID and it maps us from the unique ID to the actual centroid sub vector.


So that's where later on we, you know, we call those centroids reproduction values,


because later on when we're searching through our index,


our index is storing those 8-bit integers, but obviously we can't compare those 8-bit integers


to an actual vector that we're searching with, our query vector.


So what we do is we map each one of those 8-bit integers, which are the IDs,


back to their original centroid values, not to the original sub vector values,


but to the original centroid values.


And that's what our codebook is for.


So this K value, this 32 means that we will have a total of 32 mappings in our codebook.


Now, I know that can maybe seem confusing.


So let's have a look at what that actually looks like.


So over here we have the U and our sub vectors.


Here we would have, let's say J equals zero, J equals one, two, three.


And over here, what we have is our reproduction value.


So that those centroid or those cluster centroids.


Now, in this case, we have these three centroids.


In reality, we set K equal to eight.


So we should actually see not three centroids here, but we would actually see eight.


So we have one, two, three, four, five, six, seven, eight.


So in reality, there would actually be that many centroids in there.


And each one of those centroids can be referred to as C, J.


So in this case, J would be zero and I.


So we would have, let's say this centroid is I equals zero.


I equals zero, this centroid is I equals one, the centroid is I equals two,


and so on all the way up to, in this case, seven.


So zero to seven, giving us the eight centroids there.


So to get to this centroid here in our code book, so this C represents our code book,


we would go C, zero, zero.


Or this one over here, this number two would be C, zero, two.


And let's say down here, this green centroid, let's say that is I equals four.


So that means that to get to that in our code book, we would say C, three, four.


Okay, so that's just a notation behind it.


So our quantized subvector is one of these centroid subvectors.


And each one of our reproduction values, or those subvectors,


the quantized subvectors will be assigned a reproduction value ID,


which is simply what we have here, so this Cji.


So in this case, this value here may be the value zero, eight integer zero.


And if we start counting through all of our code book, that would relate to C, zero, zero.


Now, if we were to go, let's say down here, this is number 10, the integer value of 10.


Now, that if we count through, so J is going to go up to seven,


and that's our integer value of seven.


And it's going to get reset because we're going to take J from zero to one,


which is going to reset our eye counter.


So it will go from C, zero, seven.


And then the next one in our code book will be down here, which will be C, one, zero.


So in this actual integer value in our ID vector, that would be represented by the value eight.


So this value here would be eight, so we add one more, we go to nine,


so that would be C, one, one, and 10 would be C, one, two.


So C, one, two.


Okay, and that's how we represent those.


And we refer back to those original centroid subvectors


within the reproduction values code book.


But for now, we don't have those centroid, so we need to create some.


So what I'm going to do is from random, import randint.


C is going to be our overall code book or list of reproduction values.


And we're going to say for J in range m.


So we're looping through each of our subspaces here.


We are going to initialize C, J.


Okay, and then in here, we need to store eight different centroid vectors or reproduction values.


And of course, we refer to those as I, so position those as I, for I in range.


And here we are looping through K underscore.


So in our case, that would be equal to eight.


So we run through that eight times.


Here we have m, so you think four.


We loop through four Js and we loop through eight I's for each one of those.


So in total, we get that 32 value, which is our K value.


And what we want to do is say, we're going to say C, J, I is going to be equal to...


We want to set randint from zero up to...


It's going to be our vector space, our cluster vector space.


So we're just going to set it to the maximum value that we have in here.


So we have a nine.


So we set that.


So these are the centroid values, remember, not the IDs for underlying in range.


Here we have our dimensionality of each subvector.


And what we want to do is just append that to C, J.


C, J, I.


And then outside that loop, we want to append C, J to C.


Okay, so we've now just created our clusters, which we can see in here.


So each subspace, so each J has a total of eight possible centroids.


So one, we have zero or one, two, eight here in each one.


And don't forget, we have M of those subspaces.


So we should see four of them, although it does cut out here.


But this is, in fact, just the fourth one here.


Now, I think it's pretty cool to be able to visualize that.


So I'm just going to copy and paste this code across.


You can see here, you can copy if you want.


It will be also in the notebooks that will be linked in the description of the video.


And we're just going to visualize each of our centroids within each of those subspaces.


So this is C, where J is equal to zero, J is equal to one, two, and three.


And these here are our centroids.


Now, typically, of course, we will train these, but we are not going to train them.


We just want to create a very simple example here.


So we're not going to go that far into it.


Okay, so I mean, that's everything we see there.


And now what we need to do, so we've produced those reproduction values, all those centroids.


And now what we need to do is actually take our subvector


and assign it to the nearest reproduction value or centroid.


Now, how do we do that?


We simply calculate the distance between our subvector and each of its equivalent reproduction values or centroids.


Now, I'm sure it's going to get annoying if I keep saying reproduction values or centroids.


So I'm just going to call them centroids from now on.


It's so much easier.


So we are going to calculate, we're going to identify the nearest of our centroids to our specific reproduction values.


To our specific subvector.


So I'm just going to copy these functions in.


The top here, we're just calculating the Euclidean distance.


It's pretty straightforward.


If you need to just Google Euclidean distance, the form is pretty straightforward.


And then here, we're just using that Euclidean distance and looping through each of our values.


So the k underscore, so each of the centroids within our specific j subspace.


So all we need to do there is pass a specific j subspace and our specific subvector to that function.


And to do that, all we want to do, oh, just also note that what we're doing here,


these are the index positions, so the high positions.


And we're looping through and saying if the new distance that we calculated between,


you see here we're accessing that specific centroid.


If that is less than the previous one between our subvector and that centroid,


then we assign the nearest ID to that vector, not necessarily distance.


We don't care about distance, we just want to see the nearest ID to nearest ID.


So which is what we return.


So what we're going to return there are in fact the i values for those centroids.


So first thing to do is initialize our IDs vector.


So here we're actually going to build that 8-bit integer IDs.


In this case, there's not going to be eight of the values,


there will be four because the length of this vector is always going to be equal to m.


So we have four j in range m, i is equal.


Is equal to the nearest between c j or c j and u j.


I want to say ids.append i.


And then let's see what we have.


Okay, so that is our, that's our quantized subvector.


So each one of these IDs here represent one of our centroid positions.


So if we wanted to get those centroid positions,


so in this case I'm going to call them the reproduction or reconstruction values.


We would do this, so this is going to be our reconstructed vector.


I'm going to go between, so we're going to go for each j in range m.


Again remember, we're going to say c j i.


So the reconstruction value is equal to c for j.


And then the i position is whichever value we have in here.


So we need to say ids j.


Okay, and then we're going to do q extend c j i.


And now let's have a look at what we get for q.


So this is our fully reconstructed vector.


Now earlier on I said that we get the quantization error.


And we typically measure that using the mean squared error.


So this function here, you can see it there.


So what we can do is we can measure the error or mean squared error


between our quantized vector and the original vector.


So all we do here is we go mean squared error x and q.


And here we get the total squared error, mean squared error,


between our original vector and the quantized version.


Through sort of increasing the m value or increasing k,


we should be able to minimize this and therefore improve the performance of our index.


So I have one final thing to show you, which is how the search works in pq.


So on the right we have just a single, just a single id vector.


We would use our codebook c so that they would each go into this like so.


And from there we would output our quantized subvectors.


So these are the reproduction values.


Now on the left we have our query vector.


And we would split that, of course, like we did before with our initial vectors.


So we would split that and we would get this.


Now what we would do is we calculate the Euclidean distance between these.


And what we would get is we would simply add all of these up.


So we'd take the product of all of them, hence why this is called product quantization.


And the total distance would be all of those added together.


Let's say dist.


Now we would do that for all of our vectors.


So this is just, this is a single.


So this is like a single.


This is just, this is a single, so this is like Q u.


That's a single quantized vector or set of quantized subvectors.


We need to do that for all of them.


And then we just find the one that produces the lowest or the smallest product distance.


Now as well it's worth noting because we're taking the product,


this isn't really the distance between the two vectors,


between u or the quantized version of u.


And xq, but it is a almost like a proxy value for that.


And then that's what we use to find, to identify the nearest vector.


Now I'm not going to go into the code behind this because it's quite long,


but I am going to, I'm definitely going to record that


and just leave it as like an extra almost bonus video


if you do want to go through that.


But it is, it's pretty long, so I wouldn't say you necessarily need to.


But in the next video anyway,


we're going to have a look at how we implement all this in FICE,


which is obviously much more efficient.


And it is in that video that we will also introduce the IVF and PQ Comp Set Indepth,


which just allows us, if you have a look at the speed here,


to really reduce the speed an insane amount.


It's like a 90 times increase in the speed of our search.


So it's pretty cool. But yeah, that's it for this video.


Implementing Filters in the New Haystack Doc Store

Today we're going to have a look at the current state of the PR for the Haystack Pinecone


document store and we're going to try and work through and maybe solve all of the issues although


I tend to be quite optimistic so maybe that's not going to be realistic but we're going to give it


a go and we're going to go through and try and figure out what we need to do to actually merge


this into the Haystack library. So we'll just scroll down on it and so I have Bogdan who has


given a lot of feedback on this and the first thing or okay not the first thing because


I need to figure this out as well is make sure typing is compliant with MyPy. I mean for most


of it I was just taking methods from other parts of the framework so I'm not sure if there's been


some updates since I started doing that so my types are just kind of out of date or if it's


some other types I've kind of put in there not realized they were different I'm not sure so


we'll figure that out later it's fine. So one big one is to make use of the filter utils for


converting the filters at the moment it's just a method inside the document store called I think


build filter clause because this handles converting from a from Haystack syntax of your so Haystack


filter syntax to pinecone filter syntax. So we need to just solve that and what do you think


so it's just a load of these like little ones that we're going to go through. So for the so vector dim


this is what this is what makes me think I was looking at some old older version this has been


deprecated now and now you're just using vector or embedding dim so let's fix that.


So anywhere it says vector dim I just need to remove that.


First let me just check okay


I can just replace all of these so let's replace all of those so embedding dim


okay it's number one easy. So this is a more difficult thing I think we need to to figure out


so there's this filter utils script and when I looked it was kind of confusing before so I'm


going to let's take a look now and see if it see if we can figure out so filter utils I think it's


here so cylinder document store directory and we have this logical filter clause come down here and


ah okay okay yeah this is why it's so confusing to me so I've convert to elastic search convert


to SQL and they're all just empty so maybe I just need to I'm just going to try and add the convert


to pinecone method in here like pretty much as it is already and hopefully I don't know figure


that out but at the same time we have this logical filter clause object and I'm not sure


what that looks like exactly so if we come here class that is able to pass a filter and convert


into the format the underlying databases of the document stores require filters are defined as


nested dictionaries keys of dictionaries can be logical operators comparison operator and so on


and so on okay so the logical filter clause is here


there are the comparison operators as well okay


logical operator keys might take a dictionary metadata field names


and there so at the moment we are converting these correctly as far as I know


but I'm just not sure how to integrate that with this filter utils so let's have a look at


another document source have a look at weaviate and see how they use use that so it's it's filter


utils logical filter clause so they only use a logical filter clause here they don't use the


other one as well the comparison so let's have a look logical filter clause see where it's being


used so if there's filters this is just being called so logical filter clause pass filters


convert to okay can you see what this is so if we if we take okay let's take this and we'll put it


into a document into another file okay let me create a file here and host like test


um so I'll just call it filter utils test


okay so I'm going to test this but we need to import it so where is where is weaviate importing


this from here so okay let's take that import it here um should work I'll select the kernel haystack


let's see so filters I'm going to


do take a filter from the filter test I created before


um yeah anything there we go


let's see what that gives us a filter date


um or operation wow super weird um convert to elastic convert to elastic search


right


so I need to figure this out a little bit um see what we have into


so pass is creating one of these objects


uh what object is it filter utils or operation seems really complicated to me


so pass here okay not operation pass value okay not and


so it's like we need to modify the current filter so so we need to modify this to not


take a look directly at the keys but instead look at the objects


created over here so this and operation or so on we also need to figure out what those are so


they in here actually in this script and operation okay ah cool okay so this is where that is happening


so condition convert to elastic search where's condition coming from here um condition


self-lock okay it's just it's stored in there from the logical filter clause


ah okay so you initialize this logical filter clause which consumes


which consumes where is it um up here somewhere conditions here okay


okay self conditions are added to the logical filter what's it called again logical filter


clause and then later on when we actually use this and operation that is inheriting that value


from the logical filter clause so it's the condition self dot conditions is already in here


okay and then in here we need to specify how to deal with each of those items


okay


but then it's a condition convert to weaviate


if a condition in the operator return


okay so if we do something like let's just take let's take the weaviate one


if we do something like convert to convert to pinecone


convert to pinecone here


and the operator is not and


and so what is okay so the operator in this case is and is that right i can't remember now


so if or um let me fill this


where is um okay yeah it's and so and then the operands what is the operands here um is that


relevant i don't use it here all right let's see what that returns to us so


first i think we also need to pass convert to


convert up in the original or in the first


um class so logical filter clause


find convert to pinecone


and we just pass i'm not going to write the description in there yet


so let's go ahead and i'm just going to install that maybe i can also


let me add it for the or as well so or operation okay oh no no not that one


let me copy that there


okay where's the operation oh here so convert to pinecone again here this time we are using or


i imagine in here maybe i need to create a list so let's but let's just test it first see what we get


so install now let's go back and actually test that so


restart


what yeah run this should we should now have so we get the filter date um don't need to


have to do that again and then from that we should be able to so if we just convert to


weaviate for example we should see something uh okay what happened there did i interrupt okay


let's run again okay fine


pass we get this filter utils or operation convert to weaviate see what happens okay cool and then


if we do same again but convert to pinecone attribute error


um okay so i need to add convert pinecone to a lot of things here i think


okay um let's go back and do that then now just leave it as a weaviate we're going to reformat


everything i just want to make sure that everything is in place and i can just modify things in there


the thing i think realistically the conversion from to hate to pinecone should be super


straightforward because we don't have all these different syntaxes ours is very aligned to pinecone


so it shouldn't really be an issue or pine cones is very aligned to haste types


and so i'm just going to skip forward to this in this bit okay so that should be enough for that


and we can first need to put install again and then we'll just rerun this and make sure that


we're actually returning some sort of dictionary or filter to make sure it's everything is in place


so restart okay and then run this see what happens


okay so some issues in there let's try and figure that out okay so we've got


this now so we have like a really basic dictionary going through so all you really need to do now is


modify the code that we added to filter utils to align to the logic in here so i think what we'll


do is maybe have a few example filters and kind of figure out how we're going to write that


so let's start with that okay so we've got this working now at least for this one filter query


which is is is pretty cool a lot easier than i was expecting so that's really good and this sort


of logic of putting things or separating everything out i think makes a lot more sense particularly


if you consider how complex some of these other querying syntax must get so before i was thinking


oh this seems a really overly complicated way of doing things but now i think i'm convinced


this is probably one of the best ways to do it so the only one that we don't have is this not


operation so i don't really know what to do with that yet so i'm going to so now i'm just going to


leave it and come back to it later maybe maybe just add like a not implemented error or


something like that the only reason i say that is it would be great to say if you say not this