that classification task. Then you process all that and then calculate similarity between the


Zero-Shot Image Classification with OpenAI's CLIP

outputs and whichever of your text embeddings has a high similarity to each image that is like your


class, your predicted class. Okay so let's move on to an actual applied example and implementation


of zero-shot learning with CLIP. Okay so to start we will need to pip install datasets, torch and


transformers and what we're going to do is download a dataset. So this is the FRG-FM ImageNet data set.


We've used this a couple of times before and it just contains 10 different classes, not too much


data here we're looking at a validation set. So we have just under 4,000 items here and if we have


a look at what we have in the labels feature, so in the image feature we obviously have the images


themselves label feature. We have these 10 labels okay but they're just numbers, they're integer


values. We obviously need text for this to work with CLIP so we need to modify these or we need


to map these to the actual text labels. Now we do that by taking a look at the Hugging Face dataset


info features and then label names. Okay so most Hugging Face datasets will have a format similar


to this where you can find extra data information like the label names. Okay and then from there we


can see we have Tench, English Springer, Cassette Player, you know a few different things. All of


these map directly to the values here. So for zero we'd have Tench, one we would have English Springer


and so on. So as before we're going to convert these into sentences. So a photo of a Tench,


photo of a English Springer and so on and so on. Okay so from here before we can compare


the labels and the images we actually need CLIP so we can initialize CLIP through Hugging Face.


So we use this model ID and then we use model processor which is going to pre-process our images


and text and then we also click model here which is the actual model itself. And then we can also


run it on CUDA if you have a CUDA enabled GPU. For me I'm just running this on Mac so CPU.


NPS as far as I know is not supported in full or CLIP is not supported in full with NPS yet.


So that's like the the Mac M1 version of CUDA. For now CPU is fast enough it's not it's not slow


so it's not a problem. Now one thing here is that text transformers don't read text directly like


we do. They need like a translation from text into what are called input IDs or token IDs.


We should just integer representations of either words or sub words from the original text. So we


do that with the processor here. It's passing our text padding we set to true so that everything is


the same size. We need this when we're running things in parallel multiple inputs in parallel


essentially when we're using batches. We're not passing our images here and we're going to return


PyTorch tensors. Okay and we're going to move all that to our device for me it's just CPU so it


doesn't actually matter but it's fine. And then here we can see those tokens so we have a start


of sequence token here and then we could imagine this is something like a photo of a tench.


Something along those lines and then end of sequence over there as well. So we can encode


these tokens into sentence embeddings. All we do is this so pass our tokens in here label tokens.


Now in here we have input IDs and also another tensor called attention mask and that's kind of


wrapped within a dictionary which is why we're using these two asterisks here to iteratively


pass both of those tensors as individual items to the get text features function.


And then after we have our label embeddings over here we're going to detach them from PyTorch


gradient computation of the model and convert that into NumPy and then we can see from that that we


get 10 512 dimensional embeddings. Okay so there now the text embeddings are within that click


vector space. And one thing to note here is that they're not normalized. Okay we can see they're


not normalized so we can either use cosine similarity to compare them or we can normalize


them and then we can use dot product similarity. Now if we normalize first I find the code later


on to be simpler so we will do that here. So we're going to normalize here it's pretty simple


and then we can see straight away they're normalized. We can just use dot product similarity


now. That's the text embedding or the label embedding part. Now what we want to do is have


a look at how we do the image embeddings and then how we compare them. So we're going to start with


this image first just a single image. We'll go through the whole dataset in full later. So we


just have a cassette recorder here. Now we go down here we're just going to process the image. So


using the same processor we set text to none because there's no text this time. And what we


want to do is just pass that image in here to images and we're going to return the tensors.


It's PyTorch tensors and extract the pixel values. Now the reason that we have to process the image


is Clip expects every tensor that it sees to be normalized which is the first thing it does and


also a particular shape. It expects this shape here so three color channels which is through here


a 224 pixel wide image and 224 pixel height image. Okay so all we're doing there


normalization and resizing. Then from there we can pass it to get image features. Our image again


like here we didn't include the iterable because this is just a single tensor we don't need to pass


the two asterisks here. And we get a single embedding here one vector which is 512 dimensions.


As with our label embeddings we're going to detach them move them to CPU and then convert to NumPy.


I already have them on CPU so this part isn't necessary but if you're using CUDA it will be.


And then we don't need to normalize them. So when we're doing dot product similarity


we just need one side of the calculation to be normalized not both. Okay so with that we do NumPy


dot we have our image embedding and then we transpose label embeddings and you see that we


get scores and the shape of those scores is one dimensional that's not important. Here we have those


10 similarity scores so one similarity value for each of our 10 labels. Okay so then we can take


the index of the highest score which happens to be index two and then we find out okay which


which label is that it is cassette player. Okay so it's correct that's pretty cool. Now let's have


a look how we do that for the whole data set. So all I'm going to do is we're going to go through


in a loop through the whole data set we're going to do in batches of 32. Process everything this is


all just the same stuff okay process get the embeddings get dot product. Okay we don't need


to redo this for the labels because we we just have the 10 labels throughout the whole thing


so it's not necessary. We'll get the arg max and we're just going to append or extend a prediction


list with all of those predictions. Okay and let's see what we get or let's see what the


performance is there. So here calculating how many of them align to the true values and we can see


we get 0.987. Okay so that means we get 98.7 accuracy which is pretty insane when you consider


that we have done no training for clip here. It has not seen any of these labels it has not seen


any of these images. This is like out of the box zero shot classification and it's scoring 98.7


accuracy which is I think really very very impressive. So this is I think a good example


why I think zero shot classification or image classification with clip is such an interesting


use case. It's just so easy right you can do this for a whole ton of data sets and get good


performance. It's not going to be state-of-the-art performance but pretty good performance like this.


Super super easy. So before clip as far as I'm aware this sort of thing was impossible.


Okay every domain adaption to a new classification task needed training data. It needed


training and so on. With this it's just a case of you need to write some labels maybe modify them


into sentences and then you're good to go. So that's why I think clip is I think it has created


a pretty big leap forward in quite a few areas such as image classification. So when I think of


clip there is in these in the short time that it's been around we have multimodal search now,


zero shot image classification, object localization or image localization, object detection,


also zero shot and we'll go into that in more detail pretty soon and even industry changing


tools like OpenAI's DALI includes a clip model. Sabre diffusion as far as I know also includes a


clip model. So there's this massive range of use cases that clip is being used for and I think


that's super interesting. So that's it for this video. I hope you have found all this as interesting


as I do. So for now thank you very much for watching and I will see you again in the next one.


How to Learn Data Science | ML | Programming

Today I want to talk about learning to learn and some of my own systems and approaches to learning,


machine learning and other things in my life as well. Now there are a lot of methods that we


might talk about that may seem obvious and there are others that I believe, at least for me,


are less so. And in particular one very powerful thing for me, we'll definitely talk about that


and hopefully a lot of these things will be either reinforcing what you're already doing


or something new that you can start applying. But of course at the same time we're all different


people. We, I think, all have different approaches to learning that work best for us and for each


person that is going to vary. So I wouldn't go into this video expecting to have this perfect


template for learning how to learn that's going to work perfectly for you. It's probably more a case


of taking some bits and ignoring other bits and applying it to what you already do. Nonetheless I


do believe that this approach, although I don't believe I can cover everything in this one video


because there is a lot to cover, I do believe what we will cover is at least a good foundation


to tackling learning, machine learning and of course other things as well. So let's start.


Specific to machine learning I think it's very important that I share my sort of view of different


sort of view of different things in machine learning. I think everything kind of everything


I need to learn seems to be somewhere on a scale of pure theory to pure application.


Everything is kind of on that scale somewhere and I think both the theory and both the application


are super important and I think when we're learning something it's pretty important to


find a balance of both. And a lot of the time I really have to remind myself of this so that I


don't end up falling into one or the other too much. So I don't end up doing focusing purely on


theory or purely on application. It's very important to have both of those because what you learn


on the theory side of things is always going to support and back up the things that you are


actually building. So when you're actually applying your knowledge. We'll talk a little more about


that later but I just wanted to cover that very quickly at the start just so we can kind of put


everything in the scope of that scale of theory to application. So the first thing I want to cover


is my general way of thinking when I am approaching something new to learn and that is to go shallow,


deepen and then broaden. So this applies when you are learning to code, when you are learning machine


learning for the first time or when you are just learning something new in machine learning or


learning a new framework or so on. Now what I mean more specifically by that is going back to that


scale that we have. I start with applying something in most cases not all cases but most of the time


I was sort of trying to apply something. So I will take a library or topic and I will try and


create some sort of shallow high level tool with it or pipeline with it and using that you should


hopefully build a little bit of an understanding of the overall scope or almost like an overall


high level map of this topic or library and with that you can say okay you know I have this high


level map and now I know okay the main components in whatever this is I am learning seems to be these


three or four things so I need to learn more about these things. It is important we do this because


a lot of people might skip this and if you don't try and understand what you are applying more


deeply there is a good chance that you are not going to apply it correctly or it is going to be


a very sub-optimal thing that you are building. So I think it is really important to try and understand


things more deeply and this is going towards a theory side. So we are deepening our knowledge


understanding the theory behind whatever this application is and then once we have deepened


our knowledge we should be able to better understand the topic. And this is where we


broaden things so we then go back to the application side and we take what we have learned from the


theory and reapply it to the application and hopefully at that point we have built something


that is a bit more relevant. Now the reason that I start shallow is because for me I think


jumping straight into the theory is often overwhelming, it is confusing and there is just


too much going on. I find it really helps to have that high level view of everything first


and then from there going in and focusing on little bits. So starting shallow, keeping things


simple as possible then diving into the details and then coming back up and sort of broadening or


fixing that more high level view that you have created. The next one is courses and projects.


Now most people know and do study on courses and this is 100% the way to go. You have platforms


like Coursera and Udemy which are in my opinion amazing. You have a lot of things out there and


then you also have a lot of free courses on places like YouTube. On my channel there are even a few


free courses that you can follow along. So there really is a huge amount of content out there and


whether you want to pay for it or not you can probably find something. However doing course


after course is I don't think usually a good idea because courses are, they exist in their own sort


of isolated world. The data is usually perfect, the approach to doing something it works because


someone else has already done it and they're showing you exactly step by step how to do it.


So it's not always a good idea to just keep doing courses. Instead what I think really helps is


doing projects. So doing projects as well as courses. Projects are really good because they


emulate what you're going to be doing in real world of applying machine learning. You're actually


working on something, you're building something, you don't know if it's going to work at the start


but you're trying to figure it out. And what is very good in terms of your learning with these


projects is one I think projects can be really fun and motivating. It's really cool to start a project


and work on something and actually build something. Two they are very good at pointing out things that


you don't know. So to build something you can't just skip like a component of whatever it is


you're building. You need to be able to use it and apply it correctly. And that's why I think


projects are really good at pointing out things that you don't know. And if you don't understand


that component you will realize very quickly that you need that component in order to build whatever


it is you're building in your project. And without these projects a lot of the time you might not


have noticed that you didn't really know anything about this particular thing or maybe you didn't


know anything about it. So I think projects are really good for pointing out what you don't know.


And another really cool thing about projects is at the end of the project you actually have


something, you've built something, you can share it with people and you can be proud of whatever


it is you've built which I think is a really cool and fun feeling. As almost a continuation of the


point on projects it's also a very good idea to see if you can try and contribute to open source


projects. Now this is a pretty typical piece of advice where people say okay you should contribute


to open source and I'm including that as well because it's very good advice. If you can go ahead


and start contributing to things there are a lot of things that will happen. First you're contributing


to the open source community, you're advancing machine learning as a field which is really cool.


Two you are going to learn whatever library you are going to contribute to very well. You're going


to understand it much better than the vast majority of users, pretty much everyone other than the other


people that also contribute to that library. And you're also going to learn a lot of the best


practices from what are often some of the best ML practitioners in the world. So that is incredibly


useful even if whatever you attempt to contribute even if it isn't accepted or even if it is


accepted but heavily modified in some way that doesn't really matter so much because


either way you've learned and you have gained something from the experience.


So I wouldn't worry so much about not feeling like you're good enough to contribute to open


source because pretty much everyone even the people that do contribute to open source feel


like that it's normal. Sure you're probably not going to be as good as you know some of the


main contributors to these libraries but that's fine the majority people aren't everyone starts


somewhere so I would recommend just trying. Maybe it doesn't work out, maybe it does, either way


you're going to learn and you're going to grow from that process so that's great it's a win-win.


Point number four is to write. So I write a lot and the reason that I wrote my first book


wrote my first article was it wasn't to share anything with people or to try and make money


from articles or anything like that. The reason that I first started to write was simply because


I wanted to learn about these different things that I was writing about and the reason that I


thought okay I'm going to start writing articles is because I thought okay I thought back to


university or college and I thought about the that learning experience of writing reports on


different things and having these sort of coursework assignments and writing this report and


when you are writing those reports you realize what you don't know and you solidify these sort


of thoughts that you have on all these different topics you solidify them and as you write them


down on paper and that is really good for trying to process something and understand


something more deeply. So I really think that writing is one of the best ways to learn something


to learn something more deeply and I'm not saying you necessarily have to write articles but if you


just try and do something that emulates that university college writing experience then


I think it will probably help you a lot whether that's just personal handwritten notes or online


articles. I promise you that writing will help your learning so so much and the final point


that I want to share is to do what interests you. Now you have to be somewhat realistic with this


you can't just do whatever you want like I don't know play games all day that's not what I mean.


What I mean is when you are for example choosing what to study at university and you have all these


choices try and go for something that you actually find interesting not something that you think is


going to make you money or something like that. Of course be sensible and this is you know this is


my advice it works for me it doesn't necessarily work for everyone or even most people I just know


for me it has seemed to work just try and follow what is the most interesting to you.


Don't think so much about prestige or money or any of these other things just do what you want to do


and I say that because to be good at something it helps so much to actually be interested in that


topic and for me it wasn't until studying in my masters I realized this and during my masters I


got to choose a lot of the modules that I studied which was pretty much the first time I got to do


that and I chose a lot of the really hard modules. I was advised not to choose all of these hard


modules because it would be too much but I chose them anyway because I thought they were super


interesting and that year was you know I just got incredibly obsessed with what I was learning. I


was super interested and I found it really easy not because I didn't have to work much I worked


like crazy I did you know I basically lived in the library I worked seven days a week pretty much


every week maybe there was a odd day off probably less than once a month you know I worked like


crazy but it was super easy because I was interested and that worked out really well I got


really good grades I stumbled into AI I did my thesis in NLP and it kind of led me on the path


that has brought me to where I am now which is somewhere that I'm super happy with so doing or


following what I was interested in was for me probably the best thing I could have done. Now of


course everything varies person to person I don't know if it's the best advice for everyone but I


really do think for me at least following those interesting subjects interesting topics it just


helps so much it makes everything super super easy. So that's it you know there are a lot of


different things to talk about when it comes to learning and I realized that whilst I was putting


together what I was going to talk about in this video I dropped a lot of things so for sure in


the future I'm definitely going to do more on this sort of thing if people are interested in it but


for now I think those five points are generally pretty much what I go by what I follow now so


thank you very much for watching I hope it's been interesting and useful and I will see you again


New GPU-Acceleration for PyTorch on M1 Macs! + using with BERT

In November 2020, Apple released their latest chips, the M1 chips, based solely on Apple Silicon.


Now, the M1 chips are incredibly powerful for what they are, but they were not particularly well supported for anyone doing deep learning.


Now, TensorFlow pretty much straight out of the gate supported GPU acceleration on the new M1 chips.


But PyTorch have literally only just released it.


Now, it's been a relatively long wait and even longer because today's deep learning models rely very heavily on large model sizes and lots of data.


And in order to process that, we can't really rely on CPUs. It's incredibly slow.


So that has basically made deep learning very difficult with Macs and practically no one is going to use a Mac for deep learning when they're using PyTorch until now.


That being said, for at least the first generation M1 chips, we're probably not going to be training any large models with them anytime soon.


But we can form inference on them. And I will also show you how we can run through training, but it's not going to be anything spectacular.


So PyTorch's support for GPUs comes in version 1.12.


In the latest version of PyTorch, there comes a support for interaction between PyTorch and the lower level metal performance shaders or MPS.


So it's metal performance shaders that interact with our GPU, almost like layer between PyTorch and the GPU.


Another pretty interesting thing or positive thing about this integration is that PyTorch have collaborated directly with the Apple metal team.


The metal team deal with the new M1 chips and the MPS layer has been fine tuned for each particular M1 chips or family of M1 chips.


So that basically means the performance should be pretty optimal for what it is doing.


So if we take a look at the release announcement over on PyTorch, we can come down here and the accelerated training and evaluation.


This is on an M1 Ultra, so I believe that's the top M1 chip at the moment. It shows you pretty incredible speed up.


Now, I will say right now that this is not the same on my M1 chip or it doesn't seem to be.


So this is probably the most optimistic that you're going to get. And of course, this is in their release, so they're not going to show you the average results.


They're just showing you the best they've ever seen. So either way, this is actually pretty good.


So the game started on here is not that helpful, to be honest. This is not how you get started.


So we need to go through a few steps to get everything organized and put together.


With the new MPS enabled PyTorch, there are two key prerequisites that are not really mentioned in the announcements.


The first is that you need to have Mac OS version 12.3 or higher. So if you don't have that, you'll need to upgrade.


And the other one is that we need to do everything via the ARM version of Python.


Now, we can check all of this within our Python environment using this code here. So import platform, platform Mac version.


Right. So the first value that you get there is in this case, you can see 12.4. That is the Mac OS version. That must be 12.3 or more.


And then the last thing you see here is ARM 64. That tells me, OK, I've got the right version of Python running here.


It's using the ARM architecture rather than the x86 architecture. In that case, you need to install the new Python environment.


So if you're using Anaconda, that's great because we're going to go through the installation of ARM Python with Anaconda.


So the first thing we're doing here is specifying that we need to use the ARM 64 version of Mac, Mac OS.


So we do that here. We then say, OK, we want to create a new environment called environment ML.


We are going to use Python 3.9 and Anaconda 4 is probably in your channel list anyway.


But just in case, we're also specifying this, which is just another repository where we're going to pull all of these versions of different packages from.


OK, so once that has installed, we need to go into that environment. So conda activate ML.


And then we need to permanently modify the conda subdirectory variable to make sure that this is always going to be set to OS X ARM 64.


So this is to avoid later on when we start pip installing things in this environment. This variable may switch back to x86, which we don't want.


So we need to make sure it stays with the ARM environment architecture. So we add that in there.


And you'll probably see this where it says, please reactivate your environment. So to do that, we just do conda activate and run that.


That switches back to the base environment. And then we literally just activate ML again.


Now, the next step is to actually pip install PyTorch. And to do that, we are doing what you can see here.


So I am running pip install upgrade. You might not need the upgrade flag there, but just in case.


And we need to make sure we are going to install the nightly version of PyTorch because as of this moment, version 1.12 is only released in the nightly releases,


which is basically just a more frequent but slightly more unstable releases that PyTorch releases.


So you need all of this here. So go across and you'll be able to copy these from the notebook link that I have in the description below.


OK, so we run that. And one thing to just be aware of here. So if we have a look here, you know it's working and it's being installed in the correct version.


If you can see that it says ARM64 up here. Now, if you are just wanting to use PyTorch with NPS, that's ready.


You can go ahead and start using it and I'll show you how in a moment. But for those of you that are going to want to use Houdini Placed Transformers, there is an extra step.


If we try and pip install Transformers datasets, this will probably come up with an error for most of you.


For me, because I have already dealt with the error, it's not popping up and it would say something like error failed building wheels for tokenizers.


So the reason for that is Transformers tokenizers have particular tokenizers that are faster than other tokenizers and they are faster because they use Rust.


Now, Rust is not installed from the ARM distribution of Python that we have at the moment.


So from within our new environment, all we need to do is install Rust like this.


Once we execute that, we can just go ahead and pip install Transformers and datasets like we did before. And with that, we are ready to move on to our code.


So for this example, all we need are these libraries here with Torch Transformers datasets, which we already installed.


And we can check that our PyTorch installation has NPS using this. So Torch has NPS.


If you run that, you should see true. And that means you're OK and you're ready to go with the rest of the code.


So here I'm just pulling some data. I'm not really going to go into detail on this.


I'm not pulling much data because it's a pretty low spec M1 chip and I can't handle much.


So here I'm initializing a tokenizer model. That doesn't really make sense to you. It doesn't really matter for the sake of trying to understand PyTorch here.


I'm tokenizing my types and then this is being run on CPU. OK, this bit here.


I have not moved anything to the NPS device yet or the GPU. This is all being run on CPU.


And we get 547 milliseconds as an average time for this model processing our data.


And this is a BERT model, but BERT base on case from the Hugging Face library.


Now that's CPU performance. If we want to test with NPS, there's a few things we do.


So we have to move everything, the tensors and the model over to the NPS device.


So we set that to Torch device NPS and then we just say model to device and tokens.


So your tensors to device as well. And that's all that device is.


Now if we rerun it, we see it's faster. So we get 345 milliseconds.


So a little bit better, not a massive difference, but is a little bit better.


Now this is using a batch size of 64. I tested a few different batch sizes with this data set.


And I found that when it comes to larger batch sizes, at least for your inference,


and I imagine it's the same for training, we get a more significant difference


as we increase the number of values in each batch.


So really at 64 there's very little difference and you can kind of see that.


Then when we increase that, the difference is definitely more pronounced.


So we're not using like A100 GPUs or anything here.


We're just using the first generation MacBook Pro, the almost base specs with the M1 chip.


That's all we're using. So it's not going to blow us away.


But nonetheless, just for the sake of moving our model and tensors to a different device,


I think this is pretty cool.


Now for those of you that are interested in using this with Hugging Face,


I will quickly go through the setup for that because there's a few things to just be aware of.


So as before, I'm doing the same thing. We've got our device, it's MPS, using the same data set.


Nothing new here. And we go through everything and in this case,


anything beyond the batch size of one, so this is training the entire bare model,


anything beyond a batch size of one, it just doesn't work.


So you can see here, where's my batch? So I'm creating the batch size here,


or the data loader here, batch size of one.


We have our model, two device, because we're using MPS.


And when we're loading the different tensors for training in this training loop here,


I'm just moving them to the same MPS device.


So anything beyond a batch size of one, I even tried just two, my kernel just died.


So yeah, you just have to be wary of that.


But with one, this is actually showing a little bit lower than what I got in my other tests,


but I tried it a few times. I got around 19 minutes to train that.


Now that's a full BERT model, and we're probably not going to use MacBook,


or at least not this MacBook for training that.


Now, moving on to maybe the way that we would actually use this on Mac,


is with these NLP models, BERT and so on, we typically have two components.


We have the larger core of the model, so that would be BERT itself


that has been pre-trained by Google or Microsoft or someone else.


That includes a lot of parameters.


But then there's a smaller head on top of that, which is just like a couple of linear layers


that does something with the output from that BERT model.


So for example, it might classify the text for you.


And well, okay, we can't train a full BERT model, but we can train that head on top,


which is called fine tuning.


So let's go ahead and have a look at how we might do that.


So this is a little more complex because to do this, we need to initialize our entire model,


so the whole BERT model and the head, and then we need to freeze all of the BERT model parameters.


So an extra step, although it's not anything complicated.


So here I'm initializing my model again using BERT for sequence classification.


And this is all I'm doing.


So for param in model BERT parameters,


I'm setting the parameter that requires grad equal to false.


And taking a look at the model printout from PyTorch, we can see there's this BERT at the top,


and then there's this classification part at the bottom.


Those are all the parameters in our model.


And when we're saying model.BERT.parameters, we are accessing all the BERT parameters.


And we're leaving those classifier parameters at the end there.


So that is how that works.


And then we just go down and we would run this again.


Okay, so in this case, we have this error.


And in order to get rid of that, we actually need to downgrade to a slightly older version of the PyTorch nightly release,


because this is just a bug that's popped up in one of the more recent releases.


Hopefully by the time you're watching this, it won't be a problem anymore, so you won't get this anyway.


But if you do, this is how we fix it.


So all we do is we pip install, make sure we upgrade, and we make sure that we do this.


We don't actually need to include Torch Vision and Torch Audio here.


They're included anyway.


So we just downgrade to this nightly release.


Okay, that will fix the problem.


And then we can go back to our code and rerun everything.


Okay, so in the little GPU usage history, we can see this peak now that the model is raining.


And that's just going to basically save the top for the next four or so minutes


and go back down once we finish training.


So let's skip ahead to this finishing.


Okay, so that has just finished.


You can see GPU usage ramps down straight away.


And yeah, so it took pretty much one second shy of four minutes.


So it's relatively quick.


Nothing special, but to say I'm just on the first generation M1 using the almost base level MacBook Pro, it's not bad.


I think realistically, probably not going to do much training on the M1 chips unless you have maybe M1 Ultra.


Maybe in that case you would.


But even then, I'm not sure the chips are really at that level yet.


Nonetheless, I'm sure with the future iterations of NPS and NPS enabled PyTorch in particular,


it's probably going to improve.


So it's useful to be able to do this and even maybe just for a little bit of maybe fine tuning or at least for inference here and there.


I think this is pretty useful and it's exciting to see where this will actually go maybe in the future.


Deep learning is for or maps are for deep learning.


That would be interesting.


So yeah, it's pretty cool.


I know the sell for this is kind of finicky, so I hope this has at least helped you figure that out.


So I hope all this has been interesting.


Hybrid Search Walkthrough in Pinecone

Today we're going to have a look at how we can perform a hybrid search in Pinecone.


Hybrid Search Walkthrough in Pinecone

Now a hybrid search is where we perform a semantic search and also a keyword search.


Hybrid Search Walkthrough in Pinecone

Now we know semantic search is an incredibly useful tool that allows us to search based on the meaning or concepts rather than relying on specific keywords.


Hybrid Search Walkthrough in Pinecone

But sometimes a more traditional basic keyword search can be quite useful, particularly if you know what keywords appear in the documents that you're searching for.


Hybrid Search Walkthrough in Pinecone

So Pinecone allows you to perform a hybrid search, allowing us to both perform a semantic search and a keyword search.


Hybrid Search Walkthrough in Pinecone

Let's have a look at how that works.


Hybrid Search Walkthrough in Pinecone

So we start with our full index with all of our vectors.


Hybrid Search Walkthrough in Pinecone

And what we do is apply our keyword search to filter out irrelevant vectors from our search scope.


Hybrid Search Walkthrough in Pinecone

And then we introduce our query vector and using that query vector we find top k, in this case three most similar vectors.


Hybrid Search Walkthrough in Pinecone

And this is a semantic search portion of our query.


Hybrid Search Walkthrough in Pinecone

And those are our top k most similar results using hybrid search.


Hybrid Search Walkthrough in Pinecone

Now let's have a look at how we can actually implement that in Pinecone and start adding some basic keyword search logic in there using AND, OR, AND, NOT modifiers.


Hybrid Search Walkthrough in Pinecone

So we're going to start with a few sentences.


Hybrid Search Walkthrough in Pinecone

So here we just have 10 sentences completely random.


Hybrid Search Walkthrough in Pinecone

And what we first need to do with these sentences as we usually would with semantic search is we need to encode them.


Hybrid Search Walkthrough in Pinecone

And we'll be encoding them to produce sentence embeddings.


Hybrid Search Walkthrough in Pinecone

For that the easiest approach is to use the sentence transformers library, which you can pip install using this code here.


Hybrid Search Walkthrough in Pinecone

And of course you will also need the Pinecone client as well.


Hybrid Search Walkthrough in Pinecone

So here all we're doing is initializing a sentence transformer and we're using one of the more recent sentence transformers to produce our embeddings.


Hybrid Search Walkthrough in Pinecone

To produce our embeddings we've initialized our model up here and all we do is we call the encode method and pass all of our sentences to that.


Hybrid Search Walkthrough in Pinecone

And that will produce all of our embeddings.


Hybrid Search Walkthrough in Pinecone

And we can find a shape that it will be 10 embeddings or 10 sentence embeddings and each one of those has a dimensionality of 768.


Hybrid Search Walkthrough in Pinecone

So now what we need to do.


Hybrid Search Walkthrough in Pinecone

So that's the semantic search portion of our data.


Hybrid Search Walkthrough in Pinecone

Now we need to deal with the keyword search portion of our data.


Hybrid Search Walkthrough in Pinecone

So when we upset our data to Pinecone we're going to need to include a list of tokens or a list of words so that we can then use that list of words to filter and perform our keyword search.


Hybrid Search Walkthrough in Pinecone

So to build that list of tokens for each one of our sentences we're going to use Hugging Face's transformers library.


Hybrid Search Walkthrough in Pinecone

So for that we're going to write for from transformers import and we're going to import the auto tokenizer class.


Hybrid Search Walkthrough in Pinecone

Now it's important that we use a tokenizer that uses word level tokenization because many of these tokenizers do not split sentences into words but they split into sub words or even byte level encodings.


Hybrid Search Walkthrough in Pinecone

So we need to make sure that we're using a word level tokenizer.


Hybrid Search Walkthrough in Pinecone

And that is what this model here is.


Hybrid Search Walkthrough in Pinecone

So this transfer XLWT103.


Hybrid Search Walkthrough in Pinecone

The tokenizer for that is a word level tokenizer.


Hybrid Search Walkthrough in Pinecone

So we initialize that and we'll put all of our tokens within a variable called all tokens.


Hybrid Search Walkthrough in Pinecone

And we'll use less comprehension. All we need to do is write tokenizer and we use the tokenize method.


Hybrid Search Walkthrough in Pinecone

And then in here we want to pass our sentence.


Hybrid Search Walkthrough in Pinecone

We also need to lowercase it because this tokenizer will not lowercase our text by default.


Hybrid Search Walkthrough in Pinecone

So we just handle that and we do that for each sentence in all of our sentences.


Hybrid Search Walkthrough in Pinecone

Let's have a look at what that looks like for our first sentence.


Hybrid Search Walkthrough in Pinecone

OK. So we see that we've split our first sentence which you can see up here into a list of words which is exactly what we need in Pinecone to perform a keyword search.


Hybrid Search Walkthrough in Pinecone

So that's everything we need in terms of data. We have our dense vector representations, the sentence embeddings and we also have our keywords, the list of tokens.


Hybrid Search Walkthrough in Pinecone

So let's continue and we will connect to a Pinecone instance.


Hybrid Search Walkthrough in Pinecone

If you haven't used Pinecone before you can get a free API key over here.


Hybrid Search Walkthrough in Pinecone

So we run our initialization cell and then what we'll need to do is create a new index.


Hybrid Search Walkthrough in Pinecone

Now before we create that index what I'm going to do is list all of my current indexes to make sure I don't overwrite any existing indexes.


Hybrid Search Walkthrough in Pinecone

Now I don't have any at the moment so that's fine. I can call this whatever I want.


Hybrid Search Walkthrough in Pinecone

But I'm going to go with keyword search. Now you can name this anything you like. You don't have to use the same name as what I'm using here.


Hybrid Search Walkthrough in Pinecone

So I'm going to do is create the index and then after that I initialize my connection to that index.


Hybrid Search Walkthrough in Pinecone

So I run both of those and then just note here I'm passing the vector dimensionality when I create the index there.


Hybrid Search Walkthrough in Pinecone

And we can check this so this will be this 768 that we saw earlier.


Hybrid Search Walkthrough in Pinecone

OK so you can see the 768 there.


Hybrid Search Walkthrough in Pinecone

And now we want to do is merge all the data that we've created so far. So when we upset data to Pinecone we want a list of tuples.


Hybrid Search Walkthrough in Pinecone

Each one of those tuples is going to contain an ID, a value which is our sentence embedding and also any metadata.


Hybrid Search Walkthrough in Pinecone

Now the tokens that we're creating we will include within the metadata field.


Hybrid Search Walkthrough in Pinecone

And we'll include that using this format here so we can imagine within that metadata field for every single record or sample.


Hybrid Search Walkthrough in Pinecone

We are going to have this tokens and that will map to the list of tokens for each sentence.


Hybrid Search Walkthrough in Pinecone

So we'll execute that and then we upset all that to our index.


Hybrid Search Walkthrough in Pinecone

And we'll see a little response here telling us how many samples or records we upset.


Hybrid Search Walkthrough in Pinecone

Which in this case is 10 as we would expect.


Hybrid Search Walkthrough in Pinecone

Now alternatively if you like you can also upset with curl and for that you just reformat your data into a dictionary format.


Hybrid Search Walkthrough in Pinecone

Save it to a JSON object and then upset it using this curl command here.


Hybrid Search Walkthrough in Pinecone

Now the URL that you see here you will have to go into your Pinecone dashboard and find the URL for your index.


Hybrid Search Walkthrough in Pinecone

Now we've upset all the data into our index.


Hybrid Search Walkthrough in Pinecone

So let's go ahead and start querying.


Hybrid Search Walkthrough in Pinecone

So the first thing we need to do is create a query sentence.


Hybrid Search Walkthrough in Pinecone

So we just have this string here.


Hybrid Search Walkthrough in Pinecone

And what we do is we encode that using the same model that we used earlier to encode all of our sentences.


Hybrid Search Walkthrough in Pinecone

We then convert that to a list because it is otherwise a numpy array and we need to make sure we are sending our requests with the list.


Hybrid Search Walkthrough in Pinecone

So we execute that.


Hybrid Search Walkthrough in Pinecone

And let's start with a simple query without any keyword search at the moment.


Hybrid Search Walkthrough in Pinecone

So we pass our query vector execute.


Hybrid Search Walkthrough in Pinecone

We say we'd like to return the top k results.


Hybrid Search Walkthrough in Pinecone

I'm going to set that equal to 10 so we're just returning everything.


Hybrid Search Walkthrough in Pinecone

And I'm going to include metadata just for this column and then I'll remove this just so you can see what we have in our index.


Hybrid Search Walkthrough in Pinecone

OK so we can see we have our ID in here and we also have inside this metadata field we have all our tokens.


Hybrid Search Walkthrough in Pinecone

OK and it's using these tokens here that will be performing our keyword search.


Hybrid Search Walkthrough in Pinecone

And if we just save this and if we just iterate through those results.


Hybrid Search Walkthrough in Pinecone

We like this for x in.


Hybrid Search Walkthrough in Pinecone

We can just have a look here, results, we see we have results and then we want to enter the zero index of that list.


Hybrid Search Walkthrough in Pinecone

And then we're going to matches to get to the records that have been returned to us.


Hybrid Search Walkthrough in Pinecone

So we write in results, results, zero and matches.


Hybrid Search Walkthrough in Pinecone

You can see here that we're returning the 10 IDs of our sentences.


Hybrid Search Walkthrough in Pinecone

So what we now want to do is move on to actually implementing a keyword search.


Hybrid Search Walkthrough in Pinecone

So we'll make it a very simple query to start with.


Hybrid Search Walkthrough in Pinecone

So the index.query, xq top k.


Hybrid Search Walkthrough in Pinecone

We'll set that 10 again, we're just returning everything and then we can set our filter.


Hybrid Search Walkthrough in Pinecone

And it's through this filter that we perform our keyword search.


Hybrid Search Walkthrough in Pinecone

So we want to return only records where within tokens there is the word bananas.


Hybrid Search Walkthrough in Pinecone

And again, we will get these IDs from here.


Hybrid Search Walkthrough in Pinecone

We're going to store them in the IDs variable. Let's have a look what we get.


Hybrid Search Walkthrough in Pinecone

So you can see straight away we're restricting our search and there are only four records that contain the word bananas.


Hybrid Search Walkthrough in Pinecone

So we're now restricting our search.


Hybrid Search Walkthrough in Pinecone

So what we can now do is for i in IDs, I'm going to print each one of those sentences.


Hybrid Search Walkthrough in Pinecone

So we have all sentences i.


Hybrid Search Walkthrough in Pinecone

We can make sure that we're converting this back to an integer value.


Hybrid Search Walkthrough in Pinecone

And now we return those sentences.


Hybrid Search Walkthrough in Pinecone

So we can see each one of these just contain the word bananas.


Hybrid Search Walkthrough in Pinecone

Now what we might say is that we'd like to return sentences where we have one of two words.


Hybrid Search Walkthrough in Pinecone

So we're going to do bananas and weigh this time.


Hybrid Search Walkthrough in Pinecone

So we're going to introduce the or logic.


Hybrid Search Walkthrough in Pinecone

Now we take this code here.


Hybrid Search Walkthrough in Pinecone

And let's take this as well.


Hybrid Search Walkthrough in Pinecone

And what we're going to do in our filter here, we are going to modify this to use the or modifier.


Hybrid Search Walkthrough in Pinecone

So we have or and then using or we can pass a list of conditions.


Hybrid Search Walkthrough in Pinecone

And if any one of these conditions is true, we will return that record.


Hybrid Search Walkthrough in Pinecone

So we want to say tokens contains bananas or tokens contains weigh.


Hybrid Search Walkthrough in Pinecone

And let's return and see what we get.


Hybrid Search Walkthrough in Pinecone

So you see that we're returning one new sentence, which is this one here.


Hybrid Search Walkthrough in Pinecone

Which does not contain bananas, but it does contain weigh.


Hybrid Search Walkthrough in Pinecone

Now that's using the or statement here.


Hybrid Search Walkthrough in Pinecone

But we can also use, which is probably simpler, we can also write this using the in modifier.


Hybrid Search Walkthrough in Pinecone

So we first write tokens and then we say within tokens, within or in.


Hybrid Search Walkthrough in Pinecone

We want to search for any records that contain either bananas or weigh.


Hybrid Search Walkthrough in Pinecone

And this will produce the exact same results as what we got before.


Hybrid Search Walkthrough in Pinecone

So you see we return those same five sentences.


Hybrid Search Walkthrough in Pinecone

So that's the two tool returns we have for all logic in our keyword search.


Hybrid Search Walkthrough in Pinecone

And let's copy this one.


Hybrid Search Walkthrough in Pinecone

And what we're going to do is just modify or and replace it with and.


Hybrid Search Walkthrough in Pinecone

So now we're saying we only want to return sentences that contain both the word bananas and also the word weigh.


Hybrid Search Walkthrough in Pinecone

So we do that and we see now we're only returning these two which contain both of those words.


Hybrid Search Walkthrough in Pinecone

Now another thing that you might want to add here is let's say maybe we do want the word weigh.


Hybrid Search Walkthrough in Pinecone

But we also want to specify that we actually don't want any records that contain the word bananas.


Hybrid Search Walkthrough in Pinecone

And again we can still use the and statement here.


Hybrid Search Walkthrough in Pinecone

And the only thing we actually need to change is we have to add a not equals any to the bananas condition.


Hybrid Search Walkthrough in Pinecone

And this will invert that single condition here.


Hybrid Search Walkthrough in Pinecone

OK so now we're searching for any records that do not contain bananas and contain the word weigh.


Hybrid Search Walkthrough in Pinecone

So we execute that and we'll see that there's only actually one of those.


Hybrid Search Walkthrough in Pinecone

So there's only one sentence that contains the word weigh and does not contain the word bananas.


Hybrid Search Walkthrough in Pinecone

And what if we'd like to negate both of these.


Hybrid Search Walkthrough in Pinecone

Well we could just add this any to the weigh condition as well.


Hybrid Search Walkthrough in Pinecone

Or what is simpler using the not in modifier.


Hybrid Search Walkthrough in Pinecone

So actually what we can do is we'll come up here and you see we have the in modifier here.


Hybrid Search Walkthrough in Pinecone

Very very similar. All we need to do is bring that down here and replace in with not in.


Hybrid Search Walkthrough in Pinecone

And if we then search what we're doing here is searching for any records that do not contain the word bananas or the word weigh.


Hybrid Search Walkthrough in Pinecone

So here we're saying any sentences that contain just one of these words we're not interested. We exclude those.


Hybrid Search Walkthrough in Pinecone

So that's it for this introduction to hybrid search using both semantic search and keyword search in pinecone.


Hybrid Search Walkthrough in Pinecone


How-to Decode Outputs From NLP Models (Python)

Hi, welcome to this video on decoding methods for text generation.


So when we're performing a lot of different NLP tasks, a lot of people focus very much on the


actual model, which is fair enough. The model is very important. But a lot of people, at least when


they're self-learning, they miss the fact that you can make a big impact on your output depending on


how you're decoding your text or your tokens at the end of your model. What I mean by that is you


have your model here and then when you're outputting that you typically have a probability distribution


across your vocabulary or output vocabulary. And it is that probability distribution that allows


you to choose what token you're going to output. So we're going to focus on a few different ways


of doing that because there are quite a few different ways. The three ways that we're going


to look at include greedy decoding, random sampling and beam search. And generally I think once you


know these three, you're pretty good to go and you can make a really big difference. You'll see in


this video in the actual outputs of your models. So let's just jump straight into it. Here we just


have the setup. So we have our model setup and it's just generating text based on this input here,


which is just an extract from the Wikipedia page about Winston Churchill. We have this,


we've encoded our input here and all we need to do to actually generate our outputs from this


is we will output the generated tokens to outputs and we just call the generate method


on our model and then we pass in the inputs and then we just pass in the length that we want to


use. So the number of tokens that we actually want to output. So I'm just going to go 200.


Okay. And then to actually read that, we need to decode it. Okay. And that is our text. So I just


want to point out here that when we're calling this decode method, that is not when we are using


these decoding methods that this video is about. That is just decoding using our tokenizer. So


taking the token IDs that we do have, our predicted token IDs and decoding them into English. The


decoding methods that we're talking about take place at the end of this generate method here.


So by default, this is greedy decoding and greedy decoding is I would say the simplest approach.


And it's the one that you would probably think of when you try and just figure this out through


intuition. So obviously we have our probability distribution and we have our output vocabulary


and each value within that output distribution represents a mapping to one of our words or tokens.


What greedy decoding does is simply chooses the token that has the highest probability,


which obviously makes a lot of sense. And for shorter sequences, this is perfectly fine. But


then when we start applying this to longer sequences, it causes a lot of issues. If you've


ever seen a output that looks like this, where you have a lot of repetition, it just keeps repeating


itself over and over again. This is typically because of greedy decoding. So when we're looking


at longer sequences, what we will find is that it creates kind of like a loop. So it sees, you know,


the best match for one word and then it reads the best match for the next word. And then as soon as


it sees that first word again as the next best match, it's just going to create a loop. So that's


why we want to avoid greedy decoding for longer sequences. So the next option we have is random


sampling. Like before, we still have our outputs and we have that probability distribution across


all of those. And what random sampling does differently is it chooses one of these tokens


based on that probability at random. So what I mean by that is it will choose a token at random,


so it can choose any of those. But the probability of choosing that token is weighted based on the


probability of that token being the next token from the model. It's very similar to greedy


decoding, but it just adds that sort of layer of randomness where every now and again, it's going


to choose something other than the most probable token. And all we do to actually use random


sampling is add a do sample argument here and set this to true. And this will switch our


decoding method from greedy decoding to random sampling. And we will see that the output of this


is not as repetitive. So if we just decode this here, okay, and we can see that it's a lot more


random. So this solves our problem of getting stuck in a repeating loop of the same words over


and over again, because we just add this randomness to our predictions. However, this does introduce


another problem, which is we will often find that the outputs we are producing are too random,


and they're just not coherent because of that. This one is reasonably coherent. It's still talking


about UK politics. Sometimes it gets super weird. It will start talking about chameleons and game


sales and stuff like that within, you know, a few sentences. It was actually talking about Churchill.


So on one side, we have greedy search, which is too strict, and it just causes a loop of the same


words over and over again. And on the other side, we have random sampling, which is too random and


ends up just producing gibberish almost all the time. So what we need is something in the middle.


And that is why we use beam search. So beam search allows us to explore multiple levels


of our output before selecting the best option. Whereas greedy decoding and random sampling is


just looking at the very next token and then calculates which one to choose. What beam search


does is it goes ahead in time. It searches through multiple potential paths and then finds the best


option based on the full sequence rather than just the next token. And this just allows us to


assess multiple different options and assess a longer length of time than just one token,


which means typically we're going to see more coherent language from these outputs as well.


And the beam in beam search is all that's referring to is the number of paths that we


assess and we consider. To add beam search, all we need to do is add number of beams.


And we just set that to a value that is more than one because a beam of one is actually just


the default. So if we set this to two, we then search two different beams or two different options


at once. And this usually is pretty good. However, because we're now back to


ranking sequences and selecting the most probable, beam search can quite easily fall


back into this repetitive nature that we get with greedy decoding. So what we need to do to


counteract that is add the temperature argument to our code. So the temperature essentially


controls the amount of randomness within the beam search choice. And by default, this is set to one.


We can add more randomness to the output by increasing this, so say to 1.2. And this will


just make the outputs more random. But because we're using beam search, it will still remain


reasonably coherent unless we really turn the temperature up to a very high number,


like five or something. And if we want to reduce the randomness in our outputs,


we just reduce the temperature to something below one, so 0.8, for example. So with this,


I mean, we have our beam search, we have some good outputs and somewhat peculiar, but they're


pretty coherent, so it's pretty good. I mean, I think that's everything for this video. I think


these three decoding methods are pretty important to know. They can make a big difference in the


quality of your outputs. So thank you very much for watching, and I will see you again next time.


How to learn NLP for free

Today I want to talk about something a little bit different.


How to learn NLP for free

And instead of focusing on a specific topic,


How to learn NLP for free

I want to actually talk about how I would learn machine learning,


How to learn NLP for free

and in particular NLP, if I was starting from scratch.


How to learn NLP for free

I've tried to keep this almost like guide as simple as possible,


How to learn NLP for free

because I think a lot of learning in terms of machine learning,


How to learn NLP for free

data science, NLP, whatever else,


How to learn NLP for free

it really needs to come from your own interests.


How to learn NLP for free

And so I think a very strict order of things to do


How to learn NLP for free

can almost ruin that learning experience.


How to learn NLP for free

And so it's very important to just try and kind of figure out


How to learn NLP for free

things out as you go.


How to learn NLP for free

But at the same time, I do think it's important to be aware of


How to learn NLP for free

maybe a few particular courses or resources or books


How to learn NLP for free

that have been very helpful for others in the past


How to learn NLP for free

and therefore could also help you.


How to learn NLP for free

So I've broken it down into kind of four parts.


How to learn NLP for free

So there's the machine learning 101, applied ML, the origin of,


How to learn NLP for free

what I would say, modern NLP, and also just modern NLP.


How to learn NLP for free

And then after that, I've also added in another bit.


How to learn NLP for free

So this is obviously very specific,


How to learn NLP for free

and we'll just mention this at the end,


How to learn NLP for free

but we won't really go into too much depth on it.


How to learn NLP for free

So each of these parts here, there's almost like a stage


How to learn NLP for free

in your journey of learning.


How to learn NLP for free

And of course, at the start, you actually need to learn


How to learn NLP for free

about machine learning.


How to learn NLP for free

And this is almost not even the start.


How to learn NLP for free

So yes, you would start here,


How to learn NLP for free

but you also need to learn Python, for example.


How to learn NLP for free

So in reality, it might actually look something like this.


How to learn NLP for free

So you would start with ML 101,


How to learn NLP for free

and at the same time in parallel,


How to learn NLP for free

you might want to be learning Python.


How to learn NLP for free

Now, the reason I say in parallel,


How to learn NLP for free

rather than just starting with Python,


How to learn NLP for free

is more to do with whether you're interested in that or not.


How to learn NLP for free

Because for me, I started Python with the specific interest of,


How to learn NLP for free

like algorithmic trading, and then very quickly


How to learn NLP for free

that moved into sort of more machine learning.


How to learn NLP for free

But learning Python for the sake of Python,


How to learn NLP for free

I wasn't necessarily that interested in,


How to learn NLP for free

although I will say that once I got started,


How to learn NLP for free

I very quickly became very obsessed


How to learn NLP for free

with just coding random things.


How to learn NLP for free

So in my personal opinion,


How to learn NLP for free

and that's not necessarily going to be right for everyone,


How to learn NLP for free

I would kind of learn those two things in parallel,


How to learn NLP for free

because you have to learn Python side


How to learn NLP for free

for the machine learning side,


How to learn NLP for free

but they both complement each other,


How to learn NLP for free

and having both of them in parallel


How to learn NLP for free

and just doing Python will probably keep you more interested


How to learn NLP for free

than just focusing on one at a time.


How to learn NLP for free

So in terms of resources for just Python,


How to learn NLP for free

I use Codecademy more than anything else.


How to learn NLP for free

Now, when I use it, it was, I think, cheaper than it is now.


How to learn NLP for free

So something to be aware of,


How to learn NLP for free

but there are so many good resources out there


How to learn NLP for free

for learning Python.


How to learn NLP for free

You probably aren't going to struggle to find anything.


How to learn NLP for free

The other thing I would say is also try projects.


How to learn NLP for free

So just building projects will help your coding skills


How to learn NLP for free

and everything else.


How to learn NLP for free

Your coding skills and every other component on here,


How to learn NLP for free

so much because it shows you where you're missing knowledge


How to learn NLP for free

and also helps you learn how to apply what you have learned.


How to learn NLP for free

And at the same time,


How to learn NLP for free

I think it's very motivating to build something.


How to learn NLP for free

At least for me, I find it incredibly motivating


How to learn NLP for free

to just build something cool that I find interesting.


How to learn NLP for free

So that's another thing that I would definitely recommend


How to learn NLP for free

on the Python side.


How to learn NLP for free

So yeah, that's a prerequisite almost,


How to learn NLP for free

or something you do in parallel to what I have here.


How to learn NLP for free

So Machine Learning 101 is really kind of an introduction


How to learn NLP for free

to machine learning


How to learn NLP for free

and where you would learn about key concepts.


How to learn NLP for free

So you have neural networks up here,


How to learn NLP for free

loss optimization, activation functions,


How to learn NLP for free

and there is a lot more than just that in there.


How to learn NLP for free

So with all of these resources,


How to learn NLP for free

you can mostly get them for free.


How to learn NLP for free

So Neural Networks from Scratch is very good by Centex.


How to learn NLP for free

And Centex in general is also very good,


How to learn NLP for free

especially when you're starting Python.


How to learn NLP for free

That's probably another thing to add to the Python prerequisites.


How to learn NLP for free

You can also learn from Centex.


How to learn NLP for free

That's one of the places that I learned as well.


How to learn NLP for free

So Neural Networks from Scratch is, in my opinion,


How to learn NLP for free

a really good resource.


How to learn NLP for free

And it's one of those things that wasn't around


How to learn NLP for free

when I was learning this stuff,


How to learn NLP for free

but I really wish it had been.


How to learn NLP for free

And so much so that I bought it anyway.


How to learn NLP for free

And it is genuinely incredible.


How to learn NLP for free

Now, you don't need to buy the book as well.


How to learn NLP for free

The book is great, and I like having something physical.


How to learn NLP for free

But you can also do all this stuff online as well.


How to learn NLP for free

Or not all of it.


How to learn NLP for free

I think the first parts of the book has already been covered


How to learn NLP for free

on Centex's YouTube channel.


How to learn NLP for free

So where do we find that?


How to learn NLP for free

I think it's just YouTube Centex.


How to learn NLP for free

Yeah, so just Centex.


How to learn NLP for free

And you can find on here,


How to learn NLP for free

here it's literally right at the top.


How to learn NLP for free

Neural Networks from Scratch, intro Neuron Code.


How to learn NLP for free

And this is really nice as well


How to learn NLP for free

because you have a lot of visuals in here.


How to learn NLP for free

And obviously Centex is talking through it,


How to learn NLP for free

which is really nice.


How to learn NLP for free

But of course, also the book is really incredible.


How to learn NLP for free

And the reason I like it is because


How to learn NLP for free

it's going through the code of everything,


How to learn NLP for free

making everything very visual,


How to learn NLP for free

but breaking everything down.


How to learn NLP for free

Because what you will find later on in machine learning,


How to learn NLP for free

that everything is very abstract,


How to learn NLP for free

but it's so important to understand the core concepts


How to learn NLP for free

behind all of these things.


How to learn NLP for free

So we can see here,


How to learn NLP for free

like it's really breaking down everything.


How to learn NLP for free

So this looks horrible,


How to learn NLP for free

but this is a really nice way to actually just understand


How to learn NLP for free

what is going on in a neural network.


How to learn NLP for free

And then later on with like actually coding


How to learn NLP for free

and keeping things really simple,


How to learn NLP for free

like inputs, weights.


How to learn NLP for free

And the issue is that in most cases,


How to learn NLP for free

most tutorials, they go too abstract, too quickly.


How to learn NLP for free

Whereas in this book and on the video tutorials,


How to learn NLP for free

Centex doesn't go too abstract too quickly.


How to learn NLP for free

He builds it up from nothing,


How to learn NLP for free

which is really nice,


How to learn NLP for free

especially if you're just starting out.


How to learn NLP for free

So that's one, in my opinion,


How to learn NLP for free

very good resource for when you're starting out.


How to learn NLP for free

Another one that it seems almost like everyone


How to learn NLP for free

in machine learning or data science


How to learn NLP for free

has done this course at some point.


How to learn NLP for free

And it is, I will be honest, it is outdated,


How to learn NLP for free

but not so much in, it's not too outdated.


How to learn NLP for free

It's still very valuable.


How to learn NLP for free

So the things that you learn on this course


How to learn NLP for free

very much focus on the concepts behind


How to learn NLP for free

much of the older stuff in machine learning


How to learn NLP for free

and particularly the things that make machine learning


How to learn NLP for free

what it is today.


How to learn NLP for free

Now, one of the main reasons I say this is outdated


How to learn NLP for free

is that it uses a programming language called Octave,


How to learn NLP for free

which is almost like an open source language.


How to learn NLP for free

MATLAB, if any of you know what MATLAB is.


How to learn NLP for free

So in reality, it is actually very similar to Python syntax.


How to learn NLP for free

So it's not so much of a bad thing that you use Octave


How to learn NLP for free

because it translates over very easily.


How to learn NLP for free

And one thing that is great about this course


How to learn NLP for free

is that you can actually enroll for free.


How to learn NLP for free

I don't know if it's free the entire way through,


How to learn NLP for free

it's been ages since I did this course,


How to learn NLP for free

but either way you can order everything on almost,


How to learn NLP for free

at least almost everything on Coursera for free.


How to learn NLP for free

It just means you don't get a certificate at the end,


How to learn NLP for free

but that's completely fine.


How to learn NLP for free

And if you really wanted a certificate at the end of it,


How to learn NLP for free

you can obviously pay for it.


How to learn NLP for free

But this is an incredible course.


How to learn NLP for free

And you can see some of the things


How to learn NLP for free

that go through linear algebra.


How to learn NLP for free

This is so important,


How to learn NLP for free

just a little bit of understanding of the maths behind all this


How to learn NLP for free

to start with, logistic regression,


How to learn NLP for free

logistic regression, regularization.


How to learn NLP for free

They go through so many important concepts


How to learn NLP for free

that honestly like absolutely crucial for later on.


How to learn NLP for free

So this is, you know, without a doubt,


How to learn NLP for free

one of the best courses when you're getting started.


How to learn NLP for free

I think it's really, really good.


How to learn NLP for free

And then the final one that I wanted to talk about


How to learn NLP for free

is a 100 page machine learning book.


How to learn NLP for free

Again, obviously this is a book.


How to learn NLP for free

I do think you can get it online as far as I know.


How to learn NLP for free

So I'd have a look at that.


How to learn NLP for free

At one point you could for sure, there was a PDF.


How to learn NLP for free

But this is really nice because it's 100 pages.


How to learn NLP for free

Well, I think maybe there's a few more than 100.


How to learn NLP for free

Yeah, okay, so it goes kind of over 100 pages.


How to learn NLP for free

But because it's such a small number of pages,


How to learn NLP for free

everything in here is very important.


How to learn NLP for free

So it's kind of stripped down everything you need,


How to learn NLP for free

just so you have all the important things.


How to learn NLP for free

So we can go ahead maybe find something interesting here.


How to learn NLP for free

Let's see what this is here that I made a note of a long time ago.


How to learn NLP for free

Okay, so I'm looking at like just classification first regression.


How to learn NLP for free

This is like just a few paragraphs explaining the difference


How to learn NLP for free

between these two incredibly important things.


How to learn NLP for free

And then we have parameters versus hyperparameters.


How to learn NLP for free

It's just so much important stuff.


How to learn NLP for free

And you have charts and visuals,


How to learn NLP for free

which I personally really like to have visuals in everything


How to learn NLP for free

as much as possible.


How to learn NLP for free

So this is another really, really cool book


How to learn NLP for free

that I would 100% recommend.


How to learn NLP for free

It's also code as well.


How to learn NLP for free

Is it Python? Yeah.


How to learn NLP for free

So it's really nice, in my opinion.


How to learn NLP for free

And once you have gone past that,


How to learn NLP for free

you kind of want to start looking at applied machine learning.


How to learn NLP for free

So this is the only thing on this list


How to learn NLP for free

I haven't personally gone through,


How to learn NLP for free

although I have watched a few videos from this.


How to learn NLP for free

And the reason I recommend it is because Daniel Burke


How to learn NLP for free

is absolutely incredible at taking machine learning


How to learn NLP for free

and stripping down everything you don't need to know


How to learn NLP for free

and just keeping what you do need to know


How to learn NLP for free

and teaching you in an incredibly entertaining way.


How to learn NLP for free

So this bootcamp course as well,


How to learn NLP for free

you can, I think, possibly also find it on Udemy.


How to learn NLP for free

It has an incredibly high rating.


How to learn NLP for free

Every course that Daniel Burke is on is incredibly high rating.


How to learn NLP for free

People are super happy with it.


How to learn NLP for free

And it's really friendly to beginners.


How to learn NLP for free

So that's why I recommend this,


How to learn NLP for free

even though I haven't been all the way through it myself.


How to learn NLP for free

But either way, I'm very confident


How to learn NLP for free

that this is a very good resource


How to learn NLP for free

for those of you that are interested.


How to learn NLP for free

Now, but of course, it is up to you.


How to learn NLP for free

And another thing that I would say here is,


How to learn NLP for free

okay, I think it's very important to learn this sort of thing.


How to learn NLP for free

But at the same time, if you're really just kind of impatient,


How to learn NLP for free

I mean, it's probably best you do it,


How to learn NLP for free

but if you are super impatient, fine.


How to learn NLP for free

You can probably skip it and move on to the modern NLP stuff.


How to learn NLP for free

Because all the code in modern NLP is actually quite simple.


How to learn NLP for free

So you can maybe drop that bit just as long as you're aware


How to learn NLP for free

that there's going to be a lot of things that come up


How to learn NLP for free

where you're going to be quite confused.


How to learn NLP for free

Like pandas and NumPy if you haven't seen them before.


How to learn NLP for free

But nonetheless, you will learn a bit of NumPy in these early ones,


How to learn NLP for free

especially neural networks from scratch.


How to learn NLP for free

So it's kind of possible to skip it if you're just impatient,


How to learn NLP for free

which is fine.


How to learn NLP for free

I'm impatient all the time as well.


How to learn NLP for free

I just want to learn new things.


How to learn NLP for free

So then the next thing is, if you're really going for NLP,


How to learn NLP for free

I think it's really good to learn what is prep code.


How to learn NLP for free

What is prep current and present in NLP.


How to learn NLP for free

But at the same time, to really understand any of that,


How to learn NLP for free

it's very important to understand where it came from.


How to learn NLP for free

So I would definitely recommend Sanford CS224N.


How to learn NLP for free

It's incredible on YouTube.


How to learn NLP for free

That will take you through so many incredibly useful things in NLP


How to learn NLP for free

that are taught by some of the best people in the world.


How to learn NLP for free

And it's very relevant.


How to learn NLP for free

It's incredibly useful.


How to learn NLP for free

The only thing is, at least when I went through this course,


How to learn NLP for free

which to be fair was a few years ago,


How to learn NLP for free

they didn't really have anything on the most recent stuff in NLP.


How to learn NLP for free

That might have changed now.


How to learn NLP for free

And that is why I have kept it over here.


How to learn NLP for free

Because if they do have anything on the more recent stuff in NLP,


How to learn NLP for free

I think it's very little.


How to learn NLP for free

So that's why I kept it in sort of more the origin stuff,


How to learn NLP for free

because it mainly seems to focus on things that you see here,


How to learn NLP for free

like word vectors, recurrent neural networks, function gradient,


How to learn NLP for free

and maybe they go into attention near the end.


How to learn NLP for free

Although I can't say for sure there.


How to learn NLP for free

And the next thing that I would recommend,


How to learn NLP for free

and that I also did, so I did this course,


How to learn NLP for free

and I found this to be very good.


How to learn NLP for free

Over on Coursera, again, I do believe if you look in


How to learn NLP for free

enrolment options down here or come down here,


How to learn NLP for free

you can audit the course.


How to learn NLP for free

Can I take the course for free?


How to learn NLP for free

You can audit the course for free.


How to learn NLP for free

See, now I'm not sure how you do that.


How to learn NLP for free

It's somewhere on here.


How to learn NLP for free

They keep it kind of hidden.


How to learn NLP for free

But you can audit it for free.


How to learn NLP for free

So that's incredibly useful.


How to learn NLP for free

And this is a really good course.


How to learn NLP for free

It's actually specialization.


How to learn NLP for free

So it's multiple courses in one.


How to learn NLP for free

Again, they don't really go into the more modern stuff,


How to learn NLP for free

although I do know that they, I believe,


How to learn NLP for free

actually paired it with Hugging Face,


How to learn NLP for free

which is like modern NLP,


How to learn NLP for free

and did introduce some of that at some point.


How to learn NLP for free

So yeah, sort of attention models.


How to learn NLP for free

Yeah, they've brought in T5 and Vert,


How to learn NLP for free

and reforming models as well.


How to learn NLP for free

So that's pretty cool.


How to learn NLP for free

So even at the end of this,


How to learn NLP for free

you will get to learn some of the more recent NLP stuff.


How to learn NLP for free

So then the final bit is modern NLP.


How to learn NLP for free

So in here, there are a few things.


How to learn NLP for free

So how transformers work.


How to learn NLP for free

So this is an article I wrote.


How to learn NLP for free

And of course, there'll be links to this


How to learn NLP for free

that you can access for free.


How to learn NLP for free

And in this, I just wanted to explain


How to learn NLP for free

where transformers came from


How to learn NLP for free

in terms of going from recurrent neural networks


How to learn NLP for free

through to adding attention to those recurrent neural networks,


How to learn NLP for free

encoded decoder models, and how that attention works.


How to learn NLP for free

And I mean, obviously I'm biased,


How to learn NLP for free

but I think this is a relatively good summary


How to learn NLP for free

of what a transformer model is.


How to learn NLP for free

And transform models are really the foundation


How to learn NLP for free

of modern NLP.


How to learn NLP for free

They're incredibly important.


How to learn NLP for free

So I think that's quite good just to read through that


How to learn NLP for free

and try and understand what is actually going on.


How to learn NLP for free

And then there's two courses here.


How to learn NLP for free

So there's my course here.


How to learn NLP for free

You can always find discount on this,


How to learn NLP for free

but I don't necessarily think you need to even pay


How to learn NLP for free

because there's a Hugging Face course.


How to learn NLP for free

And the Hugging Face course will cover everything


How to learn NLP for free

I cover in my course anyway and more.


How to learn NLP for free

The only thing is that maybe my course is


How to learn NLP for free

maybe a little more applied


How to learn NLP for free

and maybe a little more concise in some parts.


How to learn NLP for free

And as well, a lot of the stuff I cover in my course


How to learn NLP for free

I also cover on YouTube for free.


How to learn NLP for free

So you just have a bit more of a structure, I suppose.


How to learn NLP for free

So the Hugging Face course is, like I said, incredible.


How to learn NLP for free

You have a nice little summary of the course here,


How to learn NLP for free

which I can't...


How to learn NLP for free

Can I zoom in?


How to learn NLP for free

That's probably the closest I'm going to get.


How to learn NLP for free

So you have an introduction, diving in,


How to learn NLP for free

and more advanced stuff.


How to learn NLP for free

And you go through everything.


How to learn NLP for free

So using transformers, Hugging Face,


How to learn NLP for free

they have multiple libraries for NLP.


How to learn NLP for free

Transformers library, datasets library, tokenizers library.


How to learn NLP for free

They're all very good.


How to learn NLP for free

I would definitely recommend this


How to learn NLP for free

as that sort of starting point,


How to learn NLP for free

or as the almost the foundation of where you learn


How to learn NLP for free

your modern NLP techniques.


How to learn NLP for free

So that's the modern NLP stuff.


How to learn NLP for free

And yeah, as I have written here,


How to learn NLP for free

I think at this point you're definitely in a good position


How to learn NLP for free

to do some pretty interesting projects.


How to learn NLP for free

So at this point,


How to learn NLP for free

I would expect you've probably done some projects anyway.


How to learn NLP for free

But if you haven't, then for sure at this point


How to learn NLP for free

you really should do some projects.


How to learn NLP for free

So these are just projects that I've done in the past


How to learn NLP for free

and actually showed you how to do on YouTube.


How to learn NLP for free

So there's building transform models from scratch.


How to learn NLP for free

I think this is really useful to understand


How to learn NLP for free

how all of this works.


How to learn NLP for free

There's a few things in there.


How to learn NLP for free

So this is like a building an Italian BERT model from scratch.


How to learn NLP for free

And I do have other series as well on that sort of thing.


How to learn NLP for free

And there's also another series I did here,


How to learn NLP for free

which again is the same.


How to learn NLP for free

It's like a Q&A app when I used to have that awful GoPro camera.


How to learn NLP for free

Not very nice.


How to learn NLP for free

But anyway, they are at least useful, I hope, as a guide


How to learn NLP for free

or as inspiration to what sort of projects you can do.


How to learn NLP for free

So I mean, that's really, you know, when it comes to NLP,


How to learn NLP for free

that's probably as much as you want to go.


How to learn NLP for free

But I will mention, of course, a lot of what I do


How to learn NLP for free

is going to sort of semantic search and information retrieval.


How to learn NLP for free

So if that's the sort of thing that you're interested in,


How to learn NLP for free

then of course, there are other resources


How to learn NLP for free

that you can go through for that.


How to learn NLP for free

And these are all free if you want them to be free.


How to learn NLP for free

So the only one that isn't sometimes free is this book here,


How to learn NLP for free

which we'll talk about in a minute.


How to learn NLP for free

But there are these two here,


How to learn NLP for free

which are just series that I've written.


How to learn NLP for free

And if you go on here, you can see there's quite a line here.


How to learn NLP for free

And it's all free.


How to learn NLP for free

So it's quite, I think it's a good deal.


How to learn NLP for free

So you come down and you can see we have all these different chapters


How to learn NLP for free

that really go through what we would call sentence transformers,


How to learn NLP for free

semantic search.


How to learn NLP for free

And for me, I think this is a really interesting subdomain of NLP.


How to learn NLP for free

And then here we're looking at vector search,


How to learn NLP for free

another important part of information retrieval.


How to learn NLP for free

And of course, again, you have to find a free link to this in the description.


How to learn NLP for free

You see there's a few parts here,


How to learn NLP for free

which is introducing similarity search, face, and a few other things.


How to learn NLP for free

So again, free costs, I think it's a good deal.


How to learn NLP for free

And then this is the final one.


How to learn NLP for free

So mining of massive datasets is a really cool book.


How to learn NLP for free

It's quite big as well.


How to learn NLP for free

It's this really interesting picture on the front here.


How to learn NLP for free

And this is more like when you start getting into your theory side of things.


How to learn NLP for free

And it's not directly applicable to a lot of NLP, I will be honest.


How to learn NLP for free

But there is so many interesting things.


How to learn NLP for free

It's almost like the theory of everything that built up information retrieval.


How to learn NLP for free

And some obviously more advanced stuff as well.


How to learn NLP for free

But this is really when you're getting into


How to learn NLP for free

trying to understand everything on almost deeper level, I would say.


How to learn NLP for free

Though at the same time, a lot of it you might not even find directly applicable to vector search.


How to learn NLP for free

As I said, it's more everything behind vector search.


How to learn NLP for free

It can be incredibly useful.


How to learn NLP for free

And some of this stuff is really interesting and definitely useful


How to learn NLP for free

when you're actually implementing these sort of things.


How to learn NLP for free

But that's another one that's really good.


How to learn NLP for free

But like I said, you can also get that online for free.


How to learn NLP for free

Which is really nice.


How to learn NLP for free

So if you go over to here, mmds.org, you can come here and...


How to learn NLP for free

Where is it?


How to learn NLP for free

I think there is a link over...


How to learn NLP for free

Yeah, so you have all the links here.


How to learn NLP for free

I'm sure there's a way to find the whole book in one.


How to learn NLP for free

Oh yeah, yeah.


How to learn NLP for free

In this as well.


How to learn NLP for free

So you can also do that.


How to learn NLP for free

It's pretty useful.


How to learn NLP for free

So yeah, I mean, that's kind of what I would recommend you do


How to learn NLP for free

if you're looking to learn NLP, machine learning, all this sort of stuff.


How to learn NLP for free

That's the route I would go.


How to learn NLP for free

A lot of those things, obviously, I went through myself.


How to learn NLP for free

And for me, it has worked out well so far.


How to learn NLP for free

So that's a good indication that at least some of these resources


How to learn NLP for free

will hopefully be useful to you as well.


How to learn NLP for free

And I get this, I get this question asked a lot.


How to learn NLP for free

You know, how do I learn these things?


How to learn NLP for free

And hopefully this video will be some sort of guide for at least some of that.


How to learn NLP for free

But I'm also super interested in what resources you recommend.


How to learn NLP for free

So definitely let me know if there's anything you think is missing.


How to learn NLP for free

And yeah, thank you very much for watching.


How to learn NLP for free

I hope it's been useful.


How to learn NLP for free

And I will see you in the next one.


How to learn NLP for free


Best Indexes for Similarity Search in Faiss

Hi and welcome to this video on, well these CompStat indexes that you can see on the screen now.


IVF-ADC, Multi-DADC and IVF-HSW.


In the last video in this series we sort of introduced CompStat indexes and the index factory in FICE.


And what we're going to do now is obviously go through each of these and build them


using what we learned in the last video, the index factory.


So let's move on to the first one, the IVF-ADC.


This is, it stands for IVF as in like the inverted file index,


followed by asymmetric distance computation.


If you watched the video or the videos or read the articles on product quantization you might remember ADC.


ADC is, well it's part of product quantization.


So during indexing, so when we first take our vectors and we add them to the index,


there's sort of two main steps. Vectors are assigned to lists or the cells.


That's the IVF part of it.


So if I just point that out, so we're taking our vector over here and IVF assignment is this part.


So this is our index IVF. We're assigning them to IVF cells,


which are the Voronoi cells that we usually visualize.


And from that you're assigned this cell ID.


So the cell ID is the IVF part of our index.


And then the second step is that those vectors that we've just assigned,


they go through a product quantization compression step.


So if I take that, it goes all the way over here and we create a PQ code, the quantized code.


And they get stored together.


So the code gets stored in the IVF list.


That is an IVF ADC index at its core.


Now during search, this is where the ADC part comes in.


So ADC is that asymmetric distance computation.


So we have two in product quantization.


Well, when you quantize your vectors, you have these two alternative ways of searching.


First, which is symmetric distance computation over here is not as popular.


And what you do with that is you take your query vector, which is XQ, and you quantize it.


And then you compare it to all of your pre-quantized pre-index vectors.


So essentially both sides of what you're comparing are quantized.


And that's why it's the symmetrical or symmetric distance computation.


The alternative, which tends to work better, is where you don't quantize your query vector.


So you just take XQ as it is.


You compare that to your pre-quantized pre-index vectors.


So the quantized speed vectors.


So that's where you get the IVF side of things.


And then you also get the ADC side of things.


And when it comes to FICE, we can use this index factor string to pull that together.


So we have IVF on the left, product quantization on the right.


So let's go to Python and put that together.


So I've already imported the data using, again, the SIFT1M dataset.


There'll be a link in the description which will explain how you can get that dataset.


From that we have, I can show you, we have XB, which is the index vector,


or the vectors that we're searching through,


and then XQ, which is just a single vector that we're using to search.


And so we put that together, we just write index, FICE, index factory, as we have been doing.


We have our number of dimensions, 128 for this dataset.


And then we want to input the string, the index factor string, which is,


so we have IVF, which is the first part.


We're going to split that into 256 cells, Voronoi cells.


And we are using PQ for the second part, which is the ADC part.


And here we're going to use an mvalue of 32.


32 and a nbits value of 8.


Now, by default, this is 8.


So we can include the times 8 there, but you don't need it.


The only way you probably need this is if you want to use a lower nbits value.


So say you want to use 4.


If you try and go for 12, well, you're not going to be able to,


unless FICE has updated that yet.


So basically, you're not allowed to go over 8 for now with the PQ index or IVF PQ index.


But that should be being removed in a future version.


And then like we usually do, we just do index train.


So we need to train, of course, using IVF and also product quantization.


So we really, really do need to train.


And then we add our vectors.


Now from there, we just do search as we normally do.


So di equals index.search xq.


And I'll run that.


We'll see, I don't really need to see what we return.


But I'll show you the performance of that.


So meanwhile, with us, I may as well explain.


So we have m here.


m is the number of subvectors that we're splitting our full vectors into


for the product quantization set.


And the nbits value here is 8,


which is the number of bits used by each subquantizer


within the product quantization set.


So you just increase that to increase recall performance or accuracy.


Okay, and then you see we have our nearest indexes there.


Now, let me show you the actual performance and the metrics for this.


So what we have here is a few things I wanted to compare.


So we have the blue lines here are what we just built.


So the IVF ADC.


Then we also have the magenta line.


That is still IVF ADC,


but we've added a optimization set to it.


Now, for me, there was no real performance increase there.


Now, I just wanted to compare that to a simple flat index.


Now, the flat index, if we use that,


the memory usage of that is something like half a gigabyte.


Whereas the memory usage for our two PQ or IVF PQ indexes


is I think around 40 megabytes.


So there's a pretty, yeah, a huge difference there.


So that's why you'd use PQ in the first place,


to minimize your memory usage of the index.


But in terms of recall, of course, you can't really beat a flat index.


So let's move on to the next one.


So we have Multi-D ADC,


which is a multi-index asymmetric distance computation.


So the ADC part is the same.


So we know, okay, there's product quantization in there.


And the Multi-D part is what is different.


So this is based on IMI, which is an inverted multi-index.


And IMI is similar to IVF,


but where we have our Voronoi cells in IVF,


IMI uses product quantization to split your vectors into subvectors


and then process them through that Voronoi cell


or cluster them using Voronoi cells.


So it's like IVF, but split across multiple vector subspaces.


So like a tiered Voronoi cell structure,


which is what I'm trying to visualize here.


So with IVF, we just have one layer, whereas with IMI,


so this is IMI, not necessarily the Multi-D ADC.


So this is just IMI.


So our vectors come in, they get split into two in this case,


and then they get clustered based on that.


And then when we introduce our query vector,


we split that again into two


and we find the cells that it belongs to


and we find a crossover between those cells


to find the nearest neighbor candidates.


Okay, and this is using just an N-probe value of one in this case.


Now we can put that together quite easily.


Using this index factory string.


So we have the IMI index on the left


and then we have again, PQ because it's an ADC on the right.


So let's go ahead and put that together.


Okay, so I'm going to come up here.


I'm just going to use the same script we use here.


So I'm going to take IMI to multiply by eight there


or two by eight.


And let me remove the by eight there.


Now in this case, it's actually better if we include OBQ.


So in this case, it does improve the performance of our index.


So we'll add that.


And what I want to do,


so I'm going to get a recall function


so we can compare the matches between this


and a simple flat index and see how many of the responses


it returns match or align.


Okay, so I'm going to add that in here.