How to Build Python Packages for Pip

just there to tell our package builder which directories are parts of our python package or


How to Build Python Packages for Pip

python sub packages and then of course we have the readme file here. So we can summarize this as this


How to Build Python Packages for Pip

we have on our top level we have the project directory and then we have the readme file and


How to Build Python Packages for Pip

then we have our code directory inside there we need to have the init file and then we in this


How to Build Python Packages for Pip

example we have two modules we have module a module b. Now when it comes to actually importing


How to Build Python Packages for Pip

one of these modules and using it in our code what we would do is we'd write something like this so


How to Build Python Packages for Pip

we'd have from and in this case it's called code directory obviously we wouldn't call our package


How to Build Python Packages for Pip

code directory unless you really want to and then we would import a for the a module or b


How to Build Python Packages for Pip

for the b module. So in our case this is going to replace with aesthetic ascii and this is going to


How to Build Python Packages for Pip

be replaced with synthesize and we'll come to actually testing that out later but for now let's


How to Build Python Packages for Pip

just work through the other files that we need in our directory. I don't want to focus on the code


How to Build Python Packages for Pip

so much here so we have our readme file and we have our code directory the other files that we


How to Build Python Packages for Pip

need are all configuration setup files and the most important of those for setting up our package


How to Build Python Packages for Pip

configuration is the setup config file. So that looks like this and if we go ahead and actually


How to Build Python Packages for Pip

open this file so we need to create it here so setup.config and there are a few parameters in


How to Build Python Packages for Pip

here that we'll need to add. Now for this project we will be using these so we have metadata which


How to Build Python Packages for Pip

is just the description or the data explaining what our package is actually doing so this sort


How to Build Python Packages for Pip

of thing will be displayed on the python package index and people will be able to see the name of


How to Build Python Packages for Pip

it the version the author they'll be able to get in touch with you through your email and they'll


How to Build Python Packages for Pip

also find a link to your website so for me obviously I'm just using git up here and all of


How to Build Python Packages for Pip

those things are pretty self-explanatory but there are a few that I think are a bit more abstract.


How to Build Python Packages for Pip

So first we have the long description and you can write something here but it's a long description


How to Build Python Packages for Pip

so I find it's better to include a file and you include your readme file and then this will


How to Build Python Packages for Pip

display on the front page of your python package index site and then we also need to explain that


How to Build Python Packages for Pip

our long description is a markdown text file so that it is processed correctly. Next we have our


How to Build Python Packages for Pip

classifiers and these are kind of like tags so we're specifying the programming language that


How to Build Python Packages for Pip

we're using it's python it's python3 we are outlining the license that we're using and for


How to Build Python Packages for Pip

that we have the osc approved and we're using an mit license which we'll cover pretty soon and then


How to Build Python Packages for Pip

we'll specify the operating system and for us it's independent you don't have to have a specific


How to Build Python Packages for Pip

operating system to run this so that's just the metadata behind our package and then down here we


How to Build Python Packages for Pip

have some of the more important things for actually building the package so here we're


How to Build Python Packages for Pip

specifying the packages that will be included as dependencies and we use this find here and what


How to Build Python Packages for Pip

this would do it will go through our code here and find any package dependencies that we have and


How to Build Python Packages for Pip

this is provided by self tools which is what we're using and I'll explain that in a moment.


How to Build Python Packages for Pip

We also want to specify that we're using python and we require either 3.7 or greater and the reason


How to Build Python Packages for Pip

that we need 3.7 or later is because we're using a package that came with python 3.7 in order to


How to Build Python Packages for Pip

include all of our resources over here so that's our dependency on that version and then here we're


How to Build Python Packages for Pip

including package data which again we're including because we have all these extra files that are not


How to Build Python Packages for Pip

code files that we need to have included within our package and that's why we've added that in


How to Build Python Packages for Pip

there so that's our setup config file and alongside that we also need a pyproject.toml file


How to Build Python Packages for Pip

and I mentioned before in the setup config that we're using setup tools and in order to tell


How to Build Python Packages for Pip

our package builder which we'll use later to use setup tools in the building process


How to Build Python Packages for Pip

we need to specify this pyproject.toml file so we'll create that file here pyproject.toml


How to Build Python Packages for Pip

and this is the code that we'll need for it so we're saying here for the build system it will


How to Build Python Packages for Pip

require setup tools which is the library we use to actually set up and process the setup.config file


How to Build Python Packages for Pip

and then python wheel and then here we're just specifying the build process that we'll be using


How to Build Python Packages for Pip

from setup tools build meta okay and that's that's all we need and that is just read by our builder


How to Build Python Packages for Pip

and it tells it how to build our package which essentially tells it to use setup tools and


How to Build Python Packages for Pip

process this and follow the instructions that this file is giving you and next up is our license


How to Build Python Packages for Pip

license now the license is quite important and it's very important that we actually include a


How to Build Python Packages for Pip

license file because if we don't include a license file we can get into a lot of issues in terms of


How to Build Python Packages for Pip

people not knowing if they're actually allowed to use our code or not and especially if you have


How to Build Python Packages for Pip

people contributing to your project if they contribute and you still don't have a license file


How to Build Python Packages for Pip

then it's quite odd because legally you're in a sort of grey zone as to whether you can actually


How to Build Python Packages for Pip

use the package that you built so a license is very important to include but it's very very easy


How to Build Python Packages for Pip

to actually set that up so all we need to actually do is we go on to this website called chooselicense.com


How to Build Python Packages for Pip

and we just select what we want for our license so for me I want it to be simple and permissive so


How to Build Python Packages for Pip

anyone can use it and it gives you a license here to just copy and paste across so I'm using the


How to Build Python Packages for Pip

MIT license go back into project and we just create a new file and it's called license like that


How to Build Python Packages for Pip

and we just copy that across change list so it's 2021 James Brooks and that's the license sorted so


How to Build Python Packages for Pip

it's pretty easy now this is all great and we have our code but there is one modification to our code


How to Build Python Packages for Pip

we need to make in order for this to work so in our code at the moment we rely on reading these


How to Build Python Packages for Pip

images which are in the same folder at the moment now of course we could have put these in a different


How to Build Python Packages for Pip

folder we probably would normally but when we are building a package we need any resources that we


How to Build Python Packages for Pip

want to include in that package to be contained within a package directory which includes a init


How to Build Python Packages for Pip

pi file so that's why they're all included in here now we have a few images and we also have


How to Build Python Packages for Pip

this font file which is a true type font and what we do down here usually is that we use the pill


How to Build Python Packages for Pip

library and we just open the image like this so we're selecting one at random so it's one of these


How to Build Python Packages for Pip

at random and we're just opening that and it's pretty straightforward now when we build a package


How to Build Python Packages for Pip

this won't work anymore because this is a relative path so you will almost certainly get issues so


How to Build Python Packages for Pip

what we instead need to do is include these files within our package so we've already done part of


How to Build Python Packages for Pip

that we've included them in this init file but there is one more step we need to include a


How to Build Python Packages for Pip

another file here which is called manifest dot in and this manifest dot in file needs to include


How to Build Python Packages for Pip

any resources that are not code files that we would like to include within our package so to do


How to Build Python Packages for Pip

that we need to write include and then here we give the name of our python package or sub package


How to Build Python Packages for Pip

so it's aesthetic ascii and this is referring to this directory name here and inside there we want


How to Build Python Packages for Pip

to include all so this is wildcard for all files that end with png which is our image files now you


How to Build Python Packages for Pip

will notice that we also have some jpeg files as well so we'll just copy that and we'll add jpeg as


How to Build Python Packages for Pip

well now that's our image files but we also have our true type font file so we add that in as well


How to Build Python Packages for Pip

and that's our manifest file now back in the config we know that we've included the manifest


How to Build Python Packages for Pip

config file through this include package data at the bottom so if this was false this would not be


How to Build Python Packages for Pip

read in and our extra resources would not be included with it being true it knows to expect


How to Build Python Packages for Pip

a manifest file and then it reads this and includes these within our package so that's


How to Build Python Packages for Pip

great and then in our code we need to use a different approach to actually opening these


How to Build Python Packages for Pip

images or other files and to actually do that we need two different libraries in this case


How to Build Python Packages for Pip

we need import lib and from import lib we need to import resources


How to Build Python Packages for Pip

and alongside import lib resources we also need to import the io library


How to Build Python Packages for Pip

so this import lib resources is specifically for reading in resources or files that we've included


How to Build Python Packages for Pip

within a package the io library is for converting the bytes that we will read when we import the


How to Build Python Packages for Pip

resources into readable file like objects because the code that we use down here reads our images


How to Build Python Packages for Pip

and font from file so this allows us to pretend that the bytes that we've read into our python


How to Build Python Packages for Pip

code is actually a file rather than a set of bytes so with that this code here this line


How to Build Python Packages for Pip

instead becomes this which is a little more complicated but this is just what we have to use


How to Build Python Packages for Pip

so here we're using the resources that we imported and we're using open binary it's in the aesthetic


How to Build Python Packages for Pip

ascii package so we need to include that that is the directory name here and then this time


How to Build Python Packages for Pip

we're using the same thing again we're just using the image and we're selecting one at random and


How to Build Python Packages for Pip

then we're just importing or reading that like with any other file and then when we use this we


How to Build Python Packages for Pip

would expect a file path to be passed to this function so we imitate a file path by using this


How to Build Python Packages for Pip

io bytes io function and passing the image binary to that now that's our images that's sorted and


How to Build Python Packages for Pip

then we also have our font down here as well okay so at the moment what we have we have our font path


How to Build Python Packages for Pip

and this is just reading from file so we've got this image font true type and then we're reading


How to Build Python Packages for Pip

it in there now we're going to be using the same thing again we're going to be using the open


How to Build Python Packages for Pip

binary function so we do with resources open binary again we include our aesthetic ascii


How to Build Python Packages for Pip

package name and we also include the font path which we've just defined there and we again read


How to Build Python Packages for Pip

that in just like we did before so we'll call it font we do fp.read and then we need to initialize


How to Build Python Packages for Pip

that font and what we were doing here was reading from path before so again we need to use the io


How to Build Python Packages for Pip

library to imitate that so we do io bytes io and font and with that all of that will now work so


How to Build Python Packages for Pip

that's everything set up and what we do once we have it all set up is we first test site works


How to Build Python Packages for Pip

locally so the first thing we need to do is navigate to that directory so for me it's in


How to Build Python Packages for Pip

documents projects test aesthetic ascii and once you're in there we can just do pip install dot


How to Build Python Packages for Pip

and this will do a local pip install from the current directory okay so you can see that that


How to Build Python Packages for Pip

worked that's great and then here you'll probably want to just test your new imported package so


How to Build Python Packages for Pip

for example for me i would go from aesthetic ascii import synthesize


How to Build Python Packages for Pip

and then i would go through the code for actually testing i'm not going to do it here


How to Build Python Packages for Pip

but as long as that all works well then we should be good to go so at this point we do python module


How to Build Python Packages for Pip

build and this package this module will actually build our package and what you will find in the


How to Build Python Packages for Pip

directory here is that we'll get a new directory called dist which means distribution and this will


How to Build Python Packages for Pip

contain two files so we can have a look at the moment we have the tar file and now we have the


How to Build Python Packages for Pip

wheel file as well now these are two files that are going to be uploaded to the python module


How to Build Python Packages for Pip

and two files that are going to be uploaded to the python packaging index and they are our package


How to Build Python Packages for Pip

so after we've done that we need to upload our code the first thing we do before we upload it


How to Build Python Packages for Pip

straight to the python packaging index is we upload it to the test version of the python


How to Build Python Packages for Pip

packaging index just to make sure that everything is working and we find that over at test pip.org


How to Build Python Packages for Pip

course you will need to register here so you go through register enter everything you need


How to Build Python Packages for Pip

you will need your username and password when you're running through the upload process in your


How to Build Python Packages for Pip

prompt window so make sure you have that set up and we'll also need to pip install twine which is


How to Build Python Packages for Pip

a package that will allow us to upload everything to python packaging index now i already have it


How to Build Python Packages for Pip

installed so i won't go ahead and do that instead what i'll do is python module twine upload repository


How to Build Python Packages for Pip

test python packaging index and here we need to pass everything within our distribution folder


How to Build Python Packages for Pip

and we run that and everything will be uploaded to our python packaging index once we've done that we


How to Build Python Packages for Pip

want to actually install our package but from the test python packaging index so to do that first


How to Build Python Packages for Pip

because we already installed our package we actually want to uninstall it


How to Build Python Packages for Pip

so


How to Build Python Packages for Pip

i'm just uninstalled that and then we can install it again but through the test package index so to


How to Build Python Packages for Pip

do that we just pip install and then we set the install index to the test python package index


How to Build Python Packages for Pip

so we just do test pipy.org plus simple and then we specify our actual package name which is


How to Build Python Packages for Pip

synthetic ascii and we should find that that will actually run and install and then at this point


How to Build Python Packages for Pip

this is where we want to test it again now if you are using resources and everything with your


How to Build Python Packages for Pip

package that is something to definitely check that everything is being read correctly and it's just


How to Build Python Packages for Pip

functioning now once you're happy that everything is working correctly this is where we finally


How to Build Python Packages for Pip

upload it to the actual python packaging index now this is the same process again so the python module


How to Build Python Packages for Pip

using twine we're using upload and we also type repository


How to Build Python Packages for Pip

now this time instead of writing test pypi we write pypi and again we use our distribution


How to Build Python Packages for Pip

we execute that it'll ask you to log in and then we're done and we'll be able to see our python


How to Build Python Packages for Pip

package over here so we'll be able to search for it or you can just come over here to if you've


How to Build Python Packages for Pip

logged in click on your projects and you'll find here we have our actual package and we can see


How to Build Python Packages for Pip

our actual package now i can view that and it will come up with the readme file on the side here we


How to Build Python Packages for Pip

have all of our metadata so we have the mit license the author and that includes a email link there


How to Build Python Packages for Pip

as well which python version it requires the language and so on we can also click on the home


How to Build Python Packages for Pip

page which will take us through to the github repository so that's all set up as we would want


How to Build Python Packages for Pip

it to which is pretty cool and most importantly we can actually just pip install it and start


How to Build Python Packages for Pip

using it so first i'll just pip uninstall and now all we need to do is pip install our package


How to Build Python Packages for Pip

and it's that easy so that's everything that you need to know for actually building your own python


How to Build Python Packages for Pip

package it's a really cool thing to be able to do and i would definitely recommend learning how to


How to Build Python Packages for Pip

do this and building your own packages so thank you very much for watching and i will see you again


How to Build Python Packages for Pip


Streamlit for ML #4 - Adding Bootstrap Components

Okay, in the last few videos we've put together this Streamlit app for open domain question


answering and what it does at the moment is we can ask you a question like who are the


Normans or we can ask somewhere, I don't know, where are the Normans from?


I'm not sure if we have, I'm not sure what topics we have in there in the Vector database


I'm just going to ask like Norman questions from now and maybe I can ask something like


where is it?


That must have come up with something surely.


Yeah okay so we see different things in there.


Okay cool so we have this and we can ask you questions and it all works but it's not very


nice to look at at the moment we just get these big chunks of text back and I mean that's


pretty much all we can do with the current backend but we can at least format it a little


nicer.


So what we're going to do is convert these big chunks of text into what are called cards.


Now to create these cards we are going to use Bootstrap.


So if I just type in Bootstrap cards, you can see it's in my history but I'm going to


just put Bootstrap cards for now so you can follow and go on here and let's make this


a little bit bigger, zoom in a little bit and scroll down.


So we have some cool examples here so this is one card we can go down a little bit more


or this one I think this is probably the one I'll use.


Okay so basically Bootstrap is like a framework which is pretty lightweight and you have all


these really nice examples of how you can build something that was super nice.


So what we can do is actually just copy this, we'll go back over to our code and what I'm


going to do is I'm going to create the definition for one of these cards.


I'm going to do that in a function so we'll make our lives a little bit easier.


We made this a lot easier to read anyway.


Okay so I'm going to, what we're going to do is return an fString.


Let me clean that up a little bit so un-indent that and that's our card.


So now, well actually this won't work yet but if I come down here I'm going to write


okay for context instead of taking this stwrite metadata I'm going to just write st.write


card.


Okay actually not stwrite we actually want to include st.markdown.


So markdown is another useful component in Streamlit.


Let me open it up here.


So it would be a text element and we have st.markdown so let's have a look at that and


it displays string formatted as markdown.


Now what it also allows you to do is if we take this put it in here and say it's true


we can also use it to display HTML.


Let's save that.


Let's go to our app and get okay always rerun at the top there.


Okay and now we have these cards and they're not very nice looking and we'll figure out


why in a moment but before we figure out why let's actually make this a little bit nicer


so or make it relevant because at the moment we don't have anything relevant in here.


So I'm going to include the title or maybe the id yeah let's go id value title and context.


Okay we'll make this an f format string and what I'm going to do is remove these links


at the bottom because I don't really have anywhere to link to with these and what I'm


going to do is replace this default text with a context.


I'm going to minimize that here.


So context the title we'll make title and the subtitle I want it to be the id value.


Okay so if I do that okay it's not going to work straight away because we need to actually


pass our id value title and context so let's do that.


So the what we know the text is here.


Okay so we have the context which is the last parameter we also have in the context or in


the metadata we also have the title and then the id is in a different part of the json


response and so that is in if I remove this that is an id.


Okay let's run this see if it works.


Okay cool so now we have like the the title so the wikipedia topic that came from the


id unique id for each context we have and then we also have this you know this chunk


of text the context.


Now there's a few things that are kind of going a little bit weird here first the formatting


of the card isn't what we saw in the in the bootstrap example.


Come over here it looks really nice okay so something weird going on there and so what


we actually need to do is if we let's search css see if it comes up okay so we have this


compiled css and js come down to here.


What we need to do is actually copy the style sheet for bootstrap so that basically how


it's been styled and we need to put that into our into our app.


So all we need to do is copy that first line which is a style sheet the second one is for


if you have JavaScript components we don't for the card so we don't need to worry about


that and what we're going to do is actually create another markdown component so st.markdown


because as before all we need to do here is add some html to our app so we're just going


to paste that in okay but as before with html we also need to use this allow or unsafe allow


html okay now if I add that in here and rerun the app our card should start looking nicer


okay so you can see now we have that nice little border and we have it just looks better


the only thing is now that it's actually like really tight it's not taking the full width


of the parent container that it's within so to change that we don't actually need to really


do anything we just need to remove this style component okay so let's remove that and see


see what happens okay so now we will see this card sticks the same width as our search component


so we're now getting something a little nicer the only issue is that now if we look between


our cards they they're like attached and we also don't really want that so what we can


do is add a margin to our cards so this is all just like html and css it's nothing like


too complicated so you can you can look this up if you if you want to I'm going to use


one rem which is a more flexible measure rather than using pixels I'm going to save that let's


have a look at what we get so now we get a little bit of a gap between one the edges


you can move that if you want by using the margin top and margin bottom but I think it


looks nice so I'm going to keep it and we have an actual gap between each of our cards


okay so yeah that's that's pretty cool if we go back to you know something that we know


works so who are the normans run that and we now get this cool response okay so I think


that is pretty much like the core of our app that's everything we need and the app is there


it's not you know it's definitely not perfect there's a lot we can do to it to make it much


more interesting and we will look into those things but for now that's I think pretty cool


it's the core of our app and with that we can actually do something useful with it we


can actually you know help people find information in a quicker more natural way but that's it


for for that video I really hope it's been useful and yeah thank you very much for watching


AlexNet and ImageNet Explained

Today we're going to talk about one of the most important events in the history of deep learning.


AlexNet and ImageNet Explained

We're going to talk about what happened at ImageNet 2012 and how that launched the sort


AlexNet and ImageNet Explained

of deep learning rocket ship that we've been strapped to for the past decade. So in short,


AlexNet and ImageNet Explained

we're going to talk about ImageNet and where it came from, why it was so important, and then we're


AlexNet and ImageNet Explained

going to have a look at, very briefly, going to have a look at convolutional neural networks and


AlexNet and ImageNet Explained

AlexNet, which is the model that triggered the massive growth of deep learning. And for me, I


AlexNet and ImageNet Explained

like to back everything up with code. So what we'll do is towards the end of the video, we're


AlexNet and ImageNet Explained

going to go through the PyTorch implementation of AlexNet and we're actually going to test it on a


AlexNet and ImageNet Explained

small ImageNet-like dataset and that will be quite useful because we can see sort of image


AlexNet and ImageNet Explained

pre-processing steps and also how to perform inference with a convolutional neural network


AlexNet and ImageNet Explained

like AlexNet. So let's jump straight into it. Today's deep learning revolution traces its roots


AlexNet and ImageNet Explained

back to the 30th of September 2012. On this day, a deep layered convolutional neural network won


AlexNet and ImageNet Explained

the ImageNet 2012 challenge. And this convolutional neural network didn't just win, it completely


AlexNet and ImageNet Explained

destroyed the rest of the competition. Now this model, you might have guessed, is called AlexNet.


AlexNet and ImageNet Explained

And the simple fact that you even use convolutional neural networks was very new. Convolutional


AlexNet and ImageNet Explained

neural networks have been around for a while, but using them had kind of been deemed impractical.


AlexNet and ImageNet Explained

Yet when AlexNet's results came in, it proved sort of unparalleled performance on what was seen as


AlexNet and ImageNet Explained

one of the hardest challenges of the time for computer vision. So this event made AlexNet the


AlexNet and ImageNet Explained

first widely acknowledged successful implementation of deep learning. And the sheer performance


AlexNet and ImageNet Explained

improvement that it showed caught people's attention. Until this point, deep learning


AlexNet and ImageNet Explained

was unproven. It was simply a nice idea that most people just decided, okay, it's impractical. We


AlexNet and ImageNet Explained

don't have enough data. We don't have enough compute to do anything like this. But AlexNet


AlexNet and ImageNet Explained

showed that this was not the case and that deep learning was now practical. Yet this sort of surge


AlexNet and ImageNet Explained

of interest in deep learning was not solely thanks to AlexNet. ImageNet also played a big part in


AlexNet and ImageNet Explained

this. The foundation of applied deep learning was set by ImageNet and built upon by AlexNet.


AlexNet and ImageNet Explained

So let's begin with ImageNet. Back in 2006, the world of computer vision was a lot different to


AlexNet and ImageNet Explained

how we know it now. It was pretty underfunded. It didn't really get that much attention. Yet there


AlexNet and ImageNet Explained

were a lot of researchers around the world focused on building better models. And the year after year,


AlexNet and ImageNet Explained

they saw progress, but it was slow. In that same year, a woman called Fei-Fei Li had just finished


AlexNet and ImageNet Explained

her computer vision PhD at Caltech and had started working as a professor in computer science and


AlexNet and ImageNet Explained

had noticed this sort of focus in the field of computer vision on the models and the subsequent


AlexNet and ImageNet Explained

lack of focus on data. And an idea came to Li that maybe a dataset that was more representative of


AlexNet and ImageNet Explained

the world could improve the performance of the model as being trained on it. Around the same


AlexNet and ImageNet Explained

time, there was another professor called Kristiana Felbom and she was a co-developer of a dataset


AlexNet and ImageNet Explained

from the 1980s called WordNet. Now, WordNet consisted of a pretty large number of English


AlexNet and ImageNet Explained

language terms organized into a ontological structure. So for example, for the term Siberian


AlexNet and ImageNet Explained

Husky, that would be within a tree structure and above Siberian Husky, you would have working dog.


AlexNet and ImageNet Explained

Above working dog, you would have dog. Above dog, you'd have canine, carnivore, and so on. So there's


AlexNet and ImageNet Explained

like that tree structure of different terms and where they and how they relate to each other.


AlexNet and ImageNet Explained

In 2007, Li and Felbom met and Felbom discussed her work on or her idea at the time of adding


AlexNet and ImageNet Explained

just a reference image to each of the terms within WordNet. So the intention was not to create a


AlexNet and ImageNet Explained

image dataset, but it was simply to add like a reference image so people could more easily


AlexNet and ImageNet Explained

understand what that particular term was about. And this inspired an idea from Li that would


AlexNet and ImageNet Explained

kickstart the world of computer vision and deep learning. So soon after, Li put together a team


AlexNet and ImageNet Explained

to build what would become the largest labeled dataset of images in the world called ImageNet.


AlexNet and ImageNet Explained

The idea behind ImageNet was that a large ontological based dataset like WordNet,


AlexNet and ImageNet Explained

but for images, could be the key behind building more advanced content-based image retrieval,


AlexNet and ImageNet Explained

object recognition, scene recognition, and better visual understanding in computer vision models.


AlexNet and ImageNet Explained

And just two years later, the first version of ImageNet was released with 12 million labeled


AlexNet and ImageNet Explained

images. These were all structured and labeled in line with the WordNet ontology. Yet, if we consider


AlexNet and ImageNet Explained

the sheer size of that, the 12 million images, if one person had spent literally every single day


AlexNet and ImageNet Explained

labeling one image per minute and did literally nothing else in that time, they didn't eat,


AlexNet and ImageNet Explained

they didn't sleep, just labeled images, it would have taken them 22 years and 10 months,


AlexNet and ImageNet Explained

which obviously is a very long time. There's just an insane number of images to be labeled here.


AlexNet and ImageNet Explained

So how did they do it? Because the team was not huge, they didn't have like an infinite amount of


AlexNet and ImageNet Explained

money to pay other researchers and students to do this. So what they eventually settled on was a


AlexNet and ImageNet Explained

platform called Amazon's Mechanical Turk. Mechanical Turk is a crowdsourcing platform where


AlexNet and ImageNet Explained

people from around the globe will perform tasks such as labeling images for a set amount of money.


AlexNet and ImageNet Explained

And because there's just the insane scale of people around the world doing this at competitive


AlexNet and ImageNet Explained

prices, that made ImageNet possible with a few adjustments to the labeling process. Because


AlexNet and ImageNet Explained

in reality, if you just have random people around the world labeling your data set,


AlexNet and ImageNet Explained

some of those people are going to try and take advantage of that, maybe game the system.


AlexNet and ImageNet Explained

So you have to have some checks and balances in place against that. So there's a little bit of


AlexNet and ImageNet Explained

system design in there. But that is how they built the data set. Now, on its release, ImageNet was


AlexNet and ImageNet Explained

the largest publicly available labeled data set of images in the world. Yet, there was very little


AlexNet and ImageNet Explained

interest in the data set, which seems pretty crazy when we look back on that in hindsight. Because


AlexNet and ImageNet Explained

now we know, okay, we want more data for our models to make them better. But at the time,


AlexNet and ImageNet Explained

things were different. So ImageNet came with these 12 million images,


AlexNet and ImageNet Explained

distributed across 22,000 categories. And at the time, there were the odd other image data sets


AlexNet and ImageNet Explained

that use a similar sort of structure and idea. So for example, the ESP data set use something


AlexNet and ImageNet Explained

called the ESP game. And people would play the ESP game and label images for the data set. Now,


AlexNet and ImageNet Explained

reportedly, they had way more images. But it wasn't publicly released. Only about a few thousand


AlexNet and ImageNet Explained

it wasn't publicly released. Only publicly released 60,000 of those images. And a couple


AlexNet and ImageNet Explained

of years later, there were a few papers that kind of looked at the ESP game and ESP data set and


AlexNet and ImageNet Explained

said, okay, it's probably not actually that useful because you can kind of guess the right answer


AlexNet and ImageNet Explained

most of the time without even looking at the image. So there were some questions around the


AlexNet and ImageNet Explained

usability of that data set. So all this to say, ImageNet was by far the biggest, at least publicly


AlexNet and ImageNet Explained

available and most sort of accurate data set for computer vision at the time. So the reason that


AlexNet and ImageNet Explained

there was very little interest in ImageNet, despite its huge size, is that people just assume


AlexNet and ImageNet Explained

that it could not work for their models. You have to think back then, they were training models on


AlexNet and ImageNet Explained

much smaller data sets, which had like 12 categories of images, for example, and the models


AlexNet and ImageNet Explained

would struggle with that. So when ImageNet comes along and it's like, hey, I have 22,000 categories


AlexNet and ImageNet Explained

here, people are just like, well, I can't deal with 12. So I'm not going to even try 22,000.


AlexNet and ImageNet Explained

That's crazy. So there was a lack of interest in ImageNet at the time. It just wasn't really


AlexNet and ImageNet Explained

received that warmly. So the ImageNet team decided to try and push it a bit more. So by the next year,


AlexNet and ImageNet Explained

2010, they had managed to organize a challenge with the data set, a classification challenge


AlexNet and ImageNet Explained

initially, and they grew into different things over the years. But initially, it's just a


AlexNet and ImageNet Explained

classification challenge. So the ImageNet large scale visual recognition challenge was first


AlexNet and ImageNet Explained

hosted in 2010. And competitors had to correctly classify images from 1,000 categories. So not a


AlexNet and ImageNet Explained

full set of terms in the ontology of ImageNet, but they had 1,000 categories in set. And whoever


AlexNet and ImageNet Explained

produced a model with the lowest error rate won. And there were a few entrants. There was not a


AlexNet and ImageNet Explained

huge number of entrants. I think it's something like four, five, six entrants in the first,


AlexNet and ImageNet Explained

five, six, seven, eight, nine, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,


AlexNet and ImageNet Explained

25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46,


AlexNet and ImageNet Explained

47, 48. So, the latest challenge results were released and one of those results was a lot


AlexNet and ImageNet Explained

better than any of the other results. And it came from a model that most people thought was


AlexNet and ImageNet Explained

just not practical. And that was AlexNet. AlexNet was the first model to score a sub 25%


AlexNet and ImageNet Explained

rate and that same year the nearest competitor was a was 9.8 percentage points behind AlexNet.


AlexNet and ImageNet Explained

And they had done this with a deep layered convolutional neural network which at the


AlexNet and ImageNet Explained

time people were not really taking seriously. Now to understand AlexNet probably best we


AlexNet and ImageNet Explained

very quickly cover a little bit of what convolutional neural network actually is. So convolutional


AlexNet and ImageNet Explained

neural network or CNN is a neural network layer that has a special layer called a convolutional


AlexNet and ImageNet Explained

layer. And today these models are known for computer vision. They have been for quite


AlexNet and ImageNet Explained

a long time sort of undisputed champions of computer vision. And actually you know that


AlexNet and ImageNet Explained

has changed a little bit in literally like the past couple of years. But right now they're


AlexNet and ImageNet Explained

still pretty dominant. And unlike a lot of the models back in 2012 and earlier these


AlexNet and ImageNet Explained

did not need too much sort of manual feature extraction or too much image pre-processing


AlexNet and ImageNet Explained

before feeding data into the model. They could just kind of deal with that themselves. CNN


AlexNet and ImageNet Explained

use several of these convolutional layers stacked on top of each other. And what you


AlexNet and ImageNet Explained

find is that the deeper the network is the more the better it can identify more sort


AlexNet and ImageNet Explained

of complex concepts or objects in images. So for example the first with the first few


AlexNet and ImageNet Explained

layers you're probably going to just kind of identify OK this is an edge. This is a


AlexNet and ImageNet Explained

circle maybe. This is this shape and maybe some textures. As the network gets deeper


AlexNet and ImageNet Explained

and you add more layers to it starts to abstract those features and identify more abstract


AlexNet and ImageNet Explained

ideas. So a deeper network will be able to identify OK this is like a living thing. And


AlexNet and ImageNet Explained

then you go you build a deeper network and it can identify mammals and then it can identify


AlexNet and ImageNet Explained

dogs and it can identify Siberian Huskies. So as the as the model gets deeper its performance


AlexNet and ImageNet Explained

and its ability to identify more nuanced things gets better. So at the time these models were


AlexNet and ImageNet Explained

overlooked because essentially to train these to get good performance from these models


AlexNet and ImageNet Explained

you need to be really deep which means that they have a lot of parameters. OK. And it's


AlexNet and ImageNet Explained

the more parameters you have the longer it can take your model to train. If you can train


AlexNet and ImageNet Explained

it at all if it's if it's too big and doesn't even fit in the memory on your computer. And


AlexNet and ImageNet Explained

also the more parameters it has the more data it has to see before it can produce sort of


AlexNet and ImageNet Explained

any any good performance of anything. As a result of this they would simply overlooked.


AlexNet and ImageNet Explained

Yet the authors of AlexNet won the ImageNet challenge in 2012 and it turns out that they


AlexNet and ImageNet Explained

were the right people in the right place at the right time. Several pieces came from from


AlexNet and ImageNet Explained

different places to create this. ImageNet provided a massive amount of data needed to


AlexNet and ImageNet Explained

train one of these deep layered convolutional neural networks. A few years earlier in 2007


AlexNet and ImageNet Explained

NVIDIA had released CUDA which you may recognize the name of. So an API that allowed software


AlexNet and ImageNet Explained

access to the lower level highly parallel processing abilities of CUDA enabled GPUs


AlexNet and ImageNet Explained

from NVIDIA. And GPU power in itself was reaching a point where this you know training these


AlexNet and ImageNet Explained

big models was becoming possible. Although it wasn't quite there yet at the time for


AlexNet and ImageNet Explained

a single GPU. So AlexNet was by no means small. And because of that the authors had to solve


AlexNet and ImageNet Explained

a lot of problems to get all this working. So AlexNet consisted of five convolutional


AlexNet and ImageNet Explained

layers followed by three fully connected linear layers. The final layer to produce the 1000


AlexNet and ImageNet Explained

classifications required by ImageNet was a 1000 node layer that used a softmax activation


AlexNet and ImageNet Explained

function to create this probability distribution over all of those classes. Now a key conclusion


AlexNet and ImageNet Explained

from AlexNet was that the depth of the network was key to to getting the performance that


AlexNet and ImageNet Explained

they got. And that depth as I mentioned before it creates a lot of parameters that need to


AlexNet and ImageNet Explained

be trained. Making training the model either impractically slow or just simply impossible.


AlexNet and ImageNet Explained

Or at least that was the case if you're going to train it on CPU. So they had to turn to


AlexNet and ImageNet Explained

GPUs. But at the time the high end GPUs only had a memory of about three gigabytes which


AlexNet and ImageNet Explained

was not enough for AlexNet. So to make it work they had to distribute AlexNet across


AlexNet and ImageNet Explained

two GPUs. And they did this by pretty much splitting the layers in two. And having you


AlexNet and ImageNet Explained

know half of the network on one GPU half the network on the other GPU. And having a couple


AlexNet and ImageNet Explained

of connections between the layers so they had a couple of points where information could


AlexNet and ImageNet Explained

be passed between those two halves. And then at the end they came together into the final


AlexNet and ImageNet Explained

classification layer. Another important factor was that they swapped the more typical softmax


AlexNet and ImageNet Explained

and tanh activation functions of the time for a rectified linear unit or ReLU activation


AlexNet and ImageNet Explained

function which again further improved the efficiency of the model and also meant that


AlexNet and ImageNet Explained

they didn't require normalization that you would usually have to do if you had tanh or


AlexNet and ImageNet Explained

softmax because both of those activation functions over many layers you can get what's called


AlexNet and ImageNet Explained

a saturation in your activations which means the activations in your neurons either kind


AlexNet and ImageNet Explained

of push towards the two limits of one of those activation functions. So for example with


AlexNet and ImageNet Explained

softmax you'd end up or your activations be pushed towards a one or zero. Nonetheless


AlexNet and ImageNet Explained

they did use another type of normalization called local response normalization but that's


AlexNet and ImageNet Explained

not really used anymore. Nonetheless for AlexNet that was still a critical component. Now another


AlexNet and ImageNet Explained

super important thing that is still used today that AlexNet introduced was the use of overlapping


AlexNet and ImageNet Explained

in the pooling layers. Now pooling was already used in convolutional networks and it essentially


AlexNet and ImageNet Explained

just summarizes a window of information from one layer into a single activation value in


AlexNet and ImageNet Explained

the next layer. Now overlapping pooling does the same thing but there's an overlap in the


AlexNet and ImageNet Explained

window that gets passed along in the preceding layer. So there's always it always sees a


AlexNet and ImageNet Explained

little bit of the previous window. And they found that this reduces overfitting of the


AlexNet and ImageNet Explained

model and improves the performance. So that is how they got AlexNet to work and a few


AlexNet and ImageNet Explained

of the details behind actually you know how it worked and why it works so well at the


AlexNet and ImageNet Explained

time. Now I think it's great to talk about all this but as I said at the start I think


AlexNet and ImageNet Explained

it's better to go through everything or back everything up with a little bit of code. So


AlexNet and ImageNet Explained

we'll go through a notebook that you can find a link to the Colab version of this notebook


AlexNet and ImageNet Explained

in the description below or if you're reading this on the Pinecone article page it will


AlexNet and ImageNet Explained

be in the resources section at the bottom. And yeah we'll start going through that.


AlexNet and ImageNet Explained

OK so we're going to start by downloading and pre-processing our ImageNet data set.


AlexNet and ImageNet Explained

So we're not using the actual ImageNet itself we're using another hosted version of ImageNet


AlexNet and ImageNet Explained

which is much smaller that we can find on Hugging Face. So to use this we will need


AlexNet and ImageNet Explained

to pip install a few things. So pip install data sets which is where we're going to how


AlexNet and ImageNet Explained

we're going to use the ImageNet data set and later on we're also going to be using Torch


AlexNet and ImageNet Explained

and Torch Vision. So install those as well. So this is a data set we're going to use.


AlexNet and ImageNet Explained

So we're using this Macie tiny ImageNet data set. Now this is a validation split and that


AlexNet and ImageNet Explained

contains I think it's 10,000 we can see here yeah 10,000 labeled images. OK and then we


AlexNet and ImageNet Explained

can see a single record in there. So we have every image is stored as a pill image object


AlexNet and ImageNet Explained

and they have these labels. So this one has label zero. We don't necessarily know what


AlexNet and ImageNet Explained

that means right now but later on we'll see how we can actually figure that out. So this


AlexNet and ImageNet Explained

one is referring to the actual training data set. So the training split of this data set


AlexNet and ImageNet Explained

does contain 100,000 of those labeled images. Now we can check the type is the object and


AlexNet and ImageNet Explained

we can see. OK. So when we're in a notebook like this we can just call this ImageNet and


AlexNet and ImageNet Explained

this is just how we show that in the notebook. So we can see it's a goldfish. So we can probably


AlexNet and ImageNet Explained

guess that label zero actually means goldfish. So there are a few pre-processing steps that


AlexNet and ImageNet Explained

we need to go through. So we need to convert all images into an RGB format so it will have


AlexNet and ImageNet Explained

three color channels. We need to resize all these images to fit the expected dimensions


AlexNet and ImageNet Explained

of AlexNet. We need to convert into a tensor for PyTorch. We need to normalize those values


AlexNet and ImageNet Explained

and stack. So when we have multiple images we're going to set them all into a single


AlexNet and ImageNet Explained

tensor. OK. It's to create our batch. So we start with RGB. AlexNet as I mentioned


AlexNet and ImageNet Explained

assumes all images have three color channels red green and blue. But there are many other


AlexNet and ImageNet Explained

formats that are supported by PIL objects. So we'll see here that we have grayscale.


AlexNet and ImageNet Explained

OK. So this is 201. This is a grayscale image because we have this L. And the other formats


AlexNet and ImageNet Explained

as well. So need to be aware of those. And we can see it's I think it's an alligator.


AlexNet and ImageNet Explained

An alligator in grayscale. OK. So we convert into red green and blue and we'll see OK.


AlexNet and ImageNet Explained

So grayscale. That's that's fine. It will still be shown as grayscale. But in reality


AlexNet and ImageNet Explained

this only has one color channel which is actually just like a brightness channel whereas this


AlexNet and ImageNet Explained

now has three color channels red green and blue but all of equal values across that across


AlexNet and ImageNet Explained

those three channels. So actually still show as being grayscale even though it is in RGB


AlexNet and ImageNet Explained

format. This is how we handle the RGB part. But we also need to resize the image to fit


AlexNet and ImageNet Explained

the expected dimensionality for AlexNet. So for AlexNet and for a lot of other computer


AlexNet and ImageNet Explained

vision models the height and width of the input images is expected to be at least two


AlexNet and ImageNet Explained

hundred and twenty four pixels. So we need to do that. We can by using this. So we're


AlexNet and ImageNet Explained

going to first we resize the images because these are very small images and they're not


AlexNet and ImageNet Explained

necessarily all going to be the square that we need to the two to four by two to four.


AlexNet and ImageNet Explained

So we resize them to be bigger and then we use a center crop to crop out any edges and


AlexNet and ImageNet Explained

make sure that is now a square image of that dimensionality. And yeah we're doing that


AlexNet and ImageNet Explained

using this transforms function from torch vision which is a very good way of pre processing


AlexNet and ImageNet Explained

your your image data has a lot of functions and we'll see we'll use a couple more of those


AlexNet and ImageNet Explained

very soon. So we have a look at our first image of goldfish image. You see it's now


AlexNet and ImageNet Explained

a bit bigger and we can also see it's kind of cropped some of it as well but we still


AlexNet and ImageNet Explained

get the you know we still get the idea of what is in that image. So if we compare that


AlexNet and ImageNet Explained

to this here we can see it's eye at the front there more of its head whereas here it's kind


AlexNet and ImageNet Explained

of almost chopped off. Now another thing we need to do is normalize all the values in


AlexNet and ImageNet Explained

these in these images. So RGB arrays tend to be in the range of zero to 255. We need


AlexNet and ImageNet Explained

them to be in the range of zero to one and we need to normalize them using these values


AlexNet and ImageNet Explained

that you see here. So this mean of zero point four and so on and the standard deviation


AlexNet and ImageNet Explained

is zero point two and so on. This is specific to the Alex net implementation from from Pytorch.


AlexNet and ImageNet Explained

So we go on the Alex net Pytorch page you can go down and it here we go. So the images


AlexNet and ImageNet Explained

have to be loaded into a range of zero to one and normalize using the values I just


AlexNet and ImageNet Explained

showed you. So that's why we're using those. And yeah so we create this process function


AlexNet and ImageNet Explained

and then we process our image through it and then we can check the size. So the final result


AlexNet and ImageNet Explained

here is going to be that normalized tensor that we want and it's in the correct damage.


AlexNet and ImageNet Explained

It has the correct dimensionality that we need as well. So yeah that's perfect. Now


AlexNet and ImageNet Explained

we want to pull this together and we don't want to do every single image like this. We're


AlexNet and ImageNet Explained

just going to put it all together for a mini batch of images. So we're going to go with


AlexNet and ImageNet Explained

the first 50 images because because they're all goldfish and we can easily check the Alex


AlexNet and ImageNet Explained

net performance on that single single object. So I'm going to redefine that pre processing


AlexNet and ImageNet Explained

pipeline using everything we've just done. So we'll resize we crop it to 10. So we have


AlexNet and ImageNet Explained

to do this by the way because Pytorch is expecting a tensor object. And before we normalize that


AlexNet and ImageNet Explained

we need to be in that tensor format otherwise we're going to get an error. And then we normalize


AlexNet and ImageNet Explained

it. So we go through every image in the first 50 images and we first convert any that are


AlexNet and ImageNet Explained

grayscale to RGB not RBG RGB and we pre process them. OK. And just append them to a list.


AlexNet and ImageNet Explained

Now that list we want to stack all those together into a single tensor. So we do that here and


AlexNet and ImageNet Explained

we get this final mini batch of our images. So we have a mini batch of 50 and we have


AlexNet and ImageNet Explained

those images that you can see with the dimensionality here. So with all that done we're now ready


AlexNet and ImageNet Explained

to move on to the inference step. So the prediction of the class label is for our images with


AlexNet and ImageNet Explained

AlexNet. So the first thing we're going to want to do is download the model which is


AlexNet and ImageNet Explained

going to be hosted by Pytorch. So we can do that here. So you can see a bit better. We


AlexNet and ImageNet Explained

import torch the Pytorch and we just do torch upload. OK. Pytorch vision is just the version


AlexNet and ImageNet Explained

that we're using. And then we have AlexNet and we're not going to train AlexNet. It would


AlexNet and ImageNet Explained

take a bit of time. So we're going to use a pre trained model weights. So this version


AlexNet and ImageNet Explained

AlexNet has already been trained. And then we just say to evaluation mode for our inference.


AlexNet and ImageNet Explained

So for the predictions we don't want to train it by default. I think it is in train mode


AlexNet and ImageNet Explained

which looks like this. We want to in evaluation mode. And then we can see the model structure


AlexNet and ImageNet Explained

here as well. So you can see AlexNet we have. So this is where we're creating the image


AlexNet and ImageNet Explained

features. So there's many of these convolutional layers followed by the radio activation function


AlexNet and ImageNet Explained

followed by the max pooling layer. And with each of those the model creates a more abstract


AlexNet and ImageNet Explained

tensor that represents the information from that image. So you can imagine here that the


AlexNet and ImageNet Explained

abstraction sort of feature that's been extracted is like OK there's some straight edges here


AlexNet and ImageNet Explained

and some some curved edges here. We go a little further and this is like OK this is a this


AlexNet and ImageNet Explained

is an animal or this is a fish. And then by the time we get to here it's like OK this


AlexNet and ImageNet Explained

is a goldfish. Hopefully. And then we move on to the classifier part. So the classifier


AlexNet and ImageNet Explained

is these three layers. So dropout this this dropout was added to reduce the chance of


AlexNet and ImageNet Explained

overfitting and improve the ability of the model to generalize. And yeah we have these


AlexNet and ImageNet Explained

linear linear linear. OK so these are the linear layers the fully connected linear layers


AlexNet and ImageNet Explained

that produce the final 1000 activations. And the highest of these activations represents


AlexNet and ImageNet Explained

the class that the model is predicting as being the class that identifies the image


AlexNet and ImageNet Explained

I saw. So that's the model initialize it. If we can it's better that we move the model


AlexNet and ImageNet Explained

over to either a CUDA GPU if available or more recently we have the Apple silicon chips.


AlexNet and ImageNet Explained

So if you are on a Mac with Apple silicon you want to use MPS. OK so that's the case


AlexNet and ImageNet Explained

to me. I have a I'm going to run all this on MPS. So we move the inputs to the device


AlexNet and ImageNet Explained

and we move the model to the to the device. Now when we move the model to the device it


AlexNet and ImageNet Explained

does this in place so we don't need to like we did here where its input equals. We just


AlexNet and ImageNet Explained

write this and then we run the model. So we set torch no grads say we don't need to calculate


AlexNet and ImageNet Explained

the gradients because we do that for training the model. We're just forming inference. So


AlexNet and ImageNet Explained

we get our outputs. We detach them from the model and then we can see the shape. So we


AlexNet and ImageNet Explained

have these 50 vectors of 1000 items. So that's 50 activations across all of our 1000 classes


AlexNet and ImageNet Explained

and we can see those here. OK. Now these are not normalized. So if we if you want to calculate


AlexNet and ImageNet Explained

the probability from this we use the a softmax function. So we would do that like this. OK.


AlexNet and ImageNet Explained

That would map everything to a probability distribution and you'd be able to get a probability


AlexNet and ImageNet Explained

of say the top five classes for example. But we don't we don't necessarily need to do that


AlexNet and ImageNet Explained

for what we're doing here. So we could actually skip this. So up here we are getting the output.


AlexNet and ImageNet Explained

So we could skip this the probability part and just replace our output and we will get


AlexNet and ImageNet Explained

the same result for what we're doing here which is taking the value or the index position


AlexNet and ImageNet Explained

of the maximum value out of those 1000 classes. So here we're getting one. OK. Now if you


AlexNet and ImageNet Explained

remember earlier on the labels that we had in this data set was zero for the for the


AlexNet and ImageNet Explained

goldfish. And the reason these are different is because the data set actually uses a different


AlexNet and ImageNet Explained

set of labels. So it's it's not actually the same. But if we if we do this so over here


AlexNet and ImageNet Explained

let me open this show you. So over here we have a text file where each class is separated


AlexNet and ImageNet Explained

by a new line character. So this is number zero. A tench. And number one is a goldfish.


AlexNet and ImageNet Explained

All right. So if we get that information. So number one we can see there's a lot of


AlexNet and ImageNet Explained

goldfish predictions here which is a good sign. We can import those those classes and


AlexNet and ImageNet Explained

we can create a spread prediction labels by just splitting the response we get from this


AlexNet and ImageNet Explained

by new line characters. And then what we do is if you print out prediction labels one we


AlexNet and ImageNet Explained

get goldfish. OK. So the text label for that prediction. And yes so we have the first 50


AlexNet and ImageNet Explained

images. All of those are goldfish and we can we can see here. So I'm just putting out the


AlexNet and ImageNet Explained

last three. See they're all goldfish. So we would expect everything all these predictions


AlexNet and ImageNet Explained

to be goldfish if the model is performing well. OK. And yeah we see for the for the


AlexNet and ImageNet Explained

most part that is the case. Now if we calculate the performance or the accuracy here we get


AlexNet and ImageNet Explained

72 percent. So that represents a top one error rate of 28 percent which beats the reported


AlexNet and ImageNet Explained

error rate of the AlexNet model in 2012 on the ImageNet challenge which was 37.5 for


AlexNet and ImageNet Explained

the top top one. However this is I will point out that this is just a single class. This


AlexNet and ImageNet Explained

is a single label. Goldfish right. And the model will perform better on goldfish than


AlexNet and ImageNet Explained

other things. OK. So when we test this across the if we test this across the whole data


AlexNet and ImageNet Explained

set one we have to map all of the different labels between the between the AlexNet model


AlexNet and ImageNet Explained

and the data set that we have here because the labels have kind of messed up. So it takes


AlexNet and ImageNet Explained

a bit of extra work. But if we do that the performance will not be as good. Nonetheless


AlexNet and ImageNet Explained

I think that is a pretty good result. So that's it for this video. That's our overview of


AlexNet and ImageNet Explained

one of the most significant events in computer vision and deep learning. The ImageNet challenge


AlexNet and ImageNet Explained

was hosted annually until 2017. By then 29 38 contestants had an error rate of less than


AlexNet and ImageNet Explained

5 percent. So over those years the model was the progress in computer vision just kind


AlexNet and ImageNet Explained

of went crazy. AlexNet ended up being superseded by even more powerful convolutional neural


AlexNet and ImageNet Explained

networks. Microsoft Research Asia was the first other team to beat AlexNet and they


AlexNet and ImageNet Explained

did that in 2015. And since then there have been many other state of the art convolutional


AlexNet and ImageNet Explained

networks that have come and gone. And even more recently there are the possibility of


AlexNet and ImageNet Explained

other networks coming in such as transformer models and disrupting the dominance of convolutional


AlexNet and ImageNet Explained

neural networks in computer vision. Now I'll leave you with the final paragraph of the


AlexNet and ImageNet Explained

AlexNet paper because it almost seems like they saw the future of deep learning. They


AlexNet and ImageNet Explained

noted that they did not use any unsupervised pre-training even though they expect it will


AlexNet and ImageNet Explained

help and our results have improved as we make our network larger. But we still have many


AlexNet and ImageNet Explained

orders of magnitude to go in order to map the inferotemporal pathway of the human visual


AlexNet and ImageNet Explained

system so to match human level performance. Now we know that unsupervised pre-training


AlexNet and ImageNet Explained

and ever greater models and deeper models were really sort of the key to all the improvement


AlexNet and ImageNet Explained

gains that we've got in deep learning in the past decade. So I hope that has been useful.


AlexNet and ImageNet Explained


Functional API - TensorFlow Essentials #2

Hi, welcome to this video on the functional API, which is the second of the two approaches for building models in TensorFlow Keras.


So we're going to go ahead and start our notebook.


I'm just going to import TensorFlow.


So the functional API can produce the same architectures as the sequential model, plus a few more thanks to a high level of flexibility.


So the functional API is ideal for when we have multiple inputs or outputs, different streams of information between layers or anything else that is more complex than a simple sort of linear model where each input and output is feeding into the next one and there's nothing fancy going on.


An example of what this might look like is something like this.


So you can see at the bottom here we have two inputs and then in the middle here we also have this stream of data being split into this one layer and then also being fed forward into the next layer as well.


This is something that you could not do with the sequential model.


You would have to use a functional API, which we're going to go through now.


So rather than defining the model first, we instead define the input layers, the network layers, which is the same as what we did with the sequential model.


And then we initialize the model with something called tf.keras.model.


Let's go ahead and define the input layer first.


So here we use something different.


We don't use a dense layer.


We're going to use a input layer, which is obviously specifically for inputs.


Whereas before we used input underscore shape to define the input shape, here we are using shape.


And we're going to build the exact same model that we did before, but using the functional model instead.


So the only difference here is that we also need to define the data type going into our inputs, which in this case will be an integer of 32.


If we don't define this, it will default to float with 32.


And then we can execute that.


And then we need to set up our network layers, which is exactly the same as what we did before.


So if we go over to this image again, we can see that we have this sort of network architecture.


So we have the 10 inputs, which is what we already just defined.


And then we have our 32 units in the middle here and two units on the output layer.


So we need to build that using densely connected neural network layers.


And we do that just like we did before.


The only difference is that here we are passing the previous layer into our new layer.


So our previous layer we have placed into this variable here, input tensor.


And we also want to pass it to our new layer here.


And then we are also putting this new layer or new transformation into our input layer.


Or new transformation into another variable, which is X.


I mean you can use these names as you want, but the inputs and the output layer,


they need to have names which are not overwritten by any of your other layers.


Because we need to pass those layers into our model initializer in a moment, which you will see.


So we will call our output layer Y.


And then we are going to use the Softmax activation here again.


And we pass X, which is our previous layer.


Now if we had more than just these layers, we could use X in the middle and overwrite,


and that wouldn't cause any issues.


Just as long as we don't overwrite the variables which define our output and input.


And the reason that we don't overwrite those is because we need them to initialize our model.


Which we do like this.


So inputs we have our input tensor.


And outputs we have Y.


So this is where if we did have multiple inputs or outputs, we would be able to put them together here.


So rather than having Y, we could have Y and Y2 for example.


But in our case we are only using one.


So now we initialize the model. And if we print our model summary,


we get the same output as we did with the sequential model with a few small differences.


So at the top here, rather than saying sequential, it says model, which obviously this is because we're using the sequential model or the functional API.


It will change what we have up here.


And then for the functional API, we also have a specific definition of the model.


We have a specific definition for the input layer as well.


Which is just because we have actually explicitly defined it here, rather than it being an input shape to one of the other layers.


And that is everything for the functional API.


So I hope that has been useful and I will see you in the next one.


How to Use Type Annotations in Python

Okay, so in this video, we're going to go through type annotations in Python.


How to Use Type Annotations in Python

So before we actually get to type annotations in Python itself,


How to Use Type Annotations in Python

I just want to explain a little bit of what type annotations are.


How to Use Type Annotations in Python

So type annotations are also known as type signatures.


How to Use Type Annotations in Python

And they're used to indicate the data type of variables and the input and outputs of


How to Use Type Annotations in Python

functions and methods in a programming language.


How to Use Type Annotations in Python

So in a lot of languages, data types are explicitly stated,


How to Use Type Annotations in Python

meaning that if you don't declare your data type, your code won't run.


How to Use Type Annotations in Python

So I want to show you an example of a hello world in C,


How to Use Type Annotations in Python

so that we can see what it means to explicitly state our data types in our code.


How to Use Type Annotations in Python

Over here, I have this quick hello world script.


How to Use Type Annotations in Python

And you don't need to read or understand C to understand what is actually happening here.


How to Use Type Annotations in Python

All I want you to focus on is that we have these two type declarations, int and char.


How to Use Type Annotations in Python

So what I'm going to do is first I'm just going to remove those.


How to Use Type Annotations in Python

So remove the char from hello.


How to Use Type Annotations in Python

And this also, those square brackets tell us that it's an array of characters, which is what char is.


How to Use Type Annotations in Python

So I'll remove that as well.


How to Use Type Annotations in Python

And also remove the int from main.


How to Use Type Annotations in Python

Now, if I save and exit and then try to compile this,


How to Use Type Annotations in Python

you have to compile your code.


How to Use Type Annotations in Python

When it is at C, which we do like this.


How to Use Type Annotations in Python

Okay.


How to Use Type Annotations in Python

We see that we get quite a few errors.


How to Use Type Annotations in Python

So mainly it says hello is undeclared.


How to Use Type Annotations in Python

And the reason we're getting that issue is because we can't declare a variable


How to Use Type Annotations in Python

without the char.


How to Use Type Annotations in Python


How to Use Type Annotations in Python

without already initializing it with a data type.


How to Use Type Annotations in Python

So it thinks that we're trying to access an existing variable.


How to Use Type Annotations in Python

So let's go back into our script and we'll just add those two definitions back into our code.


How to Use Type Annotations in Python

And let's try and compile that again.


How to Use Type Annotations in Python

You see, okay, it works perfectly.


How to Use Type Annotations in Python

So we can also just run that code and we get hello world.


How to Use Type Annotations in Python

So we can see that everything is running perfectly.


How to Use Type Annotations in Python

So I just wanted to use that example before we get into the Python to just demonstrate that


How to Use Type Annotations in Python

in these other languages, we do need to have a little bit of a


How to Use Type Annotations in Python

little bit of a


How to Use Type Annotations in Python

So with what we just saw in C, where we have to explicitly define types,


How to Use Type Annotations in Python

that's called a statically typed language.


How to Use Type Annotations in Python

Whereas with Python, where we don't have to explicitly state a data type,


How to Use Type Annotations in Python

that is called a dynamically typed language.


How to Use Type Annotations in Python

But I think that is more than enough on discussing generic type annotations.


How to Use Type Annotations in Python

Let's dive into type annotations in Python.


How to Use Type Annotations in Python

Now, the first thing I want to say is that type annotations in Python


How to Use Type Annotations in Python

are not make or break like they are in our C example.


How to Use Type Annotations in Python

They're optional chunks of syntax that we can add to our code to make it more explicit.


How to Use Type Annotations in Python

So if we added type annotations and they didn't quite match up to what we wrote in our code,


How to Use Type Annotations in Python

we wouldn't receive an error.


How to Use Type Annotations in Python

At most, we'll get a warning in our ID.


How to Use Type Annotations in Python

Okay, so switching across to Python, we define types like this.


How to Use Type Annotations in Python

So this is from Python 3.9 onwards.


How to Use Type Annotations in Python

And what we do is say we have a string type here.


How to Use Type Annotations in Python

We would define it using this syntax you see in the middle.


How to Use Type Annotations in Python

Okay, and then just like usual, we would have our string there.


How to Use Type Annotations in Python

So if we just compare that to what we normally do when we're not defining types,


How to Use Type Annotations in Python

all we get is this.


How to Use Type Annotations in Python

So all we're adding is this little bit here.


How to Use Type Annotations in Python

Now, that's when we're assigning a variable.


How to Use Type Annotations in Python

And let's say that we want to do the same but for a function.


How to Use Type Annotations in Python

So we're going to make this a very simple function where we take two integer values, X and Y.


How to Use Type Annotations in Python

And we're going to add those together.


How to Use Type Annotations in Python

So what we do is return X plus Y.


How to Use Type Annotations in Python

Okay, and here what we've done is specified the actual input types to that function.


How to Use Type Annotations in Python

Now, if we just come down here and let's try this out, let's create a new value.


How to Use Type Annotations in Python

Of course, it's going to be an integer because that's what we're expecting from our function up here.


How to Use Type Annotations in Python

And that will obviously be equal to add.


How to Use Type Annotations in Python

And here we put say four and five.


How to Use Type Annotations in Python

Okay, and everything looks good, right?


How to Use Type Annotations in Python

Now, with type annotations in Python, of course, Python is a dynamically typed language.


How to Use Type Annotations in Python

So if we run this and there was an error in our types,


How to Use Type Annotations in Python

unless it created an error somewhere else in the language, like maybe we tried to add a string and an integer,


How to Use Type Annotations in Python

of course, that would raise an error anyway.


How to Use Type Annotations in Python

But it wouldn't create an error due to type annotations.


How to Use Type Annotations in Python

And adding these type annotations won't make it so that we get type errors that wouldn't already be there.


How to Use Type Annotations in Python

All it does is it allows our IDE to essentially warn us when something doesn't quite look right.


How to Use Type Annotations in Python

So let's say here we replace that four with a string four.


How to Use Type Annotations in Python

Okay, and I'm going to save this.


How to Use Type Annotations in Python

And then we get this little warning underneath.


How to Use Type Annotations in Python

So it's highlighted. And if I hover over that, we see that it says argument one to add has incompatible type string expected an integer.


How to Use Type Annotations in Python

Okay, now, of course, usually this would come up with an error anyway.


How to Use Type Annotations in Python

So we're not really doing anything special.


How to Use Type Annotations in Python

Say maybe we did both of these four and five.


How to Use Type Annotations in Python

Now, in Python, we can add strings together. So this wouldn't raise an error.


How to Use Type Annotations in Python

But using the type annotations, we still don't get an error, but we do get this warning.


How to Use Type Annotations in Python

So at least we're aware that there's something weird going on.


How to Use Type Annotations in Python

Now, another thing that I just want to add to this here is that we can also specify the output type as well.


How to Use Type Annotations in Python

So here, of course, we're adding two integers together. We'd also expect an integer there.


How to Use Type Annotations in Python

So if we string here, we'd get another warning because we would be expecting to return a string as we specified here.


How to Use Type Annotations in Python

But this returns an integer. So we're getting a lot of warnings.


How to Use Type Annotations in Python

So let's put that all back to as it was before. So this will remove the first warning.


How to Use Type Annotations in Python

Then we do four and five. Now everything's good again.


How to Use Type Annotations in Python

Now, alongside these basic data types, so we have integer, flow, dictionary, list, we can also merge data types.


How to Use Type Annotations in Python

It's getting a little more interesting, I think.


How to Use Type Annotations in Python

So with both of these objects here, we would define both of them as dictionaries.


How to Use Type Annotations in Python

And so if we wanted to sum over every value within those dictionaries and we also wanted to include type annotations,


How to Use Type Annotations in Python

we could write something like this. So sum dict.


How to Use Type Annotations in Python

We have our variable, which is just our dictionary. So this is a dictionary type.


How to Use Type Annotations in Python

We could also put that we are expecting an integer out of that. So we can include that in there as well.


How to Use Type Annotations in Python

And then I want to return the sum of each of the values within the dictionary values there.


How to Use Type Annotations in Python

So we could just write that as say we're accessing the key for every key in the variable dictionary.


How to Use Type Annotations in Python

Now, this isn't the best way to write it, but this is fine.


How to Use Type Annotations in Python

And now that we've put that function together, let's try processing both of our dictionaries with that function.


How to Use Type Annotations in Python

So we have this string int dictionary. Now just copy that and we have int int.


How to Use Type Annotations in Python

OK, and everything's fine. There's no warnings there and we wouldn't expect there to be any warnings there.


How to Use Type Annotations in Python

But maybe the dictionary types that we would be expecting would contain a string as a key and a integer as a value.


How to Use Type Annotations in Python

So we can actually specify that by adding a little bit more to this type here.


How to Use Type Annotations in Python

So all we do is put in the square brackets. Now we have string, which is the key type.


How to Use Type Annotations in Python

And integer, which is the value type. Now let's save that and we see that now we have this warning.


How to Use Type Annotations in Python

OK, so the argument to this function has an incompatible type, which is dictionary, integer, integer.


How to Use Type Annotations in Python

Whereas we expected a dictionary with a string and integer as a key value.


How to Use Type Annotations in Python

OK, so that's pretty cool. So we can also merge different types to create more complex structures.


How to Use Type Annotations in Python

And what I want to do is return now to our previous example.


How to Use Type Annotations in Python

So this is pretty good. So for this example, do we really only want to allow integers?


How to Use Type Annotations in Python

And maybe in some cases you would do. But also it would be logical in this scenario to also allow floats.


How to Use Type Annotations in Python

So to do that, we can also use something called the union operator. Now I'm going to show you two versions of this.


How to Use Type Annotations in Python

I'm going to show you the Python 3.9 version and also the Python 3.10 version.


How to Use Type Annotations in Python

So for the Python 3.9 version, we need to go from typing import the union.


How to Use Type Annotations in Python

And then here we add our union. And we say integer or float.


How to Use Type Annotations in Python

And that's all it really is. So if we add both of those, we can now add 4.5, 5.5 and we won't return any error.


How to Use Type Annotations in Python

Now, of course, we'd also probably want it to be here as well.


How to Use Type Annotations in Python

And we could also specify that we expect it to return either of those.


How to Use Type Annotations in Python

So that's how we would set up alternative integer or float data type annotations in Python 3.9.


How to Use Type Annotations in Python

Now, with Python 3.10, we can do this in what I think is a much cleaner way.


How to Use Type Annotations in Python

So first we can remove this top import. So we get rid of that.


How to Use Type Annotations in Python

And instead of having union here, all we do is add a bar in between our data types.


How to Use Type Annotations in Python

And that is all we need to do for it.


How to Use Type Annotations in Python

So we do that for each of these and that will give us our Python 3.10 syntax.


How to Use Type Annotations in Python

Now, let's just undo all of that.


How to Use Type Annotations in Python

And the final type that I want to show you, or the final operator, is the optional operator.


How to Use Type Annotations in Python

Now, when we define a function with optional arguments, we would use this optional function in order to allow us to actually do that.


How to Use Type Annotations in Python

So this will allow us to say, OK, here I'm going to just make up another argument. We're not going to use it.


How to Use Type Annotations in Python

But this will be optional string. OK.


How to Use Type Annotations in Python

Now I've imported optional here and now using here.


How to Use Type Annotations in Python

So this will allow either non or a string in our code.


How to Use Type Annotations in Python

So immediately you can see that this means that we don't have to specify anything here.


How to Use Type Annotations in Python

So that's great. And if we do specify something, we can say op.


How to Use Type Annotations in Python

OK. No warning. And we can switch that around for the non variable and we won't see any warning.


How to Use Type Annotations in Python

Now, as well, if we didn't want to use the optional operator, we could also just write this using a union.


How to Use Type Annotations in Python

And it actually produces the exact same type. So we could write string and non.


How to Use Type Annotations in Python

And this would work in exactly the same way. There's no difference whatsoever.


How to Use Type Annotations in Python

So that's everything that I wanted to cover for this introduction to type annotations in Python.


How to Use Type Annotations in Python


How to Build a Bert WordPiece Tokenizer in Python and HuggingFace

Hi, welcome to this video. We're going to cover how we can build a tokenizer for BERT from scratch.


So typically when we're using transform models we have three main components.


So we have the tokenizer which obviously what we're going to cover here. We have the core model


and we also have a head. So the tokenizer is obviously what converts our text into tokens


that BERT can read. The core model is BERT itself. So BERT has like kind of like a core which we build


or train through pre-training. And then there's also a head which allows us to do specialized


tasks. So we have like a Q&A head or a classification head and so on. Now a lot of the time what we can do


is just head over to the Hug & Face website over here and we can say okay we have all these


tasks over here. If I want a model for a question answering I can click on here


and typically there's usually something we can use. But obviously it depends on your use case,


your language and a lot of different things. So if we find that there isn't really a model that suits


what we need or it doesn't perform well on our own data that's where we would start considering okay


do we need to build our own transformer. So in that case at the very least we're probably going to


need to build or train from scratch a core model and the tokenizer or transformer head.


So we definitely need those two parts and sometimes we'll find that we also need tokenizers


but not always because you think okay we already have tokenizers for say our task is something to


do in the English language but the model doesn't perform very well in our specific data set. It


doesn't mean that it doesn't it hasn't been tokenized properly. It can still tokenize all


that text probably pretty well as long as it's like standard English. But what we'll find is that


the model just doesn't quite understand the style of language being used. So for example if BERT is


trained on blogs on the internet it's probably not going to do as well on governmental or financial


reports. So that's the sort of area where you think okay we're probably going to need to retrain


the core model so it can better understand the style of language used in there. And then the head


is like I said before that's where we are training it specifically for a specific use case. So for


Q&A we'd probably want to train our model on a specific question answering data set so that it


can start answering our questions. Now in that case and it's in English we probably don't need


the tokenizer but sometimes we do need a tokenizer because maybe your use case is going to be


in a less common language. And in that case you probably will need to build your own tokenizer


for BERT. And that's really the sort of use case that we would be looking at in this video. So


we'll cover that building a word piece tokenizer which is a tokenized user by BERT and we'll also


have a look at how we can get or where we can get good multilingual data sets from as well. So let's


move on to what the BERT tokenizer is and what it does. Okay so like I said before the BERT tokenizer


it's called a word piece tokenizer. So this this line this text up here word piece and it's pretty


straightforward. What it does is it breaks your words into chunks or pieces of words hence word


piece. So for example the word surf is just probably most likely going to return a single


token which would be surf whereas word surfing the the ing at the end of surf is a pretty common


part of a word in English at least. So what we would find is this word here would probably get


broken out into these two tokens. Now where we see this prefix the double hashtag that's the standard


prefix used to indicate that this is a piece of a word rather than a word itself. And then we see


that further down as well so surfboarding gets broken into three tokens and then if we for example


compare that to snowboarding. Snowboarding and surfboarding are obviously kind of similar because


they are both boarding sports. The difference being one is on surf the other one is on snow.


And before we even feed these tokens into BERT we're making that very easy for BERT to identify


where the similarities are between those two objects because BERT knows that okay one of them


is surf one of them is snow but both of them are boarding. So this is helping BERT from the start


which is I think pretty cool. Now when we're training a tokenizer we need a lot of text data.


We need a lot of text data so well when I say a lot we let's say two million paragraphs is probably


a good starting point although ideally you want as much as you can. So what we will use for training


our data is something called the OSCA data set or OSCA corpus. Now OSCA is just a huge multilingual


data set that contains just an insane amount of unstructured text so it's very very good


and we can access it through Hugging Face which is super useful. So over in our code first if we


want to download data sets we need to pip install something called data sets. So pip install data


sets. I already have installed so I'm just going to go from data sets import load data set okay and


then through that we can use the data sets dot list data sets method. Let me sorry let me import


data sets as well import data sets. Okay and this will give us a very big list probably a little


bit too big showing us all the data sets that are currently available in the data sets library


which is quite a lot I think it's like a thousand just a fair bit over a thousand now.


Okay so we have all of these which is a lot. How many? Let me length data sets dot list data sets.


My internet is very bad at the moment so it takes forever to download anything. But that there are


data sets and this is one way of viewing those data sets but an easier way this is how many we


have in all of Hugging Face and there's new ones being added like every day so but an easier way


of doing this. So go to Google type in data sets viewer or Hugging Face data set viewer and just


click on a streamlet Hugging Face. So this is a streamlet app Hugging Face have built that allow


you to sort of go through their data sets. So you see over here we go over to data sets and I'm going


to type in Oscar because that's the one we'll be using. Oscar okay I type Oscar and then on the


right we should it should pop up so within Oscar we have all these different data sets so the first


one here is Africans the the language and then you have all these other ones down here. I'm going to


using Italian as my example here but Italian has a lot of data so if I click on here it doesn't


actually show you anything which is a little bit annoying but it's because it's just a huge data


set it can't show you everything. So in fact so that is 101 gigabytes 102 gigabytes of data there


so it's a lot but that's good for us because we need a lot of data for training. So if we want to


download that data set we need to do this so we write data sets or data sets and it's just a


variable name and we want to write load data set and then in here we need to write the data set


name so it's Oscar and then we need to specify which part of the data set it is so over here


it's a subset it's unshuffled deduplicated it.


If I can can't select it so never mind.


So deduplicated it and it's also unshuffled.


Unshuffled deduplicated it. So right looks looks good and then the other thing that we can do is


we can write split and we can specify how much of the data we'd like. Now when you when you use


this split it's still going to download the full data set to your to your machine which is


a little bit annoying but this is how it works. So I found that this isn't particularly useful


unless you're just loading it from your machine and you're saying okay I only want a certain


amount of data. What you can do if so this is a 101 gigabytes it's a lot if you don't want to


download all that you can write streaming if it's true and this is very useful so if you're


using a streaming device you can do that. So this is very useful so what this will do is create an


iterator and you can iterate through this object and download your your data or samples one at a


time. Now because I already have my my data downloaded onto my machine I'm just I'm going to


take the first I'm going to say 500,000 items simply because I mean obviously you want to be


using more samples than this but I'm just going to use this many because otherwise the loading


times on all this is is pretty long. I don't want to be waiting for too long.


And we also need to specify which data set or subset we're using here so typically we have our


train validation or tests in our data sets. I think we always have the train set in there


and then we can have validation test sets as well. So we'll load that and then what I'm going to do


is I'm going to load the train set and then I'm going to load the train set. So I'm going to load


the train set and then what I'm going to do is I'm going to create a new directory where I'm going


to store all of these text files. So when we're training the tokenizer it expects plain text files


that where each sample is separated by a new line. So I'm going to go ahead and create that data set


in here. So I'm going to create a new one here. I'm going to create a new one here.


And as we can see that's already created and if we go back to our train file and I want to


make directory I'm going to call this oska and then what I'm going to do is loop through


our data here and convert them into the file format that we need. So first thing I want to do


we are in that process because this can take a while. So I'm going to create this


text data list so populate this with all of our text and I'm going to use this


file count set at 0. So this is just going to loop through and we're going to


create all our text files using this here. So what I want to do is force sample


in TQDM, TQDM data set yes. Now I'm just going to pass. Okay let's run that and we


see that we get this this bar, this TQDM bar. You see we're not even doing


anything at the moment and it's already taking a long time to process the data.


So I'm actually going to let's I'm going to go down to 50,000 so I'm not waiting


too long. So let me modify that 50,000 and that should be a little bit quicker.


Okay it's much better. Now first thing I want to do is we're going to be splitting


each sample by newline characters. So I want to first remove any newline


characters are already within each sample otherwise we're going to be splitting


our samples midway through a sentence. So I want sample equals sample and in here


so if I can I show you I can show you a sample yeah we have ID and then we also


have the text. We want the text obviously so we just write text and which replace


newline characters if there are any hopefully there's not any way with space.


And then what I want to do is just append that to our text data. So text


data append sample and what I want to do so we can put all this in a single file


but then that leaves us with one single file which is huge. So I mean 50,000


samples it's not really a problem but we're not going to typically be using


that many samples it's going to be more like 5 million 50 million or so on. So


what I like to do is just split the data into multiple text files. So what I do is


I say if the length of the text data is equal to let's say 5,000 at that point I


want you to save the text data and then restart again and start populating a new


new file. So let's say with open so we need to open the file we need to save it


into this OSCA directory that we built before. So OSCA and I'm just going to


call it file count.txt. So you convert this into an f-string I'm not sure


why it's highlighting everything here and we are writing that file. So


with that that's why we want to do fp.write and then we just write our text