the cited research or is critical of it or refutes it.


[ML News] Google introduces Pathways | OpenAI solves Math Problems | Meta goes First Person

So you have positive and negative citations.


And this gives you a much more accurate picture of how a particular paper has fared in the


research landscape in how it was cited and not only whether it was cited.


This is done in part by an automated system.


And I believe they already have a giant amount of research articles in there and automating


these extraction of references and they are scoring them using deep learning model.


What else there is a paper to go along with it.


Check it out if you like and give site a try.


It isn't exactly free.


There are different tiers right here with different features.


But if this is at all helpful to you, I guess it might be worth it.


Facebook AI releases a blog post called teaching AI to perceive the world through your eyes.


This is a push by Facebook or Meta or whatever it is called right now to go away from the


standard data sets where you have some third person view of a scene to really first person


data sets.


So they have a bunch of collections of data from around the world from different people


in different life circumstances in many, many places.


And they collected first person data, meaning I guess these people had head mounted cameras


and had other sensors on and they recorded just doing everyday activities.


So the data set is called ego 4d.


And what I think is cool about it is the data set generation process is different from what


other data sets are not only the fact that it is first person and that it is, you know,


distributed all over the world and not just done by a single person or team, but also


because they just told the people, you know, just record yourself doing everyday stuff.


And then after the fact, they went ahead and they defined tasks and they annotated the


data for labels.


So they didn't have the labels in mind when they collected the data or maybe they had


them in mind, but they didn't collect the data specifically to get some labels first


collected the data, and then they put different labels over top.


So for example, different tasks that they imagine are memory tasks, forecasting tasks,


object recognition, whatnot, they have various layers of labels annotated by humans by crowd


workers on this data and the data set.


You know, you can imagine that these aren't the only labels.


In fact, it is very feasible that our different research group goes ahead and annotates the


data in a different way to create their own task.


The blog post highlights the difficulty of ego centric data, which is usually vastly


different from like a third person view.


As you can see here on the left, this object detector works quite well in third person


view.


However, in a first person view, it just kind of fails.


So is this a good way forward to build more capable systems or a step into dystopia?


I guess that's up to you.


But if you like working with data like this, then give the state set a try.


I'm not exactly sure how you can get a hold of it.


I think there is some sort of license attached.


But yeah, it's out there.


Tesla released apparently pretty randomly a guide to a configurable floating point format


and arithmetic.


So this is a very technical specification for eight bit and 16 bit floating point numbers


and arithmetic and is supposed to sort of standardize or give a format to configurable


floating point numbers.


So as I said, it's very technical.


It's actually also quite short.


And the gist here is that they say if you train AI models on really large scales, like


Tesla does, you might want to go down from 32 bit numbers to 16 bit numbers or even eight


bit numbers.


However, in these very low regimes, you only have whatever eight bit to play with.


And therefore, you can't exactly specify ahead of time how many bits should be the exponent


and how many bits the mantissa should be.


Therefore, this needs to be configurable.


So not like in a 32 bit number, you have exactly this many bits for this and this many bits


for that in these new configurable floating point numbers, this would be a variable that


you can decide as you use the number.


So that allows you to trade off what kind of range this number can potentially have


with the accuracy, the resolution that the number can have in a particular range.


We'll see whether this remains a thing that's purely used inside of Tesla or whether other


people are going to adopt it.


Microsoft introduces Pytorch direct ml they say train your machine learning models on


any GPU.


So this is a component for Pytorch that allows you to use any DirectX GPU for doing deep


learning.


And all that is necessary essentially is that in Pytorch, you don't say to CUDA like if


you have a CUDA device, now you say to DML to direct ml.


And that's it.


This works on Windows and on the Windows subsystem for Linux.


So if you're still a Windows user for whatever reason, good for you.


All right, more helpful things that I saw this week.


There are a lot of helpful things this week.


It's not only helpful libraries, it's the section is renamed to just help, like, help


me please.


PyWig is a high level batteries included neural network training library for Pytorch.


And yes, whatever you're thinking is said here at the beginning of the readme.


Does the world need another Pytorch framework?


Probably not.


We had this project when no good frameworks were available, and it just kept growing.


So here we are.


Yeah, respect.


Cool.


If none of the current frameworks please you, PyWig might be for you.


Lexa is a benchmark for zero shot reaching of goals.


This goes along with a paper by CMU, UPenn and U of T about reaching goals after discovering


the world.


So these agents, what they'll do is they'll essentially go ahead and they'll just try


out a bunch of stuff in the world without any explicit goals.


And after that, you give the models a picture of a goal to reach, and they're supposed to


reach it.


So this means you don't explicitly train the agents to reach that particular goal, or any


goal, you simply let them explore.


And after that, they have to reach a goal.


So Lexa is a benchmark that achieves this.


And as I said, this goes along with the paper that gives a very, very, very good baseline


for this benchmark already.


But the benchmark itself is available to download if you're interested in doing that kind of


research, give it a try.


Next Donnie Jarr Huffner tweets out excited to introduce crafter.


So this is a game sort of an open world game, long term reasoning, exploration, generalization,


made for reward agents and unsupervised agents.


It's called crafter and you move around and there's blocks and there's food and you have


to dig and you have to build and you have to craft things.


I've never seen anything like this before.


This is a first.


This has no relation to any game that I've seen so far.


No, it's, it's pretty cool.


So you can craft things as you can see right here, you can interact with stuff every world


is randomly generated.


Like this is a Minecraft clone, but amenable to machine learning research to AI research.


So that is pretty cool, because Minecraft just seems too complex because you can move


like in any direction and so on here, it's really discreet.


So these models, they have a much more easy time to go about it.


They've already evaluated different of these AI learning mechanisms on it, like dreamer,


PPO, rainbow agents, and so on.


And none of them really compare so far to human expert.


But I think the game is pretty cool.


It is available.


These RL agents can already do things like you know, dig holes, build bridges and so


on.


There's a very complex behaviors already emerging here it moves out of the way of a skeleton


and then another one builds a shelter.


Excellent crafter give it a try.


If this video gets more than three likes, we'll do a crafter let's play for sure.


Robert Lange releases a lightweight hyper parameter optimization tool.


This seems to be a cool kind of personal project by Robert and he released it with pretty good


documentation.


There's Colab there is an example.


And if you're just looking for like a very simple way to do hyper parameter optimization,


then this might be the library for you.


As you can see, there's different strategies for doing hyper parameter optimization and


different ways you can define them.


That's pretty much all you need even has the fancy decorator style as you can see right


here.


Very pythonic.


Sayak Paul released a Keras tutorial on mobile vid so this is a tutorial that will guide


you through implementing mobile visual transformers in Keras, which is quite neat.


So Keras still as easy to use as ever.


And this tutorial guides you through building the architecture from the ground up all the


way to training it at the end you convert this model to TF Lite so it actually runs


on your mobile phone.


Pretty cool.


John Seviero tweets out this demo is surprising combines vit with GPT to caption images with


great results.


And yes, actually, I was positively surprised.


This is a hugging face module where you take a existing text model like GPT two and you


take an existing image computer vision model like vision transformer and you combine them.


So first you start out with sort of random cross attention weights that you fine tune


just a little bit and that can have really really good results.


Essentially the model learns how to connect the latent representation from one model to


the other model and back.


So this is used right here to do an image captioning demo using GPT two and vit as I


said and training only about 7000 steps on the cocoa data set.


So you can see the result.


This is a man swinging a tennis racket on a tennis court that is very descriptive.


But that is just an unhumanly precise description of what's going on right here.


We have a blue and white street sign sitting on top of a pole.


Yes, that that is also a very, very, very precise description.


Person riding a skateboard on top of a cement floor.


Well, I guess that has some importance.


Is it just me or AI models just bureaucrats?


But yeah, pretty cool.


Bits and bytes is a library by Facebook research for eight bit optimizers and quantization


routines.


So they have a bunch of optimizers such as Adam Adam w RMS prop and so on that work on


eight bits instead of 32.


And that pretty reliably saves you 75% of the memory.


Something like Adam has two or three different buffers that for every parameter you need


to keep track of.


So this can pretty quickly get pretty large and saving three quarters of the memory has


definitely value.


Now I love that it's called Facebook research.


But if you hover it says meta research.


Is this gonna go well?


I don't know.


Also, is this supposed to be like a pretzel?


Like it is, is it supposed to be like a flat logo?


Or is it supposed to represent sort of like a Pringles chips, you know, like the saddle


in 3d?


Another helpful thing, user friendly introduction to PAC base bounce by Pierre Aucure.


Now this is something I have no clue about.


But I know it's important.


And I have learned it at some point, if you're trying to get into PAC base bounce, this is


a I believe over 60 pages introduction to it that seems to be quite well written, introducing


you to all the important concepts in it.


So if you're interested, give it a try.


Even face met whatever research releases, Xformers, hackable and optimized transformers


building blocks supporting a composable construction.


So if you're into transformers, and if you would like to recombine them, try out different


things inside of them, Xformers might be a great library on doing that.


So you see all of these boxes here, essentially, this library makes it pretty easy to just


rearrange them, connect them differently, and so on.


Xformerb is a speech processing universal performance benchmark.


This means that this benchmark has a bunch of speech tasks, so tasks in machine learning,


where the input is a piece of speech.


But the goal here is that you have one pipeline that generates a representation.


And then that representation can be fine tuned easily to all of these tasks.


So you're not supposed to solve all of the tasks from scratch, you're supposed to come


up with that pipeline that generates the representation.


If you work on speech, this might be very cool for you.


I don't know how to say this.


CCQA is a web scale question answering data set for model pre training.


This is a large scale QA data set that I guess you can use for pre training question answering


models.


Excellent.


Bagua is a library that claims to speed up PyTorch.


So they have a bunch of things in here for PyTorch.


For example, advanced distributed training algorithms, performance auto tuning, generic


fused optimizers, load balanced data loader, and so on.


So these seem to be specialized algorithms that in very certain cases where you want


to use PyTorch can potentially deliver a lot of speed up.


So if your problem doesn't fall into like the standard bucket where the library is optimized


for maybe you can find something inside of Bagua that is going to help you.


Bagua?


I don't know.


Tree extracts is a PyTree module system for deep learning in JAX.


So if you work with PyTree, this is it in JAX.


Good job.


PyTree for those of you don't know are essentially trees out of Python structures.


So here for example, a list which contains numbers and dicts which themselves contain


tuples and so on.


So a PyTree works with these kinds of objects and now you can use them inside of JAX and


Tree X helps you to do that in a more module oriented way or a more object oriented way.


Reuters writes AI can see through you CEOs language under machine microscope.


This article essentially says that things like NLP and speech sound analysis, they now


go after CEOs quarterly announcements, they analyze their voices, and they're trying to


just recognize when they're nervous and so on.


And they actually have a point in that they claim they can make better investment decisions


if they do something like this.


But as you know, as soon as you pay attention to anything like this, the CEOs are immediately


going to adjust and train to trick essentially these AI systems.


So they will use scripted speeches much more in order to not trip the NLP systems, they


will train their voice acting more, I guess, or let some press secretary speak for them.


All in all, it just seems to be like, you know, if you analyze a CEO's speech and to


to detect when they're lying and when not, and then make investment decisions, you'll


simply reinforce the like the sociopaths that have no problem with just straight out lying


that have no difference in their voice whatsoever.


So if you want to create a world of more sociopathic CEOs than it already is, I guess, then go


right ahead.


Just just do this.


This This is fine.


Excellent.


Cadbury, the company has apparently made this ad for Indian local businesses.


And it's not just an ad, but they've paid this Indian celebrity to record essentially


one ad and then they modified that ad using deep learning.


So they have like three product categories like shoes, and I guess glasses and watches


or something like this.


They've recorded the different ads for the different products.


And whenever the actor says the company name and the location of the company, they use


deep learning to change whatever the small business is.


So essentially, this is a deep fake from the same actor to his own face, but to make him


say something else.


So as a small business in India, you can go there and get your ad for your local business,


the system will actually make sure that people that are in your area are advertised with


your particular business and people in different areas will see I guess the same ad but the


actor mentioning a different business that is in that area.


Pretty cool.


There's a form if you're in India, you know, check it out.


And lastly, this shoe does not exist.


This is a website I guess it's analogous to this person does not exist, which was a famous


website that trained style gun to on a face data set.


So this is style gun three, which was recently released the alias free gun and it's trained


on a shoe data set.


So you can just refresh and look at shoes that the model has come up with.


I guess these shoes all look like they exist.


They might as well, who knows?


But yeah, if you're looking for unique design ideas, check it out.


I'm looking forward to many more things where style gun three is applied.


It seems to be the quality of these models and the ease of training them has come a long


way such that it is in fact possible to do this for many types of things where you have


decently amounts of data such as shoes, I guess.


All right, this was it for this week's ML news.


Thank you so much for being here.


Don't forget to like and subscribe and let me know what you think in the comments.


I value your opinions.


Definitely.


This is not just a trick to get the YouTube algorithm to promote the video more and all


of that kind of stuff.


Tuples, Strings, Loops - Python 3 Programming Tutorial p.2

What's going on everybody and welcome to part two of the updated Python 3 basic


series. In this video I'd like to talk about variables, tuples, strings, and


loops all in one go.


Let's get started. So first of all I'm going to make a new directory. I'm just


going to call this tutorials. Then I'm going to open up Sublime here and I'm just


going to do a file save as. We're going to save it to that tutorials here and I'm


going to say tutorial2.py. I'm going to try to save these in parts that way later on


when things we have actual thicker code I can share it on pythonprogramming.net.


This one probably you're not going to need the source code for but


eventually you might want it just in case you can't figure out what's


different. So first of all let's talk about variables. So a variable is just


anything that is a variable is everything in Python is an object and a


variable is just a word representation basically of that object. So for example


we're going to have a variable called programming underscore languages. Now


variables have a certain rules associated with them. You can't start a


variable with a number but you can have a number in a variable like programming


one equals some sort of thing. You can have underscores and so on. You can't


have like a string be a variable or something like that and obviously since


you can't start with a number you also can't have like an integer as a


variables you know name. But programming language is a variable so to define a


variable use the assignment operator not to be confused with a double equals


which is a question of you know does this thing equal this other thing more


on that later but you use a single equals and then we can pass something


like the following. So the reason why we're covering tuples now is it just is


going to make sense to cover tuples now. I mostly actually just want to show you


guys how simple Python is both to write and read and this is why I fell in love


with Python to start. So programming languages equals Python Java C++ C. I


think anybody can look at this and understand okay I get it it's a list of


Python or it's a list of programming languages. Unfortunately in Python it's


actually not a list and if you're ever curious to find out what is this thing


I'm dealing with you can use a the built-in the built-in type so you can say


type programming languages like this. Since sublime might be new to some


people that was a tab complete so programming languages and then you can


see it made a little suggestion I hit tab done. So that's how you can get what


what is the type of variable like what is the objects type. So if you wanted to


output that well we already saw a print before so we could just print type in


languages. So let's go ahead and run that so again ctrl B I just restarted this


machine so I'll pick Python eventually we'll set that as a default but for now


class is tuple. So this is actually a tuple so generally you'll hear lists and


tuples later we will cover lists but for now it's not necessary. And the purpose


of this tutorial my goal here is to just cover what we need because this is


really the way you're going to learn to program you're not going to learn you


know this fundamental thing and this fundamental thing that no you're going


to try to solve a problem and on your way of solving that problem you're going


to learn new things along the way use the things that you know and use Google


and a lovely lovely website called Stack Overflow and maybe even some reddit in


there or you can come into certain chats like ours is discord.gg slash syntax all


that kind of stuff that's how you actually learn. So right now we don't


need lists I'm not going to cover lists my goal here is to show you guys how


simple Python can be. So we know it to be a tuple the reason why it's a tuple to


define a tuple either you could do something like this where it's just a


thing comma a thing comma a thing or object comma object comma object or you


could encase it in parentheses like this so if we do this nothing has changed so


just to prove that to you let me just put these back we run this again and you


can see it's still class tuple nothing has changed but to make it pretty and


easy to read I'm going to remove those and now let's talk about a final thing a


loop in this case I'm going to use a for loop just because again it's going to be


a little easier to read than a while loop at some point we probably will


cover a while loop I just don't need it right now so if we wanted to like


imagine if you want to just dictate how you would how you would go over this you


would say you know for each of these let's you know maybe output them in


some way so we know print to output so it's you just using natural language we


would say you know for each of the languages in programming language print


the language or something like that so so what you could say is literally for


language and so language becomes this little temporary variable here so for


language in programming languages colon and then enter because we're going to


start a new code block underneath this for so anytime you've got you know the


definition of a function or a for loop a while loop later on we'll do some if


statements stuff like that if something is the case colon new line and then


you'll see that the editor just automatically tabbed it over for me also


let's talk about tabs really quick I see that it has given me a silly tab that's


a mistake let me see if I can fix that right away


convert okay great so if everybody's following along and no matter what


editor you're in let's make this absolutely central change immediately so


as I highlight this I can see it's just one big line that means it's a tab the


problem with this is you obviously don't want to just pound your spacebar four


times but a tab should always be four spaces and the reason why is as you


upload this maybe to github or share it with your friend who uses a different


editor they might open it up and their settings might be totally different from


your settings and then suddenly the the tabs are the way that you sent them and


then they start adding things and then suddenly they don't mesh you've got some


tabs some spaces and it just goes goes down very quickly so I'm just gonna left


click on this I do want it to be for Google likes to but most of the world


accepts four spaces so we'll keep with four and then I'm gonna say convert


indentation to spaces so hopefully now you'll see dot dot dot that's what you


want anyway for language in programming languages let's go ahead and just print


the language ctrl B to run it boom we've iterated now over this we would


initially want to call this a list but in programming terms it's actually a


tuple the differentiating factor of a tuple is it's immutable meaning you


can't change it you can't add to it you're not going to remove from it so if


it's a thing that you intend to not be changed in your program should you maybe


accidentally you know run something that might affect it you can't do it whereas


a list can be things can be deleted and popped and added or appended and so on


so anyways later on we will see some slightly more tricky reasons why there's


a differentiation between these just for now know that at any point you can


always do a type also since I wanted to last I'm gonna right click here and pin


to taskbar so just to really drive it home again I know I brought it up before


but don't forget about the Python 3 docs so Python 3 tuple hopefully one of the


top results will be talks and Python 3 3 7 unfortunately Google's not the best at


indexing you know the newest greatest language but that's okay because tuples


really haven't changed much and you can probably just come up here and edit this


so rather than 3 3 let's try 3 7 see if we get away with that yes so you can see


first of all we've talked about lists but we haven't done anything with lists


but you can see right here they've got just a ton of things these are all the


little methods that you can use on a list again that doesn't need to make any


sense to you right now we'll talk about that as it comes up but here hopefully


I'm hoping they're gonna do a slightly more I'm guessing at some point they go


a little more in depth into what exactly tuples are but it says okay it starts


here so so yeah you could definitely read up on this learn more about tuples


if you're a little confused or you want to just learn more or not if you don't


really care to learn any more than where we are right now that's totally fine too


because we're gonna just try to use Python to solve actual problems not try


to learn everything in the atmosphere that is Python and all that so anyways I


think I'm gonna stop it here and in the next tutorial we're gonna actually start


our overarching objective which will be to make a simple text-based tic-tac-toe


game in Python which actually will should let us cover pretty much all of


the principles of Python I think you need before you can get out of the


basics and start doing things that actually interest you so that's all for


now if you have questions comments concerns whatever feel free to leave


them below you can also come join our discord server that's you can just go to


any old browser and discord.gg slash syntax should give you the invite you'll


need an account and all that but anyways yeah you can come and join our server


and we have a help channel so if you're having any problems come in there and


ask for help so anyways that's it for now questions comments concerns whatever


feel free to leave them below if you like the content give it a subscribe


thumbs up if you didn't like content thumbs down and if you would be so kind


tell me why the thumbs down so that's it for now I will see you guys in the next


List comprehension and generator expressions - Intermediate Python Programming p.4

What's going on everybody welcome to part 4 of our intermediate python programming tutorial series in this tutorial


We're going to be talking about list comprehension in sort of generators


But really generator expressions, and we'll be talking about generators more


Later on so first of all to start


One of the most common generators that you've probably been using is range


So when you say for I in range of five


This does not generate a list 0 1 2 3 4 it's a generator


And it generates a stream in the range of 0 to 5 which is produces 0 1 2 3 4


So but it doesn't do that and save it all into memory as opposed to say list


Comprehension or a list that actually obviously would would store that into memory so in Python 2


Range actually isn't a generator, and it does store that entire list into memory


Which is why if you in Python 3 you can get away with saying like I equals range


This that's okay. You're going to be just fine when you do that right we can do that real quick


Okay, boom. It's already done. All right, but in Python 2 if you run that code


It's going to be like like it's just not going to happen and you're going to close out of Python


Okay, so because it's generating a list, and it's just going to blow your memory


so generator now


fundamentally


Well, I'll have to keep hitting this point as we go through this this entire course really but list comprehension


Or a list is going to be faster, but it's going to use your memory right it's going to use your ram to store that list


And that's why it's faster. It's already loaded boom. It's in memory


Generators on the other hand are going to be slower, but they're not going to use as much memory and


slower is like with an asterisk because building a list actually like when it comes time to


build and save that list to a variable that can actually sometimes be a


Pretty time intensive process so actually sometimes generators are faster


But in most like raw cases the generator is going to be slower. It's just not going to blow out your memory


Cool now that we have that out of the way how do we build these things so maybe you've seen things like I4i in


range 5 and maybe it's


XYZ equals I4i in range 5 what the heck is that so?


What that is is basically? It's the same as saying xyz equals empty list


4i in range not the same, but it produces the same output for some of christman that one


4i in range


5 x y z dot append I


Okay, these two things do the exact same thing we can print


XYZ and then do the same thing again


Okay, same output


So that's list comprehension now. What's it? What about a generator?


Expression so a generator expressions literally the exact thing


just instead we use parentheses


Generator boom okay, but those parentheses make all the difference because this is not going to store this into memory


It makes it's a generator now, so


so for example


if we do print


Let's do up here xyz


xyz run


One of them's an actual list. It's in our memory the other one


Hmm weird it's a generator object, and what do we do with the generator object?


Well as you might guess from before when we did I for I in range 5 you iterate over a generator object so in this


case you would say 4i in


xyz


print I and


Then you're able to iterate over it, but the magical thing about a generator is it was not it didn't store that range


Or didn't store this right as a list 0 1 2 3 4 as opposed to this one so if this range instead was this


Don't try this at home. I'll probably regret this let's see what are we using here 500,000. Let's do that


Well actually do that help us sort of visually time it I?


I thought it would be done by now, but maybe not


My goal is to show you that as soon as pretty much list comprehension is done the other range would be done


Truly range of five thousand is not too much to ask


It's too much to ask everybody. I'm so sorry


Oh my gosh, we're silly. Oh the problem is we're printing okay. Yeah, no, that's the problem


Sorry, dumb


of course


Let's open that back up again


I'm pretty sure that's the problem. Let's just say done


Try again, okay? Yeah, so


rather than this now, let's add let's go back to


500,000 or


What is this five million? That's five million? That's okay? We'll do five million. Yeah, okay five million is acceptable


five billion okay that one's taking a little longer


I'm sorry. I'm sorry. I'm sorry


Took off one zero. We'll see if that's any fast there. We go so okay. That's a good one so


waiting


Done, and then the generators like bam right because we didn't need to store it into memory


So once you start to actually want to iterate through it or do something with it the generator is going to take


Take a longer time


Okay, so that's an introduction to list comprehension and generators in the next tutorial


we are going to apply a function to one of these and


Also talk about kind of embedding them into themselves and stuff so if you have any questions or comments concerns


Matplotlib Tutorial 16 - Live graphs

What is going on everybody welcome to part 16 of our data visualization with


Matplotlib Tutorial 16 - Live graphs

matplotlib and Python tutorial series. In this video we're going to be talking


Matplotlib Tutorial 16 - Live graphs

about live graphs with Python so if you're following along linearly this is


Matplotlib Tutorial 16 - Live graphs

the code that we're we've left off on we're going to go ahead and delete it


Matplotlib Tutorial 16 - Live graphs

but if you're following along and you're going to continue along in the series I


Matplotlib Tutorial 16 - Live graphs

suggest you save this code somewhere otherwise you can go ahead and head to


Matplotlib Tutorial 16 - Live graphs

pythonprogram.net and like re-get the code. So anyway I'm going to delete this


Matplotlib Tutorial 16 - Live graphs

code and now what we're going to do is just purely post code or write code here


Matplotlib Tutorial 16 - Live graphs

that's applicable to this specific video. So first of all we want to import matplotlib.pyplot


Matplotlib Tutorial 16 - Live graphs

as PLT that doesn't change we've been doing that this


Matplotlib Tutorial 16 - Live graphs

whole time and really that will never change then what we want to go ahead and


Matplotlib Tutorial 16 - Live graphs

do is we're going to bring in the animation functionality so import


Matplotlib Tutorial 16 - Live graphs

matplotlib.animation as animation then we're going to bring in one more thing


Matplotlib Tutorial 16 - Live graphs

and that would be from matplotlib import style so we've kind of covered styles


Matplotlib Tutorial 16 - Live graphs

already we just use it to quickly make our graph look a little more appealing


Matplotlib Tutorial 16 - Live graphs

than it does naturally so first we'll do style.use and we're going to say


Matplotlib Tutorial 16 - Live graphs

style.use we'll do that 538 that seems to be my favorite one. Next what we're


Matplotlib Tutorial 16 - Live graphs

going to do is we're going to say fig equals plt.figure and then we'll add an


Matplotlib Tutorial 16 - Live graphs

axis whoops we'll fix this figure now we're going to add an axis so we'll say


Matplotlib Tutorial 16 - Live graphs

ax1 equals fig.add underscore subplot and then we'll do a 1 by 1 by 1 so add


Matplotlib Tutorial 16 - Live graphs

subplot I don't think we've really covered much of this we did subplot to


Matplotlib Tutorial 16 - Live graphs

grid before neither of these we've really talked about depth so really


Matplotlib Tutorial 16 - Live graphs

don't worry about it but this just means it's a subplot that's on a basically a


Matplotlib Tutorial 16 - Live graphs

grid sort of system that's a 1 by 1 and this is plot number 1 but we'll talk


Matplotlib Tutorial 16 - Live graphs

more about those later on. Now what we're going to do is we're going to say define


Matplotlib Tutorial 16 - Live graphs

animate and then we'll pass one parameter will be i and that'll be the


Matplotlib Tutorial 16 - Live graphs

interval and then now we're going to say is we're going to call graph underscore


Matplotlib Tutorial 16 - Live graphs

data and we're going to say graph data is equal to open and then we're going to


Matplotlib Tutorial 16 - Live graphs

create a sample file it doesn't exist yet but we'll just say sample file dot


Matplotlib Tutorial 16 - Live graphs

text with the intention to read and then we'll go ahead and read it now again we


Matplotlib Tutorial 16 - Live graphs

don't have that file yet so let's go ahead and create it now so this is where


Matplotlib Tutorial 16 - Live graphs

I'm coding all this this is the actual script that we're writing right now so


Matplotlib Tutorial 16 - Live graphs

just down here or actually we already have an example file from the previous


Matplotlib Tutorial 16 - Live graphs

tutorial so let's just use this one actually so double click on that just to


Matplotlib Tutorial 16 - Live graphs

bring it up basically it's just 1 2 3 4 5 6 7 8 9 10 and then for those are the


Matplotlib Tutorial 16 - Live graphs

X's then we've got some random Y's it really doesn't matter you can put in


Matplotlib Tutorial 16 - Live graphs

whatever coordinates you really want these are just the ones I'm going to


Matplotlib Tutorial 16 - Live graphs

work with so I'm going to leave this file up I'm just going to put it over


Matplotlib Tutorial 16 - Live graphs

here for now so this was example dot text so let's just change this to


Matplotlib Tutorial 16 - Live graphs

example text save that so now we have the file and then now what we're going


Matplotlib Tutorial 16 - Live graphs

to go ahead and do is we'll say lines equals graph underscore data dot split


Matplotlib Tutorial 16 - Live graphs

and we're going to split this by the new line okay so backslash n is new line


Matplotlib Tutorial 16 - Live graphs

then what we're going to do is we're going to have the X's equals to an empty


Matplotlib Tutorial 16 - Live graphs

list and the Y's equal to an empty list and then what we're going to say is


Matplotlib Tutorial 16 - Live graphs

for line in lines what do we want to do well we're going to say if len of line


Matplotlib Tutorial 16 - Live graphs

is greater than one so sometimes you might have like a line like an empty


Matplotlib Tutorial 16 - Live graphs

line at the end of your files so I always like to handle for this to just


Matplotlib Tutorial 16 - Live graphs

check to make sure that line is not totally empty it's a really common thing


Matplotlib Tutorial 16 - Live graphs

people are asked people ask like you know how do I trim away that empty line


Matplotlib Tutorial 16 - Live graphs

and stuff well you can just use some handling you can trim away that empty


Matplotlib Tutorial 16 - Live graphs

line with some code or you can just handle for hey if that line is empty


Matplotlib Tutorial 16 - Live graphs

ignore it and this is usually less less logic required so anyway so if the


Matplotlib Tutorial 16 - Live graphs

length of line is greater than one we're going to say X comma Y is equal to line


Matplotlib Tutorial 16 - Live graphs

dot split and we split by a comma because our delimiter here in our file


Matplotlib Tutorial 16 - Live graphs

was commas so we'll split by comma and then we're going to say X's dot append X


Matplotlib Tutorial 16 - Live graphs

Y's dot append Y and really that's it so then what we're going to do is after


Matplotlib Tutorial 16 - Live graphs

we'll get out of this for loop and then now what we want to do is we want to


Matplotlib Tutorial 16 - Live graphs

modify our axes so first of all to plot we already know how to plot we could do


Matplotlib Tutorial 16 - Live graphs

ax1 dot plot and we plot the X's and the Y's and that's all we have to do in our


Matplotlib Tutorial 16 - Live graphs

animation function in theory but the problem is if basically this animation


Matplotlib Tutorial 16 - Live graphs

function is just going to keep running it'll keep updating our figure and every


Matplotlib Tutorial 16 - Live graphs

time we draw something to the screen there's nothing to get rid of that


Matplotlib Tutorial 16 - Live graphs

something that we've drawn previously so we also want to call an ax1 dot clear


Matplotlib Tutorial 16 - Live graphs

before we plot the new data so that I'll wipe the slate clean and then redraw


Matplotlib Tutorial 16 - Live graphs

including new data if there is new data or not it's a relatively computationally


Matplotlib Tutorial 16 - Live graphs

light really the heavy computation is the actual drawing of anything and then


Matplotlib Tutorial 16 - Live graphs

the rendering of those things so clearing everything out is really


Matplotlib Tutorial 16 - Live graphs

doesn't cost us anything I mean it costs a little bit but nothing much so like


Matplotlib Tutorial 16 - Live graphs

some people kind of consider that like you're like wasting or something like


Matplotlib Tutorial 16 - Live graphs

you're clearing away and then you're drawing like 99% of what you had before


Matplotlib Tutorial 16 - Live graphs

but it really doesn't cost you much it's the rendering that cost you the most and


Matplotlib Tutorial 16 - Live graphs

having many many lines cost you a lot so anyway clear so that's our animation


Matplotlib Tutorial 16 - Live graphs

function here now we're going to do is come outside of this function that we've


Matplotlib Tutorial 16 - Live graphs

been writing and now we'll just do Annie or ANI for animate and we're gonna say


Matplotlib Tutorial 16 - Live graphs

that is equal to animation which is in reference to this right here that we've


Matplotlib Tutorial 16 - Live graphs

imported equals so animation dot capital F funk animation so this is a method


Matplotlib Tutorial 16 - Live graphs

that allows us to animate based on a function if you can guess we're gonna


Matplotlib Tutorial 16 - Live graphs

animate based on this function here so funk animation and then the question is


Matplotlib Tutorial 16 - Live graphs

where are we going to animate this function to well figure we've defined


Matplotlib Tutorial 16 - Live graphs

that up here then what is the function that we're going to animate that's


Matplotlib Tutorial 16 - Live graphs

animate and then finally any parameters that you're gonna pass and stuff we


Matplotlib Tutorial 16 - Live graphs

would do that and we're just gonna say interval equals a thousand this is in


Matplotlib Tutorial 16 - Live graphs

milliseconds so a thousand milliseconds is one second so every one second this


Matplotlib Tutorial 16 - Live graphs

graph will update along those same lines in order to this will update in the


Matplotlib Tutorial 16 - Live graphs

background but we don't have a PLT dot show so we need to call that now so PLT


Matplotlib Tutorial 16 - Live graphs

dot show and that's it so now we'll save and run this and up pops our current


Matplotlib Tutorial 16 - Live graphs

graph it is updating whether or not you know it and we'll come over here let's


Matplotlib Tutorial 16 - Live graphs

put the graph like right here and then this is the the sample file that we've


Matplotlib Tutorial 16 - Live graphs

been working on so I'm just gonna add I don't know 11 5 so 11 comma 5 control s


Matplotlib Tutorial 16 - Live graphs

to save and there you have it our graph is updated and I mean we can keep going


Matplotlib Tutorial 16 - Live graphs

like 12 11 if we want so you can see it just keeps updating 13 7 and so on so


Matplotlib Tutorial 16 - Live graphs

you can use this to really have updating graphs anywhere right like it just so


Matplotlib Tutorial 16 - Live graphs

happens that we're pulling from a static text file that only gets updates from us


Matplotlib Tutorial 16 - Live graphs

but as I've already shown you guys we can pull information for stocks offline


Matplotlib Tutorial 16 - Live graphs

for an example there and so you could have graphs pulling from some sort of


Matplotlib Tutorial 16 - Live graphs

stock price API and the graph is actually updating live on your screen so


Matplotlib Tutorial 16 - Live graphs

that's pretty useful and also another useful aspect people use live graphs for


Matplotlib Tutorial 16 - Live graphs

is like sensors so people will use matplotlib to display data from a sensor


Matplotlib Tutorial 16 - Live graphs

but a lot of times that sensor data is really live and it's coming in


Matplotlib Tutorial 16 - Live graphs

constantly so they would use these this animation function so they don't have to


Matplotlib Tutorial 16 - Live graphs

keep like floating their graph over and over or something like that anyway


Matplotlib Tutorial 16 - Live graphs

that's it for this tutorial if you have any questions or comments up to this


Matplotlib Tutorial 16 - Live graphs

point please feel free to leave them below otherwise as always thanks for


Matplotlib Tutorial 16 - Live graphs


Deep Ensembles: A Loss Landscape Perspective (Paper Explained)

Hi there, today we'll look at deep ensembles, a loss landscape perspective by Stanislav Fort,


Hui Hu and Balaji Lakshminarayanan. This paper on a high level explains the loss landscape of deep


ensemble models, so ensembles of deep neural network. And it hypothesizes and it shows through


experiments that each member of the ensemble by means of being initialized at a random point


will go and through optimization, go and end up at a different place in weight space. And therefore,


the deep ensemble is able to capture different modes of the functional space of the space of


solutions. They compare this to Bayesian networks, which are sort of promised to do the same thing,


but they often only characterize a single mode and therefore they don't generalize as well.


So join me exploring this paper, I think it's pretty cool paper, the experiments are cleverly


designed to show what they're supposed to show. And I generally enjoy this type of research because


it's kind of an explanatory research that shows you what's going on inside of these networks rather


than you know, chasing the next state of the art number. It's also an example of research that you


can still do while you're you don't have, you know, giant resources of compute, even though this is by


deep mind. But I do believe that this kind of research is still, you know, wide open and


available to academia. And whereas the other kinds, the state of the art kind, slowly goes


into more and more of the money game. Alright, in any case, join me in reading this paper. If you


like it, share it out, leave a comment to think to tell me what you think, and leave a like if you


enjoyed it. Alright, so we'll start off the abstract says deep ensembles have been empirically shown to


be a promising approach for improving accuracy, uncertainty and out of distribution robustness


of deep learning models. So what are deep ensembles really quick and an ensemble model and


we're in the classification setting. So in the classification setting, we have data points and


each data point has features. So which are the x x is some kind of D dimensional feature. And then


you have y which is the label. So that's in some, let's say that's some natural number or something


like this or is element of a class set. Now that's the complex numbers. It's its element of some,


some some bounded set of class labels. So it's either a cat or a dog or, you know, what, what


whatever you want. So you have a data set of these things and your plan is to use x to predict y.


If you build an ons, if you build a model, a deep neural network, for example, for this task,


you would simply characterize this function here, you would parameterize it as a deep neural


networks of many, many layers. If you build an ensemble, now, what you would do is you would


take the data set and simply train multiple different ones of these deep neural networks. So


you'll train multiple different ones. And if you now want to classify data point, you'll input that


data point into all of these three. And at the end, you would somehow aggregate and there are


different methods of doing this. But the most obvious one is simply either to aggregate by


the mean or the mode. Median, whatever you want, you could also kind of also learn something here,


but you can just average the predictions. And that will usually give you a better prediction than if


you only have one model. So this is called an ensemble model. And if the ensemble members,


these thing here are neural networks or deep networks, this is called a deep ensemble. So


why do we hope to become better? That's the point of this paper is to show what happens in the lost


landscape of these deep neural networks. And why do they perform better than other methods that


are supposed to achieve the same thing. So usually, when you build an ensemble model, what are you


hoping for, you're hoping to sort of learn a generalizable function, and they have this


drawing right here, where it's a bit of a you have to sort of think differently than you usually do.


So on the x axis, you have the space of solutions. So imagine that your neural network only has a


single weight. So this axis here is that single weight, or you can project or whatnot, this is the


space of different solutions. So after you optimize, you land somewhere on this axis. And you


can see that there is a solid line, which represents the accuracy on the training set. And


then there is a dashed line, which represents the accuracy of the validation set for a given parameter.


So what you usually do is you optimize one neural network to its very best training accuracy. So


let's say you start off here, what you would do is you would see my training accuracy is this high,


I need a different color, right here is this high. And you calculate the gradient and you could do


gradient descent. And that means you go down the loss up the accuracy. So you go over and over and


over until you reach this point right here, where you have maximum training accuracy. And then you'll


suffer some generalization loss like you're right here, it suffers some generalization loss because


the validation accuracy at that point isn't as high. But generally, it's correlated as you can see,


by the general overlap of these two lines of these two shapes right here. Okay, so this is called a


maximum a posteriori estimate, you simply optimize one neural network until the best training error.


There are different approaches right here, there are approaches that say, okay, we can do we could


do better. So first of all, what you see right here is rather peculiar. And you might not be


used to this that there are different peaks right here, there are different peaks, as you can see in


the training and the validation error. So they're correlated. And the idea is that neural networks


are very nonlinear. And we've known from other papers that they have many, many local minima.


And in fact, so this is one of the astounding things about neural network, most of these minima


are performing equally well. So even though the neural network has different local minima, they


all perform about equally well. And other papers even say they're all sort of connected on a


low loss landscape. So there are many, many things that are still mysterious about neural network.


But we know that there are multiple minima. And we know that we basically need to find one of them.


And it doesn't really matter which one they all perform sort of equally well.


Now, as you can, as you might imagine, there are people who aren't really satisfied with the


model. And their approach is to say, why don't we just capture this entire curve right here.


So if we could build a model that could not only, you know, not only tell us at this point right


here, you're this good, but could tell us that at any point, right? How good we are capture the


entire distribution of solutions. And these are usually in the category of the Bayesian neural


networks, they try to capture the entire distribution. Of course, that's not really


feasible, because you know, you always have to calculate that posterior. So what they end up


doing is they do some approximation. And usually they do some sort of a multivariate Gaussian


approximation, because you can calculate posteriors in closed form and so on. And this


paper, this paper's hypothesis is that these can only be calculated in the closed form. And so


that these can only usually capture one of these peaks. So they are very able to capture the


surrounding right here, they're taking capture very accurately, what happens around this particular


peak, they are very aware of the shape of the curvature here, and can tell you a lot of things


about it. So they can tell you, for example, that the validation so that you might want to be a bit


over here, rather than over here. But they cannot, they don't generally know about these other modes,


because they are only approximations. They generally don't produce multimodal solutions.


Another approach is a deep ensemble. Now, this paper shows that in general, if you train a deep


ensemble, what will happen is because you randomly initialize the deep ensemble, at some points, it


will happen that if you do gradient descent on all of them, they will end up sort of covering


all these different modes, they still they don't have an idea of you know, the curvature, oh,


sorry, this one shouldn't go here, this one should go here, the curve, they don't really


know about the curvature, but they will give you these different minima right here. And therefore,


they can capture the landscape much, much more easily. If you know that these three are minima,


you sort of, it might look something like this. And that's a hell of a lot better than simply the


Bayesian approximation that only is able to capture one of the peaks, but really accurately.


So their hypothesis here is that deep ensembles do this job of capturing the different modes of


the functional space much better than the Bayesian methods. And it is why the deep methods sorry,


why the deep ensembles work so well, because they end up in different minima. And that is,


it's really interesting proposition. And what I find really interesting as well are the experiments


that they do to show this. So they have a lot of these experiments right here. First of all,


to the setup, they use c4 10 c4 100, and so on. And on c4 10, you can see right here, they use a


small CNN, medium CNN, and a resnet. Now the small and medium CNNs, their accuracy is really, really


subpar. So take the results here with some grain of salt, because there are effects in these neural


network that are really qualitatively different. If you are seriously underperforming, like this


one, like if you have a seriously too small network rather than a large network. Now they do verify


all of their things also with this large network and 90% accuracy is acceptable for c4 10. I don't


think there's the big qualitative difference between 90 and 95 and so on. But the 64 if it were


only this, I would be rather critical of this work. But it's fine to if you if you see the effect at


64 and then some of the effects you check to carry over to the 90% one, I'm going to generally believe


you. Okay. So first of all, what they do here is they look at a training trajectory of just a single


run. So this paper is half about ensembles, but also half generally about what what does training


of neural networks do and they reach some very, very cool conclusions that even are independent


of deep ensembles. So here, the first thing we do is we have some initial random initialization


in weight spades of your weight. And then you do gradient descent and you run and you run right and


you get to some minima right here, some minimum right here. And then you do a second one. So you


initialize somewhere else. And because you initialize somewhere else, you run, you run,


you run, you end up at a different minimum. Okay, this is a property. So these are not convex


functions, right? We know about neural networks, you'll end up a different minima. But the minima,


they will, they will perform about equally well. So the question is, do those different minima


that perform equally well, describe the same function? Or are they fundamentally different


functions that just happen to reach the same accuracy? And the question is very interesting.


And this paper attempts to answer that. So here you can see in the description, on the left,


cosine similarity between checkpoints to measure weight space alignment, along optimization


trajectory. So we only consider one of these runs only consider the left one, for example,


and you plot it here. And here, this later one comes later, sorry. So you plot the left only a


single run and you ask yourself, the checkpoint that I have after epoch 20, how similar is it


to the checkpoint that I have after epoch five, that will be right here. Now, we have to read up


how they compare the checkpoints. And this is weight space alignment. Okay, so weight space


alignment, it basically means how much do the weights align in the cosine fashion, as you can


see right here, this is simply the cosine between the weights, this is one way of comparing two


functions. If two functions align in weight space, there's a decent chance that they describe the


same thing. So as you can see here, we go as we go down the optimization trajectory, of course,


each one is similar to themselves. But you can see that there is kind of a shift right here.


So at the beginning, the zero of checkpoint is very dissimilar to the checkpoint at the end.


But after very short while, you kind of cross over, and then all these checkpoints right here


are sort of similar. So the if you just look at two rows, you look at the bottom row, and you look


at the top row, the bottom row tells you how similar are the checkpoints during training to


the initial checkpoint. And you can see pretty quickly, they are very dissimilar. So at this


point right here, there's kind of a dissimilarity happening where the checkpoint goes away from its


initialization to something else. And the top row tells you how similar are they to where the network


ends up. And you can see that there appears to be a period in, let's say here, where this shift away


starts up until here, where it's kind of not similar to anything. But then after that, after here,


everything is similar to the final checkpoint. Okay, so this is sort of tells us a hypothesis


is that you initialize randomly somewhere you have this lost landscape, right? initialize randomly


somewhere here, and then you go go go and at some point you fall into one of one of those valleys,


and then you simply go to that to that valley. If you initialize somewhere differently, you can see


that at the beginning, you might be here somewhere, and then you fall into that valley over here. And


after that, you're pretty much set. So this is going to be our hypothesis from now on that


in these neural networks, you the initialization is basically you you are somewhere and you kind of


meander around a bit until you happen to go into one of these directions, which happens pretty


quickly. And then you fall into a hole basically. And that's that's rather a convex setting in that


thing. Okay, a really interesting thing that they do is a really interesting thing is that they


check the disagreement of predictions. So you might think that if a neural network achieves


65 or 90, let's call it 90% accuracy on c410, right, that there are just you know, there there


this data set, that's 100%. And there are just these 10% over here, that are just the hardest,


right. And the more you train, the more are you you're able to push this boundary to the right. So


if you train more, if you have a better network, you're just able to explain more and more of the


samples. However, this this experiment here is going to show that this is not the case. What they


measure is the disagreement in predictions, which basically means that if I there is this data set,


the validation data set, and if I have one random initialization, and I train it to 90% accuracy,


it will have it will say these, it will not be able to classify these here. But if I have the


same network, but a different initialization, it might not be able to classify these over here,


but will be perfectly able to classify these over here. This is a very also very interesting


property. And you can see right here, the disagreement of predictions as you go through


the training. So again, we're going to look at the bottom and the top row. So the bottom row,


and the top row red is very disagreeing, blue is very agreeing, you can see again, that


Oh, I introduced, again, I introduced the different runs, I'm already taking this away from


later. We are just looking at one single run for now. This this is a result that's going to come


up later, when we look at two different runs of the same neural network. And that's the astounding


part. Okay, here, we're just going to look at one run again during training. So we can see right


here at the beginning, of course, every checkpoint agrees with itself on the predictions. However,


you can see that pretty quickly, the checkpoints start disagreeing very quickly, everything is red


right here. However, on the top, you can see how much how much do these checkpoints agree with the


end with the 30th epoch checkpoint, and see that there is a period that is red, right from here


to let's say here. And then after that, they all start agreeing. So from here on out, it's all


pretty blue, which basically means that that they agree with the last checkpoint. So with the


that all of these agree with the end of the training. Again, this is our hypothesis here


that once you're in this valley, that the function kind of stays the same, and you only sort of


micro optimize the function. However, at the beginning, you decide into which of those valleys


you want to go. And these are different initializations will lead you to different


valleys. And that's what they show right here. So they do a t SNE plot of predictions. t SNE is a


method to project to down project high dimensional vectors. And this is the weight space projected to


two dimensions. So t SNE axis one, and two, these are rather arbitrary. It's just the if you think


of a PCA, it's the directions of maximum variance. And you can see the three different runs, they


immediately at the beginning right here, they immediately go, you can see they have they do


large distances at the beginning, between the steps of optimization. And they do in very different


directions just by means of being initialized at different points and having maybe a bit of


noise in the training process. But once they are at the particular location, they sort of just


kind of bounce around right here and try to find the best minima in that region. So


this is our first indication that the if we train the same network multiple times with random


initializations, it's going to end up at multi at different places. And what we're wondering is we


already know that a single network is very different at the end than at the beginning of


training. What we want to know is our two networks also very different, even though they're trained


on the same objective, just because they are at different places in the weight space doesn't mean


they are functionally that different, there are symmetries. And it's going to turn out yes, they


actually are very, very different. So this is right here, here, you can see two different things.


And we're going to read the plot along with it. Just so I remember what I'm seeing here.


So using two different architectures, okay, for each of these architectures, the left subplot


shows the cosine similarity between the different solution weight space. And the right subplot shows


the fraction of labels on which the predictions from different solutions disagree. Okay, so it's


the same as before, the left is the alignment. And now it's not during training. Now we restart


independently, we train the same network 10 different times. And after that, we're going to


compare the 10 different solutions. Remember, these all achieve roughly the same accuracy on


the data sets. And this is the same whether you go to a big architecture like this ResNet 20,


or to a small architecture like this small CNN right here, you can see that every single solution,


of course, agrees a lot with itself. That's the diagonal right here. But it's completely,


it's not aligned, it's completely orthogonal to all the other solutions. So all the solutions


in weight space are orthogonal. Now, there's still the chance that there is, you know, some symmetry


in weight space, because, you know, if I, if I have a neural network, I can just exchange the


exchange the connections. And if I also exchange the neurons, then it will be the same function.


However, you can see right here that they completely disagree. So this small CNN,


remember, it had like a 65% accuracy, the solutions, the red here, they disagree on 25%


of the labels. So that this is exactly this effect we saw before, we train one solution,


and it will not be able to classify these parts of the validation data set. And we train the same


network with the same data set with the same loss with everything the same, again, just from a random


initialization that's different, it will end up equally performing equally well, but it will make


the mistakes on an entirely different set of the validation data points. This is rather astounding,


I feel because I think most people are of the of the idea that all the kind of data points have an


intrinsic hardness. And if if we get to 70% accuracy, it will always be the same 70% of data


points that we misclassify or sorry, that we correctly classify. This is not the case. And


this is one thing I think this paper and this line of research does pretty cool is to look at


these networks in terms of their prediction agreement. So they go further and they compare


this to four different methods. So they say, okay, ensembles, ensembles are one method of kind of


doing these getting different solutions, which means we start from random initializations. But


there are other ones. So for example, there is just place this correctly or random subspace sampling.


So what does it mean? They say we start at an optimized solution. So you train a network,


one single network. And then we choose a random direction at v in weight space, we step in that


direction by choosing different values of t, looking at the predictions at configurations,


the theta zero plus TV, we repeat this from many different v, but always the same theta zero. So in


our original kind of drawing of this thing, we optimize one single network, let's say that's here.


And then we sort of wiggle around in here into different random directions. Now, of course,


there's only one random direction right here. If maybe we can look at this at at the so if here,


we have the loss landscape, and maybe over here, there is a bit of a of a thing. And over here,


there is a bit of a you you thought I was gonna draw. Okay, so we start here, and maybe that


converges to somewhere here. So what we're going to do is we're going to select random directions


in that space. And we're just going to go a few steps into this direction and compare the weights


now here, you can already see by the way, I'm drawing it that this will probably make you stay


in the same region. And our hope with ensembles is, of course, is that they are able to capture


all of the three different modes right here. But it is a way to obtain different solutions that all


also perform quite well, if you only perturb your solution by a little bit, it also works quite well,


and you can build an ensemble out of these methods right here, you can build an ensemble out of


these. In fact, these Bayesian methods, if you do these approximations with Gaussians, that's pretty


much what they will end up doing is they will end up characterizing the local, the local landscape


around one of these minimum. But here, we simply do it by randomly stepping into a direction.


That's the first method that we're going to investigate to obtain an ensemble of different


solutions. So deep ensembles means we initially random, we randomly initialize many times and then


train from scratch each member. This method here means we start from an hour from a solution and


we simply perturb it into random directions. The next thing we can do is we can do dropouts.


Subspace dropout. We again start at an optimized solution, apply dropout with randomly chosen


probability. Again, our hypothesis is going to be that this is going to keep the network rather in


the sort of same kind of functional mode and not switch over. Diagonal Gaussian subspace. Again,


we start from an optimized solution, you can see the pattern. And here we actually do some sort of


a Gaussian approximation to the functional space, we calculate a mean and a standard deviation,


and we draw samples of the parameters from that distribution, and then the same in a low rank


regime. And here you see what happens. So here is these things. For example, the random subspaces.


Here is this overlapped with the plot we saw before. So here we have three different trajectories


of runs. And then at the end of each trajectory, we take the best solution. And we do this random


exploration around it. And this here is the t-SNE, the t-SNE projection. Now this isn't,


I have to say this isn't a projection of the weights itself. Sorry, I did not say that before.


This is a projection of the predictions, I believe, of a subset of data points. So this is


the prediction projection of that. And you can see that if we perturb the solutions like this,


all of these solutions, all of these ensembles, they rather they stay in their basin of attraction,


as you can see right here. So with a deep ensemble, we would build an ensemble that,


sorry, sorry, sorry, we would build an ensemble that combines this point and this point and this


point. Whereas here, we will simply either build an ensemble that that combines points in here,


or we'll build an ensemble that combines points in here, and so on. And you'll see this for all


the different methods that we consider here, especially the Gaussian methods. And that's a


hint to why even though Bayesian networks explicitly try to capture the entire distribution


right here, what they'll end up doing is they'll simply end up capturing a single mode. And not,


and that's important because the single mode is always functionally sort of equal. We saw that


this is a trained training trajectory. And at the end of training after like this step right here,


all the functions are pretty much the same, right, they pretty much agree with the end optimum.


Whereas between the runs, they these functions completely disagree with each other. So it is


important if we want to build an ensemble to capture as many of these modes as possible. And


only the ensemble, the deep ensembles can do that. So far. So this is another experiment where they


show this lost landscape. And I really like these kind of plots. So what you see here is a plane,


it's a 2d plane. And the 2d plane is described by three points. So one point is the origin,


you see right here, the origin, that's the origin of that's the zero in weight space. Okay,


then what you have are the two optima. So you'd run an optimization two different times. Once it's


initialized here, and it runs to here. And once it's initialized here, and it runs to here. Okay,


so that defines the plane that we're going to look at. Now they for each single pixel in this


plane or actually for each single pixel in this half circle right here, they're going to run


they evaluated the networks. So what you'll do is simply you do linear combination of these


weight of the weight vector here at this optimum, the weight vector here at this optimum. And you


can for each point here defines a neural network with those weights. And you can evaluate it. And


that's what you get. This is the accuracy of the neural network at that point. So here you can see


very, very clearly that there are these two different modes right here. So each even though


they're initialized super close to each other, right, you can see this right here, they're


initialized super close to each other, because they are this is the flat area right here that


we saw before, because they are in the flat area. They're even though they're initialized pretty


close. The red one is a little bit more to this basin of attraction, the blue one is a little bit


more to this base of attraction. So they move over and as soon as they're in, it's like boom,


they go to the minimum of that basin. And this area is rather convex. And this area is rather


convex. And in the middle, you can see is a less is more loss. So no solution will go there. That's


how you get these different minima. That's how you get these different modes. And you can see the


accuracy from the color is going to be the same in each of the in each of the valleys,


consistent with our what we know so far. Now here, the pink stripe is a Gaussian exploration.


So if you now do a Gaussian perturbation Gaussian exploration around this minimum, you basically you


can see again, you don't get out of this valley, you don't you're not going to go to different


modes, the weight space is just too large, and you're going to simply be stuck in there. So the


only chance almost you have is to initialize again, and hope that you end up in a different place.


And I guess my hypothesis is that there are many, many more of these valleys of these basins, then


that you can you could ever capture. So basically, every single initialization that is different,


will lead you to a different one of these basins. I guess it's only a matter of the size.


So here, again, they do a function similarity. So in this case, it's the function similarity to


optimum one. And this is again, how many of the labels agree with the optimum right here. And you


can see that within this basin of attraction, you have fairly high overlap of the functional


similarity. But here, none, right? So 15% or so, it's not going to be zero, because they're going


to agree on like some of the examples, I guess there's still something like intrinsic hardness,


but they agree on almost none of the labels. At least, I guess if you normalize by their base


accuracy. So even though optimum two is performing as well, it is functionally extremely dissimilar


to optimum one. So these describe really different functions. And I really don't know


what to make of this, other than, you know, this each one of these is maybe sort of deciding to


to look at different features in the in the data set, right to or to maybe build different high


level features from the same low level features. And maybe we're still under parameterizing these


models, because not a single model can sort of look at both features at the same time,


as evidenced by the fact that each of these is always going to one of these things. Or it could


be that in fact, the task is way too simple. And it's it can be solved in like 500 different ways.


And each of these optima is simply one way of solving the task one way of combining feature.


And it's actually completely an over specified problem. That's another hypothesis, it would be,


I guess, interesting to look at these things. And I'm sure there's work on this. So you can see the


same thing for optimum two right here, where, okay, go away, where you can see that optimum one,


it agrees almost nothing with optimum two, right? It doesn't, it doesn't agree, there's not even a


hint of a valley right here, in the terms of functional similarity. That is very, very interesting.


So it really means that these two things describe two different functions.


They do these other plots right here that they call diversity versus accuracy plots. So


what they'll do is, they are going to have different models, and they're going to look at


them in terms of their diversity and their accuracy. So here, the y axis is going to be how


different are these functions. And that's going to be again, in fraction of labels changed normalized


by the base accuracy. So here, you can see, we're always start from this baseline optimum, this


baseline optimum has zero diversity zero, because it agrees with itself on all the on all the


on all the different labels, of course, and then we're going to disturb that using our four methods


that we defined before. So we're going to randomly subspace, we're going to drop out, we're going to


Gaussian, perturb it and so on. And the more we perturb it, the more diverse our function is going


to be naturally, right? Because we perturb the function, it starts disagreeing more and more and


more and more with our original optimum. However, what we also expect is, you know, if we are in the


local optimum, and maybe here, and you know, maybe the validation accuracy is sort of beside it,


or a little bit larger and so on. So if we perturb it a little bit, you know, that might not make too


much to our accuracy. But if we perturb it a lot, you know, we go actually up the loss landscape,


and then we get less accuracy also on the validation set. So that's what you see right here


in these in these this curve right here, as you make the function more diverse, so you perturb it


a little bit, you see that your accuracy doesn't suffer too much, you kind of stay at the same


accuracy. But as if you make it more and more and more diverse, you can see that the accuracy suffers


until the diversity of one basically means that you disagree on the maximum amount of labels that


you can. And so you're sort of sort of out of this valley right here. And also, you can see that your


accuracy goes to zero. So the more these functions disagree, the less their accuracy is, that seems


natural. However, you can see that these red stars right here, they seem to be also very different,


they seem to not agree with the original baseline optimum, but they seem to be doing perfectly fine


in terms of accuracy, be at the same accuracy. And those are the independent optima. Those are


the optima from runs where we initialize at a different point and then also trained.


And that, again, is evidence for the fact that there are probably other optima far away,


where these different initializations find this here. So they are very different in terms of


functional space, they're quite far apart, they predict different things. However,


in terms of loss, they're almost the same or actually the same. So this is very, very cool


experiments right here. And they do this for the different for different architectures. And you can


see that especially the larger architectures, this actually happens more pronounced. And they also


make the point of saying, if you go to harder problems like a CIFAR 100, or an ImageNet, this


this effect is more pronounced that you can see, these here are closer together as a curve. And


these, these are the the independent optima. So I hope you're all you're already on board,


and still know why we're doing these things. We're doing these things because we want to build some


sort of ensemble that captures the distribution of solutions in order to generalize better. Now,


we have two options. Either we start kind of from a an optimum and characterize the space around


that optimum, which is what these methods do right here. And what also the Bayesian methods do this,


even though they don't want to, because they do these approximations, because they're intractable,


they're going to end up doing this. Or our other our other option is to do the same thing,


our other option is to restart training a bunch of times. And then we end up at different optima.


And the point of the paper is it's better to do that, than to build the ensemble out of the of


these Gaussian methods or of these perturbation methods. And the paper, I guess the main claim


of the paper is why that happens. And it happens because the ensemble members obtain different


or functionally different. Okay, so exactly, that's what they do here. So they now build


ensembles out of these different things. And you can see that here on the x axis of the ensemble


size. So how many ensemble members do you have? And the dashed lines here are the baseline


accuracies if you just have a single model. And the test accuracy is plotted on the y axis. Now,


I actually was, I was not, that's not correct. You always build the ensemble out of random


initializations. But on top of that, you do these things. So what you can see right here is, if I


have this classifier, which is my original classifier, and I add on top of that, this PCA


Gaussian, you know, perturbation stuff, I increase in accuracy, okay. However, if I build an ensemble,


I increase in accuracy, if I build an ensemble out of 10 members, I increase in accuracy this much.


Okay. And then if because I can do both things. So if I increase, if I do an ensemble, and then on


top of that, do the PCA Gaussian, I gain another this much right here. So that's sort of evidence


for the fact that you'd rather build an ensemble than do this, these other methods of, of


approximating the Bayesian posterior of weights. So, yes, I'm, I'm sort of convinced. I hope you are too.


And they do a lot of, they do some more experiments right here where you can see that the difference


between, so this is single model, sorry, this is, I guess, here, accuracy. Oh, yeah, if you, this is


the out of distribution test. So you can take a data set, and you can corrupt it by corruption.


So there are predefined data sets, but you can also do it yourself, you crop it, you can do luminacy,


whatever, you can destroy parts of the image, you can see that having more ensemble members. So this


is your original models. Here is how they sync with increasing corruption. It almost doesn't matter


which ones of these methods you do, you see the bottom one is the original model, and you gain a


little bit by doing these things, but not nearly as much by building an ensemble and going here,


or actually an ensemble of two members or a five members, in which case you jump this much in


accuracy. So these ensembles from different initializations are also very, very, are also very


good at countering corruption, which you see also here. Yeah, so this is the JS divergence. Okay,


I've read that, but let's not go here. Video is already too long. And this is the last thing is on


image net test set and the image net corrupted set, where they pretty much show the same thing.


It's not as pronounced here. But you can see pretty much how the different if you go from


single model to ensemble with two members to ensemble of four members, there is a general


upwards trend. And the general upward trend is much less pronounced within each ensemble. So if


you go just go from method to method, then it is between the different groups of ensembles,


meaning that the ensemble is a much more pronounced effect that these other effects.


So I hope I have convinced you a little bit of how these subspaces look like how the lost landscape


neural networks look like, especially the fact that there are these different minima. And the


random initializations will almost always hit these different minima. And the interesting part


is that even though these different minima perform equally well, they are functionally very different.


And an ensemble of differently initialized and independently optimized models can actually


capture these different modes of the functional space. And therefore, if you build an ensemble


out of that, it will generalize better, because it kind of can draw information from all of those


different modes, rather than if you do some sort of Bayesian network, which will, because you have


to approximate usually with Gaussians, will end up only covering one of these modes. That was that


is sort of a, a good summary of what this paper says, I again, I enjoy research like this because


it's easy and it gives it kind of makes you thinks right. So I'll be thinking about these things for


a while now and thinking of new kind of experiments that walk do. And yeah, as I said, this research


is still wide open. We don't know so many things about neural network. And you know, tell me tell


me what you think is going on actually that that would be very interesting. And yeah, I'll see you


Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions (Paper Explained)

Hello there. Today we're looking at Implicit MLE Backpropagating Through Discrete Exponential Family Distributions


by Matthias Niepert, Pascal Minervini and Luca Franceschi.


This paper is a paper that we've discussed in our regular paper discussions on Discord


and so it is informed by everything that I have heard there.


If you want to take part in these discussions and influence my opinions, you're very welcome to do so.


The link to the Discord is in the video description.


Alright, let's get into this paper right now.


This paper proposes essentially a discrete layer for neural networks.


This is maybe how I can describe it.


And the basic setup is in this figure right here.


So let's say you have an input X which might be some sort of a continuous input like an image.


They do give an example.


By the way, the authors, they have quite helpful code that's available,


but also they have made themselves a little video about the paper.


And I also recommend that you go watch that video because it's quite helpful.


So what they give as an example in the video, which I find a good example,


is you have a map of, and I think they use even Warcraft maps,


but you have a map and there's like a lake somewhere and then there's like a little house right here and so on.


Your task is to go from the top left here to the bottom right.


So you need to plan your way somehow through that.


Now, you don't get this as a graph that would be directly input into Dijkstra's algorithm.


However, you get this as an actual image, right?


Yet the solution here is going to be some sort of a path, some sort of a gold path.


That's the label.


Or maybe something even derived from the gold path like how long the gold path is.


So maybe that's five long or something like this.


So it's very complicated.


You first need to recognize where can I even go based on the image on the left.


Then you need to find the shortest path based on you've determined where to go.


Then you need to evaluate based on that shortest path, you need to evaluate some property.


For example, as I said, how long is the shortest path or just, you know, follow the shortest path on the actual map.


So it's a mix of continuous and discrete elements and specifically the part in the middle that's described by this P of Z right here.


That is going to be some sort of a discrete solver.


In the case here, it's going to be a shortest path algorithm.


Now, the question is how can we run backpropagation if we only have the label on the right hand side?


How can we backpropagate?


I mean, we can backpropagate from the label through here, right?


This is a neural network that maybe determines some property of the shortest path.


But then how are we going to backpropagate through this layer right here back to this neural network


that's supposed to extract the input graph to the Dijkstra algorithm from the image.


And that is a challenge.


There have been some solutions already, for example, one famous example is a score matching.


Sorry, that is also an example.


But the famous example is the straight through estimator.


However, it doesn't always work.


It fails sometimes.


And specifically here, the authors propose a different framework in this implicit MLE framework.


We're going to look at how that's built up.


This is a very technical paper.


And I'm by no means an expert in these things.


I just try to give you a little bit of the idea of what's happening right here so that you know what's going on.


And if you have something like this in your neural network, like a combinatorial optimization solver,


or anything like this, then you can just go grab their code and use that as a layer.


It is really super simple.


All right, that was the overview.


Now let's get into the paper.


Hold on, this video is sponsored by Weights and Biases.


Weights and Biases is your one stop shop for all your machine learning needs.


It will track your experiments with a single line of code.


It will upload automatically all your logs, all your configurations, everything to your cloud.


It will automatically grab all the output, all the metrics, all the configurations of your experiments,


and store that in one neat location.


So you can see your experiments, you can track them wherever they run.


You can compare among the experiments, but you can go further, you can then tune your hyperparameters


according to the results of those experiments.


And all of this is done automatically in a distributed way.


You can literally sit on your toilet on your smartphone and tune your hyperparameters and start new experiments.


But it's not only experiment tracking and hyperparameter tuning.


Weights and Biases has tools for the entire pipeline of machine learning research from the initial idea


up until the deployment and beyond that when you actually want to track what you've deployed.


Weights and Biases has cool methods to track all of your data set and their dependencies to each other,


as well as your models and all kinds of other artifacts that you might produce.


They're very powerful visualizations for all the inputs and outputs of your pipelines,


as well as the models themselves.


All of this runs in the cloud. But if you're concerned about privacy, there are options to self host.


The system is free for personal use and for academics and they have great plans for enterprises,


small teams, large teams, doesn't matter.


So thank you very much Weights and Biases for sponsoring this video.


If you don't know them yet, absolutely check them out.


It's free, it'll make your life a whole lot easier.


Now let's get into the video.


As I said, the problem right here is that you have these kind of discrete tasks sometimes as a part of an entire learning setup.


So the paper makes different contributions, but here they're listed out.


They say we propose the implicit maximum likelihood estimation as a framework for computing gradients


with respect to the parameters of discrete exponential family distributions.


So what we want is, of course, gradients, gradients of this discrete process in the middle.


And the discrete process specifically is going to be formulated as a exponential family distribution.


And we're going to see how that happens.


They say we show that this framework is useful for back propagating gradients through both discrete probability distributions


and discrete optimization problems.


And that would be the example right here would be a Dijkstra shortest path algorithm or an integer linear program solver or anything like this.


In fact, they're one of the general formulations they have is for integer linear program solving.


IMLE requires two ingredients, a family of target distribution Q and a method to sample from complex discrete distributions.


We propose two families of target distributions and a family of noise distributions for Gumbel max based sampling.


So we're going to look into how that works and exactly what it contributes.


And then, yeah, we show that this simplifies to explicit maximum likelihood learning when used in some studied settings and experimental evaluation.


These points we're probably not going to go into too much.


Essentially in point four, they show that for some settings, this reflects already established methods.


It's in sort of a generalization of methods that have already been around of methods that are maybe specific to a given setting or problem.


And the experimental results, well, you just like their experimental results essentially show that their method, for example, outcompete the straight through estimator method.


So what's the deal with discrete things in neural networks?


The problem is, of course, that we can't compute gradient with respect to discrete things.


Now take, for example, the straight through estimator, the problem it's trying to solve or one of the problems.


You can formulate it like this. You have some X, you put it into neural network and out in the middle somewhere.


You are required for some reason to sample from some sort of distribution.


For example, you're required to, this produces a probability distribution over a few classes, let's say over four classes.


And then what you're going to do is you're going to sample one of the classes right here.


And then you're going to continue with that through the rest of your neural network until you're at the label.


Now, again, as before, you need to back propagate in order to learn through this network, which is easy.


But through the choice, through the sampling procedure of that of that inner layer.


And that's hard. So what the straight through estimator does is it's a bit of a trick.


It essentially in the forward pass, you do the discrete optimization, you do the sampling.


But in the backward pass, you act as if you simply propagated the distribution as such.


So to the forward pass, it is really a discrete sample.


But to the backward pass, it looks like you've simply you did you never sampled.


You simply pass the whole distribution and say, well, I'm not sure it's like 70 percent this and 30 percent this.


The way you would implement that usually is you have some signal. Let's call that H for maybe that's the histogram right here.


And what you would do is you would if you sample from H, that was going to give you like S.


Well, let's say let's say we take the most likely state. Right.


So we determine H and we take the most likely state, which which is let's say S is the arg max of H.


That is your sample. Now, what you would do in your forward pass is you compute the next layer H prime as S,


which and then plus H minus a stop gradient of H.


So the stop gradient. Am I doing this correct? No, of course not. Of course not.


Yes. Oh, yes. I'm doing this correctly. Of course. OK.


So let's analyze this in the forward pass. The stop gradient has no effect on the forward signal.


So these two here essentially cancel out these cancel out to zero.


However, in the backward pass, right, since derivation is distributes over addition and subtraction,


what you would do if you were to derive the gradient of H prime,


that's essentially the gradient of S plus the gradient of H plus the gradient of stop gradient of H.


Now, stop sorry, minus minus stop gradient of H obviously has no gradient.


So that goes to zero. The gradient of S is also zero because it's a discrete operation.


And most of these frameworks simply tell you, well, the gradient is zero. It's a discrete operation.


If you're not sure that this is happening, you may in fact also put a stop gradient operator around S.


And you can see what remains is the gradient of H.


So you see the trick in the forward pass. These two cancel out.


However, since in the backward pass, this by itself is already zero because of the stop gradient operation,


the gradient of H remains right here. This is a trick.


You can you can simply swap out a gradient in the backward pass for whatever you like with this trick.


People have used this to get gradients with respect to discrete operations like this.


But this paper right here is an alternative. And as they show in some situations, it is more appropriate to use that alternative.


However, it is also quite a bit more tricky. So what's the first thing we're going to do?


The first thing we're going to do is we're going to take that inner thing right here, that inner procedure.


And again, let's go back to the task of finding the shortest path.


So what's the input? The input is some sort of a graph, right? Where you need to find the shortest path with cost associated with each of the edges and some start and some end goal.


And what we want is the shortest path. Something like this.


Now, the first thing we're going to do is we're going to encode this problem into a binary vector.


Now, how exactly we do this is, I don't really know for shortest path problems, but we're going to encode this into essentially, well, not a binary vector, but I'm going to encode the problem into this vector theta right here.


So theta, in this case, what you would do is your theta vector, this is the theta vector, it will have, I guess, it will have probably for each edge, it will have an entry with the negative cost of that edge associated in the vector.


So the negative cost of edge one, the negative cost of edge two, the negative cost of edge three.


Now, why are we doing this? You can see that we are going to multiply this theta with another vector called Z.


And Z here is the, let's call it the solution or the proposed solution to this inner problem.


And Z is now a binary vector. So Z can either be one or zero in each entry. And it's going to be one if and only if this edge here is part of the proposed solution.


So any path in this graph can be represented by a given Z variable, right?


By simply setting a bunch of things to one and zero, I can select some of the edges. And if I've selected the correct ones, they will form a path.


And if I have selected the absolutely correct ones, they will, in fact, form the shortest path.


You can immediately see that for the shortest path, the inner product between the two vectors will be the highest among all the paths.


So this is how I formulate my problem. I'm formulating my problem between, as an inner product between a binary vector and some sort of a weight vector theta,


such that for the solution of the inner problem, like the shortest path algorithm or the k subset selection or the integer linear program,


such that for the solution of this problem, it is the case that this inner product is the highest possible.


Now you immediately see that, of course, I can make that inner product even higher by putting all of the edges to zero, right?


So, you know, Z right here, I can simply say zero, zero, zero, zero, zero. All the costs here are negative.


Ergo, I have no negative cost. Ergo, that is going to be zero. And that is going to be the largest possible. I've solved the problem.


What's the problem? This isn't a path in the original formulation.


So the last ingredient we're missing right here is what they sometimes here call capital C.


This thing right here, capital C is a constraint set. So capital C would define, in this case, what the valid entries for the Z vector are.


So Z must be in this capital C class. And I think C must be in the, yes.