Because above I was trying to call a refresh method.


Language Models are Open Knowledge Graphs (Paper Explained)

This is something that I haven't seen any other completion engine do yet compared to


a regular coding engine tab nine is trained on lots of open source projects.


And it combines this with your code and it predicts what you want to do compared to predicting


what's possible, which is what a classic engine does.


Tab nine it uses a GPT based model.


And it downloads that model onto your machine.


So the code never leaves your machine.


There is an opt in feature where you can run that in the cloud.


And that will just give you a bit of a better beam search and better quality predictions.


And it saves you a bit of RAM.


As you can see, I myself use tab nine, I just have it on by default.


And I'm pretty happy with it.


I use it through CFC integrated into my neo vim.


But you can also get it in sublime, atom, IntelliJ, VS code, even like Jupiter notebooks,


and you can use it together with classic completion engine.


So you can really get the best of both worlds.


So whenever you see me code in a coding video, look out for this TN marker next to the completions.


That's the completions by tab nine, it doesn't only work for Python, it actually works for


pretty much any programming language that isn't completely obscure.


If you go to this link within 72 hours of when this video is released, you'll get three


months of tab nine professional for free.


The professional version removes the project size limit of the free version.


And it also gives you access to that sweet, sweet cloud inference.


After the three months, you're automatically kicked out of the pro version, there's no


auto sign up, there's really nothing to lose.


I mean, the only bad thing here is that tab nine itself is written in rust.


If that's the worst thing about an offer, it's it's a pretty good deal.


Again, I use this myself, and I'm pretty happy with it.


So again, if you sign up at tab nine.com slash promotion slash Yannick culture, within 72


hours of when this video is released, you'll get a free three months of tab nine Pro, no


strings attached and now enjoy the video.


Thanks.


All right, I hope that was fun.


Let's get back to the paper.


Let's get into the paper.


So first of all, what is my first criticism of this paper?


Yes, the title, there are some disturbing trends in the last few years in in in machine


learning papers, and the disturbing trends can be maybe encapsulated with the phrase


is all you need.


So people have sort of since attention is all you need, since this paper, people have


discovered that if they just append this to whatever their paper is about, then the paper


will get much more notoriety.


And the same thing, I think is a bit at play here with this with the R, because in recent


times, we've kind of seen a bunch of papers that show equivalences between models such


as famous example is that the transformers are hopfield networks in some kind of in some


regard.


And it these papers are pretty cool, right?


Even if the two things are not exactly equal all the time, if you can say, look, there


is a setting there are, you know, under these assumptions under these settings in this situation,


these two models actually are the same.


That's a pretty cool recognition, pretty cool thing to show.


And it's very useful for academia and, and practice, I believe.


However, I believe the R keyword, the is keyword should be sort of reserved for when two things


are equivalent.


Whereas here, in the very first, at least they're honest, right?


In the very first sentence, they show they say, well, we show how to construct knowledge


graphs from pre trained language models.


So essentially, they're going to use a language model to approximately construct a knowledge


graph.


And they're also going to use a bunch of other auxiliary models that come all pre trained,


but still, they do not show an equivalence of language models and knowledge graphs in


this paper, not at all.


So I would sort of I see that you can get somewhere with these titles.


But yeah, maybe people will be disappointed kind of if they read the paper, which it is


actually a cool paper, believe me.


All right.


So, as I said, what we have usually is a corpus.


Okay, a corpus is simply a bunch of text pieces, you can think of maybe just the text in Wikipedia.


Okay.


Here, you know, the this Wikipedia page about Bob Dylan, Bob Dylan is a songwriter was awarded


the Nobel Prize signed Albo Grossman.


These are easy sentences, right there, there can be sentences are usually larger and longer


and so on.


And what you want to do is you want to extract a knowledge graph.


So the knowledge graph has two distinct things, it has entities, and one entity here would


be kind of Bob Dylan songwriter is an entity Nobel Prize and it is an entity, you can sort


of think of them as nouns, okay.


And then the second part in knowledge graphs are the relations here occupation, sign, award


received and so on.


So the relations connect to entities, there's always what's called a head of an entity of


a triple.


So ahead of a fact, which in this case is Bob Dylan three times, then there is a tail,


which is sort of like the object of the verb.


And then there is the relation which is described by the verb.


So here you can see there are two stages of constructing such a knowledge graph, any system


that does this probably goes through these two stages.


So first, you extract a set of candidates, which it's not the knowledge graph yet, because


these are still strings, right, you extract, direct a bunch of string triplets, as you


can see here.


And as we said, as the sentences get more complicated, it gets more and more difficult


to extract these kinds of triples and then the second part is that you need to map it


to a to a scheme to a to a schema.


And these schemas are usually defined by humans.


So here we're still going to rely on humans to define the schema.


So there is one list that says entities.


And the entities there are just the entities are listed, okay, by the humans.


And at some point, it says Bob Dylan, Bob Dylan, and it has a bunch of mentions of Bob


Dylan associated with it.


And it has a clear ID.


In this case, you see the ID is q 392 in that knowledge graph.


And the system not only needs to extract these facts, but then also map these facts to the


correct entities.


Sorry, map these facts to the correct schema entries.


This second stage right here is a bunch of standard tasks.


So especially mapping something like the, the word Dylan in its context to this entity


Bob Dylan, which you can think of it as like the Wikipedia page of Bob Dylan, right?


That's how the system usually work.


That is a task called entity linking, okay, entity linking, and similar tasks exist for


like for sign like the relation awarded mapping this to award received to this.


So maybe there's some kind of a dictionary entry award received and what it means and


a bunch of examples.


And you're supposed to map this to that.


These are standard tasks.


And the system that we are going to look at right here is not not much concerned with


these tasks.


It simply uses pre existing methods to do these things.


So the system we're looking at today does this first part right here.


It takes text, okay, this is text.


And it comes up with these candidate facts about the text, whether how this is then mapped


to the schema, that is a different question.


And it's so there, there are pretty cool things in this paper about this step.


But we're first going to look at the first step and then at the second step.


All right.


So how does this system do this?


And how does it do it that there have been machine learning models before.


But being machine learning, they all have like some sort of a training corpus where


you have kind of the facts as a training set.


And then you have a separate set of facts as a test set.


And you try to learn from the conjunction of the text and the training facts, how to


extract facts, not this system.


This system simply uses a pre trained language model.


So what's the reasoning?


The reasoning is the following.


We used to think that we could do NLP probably best with having a knowledge graph, right


with having this set of very structured data, we can answer something like what's the what's


the age of Barack Obama's wife, and then you could go to the entity of Barack Obama, you


could look at the relation spouse, you could go to Michelle Obama, you could look up her


birthdate, which would all be structured information in this graph.


So you could sort of answer questions like this and search engines like Google and so


on that they have this built in.


So there is kind of a knowledge graph entry, sometimes when you search an entity in Google,


that pops up.


And these have been very useful to answer questions like this.


However, in recent years, language models have become better and better things like


BERT or GPT-2 have become better than these expert systems, let's call them at answering


questions.


By the way, if you want to, if you want to hear a very, very cool and solid argument


of where these kind of expert systems where this kind of structured human annotated or


maybe extracted information can still come in in natural language understanding, I would


recommend the machine learning street talk episode we had with Wally Saba, extremely


interesting person.


And I just I can recommend listening to that this should be out any day now if it is not


already.


So the language models have become better and better at these tasks without having this


structured information.


So the hypothesis is, maybe these language models can already contain the information


that's necessary to construct the structured facts because the structured facts is what


we, you know, let's say should use to answer these questions because we feel that structured


information is better than unstructured.


The language models are pretty good at these tasks.


So maybe we can get the structured information out of the language models.


So that's what they do.


They say the contributions are as follows.


We show how to construct knowledge graphs from pre trained language models.


The knowledge graphs are constructed with a single forward pass of the pre trained language


models without fine tuning over the textual corpora.


I think this is the, this is kind of a very strong point about this paper.


And it's also shows that if you're some PhD student somewhere and you don't necessarily


have the resources to train the next GPT-3 model or even fine tune it, there is still


research to be done.


Simply if you have enough resources to forward pass your data, which is often much fewer


than to train one, you can still do very cool research.


I think this paper shows this explicitly.


Okay.


This helps researchers explicitly understand what the language models learn, bridging the


deep language model and the knowledge graph communities through enhanced model transparency.


Okay, they say we propose an unsupervised two stage approach, MAMA, M-A-M-A, which stands


for match and map to first match the candidate facts in the corpora with the knowledge stored


in language models.


That's the first step we looked at, then map the matched candidates facts to both fixed


and open schema to produce a knowledge graph.


And then they say they produce a new type of knowledge graph, which simply is the facts.


Sometimes the facts they extract, they can't really map to a schema entry.


And we're going to look at that because I think a bit critically of this, namely the


open knowledge graph consists of mapped facts in the fixed schema of existing knowledge


graphs annotated by humans and the unmapped facts in the open schema that are new in the


reference knowledge graph schema.


So what they claim here is that their system is finds these new relations that don't even


exist in the schema and is able to uncover, kind of build new additional schema entries.


And they call this the open knowledge graph.


I'm a bit skeptical of this as we're going to see.


So the first step, how do you come up if you have a sentence?


And this is a very poor example, I feel honestly, to do this.


I get it must be short, but it's a poor example.


But stay with me.


So you have the sentence, Dylan is a songwriter, and you would like to extract a fact from


this.


The paper is not really written clearly on how I mean, it is I could you can parse it


out.


But the description is kind of distributed.


So step one, step one is run spaCy, run spaCy.


This is a standard kind of library for NLP to extract noun phrases, or they call them


noun chunks.


Okay.


So step one is not, there's nothing to do with the language model, it is simply you


want to find the noun phrases in here.


The noun phrases are Dylan, and songwriter.


Now these noun phrases now define your head and your tail of the facts.


So you already have two things, right?


So the the entire task of what of their method they're proposing is so the step one is run


spaCy to find the head and the tail of facts.


Step two is question mark for now.


Step three is going to be use the entity linking system and the relation linking system to


construct the knowledge graph.


Okay, so step one is steal underpants.


And then step three is profit.


So what's step two, step two is obviously step two is where their system comes in.


Step two is here is the head, and here is the tail in the text, some hat where in between


there might be a relation, and we need to figure out where that is.


So how does this method figure it out?


You already see the assumptions here are very, very restrictive, right?


So you use spaCy to extract basically noun phrases, which means you probably already


going to miss a lot of things that are not recognized as noun phrase.


And they all they also say that that spaCy annotations are sometimes error prone.


And that's why they miss a lot of things.


And then secondly, the assumption that the relation must be in between the two things


textually.


Now, you can run the algorithm forward and backward, but still, it must be in between.


And it must sort of be encoded, let's say, as a semi accurate string in there.


I guess then that's up to the relation linker.


But already, these assumptions are super constraining in the the kind of things you can find.


And you'll see in the experiments that their biggest flaw is that they have a very, very


low recall.


I mean, so do all the systems on the task, apparently, but they still have a very low


recall.


And it's because they constrain their problems so much, I'm going to guess if they wouldn't


constrain their problems so much, then they would have maybe a better recall, but their


precision would just plummet because these these things if you let them run wild, they


just over extract.


So basically, every, every set every verb in every sentence is going to be a relation,


right?


So like I ate a banana.


I ate banana would be a triple, not necessarily a really valuable entry in a sentence.


In any knowledge graph, though, banana has a lot of carbs.


So I would want to know about that.


Okay, so you see that the task is now reduced from building knowledge graphs to simply given


a head head annotation had piece in the string span, and a tail span.


Extract any span in between the head and the tail that describes the relation between the


head and the tail.


So the way this algorithm does it, that's where it uses the language model.


Okay.


So here, it's going to do something that is going to be similar to dynamic programming.


If you've seen kind of the dynamic programming, and search algorithms, let's say, you know,


string matching algorithms, and so on.


This is going to be sort of similar in that, what we're going to do, we're going to start


from here from the head in the string, there could be text before it, right?


We're simply going to locate the head, Dylan, right here, and going to start, then we're


going to look at its attention matrix.


Now the attention matrix is we're going to cross out here, the attention matrix, if you


have done many, many videos on attention, the attention matrix basically in a sequence


means how much each token attends to each other token, right?


How much information is kind of sent from each other token to this token right here.


So this up here would be the query, and these would be the keys, the attention matrix specifies


that.


So since we locate things between the head and the tail, what we want to do is we want


to cross out, we want to disregard everything that's kind of behind the query, and only


look ahead in the sentence.


Okay, so that's why the sum of the attention matrix here is crossed out.


As you can see, these are the X's.


This is exactly because we only search in one direction.


So from each from the token, Dylan, we can look at three things we can look at is a or


songwriter.


And this question is simply, where do we go next with this algorithm, right?


There's no interpretation yet.


It's simply, where do we go next?


And the where do we go next is simply answered by just taking the highest scoring thing in


that column of the attention matrix, look at the attention column where of the token


Dylan, I take the highest scoring one, that's point three here is higher.


Okay, then I go to point three, and that means is gets into my candidate fact.


Okay.


And once I put is into my candidate fact, I then go to is so the next thing I do is


I go to is, and then I again look in the corresponding attention column.


And I see what's now the biggest entry here.


And the biggest entry is point four, which is songwriter.


And you can see here, now we skip the A, that's how we leave out some text, okay, by skipping


it basically.


So you can see that this this can create artifacts, right?


This can create like kind of holes in the middle and so on.


But we skip a we go directly to the point four, and then we discover up the point four,


that is our tail.


So now we put our tail into here.


And since our tail is the last word, we can stop the algorithm.


I yes, so there is no need to go on even if there were text behind the tail.


As soon as we are at the tail, which we already know, right, we're given the head and the


tail, we stop.


Alright, so the we simply go forward with always the biggest entry in the attention


matrix, until we reach the tail.


That's the algorithm.


This this there, it's described here.


But it's kind of described in this in this way where it has these actions like start


yield and like this, maybe I'm not understanding something, but it seems completely unnecessary


to kind of describe these actions.


And it basically start the search from the head, the head is added as the initial candidate


and so on, then in yield, it sometimes says with the largest score from the attention


matrix is appended to the end to yield the new candidate and so on.


But still, and then stop, we stop.


And the algorithm description here, it basically just says, while we're not done, if we're


if it's not the stop action, we continue.


It's it's sort of, it doesn't tell you anything like this is this is a super unclear description


of this algorithm.


Basically, the whole logic that you would want to know about is here in this action


manager, right.


So the action manager that gives you the action is doing the actual logic of figuring out


which token you know, you should do next and where you should go next, and so on.


This is nowhere in the algorithm, the algorithm just describes beam search.


So you can do this a little, yeah, the little more sophistication that comes in is that


you don't do this deterministically, but you actually do it via beam search.


Okay.


But you can you can just generalize this.


All right.


So the description is a bit floppy with the whole actions and action manager and whatnot.


And not describing the only thing that don't describe formally is how actually to select


the next token, which is basically the entire kind of meat of the algorithm.


In any case, you might, this is something that confuses me right here.


So fair enough, you know, they say here, we take the attention matrix and we cross out


these X's.


But they say they can take things up here, right, they can take things like Bert and,


you know, as I said, fair, Bert has a full attention matrix, everything attends to everything.


But they can also take things like GPT-2.


Now GPT-2 is an autoregressive language model.


That means that in GPT-2, if you look at it, then you produce each token one after another,


which means that when you produce, so each token, when you train, or when you evaluate,


even each token can only attend to the things in front of it, right?


You see the problem with what this thing requires.


This is also the same.


Okay, let's do that.


You see the problem with this method.


This method is the exact opposite.


Each token attention matrix is deleted such that only the entries ahead of it are in the


attention matrix.


You don't actually get GPT-2 to give you an attention matrix that looks ahead because


it only ever looks behind.


So maybe, maybe what's happening is that the query and key matrices are switched up in


some way.


In that case, when we want to interpret the algorithm, the way they write it down is,


if I am at a particular part of what I think is the relation between the two entities,


how am I going to find whether or not there is more to the relation, right?


There could be a multi-word relation, like has a child with or I don't know.


Can't think of any multi-word relations, or whether we kind of are done with the relation


and go to the tail.


What this thing is saying is that we should look at the language model.


So if this is really how it is here, and you are at the word is, what you want to know


if this is BERT, if this is a BERT language model, what you want to know is, if I were


to cross out is, if I were to delete this word, which other words in the sentence right


here that are ahead of me are very, very informative to predict this particular word.


That's kind of the query style.


And if the answer turns out to be songwriter is quite important for that.


Maybe Dylan is too, but we only look ahead.


If it turns out A, the word A is not as important as the word songwriter, right?


Because songwriter, yeah, it gives an indication that there should be is, because songwriter


is kind of a profession, and there's a person in front of it.


We don't look at that, but the attention matrix would have that in mind.


That's valid, right?


So that's how this construction is made.


However, if this is the key, we have to think of the other way around.


If we are at is, we look ahead and say, if I were to delete the word A, how well could


I reconstruct it from this word is, or if I delete songwriter, how well could I reconstruct


that from the word is.


I think both are, you know, there is interpretations probably for both of these methods.


But what I want kind of to convey is that none of these things are really amenable to


constructing a knowledge graph.


It's quite interesting that this stuff actually works, because all it asks is, how well does


one word inform about the presence or how well can one word predict another word.


And from that information, we construct this knowledge graph, which probably is a testament


to the fact that knowledge graphs maybe aren't so much about knowledge.


If you extract them from a corpus, but more about grammar, I would think that's the thing


that goes on here, because these language models are a lot about grammar, right?


A lot about how different words appear together frequently.


So given that songwriter is kind of a mix between grammar and basic word knowledge,


given that songwriter is kind of an object here, the word is being the verb is probably


quite important for it.


And that's exactly these triples, they always appear a bit like compressed sentences, which


are very grammatically relevant.


So I'm not buying this hypothesis that there is much knowledge in these language models,


and that's why this works.


What I much rather think is that they are really, really, really good at kind of grammar


and statistical association between words across the language, and that's why they can


extract these candidates facts so well.


So that's what I think about the algorithm.


They do constrain it some more, as if it doesn't already have enough constraints, but they


all make sense.


So they say the matching degree, which is simply the sum of all these attention matrix


entries that we've encountered during our search.


So all the ones we didn't skip, or to count it together are the matching degree of this


triple.


The matching degree must be above some threshold.


That's the first constraint.


Because so they give an example right here for the sentence.


Rolling Stone wrote no other pop song has so far really challenged artistic conventions.


And the extracted candidate fact is Rolling Stone wrote pop song.


Again you can kind of see here it's mostly going into grammar-ish.


So spaCy extracts Rolling Stone and pop song, and the language model here extracts like


the only verb in between, wrote.


So yeah, to limit, to kind of limit the, to limit the matching degree to say it must be


at minimum kind of some number, it makes a lot of sense.


Because if the matching degree is high, that means if we go by this attention matrix, it


means that these words that are in the candidate fact, they kind of as themselves, they follow


from each other.


So the language model thinks that wrote is a very good follow to Rolling Stone and pop


song is a very good follow for wrote, or the other way around depending on which way the


attention matrix is.


But that's kind of the language model thinks that these words together make sense in the


context of the sentence, of course, like in the context of this entire sentence.


So as I said, it's sort of, think of it as a bit of a summarization paper, but with more


constraints.


Candidate number two is that the frequency of R is above a threshold.


So the relation itself shouldn't be too specific, it actually should appear a bunch of times


in the corpus.


So what you do is, you know, you go through the corpus once extract all the facts, my


pen just dropped, you extract all the facts, or the all these candidates, and then you


kind of count them and go through the candidate facts again and delete all the ones that are


below a certain thing.


That's people usually do this with things like stop words or rare words and so on.


It's pretty standard makes a lot of sense.


And constraint number three, relation R is a contiguous sequence in the sentence.


Okay, so we have an example here from the same Rolling Stone wrote challenge conventions,


which the language model would like to extract, because again, these in the context of that


sentence, these words sort of, you know, they jump to each other in the attention matrix,


because you can predict them from each other very well.


But they say this must be a contiguous sequence.


So what I said before, I said this could happen with this constraint, they excluded.


Okay, so for the second part, where they actually have to map a candidate fact to a fact in


the schema, as I said, they use kind of pre pre made solutions, entity linking and relation


mapping with the schema, I won't go into this except to say that whenever they find a match,


they say that this is a mapped fact, whenever they don't find a match, they say, Oh, this


is an unmapped fact.


Okay, an unmapped candidate means that at least one of HR and T is not mapped to the


schema.


There are two types, partially unmapped facts is where some are mapped and completely unmapped


facts indicate that all HR and T are not mapped to the schema.


For example, Jacob was a registered Mennonite.


Now here, they so they, they say they have these different facts.


And, you know, it's a cool thing if a model like this can actually come up with new facts,


not so not only new mapped facts, which is something you would expect, right?


If humans provide some kind of a schema, then build a knowledge graph, this is never complete.


So if you can automatically kind of fill in missing facts, that's very, very cool.


Though I would say humans, if you construct knowledge graphs, humans should probably also


build kind of like negative connections, saying like, yes, it is conceivable that Elvis was


a vegan, because a lot of texts talk about it, but in fact, it is explicitly not, I don't


think that's what we have in the knowledge graphs so far.


But it would be cool if this model could fill in new facts.


Yes, to the schema, it would also be cool if it could uncover completely new relations


that haven't been hadn't been considered by the human makers of the knowledge graph, like


if the knowledge graph itself is incomplete, the schema is a man, you know, same argument,


the schema is probably also incomplete.


This paper is sort of trying to sell their system as something that can do that.


And I believe that to a degree, but also, also, Jacob was a registered Mennonite.


Okay, now, maybe I'm completely wrong from the sentence Jacob was a registered Mennonite


in Amsterdam.


I might be completely wrong, but Mennonite is a religion, I think.


And I'm very, very sure that any of these knowledge graphs with the schemas that they


have, have being in a religion or being of a certain faith in their relations table somewhere.


And I'm also pretty sure that Mennonite large enough that that would actually appear as


an entity, maybe Jacob not right.


Maybe Jacob is an unknown Jacob.


We don't know who Jacob is.


But this seems more like a failure of the entity linker and relation linker, then an


uncovered new relation or an uncovered new entity.


So yeah, take this stuff with a grin.


Now they they are very honest about this.


But just to say that that's probably what happens most often.


So here, you can see the graph for Bob Dylan, constructed from the Wikipedia pages that


are kind of, they say around the page of Bob Dylan.


So I guess one or two or three hops away, something like this.


And you can see the blue stuff is stuff that we already knew so that the human humans also


found when looking at this.


Then yellow stuff, I believe is either new relations.


So whenever things are annotated, it's a new relation in the schema.


So you can see, this is an entity in the schema because it's annotated.


This is a relation in the schema, but the arrow is new.


So the humans hadn't yet extracted the fact that Bob Dylan was or was a member of artists


united against apartheid.


Then the yellow also sometimes means that there is a new thing.


So here tour with is a relation that's extracted, that is not in the knowledge graph yet.


Also this one.


And you can it's pretty, it's pretty cool, right, that you can extract these things automatically.


There's a lot of yellow stuff here, which means there is not a lot of new information


that this extracted.


And a lot of this new information is actually mapped to the schema, right?


Bob Dylan residents in Duluth.


I don't know how to pronounce that, by the way.


Yes.


So that's fairly, fairly cool.


They do some of these tasks of these knowledge based tasks.


So in these tasks, what you'd have, I believe what you'd have is always you'd have like


a head and a relation given.


So you have a document and you are given a head and a relation and you're asked, what's


the tale of this?


And then you ask the system and the system will tell you.


So you have these baselines and these baselines, I believe they are specifically made to extract


these knowledge representations.


They might even be trained.


I don't know that, but you can see that the MAMA, even the smallest one here beats those


by quite a bit.


Now you can see that the recall is significantly lower than the precision, which is a direct


result of how many constraints on the system there are and tells you sort of what the going


forward, what the improvements can be.


So they analyze a lot of this.


And yeah, so a first recognition is that larger and deeper language models produce knowledge


graphs of higher quality.


Third language models outperform GPT-2 language models under similar model sizes, which is


interesting, is scalable to larger corpora, which again, as we said, you don't need to


train it and larger corpora embed more complete knowledge graphs, which is something we would


expect.


The other interesting part is the unmapped facts.


So the numbers you can actually compute only for the mapped facts, right?


Because that's where you have data, humans produced the knowledge graphs from this.


That's what you can compare with.


Now the unmapped facts, they say, they analyze, we turn to study the quality of the candidate


facts that are not mapped to the above reference knowledge graph schema, but are in the open


schema generated by MAMA.


We manually judge such unmapped facts generated by our best method from 100 sample documents


in Wikidata and TAC KBP respectively.


So they go as researchers, they look at these things, and they judge them whether or not


they're true given these documents in Wikipedia.


They say the quality of unmapped facts is very, so the claim is that they've looked


at them, and they are good.


We find that 35.3% of the unmapped facts are true on Wikidata.


We find that 83.2% of those true facts are partially unmapped facts.


For example, Bob Dylan tour with the Grateful Dead.


And yeah, here is an if this really isn't in the schema, right?


This is a nice relation that you might think humans would miss because touring with someone


is not the first thing that would come to mind if you had to come up with a bunch of


relations between entities, but it is something that is regularly useful, regularly used for


musicians.


So that is an application where certainly an automated system can even extend the schema,


right?


Whose relation is not within the scheme of Wikidata?


Well both head and tail are in the schema.


The registered, the remaining true facts are completely unmapped facts.


For example, this red Jacob was a registered Mennonite.


And they also say accurate entity detection is desired where they say a lot of the errors


are due to spacey detecting wrong, incorrect entities or due to incorrect or missing entity


linking by the, by that, those systems.


The rest errors made by mama are incorrect relation phrases such as uninformative relation


phrases.


For example, Bob Dylan made and his breakthrough.


What can you do?


What other, what other one, what other verb would you put there?


Yeah, but okay.


We're going to look at a few last things right here.


They have a bunch of, a bunch of experiments right here, which where they show, you know,


the beam size has an influence, this constraint number one and number two that we looked at


has an influence, right?


So you can tune these things a bit.


What is interesting here is that they try, they try to look at either the attention matrix


of the last or of all the layers.


And interestingly, the system performs better if you only look at the attention matrix in


the last layer.


Now they reduce that attention layer because there are multiple heads using max or mean.


You can see they perform similarly, but it is interesting that only the last and they


argue, they argue in the text that we know that the last layers kind of have higher level


features than the lower layers.


But I recall there are multiple papers like I've done videos about them, what does BERT


learn and so on.


I think even something in constraint in conjunction with lottery tickets and so on that show that


in a transformer at least, I think it is the middle layers that encode the most kind of


semantic knowledge because the lower ones, yes, they are for kind of low level features,


but the upper ones, they are again for low level features because the task right here


at the end is to predict an individual word or token, right?


So you'd expect that the features in the attention matrix there go back to kind of sort of more


grammatical features and so on and that the highest level features are actually somewhere


in the middle.


I don't know if they tested, if they only tested like all versus last, in which case,


yeah, I believe that.


But if they tested each one individually and it still turned out that last is the best,


that would kind of add to my hypothesis that what happens here is more kind of a grammatical


effect of extracting this correct candidate verb in between the head and the tail.


So that kind of gives more weight to my hypothesis.


So to repeat, my hypothesis is that it's kind of a grammatical thing that's going on here


because the only task of this model is basically to find the correct string span for the relation


between head and tail because it's already given head and tail.


And there from the text, their hypothesis is more like the language models have a lot


of knowledge built into them and we can extract that knowledge kind of, they make it sound


like the language model has this semantic knowledge in them.


Okay, okay.


So let's look at a bunch of mapped facts right here.


You can, okay, you can maybe check out a lot of them yourself, but we'll just look at like


one in each category.


Blah blah blah, mail, yada yada yada yada is in worse shape, however Klaus told press


conference at the Western city of Essen where yada yada yada.


And it extracts this company and it maps it to the city of headquarters.


Maybe they leave out some text here.


What I want to get to is the unmapped facts.


Where are the unmapped?


facts to just kind of show you mapped facts unmapped facts okay so the unmapped


facts what I feel and you can judge for yourself please what I feel just to


pre-bias you before we look at them is that a lot of times simply it extracts


things that are that are it extracts things that are not it simply can't


can't assign things right it's a failure to sign it's not a new thing because in


these schemas like you haven't seen the schemas but you kind of get a feel the


last which is the last table you kind of get a feel of what contains in it so


maybe get a feel for for what okay Ernst Haeckel was born 16th of February 1834


in Potsdam okay so the extracted thing is Haeckel was born on 17th of February


1833 in Potsdam okay so that it maps to this is in the knowledge base a schema


this is in the schema but was born on 17th of February 1833 in is simply a


failure of the relation linker okay he was also a pacifist until the First


World War yada yada yada Ernst Haeckel and then was on a and a pacifist are


both not in the schema now maybe pacifism isn't in the schema maybe


maybe though I would guess pacifism has a Wikipedia page so it must be in the


schema because it's a wiki data but was as you know the relation here with


something be like a political leaning or something like this which is certainly


certainly in the knowledge base right then you have things like Haeckel was


awarded the title of excellency so you have correctly Haeckel again recognized


award received is in the schema nice excellency as a tail and excellency you


know what what do you want like this is this is a this is not a fact right this


is the award or the title of excellency would be kind of the thing so this is a


failure of spacey so again I have I've seen little facts here that would


actually be of genuine a genuine addition to the schema that should be


considered and I absolutely believe that the schema is incomplete don't get me


wrong I like a 100% the schema is probably less than 1% of what it should


be right if we did a thorough job I just don't think that this system here is a


good like I think that the things that this system comes up with mostly are


simply failures of its subsystems rather than genuinely new entries to the schema


that's different from when it genuinely discovered when it discovers a new


mapping between already established things for example Pauline Baines


educated at this college right so these are new facts all fit in the schema and


the system might be very very nice for that all right so that was my kind of


estimation of this paper I hope I didn't rag on it too much as I said it's it's


very cool work actually I look at this appendix is giant go look at it check it


out please tell me what you think about it in the comments any feedback is


[ML News] DeepMind controls fusion | Yann LeCun's JEPA architecture | US: AI can't copyright its art

DeepMind controls nuclear fusion with reinforcement learning. Jan LeCun proposes a new


architecture for world models. And according to a reliable source, it may be that today's


large neural networks are slightly conscious. Welcome to ML News. It's Monday.


Stop. Let me tell you about a feature that I have just learned about recently. And it's crazy.


See, this video is sponsored by weights and biases. And they have this alert thing that you


can send yourself emails whenever your runs finish whenever your runs crash and at your own choice.


So if you're log your experiments with weights and biases, and if you don't, you should in your code,


you can simply put this alert statement right here. And that will send you an alert as soon as


that piece of the code is reached. And for some things, it's obviously helpful to pack that into


some sort of an if statement. But you can send yourself updates about your runs. This way,


you won't have to check them constantly. So you can keep track on the go. If you go to your account


settings, you'll see that scriptable run alerts are already activated. That means whenever you


put the alert statement inside a code, it will actually send you an alert. This works whether


it's in a script on a server or in a Jupyter Notebook. Now you can also activate notifications


on email or on slack whenever a run finishes or if it crashes a certain amount of minutes into the


run. This is super helpful if you are dealing with various problems such as nans appearing after a


while, and you want to keep constantly checking your runs. This could really help you as I said,


these notifications will show up in your email inbox or in slack. And they even have different


levels as you may be used to from logging statements. So this is an absolutely cool method


of keeping track of your runs without having to check if you go to this URL, Wanda b.me slash


Janek, then you'll get straight into the documentation of the alerts, please check it


out. There's also a collab to go along with it showing you directly how to use this thing. And


it's most beastly when you combine it with configuration frameworks such that if any run


exhibits any suspicious behavior, you can immediately get an alert see what's up. Thank


you so much to weights and biases for sponsoring this video. Again, weights and biases is your one


stop shop for ml ops, whether you're a researcher or in industry or a student or just someone who's


doing this at home for a hobby, they have something for you personal use is free forever. And please


check out the link to learn more about their awesome alerts feature. Let's get into the video.


All right, we have a lot to get through today. So buckle up the first story DeepMind has a new


blog post called accelerating fusion science through learned plasma control. Now this is


insane. They control plasma in a nuclear fusion reactor using deep reinforcement learning. So this


thing is a research apparatus called the variable configuration tokamak. And it produces this thing


called plasma inside of it. Now the goal here is to generate and control this plasma over time,


which would allow us to harness the energy of nuclear fusion. How exactly? I'm not sure, but it's


important that we can do it. Yet controlling this thing is really hard. There are a number of


magnetic coils that are super strong that go around and inside of this device and decisions about how


to steer them need to be made in very short time. And this is a highly nonlinear complex problem.


So DeepMind used a simulated environment in which they trained a reinforcement learning policy


in order to steer these coils. This augments the traditional control system that was already in


place and allows them better controllability of this plasma. This is really important because


if the plasma is mishandled, like if it touches a wall or anything like this, that can cause a lot


of damage to the apparatus. Now hold on, hold on right here. Are you telling me that deep


reinforcement learning, you know, this thing or, you know, you know, that thing right here? Yeah.


Yeah, that controls a nuclear reactor. Absolutely fantastic. Oh, yes, yes, please. More plasma,


more plasma right here, right here, right here. Yeah, more plasma is gone. Well, in any case,


they do actually achieve stable configurations of plasma, including ones that the human controllers


couldn't do before, such as this droplet configuration on the left. There is a paper


in Nature to go along with it. And this represents a significant step towards making nuclear fusion


more plausible for possible future energy source. The Google AI blog has a piece called Good News


about the carbon footprint of machine learning training, in which they analyze how they've


reduced the carbon emissions of machine learning training in the last years. They have this


principle called 4M, which stands for model machine mechanization and map optimization,


into which they categorize the various things that you can do to make your machine learning


model training use less energy and therefore emit less carbon. This starts by figuring out smarter


model architectures themselves by using more efficient machines by pooling machines into data


centers where you can cool them very effectively. And then by locating these data centers on the


planet where energy is maybe more green, or more readily available and emits less carbon as such.


They say these four practices can reduce energy by 100 x and emissions by 1000 x is pretty cool.


They have a plot right here and no, these aren't carbon emissions. These are actually


reduction in carbon emission. I know the plot is a little bit weird, but a y value of one would be


the 2017 state. And then, you know, reduction. Now at least part of this comes as a response to


a different piece of work by the University of Massachusetts, which completely overestimated


the carbon emissions of a previous work by Google on neural architecture search. So Google had this


paper on an architecture called the evolved transformer, which they found by neural


architecture search, one of the goals of the architecture search was to build a more performant,


more efficient model, which the evolved transformer is. So this external study had estimated the cost


of doing this architecture search. And it turns out, according to Google's own numbers, they


overestimated the amount of carbon emissions by 88 times. That's an 8800% error rate. What's even


crazier is this right here, they say, unfortunately, some subsequent papers misinterpreted the NAS


estimate as the training cost for the model it discovered. So the study was criticizing the


architecture search itself. Now, given the fact that you do architecture search, you can actually


waste some resources because you're going to get them in again, once you use the discovered model,


if it's really that much better. But apart from that, they did criticize the architecture search,


they overestimated by 88 times. And now other papers come in, and they just misquote the study,


and they claim that the estimate is the cost of training one evolved transformer model,


Google writes, in reality, training the evolved transformer model on the task examined by the


UMass researchers following the four M best practices takes 120 TPUV two hours and costs


$40 and emits only 2.4 kilogram of carbon dioxide. That is an error rate of a factor of 120,000. Now


I'm not saying there's nothing to be worried right here about the cost of training large


machine learning models, there certainly is. And within Google, they say that machine learning


uses about 10 to 15% of Google's total energy use split into three fifths for inference and


two fifths for training. So that's not nothing. There seems to be a narrative about big companies


bad, which I get, you know, understandable, but still, if the desire to complain becomes


stronger than actually looking at true numbers, the criticism might be a bit misplaced. And yes,


these big companies train larger and larger models, but they're also building more and more


effective data centers. And these larger models could potentially down the road, save a lot of


stuff and advances beyond the need for carbon based energy much faster. But I have one issue


with this article, whoever thought it was, it was a good idea to call this thing good news about the


carbon footprint. Like, could you make a title that anymore screams lobbying piece is like some


Christians at your door. Have you heard the good news about the carbon footprint of machine


learning training? I mean, the title here almost invalidates the article itself, but I'm gonna


believe them on their numbers. The meta AI blog, which surprisingly is still hosted at AI dot


facebook.com. Wait, wait, the certificate is made out to Facebook Inc. They never changed their name.


It's all just a big conspiracy. The metaverse isn't real. In any case, the meta AI blog has a


new post called young Lacan on a vision to make AI systems learn and reason like animals and humans.


And other than implying that humans apparently aren't animals, it details a new vision of a broad


architecture guidelines for building the next generation of autonomous intelligence. Now,


diagrams like these have existed for a while, you know, the different modules that we need in order


to interact with the world in order to build autonomous systems. But the specific focus here


is on the green bubble called world model. And the main ingredient Lacan suggests here is called


JEPA, the joint embedding predictive architecture. Now this pulls together a number of threads that


young Lacan has been pursuing and advocating for in recent years, such as energy based models,


such as self supervised predictive learning, and he advocates strongly for using regularizers


instead of contrastive learning. Now all of this by itself is of course nothing new. Meta previously


Facebook has investigated all of these things in a number of architectures which have been really


successful. But to get it together into one model is sort of a suggestion by Lacan. And he says they


haven't built this model yet, but it is a plan on the horizon. The cool thing about this model is it


can be composed, for example, it can be made to do short and long range predictions, it can do


supervised as well as unsupervised as well as reinforcement learning task, it can be arranged


in a temporal and hierarchical fashion to learn layers of abstractions and build up plans into


the future. Focusing on world models is definitely a breakaway of other reinforcement learning


efforts, which directly try to go model free from input and perception to action without having


some sort of an intermediate model. Now if this all seems pretty vague to you, then yes, it's just


a plan for now. But it sounds pretty cool. Young Lacan has also given an extensive talk about this


which is on his YouTube channel. Yes, young Lacan is a YouTuber bet you didn't know that. So leave


a comment right here if you would like me to make a dedicated video about what we know so far about


the JEPA architecture and how it might work. Researchers from the Max Planck Institute of


neurobiology published a paper called a biophysical account of multiplication by a single neuron


investigating a real neuron biological neuron in a fruit fly that can do multiplication or something


akin to multiplication, which is really cool, because a lot of models and what we knew so far


about neurons, we're always dealing with sort of input output linear relationships. So we could


weigh inputs, and we could add them, but we could not necessarily multiply them, or there wasn't


necessarily a mechanism by which that could happen. These researchers study fruit flies under


different visual stimuli and discover that under the right conditions, they can see a multiplication


like nonlinear behavior in a neuron. If you want to learn more about this, check out the paper,


it's on nature. Lillian Wang publishes emoji search.app, which is a pretty neat search tool


where you can find emojis pickle. Yeah, works nicely. So the code of this is online, it


essentially does a call to the open AI API gets embeddings from the new embedding endpoint,


and then compares those embeddings of whatever you entered to the embeddings of a predefined list of


emojis. Pretty simple application of the embeddings API and works pretty well for the stuff I've


tried. If you want some inspiration, check it out. We've previously reported on our five, which is a


pond on do archive, where you can view any paper as an HTML page instead of a PDF. And we're happy


to report that our five is now an official sub project in the archive labs. So hopefully pretty


soon you'll be able to have a unified experience across archive where you can go to website instead


of a dumb PDF. Elia Sutsgever has tweeted out it may be that today's large neural networks are


slightly conscious and the whole world came crushing down. This to me is essentially kind


of a shower thought. And just you know, it's Twitter, you just tweet it out. It's a musing,


it's an interesting thought. It may lead to an interesting discussion, who knows. However,


the world seemed to freak out about it. And I just can't understand that people legitimately get


mad at this. Like either you're looking to get mad at something, or you're just so far down some


rabbit hole of construing this as bad. I don't know. Now, of course, there were legitimate


responses and people discussing this in seriousness. But then also, the news happened


futurism.com open AI chief says advanced AI may already be conscious interesting engineering.com


open AI top scientists as AI might already be conscious. Researchers respond furiously,


furiously you hear but there were some brave souls coming to help another one from futurism.com.


MIT researchers don't ignore that possibility that AI is becoming conscious Daily Mail,


artificial intelligence expert warns that there may already be a slightly conscious AI out in the


world. Oh, no. And interestingly, you see the phenomenon that happens often with media in that


they just kind of translate that may and could to open AI co founder Ilya Satsuki, who claims


artificial intelligence is conscious is experts called out his claim as being off the mark and


called him full of it. Like the word experts has got to become a meme sometimes in the near future.


I guess soon as you start a sentence with experts say is like who listens nobody listens, especially


if their argument is you're full of it. Oh, ah, the convincingness Oh, no, gee, I've just meditated


three days about the metaphysics of what could and couldn't be consciousness and the inner workings


of deep learning. But now that you're saying I'm full of it. Ah, that does it. Thank you experts.


Again, futurism researchers furious over claim that AI is already conscious. They're mad. Oh, no.


They're mad. Anything but mad, not the mad and the Daily Star conscious AI may already exist as


expert receives backlash over terrifying warning. Well, there you have it. I personally have no


stake in this. I just found it funny how the media reacted right here. If you want my opinion and


this is not me coming up with this by myself. It's helped by a lot of people is that consciousness,


whatever it is, is clearly a physical process that happens somewhere in the brain as a result of


matter interactions. And therefore, it's absolutely possible that there is something like consciousness


happening in a non human brain system. Secondly, I think consciousness is not something that is


binary. It's not like you're either conscious or you're not most things in biology are not that


clear cut. I mean, even the concept of alive is sort of undermined by the existence of viruses.


And I think the two qualifiers here, first the it may be and second the slightly conscious work


very nicely with that. Now, of course, a lot of people are pointing out that we don't actually


have a good definition of what consciousness is, we don't know too much about it, to be able to


make these kinds of statement, which is absolutely true, guaranteed. However, very often, those are


the same people that say, absolutely not our large neural networks conscious. And it's like,


well, which one do you want? In any case, carry on. There's a new paper by Meta AI and in RIA


claiming that vision models are more robust and fair when pre trained on uncurated images without


supervision. So what they've done is they've gone out into the internet, and they just collected


without any filters, a humongous amount of images, no processing, no filtering, and they've just


trained the models on those images. And it turns out on a lot of these metrics about fairness and


robustness, that model performed better than models that were trained on curated data sets,


such as ImageNet. Now, of course, this is interesting, because it cuts directly against


the people that claim often very loudly, that you need to heavily curate your input data, you need


to heavily curate your training process. Otherwise, the models they will become also


bad because they'll pick up all of these stereotypes and all of the biases in the training


data. And while I generally agree with that statement, the evidence here seems to be that


exposing just the model to a wide variety of data and the diversity of data may actually achieve


that in practice. And if you ask me, I'd rather go with the people that actually tried it out and


measured it rather than the people who are simply claiming we should do something and how it would


turn out on a more philosophical level, I think that much like humans, we shouldn't shield our


models from bad material. Instead, I think we should expose our models to all sorts of material,


all sorts of inputs, and then build the types of models that can actually reason across those


inputs and reason why particular things in particular contexts may be appropriate or


inappropriate or warranted. And yeah, that's just my opinion. Leave yours in the comments.


This project is pretty cool. Clip Poso is a system that comes out of a paper, and it uses clip


together with a differentiable renderer for drawings in order to create sketches of pictures


in various levels of abstractions, as you can see right here. Most notably, this process is based


on clip and does not require a sketch data set to be present. And because the sketches are


parameterized as Bezier curves and not pixels, you can do things like change the brush styles


and use all kinds of weird things that you can do in vector graphics, as opposed to the classic


pixel graphics. Check out their project website, it's pretty cool to see what the model outputs.


And yeah, give it a shot. Alright, here's our section of helpful things. First helpful thing


is this book right here, natural language processing with transformers by Louis Tunstall,


Leandro von Vera, and Thomas Wolff. Now what's interesting about this book right here,


other than it might be interesting to a lot of you is that this one comes with a dedication.


It says Hi, Yannick, have you heard of them transformers already, we think they're pretty


cool. Maybe you'll learn a thing or two about them in this book, the book itself goes into


concepts of how transformers work, how they operate on language, but also the direct code


examples of the Hugging Face library. So essentially an all in one tutorial on natural


language processing in 2022. Thank you very much. The book is available on Amazon as an


ebook and also as a well, paper, I think it's called paper. So you know, give it a try. Alright,


some more helpful things. TB parse is a library that parses TensorBoard events. Very cool. If


your stuff outputs these event files that you want to read with TensorBoard, but you actually want


to import them somewhere else. This may be the way to go. Anomalyb is a library for benchmarking,


developing and deploying deep learning anomaly detection algorithms. akb 48 is a database of


articulation objects. These are objects that you can somehow interact with by articulation. So


books, bottles, knives, dispensers, compasses, glue sticks, nail clippers. So this is a database


of properties and 3d models of those things. Evo sax is a library that contains jacks based


evolution strategy algorithms. This library is by Robert Lange. You might know him, he's a writer in


the ML space. And it implements a number of different evolution strategy algorithms in jacks


related to this Google releases Evo jacks. Yes, the previous one was Evo sax. Now it's Evo jacks.


This is hardware accelerated neuro evolution. Now Evo jacks focuses on the acceleration,


the distribution and the efficient methods of rolling out episodes in neuro evolution algorithms,


while Evo sax focuses on implementing the actual strategies. Now what's cool is that Robert has


already made a pull request to Evo jacks, and the two projects are integrated with one another. So


if you're into evolutionary algorithms, give these two projects a try. Textless lib by meta AI


research is a library for textless spoken language processing. That's essentially NLP without an


intermediate text representation, going from sound waves directly to whatever the task output should


be. Standard sim is a synthetic data set for retail environments. So this is a rendered data set of


stores like the inside of stores. It's pretty cool. And it looks like super real, except it's


too clean. Like it's way too clean. Patrick von Plotten tweets that new T five checkpoints are to


be found in the hugging face hub. This is after research by eat I and others who trained a lot of


T five checkpoints in various sizes and analyze their scaling properties. So now there are a


number of T five checkpoints that are potentially much more performant than the original ones,


they are available in large and small, especially one is called T five efficient tiny n 18, or nl


eight, who knows, but it does require less than 100 megabytes of memory, which is very small for


a transformer. Motson is by its own description, a huge data set for pedestrian detection and tracking


in urban scenarios, creating by exploiting the highly photorealistic video game, Grant theft auto


five, say a GTA five in game footage is now used to create high quality data sets. This is how far


we've come as a civilization. Better hope Sasquatch isn't in there. Hash nerve pytorch is a pure


pytorch implementation of the paper on neural graphics primitives. So neural graphics primitives


or instant NGP was a paper by Nvidia that made it possible to render nerves a lot faster. Now that


implementation was only available in c++. So the researchers here have ported it to pytorch. And


that is not as fast, but it allows researchers to play around with it. diff racks is a library of


numerical differential equation solvers in jacks, they're auto differentiable and GPU capable.


Excellent. Fin RL is a deep reinforcement learning for quantitative finance. If you ever wanted to


predict the stock market using deeper enforcement learning, don't do it. It doesn't work. But for


anything else, use Fin RL, the AI Nordics Discord is releasing Swedish models. Specifically, there


is a Bert large Swedish case, which is Bert trained on Swedish. Excellent. They also have a GPT model


in Swedish, but they're only giving it out if they like you because of potential misuse of the model.


Well, I guess whatever floats their boat yet. Mold is a benchmark about long documents and


multitask learning. It's a set of six NLP tasks where the input consists of at least 10,000 words


and has various tasks such as translation, summarization, question answering and more.


Interestingly, there seems to be tasks where you need to create an output that is even longer than


the input text breaching is a framework for attacks against privacy in federated learning.


So federated learning is this idea that users kind of keep their own data and just kind of send you


back gradients for your models. And there are a lot of techniques that claim that this can be done


with privacy sort of guaranteed that I can send around my gradients without the central instance


being able to reconstruct my personal data. So this framework includes a number of what's called


gradient inversion attacks that allow you to do it nonetheless. So it's a little bit like the field of


adversarial examples. If you're interested in this kind of stuff, this might be a cool way to start.


Meta shift is a data set of data sets for evaluating contextual distribution shifts


and training conflicts. So this is a benchmark about distribution shifts. And one thing it does,


for example, it presents objects in various different contexts to analyze how models react


to that. For example, on the bottom here, you see a cat on a keyboard in a sink in a box with a remote


control, you know, just cat things. It's really cool that we go beyond sort of the classic one


image is one object of one class setting and take the next steps in order to deploy these models in


the wider world. Alright, that was already it for helpful things while not already that that was a


lot of helpful things. Our last story for the night, the verge writes the US Copyright Office


says an AI can't copyright its art. Now if you click through, you'll get to an article of urbazm.


And for whatever reason, their background picture has a bot in it. But okay, cool. But this turns


out to be about an old friend of ours. It's Dr. Steven taller, the inventor of a system called


Dabas that makes autonomous inventions. Apparently now it also makes art. Now taller has previously


applied for patents of inventions that his system has made and actually succeeded in some countries


and failed in others. Now apparently he's also trying to patent his art. Sorry, the AI is art,


of course. Now I've looked into his systems and they seem kind of sketchy to the point where I'm


not sure if these are just kind of sort of pixelated versions of things that exist. And


that's just disturbing. I mean, that's just an image of Einstein overlaid on a tunnel. But yeah,


Dr. taller seems to be on a mission to establish that AI can own patents. But he's now been


smashed down by the Copyright Office that says that in order to grant a patent, there must be a


human intervention. So their definition of patentable creativity includes essentially


the interaction of the human intellect or the human brain with the world. Now that is the law


currently, but who knows how this goes on in the future? It's a difficult question, because for the


first time, probably, it is probably legit to ask who owns the copyright of AI produced stuff,


and whether or not this counts as an invention, and then who made the invention. And if AI is


capable of being an inventor, what kind of implication does this have down the line? It's


a set of interesting questions, but I don't have the answer to those. Let me know what you think.


As always, this was it for ML news was wonderful to have you here. Please check out Weights and


Curiosity-driven Exploration by Self-supervised Prediction

Hi there. Today we're going to look at this paper Curiosity Driven Exploration by Self-Supervised


Prediction. It's a relatively short idea so it shouldn't take too long. So the fundamental


idea of the paper is to tackle the reward sparseness problem in reinforcement learning.


For example, if you have a Super Mario game like here and there's a number of ways you


can think of the reward but one way you could formulate it is that you simply get kind of


a plus one reward when you finish the game or the level. Let's say you finish the level,


you get plus one. If you die or don't make it in time, you get negative one. I think


there's no way to not make it in. Oh yeah, there's actually a time limit. So the


problem here is that your algorithm kind of needs to learn to make things now such that


it gets to the end of the level but the reward is only at the end of the level. So basically


step by step it has no signal to go on because the reward is always zero and it kind of needs


to learn these long-range dependencies and that's notoriously hard in reinforcement learning


to step by step learn actions that kind of maximize some very long-term goal. So you


can also think of a game of chess where your reward is going to be whether you win or lose


at the end but step by step it's kind of this reward is 50ish steps away. So you have no


way of step by step optimizing your actions in a meaningful manner. So there are many


ways to get around this. One way that people have done is what's called reward shaping.


And reward shaping is you're trying to introduce additional rewards kind of as a designer of


the algorithm that you know are kind of good or helping to solve the problem or at least


correlate it with the reward you're going to get at the end. So in Mario this could


be like the further right you go the more reward you get. You get kind of an additional


reward if you go right. Coincidentally I think in real Mario this also gives you points but


our situation is that the reward is just going to be at the end. You could also say like


if you kill or if you stomp the Goomba, one Goomba you stomp that actually gives you also


a bit of reward. In chess you could say like the more pieces you have that gives you a


bit of reward if you have more pieces than your opponent. If your opponent loses pieces


you don't and you also get a bit of reward if you get more territory on the board and


so on. So these are all things that we know kind of correlate with the end reward. Like


because in Mario for example the end of the level is actually on the right. But of course


it's not perfect because sometimes there are situations where you kind of have to go back


and go around something or go over something and not immediately go to the right. As well


as in chess there are good sacrifices that you can make. So these kind of additional


rewards they help but they're not perfect and the biggest problem with them is they're


very domain specific. So a developer of the algorithm you basically have to know the domain


like Super Mario and you have to know the goal is on the right so you have to construct


your reward in order to kind of reflect this. And this is very domain specific. Basically


you have to do it for every domain again and again and again. In chess you have to know


something about chess to play and so on. So one way around this and this paper proposes


one method to do this is to introduce an additional reward not based on the domain specifically


but based on what they call this curiosity. And it's specifically curiosity by self-supervised


prediction. So what does that mean? The idea is not new in that people have kind of done


this before. If we go for example here. So here is this kind of doom environment and


what you could say is in my agent I have kind of a little module that's going to predict


the future. So like if I'm if I'm here then I will basically choose an action my agent


will choose an action like move forward like press the forward key and then I will predict


how that's going to look. And of course we know this is kind of a 3D environment so this


probably going to be this part of the screen is going to be the full screen because you're


now closer and so on the perspective changes a little bit. Basically this is this should


be a learned like a neural network that predicts the future from the state now and the action


now. And basically the you can train this in a supervised fashion because you will perform


some actions you will collect some data about this so you can learn a network that is going


to predict one step into the future basically how the environment will look. And then and


this is by no means kind of a new idea to introduce rewards based on this type of learning


how the environment acts. We've seen this in like the A3C paper the original one where


the additional reward is something like pixel control where they consider like okay this


pixel here how much can I control it by my action like how does my action influence it


can I predict this and so on and to to learn how to control the pixels on the screen by


your actions and to give a reward based on that so that's that's been around this idea


and what this paper here does specifically is they say well I'm going to predict the


future and if I am wrong about the prediction then that gives me a reward and that's that's


the curiosity part basically means like if I have a good model of what's going to happen


in the future and then I predict the future and then I'm wrong it means something new


has happened something special something that I hadn't expected and therefore if the goal


is to get the algorithm to explore by itself which is what you need to do when you don't


have a reward right when you don't have a reward what you want your algorithm to do


is simply to go around and explore and in a sense they're saying okay the way to do


is is to go by curiosity which means is to go to actively seek out environments that


you haven't that you wouldn't expect basically so whenever you don't expect something that


means it's something new that means you haven't had this experience before right and that


means that it's kind of a new state to explore that you haven't you've not seen this before


so kind of in absence of any reward might as well go where you haven't been before and


that's that's kind of the the essence so they outline a number of problems that you might


have with this approach they they give the example let's first actually go to what the


the model actually looks like so that's here you can see this is kind of what they call


an intrinsic curiosity module so you have a state here you're in a state you have your


policy and your policy gives you an action and the action goes to the environment and


the environment gives you the next state and also the what's called the the reward they


call here our e is the extrinsic reward that you get from the environment but they also


combine this within so what's called an intrinsic reward that you get from here that you get


from the curiosity module and that's what we've discussed it kind of tries to assess


how new is the state that I'm gonna be in how surprising it is for me so the the thing


is that I'm gonna I'm gonna first describe like the model how you would build it and


how that gives gets you into problems and then how to fix it so how you would build


this is to have this what's called this forward model so the forward model takes the action


and the current state and it kind of predicts the next state that's in here don't worry


about the Phi hat right now it predicts the next state and then you compare this to the


what's here to the actual next state you here is subtract you just look at the difference


between what you predict the next is gonna be and what the next state really is and that


gives you the intrinsic reward the more different these are the kind of higher the reward that's


what we've discussed like how how much different is it from what I've expected so how does


that get into problems and the authors give a very very good illustrative example of say


you are in an environment let's actually go over here you're in an environment and you


have your screen right and here is kind of a road that you need to maybe walk after and


here are some leaves in the wind I'm very bad at drawing leaves so imagine these are


leaves and there's wind right like winds coming from here and kind of shaking up these leaves


and so on so if you simply try to predict this entire screen from as your forward model


what's gonna happen is you will never be able to predict how these leaves are gonna move


because they're basically you can't influence them no no part of like you can predict a


bit from the current state but the action you take has no influence on how these leaves


are going to move because they're influenced by the wind and the wind is kind of this random


ish process that that you you can't control so the the authors say because of this to


your algorithms always going to find these leaves basically interesting curious be curious


about it because it can't predict them and we've seen that the reward that they model


to give an addition is based on how well you cannot predict a certain state and they say


okay if we do like this then the the kind of these things these random things that we


can't influence will always be surprising and therefore we will always be curious about


them and therefore we will always kind of look at the leaves and be amazed and get reward


after reward because we can't predict them that's not the goal so what what they're arguing


is that why are these leaves not important for curiosity because we can't influence them


with our actions like we can influence where we go on this road because we can kind of


move and the road is kind of static not governed by these random processes but the leaves we


we we would like to discard them we can't influence them and therefore what they say


is what we need is an encoder that takes a state and try to delete this annotation so


we need an encoder here features that takes a state and it outputs features of the state


and then we're not our forward model isn't fed with the state it's fed with the features


of the state and is not going to output the next state as such but the features of the


next state like the predict the features and then we're going to compare that with the


features of the true next state and that's what we compare so how does how does this


encoder these features need to look and they're saying well these features should kind of


only consider things about the state that are actually dependent on our actions and


they have a I think a very interesting way of achieving to train such an such an encoder


such a feature producing function in that they say it's going to be a neural network


that we train by this training this so-called inverse model so we take this encoder and


we train this inverse model on top of it and the inverse model takes the features of the


last state and the new state and is trying to predict this action this action right here


so this this is this action the action we took to get from the old state to the new


state so this inverse model is trained to predict what action was taken to get from


the old state to the new state and by training the encoder with this inverse model like training


this end to end you will make the encoders such that it only considers things that are


actually relevant to predicting this action so in the leaves example it would discard


the leaves it will discard anything that you can't influence with your action and therefore


it will only retain features that are dependent on your action I think that's quite interesting


way to get kind of rid of the irrelevant information that they don't want and then they can use


this encoder to train this forward model and to essentially get this intrinsic reward so


I find this idea quite interesting and as I said that the idea of intrinsic reward and


curiosity to go for exploration is not new but I think this this this kind of approach


and I'm sure it's been around in some variants but I've just stumbled across this and this


is oh sorry this is quite interesting cool so we're gonna take a look and you can go


by the math yourself but they they do these kind of experiments and they corrupt as you


can see part of the screen with noise here and they of course show like okay since the


noise is not dependent on our action our features do actually discard this noise only focus


on the part that we can actually influence our actions so that's I think all in all pretty


interesting they show of course that their algorithm then outperforms the kind of baseline


of a3c on these sparse reward tasks and the sparser here you can see like the left is


like dense reward and then sparse reward and then very sparse reward and at some point


you see the a3c simply doesn't doesn't do it anymore but what's also interesting is


here you have the ICM in pixels which which kind of means pixel based curiosity so where


we don't have this encoder where we simply trying to predict the pixels of the environment


and that works if you have like this kind of sparse reward thing but if you want to


if you have the very sparse reward that also fails and you actually need this this encoder


that discards what's not relevant for predicting the actions yeah so you can you can take a


look at the rest of the paper yourself I find I find it quite interesting they analyze how


their their agent explore these mazes and things and they have more experiments on like


Neural Architecture Search without Training (Paper Explained)

Hi there. Today we're looking at neural architecture search without training by Joseph Meller,


Jack Turner, Amos Storky and Elliot J. Crowley. On a high level, this paper performs neural


architecture search by looking at the correlation matrices of the Jacobian of the data when


you pass it through the network. And it does so at initialization. So you pass the data,


you look at the Jacobian, and if it's very correlated, then the network is bad. And if


it's very uncorrelated, then the network is good. And by simply observing that they can


already achieve a very good score on neural architecture search benchmark. Alright, that


was the high level and maybe a bit too simplified. But that's sort of what's going on. Okay,


let's dive in. So what's neural architecture search neural architecture search is the


discipline of you are given a data set, let's say here, we have a data set, which could


be something like CIFAR 10, which is an image data set. And you are given a sort of a training


procedure, let's say Adam or SGD for 100,000 steps or something like this with many batches


of size 64. Okay, and you're given a loss function, which the loss function here could


be the cross entropy between the outputs of the network, which we'll call L, and the label


Y. And your task is now to find a neural network architecture that conforms to these specifications,


but gives the lowest possible loss or the sorry, the highest possible validation accuracy


in this case. So this here would be like the train and then you'd have the test accuracy


or the validation accuracy. Okay, so you could decide, well, I'm going to go with, you know,


first, like three convolutional layers, each one having like a real non linearity. But


you could also say, well, I'm going to build like a skip connection from here to here.


You could also say that I'm going to downsample by two, you could have maybe a bigger stride


and so on. So the kernel size of the convolution, you can vary. Until now, people have done


this by hand, right? In effect, we all use like the same 10 to 20 different architectures.


So if it's an image problem, we tend to go for like a resnet or a wide resnet, like a


VGG style architecture. Someone has come up with those at some point with each of those


discovered that it works well. And we don't really do much exploration, we simply kind


of use the same things over and over. And the the truth is that there might be much


better architectures that we're simply not exploring, right, there might be much better


building plans for networks that we don't know of that might perform a lot better with


the same data and the same training. So neural architecture searches the process of automatically


searching for these better architectures. Of course, that's a combinatorial problem.


But the idea is that, you know, you can actually learn to construct good architectures. And


by doing so, you can, you can sort of speed up this process that is manual otherwise.


And the idea behind it is there some regularity of when an architecture is good, there's some


like high level pattern that you as a human maybe cannot really grasp, but like a machine


can figure out which architectures are good and which ones aren't. So there have been


a few inventions in this in this area, but they are mostly costly. And that's what they


say here. The time and effort involved in hand designing deep neural networks is immense.


This has prompted the development of neural architecture search techniques to automate


this design. However, neural architecture search algorithms tend to be extremely slow


and expensive. They need to train vast numbers of candidate networks to inform the search


process. So what neural architecture search methods do is what they'll have is they'll


have something like a controller, the controller itself, of course, is going to be a neural


network. So there will be this thing that will be the controller, and the controller


will emit like a building plan. So the controller will emit like a building plan for this network


right here. And then you train the entire thing once through for the entire 100,000


steps. And then you observe the final validation accuracy, which might be something like 80%.


And then you know, okay, this is 80%. So you feed the 80% into your controller and the


controller outputs the next building plan that it thinks will score higher. And then


you train the entire thing again, and you maybe observe our 70% accuracy, you again,


feed that in, right, and the controller realizes, oh, I may have done something wrong, let me


try something else. And does again, if this looks like reinforcement learning to you,


that's because this is reinforcement learning. So the re in the DC here, the controller would


be the agent, the percentages here, the accuracies would be the reward. And the environment, the


observations would be basically, this thing here, this thing would be the actions, but


sometimes it's the observations and you need to score the different things. Okay. So the


problem, of course, with this is that the reinforcement learning requires a lot of data,


it requires a lot of steps to converge, because the signal from the reward is just so weak,


you simply get one number for your action. And you don't know what you can change to


make it better, you simply have to try. So you need a lot of steps. But this thing here


is mighty slow, because each each single step in your reinforcement learning procedure involves


training an entire neural network for like this many steps. Okay, so all of this is ginormous


Lee slow, and resource intensive. And that, of course, blocks a lot of research, because,


you know, we started with the plan to automate this part right here, but automating it itself


is super expensive. So they go for a different solution. They say this could be remedied,


if we could infer at net, sorry, if we could infer a network's trained accuracy from its


initial state, okay, it seems a bit out there. But let's, let's give them benefit of the


doubt. In this work, we examine how the linear maps induced by data points correlate for


untrained network architectures in the NAS bench 201 search space, and motivate how this


can be used to give a measure of modeling flexibility, which is highly indicative of


a network's trained performance. We incorporate this measure into a simple algorithm that


allows us to search for powerful networks without any training in a matter of seconds


on a single GPU, okay, and they have the code available right here, if you want to go and


check that out. So let's go in, let's go into that. The claims are pretty big. And the reasoning


behind the claims is the following observation, you can already sort of see in this graphic


right here, we'll, we'll go over what it means in one second. But what they do is they take


different networks in this search space. And the search space in this case is given by


this benchmark. So this benchmark basically has a long list, I think, of architectures


that you could consider. Actually, so it's a, it's a constructive list. So they don't


actually give you the list, but they give you like a, a way to construct architectures.


And they took those architectures and they rank them by how well they score on the C


for 10. So there are very good architectures, which are here, there are good ones, there


are mediocre ones, and then the bad ones. Okay. And you can see that the histograms


here of whatever they measure, they look quite different. So the histograms with the good


ones, they all have kind of spiky around zero, and the histograms of the bad ones all sort


of look spread out. So this is the measure that they're going to propose is they have


some sort of number, some sort of histogram that they produce. And if the histogram is


very spiky and close together around zero, then they conclude that this network is good.


And if the histogram is very spread out like this, they conclude that the network is bad.


Now these histograms, as you might expect, they are computed not from the final trained


network, but they are computed from the initial network. So here they show, at least, you


know, in this case, it seems to be that there is a general correlation between the trained


accuracy and how this histogram looks. And we're going to explore what they do. So it's


essentially it's pretty easy. They compute the linear map around each data point. So


what is that? If you imagine a neural network as a nonlinear function, which I guess you


should because it is. And so let's imagine it as like a nonlinear function from x to


y. What they'll do is simply, they'll look at a given date training data point, which


could be here, right, this could be the x, and this could be the the y. And, in fact,


let's look at it in loss landscape, not even in y, but in L in terms of the loss, because


we don't need necessarily a single label, this could be for unsupervised, this could


be for anything, okay, so it maps a data point to a loss. Now, what we'll do is we'll simply


linearize the function around that point, which means we'll just freeze all the non


linearities in place. And that will give us this linear function right here. Okay, we


just observed that this linear function can exist, it's the tangent to the loss landscape.


And it's at a particular data point, right, it's in data space, not in weight space. Then


we look at a different data point. So we look at this data point right here, another data


point, what's the linear function around this one is sort of like whoops, D is like that.


And then around this one is like this. Okay, so this is one function. Now let's look at


a different function right here. So L, x, and we'll look at this function, the linear


function, okay, so for some reason, this is like this. And if we consider two data points,


their linearization is very similar. Now imagine that these two have been produced by the same


sort of neural networks, it's just the architecture is a little different. But they have been


produced like they have the same number of parameters in the neural network, which neural


network would you prefer? Remember, you can in by training the neural network, you can


actually shape this loss function, you can kind of shape that around. So which one would


you prefer? I personally would prefer the top one. Because the top one already tells


me that, hey, you know, I might have 10 parameters here. And this already sort of looks like


each of the 10 parameters is doing something. So if I then go into my 10 parameters, and


I, you know, turn this knob right here, then I might, you know, up this bump or down this


bump or do something with it. But the sort of frequency curvature, the randomness of


the function, the way that it fluctuates tells me that all of the different parameters must


have some sort of effect, right? Because it's quite an expressive function. Whereas if I


have the same number of parameters for a function like this, this sort of tells me, well, maybe


only one of the when the only one of the weights is actually doing something, maybe only one


of the dimensions is doing something, this seems odd, right? That even though I've initialized


it randomly, a super regular function like this comes out. So maybe all of the all of


these parameters down here, they don't do anything. Or the so somehow the signal doesn't


get through. So that's, I, they don't explicitly say it in these terms. But this is how I make


sense of this. What they're saying is that if you look at the linearizations of the functions,


and you look at the, the angle right here, so the angle in this case is that and in this


case is that and in this case is that so you look at the slope here. And the slope is basically


the gradient of these linearized functions. And what you want to do is you want to look


at the correlation between those of the different data points. So here we have three angles,


one is very short, one is very bit longer, like this, and or no, even like this. And


one is even over 90 degrees like that. They are not correlated at all, right? They're


all very different. However, the angles here, they're all quite the same, as you can see.


So what they propose is the following. Let's send all the data points or in that case,


all the data points in a particular mini batch, let's send them through the function. And


let's calculate their linearizations. So the linearization is nothing else than you send


them through the network to obtain the f value for the x value. And then you calculate the


gradient with respect to the input, right? Now you have to get used to this a bit, because


usually we calculate the gradient with respect to the weight. But now we calculate the gradient


with respect to the input, which if this is a linear function, so if you have a lit f


of x equals w x, like a linear function, then this gradient del f del x would just give


you the W will give you the slope of the linear function and the same in the neural network


when you linearize it. Alright, so we're going to obtain all these linearizations. And that


gives us the this matrix J right here. And what we can do is we can then observe the


covariance matrix of J of all these linearizations. The covariance matrix simply tells you how


two data points vary with each other. And in fact, they don't look at the covariance


matrix, but they look at the correlation matrix, which is simply the scaled covariance matrix.


So one entry in this covariance matrix, so you have n data points, and this gives you


a matrix that's n by n. And that particular entry here, like the entry ij would simply


state how does the angle of data point i correlate with the angle of data point j? Okay, that's


the that's the covariance matrix. And now the hypothesis is, if all of these data points


are sort of independent, like in our very expressive function here, then the these correlations,


they should not be high. In fact, most data points should be rather uncorrelated. However,


in this case, right here, if the function is sort of kind of degenerative or something,


not very expressive, then all of these all of these angles or of these linearizations


should be highly correlated. And that's what you see in this graph right here. This right


here now is these correlation histogram of the correlations between local linear maps


across all pairs of items in a mini batch of c410 training data. Each part is a histogram


for a single untrained NAS bench 201 architecture. So remember, the expressivity is important


because we want to train that function and therefore it's important that every parameter


does something. And if it's degenerate, we can't train it well. And that's, I find that's


the reasoning they sort of say this, but I'm not, I might make I might make the wrong sense


out of it here. But it seems to me like that's what's actually going on. So you can see this


is simply these matrix values rolled out and then plotted as a histogram. So what does


it mean when an histogram is like super spread out like this? It means that there are a lot


and I think down here are axes. Yes, there are a lot of data points that correlate highly


or anti correlate highly with each other. Okay, which means that exactly this degeneracy


happens. So either too high or too negative high correlation means that they're very much,


they're kind of the same thing. So there is, if you have as many parameters as data points,


that means that one parameter can potentially serve these two data points or these two that


are correlated by one or negative one, you don't need both parameters and therefore you


have a lot of parameters doing nothing. Whereas over here, with the good networks, you can


see that this spikes around zero, meaning that the data points are not correlated, or


the linearizations around the data points are not correlated. And therefore, you can


sort of shape the function around each data point, however you want, which we sort of


know that neural networks, what they do is they're so over expressive, that they're actually


able to shape the functions around the data points without necessarily looking at other


data points nearby. And that expressivity is what what you want. And that expressivity


is what this in part measures. Okay, so they make a, they have some experiments here where


they validate this. So for all these architectures in this benchmark, and maybe I should tell