Yeah, so we did add like two different themes of tasks.


Author Interview - Memory-assisted prompt editing to improve GPT-3 after deployment

One is ethical reasoning.


And the other is more word reasoning.


In ethical reasoning, this is a recent topic on ethical AI, which is as an example, if


I have turned on the blender at 3am, I ask the system, is this ethically correct to do


or not?


And the system will probably should probably say that it is not okay to turn on your blender


at 3am because it might disturb your neighbors.


That's one theme, which is ethical, ethical AI.


And we have two different tasks within that.


In one case, the input is, you know, a string, like I said, turn on the blender at 3am like


a situation.


And the output is whether it is good, bad or not.


And like with some clarification, or some understanding, sorry, not clarification, just


understanding of the model.


Why it believes this is the case.


And we have two different types of understanding in it that makes up the two, you know, two


different tasks.


One is it clarify it, the model presents its understanding based on an explanation of the


sort that it's not good to wake up your neighbors or disturb your neighbors in the night.


That's one.


And the other setup we have, which makes up a different task is, you know, it says, this


is about care or harm.


This is about, you know, the topic, what this situation is intended to bring out.


So that's one task, one theme of task.


The other one is more word reasoning tasks.


So we add on to the synthetic lexical relation tasks that we had in this, in the V1 paper.


And we add on to word scrambling and other tasks, which are involving, you know, anagrams


and how to fill up, how to correct a word misspelled and so on.


So those are like two different themes of tasks we have.


Aman, do you want to say something on the second task?


So I think we also added one other task, which is factual question answering.


So suppose that user wants to ask factual questions like who is or where was a certain


person born or where did they go to school?


So things like that.


So in those cases, there is no understanding that the model can display of the instruction


other than the answer itself.


So for example, if you ask where did Albert Einstein go to school, if the model says Stanford,


then you can correct the model and say no, it goes EPS, Eurek or something.


And then you can store these corrections in the memory again.


And then when you create the prompt, you would bring in some examples which are similar to


the question on this the model has been wrong before to make the prompt.


So for example, if the question comes in where did Vincent Church go to school, then you


would already have an example in memory of the Albert Einstein example.


And that we show is helping the model in getting the correct response.


So it would have been that really effective.


Have you so?


Um, yeah, so this this, it's pretty cool.


And I've had a flick through this paper that it the tasks seem to be much more extensive.


Now, that's not it.


It's a so you had the ethical one, you give a few examples right here.


On the right, we can see, for example, the understanding this question is about loving


your partner this question about seeking medical attention, if you feel there's something wrong,


which is a lot, I think, you know, the, the gap to what we what people usually call common


sense gets smaller and smaller.


Have you let any users any actual users use this system with GPT three, so you came up


with your own data set as if I understand correctly, your own sort of feedbacks, sometimes


heuristics and so on.


Did you ever just, you know, set this in front of someone and say, you know, here you go,


try it out?


No, we have not.


That's one of the things we would like to do.


So we have not done that yet.


And in fact, in just to clarify, the the data sets that we have here are the feedbacks on


ethical reasoning, for example, is not something that we came up with this was present in the


data itself.


So this was a data which was crowdsourced through mechanical Turk and there were actual


users who are actual mechanical Turkers who gave this feedback.


But on the other hand, we have not tried this on any real users.


This is the closest we came to reality in some sense.


And we would like to do this in the future.


Yeah, it'd be super cool to see how real people interact with this.


Sorry, Aman.


Yeah, so I think so like Nikit said that for all these data sets, the data set is real.


So you're right in the first version, we had one of the data sets that we collected ourselves.


But in this case, the feedback is given by humans.


So in some sense, we are approximating that process by a linear data collection process


as opposed to a bunch of Turkers working on it at the same time.


But yes, it would be great to kind of see, you know, once deployed, if this actually


does better on one of these tasks or one of the new tasks that we discussed.


I'm going to guess that specifically for GPT-3, the restriction of open AI on what you can


build with it and the approval process would prevent you from actually releasing this,


say to the public as a service.


But one could think of maybe using another model or just, I mean, your code is online,


so people could use it with their own API key, if they really wanted to.


Yeah, that is correct.


And in fact, just outside of this paper, also, we had been working on T5 model with a very


similar architecture, T511b.


And so that's one of the models we could, you know, release in the future.


Is there a difference between smaller models and larger models in how much this type of


feedback is needed?


Like you specifically work with GPT-3 and you know, I get it, that's the model that


we cannot train.


But is it also more necessary to provide feedback?


Can you tell us a little bit about the differences between small and large models or different


models?


Let me just start with that.


So it's a really good question, first of all.


So our general experience with injecting, you know, some knowledge, external knowledge,


like you know, common sense knowledge into models has been as the model capacity keeps


increasing, it requires comparatively less knowledge injection.


So smaller models like, you know, let's say, Bard-Base would require, would benefit a lot


by, we have seen this in experiments in the past on, and others have also reported it.


If you inject external common sense knowledge, then those models get much bigger boost than,


for example, T511b.


Bigger models get less boost.


So we have tried the same, very similar architecture, actually almost the same architecture, there


is a paper under review on T511b and what we also observed there is that there is substantial


gains with T511b.


The only difference in mechanism is that, you know, there we were able to fine tune,


have a fine tuned T5 model, which understands the task a lot better than in GPT-3 where


there was not even an opportunity to do that.


So probably because of that reason, we are seeing bigger boost in GPT-3 than we did with


But in both the cases, there is substantial boost in performance by doing so.


Cool.


And have you tried, so what you are doing right here, it goes very much into the direction


of correcting the model if it, let's say, makes a mistake, or if misunderstand something.


I had the sort of the opinion that personalization very much in the sense of how you Aman said


this before, you know, I want my IDE to do something in a particular way, would benefit


hugely from that.


Is this something on your mind too?


Are you looking into various like personalization aspects of these models?


Or is this something that is for some reason not possible?


Yeah, I think that's a very good point.


In fact, in the first version, in this version, we have some experiments in the appendix,


also in the earlier version, where we simulate users who sort of interact with the model


in Hindi or Punjabi.


And that's some sort of personalization, it's kind of a language personalization.


So there's a person who's speaking in a dialect of Hindi or Punjabi, and there's a certain


phrase they use repeatedly.


And if you can store that in memory, then sure, the first time the model is not mitigated,


but the next time someone comes and uses the same word, you know, hopefully it will be


patched.


So we did kind of create some experiments on that angle.


And we also have examples in the ethical AI setting where the model was able to correct


or kind of work with slang usage.


When people were saying the same thing in slang, right?


So one person comes and they give feedback.


I think it's a very promising direction for personalization.


And I anticipate that in the near future systems that do it successfully, we'll use a similar


architecture where they have this memory that kind of has a track.


If we get into the paper a little bit, like into a bit more sort of the technical aspects


here, I want to jump over to the experiment section.


And you had an interesting plot where you show not this one, not this one.


This one is one of them.


And interesting, no, this is the out of vocabulary, I think the main ones are, I missed them.


Oh, here, I've drawn so much over them that it's a mess.


Specifically, I was wondering this PFB of 0.5.


Did I interpret this correctly, that this means that you only get the feedback half


of the time?


Does that mean the user can only give feedback half of the time, or the model only receives


sort of this feedback or the model only gets to go through this feedback loop half of the


time?


The user gives feedback half of the time.


Okay, because then the memory grows slowly.


Then it makes total sense that they end up sort of converging to the same place because


I was wondering, you know, if your procedure was only active half the time, it should fail


half the time.


But if the user is able to give feedback half the time, it would still learn slowly, but


it would still learn over time.


Okay, that's...


We wanted to simulate reluctant users who might not always give feedback.


So sometimes you want to give feedback, sometimes not.


Yeah.


Have you thought about pairing this with recommender systems?


Because in recommender system, sort of a recommender system would group me together with other


users who have like similar preferences as I do.


So you know, conceivably, I could say, well, maybe I'm able to sort of profit off of feedback


of those users, right?


If I give some feedback and I'm very similar to these users, it might be the same.


Is this something that could be done or?


I think this is a really neat idea.


We did not think about it, but now that I think about it, when you mentioned it, I think


it is a...


It makes total sense to have a community of similar users all having similar preferences.


It makes total sense.


And I think it would be very cool to try this in the future.


Well, maybe.


Or you always know who the feedback comes from.


Like, ah, your dumb friend entered.


I'm thinking of these people who enter, who like altogether enter dumb things into Google


so that Google autocomplete suggests the dumb thing.


You know, that brings to a very good point about sabotaging our system.


It is possible.


I mean, if you keep giving it really bad feedback, eventually it is going to apply bad feedback


to newer examples.


And this is a valid point, a valid concern.


We also don't know if our memory can be consistent over time or it can start deteriorating and


becoming inconsistent among itself.


I could just give different examples with different feedbacks.


So there is not our work, but there has been other work on how to maintain consistency


in a memory over time.


But that's an additional direction of research which we can employ within our system to keep


it healthy and consistent.


Are there you another in another point in the paper, you mentioned these different pieces


of the puzzle in this framework you propose.


You've added more tasks.


Have you also thought about amending or augmenting some of these things to be more, let's say,


more complicated, maybe replace some stuff with learn things.


So far, you have to look up which is a language model or an embedding model.


Yet the other pieces of the puzzle here are fairly simple so far in your experiments.


Are there any obvious next steps where to make this more powerful in any of these four


parts?


Yeah, so that is true.


In fact, the current implementation is for the combiner is as simple as you know, it's


just a threshold is just thresholding over the inner product.


You know, it's that simple.


But eventually we are in the process.


So this is very much work in progress where we are trying to, you know, beef up the other


components also.


Right now, our only focus was on lookup and memory and the other components are very simple.


But eventually this is where we are getting at.


You know, work in progress.


And I think there are lots of lots of details where, you know, our current system is very


primitive in the sense that it only assumes that the users are, you know, really nice


and that they don't give you bad feedback.


That's one.


It also assumes that the users can, you know, you can effectively retrieve from the past.


And that's not always the case.


You know, we there are cases where we are not able to do that.


That's why we had to set, you know, a higher threshold where we only get good matches and


like good feedback, which are very similar.


But you know, something which we would like to do in lookup, I'm just giving an example,


just like suppose your input is turn on the blender at 3 a.m. and now a new input comes


in which is saying playing drums late night.


You know, both of them are in the analogy space of errors.


They're actually very similar.


But that's not something which our current system can match.


It can at most say, oh, well, if if I find something like turn on the mixer at 2 a.m.


that's similar to something I found and it will pick that feedback, you know.


So this kind of really recursive reminding to a model based on similar error space is


the next step where we are getting to with this lookup.


I think also in the space of the combiner and the prompter specifically, there is probably


a lot of potential to still be gained.


I mean, instead of concatenating, you could you could imagine any, you know, many smart


ways of combining what you retrieve from the memory with what you already have.


Potentially, you could even ask the model itself to come up with sort of like a better


prompt or to sort of you can maybe abuse the model again to suggest better things to you.


I mean, I think that the possibilities are quite quite open here to make this very, very


cool, very powerful.


Another thing that I wasn't sure about is your baseline, this grow prompt baseline right


here.


And I think I tried to explain this a little bit.


Do I understand correctly that the grow prompt baseline, you take whatever the contents of


your memory are and you just append them to the prompt before the question?


Okay.


Yeah, my concern was a little bit that it's not exactly right that the baseline because


that the prompt is structured differently.


But I don't know how important that ultimately will be.


Probably not.


So I think we do structure the prompt in the same fashion.


So we get examples and the structure of the prompt does not change.


It's just like a longer prompt.


So in the video you show an example prompt, which is in the appendix.


It's the same format.


It's just much longer.


It's basically as much as we can fit.


Yeah.


So wait, we can I mean, we can look at we can look at one here.


So there this is the entire prompt, which I found pretty cool that not only do you prime


the model to sort of give you the answers and give you the understanding, which is,


you know, that's, I think that's pretty cool idea in itself to get side information with


your main information out of these models that you can then use to query them again,


I think the applications for this are much larger than just this one.


You also train the model to specifically view or regard or pay attention to the clarifications.


My question was that, let's let's know this is a bit fat.


When in your main method, when you retrieve a clarification, do I see this correctly that


you append it at the end right here to the to the question currently, and this, this


grow sort of this baseline would append something like here in between?


Or do I see this incorrectly?


Right, so in the grow prompt, what we do is, we essentially add more examples to the prompt.


So instead of retrieving something from the memory, it's added to the prompt itself.


Okay.


So that's cool.


Yeah.


Then I've understood correctly.


Sorry, mechanism is kind of very similar to our own method sort of like, you know, retrieve


the right feedback in some sense.


The only thing is we now we are allowing GPT-3 to attend over those to attend over it rather


than you know, be providing a retrieval function from the memory.


We hope that GPT-3 will be able to attend over it itself.


Yes.


I mean, yeah.


And if it fits into the prompt, it's pretty, pretty certain that at least it might pick


up on it, right.


And you make good points here, you say that these this grow prompt, it is quite a bit


larger, and it cannot scale up.


So as soon as things fall out of your memory without a good retrieval function, you're


essentially limited to a very short time horizon.


There is this experiment here, this plot right here, which I haven't touched at all, which


it goes a little bit into out of vocabulary domain a little bit into the domain of different


languages, maybe lower resource languages.


Do you want to comment a little bit on what you what you did there and what your findings


were?


Yeah, so the idea is essentially very similar to what I was talking about earlier.


So the prompt itself has examples from Hindi, for example, and then the questions also come


in Hindi.


And, you know, for the first time, when the question comes, GPT-3 would not know because


it's primarily English.


Funny thing is for Hindi actually, sometimes it gets it, or apparently there's lots of,


you know, English, English or plus online.


But for Punjabi, it struggles.


So the idea is that user comes in, enters something, the model doesn't get it, it goes


in the memory, next time something comes as a similar question, so the model retrieves


understanding from the memory, and hopefully is able to do the task.


So to clarify that the questions are in Punjabi, for example, that you would like to have answered,


and you also construct a prompt in Punjabi, or is the prompt still in English?


The prompt is transcribed in English, but the question parts are all in Punjabi.


So the script is not the Punjabi script, it's still English, but parts of it are in Punjabi.


So we have an example in the appendix.


Yeah.


Oh, yeah, that's a good point.


We should go.


It's a yeah.


No, this?


So I think one of those.


This is the end right here.


I think this one might be.


So those are in Hindi.


And the one in the bottom is in Punjabi.


So the person is, you know, trying to, the scenario I had in mind was someone's trying


to learn English and they're trying to look up words.


So in the first case, they are saying, what is the opposite of edit?


So they say, they ask it in Punjabi.


So they know that they want meaning of this word edit, and the rest of it they ask in


Punjabi.


And the model says something that the opposite of this is something else.


And then the person can say, no, I want synonyms.


And there's like one missing piece here.


This is that you tell the user the end and the means opposite in Punjabi so they know


what the model is.


You know, it's trying to say.


But okay, so you could interact with this thing sort of across languages, and you could


prime it to say, which parts do I want in which language because it would obviously


not know I guess, what what you want the answer in?


Yeah, yeah, you can definitely add language tags.


I mean, it's a pretty cool example of exactly of personalization, right?


Because you can imagine you personalize this exactly to sort of how you want to interact


with it.


And someone else who might be more or less skilled at English or in reverse in Punjabi


might do a different thing.


That's pretty cool.


Yeah, there's one point I wanted to touch here, which you kind of mentioned earlier


with respect to the prompt.


So as you notice in our prompt, the model does not only give out the answer, it also


gives out its understanding of the question.


And I think that's a very crucial piece in this design, because one of the bottlenecks


for us earlier was the system, a system that assumes that the user knows the real answer


is not really practical because if the user knew the answer, why would they be playing


with the model, right outside of a notation setting?


So this kind of breaks that barrier.


So you might not know what the answer is, but you know for sure what you asked for.


So you can always tell the model, no, this is not what I don't know if you're right,


but I know for sure this is not what I want.


And that kind of helps in improving the performance.


The performance of the model itself might be whatever it is, but we are helping the


model in understanding that intent more precisely.


That's the, I guess, the main trick here.


Yeah, I like this, this getting the answer with the understanding.


I think that's, that's pretty powerful.


Not only yet to interact with the model, but also just to understand what it does, instead


of just getting a simple answer.


Could be a good recipe for other applications as well.


Did you have to fiddle around a lot with sort of the prompt structure or the structure of


what to add right now you have, you have a bar and then clarification and then colon.


Is this the first try and it worked or is this the result of many hours of sweat and


tears?


No, so it's a first try and we did not, and it was intentional because our goal was not


to show, our goal was to give it words.


And you know, this weird hash and new line, this is what we took from OpenAIR's website.


They had a bunch of instructions on best practices for formatting your prompt.


I think they have changed fixes, but we just took it from OpenAIR's website.


Yeah, and this was also one of the main motivations, like, you know, even if I don't know how to


exactly have the prompts here, let me, you know, there are two ways in which you could


gain improvements here.


One is in the in context examples within the prompt and the other is at the question side,


you know, there are like just two aspects to for fiddling with this.


And there has been a lot of work on, you know, how to give the right in context examples,


what order, what examples, how to select them.


Our focus is on the question part, like, you know, only on the input part, which comes


from the user and we are trying to pull all the knobs, like turn all the knobs at that


end.


And in some sense, we were able to overcome, you know, some limitations which our prompts


probably have, like maybe there are much better ways of coming up with a prompt than we have.


But I think all those methods are just, if we plug in any of the nicer methods to, you


know, come up with a better prompt, that's just icing on the cake for us.


Could you, if this was first try and it's still in there, so obviously it worked, was


there things that didn't work out over the course of this research, like things where


you got stuck or maybe even ideas that you had to discard halfway through?


I can tell one which really bothered us for a long time.


It's on contrastive prompting, which is we wanted to also give like negative answers


like can the user just say, you know, no, that's not the right answer, you know.


With the autoregressive models, it is really difficult to somehow give them, steer away


from probability mass towards certain tokens, like it's really difficult to do that.


We are still not able to effectively do that.


Like ideally, you know, in the real world, users will give, I think users will give feedback


of the kind, you know, instead of clarifications.


In addition to clarification, they can also say, no, this is not right or this is why


it's not right.


Like the model came up with what's the capital of India and it says the capital is Mumbai.


And I just want to say, no, it is not.


It is like Delhi or like you're looking at the wrong places.


And that's something which we were not able to do.


And I think it's an open problem, like this kind of negative prompting.


It's valuable from a feedback perspective for the future.


We just don't know how to solve it right now.


What is your, maybe what did you, you played obviously a little bit with these large models


with the API, presumably also tried out yourself a lot of things I can only assume over the


course of this research.


Is there anything maybe also a bit independent of the research itself?


Is there anything that you came across that surprised you about these large models and


how people can interact with them?


I think for me, one of the things that's actually stood out from early days is how good the


compiler was.


And I think if you really have been using it on a day to day basis, and I have been


using it for a few months now, it has consistently gotten better.


And initially it had these small weird quirks.


So, you know, these models can basically generate left to right or top to bottom.


So if I have some, but when you program, you would write some functions below and then


you go back up to a function and you want to reference the function below.


So that did not work earlier.


So it would only condition on things that it had seen so far in the file.


But they have improved over that stuff also.


So I think it's astonishing that at least in the structure setting, how good they are


for generating things.


At the same time, it's also interesting that even when you have 175 billion parameters,


how poor the model is at common sense, because it's very clear when you go from these structured


settings to a more open ended setting, like common sense generation or common sense reasoning,


I still think the models struggle a lot.


So it still is clear that there's a long way to go.


So there's a bit of hope.


So I think you have to choose your end application wisely, but there are clearly very cool applications


that can be built for which you don't need AGI, so to say, as long as you have a very


good pattern manager.


One of the surprises for me was on the fact that these models are correctable.


A model can make mistakes which are hopeless.


It's just total understanding is wrong.


But I think over time what has happened is with larger models, even though there might


be many claims that it is missing common sense and these models are dumb and so on, but I


do believe that for a certain question, yes, there might be cases where it's not coming


up with the right answer, but they're still correctable.


They're not dumb anymore.


I think these models are getting, they're correctable in the sense that their output


is not completely off and with some guidance they can get to the right answer.


Awesome.


Is there something other than that that you feel I have maybe not touched in my review


that you would like viewers to know or be able to understand or anything that I've maybe


gotten wrong?


I think most of the stuff you said was correct, like it was nothing was wrong, really.


Your understanding in almost everything was correct.


I'm not fishing for compliments.


If there's something that you feel like people should know about this that we haven't talked


about at all.


Yeah, I think the part that you mentioned in your video about the feedback could be


misleading.


I think we've really touched upon it, but I think that's a valid criticism that still


holds.


And that was one of the things that we have not been able to solve even now.


We are trying different kinds of retrieval, conditioning on the expected output, doing


something like you said, more complex in one of those four modules.


But I think that remains a valid criticism of the work that there will be cases where


feedback would distract.


So the model was going to say the right thing, but because you have this thing, it's saying


the wrong thing.


But we think that problem is kind of, there's an easy way to solve it.


It's to show both the answers to the user and let the user pick one.


So you show this is the answer that I would have given you.


This is what I would give you with some retrieved feedback, pick one.


But if you don't want to do that, then it's kind of very challenging because the model


somehow has to know that it's going to make a mistake.


And only then it should pull up feedback etc.


And those are kind of having, it's very hard for models to know that they are wrong or


to know what they don't know.


So that's a big challenge and kind of one interesting research direction that we are


pursuing outside of this, which is how can we let a model know that they don't know or


they started doing wrong and what can we do in those cases.


I agree.


And if you can do that with a model that you don't even have access to, I think that would


be a little bit of a grail of research.


That would be seriously cool.


And I think it would improve a lot of applications of these models around, you know, all around


technology.


Cool.


Well, Nikit and Aman, thank you very much for being here.


Was a pleasure.


And I hope this work goes on and becomes more powerful over time.


Thanks, Anik.


Dynamic Routing Between Capsules

Hi there, today we're looking at dynamic routing between capsules by Sarah Sabour,


Dynamic Routing Between Capsules

Nicholas Frost and Jeffrey Hinton of Google Brain. This paper is a bit older


Dynamic Routing Between Capsules

but it's made quite the impact at the time and so we'll go through it. I find


Dynamic Routing Between Capsules

it's a pretty hard paper to read and kind of understand because a lot of


Dynamic Routing Between Capsules

things are very implicit and hand-wavy. So we'll kind of go through it and try


Dynamic Routing Between Capsules

to get the best out of it, try to explain what capsules are and what they do and


Dynamic Routing Between Capsules

how they you know stack against current networks. So capsule network in essence


Dynamic Routing Between Capsules

is a new type of neural network made of capsules and here it says a capsule is a


Dynamic Routing Between Capsules

specific is a group of neurons whose activity vector represents the


Dynamic Routing Between Capsules

instantiation parameters of a specific type of entity such as an object or an


Dynamic Routing Between Capsules

object part. Kind of cryptic but so what they're saying is that in a capsule


Dynamic Routing Between Capsules

network, let me try to draw one here actually, in a capsule network you have


Dynamic Routing Between Capsules

what's called capsules. The capsules you can imagine as just little blobs of


Dynamic Routing Between Capsules

things right and they're also ordered in layers in this case. Let's actually leave


Dynamic Routing Between Capsules

away the second layer and each of these of these capsules will correspond to an


Dynamic Routing Between Capsules

entity in the input. Let's say the input is an image so somewhere here there is


Dynamic Routing Between Capsules

an image right then maybe this capsule here will be responsible for detecting is


Dynamic Routing Between Capsules

there a wall in the image and this one will be responsible for detecting is


Dynamic Routing Between Capsules

there a roof. This one will be is there a door and this one will be responsible


Dynamic Routing Between Capsules

for detecting is there a lake in the image right. So now each of these each of


Dynamic Routing Between Capsules

these capsules can for on one hand can either be high or low so if you if you


Dynamic Routing Between Capsules

imagine now a situation where wall high, roof high, door high, lake low, it means


Dynamic Routing Between Capsules

probably the image has a house on it right. But second of all not only can it


Dynamic Routing Between Capsules

predict whether or not a given entity is present in an image but the individual


Dynamic Routing Between Capsules

capsules are also responsible for encoding the exact way or shape or form


Dynamic Routing Between Capsules

that this entity takes. So the wall could have different aspects such as color


Dynamic Routing Between Capsules

color green, it could have size tall, it could have orientation, is like I


Dynamic Routing Between Capsules

don't know vertical, then roof could have angle right angle wide so it's a wide


Dynamic Routing Between Capsules

roof or a flat roof right these are these are kind of attributes of these


Dynamic Routing Between Capsules

things that also the capsules would encode. So ultimately what these capsules


Dynamic Routing Between Capsules

that they are proposing will output is the roof capsule here for example would


Dynamic Routing Between Capsules

output a vector. So the output of the roof capsule is a let me draw a coordinate


Dynamic Routing Between Capsules

system is a vector. Now the length of the vector will represent so that the length


Dynamic Routing Between Capsules

draw this norm here will represent the probability that the roof is in the


Dynamic Routing Between Capsules

image. That there is a roof in an image right the roof is an element of this


Dynamic Routing Between Capsules

input image. This is simply the length and the individual coordinates will


Dynamic Routing Between Capsules

encode these attributes. So this here for example this axis could be the angle of


Dynamic Routing Between Capsules

the roof and this axis could be the color. Let's say just that the angle is


Dynamic Routing Between Capsules

like some degree number that can be positive or negative maybe a roof can be


Dynamic Routing Between Capsules

like this. Right this so this is but in essence this is a flat roof and this is


Dynamic Routing Between Capsules

a very narrow angle roof so you can imagine something like this and then the


Dynamic Routing Between Capsules

color could also be maybe parameterized on a one-dimensional. It can have more


Dynamic Routing Between Capsules

dimensions than two I just can't draw more. So the depending on where this


Dynamic Routing Between Capsules

where this arrow now points the for example this vector here has the same


Dynamic Routing Between Capsules

probability that there is a roof in the image like if the output is this but the


Dynamic Routing Between Capsules

color will be different. The angle will be the same because they're roughly on


Dynamic Routing Between Capsules

the same this axis here but the color of this will encode a different different


Dynamic Routing Between Capsules

colored roof and then if the vector is something like this a very short vector


Dynamic Routing Between Capsules

it will encode the same the same angle and and color directions so maybe I


Dynamic Routing Between Capsules

shouldn't say the position on the axis it's more like this angle and this this


Dynamic Routing Between Capsules

angle that encode the attributes so the kind of the angular components if you


Dynamic Routing Between Capsules

will encode the attributes and the length encodes the probability. So this


Dynamic Routing Between Capsules

small vector has the same direction in terms of color and angle of the roof but


Dynamic Routing Between Capsules

it's much less probable much less likely so this if the capsule outputs the


Dynamic Routing Between Capsules

little blue vector here it says well if there is a roof it's going to be this


Dynamic Routing Between Capsules

color and this angle but I'm really that really don't think there's a roof in


Dynamic Routing Between Capsules

this image whereas if it outputs the large green one then it says I'm pretty


Dynamic Routing Between Capsules

sure that there is a roof and it's going to be this angle and this this this


Dynamic Routing Between Capsules

angle and this color. Alright so that's that is what each capsule is supposed to


Dynamic Routing Between Capsules

do. Each capsule takes the input and outputs a vector that encodes if the


Dynamic Routing Between Capsules

entity that the capsule is responsible for is present in the image A and B what


Dynamic Routing Between Capsules

properties this entity has and then we get to the point where there's the next


Dynamic Routing Between Capsules

layer of capsules. So the next layer of capsules takes information that each


Dynamic Routing Between Capsules

capsule here takes information from each capsule in the lower layer like like


Dynamic Routing Between Capsules

you're used to from your neural network and integrates this information and


Dynamic Routing Between Capsules

we'll talk about how this works it integrates all of this information right


Dynamic Routing Between Capsules

all of these are vectors now that come from the lower integrates all of this


Dynamic Routing Between Capsules

information and again each capsule in this next layer is responsible for a


Dynamic Routing Between Capsules

entity. Now these entities in the higher layers are usually composite entities of


Dynamic Routing Between Capsules

the lower layers so this one here could be responsible for house this one could


Dynamic Routing Between Capsules

be responsible for national park national park and this one could be


Dynamic Routing Between Capsules

responsible for beach or something like this right and then each of these will


Dynamic Routing Between Capsules

integrate all of this information from the lower layers and then come up with


Dynamic Routing Between Capsules

their own output vector encoding whether or not a given entity is present in the


Dynamic Routing Between Capsules

in the image of course the house class will pick up if there is a door a roof


Dynamic Routing Between Capsules

and a wall in the image the house class is it will pick up on that or that's how


Dynamic Routing Between Capsules

it's meant to work house class is meant to pick up on that and then itself


Dynamic Routing Between Capsules

output a large vector saying there's probably a house in this in this image


Dynamic Routing Between Capsules

so each of these capsules in by itself is responsible for encoding the presence


Dynamic Routing Between Capsules

and attributes of a object or object part or entity or part of entity in the


Dynamic Routing Between Capsules

given input data and of course the last layer here it will simply be your


Dynamic Routing Between Capsules

classification layer so in the last layer you have as many capsules as you


Dynamic Routing Between Capsules

have classes in your classification task if so this is mainly for a classification


Dynamic Routing Between Capsules

task and then you can classify and you can kind of train the whole system like


Dynamic Routing Between Capsules

this so how exactly this happens we'll see next all right so they make kind of


Dynamic Routing Between Capsules

analogies to the visual system and so on and we'll we'll jump these you can


Dynamic Routing Between Capsules

everyone that ever does deep learning in some way is trying to to make that we're


Dynamic Routing Between Capsules

rather going to the specifics of how these capsules work and how their


Dynamic Routing Between Capsules

specific suggestions for them note that they say this is in no way the only


Dynamic Routing Between Capsules

implementation of capsules is just kind of an example to show how one could do


Dynamic Routing Between Capsules

it all right so first of all they present their what you might call


Dynamic Routing Between Capsules

nonlinearity so their nonlinearity what it needs to do is if you look at these


Dynamic Routing Between Capsules

capsule networks the outputs here the length of the outputs of these vectors


Dynamic Routing Between Capsules

right they're supposed to represent probabilities and as such they they need


Dynamic Routing Between Capsules

to be so here it roof this door may be a vector like this wall may be a vector


Dynamic Routing Between Capsules

like that so initially it we simply specify the output is a vector and in


Dynamic Routing Between Capsules

essence these capsules are implemented in much the same way like your classic


Dynamic Routing Between Capsules

neural network layer would be implemented so each of the each of these


Dynamic Routing Between Capsules

each of these capsules will be in essence a neural network layer by itself


Dynamic Routing Between Capsules

that outputs a a vector and there's nothing constraining the length of the


Dynamic Routing Between Capsules

vector initially so their nonlinearity does constrain the vector to be of


Dynamic Routing Between Capsules

maximum length one and of minimum length zero that's this nonlinearity here so s


Dynamic Routing Between Capsules

here is the unscaled output of the capsule and you can see here if the


Dynamic Routing Between Capsules

length of s gets close to one or sorry it gets really large then this here is


Dynamic Routing Between Capsules

becomes irrelevant this whole term will be one and then the length of the final


Dynamic Routing Between Capsules

output of v here will be one right so if this is very large then the the length


Dynamic Routing Between Capsules

of the scaled output will be one however if the if the length is really small of


Dynamic Routing Between Capsules

the original output so if this goes towards zero then this becomes


Dynamic Routing Between Capsules

irrelevant this becomes irrelevant this will go towards zero and the entire


Dynamic Routing Between Capsules

length will go towards zero so this is kind of a nice way to scale these


Dynamic Routing Between Capsules

outputs always to be between length zero and one then next thing is so how this I


Dynamic Routing Between Capsules

find I find the the most complicated part right so we'll jump ahead actually


Dynamic Routing Between Capsules

to how a capsules network is implemented and this is the the capsule network they


Dynamic Routing Between Capsules

implement so first it's an MNIST classifier you have an MNIST image here


Dynamic Routing Between Capsules

and it first goes through a simple convolutional layer that's that's nothing


Dynamic Routing Between Capsules

new this is a classic convolutional layer is there's 256 channels it has a


Dynamic Routing Between Capsules

nine by nine filters and stride one so it will output a 20 by 20 time by 256


Dynamic Routing Between Capsules

tensor then each of these so each of the outputs here is sent to each of these


Dynamic Routing Between Capsules

capsules and now they're convolutional capsules so that makes it a bit more


Dynamic Routing Between Capsules

complicated but don't you know don't worry primarily about them being


Dynamic Routing Between Capsules

convolutional capsules the analogy is exactly as in a classic neural network


Dynamic Routing Between Capsules

you can implement these capsules as four feet forward capsules or as


Dynamic Routing Between Capsules

convolutional capsules and well maybe also as transformer capsules I don't


Dynamic Routing Between Capsules

think anyone's done that all right there's a paper for you the so you'll


Dynamic Routing Between Capsules

send you'll send the output of this convolution layer to each capsule and


Dynamic Routing Between Capsules

then you have basically just two layer of capsules here the first layer consists


Dynamic Routing Between Capsules

of 32 what they call primary caps sorry the these 32 capsules each will output


Dynamic Routing Between Capsules

an eight dimensional vector and I'm simplifying here it's it's convolutional


Dynamic Routing Between Capsules

but they will just for simplest they will each output an eight dimensional


Dynamic Routing Between Capsules

vector right and these are exactly as we said before so each of these will be


Dynamic Routing Between Capsules

responsible ultimately for a given entity or part of entity being there


Dynamic Routing Between Capsules

like in MNIST this could be is there a little curve on the bottom left side


Dynamic Routing Between Capsules

right this might indicate the presence of a six or an eight something like


Dynamic Routing Between Capsules

this and then the these capsules here each is they represented as rows so each


Dynamic Routing Between Capsules

of these rows here is a capsule and we have ten of these and these are your


Dynamic Routing Between Capsules

simply your final classification capsules so each capsule is responsible


Dynamic Routing Between Capsules

for indicating the presence or absence of one particular class of digits so


Dynamic Routing Between Capsules

this will be of a one of a two of a three of a four and so on of a zero I


Dynamic Routing Between Capsules

guess somewhere as well so these are ten capsules and the question is how does


Dynamic Routing Between Capsules

information go from a capsule here from the output of a capsule or to any of


Dynamic Routing Between Capsules

capsule here and the easy way to do this is simply to say as in a classical


Dynamic Routing Between Capsules

neural network the output here simply goes to the input here just you just


Dynamic Routing Between Capsules

put it there basically on unchanged now there is a bit of an issue here with the


Dynamic Routing Between Capsules

dimensions but you can simply say well we simply put a weight matrix in to


Dynamic Routing Between Capsules

route into the capsules but the idea of these capsules and this paper is to say


Dynamic Routing Between Capsules

wait wait these capsules actually we want to make them decide to which capsule


Dynamic Routing Between Capsules

in the next layer will they send their input right so the capsules can kind of


Dynamic Routing Between Capsules

decide where they want to send their output to like where is this where is


Dynamic Routing Between Capsules

the capsule that detects the maybe this one detects is there a line in the right


Dynamic Routing Between Capsules

side of the image right indicating maybe a seven or a one this is probably most


Dynamic Routing Between Capsules

relevant for the one class and for the seven class so it might decide to route


Dynamic Routing Between Capsules

its output there and the idea of how this routing happens is basically the


Dynamic Routing Between Capsules

topic of this paper so the the capsules route their output to the appropriate


Dynamic Routing Between Capsules

next layers capsules how is this done all right this is done via the what's


Dynamic Routing Between Capsules

called the routing mechanism I find it quite poorly described here so I will


Dynamic Routing Between Capsules

simply draw it I will I will simply try to make it up all right so we have


Dynamic Routing Between Capsules

capsules and as I've drawn them before right we have one two three capsules and


Dynamic Routing Between Capsules

we maybe have two parent capsules each of these capsules here will output a


Dynamic Routing Between Capsules

vector as we said and we'll only do it for this this one sorry vector here so


Dynamic Routing Between Capsules

this will output this vector and needs to decide where to here or to here do I


Dynamic Routing Between Capsules

send to this output now what it does is there is an iterative procedure that has


Dynamic Routing Between Capsules

multiple steps and this is I think this is at least the way I understand I think


Dynamic Routing Between Capsules

the important part to understand this that if we forward pass data through


Dynamic Routing Between Capsules

this network it actually doesn't go forward in a straight line what it


Dynamic Routing Between Capsules

actually does is it goes through a layer and then it does multiple steps in


Dynamic Routing Between Capsules

between layers until it has decided where it wants to go in the next layer


Dynamic Routing Between Capsules

and then it goes on to the next layer and if there's another another capsule


Dynamic Routing Between Capsules

layers it does again multiple steps before it goes on so that's that's my


Dynamic Routing Between Capsules

take on it and the multiple steps are as follows first I'll send my output vector


Dynamic Routing Between Capsules

to to all of the all of the layers like equally all of the parent capsules and


Dynamic Routing Between Capsules

so will will everyone else right everyone will send theirs equally to the


Dynamic Routing Between Capsules

parent now this isn't just done and this may be here this isn't just done just by


Dynamic Routing Between Capsules

sending it but this is actually done by modulation of weight matrices so each


Dynamic Routing Between Capsules

thing here if this is capsule I and this is capsule J there is a weight matrix in


Dynamic Routing Between Capsules

between w i j that is learned right this is a static weight matrix and each one


Dynamic Routing Between Capsules

of these red red arrows you see here has such a weight matrix attached to it so


Dynamic Routing Between Capsules

each each line you see here is actually modulated by such a weight matrix so


Dynamic Routing Between Capsules

there is an a quadratic number of these weight matrices flying around and this


Dynamic Routing Between Capsules

will also then allow you that maybe this vector is eight-dimensional but the


Dynamic Routing Between Capsules

input vector here is 16 dimensional what we saw before all right so the out the


Dynamic Routing Between Capsules

input of capsule J here it will receive let's see what it receives it will


Dynamic Routing Between Capsules

receive the output of capsule will the output of capsule 1 v1 modulated by the


Dynamic Routing Between Capsules

let's let's call this yeah let's call this J modulated by 1 J w 1 J and it


Dynamic Routing Between Capsules

will also receive this as a set the output of capsule to modulated by the


Dynamic Routing Between Capsules

weight matrix for sorry weight matrix for capsule 2 and so on now what it does


Dynamic Routing Between Capsules

is it adds this these all up into a softmax so sorry let's write this so


Dynamic Routing Between Capsules

soft it will add those all up in a softmax weighted fashion so it will


Dynamic Routing Between Capsules

actually compute a a weighted average of those now the weights at the beginning


Dynamic Routing Between Capsules

are are just one because it gets each from each lower capsule it gets equal


Dynamic Routing Between Capsules

amount of this vector but then this will give you an output so this will give you


Dynamic Routing Between Capsules

some output let's put this in green this will give you an output that's I don't


Dynamic Routing Between Capsules

know how they call it in the paper let's just call it oh J right and then what


Dynamic Routing Between Capsules

you do is all right you compare how much do each of the individual


Dynamic Routing Between Capsules

contributions agree with OJ so you actually compute for each of these you


Dynamic Routing Between Capsules

would compute the inner product so you would compute the inner product of w1 J


Dynamic Routing Between Capsules

v1 with OJ and you would compute the inner product of w2 J v2 with OJ all


Dynamic Routing Between Capsules

right the inner product and then these inner products here will become the


Dynamic Routing Between Capsules

weighting coefficients for the softmax in the next iteration all right so this


Dynamic Routing Between Capsules

I mean this this is a bit convoluted but ultimately what you're saying is if


Dynamic Routing Between Capsules

you're a capsule here you'll send your output forward you have an output you'll


Dynamic Routing Between Capsules

send it forward right to the other capsule and the other capsule will so


Dynamic Routing Between Capsules

this is this is your output and we'll forget about this weight matrices for


Dynamic Routing Between Capsules

now this is your up the other capsule will output its own its own output


Dynamic Routing Between Capsules

computed from the lower layers now we do an iteration again if your output now


Dynamic Routing Between Capsules

aligns with this you will send more of it and these these two that I've drawn


Dynamic Routing Between Capsules

here actually aligned pretty well right so you'll send more of it is more more


Dynamic Routing Between Capsules

more right and now the maybe the output that next computed output of the same


Dynamic Routing Between Capsules

capsule will be even more in that direction because you've contributed


Dynamic Routing Between Capsules

more right you'll send more and then you're like in the next iteration wow


Dynamic Routing Between Capsules

these two are really equal sorry this should be red here yours just keeps


Dynamic Routing Between Capsules

being the same and then you say well I'm gonna send even more to that one right


Dynamic Routing Between Capsules

whereas another capsule that it's whose initial output was basically whose


Dynamic Routing Between Capsules

initial output was basically like this it will by itself compute the inner


Dynamic Routing Between Capsules

product with the original this original it will send it here right it will


Dynamic Routing Between Capsules

compute the inner product with the original output and it will realize well


Dynamic Routing Between Capsules

these do not align very much and then it will send less right it will ascend less


Dynamic Routing Between Capsules

at the next step and because it sends less in the next step of course the


Dynamic Routing Between Capsules

output will then probably align even less with that vector and then it will


Dynamic Routing Between Capsules

send less and less and less so this is called dynamic routing the the idea


Dynamic Routing Between Capsules

behind it is kind of that you route by agreement so you will route to the


Dynamic Routing Between Capsules

parent capsules that agree with your output and by agreement we mean kind of


Dynamic Routing Between Capsules

the inner product is high after modulating by this weight matrix and


Dynamic Routing Between Capsules

that sort of so that basically means this weight matrix is responsible for


Dynamic Routing Between Capsules

deciding which information is relevant together whenever you have two vectors


Dynamic Routing Between Capsules

that align in the same layer then the in the sense of the capsule networks those


Dynamic Routing Between Capsules

represent the same kind of information and those will be routed together to the


Dynamic Routing Between Capsules

same capsule in terms of the examples we made maybe if a door and the roof is


Dynamic Routing Between Capsules

present then these these these weight matrices that connect door and roof to


Dynamic Routing Between Capsules

the house class they will transform a high vector in door and roof into


Dynamic Routing Between Capsules

aligning vectors for the house class and thereby saying look these two if I look


Dynamic Routing Between Capsules

at them through if I look at a door and a roof through the perspective of trying


Dynamic Routing Between Capsules

to be a house right then they are in much agreement on the presence of a house


Dynamic Routing Between Capsules

so if I am a house right I am a house and I look at a door and I look at a


Dynamic Routing Between Capsules

roof through the kind of from the perspective of being a house right this


Dynamic Routing Between Capsules

is this is what these weight matrices do they always have a perspective of the


Dynamic Routing Between Capsules

parent capsule then these two things they make a lot of sense together and


Dynamic Routing Between Capsules

thus I will route them to the same place so they can both contribute to there


Dynamic Routing Between Capsules

being a house now from the perspective of a house if I look at a little beach


Dynamic Routing Between Capsules

with a tree on it right then that does not that is not the same that does not


Dynamic Routing Between Capsules

really is not the same information as a door or a roof so I will not route this


Dynamic Routing Between Capsules

to the house in the in the same strength that is sort of the best way I have of


Dynamic Routing Between Capsules

explaining it how these capsules work basically the lower entities will always


Dynamic Routing Between Capsules

be routed for the relevance of the higher entities that are trying to they're


Dynamic Routing Between Capsules

trying to combine the lower entities if that wasn't it's not entirely clear to


Dynamic Routing Between Capsules

me either yet but it's the best shot I I can give and the routing is here


Dynamic Routing Between Capsules

formalized I I find it hard to follow the important thing is that there is an


Dynamic Routing Between Capsules

inner loop in all of this so there is an kind of an inner iteration and this


Dynamic Routing Between Capsules

inner iteration is computed in every forward pass and so the this routing


Dynamic Routing Between Capsules

where the information goes in the next layer that is only the prior probability


Dynamic Routing Between Capsules

for that is learned but the actual routing coefficients those are


Dynamic Routing Between Capsules

dynamically computed in every forward pass every forward pass goes it goes


Dynamic Routing Between Capsules

information goes through a layer then it goes multiple steps between two layers


Dynamic Routing Between Capsules

until it decides exactly what the distribution for the next layer is and


Dynamic Routing Between Capsules

then the next layer computes its outputs and that goes again multiple steps


Dynamic Routing Between Capsules

between these layers and the next layer so that's the the basic thing to


Dynamic Routing Between Capsules

remember there's also some normalization involved the squashes the non-linearity


Dynamic Routing Between Capsules

we discussed so what do they actually train now at the end here they have a


Dynamic Routing Between Capsules

they have these ten capsules and each capsule will be responsible for


Dynamic Routing Between Capsules

recognizing one the presence of one digit in the ms data set of course and


Dynamic Routing Between Capsules

so what they do is they take the length of these vectors that are output by


Dynamic Routing Between Capsules

these capsules these capsules are feet forward capsules as opposed to the


Dynamic Routing Between Capsules

convolutional capsules here so the feet forward capsules output again a vector


Dynamic Routing Between Capsules

the length of this vector is taken and then it's basically trained like you


Dynamic Routing Between Capsules

would train a regression problem and the loss here is specified up here so if the


Dynamic Routing Between Capsules

if the image actually does contain this if the training label actually has this


Dynamic Routing Between Capsules

digit present this T here encodes that so if if K let's say K is 2 right so if


Dynamic Routing Between Capsules

K 2 if there is a 2 in the image when we know that because it's a training image


Dynamic Routing Between Capsules

then the length of the output of capsule number 2 should be high and this simply


Dynamic Routing Between Capsules

encodes that it should be very close to this m plus an m plus here is that I


Dynamic Routing Between Capsules

think they said it to 0.9 so they say you should be the length should be as


Dynamic Routing Between Capsules

close as possible to 0.9 whereas if the 2 is not present then T K will be 0 then


Dynamic Routing Between Capsules

this part will be active so it's only one of these two parts will be active


Dynamic Routing Between Capsules

then the length of the vector so of capsule number 2 should be close to this


Dynamic Routing Between Capsules

m negative which is 0.1 it's basically a regression problem saying if if the if


Dynamic Routing Between Capsules

the given entity is in the image then please make the length as close as


Dynamic Routing Between Capsules

possible to 0.9 and if it's not make it as close as possible 0.1 so this this is


Dynamic Routing Between Capsules

a classic say regression loss on the length of the output vectors the the


Dynamic Routing Between Capsules

lambda is just a factor to to dampen the contribution for all the negative


Dynamic Routing Between Capsules

classes with respect to the one positive class of course per capsule it turns out


Dynamic Routing Between Capsules

this is actually not enough so this will be the classification output but it's it


Dynamic Routing Between Capsules

seems not enough they don't say it's not enough but they simply say we


Dynamic Routing Between Capsules

additionally do the following so they also do is they introduce a


Dynamic Routing Between Capsules

reconstruction loss now if this model is trained correctly then these capsules


Dynamic Routing Between Capsules

here these last capsules especially this one maybe that's the capsule


Dynamic Routing Between Capsules

corresponding to the class of the digit 8 will not only encode if an 8 is there


Dynamic Routing Between Capsules

or not as in the length of the vector output but it will also encode the


Dynamic Routing Between Capsules

properties of the eight it is a 16 dimensional vector so it will encode


Dynamic Routing Between Capsules

hopefully things like the stroke width so it then it might encode the maybe


Dynamic Routing Between Capsules

the rotation of the digit then it might be controlled the tightness of the of


Dynamic Routing Between Capsules

the loop so you can have an eight with very large loops or it can have an eight


Dynamic Routing Between Capsules

oh sorry this is a smaller eight I can have an eight with very tight loops so


Dynamic Routing Between Capsules

it might you know encode things like this so technically it is it will be


Dynamic Routing Between Capsules

possible to reconstruct from this description reconstruct say the width is


Dynamic Routing Between Capsules

high the rotation is zero and the tightness is low then maybe I have a a


Dynamic Routing Between Capsules

wide widely stroked not tight eight that is not rotated right so it should be


Dynamic Routing Between Capsules

possible to reconstruct this and they do exactly that so they take this last


Dynamic Routing Between Capsules

capsule of the class that is the actual training label that's called the


Dynamic Routing Between Capsules

reconstruction target and they feed this to a simple feed forward neural network


Dynamic Routing Between Capsules

that at the end you see this is exactly the m nest size will try to reconstruct


Dynamic Routing Between Capsules

the image so if the image here this image goes in then it goes all through


Dynamic Routing Between Capsules

here it will take the class for here feed it through this network reshape it


Dynamic Routing Between Capsules

to an image again and hopefully what will come out is again this for here and


Dynamic Routing Between Capsules

it will then have an auxiliary auxiliary loss in addition to the loss of this of


Dynamic Routing Between Capsules

this classification loss here will auxiliary loss that tries to


Dynamic Routing Between Capsules

reconstruct the original image right and that's simply a I believe it's just an


Dynamic Routing Between Capsules

l2 reconstruction loss that is that is scaled down that it doesn't dominate so


Dynamic Routing Between Capsules

they also train the network basically to reconstruct this and I believe they do


Dynamic Routing Between Capsules

this because the length isn't quite enough to make it do what they want it


Dynamic Routing Between Capsules

to do thus they by having this reconstruction here they really kind of


Dynamic Routing Between Capsules

enforce that the individual capsules the individual dimensions must encode some


Dynamic Routing Between Capsules

kind of information about the original image and since the original images in


Dynamic Routing Between Capsules

the m nest data set at least vary by those things by stroke with by rotation


Dynamic Routing Between Capsules

by tightness that by this loss will be reflected in the in the reconstruction


Dynamic Routing Between Capsules

all right so how are they doing here you see different examples of inputs and


Dynamic Routing Between Capsules

then reconstructed outputs and this you know seems pretty good actually so you


Dynamic Routing Between Capsules

see here all of these the input image is reconstructed fairly well so the numbers


Dynamic Routing Between Capsules

up here in the following so the right are the failure cases here it the input


Dynamic Routing Between Capsules

image is a five labeled in the training data but the network actually classifies


Dynamic Routing Between Capsules

it as a three but then if you now you have two choices right this this is the


Dynamic Routing Between Capsules

same sample now you have two choices for reconstruction either you reconstruct


Dynamic Routing Between Capsules

the capsule that is actually the is that you know is the true capsule that should


Dynamic Routing Between Capsules

be activated and you reconstruct from that or you reconstruct from the capsule


Dynamic Routing Between Capsules

that the network says the it classifies it as so here it mixed up a five four


Dynamic Routing Between Capsules

three if you still take the five capsule and reconstructed you see it actually


Dynamic Routing Between Capsules

looks like the original image but it looks much more like a five and if you


Dynamic Routing Between Capsules

take the three capsule to reconstruct which is what the network classified


Dynamic Routing Between Capsules

this as it still it looks like the original image but it looks much more


Dynamic Routing Between Capsules

like an actual three right it's it's missing the the part up here whereas


Dynamic Routing Between Capsules

over here it's it's missing this part here so that the network really seems to


Dynamic Routing Between Capsules

kind of learn the different variations of these digits and in an ambiguous case


Dynamic Routing Between Capsules

such as this one it you know it can it can actually go either way and it can


Dynamic Routing Between Capsules

actually reconstruct the original output in either interpretations once as a


Dynamic Routing Between Capsules

three and once as a five it will be interesting to see what the actual


Dynamic Routing Between Capsules

length of the vector of both of these classes were that were mixed up and here


Dynamic Routing Between Capsules

they compare their accuracies so they have a baseline model which I believe is


Dynamic Routing Between Capsules

just like a CNN where they get a decent kind of error and then the capsule


Dynamic Routing Between Capsules

networks they get a lower error and here you see as you add the reconstruction


Dynamic Routing Between Capsules

loss and as you add routing more so one step of routing simply means the first


Dynamic Routing Between Capsules

step is where you send your output equally to each parent that is as in


Dynamic Routing Between Capsules

the classical neural network case but if you introduce three steps of routing


Dynamic Routing Between Capsules

then your error drops even lower so they they kind of are on par with baseline


Dynamic Routing Between Capsules

CNNs on MNIST here they also explore what their capsules learn so as I said


Dynamic Routing Between Capsules

the individual capsules the dimensions should encode kind of properties of the


Dynamic Routing Between Capsules

variations of the of the class class samples and here they explore this in


Dynamic Routing Between Capsules

the different capsules so they change some dimensions and they run it through


Dynamic Routing Between Capsules

their reconstruction networks and indeed they discover that there is like a scale


Dynamic Routing Between Capsules

and thickness dimension stroke thickness dimension there's a skew dimension and


Dynamic Routing Between Capsules

so on width and translation so that that this is pretty remarkable these networks


Dynamic Routing Between Capsules

really if you train them in this way they really seem to learn about the


Dynamic Routing Between Capsules

entities and about the properties of the entities and that seems to be quite


Dynamic Routing Between Capsules

interesting you see that there's everything here stays well within the


Dynamic Routing Between Capsules

class that the capsule is assigned to they also yeah there's robustness to


Dynamic Routing Between Capsules

affine transformations where they improve over the baseline it's kind of


Dynamic Routing Between Capsules

an auxiliary experiment the next interesting experiment is what they


Dynamic Routing Between Capsules

call the multi MNIST experiment the multi MNIST experiment is done by taking two


Dynamic Routing Between Capsules

different MNIST digits and basically just overlapping them so that they have


Dynamic Routing Between Capsules

you know shift them slightly but as you see here or here they are overlapped


Dynamic Routing Between Capsules

heavily and the task of the network is to figure out which two overlapping


Dynamic Routing Between Capsules

digits are in the image and the the network is very very good at doing this


Dynamic Routing Between Capsules

the capsule network that is and better than the the baselines because the


Dynamic Routing Between Capsules

capsule network simply encodes the presence and properties of a particular


Dynamic Routing Between Capsules

instance in the image if you simply take the top two length capsules and then


Dynamic Routing Between Capsules

reconstruct those independently then you're you can you can you can basically


Dynamic Routing Between Capsules

segment the image and you see this here so the different colorations come from


Dynamic Routing Between Capsules

two different reconstructions of the image from two different capsules so


Dynamic Routing Between Capsules

green is from one capsule and red from the other capsule so the network


Dynamic Routing Between Capsules

correctly identifies that it's a six and zero right and it also correctly


Dynamic Routing Between Capsules

identifies not only which pixels belong to the six and which belong to zero but


Dynamic Routing Between Capsules

also pixels that belong to both so that's not a not a problem if you use


Dynamic Routing Between Capsules

capsule networks as they are a notable to say here they the way they train is


Dynamic Routing Between Capsules

is they train the actual reconstruction by only reconstructing one at a time so


Dynamic Routing Between Capsules

the kind of the premise of the data set is that you actually have access to the


Dynamic Routing Between Capsules

underlying individual digits while training so like the images of the


Dynamic Routing Between Capsules

individual digits you don't know not only have this this label here but


Dynamic Routing Between Capsules

that's a detail here are some kind of failure cases where it it misclassified


Dynamic Routing Between Capsules

or you misspecify the capsules and it's kind of unable use here you see to to


Dynamic Routing Between Capsules

assign the digits of the misclassified or the pixels of the misclassified thing


Dynamic Routing Between Capsules

it's quite interesting to look at the failure cases but I find it more


Dynamic Routing Between Capsules

interesting to look actually the success cases and the kind of ease at which the


Dynamic Routing Between Capsules

at which the capsule networks can do this simply by how they're structured


Dynamic Routing Between Capsules

all right so then lastly they also experiment on CIFAR-10 and interestingly


Dynamic Routing Between Capsules

the CIFAR-10 experiments show that the capsule networks don't perform as well


Dynamic Routing Between Capsules

there and as you know CIFAR-10 is a data set that is about the same size as


Dynamic Routing Between Capsules

MNIST but it's first of all color and second of all is natural images and so


Dynamic Routing Between Capsules

they have quite a bit of clutter it's not black and white black background


Dynamic Routing Between Capsules

white digits it's actually there's a sky and like on an image there's lots of


Dynamic Routing Between Capsules

things going on and right there's my tree and there's stuff here and there's


Dynamic Routing Between Capsules

stuff here and the the capsule networks they like to account for things in the


Dynamic Routing Between Capsules

image so they like to have a capsule corresponding to everything that's going


Dynamic Routing Between Capsules

on here and here and here and here and here if the whole background is black


Dynamic Routing Between Capsules

that is not a problem you can account for simply the the background but if


Dynamic Routing Between Capsules

there's lots of things going on then these capsule networks get they get they


Dynamic Routing Between Capsules

get a bit over explanatory they want to explain everything and that degrades the


Dynamic Routing Between Capsules

performance now this paper basically says yeah you can have a something like


Dynamic Routing Between Capsules

a none of the above category and they found that it helped to introduce that


Dynamic Routing Between Capsules

in my opinion that it I think the the the solution will be more towards


Dynamic Routing Between Capsules

introduction of a better loss function for this because like such that you


Dynamic Routing Between Capsules

don't need kind of to explain the entire thing rather than here well here what


Dynamic Routing Between Capsules

you do is you simply explain it by saying it's none of the above but it's


Dynamic Routing Between Capsules

incredibly hard to balance that my opinion all right so that is basically


Dynamic Routing Between Capsules

the end of this they say they have a discussion here where they compare


Dynamic Routing Between Capsules

capsules against other related work but I hope that you kind of got an overview


Dynamic Routing Between Capsules

of how this works now and as much as possible and with that that was it for


Dynamic Routing Between Capsules


GPT-NeoX-20B - Open-Source huge language model by EleutherAI (Interview w/ co-founder Connor Leahy)

Big announcement by Eleuther AI releasing GPT Neo x 20b. This is a 20 billion parameter


large language model, and it will be publicly released in about a week from now so less


than a week from when you're seeing this, we have a blog post right now. So there will


all also be a paper coming up the blog post details a little bit about the effort a little


bit about the model and releases some results on language modeling tasks and on factual


knowledge tasks where the model compares pretty good pretty well against comparable baselines


not as good as something like GPT three which of course is 10 times larger, but it holds


up quite well. And now I'm happy to welcome Connor Leahy, who is one of the founding members


of Eleuther AI and worked on GPT new x 20b over the last months and even years I guess,


and we'll see what he has to say about it. Cool. Hey everyone. Today I have with me here


Connor Leahy, who is one of the team members founding members of the Luther AI and creators


of GPT neo 20 GPT neo x 20b model. Connor, welcome. Thanks for having me on the show.


It's really cool. I saw the announcement and let's, this is a big release, right? Yeah.


So this whole thing was definitely like a year in the making overall. So we first started


seriously working on, you know, larger model like this with core weave around, yeah, about


a year ago. It was like probably like last February, maybe March. We had like starting


harm series discussions. The chip shortage hit us. That was like a big problem to building


the actual cluster and stuff. And you know, just write the code and whatever. And yeah,


finally we got to training about three months ago and yeah, got the model done like in the


last couple of weeks and now pushed for release. So the cluster, you built a cluster for this


whole. It's not like there was one available, but you actually had to get hardware and so


on. It's pretty cool. Like how, how does that work together with a hardware sponsor like


core weave? So core weave have been really great to us. This wouldn't have been possible


without them. Basically after we released the pile about a year ago and we kind of first


had some variety or whatever, core weave either December or January, I don't exactly remember


when we first approached us, but they kind of first approached us and they're like, Hey,


let's do this. Like, you know, we want to get into large model training for our customers


anyways. We don't, and we would love you guys like test our hardware to like help us find


the right configurations of hardware. It was kind of like a, it was kind of like a back


and forth kind of like, you know, we give them, you know, free testing, free advice,


free consulting, and in return we get to use their cluster to build big models and release


them. So like there was no financial. Yeah, it was great. You know, no financial exchange


either way. It was just, you know, both helping each other. So, and you said, sorry, you said


you delay the release of the model, the weights for seven days due to your sponsors. Like


what's that? Like why seven days? They asked for an exclusivity period. So people would


try to use AI. Okay. That's basically it. I mean, it's just kind of the initial press


bomb boost leads them. I mean, I tried it, so it worked. Yeah. So, you know, we thought


this was a very reasonable thing that we think doesn't like, isn't like a big compromise


on our values or anything here. You know, we, our paper isn't finished yet anyway, so


we probably would have delayed it anyways because we have finished writing our paper,


which we want to release at the same time as we release the model. So this cost us basically


nothing. It's good marketing for our friends. Everyone wins. Excellent. Give us a bit of


a, like just the dimensions of the model right here. 20B is like, we've heard, like we're


accustomed almost to this billion parameter models. What is it like scale of hardware,


scale of just stuff that goes into it? What is it like? So the 20B model was trained on


96 A100s all interconnected with SPX4, you know, NV switch interconnect and HDR InfiniBand.


So this is all super high end data center quality hardware. It was one of the things


we learned while building the cluster and while we had built an actual cluster is at


first, you know, Core has like a ridiculous number of GPUs, like one of the biggest crypto


miners and they provide like GPUs for like lots of like other services and whatnot. And


so they have like thousands and thousands of GPUs. Unfortunately, the kind of GPUs you


might use for crypto mining or for cloud gaming or for something like this, usually


a single, you know, like single PCIe type GPUs. And those will not work for these large


kinds of models where the bottleneck is really the communication between the individual chips.


So you need this really low latency InfiniBand, you know, GPU to GPU direct interconnects and


stuff if you want to have any hope of, you know, training these things. So, you know,


we tried like a bunch of like demo nodes that like didn't have NV switch or it didn't have


InfiniBand or whatever. We haven't really worked our way up. And ultimately, really,


this is the only thing that was possible and that's what we had like kind of build it this


way. So it was trained for three months on 96 A100s, which is quite a lot of, quite a


lot of compute. And now the final model, if you want to use it for inference, it should


run fine on any card, any GPU with about 48 gigabytes of memory or so. So it runs on an


A6000 or an A40.


Cool. Excellent. So the model, we'll get into a little bit of the results right here. There's


not too much yet. There's a press release. Your paper is going to come out. The model,


as we said, are going to come out in about a week or so from time of where we record


this. But you have released some of the results. Can you give us maybe like a summary of the


results? Maybe something that was surprising to you or especially noteworthy?


Yeah. So there's definitely a few interesting things that happened during the training and


also with the eval results. So one funny thing that happened is during the training, our


evals were really bad and we were kind of disappointed. But it turns out we actually


had a bug in our code in like one of the operations, the Fuse Softmax. The way it was implemented


caused it to give you bad results if you don't use the full context length for some reason.


So the training was actually totally fine. And once you fix that bug, all of our benchmark


jumped by like three or four percent. So that was nice. So the way the results currently


look is the way I would describe it is it's a little less good at like natural language


than maybe you would expect of a model of this size. But it is like a good bit better


at like knowledge. This makes sense given the amount of the kind of data we've trained


on. We trained on a lot of code. We trained on a lot of scientific papers, medical papers.


So one of the things we did different in this model is we actually use a different tokenizer.


So that's why, you know, like comparing loss, it doesn't make sense to compare like complexity


or loss to the other musts, why we show like these accuracy numbers. So we use a tokenizer


that we trained on the pile. And also we add like a bunch of like custom tokens or like


multiple white space to like make code more efficient. So we tried like a bunch of different


things which in retrospect, we should have tried everything at once for the big model.


We probably should have done more ablations before we started. If I have one piece of


advice to people building big models, do ablations, do hyperparameter sweeps on small models.


Really really do that. It's really really important. So yeah, so as the final results,


I'm generally pretty happy. You know, it's not GPT-3 level, of course not, because you


know, Givenchy is a huge ass model and a really very very well designed model. It compares


pretty favorably, I think, in most tasks. It's not, it doesn't knock anything real to


the park, I would say. It's pretty good. It has a lot of very good knowledge, very good


scientific knowledge. I haven't tried it yet very extensively myself to give you like a


subjective impression of how it works. And one thing worth mentioning is the HellaSwag


results, which are just weird. We don't really know why they are so low, like the HellaSwag


results specifically are like much lower than we would have expected them to be. We do not


have an explanation for why that is. Okay, short interjection. Connor actually told me


later that they've mixed up two of these numbers, which means that HellaSwag actually performs


quite well. Yet it is the WSC that is not really explained why it's so bad. They suspect


that it's the data set because JPTJ was already kind of bad on that model. But we don't know.


Yet to be seen.


But it seems it seems that on on the what we call standard language modeling tasks,


it kind of holds, holds itself, you know, holds par with, let's say, Fairsec or so is


a bit behind DaVinci. And then on the factual knowledge tasks, it is quite a bit better


than something like Fairsec. Right. Is that is that a function of because there is I don't


know. Do you know Fairsec? What kind of data was trained on?


I don't know off the top of my head.


Okay, because there might be like a trade off between, you know, model size may be responsible


for some things and then data size or quality or nature might be responsible for another


thing. It's pretty cool.


Yeah. So I expect this to probably be down to the data. So because, yeah, just the way


the pile is built up and like because we also have a tokenizer specialized for the pile.


It's like the original GPT2 token. So honestly, no one knows what tokenizers actually affect.


Like no one has done any good studies on what different tokenizers do, whether large or


small vocabularies are useful, whether you want whether having words in your dictionary


is good or bad. Like no one knows. This is all guessing, basically. And so, like, for


example, our tokenizer has like, you know, really long medical terms as single tokens


in it. But, you know, sometimes lacks like some common, you know, words you might see


in a book or something in its tokenizer, unlike other models. So I'm not too surprised that


our model does pretty good on scientific things, which is generally, I think, something we're


interested in. I'm pretty sure if you would fine tune it, you would probably get really


good results for other tasks as well. So like, as you know, it's always important to caveat


that this is, you know, an untuned model. This is a generally trained model. It can


still be fine tuned. And yeah, we also don't know the best sampling parameters or whatever


yet. So I'm sure people get a lot more performance out of it. Same thing was happened with GPT-J


when it first came out. When GPT-J first came out, it was horrible. Like every time you


use it for anything, it was just awful. And then we turn it, for some reason, like GPT-3


is pretty decent. If you have it at temperature one, it's like not that bad. But for some


reason, GPT-J just hates that. And you have to turn down the temperature to like 0.8.


Otherwise it's just awful. I can't explain why. It's just, models have personality.


And so there is this difference, right? There's GPT-J, which I understand is a JAX implementation.


GPT-NEO-X has like a different code base. And the X is also an iteration on GPT-NEO,


which was sort of your first project. Can you explain us a little bit, are these different


people working on the different things? Why isn't there a GPT-J20B? So what's the reasoning


behind sort of building these models, choosing the code bases, choosing what technologies


to use? So it's mostly all by necessity. So we started with GPT-NEO when we only had access


to TPUs from the TensorFlow Research Cloud as our sole compute. NEO is an incredibly


cursed code base and should never be used by anyone. So NEO is fully deprecated. Do


not use NEO. We do not support NEO. We do not, don't even look at it. J is an offshoot


in the sense. So yes, it is written completely in JAX, but it's done basically exclusively


by Ben Wang. He basically just did that by himself, absolute mad lad. So it's kind of


like an offshoot of the Eleuther AI project. So it's like a different type, different people


worked on that than worked on NEO. The reason there is no J20B is that MTJ, so the actual


code used to train 6B in my, if I remember incorrectly, lacks certain kinds of parallelisms


that you would want for these large animals. You can do it, like we've tested it. It does


kind of work, but it's pretty slow and we just can't reliably get enough TPUs to actually


make that happen. So like, you know, we can get, you know, we've got like with 6B, you


know, we just kind of just enough TPUs. I think it was 256 for like three weeks or so.


And you know, that took its time and it's very dependent on how much TPUs Google is


currently using internally, whether we get access to some, cause they're all preemptible.


So we moved to NEOX, which is written in PyTorch because we got GPUs, which is much nicer than


TPUs. So yeah, that's basically the whole reason for that. So the people that worked


on NEOX are basically kind of the same people who worked on NEO. So big shout outs in particular


to Sid Black, who is like, you know, the figurehead for most of the NEO projects. Also, of course,


too many people to name, but there's a lot of other people who have also contributed


a lot. It's pretty cool to see like different technologies


matter because people are always like, well, you know, do you prefer TensorFlow or PyTorch


or JAX and people are like, you know, whatever you want, like whatever fits. But as soon


as you get to like these frontiers of engineering, it actually matters kind of. I mean, you could


probably, as you said, implement anything in anything, but there the differences between


can I do parallelism, can I do this or that, how easily can I do it? It's cool to see that


there's still kind of a distinction between stuff and it's not just all like the same.


My question is a bit, as you train these big model, you said ablations on small models,


you know, to know your hyperparameters, how much handholding is required for the big models?


Like how often do you have to like stop training, change something and then continue from where


you stopped or this does not happen at all? Do you just restart and hope for better luck


with some other parameters? So with 20b, we didn't have any like terrible


problems, like things diverging and stuff like that. We of course did a lot of testing


with hyperparameters, whatever, but honestly we could have done much more. And like, again,


I can't, it's like large model training is very much alchemy. Like you think ML is alchemy,


this is the alchemy of the alchemy. Like it is very much secret recipes of like, like


for example, you know, knowing that you set the atom beta two parameter to 0.95 instead


of 99 is really important. Like if you don't set it to 95, if you set it to 99, which is


the default, if you can't train large models, like it's like way more unstable and like


you would never. Come on, that's common knowledge.


Oh yeah, common knowledge. Everyone would know these things. So yeah, it's just like,


and like there's like so much of it is like folklore too. Like I remember someone asked


someone at OpenAI, like why do they use weight decay? And the answer was because Alec Redford


said it helps. Like that's the, that's the whole reasoning why people use weight decay


is because Alec Redford said it helps.


Isn't there also like a difference between, I believe, isn't there a difference between


like the atom parameters in the different frameworks, like the default parameters?


Yeah, I think that is true. I don't know about the top of my head, but yeah, so like there's,


there's a lot of like little details like that that don't matter as much as smaller


networks, but can really matter on large networks. So 20B, I think is kind of like on the frontier


of models that are still trainable in reasonable circumstances. So for example, the big science


project from Hugging Face has been having an absolute hell of a time trying to train


a hundred billion parameter model and it just keeps diverging and then they roll it


back to try something else and it diverges and they roll it back. And we didn't have


to do that with 20B. 20B was actually pretty well behaved, all things considered, once


we had a set of parameters down and a pretty decent data set. Also very important, data


set really matters. Like it really, really matters. Even the pile is like, we could do


better now in retrospect. We're seeing like there's a lot of things like deduping and


stuff that we could have done that we think would improve it quite a lot. So I remember


for example, the big science project once had this like huge divergence that like keeps


happening and then they looked into the data set and they found that it was like 500,000


backslashes just consecutive.


I mean, you gotta see it, right? If you see, if you're gonna, yeah, it's better than 4chan,


I guess. So people can try out this model. If they go to Google, say I, you can make


an account and you can play around with it a little bit. It is the default model currently


right here. I tried Hello and it did give me some code. Yeah, it gives me some code


again. You said you haven't played around with it much, but is like what kind of stuff


would you expect to work nicely? Do I have to set now the temperature to point A?


I have no idea. So like I'm just saying like that's how it was with J. I don't know Neos'


personality, so I expect people to still find better and better parameters. Also like the


playground, you can see it's brand new, so I'm sure they're gonna add like more features


like repetition penalty and stuff, which helps. So what I would expect Neos to be best at


is code and like scientific tasks. So like for example, I used to know a doctor who used


like J and our Neos models to give him ideas for new research topics. He would like prompt


like you are a brilliant medical epidemiologist working in the field of XYZ and you are going


to study and then it sometimes came out with really interesting experiments. I know that's


like a common use case or whatever, but I would expect that to work. I'm sure it's fine


at like, you know, story generation and stuff like that. I would expect that like fine tuning


it on more of those tags will probably make it a lot better. But yeah, his knowledge should


be pretty good. It should be pretty decent in coding, not as good as codex or God forbid


alpha code, of course, but I would expect it to be pretty decent at all of these tasks.


And this is still language modeling. So this is still like likelihood next token prediction.


This isn't any contrastive training or anything like this.


Yep. This is just plain GPT-3 type training.


Nice. Cool. Is there anything else you want to shout out about this model people code,


anything?


Well, I guess I just wanted to say, you know, thanks to the Aluthre AI people also like


to shout out maybe Ann Lanton and Aaron, who is their CEO, who has been very instrumental,


including some of his employees have been really instrumental in helping with this.


So this wasn't just Aluthre AI, we also got a lot of help from them and some of the cluster


stuff. And as you can see, they're also a partner on the Goose AI project. So we're


very thankful for their help. It's been quite the ride. It's been good fun. We don't intend


to stop here. If you're interested in Aluthre AI in the kind of work we do, or if you're


an academic or research that wants to work on this kind of model, we'd love to hear from


you. Check out our Discord. Love to hear from you.


[ML News] This AI completes Wikipedia! Meta AI Sphere | Google Minerva | GPT-3 writes a paper

Meta AI releases a model that can check Wikipedia citations for accuracy. Google research releases


a model that can solve math problems just by reading research papers and GPT-3 writes a


paper about itself. Welcome to ML News. I was going to start the news but I had word


dully open from last time and I'm pretty sure it's doge to the moon. Check it. Nice. Excellent.


Excellent. Let's dive in. The Meta AI blog has an article called how AI could help make Wikipedia


entries more accurate. This is about a system called sphere. The article starts off by describing a


common problem on Wikipedia. The example here includes Joe Hipp. Hipp was a member of the


Blackfeet tribe and was the first Native American to compete for the World Boxing Association's


heavyweight title and Wikipedia actually does know and state that fact. However, if you go and check


the citation, at least if you did so about a month ago, then that citation would have nothing to do


with either Joe Hipp or boxing the citation would be wrong. Wikipedia has systems to detect kind of


spam people entering gibberish people entering some sort of ads into articles, but they don't


yet have good systems for detecting references that have nothing to do with the claims they're


supposed to prove. The article states that Wikipedia receives about 17,000 new articles


each month. And that is a volume that no human moderator team could conceivably all check and


cross verify and reference and checking references is a difficult topic because you need to go and


actually look at the thing that is cited and decide whether or not it actually proves the thing


that it's supposed to prove not just contains the same words or something, but whether that's


actually a credible verification of a claim being made. So here's where sphere comes in. This is an


open source system and it can check citations. It's been trained on Wikipedia citations and it


has a giant corpus of web pages that it can search across. So you get a claim to verify this is then


run through the retrieval engine, which we'll look at in a second. And the retrieval engine will


suggest citations, it will also at the same time verify whether or not the original citation actually


does support the claim being made. And if it doesn't do that, then it will suggest the best


ranking retrieved citations to the human editor. All of this results in an interface that you can


try online right now. This is not implemented as of yet in Wikipedia as far as I understand,


but that is the plan. So the interface will look like this, there's going to be an article,


for example, tulip mania, there's going to be a claim highlighted. For example, many modern


scholars feel that the mania was not as extraordinary as McKay described and argued


that there's not enough price data available to prove that tulip bulb bubble actually occurred.


That is interesting. I actually always thought that was a real thing. Now, right now the article


has citation needed. So this claim has no citation yet. And what we'll get is some suggestion, in fact,


two suggestions by the system. And we're supposed to choose which one would actually prove that


claim, we can select either one, the other or none of the above. The top one here, in fact, states,


however, many modern scholars believe that tulip fever is not so serious, nor is it a major economic