you can do is you can build a model. You can build a model you can say YIJ


Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment (Paper Explained)

that's the score that the J of the reviewer gives to the I of paper and you


know being the conference chairs these these authors here would have prime


access to that data. So what you observe is Y. Now you can say we assume this is a


combination of three things. First of all we assume that there is some sort of a


objective paper quality which is FI. This is the objective quality of the paper.


This is actually what the reviewers are trying to predict. So when the reviewer


posts the number Y into the system they're trying their best to actually


assess FI. However there is also this BJ right here and this is the bias that the


J of reviewer has in calibration. So not everyone not everyone sees the one


through ten or one through nine scale that we have in the same fashion and


therefore what's like a three to me might be a five to you. So we have to


correct somehow for this and the inclusion of this BJ factor is how we


account for that. And then lastly you have this EIJ factor right here and this


is the subjective portion of the score. So this is independent of the objective


quality of the paper. This is sort of the subjective bonus or penalty that


reviewer J gives to paper I. And their goal is going to be to figure out how do


these two numbers compare to each other. How much of the score is objective


versus subjective after we have calibrated for reviewer for general


reviewer bias for calibration bias let's say. Keep in mind this is a model this is


how we imagine the world all we observe is this Y thing right here. What we can


do is of course we can put up a linear system of all the scores right and of


all the scores because every reviewer does give more than one score in this


conference and every paper gets more than one reviewers scores. So we can put


up a linear system but it turns out this is over parameterized because you only


have as many numbers as you have these parameters right here. So the rest both


parameters they don't you don't have enough data points to assess that. Now as


much fun as over parameterized models are in deep learning they're actually not


that good if you want to estimate a linear system. So what people do they


come up with regularizers and Bayesian approaches and yada yada yada. I'll skip


all of this to just give you the numbers. So the model that these authors come up


with determines that the factors of the linear systems are as follows. This here


is the factor that goes with the FI. This one is the one that goes with the BJ.


And this one is the one that goes with the EIJ. And you see you pull out this


one and then you simply compare the number on the left to the number on the


right and you'll see they're almost exactly the same. And that means and they


formulate this here in other words 50% of a typical reviewer score is coming


from opinion that is particular to that reviewer and not shared with the other


reviewers. This figure may seem large sorry about that this figure may seem


large they say but in retrospect it's perhaps not surprising. So this is pretty


I guess this is pretty surprising to me but it is not that it is not that I


didn't expect it and I think anyone who's participated in conference peer


review would expect a number that is in approximately this range because we know


that the review process is pretty noisy and very very often individual reviewers


just kind of give weird scores that you don't understand and here's the reason


you don't understand because it's the source of them are subjective and


largely not shared by other reviewers. So what having figured that out having


figured out that about 50% of the variation is due to just subjective


feeling of a reviewer about a paper now they sort of try to validate their


findings and for that they run a simulation. So the simulation is it's a


simulated conference so we assume that each paper was scored according to the


model we've given above and we estimated the accept consistency through averaging


across a hundred thousand samples. So now they're simulating the conference


with this experiment done and they ask themselves if this is really the correct


model then we should get back we should get back a consistency of the 50%


we found above right so because above the results of the experiments were that


there was about a 50% consistency in acceptance in the experiment and now


they go and they look at all the papers and all the scores and they determine


that there is about a 50% subjectivity in scoring and now they ask themselves


do these two numbers match and they run a simulation where every reviewer has a


50% subjectivity and they ask themselves if we do if we simulate this


splitting up into two committees and then every committee agrees by


themselves do we see the numbers that we found in the experiment and the answer


is yes actually. So you can see these are conferences for a bunch of for a bunch


of different scenarios namely for different number of reviewers as you can


see here these are reviewers per committee so random means there is no


reviewer per committee committee decisions are just random and you can


see that as the accept rate of the conference goes up the accept precision


of the committees go up because they simply they they would more papers are


accepted and therefore more papers would be the same if you were to change the


committee. What we're interested in is of course the one with three reviewers


which is the most common reviewer scenario in these conferences and that's


this curve right here. So the way to read this is that for example if the


conference had an accept rate of 50% right here then we would expect a


reviewer consistency or an accept precision of 0.75 of 75% which means


that if we were to switch the reviewers for a particular or for all the papers


75% of the paper would still be the same. Remember that in our experiment only 50%


of the papers were still the same if we switched committee but the conference


also didn't have a 50% accept rate so for that we actually need to go to the


accept rate of the conference which was something like 23% right here and then


if we look that up we are at about a 60% accept precision. Now this might still be


away from the 50% we found in the experiment however the experiment had so


little data that if you calculate the bounds on what the true accept


precision was from that experiment you can determine that it was


between 38 and 64% and the exact number we got is 61% so this is still


within the bounds of what we found in the experiment. So pretty interesting


this actually means that the model they put up is a close enough approximation


to reality such that it predicts the experiments outcome and this gives us a


little bit of a this gives us a little bit validation that we're on a good


track right here so we can sort of confidently say that about half of a


reviewers decision on a particular paper essentially comes down to subjectivity


is consistent with what we found in the experiment and it'd be interesting to


see how this develops this year when we repeat the experiment. So lastly what


they were trying to figure out is well are these reviews even worth it so to


say do they actually predict how good a paper is and you know how do you measure


how good a paper is of course by the number of citations so here they define


the citation impact as the log of the number of citations and yes there is a


debate about whether citations really mean a paper is good or influential or


a blah blah blah but we don't for better or worse we don't have a different


measure right now than number of citations and it's been seven years


which is like three generations in machine learning so there is a long


enough time that these papers had to accumulate citations so do let's let's


just look at the accepted papers do the scores that the reviewers give to the


papers predict in any way whether or not the paper is going to be cited more or


less so do higher scores indicate more citations and the answer is no not at


all so here is a plot the correlation is 0.05 this is ever so slightly


statistically significant but not not really so you can like at least for this


particular conference right here there's no correlation between reviewer scores


and between reviewer scores and impact of the paper in the future it becomes a


little bit interesting when you ask specifically so because here the


question is you know is the paper novel is it correct is it well written and so


on these are not necessarily indicators of significance right if you accept the


paper to a conference only a small part of it is is it significant if you


actually ask reviewers do you think this paper will have a potentially major


impact or not you get a slightly higher correlation but also not really which


means that reviewers are kind of bad estimating whether any given paper will


have a big impact or not though to be fair for most papers the answers is


probably no by default however the interesting part is when you ask them


about their confidence in their rating and it is if I understand correctly


doesn't even matter which rating but for the rating that you give at these


conferences you have to provide a confidence score like you say okay I


think this paper is really good but I'm not very confident and if you simply


correlate the confidence scores as you can see here the average confidence


over all your sort of confidences of the paper with the impact then you do get a


slight correlation which is interesting right so the the authors here argued


that it might be that there might be something like clarity in the paper so


if a paper is written very clearly then you will also be able to understand it


better as a reviewer which makes your confidence higher but also since the


paper is more clear it means that the rest of the world will have an easier


time understanding the paper and therefore cited more often so this is a


is a good hypothesis but it's quite interesting that that the confidence in


papers it seems to predict the impact better than the actual assessment of the


impact that's astounding it's not super astounding that confidence by itself


would predict it but that it does does so more than if you directly ask people


I wonder what else we can ask okay I wonder what weird questions we can ask


that will then up correlating with the do it with the future impact like do you


like the colors of the paper do you like the pictures so these were for accepted


papers they also interestingly trace the fate of the rejected papers so they say


only 414 were presented at the final conference so they want to trace the


rejected papers and they go through a lot of work to try to figure out where


these papers ended up so they search for papers with similar titles and authors


or same titles and authors and of course this is not a perfect process but it


seems like they've been able to trace a lot of these papers to their final


destination you can see a lot of papers are discarded or some are simply posted


on archive or somewhere else of course the discarded papers you don't know if


they somehow morphed into other papers or something like this but it's still


pretty interesting pretty interesting to see though they say there are various


error sources in these plots lastly yeah here is the fate of the rejected papers


now they don't say exactly what blue and green means in this particular thing in


other plots in the same papers they differentiate for example between papers


that have been accepted somewhere else ultimately and papers that have not been


or that they have not been able to trace so this might be blue and green I'm not


sure I haven't been able to maybe I'm just stupid at reading but as you can


see if you look at their rejected papers so this is the calibrated quality score


for the rejected papers and here you can see that there is in fact a correlation


which means that for the rejected papers the assessment of the reviewers really


does correlate with how the papers will end up doing ultimately though I'm gonna


guess well if if the citation count is in here I'm gonna guess the discarded


paper must not be in here yeah sorry but the the conclusion is that for the


rejected papers reviewers can tell whether they're better or worse for the


accepted papers not so much and that's what they said at the beginning the


review process is probably good at identifying bad papers but bad at


identifying good papers and this is it's not too surprising because bad papers


you know you can find it's really easy to recognize a very poor paper but it's


it's harder to recognize really how good a paper is you know compared to other


good papers so that was the paper they give some recommendations for example


they say well maybe we should we should assess papers on on on some on different


on different criteria than we do now but they do guard they do warn against


saying we you should do away with with subjectivity all together because you


know as annoying as the subjectivity is they argue is it also guards against


sort of the the the collective dominance so it guards against sort of making


consistent mistakes so if all the like if if the entire conference for example


if the entire conference makes consistent mistakes in in some direction


then the subjectivity might counter that a little bit I'm not sure if that's a


super good argument I am generally for noisy processes over super duper rigid


ones it seems though that the conference review right now is a bit too noisy I'd


rather do away with just having three reviewers and not having this accept


barrier this is my personal opinion I would just do away with the accept


barrier all together you know you submit to a conference you get a bunch of


scores and then you have the scores like why do we need to divide papers up into


accepted and rejected or you know like it seems better to just put papers out


there and let the future let the future researchers assess them in retrospect


rather than having three random people with highly subjective opinions assess


them but yes probably a bit of noise is good in a process like this if you do a


process like this they also say well maybe we should not put put that much


value at publishing at top tier conferences now I don't know how that's


gonna work you know like whenever whenever and yeah I wish I wish as well


that we could like change the collective the collective thinking about our field


I don't I don't see that as a super easy task though in any case this was the


paper let me know your ideas let me know how you think this year's experiment is


gonna turn out like are we gonna find more subjectivity are we gonna find less


how much disagreement do you think we're gonna find this it's gonna be


[ML News] DeepMind does Nowcasting | The Guardian's shady reporting | AI finishes Beethoven's 10th

cut my hair, but not the beard. I have a giant cold sore here. It just looks weird without


the beard. I was just gonna wait. Well, we'll Yeah, intro. DeepMind can predict rain better


than anyone else. The Guardian is not so really truthful about truthful language models. And


an AI finishes Beethoven's 10th Symphony. Welcome to ML news. It's Monday. For centuries


upon centuries, millennia upon millennia, humans have shaken their fist at the sky for


the rain which they could not predict. But while the gods of the heavens curse us with


the falling precipitation, the gods of the earth, namely DeepMind have now blessed us


with a system that can tell us when and where it's going to rain. DeepMind has been looking


into what's called now casting, which is an area of weather prediction that concerns just


the next one to two hours. The reason being that apparently longer term forecasting can


be done pretty accurately by sort of modeling the global weather, seeing how stuff moves,


considering the physics and blah, blah, blah, but very short term predictions are not as


accurate as we would like them to be. They've published this in a paper in nature because


where else would DeepMind publish? And it's actually a pretty interesting reads, they


cite the availability of high quality data, at least in the UK, where radar data is available


at very high resolution, and the lack of current systems that work well. Now instead of directly


predicting their model is a generative model. And from the paper, it looks like it's sort


of a GAN with a bunch of GAN losses. So there is a temporal discriminator that discriminates


between real and fake, I guess, temporal rollouts, there is a spatial discriminator, and there's


sort of a regularity loss as well. So essentially, what they do is they take a context of 20


minutes of radar data. And from that they generate how the radar data looks about two


hours ahead. And as you can see, this looks pretty good. So on the top left, you have


the target on the top right, you have the DeepMind system. And on the bottom, you have


two baselines, you can see that the DeepMind system is quite a bit more accurate. And not


only is it more accurate as rated by the metrics and also by human climatologists or weather


people, I don't know what exists in this case. And while the DeepMind system is more accurate


in terms of metrics, and in terms of humans rating it, DeepMind also advocates for a more


impact based metrics. For example, they highlight that the prediction of heavy precipitation


at long lead times remains difficult for all approaches. And this is one of the crucial


events that you would like to predict. So the paper advocates that maybe we should pay


more attention to the things that actually impact such things as farming or air travel,


or deciding whether or not you can hold an event outdoors. Along with the paper, they


do provide the data set and also a snapshot of the trained model. There's a collab where


you can download the data set and try out the model. So no longer do you need to have


a wet head simply go here and see whether or not it's going to rain in the next hour.


The Guardian has an opinion piece by John Norton that says the truth about artificial


intelligence, it isn't that honest tests of natural language processing models show that


the bigger they are, the bigger liars they are, should we be worried? Now, isn't this


exactly what I predicted? I reported on this in last ML news, I made even a dedicated video


about this benchmark called truthful qA, which is where the authors create a data set specifically


designed to trick these language models going as far as throwing out questions that the


language models get right and defining the word truthful in a way that if you answer


complete garbage, it counts as truthful and therefore, the smaller models are better because


they're just worse. Now, if you get the impression that one should mention these things when


discussing this data set, then you'd be right. And I advocated for the same thing. I said,


if someone gives this as an example of how bad large language models are, and doesn't


explicitly mention these things, they either don't know or they want to deceive you. Well,


enter john Norton, who writes an entire opinion piece about this article. So given that he


writes an entire opinion piece, the possibility that he hasn't read the paper is out. The


only thing that comes even a little bit close to mentioning the way that a data set was


created is this sentence, they composed questions that some humans would answer falsely due


to a false belief or misconception. Really, really, do you, dear viewer, do you feel that


is an adequate characterization of this benchmark? And do you feel that giving only this sentence


draws the correct conclusion for people? I mean, it's not wrong, they did this, it just


leaves out all the other stuff that you would need to know. And why does it leave out all


the other stuff? Because of course, john wants to make an argument, and the argument will


completely fall apart if you include this other stuff. And this is how science reporting


goes when you have a narrative already in mind, it goes from a paper that does describe


the complete process, but uses words such as truthful in very weird ways, and is already


framed in a particular manner to the Twitter announcements of the authors, which hide all


of these facts in very specific wording in somewhere down the thread to the more popular


hubs in the AI space completely leaving away these details, and then to the mainstream


media that just picks up the talking points and writes big articles about how bad these


things are. Good job, everyone. Now, if only there were some kind of independent news source


that you could get your machine learning news from that never ever, ever makes mistakes.


Where could one find that?


Moving on, there is an interesting new paper on archive that's called stochastic training


is not necessary for generalization that argues that if you tune full batch gradient correctly,


and if you regularize correctly, and all of these kinds of things, then you can achieve


the same performance with full batch gradient descent, then you can with SGD. And this casts


doubt on a lot of theoretical explanations of why neural networks generalize so well,


because many of these rely on the stochasticity of SGD. It's long been believed that the stochasticity


plays some kind of a role in the generalization capabilities. And at least in part, this paper


provides evidence that this might not be fully the case. However, that being said, you do


need to regularize the network. So you do need to bring some of the implicit regularization


that SGD appears to do through stochasticity into the world of explicit regularization.


If you don't want the stochasticity in there, this appears to be true with and without data


augmentation. And the paper argues that the community has essentially just spent a long


time optimizing stochastic optimizers and hyper parameters and hasn't put that much


effort into full batch methods. If this is of interest to you give this paper a read.


Google AI releases the efficient partitioning of road networks. So this is a method to partition


road networks. Because if you simply look at a road network and try to do planning,


it quickly becomes ginormous. If you just consider your own city, then already, that's


a pretty big graph, if you really model all the connections, and then you consider a country


you consider a continent, it quickly becomes so huge that something like a Dijkstra algorithm


cannot plan efficiently anymore. So what you have to do is you have to partition and they


give the example of state island, which is an island in New York City. And while state


island has a lot of roads and the surrounding city has a lot of roads, the access between


the city and state island is limited to four or five different bridges. So a smart algorithm


would sort of clump state island into very few nodes. And then you can essentially plan


on these super nodes until you get to state island and then inside state island, you can


plan locally. This relies on the fact that our road networks very often are comprised


of large interconnections between clusters of local roads. And in order to do this, they


leverage random walks. So they simply start from some point on the map and they do random


walks on the map. And the idea is that if you have super duper connected networks like


inside state and island, then the random walks are probably going to stay in that area as


they walk because the amount of connections inside the area is just so much larger, and


they're not going to traverse very often these interconnections between the clusters. So


therefore, using random walks, you can figure out what are the clusters that are tightly


connected and what are the clusters that are only loosely connected and therefore you can


partition the graph. This is then refined using some flow algorithms. And at the end,


we all get Google Maps. Thank you. There is a paper to go along with it. Have a read if


that is of interest to you. Facebook AI research releases mini hack a new sandbox for open


ended reinforcement learning. This is an iteration on the net hack learning environment, which


we've reported previously is available. Net hack is this game where you're in a dungeon


and you need to do certain things, battle certain things and so on. And the cool thing


is that it's entirely described in kind of an ASCII way. So on the left here, you see


the way that players or level creators would design levels and then add items and certain


effects to it. Now the net hack game is very difficult game. And if you do reinforcement


learning inside the game, there are a lot of tasks, there are a lot of things to do.


And there is essentially just this one game. So mini hack is an environment where you can


create small parts of the game, different sub levels, very simple tasks to test the


individual abilities of agents. So you could, for example, make a mini level where it's


just about avoiding obstacles, or you could make another mini level where it's simply


about fighting opponents. So essentially, it's a level editor for the net learning environment.


Pretty cool. Give it a try. I do releases Plato XL, the world's first 11 billion parameter


pre trained dialogue generation model. Now, whenever you say the world's first, you just


have to make whatever comes very specific, then you're always the world's first. Like


even if there were a 12 billion parameter pre trained dialogue generation model, Plato


XL would still be the world's first 11 billion parameter pre trained dialogue generation


model. However, this is really so far the biggest model that is specifically made for


dialogue. It's available in English and Chinese. And it is specifically trained to do long


dialogue that keeps the context alive of what's talked about. Also Baidu says that they will


release the source code together with the English model on GitHub soon. Next web news


writes Beethoven never finished his 10th symphony computer scientists just did. This is a description


of how a team of computer scientists and music scholars went about finishing Beethoven's


10th symphony. So the ninth symphony concluded with the Ode to Joy, they said, but the 10th


symphony is unfinished. There are some scribbles by Beethoven, some ideas, but it's by no means


a finished piece of work. So the article details how the team went about recreating something


that Beethoven might have written. And this is the important part to get right here. They


do not claim that what they produce is Beethoven's 10th symphony as Beethoven would have written


it they say that this is given the ideas something that Beethoven might conceivably have come


up with. Now that being said, there is a lot of iterations here. There's a lot of hand


engineering of course. So rather than this being fully AI generated, so I would rather


call it a computer human collaboration to come up with something that plausibly could


have happened had Beethoven lived for a bit longer. The article is fairly long, but it


concludes with an excerpt from what these people created. That sounds like music, correct.


So it seems like a cool practical applications of some of the techniques, the combination


of AI and art is more and more explored. And it's good to see that music is not an exception


here. Speaking of AI and art, the Smithsonian magazine writes, did Peter Paul Rubens really


paint Samson and Delilah AI analysis renews doubts over the authenticity of a star painting


in the London National Galleries collection, right. So there's this painting by a painter,


I have no clue about art, I'm very sorry. But apparently the painting has been painted


at some point and then went missing for a while and then it reappeared. And there is


an entire debate about whether or not reappeared painting is in fact the original painting


or a fake. And there is this company called art recognition, which supposedly can give


you a report about whether or not a given painting is actually from a given painter


or not. And when this company analyzed the painting, the algorithm reported a 91.78%


probability that Samson and Delilah was painted by someone other than Rubens. So the company


claims they have had quite a lot of successes when they assessed non disputed works with


the algorithm being generally very correct in these assessments. So given this track


record, the statement that this painting is probably fake is quite a bit of a shakeup.


Now now now I have many questions about this. Like why does this need seven days to generate


a report? Do these people actually go out and collect training data once you submit


your thing? I don't know. Also, these systems got to be like super duper vulnerable to something


like adversarial examples, they give you like a certificate of authenticity. Now I'm going


to guess this is like a CNN and the CNN is trained on a bunch of paintings of that painter,


then you get some sort of a closeness estimate. Now, are there negative samples that this


is trained at? Is this a one class SVM? I don't know. And actually found anything in


the FAQ about how exactly this works. Apparently, the entire service is just digital. And you


don't actually need the painting itself. And I know a lot of these scholars, they look


at the paint strokes themselves and the thicknesses and x rays and whatnot to determine if art


is authentic or not. Now, I have no doubt that something like this might actually work


and might actually work better than human art experts can assess this. But at the same


time, there are a lot of vulnerabilities in these systems. And I also wouldn't trust them.


Now would I trust them more than human experts? Not sure. I think what is safe to say is that


simply because this company says this is probably fake, it probably won't convince anyone in


the art world to change their minds about this painting. But interesting to know this


exists.


Artie writes AI driven community surveillance us cops reportedly using invasive tool to


grab suspect social media, Pornhub and Tinder data. This report is about a company called


Shadow Dragon that produces tools that scrape social media and pull together all kinds of


information about individual people. And they sell this to law enforcement such that essentially


anything you do across social media is neatly pulled together and analyzed in one place.


This can then be combined with other surveillance mechanisms such as facial recognition from


surveillance and all your data from various government databases. And it could technically


be used to do predictive policing, which is a very controversial practice where you don't


react to crime, but you try to react to pre crime, which gives it a sort of dystopian


feeling. The company's founder says the company disagrees with predictive policing and does


not build products with predictive capabilities or even suggestions. However, also their website


praises the product for being able to predict violence. So, yeah, another question is where


exactly Shadow Dragon has all this data from they themselves claim they do not intercept


any private chats, and they do not access anything that's proprietary or private, but


simply scrape information from public websites. And again, that is highly disputed. Now, even


if they only collect data from public websites, it's still quite worrisome to see that police


are using these kind of systems. Of course, if you are a suspect police has every opportunity


to go look at all of your social media all across the web and cross reference that but


this is now being done in an automated fashion that is available to search and yes, train


predictive models on top of it. Now whether or not that's a good development, I leave


that up to you. But a good recommendation is that simply assume that all of your activity


online is being carried together at some place and just put all into one neat package. So


while in the previous life, you could be one kind of person on Twitter and another kind


of person on LinkedIn in the future, these things are going to morph together more and


more right now it's simply for law enforcement and the government. But given that these products


seem to exist, you can expect that to be more the case in general in the future. So now


you have the opportunity, do you want to behave more professionally on Twitter? Or do you


want to just spew random opinions around on LinkedIn, I know what I'm gonna do. I'll also


link a more in depth article by the intercept about shadow dragon and its connections to


law enforcement if you're into that. All right, helpful libraries, we have a lot of helpful


libraries and data sets this week, like so much help on the internet. It's crazy. I'm


suffocating from helpful libraries. I can't labor anymore. That being said, you should


totally check out hugging faces infinity, which is a Docker container that you can deploy


yourself and that brings inference of transformers down to a millisecond. So if you read more


into this, apparently it's about three milliseconds for CPU based transformers like Bert and Roberta


and one millisecond if you host them on GPU. Now this is pretty massive. It represents


about a 10 x improvement over previous attempts at speeding up these transformers and you


can deploy this on premise fits neatly within a Docker container. Now infinity is in a closed


beta right now, but I guess they're going to release it at some point. I don't know


there is a website but it doesn't say a whole lot of things about it. But I guess being


in beta this is bound to develop further. If you are interested, click the request trial


button and see what happens. Next up the text based NP enrichment tasks text base, text


based, not sure which one it is I'm gonna I'm gonna guess text based. So this is a data


set for NLP. And by that I mean rather how NLP used to be before deep learning where


every noun phrase is sort of annotated with all the possible cross references that exist


in the text. So for example, the sentence here, Iranian student protesters face expulsion


would be annotated in the following way. Iranian student protesters would be annotated at Amir


Kabir University, it would also be annotated with against Ahmadinejad and face expulsion


would be annotated with expulsion of 54 students expulsion by university Chancellor Ali Reza


Rahai or expulsion from Amir Kabir University. The goal of the data set is to do these annotations


exhaustively, which I'm going to guess was a lot of work, but they do end up with 5497


documents that are exhaustively annotated with all possible links between noun phrases


in each document. So pretty cool. If you're more into old school NLP, definitely give


this a try. If you are into new school NLP, you should probably learn a bit about old


school NLP. Next there is TROCR transformer based optical character recognition with pre


trained models by Microsoft along with code. This is a new OCR method that uses transformers


code is available, give it a try. Kau Kore, which is joint work of Google research and


collaborators from Japan's National Institute of Informatics and the University of Cambridge


released this data set right here of Japanese art depicting faces. So they wonder whether


or not they can teach machines to recognize facial depictions in Japanese art and classify


them into various categories. So the data set is created from a larger Japanese art


data set by cropping out all of the faces and then manually labeling them. The labels


are things such as the social status, which is divided into noble warrior incarnation,


which is a depiction of a god or goddess and a commoner, which is I guess the rest of us.


You can also train GANs on these data sets. And it seems to be just a pretty cool data


set for doing research again, intersection of AI and art. This could be like a theme


for today. Raft is a data set of real world annotated few shot tasks. This is a data set


where both the task itself and the examples are given in natural language. For example,


the task here is the data set is a list of institutions that have contributed papers


data data data data data. The goal is to classify these institutions into one of three categories


university company or research Institute 50 labeled examples are provided and then there


are a bunch of labeled examples but not too many. Thus the name few shot tasks. So this


could be pretty cool because especially it has a lot of practical applications. If you


can specify the task in natural language and you don't need a whole lot of examples for


the model to learn a task a lot of new possibilities in applying NLP open up there is a paper and


a leaderboard if you want to give it a try. The next helpful thing is a data set Edgar


data set is a data set of financial texts. Edgar is a database where all the public companies


have to send in their annual reports and Edgar corpus is a data set of that they do provide


a script with which to mine the Edgar database and they do train a set of word vectors which


for specific tasks in finance perform much better than standard glove word vectors. So


if you ever wanted a corpus of a giant amount of text that says absolutely nothing important


of any informational value because all of these finance departments basically just cover


their own behind. There you go. The next data set is pass an image net replacement for self


supervised pre training without humans. The pitch is they have 1.4 million images 1.4


million of them are CC by licensed and there are absolutely zero humans in the data set.


Not only aren't there any depictions of humans, there are also no license plates or other


personally identifiable information. The catch is this data set comes without labels so you


cannot train your classic computer vision image classification task but it is supposed


to be another data set that you can use for pre training your models without having to


worry about there being some personally identifiable information in there and also without having


to worry about the licensing of the pictures that are in the data set. Now are people going


to replace image net by this one or are people simply going to add this data to their image


net data and therefore the problem simply remain well you take a wild guess which one


of those two things is going to happen. In any case the data set is available to download


have fun. And lastly torch data by Pytorch is a very unstable prototype but it is primitives


in order to build data loaders in order to make data loading from various sources more


effective. So if data loading is your bottleneck and the standard data loaders don't do the


job, maybe give this a try. The API's might break but you know that's life. Last news


for today Engadget writes Samsung hopes to copy and paste the brain to 3d chip networks.


Essentially their idea is to stick a bunch of electrodes in there stimulate the neurons


see how the neurons stimulate other neurons from this you can figure out which neurons


are connected to each other and how strong and then you can simply map that connection


pattern onto a neuromorphic chip. Now this might actually be an interesting way of getting


a neural network with the general connection pattern of the human brain like the sparsity


pattern or how exactly the things are connected. So it might be a neat architectural investigation


into the human brain. However, the article also writes the move could serve as a shortcut


to artificial intelligence systems that behave like real brains, including the flexibility


to learn new concepts and adapt to changing conditions. You might even see fully autonomous


machines with true cognition according to the researchers. Nah, nah. Simply because


you map out the connection pattern doesn't mean at all that you will get any sort of


brain like activity connection pattern between neurons is only one of many, many, many things


that is going on in the brain, especially things like learning require forming of new


connections dynamically strengthening connections or strengthening synapses inhibiting expression


of genes that lead to faster or slower re uptake of synaptic material. And all of this


is simply not captured by simply mapping out the connection pattern. Forgive me, but no,


you're probably not going to see fully autonomous machines with true cognition simply because


you can map the brain's connections. Now these things are supposed to run on neuromorphic


chips, which means they will have some of these additional abilities, but still highly


doubtful. That was it for this week's news. So much stuff happening if you have something


interesting that's happening in your life. And if it is in any way related to machine


learning, let me know we have no standards here at ML news. Anything goes. I'll see you


Dynamic Inference with Neural Interpreters (w/ author interview)

how do you prevent all the signatures from collapsing on each other, right? Because that's


a very nice way to cheat. Right? I mean, you know what neural networks like to do, right? They like


to cheat. Hi there, today, we'll look at dynamic inference with neural interpreters by Walid


Gondal, Naseem Rahman, and others. So this is again a paper where I will interview the authors


of the paper. In fact, we had three of them, the two first authors and Francesco Locutello as well.


So that's going to happen in, I want to say, 10 minutes or so. If you already feel comfortable


with this paper, you please skip ahead to the interview part. It is by far the best part


of this video. I will give a very brief introduction to this paper by myself right now,


so everyone knows what's going on. We'll go over the architecture again in the interview,


just because I want to get it from the authors as well. So briefly, this paper takes a new look


at how we build neural networks. Specifically, it supposes this architecture called neural


interpreters. And these neural interpreters, as the title says, they're able to do something that


the authors call dynamic inference. I want to jump right into this, what is a neural interpreter?


Now, what we're going to see in this model is a lot of things, a lot of maybe things that are


familiar, are going to be rephrased in sort of a new terminology. And that is because the thinking


that leads to this model is different. The thinking is something like, well, if I process data,


I'm going to do so in a series of abstract, let's say functions, a series of abstract


functional modular components. And what I do is I compose these modular components in my head or


in the computer when I program it in order to achieve a certain goal. And we're trying to


replicate this in a neural model, this is going to look a lot like a transformer model, but it has a


few intricacies that make it particularly suitable to this to this formulation. And this model here,


it's not going to be like the next state of the art model on ImageNet or anything like this.


But what it is going to be able to do is going to be able to generalize to unseen data, especially


in terms of data that needs some sort of logic processing, abstract reasoning or anything like


this, it's going to be able to do this. And it's going to be able to handle things like distribution


shifts, or just very little training data much better than a regular model that I just train


on a training set. Of course, this model is trained on a training set as well. But


you know what I mean. So we're going to focus on models that abstractly work a little bit like


transformers, in that the input is going to be some sequence of tokens, like a set of tokens.


In this case, it's a set of visual tokens, which are going to be converted to visual embeddings.


But you can imagine any sort of token based input such as text, or little snippets of sound or


anything like this, whatever you would do to a transformer currently, that's the same input


that this model gets. The model is made up of this series of scripts, what they call scripts.


And these scripts are nothing more than just blocks of computation. A regular transformer


also has blocks of computation. This model does as well, nothing special here. However, inside of a


script, now the differences start. So inside of a script where a regular transformer will still will


essentially only have what you see on the right here, it will have a stack of attention layers,


and multi layer perceptron layers interleaved with each other. This script right here,


it modularizes this into what is called functions. So each function by itself, for example, here you


see f three consists of a bunch of these layers. Yet, there is also f two, and f two will consist


of different ones of these layers. In fact, they do share some parameters, but we'll get to that


in a minute. So f one, f two, and f three, they are all independent functions independent from


each other. And any token can be routed to these functions independently. So the idea is that we


take the tokens, and the tokens are routed to either one or more or multiple of these functions,


but not all of them. The goal here is sparsity. So each token is routed to a few functions, each


function takes as input all the tokens that are routed to it, it internally runs these layers


right here. And then it outputs the tokens again. And what happens then again is special in that in


a transformer, we will just go on to the next layer with the next set of parameterized functions.


However, in this case, we apply the same layer again, you can see here at the top and the bottom


are the same. So we do the same thing with the output, we again evaluated, we route it to different


functions. So the same token can first pass maybe through functions one and two, and then the output


is combined from them. And then in the next step, we apply the same layer again. And maybe then the


token is routed somewhere else, maybe then the token is again routed to function two, but now


also to function three instead of function one. So this is supposed to represent this, first of all,


modularity in that we have different functions, but also composability in that we repeatedly apply


the same layer. So weights, the computation is shared, this is essentially a recurrent layer


right here. And you can imagine that there's not only two applications. In fact, I think they go


up to eight applications of the same function within one script block, yet the functions,


they're modular, they're independent from each other. And the between each recurrent step,


a routing table is computed a new note that this is not the same routing as happens in the attention


layer, the attention layer here is part of this. So this attention layer here would be, for example,


right here, it would actually be a part of a function. So the attention layer would only


consider tokens that have initially been routed to function three. So now the question is,


how does this routing work? And I also said a little bit that code is shared or weights are


shared between the functions, I want to tackle the second thing first, the way this is implemented,


and we come to this all in the interview in more detail. And and I think what I understood


is not necessary what I'm telling right now, what I'm going to tell namely that these function one


function two and function three, they do share some of their parameters, but not all of them.


In fact, the authors here imagine a scheme where each function has a global set of parameters,


let's call them w. And then so function gets w, which is a global set of parameters,


it gets the input x, right, which is the token that is currently being processed. And it also


gets c, which is the code of the function. So this, this here, the authors call the code.


And the idea is that this thing right here, that's shared across all the functions,


there are parameters that are shared. And then there are parameters that are just individual


to each function. And the idea is, as I said, the c is the code. And this w right here,


you can see as the interpreter, interpreter. So the this is in analogy to you have sort of


an interpreter, like the Python interpreter that is able that has code by itself, right,


Python is written in C, Python is written in C. So there is shared code among all Python programs.


But then obviously, there is also individual code for each Python program. They see these functions


in the same way, they have a set of learnable parameters that are across all the functions,


which are kind of global interpreter parameters. And then there are local parameters, which they


call C, which they call the code, which are also learnable parameters, they're nothing else,


but they're just localized to each function. And that's what makes one function different


from another. Every function here has learnable parameters with the code, which is individual


teach function, as I said, is not necessary. Like you could just imagine that these be completely


independent neural modules, they will have independent attention weights, they will have


independent multi layer perceptron weights, this will still be the same model. But it's just in


sort of an homage to this thinking of, of dynamic computation that the authors here build in this


weight sharing scheme. So the second thing each function learns by itself, it's what they call an


and like an S variable, I think I believe it is S. And this is what determines which token is


routed where this is a little bit like in the attention mechanism, every token exposes a key and


every token exposes a query, and then keys and queries are routed according to inner product.


Here, every token is run through essentially an embedding layer, a linear layer, which determines


to which function is going to be routed. Each function has, as I said, these S variables,


which are just vectors. And then we compare inner products. So again, each token goes through this


type inference function, which is a stack of neural network layers, we obtain an embedding T,


we look at the inner product between T and what they call S, S is an is a vector for each function,


and that determines what's routed where right, this is an exponential, this is a normalization.


So this is a Softmax routing based. So if you have function one here, sorry, function one,


if you have function two, function three, and you have your tokens, right, each token is sent


through the same neural network to determine its what they call type. Essentially, it's an embedding,


right, this is type one, type two, this is the type of token three, and so on. Each function


will already have as a learned or fixed parameter, right, they, in their model, they suggest one can


learn these as well. But in fact, they say it works the same as you know, when you just leave


them fixed. So they initialize uniformly initialize these S variables, this is S of function one, S


of function two, S of function three, S stands for signature, I believe. So the idea here is that the


function exposes signature that tells which variables are even allowed to go in the function,


and the tokens express a type, and the type and the signature needs to match in order for a token


to be routed there. And the matching is done obviously via inner product. So for example,


these two are certainly going to be routed together because their inner product is going to be large.


As I said, this is not too different from like attention based routing, except that this is


dynamically computed. However, this here is either learned or fixed. So in attention, this would also


be dynamically computed. But since the functions aren't dynamic, this, yeah, this could be a good


extension somehow thinking that the functions themselves could be sort of dynamic. But then,


I almost believe we'd end up kind of at the classic attention mechanism, because maybe you want to


compute the functions from the sequence here itself, right, the code and the routing, which would


essentially be the keys and maybe the values, not super sure. But yeah, so another idea I had just


after the interview is that they say something like, you know, these these signatures right here,


if we want to make them learnable as well, the neural network could sort of cheat and


collapse them all together, so that every token is routed everywhere, which I think due to the


nature of gradient based learning would be advantageous at training time. But it's not


necessarily what you want for generalization, you would like to preserve the sparsity aspect


for generalization. And they talked about having repulsion losses between these. Maybe


one could also take ideas from, from VQ, like from vector quantized VAEs or so, they do have the same


problem, right, that their codebook needs to be not collapsing. And I think that the quantization


procedures that they have there, as well as the methods they use to construct the codebook could


be instructive for here, rather than just leaving the things at sort of the uniform initialization.


In any case, this determines how the things are routed to the individual functions. It by the way,


also determines how they are combined again, which is something we didn't talk about in the interview.


So the routing table determines how the tokens are routed. But then also, the combination happens


in a sort of like the reverse routing table, obviously, because you only want to get output


from where you got input, I believe at least that's what happened. Yeah, I think so. Otherwise,


I might be wrong here, the attention mechanism inside of the function happens as a regular


attention mechanism. It is obviously gated by these C values right here. And those are simply


the values we've just computed these routing values. This essentially just means that every


function only has access to the tokens that had been routed to that particular function,


then there is an MLP after the attention layer. And the MLP is represented here. Again, these are all


these modlin functions, these are linear layers that as an additional input get this code. And


the code is essentially the parameters that are special, as we discussed before. So they have a


formula here for these modlin layers. And the modlin are also used in the attention layers in


order to compute keys, queries and values. So you can see right here modlin is computed out of an


input and a code vector. It is computed in the following way. It's in a classic fashion, it's


w something plus b, so it's a linear layer. However, w and b are shared across all of the


functions in the same script. Then the something right here is the different part. So there's the


input, but it's element wise multiplied by the okay, this is layer norm. And then this thing


right here is the code projected through this linear layer. The wc linear layer is also shared


across functions, but the code is obviously different. You can see what differentiates one


function from the other one is simply the code that is input to the function. Everything else is


shared across all the functions in the same script. This is, again, not something that is,


I believe, necessary to the splitting up of the functions. You can imagine every function


having its own independent set of parameters, but it does sort of goes one step more into


this directions that the authors want to go right here. So then you can build up things. So you can


build up MLPs as a combination of these mod linear layers, you can build up mod attention layers,


which is simply attention, but instead of linear layers to compute keys, squares and values,


you use these modlin layers that get the code. So even in the attention, most of the parameters


are shared between the functions in the same script, then you can build all kinds of things


from these. So a line of code is essentially one of these blocks that has an attention


followed by an MLP, which we saw like on the right side. So this, this thing here,


that's a line of code, then you get from lines of code, you can determine what is the here the


interpreter, which is many lines of code after one another, then you can iterate these functions,


as we said, we apply the same function multiple times. So the interpreter now is applied multiple


times, right, but always, always, we always do the type inference, the routing in between. And then


obviously, you can apply that many times over and that will be the whole model. So these are the


different scripts that you apply. So here inside a script, function iterations enable sharing of


computational units in depth, increasing the number of function iterations can increase depth


without increasing the number of parameters. So it's a little bit of a recurrent scheme inside of


the big blocks that make up the entire model. Right, that's the entire model right here. I hope


you could sort of follow we go through it again in the interview. So I want to keep this really


brief right here, I want to go down a little bit to the experiments. Right now they do experiments


on learning fuzzy Boolean expressions. So they have these Boolean formulas and not and or these


are fuzzy. So they deal with real numbers. And on the other hand, they also look at actual real data.


So there's image classification, as well as these abstract reasoning matrices over here,


they make some interesting discoveries. For example, they can learn the Boolean formulas


by only adjusting the routing parameters. Oh, I've scrolled too far. So by only adjusting


the routing parameters, they can learn these Boolean formulas and generalize to new ones.


I said this wrong, they learn the Boolean expression task by training the model regularly,


then they can generalize, they can transfer learn to new ones by only adjusting the routing


parameters, which kind of tells them that the function modules they learned, they are in some


way kind of universal, they, they represent these Boolean functions on a more fundamental level,


because you only have to adjust the routing in order to make them adapt to new things.


The second thing they look at sort of how samples propagate through the network, I'm gonna I'm gonna


ask them in the interview about this graphic right here. They look at the inferred type embeddings


and see that really, they do not all collapse to the same thing, right, as you can see, right here,


they also look at how this develops in function iteration. So they say types are more clustered


in the later function iteration, suggesting that the input set elements gradually develop a type


as they progress through the network. They do a lot of these kind of little experiments right here


on toy data on real data, they even look whether they can drop out functions or add in functions


after they've trained. Yeah, so that's, that's what they did. But I will ask them all of this


in the interview. So I again, I don't want to go too much here. In essence, I have to say I did


like this paper, it is quite hard to develop a model like this and then design experiments that


really validate that what you think is happening is happening. And I think the authors did a good


job. I'm still not 100% convinced, but I think that's never going to be possible. I think the


authors would would agree with that statement. It is hard to peek into these models. They do


test on real data right here on against some some baselines, you can see the results are kind of


all over the place. Their models is ahead a lot of the times, not all the time, though. So I think


this, the problems are still open. And these models are still out there to be developed. If you


have ideas more than happy that you play around with it. I didn't ask them if the code was available,


actually, I'm going to do that. And if it's available, I link it in the description. If it's


not available, then I'm sorry, you'll just have to to guess around. Yeah, that was it. Now over to


the interview. Welcome, everyone here back. Very, very fortunate today to not be joined by one


author, but actually three I have with me today. Walid Gondal, Nassim Rahman, Nassim Rahman, and


Francesco Locatello, that all work together on the dynamic inference with neural interpreters


paper. Welcome, everyone. Thank you so much for being right here. Thank you for having us.


Yeah, it's it's, it's really cool. This the paper, because I think it takes maybe a little bit of a


first principles approach to the whole idea of of computation, it is really framed in terms of


computation, it is framed in terms of I want to have different modules that do different things,


then it's somehow being connected with neural networks and so on. Can I can I ask you what was


your? What was your motivating thought behind all of this? Like, how did you? How did you get to it?


Did you sit down and say, Well, I want to build a computer like neural network? Or


what were the kind of leading thoughts that you would tackle such a problem in this way?


Okay, so I guess I'll start maybe so. So, you know, like, of course, I've been in Bernard's group for


I think, two years or more, and also Joshua and, you know, the thing that they've both been very


excited about is, like has to do with principles of causal mechanisms, that's like, you know,


like that, you know, you can decompose the world as a system of modules that kind of interact with


each other. Right? So that was kind of like, always, at the back of our heads, right? And then


we thought, Okay, look, this is actually, you know, the intuition there is actually not very


different from what we do as programmers all day, right? I mean, it's kind of we type functions,


we use functions, and then we kind of recompose stuff. And it's, it's maybe it's not as different


as we think, like, these two things are maybe not very different. And, and then, of course,


since we're deep learners, you know, like, how do we mash these three things in and make something


cool out of it? So I think that was kind of the, I think the initial motivation that kind of drove


us to it. I don't know. I mean, I mean, I guess we had like this chat, I think, like, and then we


were like, Okay, this does not sound too shabby. Like, yeah. Yeah. And I just I have to say, like,


I read the title, I read the abstract. And I thought to myself, like, this is such a banjo


paper. Like, this is like Yoshua banjo has all over it. And then I and only then I looked at the


author list. And I was like, of course, it's it's it is astounding. So maybe, you know, I want to I


want to get with I've, I've, you know, by the time people watch this, I will have done a little intro,


but I maybe want to go with you again, just briefly over sort of the main structure of all of


this. So this your your method essentially takes the form of, let's say, what we would know nowadays


as a transformer network just broadly, right. So you have a bunch of like input tokens, like down


here. In this case, you do multi, you can do multitask classification. That's why you have


like multiple CLS tokens. But you could imagine anything, let's say a transformer network could


do as long as it sort of takes a set of inputs, and it outputs like a set of outputs. And every


every layer as in a transformer will take a set of embeddings, presumably, and output again,


a set of embeddings. So, so far, so good. But then here is the first thing that's that's kind


of different. So you your your model, whereas a transformer is made up of multiple, I guess they


call them transformer blocks. Your model is made up of multiple of these scripts. What is kind of


the high level idea? What is a script? So a script is I mean, you can think of it as essentially like


a group of layers. Like the core of the architecture truly are like the functions and


and the objects because we I really like this this analogy with programming languages where


you have objects that are processed by functions. And then when you have a bunch of functions,


you stack them together into a script. And then you can compose a hierarchy of scripts. So the


this split in scripts is just a convenient way to share parameters in different ways across the


architecture, because maybe the early layers want to do things that are slightly different than the


later layers. And since the all the functions within a script share certain parameters,


then then you can have a differentiation this way. Okay, so the this is this is simply because this


is the script three is independent from script to script two is independent from script one and so on.


Yet within the script, certain stuff is shared. Okay, so it is okay. Then within a script, right,


you have these what you call functions. And we see right here, there are three functions in this


example, they appear multiple times. They they're sort of functions next to each other. There are


also functions one after another. Can you maybe comment a little bit what are what are functions


and how do you sort of stack and combine them? Right, so we could think of functions as like


really the fundamental like, in this case, like a building block of like, it's an abstraction,


of course, right, but it's kind of like a building block, it's kind of a unit of computation that can


be shared. And how is it shared, it shared along depth, right? So you see, like, you can have a


token that goes to f three, and then it may come back to f three again, or it may go to f two,


depending on how it's routed. Right. So so you know, like, there's this reuse of parameters,


like a dynamic reuse of parameters, and you actually learn how to reuse the parameters.


Yeah, the whole function abstraction is kind of is what enables that. Right. So along with it's


kind of like, like, horizontally, it's kind of essentially a way of adding more capacity,


if you can think of it that way, right? And horizontally, sorry, vertically, it's kind of


like, adding, yeah, I mean, it's adding depth, of course, but like, it's adding more computation,


more computation, exactly. And it's, and we have some actually pretty fun results downstairs,


where we actually show that this amount of computation is kind of flexible. Yeah, even at


test time. And yeah, you split this up right here on the right side. So this would be this would be


sort of what a function looks like internally. In fact, you have a stack here, this stack in depth,


do I see this correctly, that sort of the front, let's say that the front of the stack is the


front thing of the stack corresponds to the pink function. And the second one would correspond to


the blue function. And the last one corresponds to the green function. So each function essentially


is a stack of neural network layers abstractly spoken.


Yes, parameters, right, because this distinction is modulated with the code of the function. Again,


if you if you follow this programming language interpretation, that you have the code that


determines what the function does. And then to to make it easy, then all the functions actually


share parameters, but then they are differentiated by the code and their signature, of course, that


determines what they are able to digest. Exactly. Yeah, that's exactly that. Those are two things


that so you just said, which is one of the questions I actually had about, you know, where


exactly are parameters shared? Because if anything, this this method seems like a sort of an intricate


way of sharing parameters between things, right? You just said there are parameters shared between


all of the functions? Yes. Is that correct? Okay. Now, what differentiates the functions from each


other is what you call their code and their signature. So if I have any sort of talk, let's


let's take one of these these tokens right here. So what do I do? This is like x i my token i,


and I have a bunch of functions right here. Every function is characterized by its signature and


its code and its signature is you call it s or some s, I believe. So s j and s determines whether


or not the token is even routed to the function kind of right. So s is about what the function is


able to process. And then you'd have to look at what's the type of your object. And then if the


type matches what the function can process, then it can be processed. So you run a function through


this type inference function, which I'm going to guess is like a linear layer or something like


this to be there. Or multiple layers, you get out an embedding, you calculate the inner product


between this thing and whatever this signature is per function. Now, I understand correctly that


your framework suggests this could be learned these types of functions. So every function would


sort of expose a type very similar to how in an attention layer, every token exposes like a key


to be addressed, right. And then whatever the type inference module expresses could be interpreted as


sort of the query. And you would route queries and keys that have large inner products together.


However, if I understand correctly, in your experiments, you just left the type signatures at


the initialization. So you were happy, saying, we initialize them uniformly, and that's it.


That's one way of doing it. I mean, even if you don't do it, it kind of still works. But I mean,


this is just like, we also experimented with some sort of a repulsion loss,


where we could kind of, you know, like, so that's, so like to give some more context,


that was kind of, you know, like, how do you prevent all the signatures from collapsing on


each other, right? Because that's a very nice way to cheat. I mean, you know what neural


networks like to do, right? They like to cheat. So like, this, like not learning Si is just like one,


it's just the simplest way to, you know, like prevent this from happening.


There are we experimented with some other ways, like, I don't know, the repulsion term,


like a hinge repulsion term, right? That would kind of just push everything away if they're,


like too close to each other. It worked just as well. But you know, like,


had more hyperparameters and thought, okay, how can we simplify this? How can we break it down?


And then we kind of, you know, just froze it and saw, oh, okay, the performance is not too bad.


And the reason we understood was that, okay, the type inference can also do, you know, like,


because Si is a learnable parameters, but the type inference MLP is a two layer MLP,


it also has learnable parameters, right? And their roles are kind of sort of interchangeable


and to some extent. So we kind of, okay, that's like, we figured out this like one way


to save some complexity. So yeah, you have two things that essentially are learnable that


whose mission is the same. Exactly. Exactly. Okay, I see. Yeah. And then you do this,


this is a, this is, I think the whole thing is a softmax then it results in sort of a softmax,


there is an exponential of the inner product, and then there is a normalization involved,


right? So this is essentially, let's say an attention mechanism over functions.


Over the input, right? Because this then produces a mask. Yeah, exactly. Yeah. Yeah. So this


determines you have this nice graphic right here, this determines which token is routed to which


function and the same token can be routed to multiple functions. But hopefully you've configured


your threshold such that it's reasonably sparse. So did you, I don't know, was it your plan to have


this as sparse as possible? Or what is the reasoning behind here? I know sort of the people


who argue from neuroscience and so on, they always argue for sparsity, only a few of these modules


are activated at any point. Is it hard to impose this in a system like this? Or how does that work


out? I imagine it's going to be easy to have one function per token. I also imagine it's going to


be easy to have all of them. But to have like a sparse in between thing, is this a hyperparameter


that is very hard to figure out? Or is this relatively easy? So I think we found a good range


of hyperparameters that kind of did the work. And actually, we have a whole big appendix on how


we, how to set hyperparameters based on the problem that you're looking for, the behavior


that you're looking for. Because in the end, what you have is a spectrum. I mean, if you have


too much sparsity, your thing is going to be really hell to train. Right? I mean, that's,


you're doing gradient based learning, right? I mean, there's, you know, like only so far you can


go on unless you try some combinatorial magic and get it working. So it's like not a silver bullet


in the sense, right? But there's a trade off between training stability and kind of like


sparsity and out of distribution performance. We found that under some configurations,


training went like super smoothly, right? And then when we tested it on some adaptation tasks,


it was meh. But then we had like, when we cranked up to sparsity, like most of the runs diverged


when we cranked it to the extreme, right? But when you crank it a bit less, like on the edge of chaos,


that's where the magic happens. And that's where you get these models that perform well


in distribution and are also pretty good for adaptation tasks or the other tasks that


we're also interested in due to the inductive bias. So it's kind of always like playing the


daredevil, you know, like how far will you go? Given that this seems to be a distinct point,


is there a hope that we can automatically discover this point? Maybe in the future


without having to set the hyper parameter to be like, you know, at this edge of chaos where the


magic happens? Yeah, so like, I mean, it's, that's what hyper parameter search is kind of about. No,


I mean, I mean, that's what that's what you're kind of, I mean, I mean, the edge of chaos is


definitely not a new thing. I mean, there's I think, pretty sure there are a few papers on it


as well. Like, it's a pretty common thing also in neural networks in general. So, so it's definitely


not a new thing. And maybe there's a more principled way that we're not seeing yet. It'd be


cool. But I think so far, this paper, what we just did is just like, do some hyper parameters. And it


is not awful, right? I mean, if you did not say if you kind of, even if you did not go too sparse,


it kind of worked. Right. And then the performance, you know, like, I mean, the less sparse you go,


the more training, the more stable the training is. But, but, but, you know, like, there is like


some leeway. And it's like not an awfully tight tolerance. Sorry, Francesco. Yeah, I think like,


if you want to be extreme about this, you could also think about like, you know, playing with it


during training, right? Like, instead of like fixing a value, you can train the network to


exhibit different behaviors, depending on like, you know, different type of sparsity you may want


to use at test time. And then you just decide later on, okay, do I want to be conservative


in my prediction or not? Whether, you know, I believe this data is in distribution or not,


and then you use different degrees of sparsity. This could also be done. We haven't tried. Maybe


it helps stabilizing training because you also allow for like less sparsity. Also, at the end of


the day, there is an exponential in front, right? So like small values get killed anyway. Yeah.


Okay. Yeah, that's what I what I meant. Like probably one, like having having one dominant


value is also pretty easy. Multiple might make it a bit okay, but now this is this is the part that


routes, let's say the tokens to to these function, you call it type inference. But in essence,


like, I think it's fair to just call it sort of attend, like, it's a little bit like attention


based routing. So it's a softmax over inner products with, you know, with the things that


are available. And the inner products determine sort of the routing table.


Not entirely right. I mean, it's, it's not like exactly an attention mechanism. But it kind of,


this is kind of like another layer of attention, right? It's kind of like, if you want to think of


it like a nested attention, right? Very second attention, like the higher level attention


decides which token get to interact via that function in the lower level, right? So it's kind


of like a hierarchical attention of a sort if you want to think of it that way, right?


So it's an attention in front of the attention, because now we actually get to the attention,


which happens inside of these functions, you've already alluded to the fact that


there are there are inside of these functions. There is there is something happening. So I also


had a bit of trouble understanding this just from parsing your, your math a bit. But I think


what you said right before help namely, you have these you have what you call mod lin layers.


Mod. What does mod stand for, by the way, modif modulated, modulated linear layers, of course.


Now the modulated linear layer is a linear layer, right, which means it's it's a w, or it's w times


something plus B. So there is a weight matrix, there is a bias. But the something in between


is a little bit different. You don't only have the input, but you have the input. And this is the


the element wise product with this thing right here. And okay, we have a normalization layer.


But essentially, again, it's a linear projection of this C value. And so this is w c times c.


And the C comes from here. So that's actually an input to the function. Now, is it like,


do you understand this correctly? This here, this is a learned set of parameters that is shared


among all the functions in the same script? Is this? That is correct. This is what I understood


that is absolutely correct. That's totally correct. Good. Yeah. So and but yet the this


this C right here, that obviously says how one function is different from another function


otherwise, and this w is also shared between all the functions, which one? Yeah, yeah, yeah,


all the parameters are shared, except whatever x is element wise multiplied with, that is the thing


that's not shared. So this is, I think your analogy is that C here is the code of the function. That's


how you tell the function what to do. And then x is the input. And the rest is the rest is


essentially the same across all the functions. So you kind of parameterize the function with its


code. It's a bit like a, like a like a touring machine or so. I mean, it's it's not a new like,


it's not a totally new thing, right? I mean, I think we cite a bunch of papers as well. But like,


there are like a class, I mean, styleGAN uses something kind of similar, if you think about it,


right? And as the SIPs, like conditionally independent pixel synthesis, and like,


they serve as pretty strong inspiration for this setup, right? So I mean, I don't. So yeah,


I don't think in this part, we reinvented the wheel. Yeah, sure. No, no. It's like,


there might be this. This, my question is, where does the code come from? Because I clearly see


that, you know, here is, you know, a token, the token gets routed to one or more functions. So the


talk the x comes, that is the token that is the input. Now, where does the code for a particular


function come from? Is that also just like a learned parameter for each function? So epsilon


has its c, f2 has its c, f3 has its c, okay. I mean, another way to, I think another way to write


this, if I were to sort of draw this up, is that instead of drawing this, right, I can, I can also


say, well, there is a, this here is, is also just kind of a weight sharing thing, right? It's a


learned parameter times another learned parameter, except that it's not a weight sharing parameter.


It's a learned parameter, except one learned parameter is shared across all the functions and


one learned parameter is separate per function. So this, if we leave away the weight sharing part


of that, it's a set of learned parameters for each function. So I can, I can imagine my x, and here


is my c, that is one per function, the x and the c, it gets multiplied element wise. And maybe here


is c2 and c3. So that gets multiplied element wise as well. And as well, so the x goes into each of


these and then all of them individually, all of them go through this same linear layer, this outer


linear layer, which seemed to be completely shared, right? This w, so this is like w, x, whatever


comes out of this calculation plus b. So essentially, yeah. Sorry. A good abstraction to think about this


would be like really an interpreter in Python, right? So that is kind of like a nod to the


whole name, right? Because like you may have different functions, right? But, and these


functions, they have their code, they may do different things, but in the end, the actual


interpreter that's doing the computation, it's shared between all the functions.


Okay. Right. So like, this would be the interpreter here. Yeah. Like the orange, that's shared. Yeah.


The part that is shared between the functions, those are the interpreters. That's, I mean,


you can think of it as parameters of an interpreter. So this is a way to, I would like,


it's a, again, what I said at the beginning, it's sort of just an intricate weight sharing scheme


to characterize a linear layer. This is, it would work independently, right? The type inference module


would work independently from sort of the mod linear layer. These two could, we could make


separate models with either one or the other. Yeah. So you can, I mean, if you, so like the


Xs, right? I think that's where the signatures and the type inference thing comes in, right? So


if a function C1, like sees X or if it just sees zeros, like in a naive


implementation, right? That is what's determined by the type inference mechanism. Right? But


otherwise, yeah, totally. And that's what makes the symmetry a little bit, right? Because


now we are sharing parameters in both widths and depth. So, I mean, it's clearly you want to


differentiate a little bit what happens, right? And that's how it happens essentially.


Cool. And you use this mod linear layer, which, so it's input and output are as a linear layer,


or you input a token, and that will output another embedding for that token. And you use that


in sort of the rather classical way of doing attention. So you compute keys, you compute


queries, and you compute values from the token using these now these mod linear layers instead


of just linear layers, as we would do it in regular attention. And then the attention mechanism


is essentially the same, except, of course, it needs to respect that routing table, right, that


we computed initially. So any function, so the functions here are essentially attention mechanisms.


Yet, they only get access to the tokens that the routing mechanism determined would be appropriate


for those ones. Yeah, you can really think of it as a sparse attention that doesn't get all the


tokens going to get a subset of them. Yeah. Okay, so now we have we have the sort of attention matrix,


and we can again use the linear combination here and send that as a classic attention mechanism


does through another linear layer in order to get the output embedding of a particular token.


Is it usually in, let's say, regular attention mechanisms, this part here takes quite a lot


of parameters, which is, it doesn't seem like it like it doesn't sound like it doesn't look like


much right, but it does take like a lot of parameters and, and computation does your does


the fact that you parameterize things with these codes? Does that change how many parameters there


are in different parts of the model? Can you save a lot of parameters using these these weight


sharing schemes? Or like, what do you is it? Is it a side effect of your architecture that you also


have less parameters, let's say in? Yeah, because they're shared everywhere, right? So at the end


of the day, you have less parameters, it doesn't mean that your inference is gonna be faster,


right? Yeah, but definitely it's a very lean architecture in terms of number of parameters.


Yeah, that's what it seems to me. You're sharing it in depth, right? So that's kind of the,


that's where, yeah, but you're also kind of not totally sharing it because it's codes that kind of


also on the show, you're kind of sharing and not sharing at the same time in a very special way.


So I think that's kind of that's the stick. Yeah. So coming back, yeah, exactly. Coming back to this


diagram right here. So f, f1, f2, and f3, they all share the same sort of global parameters. But then


f1 has its own code, and f2 has its own code, and f3 has its own code. And what you do now, that's


what you said just now, is we apply this layer recurrently, right? We apply it multiple times


within each script. So potentially, that allows one particular token to first go through function


one, have a result computed from that, then go to whatever function three, have a result computed


from that, and so on. So the code for function one would always be the same, independent of which


step it is applied. So this really feels like I'm a programmer, and I'm composing functions one after


another. Exactly. And it's nice because you can do it in a totally flexible way, right? After function


three, you could go back to function one again, or use function three again, like it's completely,


like, the way each example is routed through the network is completely independent. Yeah. And


it's determined on a per sample basis, right? And the routing itself is completely learned.


That's why it's a very interesting architecture. Yeah, I see that. Yeah, that's pretty cool.


It is implemented, I see, as a recurrent neural network, right? I mean, essentially, I apply


the same function over and over again. Parameters are shared across the depth, which is kind of akin


to a recurrent neural network. Did you have to do any sort of tricks in order to get this to run


stably? Or anything like this? Did you notice anything? Like how much stacking did you do in


depth? How often did you apply each line of the script? Oh, that's a very, very, very, it's an


excellent question. So like, so I think we talked about it at great length in the appendix. But


nevertheless, what I, so like, what we tried was kind of, so what, so we went, I think, in our,


the biggest models we trained were like, two scripts, each with eight such recurrent layers,


if you want to think of it that way, right? Such function accretions, that's, I think, how we call


them. And that worked, essentially, without much of a problem, right? And we tried even some


combinations, for instance, like two, two, two, which means like we had two scripts, and like we


had two scripts, and then two, two iterations per script. And then, and then there is another,


there's another aspect over here, which is kind of how many, you know, like inside, you know, the


MLP attention, MLP attention, you can, yeah, yeah, you can kind of keep adding more to it, right? So


parameter, and we found like two, two, two also works pretty well, like absurdly well, which


this was interesting. And like eight, one, one also works pretty well, like eight function


accretions, one script, yeah, one. But that's also why you want the scripts, right? Because it


breaks this chain and allows you to not have a chain that is too long. Exactly. Exactly. You can


think of it as a way to break the recurrence. And between the ones inside here, like this,


let's say this MLP and this MLP is, are these the same functions or the parameter shared? Or are


these two different functions that, that now, you know, inside that live inside of these functions?


They're different. They're different. So you have different functions here


that get repeated here. And then you have different stacks of these repeated applications


of different functions. Yeah, that's right. So the real, the real recurrence, the recurrence here


happens in step number two, that's where you recurrently apply the things and inside of the


function, it might take more than one layer to make the function, you know, powerful enough to


do its computation. Exactly. That's the right intuition. Okay. Yeah. So I see, I see kind of


I see, yeah, sorry, go ahead. Yes, there is some lag, but I just wanted to say, I mean,


we also tried to increase the recurrence to depth 16. And I remember it worked as well. Like, I mean,


there was no issues and we shifted from two different tasks that like from multi,


this multi-task classification to this, to this reasoning task and the parameters did,


I mean, we kept them the same and they were out of the box. Yeah, it's a little bit like, so I,


again, I see a little bit, you combine a lot of things, obviously here, because I can also think


of a regular attention based transformer where I have, let's say I have this here as a block,


and I just repeatedly apply the same block. I think the, didn't like the Hopfield network


paper or so even make an argument that that's sort of connects transformers to Hopfield network.


But that was always sort of the entire attention. So I can imagine that by itself. I can also


imagine what I said before this routing just by itself. In fact, I believe the sort of mixture


of experts idea is very much akin to that where you say this, the MLP here, as I said, it takes


up a lot of computation or, and I can route the tokens sparsely to the individual experts. Yet you


decide to sort of split up the entire layer all together. And that, yeah, I think, I think it,


it comes from different motivations because the mixture of experts obviously comes from


the motivation that I don't want to, I want to shard my model somehow. But that means that doesn't


work super well with the sparsity that you do in the attention mechanism. But I don't want to,


don't want to, but if you think about it, if you think about it, like this could also be a


limitation of our approach, right? Because now every example has its own independent path to


the network. And now you cannot really exploit like batch statistics, right? Like now I could


say, okay, I have this batch of examples and they, you know, they look like all they would,


they would all benefit from this particular path in the network, but you're still deciding on each


of them independently. And this is the drawback that you need to keep every expert around.


Yeah. So if, if I were to describe your model without, let's say the language of this functional


approach and so on, because as you introduce a lot of new words like scripts and functions and


lines of code, like there's lines of code, which, so there is the interpreter, right? And the


interpreter goes across scripts and every script is, wait, I had it before, is like composed of


different lines of code and each lines of code shares the same functions and so on. If I describe


this, let's say without any of the language, I would say this is a transformer like model where


each, each data is divided into blocks. Each blocks is a recurrent application of a fixed layer of


attention mechanisms. The attention mechanisms are separated into individual modules that in parallel


can process data. And then you route that data sparsely to these individual modules that you,


and you do this recurrently. So the modules can dynamically process these, these inputs. Okay.


What you did sounds a lot cooler than, than this. And I can totally see that you, you know,


you come from this very different point of view. And I think it gives, it gives rise to a very


interesting model. So now it came to the point where you did some experiments to validate this.


And what is especially interesting, of course, is are your hypotheses, what kind of hypotheses can


you even make building such a model? Like, can you lead us a little bit about how you approached


the experiments? Because what's boring is, you know, we get better on ImageNet. Also, probably


that's not going to happen, right? But you need to, I think this is maybe a little bit for researchers


who are starting out. When you have this new architecture, where you think, aha, okay, this


does something new. How do you design an experiment that sort of validates? Yes,


that's really what's happening. Like, how do you approach this? Yeah, so for me, like, I mean,


we have three experiments, right? But for me, there are really like two cluster of experiments,


right? The one on more like real data is about, can I solve both classification and reasoning with


the same architecture? For me, that was the most interesting part. And then, of course, like,


I want to see all the advantages of the modular computations, I want to be able to change the


inference time, adding modules, dropping modules, and whatnot. But for me, the crux was really,


like, can I do these two tasks that are seemingly very different? And then the other part on the


toy experiment, it was really about, can we truly validate that these functions can be composed in


novel ways? Because when you go about it in the visual data, for example, like, I mean, it's really


hard to say exactly what is happening, right? But then my favorite experiment is when you train the


neural interpreter on to like logic rules, and then you can extrapolate to unseen logic rules that


can be obtained as a composition, right? And you can do that only changing the routing, then it


means that the network actually did learn some compositional knowledge, which was our goal to


begin with. Yeah. And so that's what you did, you did here, you took these logic formulas, and you


built you built a data set from them, you know, you have and and not and or and these are all fuzzy.


So these are not Boolean logic, but they're they're, you know, made out of real numbers and is


a multiplication. Not this is one minus and so on. And you can now build these build Boolean


functions, you can sample them, of course, in some interval, you can train your network on that. And


now you wonder, can it generalize to unseen logic formulas, which would sort of be if this works,


one would think that the network has learned these fundamental primitives of logic, if I train like


learning and or not, then I should be able to recompose my primitives to perform the store. And


they should be able to do that without changing the core, the parameters that the site has,


that decides the computation, but only how you stitch together the functions that in our case is


the routing. Yeah, so you Yeah, you only you only changed the routing parameters. Nothing else,


of course, of course, if you change everything else, it works better. But it still works


surprisingly well, if you only change the routing parameters. And that was the interesting part.


There was this recent paper about similar things that that essentially said, I only have to adjust


the layer norm parameters in order or there are also these adapter layers, right? Did you did you


compare to any anything like that? Or do you see parallels between what you're doing? And essentially,


you know, people saying, Well, if I just change people saying, more generally, I can adapt a


transformer if I just change like very few parameters in between the layers. Do you see


parallels or I think the motivation is different, like, so that there are papers that adapt the for


example, like the batch norm parameters, when you are on a new distribution, right, and then you


you can get much better robustness. But for us here is really about getting some evidence that


the architecture is able to reuse and recompose these primitives in novel ways. So I mean, of


course, like, methodologically, it's the same writer, like a very few parameters that you want


to adopt. But the reason why we do this is completely different. Sure. But but I think at


the same time, it's also kind of, I mean, they also in a very different from a very different angle,


they make kind of a similar point, you know, like, I think the paper you're referring to is that


transformers are universal computational engines or something. Exactly. Yeah. I mean, I love that


paper. I think is one of my like favorite for like, since the past few years, I really loved it.


And, and I think, you know, like, the message they're trying to say, send this kind of Yeah,


look, if you have a pre trained Bert, it's in some sense, it's universal, right? Because the


inductive biases of even attention is a good place to start. Right. And this is kind of like


taking that a bit step further. And in my mind, yes. Right. And you know, like, yeah, you go,


you go ahead and you say not only not only should we train like layers of these attention


computations, but if we structure them in certain modular way, that might lead to even more, let's


say universality. Yeah, yeah, yeah, that's it. And another not star, I think has also been kind of


like our 2019 work, the Benji hotel work on meta transfer objective for learning disentangled


causal mechanisms. And I think the argument there is like, if you have a good modularization of


knowledge, then you know, like when you when given a new data distribution, you should be you should


get away really easy when it comes to adaptation, right? Because you're already kind of like,


right, because you're already kind of have all the pieces together. And when you see a new


distribution, you just need to change, like, do small localized changes, and then you're good to


go. Right. That's kind of also been a not star, as you see, like for the adaptation experiments


that come below, right, that, you know, like, if, if, you know, like, yeah, that also connects with


the whole causal picture in some way, right? Not the classical causality, but the causally inspired


class of models that so yeah, I think that's just like another not star that has been guiding the


hypothesis because he asked, you know, like, how we came up with these hypotheses. Yeah, like,


I think that's one of the angles as well. And like, generally, like, if you want to connect it


back with causality, that has been like a core guide for my research agenda, taking ideas from