paper B yet was before paper a so when the same logic indicate that paper a copied from


[ML News] Plagiarism Case w/ Plot Twist | CLIP for video surveillance | OpenAI summarizes books

paper C, the authors of paper a actually comment on this and say they did not know about paper


C when they wrote their paper, they now highlight the differences between the two papers, they


strongly deny having plagiarized paper C and the whole thing is just a mess. Now, is there


something to learn from this? I think yes. And I think that's what makes it so hard in


these plagiarism cases. I don't know anything more than you do. But if I had to guess, I


believe the authors of paper a here that they didn't know about paper C, but it just shows


you how multiple people and they self admit the idea is relatively simple and works how


multiple people can have very similar ideas and then write papers that essentially turn


out to be very similar to each other. And among the 1000s of papers that are released


each month, it's bound to happen that some of them with the same idea doing the same


sorts of applications will turn out to be quite overlapping without ever having seen


each other. And that might be indistinguishable from a paper that has actually plagiarized


another paper but has done so putting in a little bit of work to reformulate and redo


experiments. So while plagiarism is certainly a problem in our field, and it's probably


happening a lot more than we realize in this smarter way that is undetected. It is also


the case that you have to be quite careful with these allegations. And in general, probably


the best thing you can do is simply to publish your ideas, write them up as well as possible


and just make it easy and nice for people to cite you instead of citing someone who


copies from you. And yes, that means that there is a little bit of a marketing aspect


involved in this and it also leads to problems where people with bigger followings will attract


more citations. But ultimately, it is your best shot. With regard to this particular


story, I doubt that anything more is going to happen here. We'll keep an eye on it. Next


news GitHub user Joan modern demonstrates how you can use clip so open AI is clip model


to search through videos. Apparently in the original clip paper open AI claimed that this


is not really an application that this doesn't really work well. However, as this little


project demonstrates, it appears to work quite well in the way that you can search surveillance


footage for a descriptive sort of text. So what you do is you take your video, you encode


each frame with clip and then you encode the text that you're looking for also with clip


and then you compute the inner products between all the frames and what you're looking for.


And if any of the frames exceed a threshold, then you will show that frame. So here the


author searches for the text a truck with the text of Walla and directly finds the frame


corresponding to that a white BMW car a truck with the text JCN a bicyclist with a blue


shirt, a blue smart car, and it's pretty easy to also do this yourself you clone the repo


you put in your own video, you can search through it. Now this raises a lot of questions.


This gives essentially a new superpower to people who have access to this kind of material


tracking was possible before but not with this ease, you'd have to craft some sort of


a detector in order to label a bunch of things in some of the images and then you might be


able to track it through the footage. But here you can simply enter what you're looking


for in many different ways. Now you can of course ask what's the purpose of having surveillance


apparatus in the first place if it's not for you know, surveilling. So rather than criticizing


the possibilities here, one might criticize the implementation of surveillance in the


first place. But and it's also the case that you might simply have the surveillance cameras


for the purpose of proving someone like running a red light or something like this. But once


it's in place, it can obviously be misused for other things. And with the addition of


clip now that's an easier possibility. I don't know, I don't have the answer here. I just


like people to know that things like this are now totally possible not only to the government,


but pretty much anyone who has access to this camera feed and a home computer. So make of


that as you will. Next news, the DARPA subterranean challenge has concluded that this is something


extremely cool. The task here is that submissions to the challenge are teams of humans and robots


that explore underground areas. So this can be mine shafts or underground tunnels or anything


like this. So this is a competition and the way it works is that the robot or robots and


usually there's multiple robots are deployed into this underground system and are tasked


with doing certain tasks like finding things, retrieving things, mapping the area while


the humans aren't allowed to go into the underground areas, they can communicate with the robots.


However, this being mine shafts and so on, there isn't always reliable communication.


So the robots must largely be autonomous. And this isn't only simulated, this is actually


real world robots. For example, here is a drone in one of these underground bunkers


being hit by a plastic bag that itself has thrown up with the wind. So Evan Ackerman


on Twitter has a number of really cool clips from this challenge. So the challenge has


concluded you can no longer participate this year, but you can look at the participants


at the trials on YouTube. This is just really cool. Jürgen Schmidhuber pumps out another


blog post claiming to correct mistakes in citations in historical references by others.


This time he criticizes the 2021 Turing lecture by Yoshua Benjo, Yann LeCun and Jeff Hinton,


which they gave after receiving the Turing award. It also criticizes the announcement


of the Turing award, all of them for as I said, making wrong historical claims and not


properly citing things. Schmidhuber himself starts out the blog post by saying, we must


stop crediting the wrong people for inventions made by others. And in the abstract, he states,


most of these breakthroughs and tools, however, were direct consequences of the breakthroughs


of my lab and other labs in the past three decades. And he makes 15 distinct claims


about the Turing lecture such as LBH, which stands for LeCun Benjo Hinton, site Hinton


for dropout without mentioning that dropout is just a variant of Hansen's 1990 Stochastic


Delta Rule or such as LBH site Benjo's 2014 paper on generative adversarial networks without


mentioning that GANs are instances of the adversarial curiosity principle of 1990. And


he follows this up with detailed references to his claims, as well as over 250 references.


A lot of which are to himself, I have sided with Schmidhuber a lot of times in the past,


it is true that his labs have done a lot of fundamental work. It is also true that sometimes


this work is not properly credited. And I can even understand that he's pretty salty


about LeCun, Benjo and Hinton receiving the Turing award and him not but this is pushing


it a little bit like just the sheer length of this article. He sees himself as something


like a crusader for the correction of scientific history for making sure everyone cites properly


and so on. And I agree that is an important thing. But I asked myself, is this really


what he wants to be remembered for? Does he want his legacy to be Oh, Schmidhuber, the


person who did a lot of cool work? Okay, we might not credit him for all the cool work


he did. But still people remember him for a lot of cool work. Or does he want to be


remembered as the person where every single time someone invents anything, he finds a


vague relation to what he did in the 1990s. And then claims, Oh, this is just a special


version of my case. And look at the length of this article, the amount of work going


into this is just absurd. Like he's so smart, clearly, he could do something better with


his time. And this isn't even productive at the frequency and intensity that Schmidhuber


is doing this. This is completely counterproductive. No one is even going to respond to this. People


simply say, Ah, here he goes again and ignore him. And the claims get more and more wild


while you can make the claim that something like a resnet is essentially a highway net,


but simpler. The claims that GANs are just a special case of artificial curiosity. It


might be true in an abstract level, but certainly not on a practical level. And then his newest


claims that transformers are essentially nothing else than fast weight programmers and so on.


I mean, come on, if this are actually all special cases of your things, then please,


please tell us what the next big thing is. Transformers have not only sparked a revolution


in an LP, they have widespread consequences. People worry about do language models really


understand people can solve new tasks with them. Google search is now powered by BERT.


And Schmidhuber claims to just have been sitting on this for 20 years. Well, please next time


tell us beforehand so we can rein in the revolution faster. In any case, read this if you want.


I don't think it's worth your time. OpenAI has a new blog post called summarizing books


with human feedback and a paper to go along with it called recursively summarizing books


with human feedback. I don't know why they've left out the recursive from the blog post.


But in any case, the algorithm works by taking a book chunking it up into sections and then


summarizing each of the sections and then putting together the summaries of those sections


and then summarizing those into super sections and so on. Every summary generation is conditioned


on the section it's supposed to summarize, but also at the summaries that have been produced


from sections that come before it at the same level. This is something you can see here


at the height one. So generation of the super summary here would not only receive the things


it's supposed to summarize, but also the summaries that have been generated before it. So essentially


you're telling the model, here's a bunch of text I want you to summarize. It's from the


middle of a story. And here is a high level summary of what already happened in this story.


Please continue this high level summary. So this is cool because doing this at this chunking


level and not as a please summarize the whole book task you get more accurate, you can leverage


humans in a better way because humans can now simply check whether a reasonable length


text like a couple of pages have been summarized correctly and not whether an entire book has


been summarized correctly. And also this allows you to summarize arbitrarily long texts because


you can just always add levels and therefore if your original text is longer, you simply


recursively summarize it more often because with each recursion the text gets chunked


then each chunk gets summarized and then all of this goes together. So this is a neat combination


of the principles of learning from human feedback, which is a thing that OpenAI has shown interest


before and also recursive task decomposition where you can divide a task into essentially


the same task at lower levels. Therefore, you can learn one model to do the task and


then simply apply that model over and over again. The model they end up using is a fine


tuned version of GPT-3. And you can read some of the example summaries on the blog post


for example, this one from Alice in Wonderland. Now I've read the summaries and I have to


say they're not exactly what you would expect from a summary of a book in that they seem


to pick out important events that happen in the book, but the highest level summaries,


they don't really give you like a sensible overview over the plot of a book. And this


might be due to this recursive decomposition. So while it might be appropriate at the lowest


level to simply sort of leave away all the in between whatever the author sprinkled in


and simply mentioned the important events of a chapter, if you go higher level, you


most often want sort of a more abstract summary, you want to condense the plot somehow. So


there's still room for improvement here. But it's pretty cool to see what these language


models can do when you bring the human into the loop. CNN Business writes, a startup says


its software can spot racial bias within companies will the surveillance scare employees. Now


this is a product called unbias it eliminating bias with technology one alert at a time.


So what this product does is it monitors the employees of a company for example, their


email communication, and it tries to detect instances of bias. So the CNN articles mentions


this example. For instance, she said if an email from one employee to another alluded


to a diversity hire, that's the kind of thing the software would be expected to flag. So


the way it works is here if unbiased it scans an email and finds wording that may be objectionable,


it will send an alert to a small group of employees working in human resources and diversity


equity and inclusion with the wording in question highlighted in yellow. The spokesperson says


it's not looked at as a gotcha for employees because the bias might be unconscious. So


the consequences might be that you offer an employee bias related training or other education.


The interesting thing is that it says it doesn't use artificial intelligence to determine when


to send an alert because of concerns surrounding the possibility that bias could be contained


in AI itself and that it essentially relies on keyword and phrase spotting the product


website makes a big deal that the companies applying the product are in control, they


can define what the criteria are and so on. And they frame it more as a compliance issue


comparing it to similar tools which detect instances of for example, insider trading.


However, if this doesn't scare the crap out of you, then I honestly don't know. And it's


only a matter of time before machine learning is actually used in these systems because


as they are, they seem to be pretty easy to evade. And when the company wants to improve


their detection, and they'll implement some sort of an NLP system that's certainly going


to make things more interesting, but not necessarily more pleasant. And I highly doubt this is


going to change anyone's mind or unconscious biases or increase in substantial ways the


workspace climates. Speaking of surveillance, Apple is working on iPhone features to help


detect depression, cognitive decline, the Wall Street Journal writes. So this story


is about Apple monitoring users in order to detect things like depression and mild cognitive


impairment, which is a precursor, for example, to Alzheimer's or other forms of dementia.


Now for this, I'm honestly not that skeptical, given that I hope you will have the ability


to turn it off. But if this is an optional feature, it could potentially be quite helpful.


People generally let their smartwatches and their phones track other health related data


such as pulse, oxygen saturation, number of steps, heart rate, heart rate variability,


well heart rate is the same as pulse, right? Doesn't matter. So while I certainly agree


that mental health data isn't exactly the same, it probably requires monitoring more


personal data than simply a number which is your pulse, we do face a lack of mental health


professionals and having the system monitor you for something like cognitive decline might


be helpful in that you might be encouraged to go look for treatment a lot sooner than


you would if you simply had to notice it yourself. Because if something declines mildly over


time, you're unlikely to see it yourself. But of course, the privacy implications for


something like this, especially if this data is then sent around and analyzed and potentially


even sold are pretty great. So treat this with a grain of salt. Next news CNBC writes


the UK publishes a 10 year plan to become an AI superpower seeking to rival the US and


China. So this article details the UK strategy to become a leader internationally in AI technology.


It's something like a 10 year plan and it outlines a strategy and this strategy goes


from providing more compute to launching centers where researchers from the whole country can


communicate with each other and coordinate AI research. It also outlines some better


regulations for intellectual property and so on. And it appears to be just a general


indicator that the government is looking to push this area. However, there are multiple


problems with something like this. First of all, academics are very likely to move not


only academics, also employees of tech companies, they're pretty move happy. A lot of them are


not bound to individual location. And it is even considered a good career move, for example,


in academia, if you have spent time at various different places. So as a country retaining


knowledge is quite a hard task. If it comes to people like this, it is a bit easier with


industry where a company actually needs headquarters and so on. But also their employees frequently


rotate. The other problematic aspect is actually also outlined in this article. And that is


that AI startups like many startups get bought. And very often they get actually bought by


us or Chinese big corporations. So in this case, Britain might have raised the startups


given them tax breaks or subsidies or grants and whatnot, built up all this knowledge in


the country only then for it to be bought by a US firm. The article, for example, names


DeepMind as such an example. Now while DeepMind is still in London, it now belongs to Google.


It's good to see that countries are pushing AI technology, but it does detail the problem


you have when trying to achieve something like this, especially as a country that is


not huge, such as the UK. Okay, let's dive into some helpful libraries scikit learn is


a lie. I'm kidding. You know scikit learn, but scikit learn has just released the one.


Oh, release for some projects, one dot or release is sort of the initial release first


stable version and so on. For other libraries, the one dot or release is actually the last


release saying okay, we're done with this releasing one dot oh, that's it scikit learn


doesn't appear that either of these are true. Of course scikit learn is already an established


library, but it doesn't seem like they have any intention of finishing or killing the


project. There are also no major changes in the library. One of the changes is that lots


of functions now have to be called with keyword arguments, which let's face it in NumPy and


scikit learn and all of these functions is a good change. Now while I think it would


be better to simply educate the users to do this as a good practice and leave them the


option of killing their code with non keyword arguments. Now it's their library, they can


do whatever they want. They're also a bunch of new models and the plotting library has


been also improved. Also new release dopamine version four is out. So dopamine is a library


for doing reinforcement learning research with lots of implementations of common agents


and environments. And the major new additions are things like soft actor critic for continuous


control and the up tax optimization library for jacks based agents. Also new is that it's


now compatible with Docker so it will become a lot easier to set up the required environments


in the future. Microsoft releases music which isn't necessarily a library. It's simply an


umbrella project for music generation research. So this repo holds code for a bunch of different


papers in various aspects of synthetic music generation and also artificial understanding


of music that already exists. This can go from classification of genre to transcription


of lyrics all the way to arranging and synthesizing new music including lyrics. Now what's cool


about music is that not only does it have this picture logo, but they actually do have


their logo in MIDI and you can listen to their logo. Excellent Facebook AI releases dynatask


a new paradigm of AI benchmarking and this is an iteration on dynabench. So this is a


system for benchmarking AI systems, specifically natural language processing tasks. So this


is supposed to combine tasks which are essentially data set and their associated labels and on


the other hand models that people submit and it evaluates the models on the task. But also


there's the option to have the human in the loop something like a mechanical Turk worker


that goes and tries to come up with some sort of adversarial examples against the models


or examples about a particular aspect of the task. The human created data is then fed back


into the system and used as further evaluation data. So this is supposed to give a more complete


picture of models capabilities, rather than simply evaluating them over and over on the


same limited set of static benchmarks. So if you're interested in that sort of thing,


this seems like a pretty good framework to go about it. Next up, Fi flow has a new release


out and this is a framework for solving partial differential equations in a differentiable


manner. So as you can see right here, this can be for example used for fluid dynamics.


Now I'm a total new but any of these things but if you're in these fields, this library


might be interesting for you. The next library is Dora the Explorer, a friendly experiment


manager by Facebook research. And this is an experiment manager that focuses on specifically


things like grid searches. And the special thing here is that the experiments themselves


are defined in pure Python files. So there's no yaml, there's no web interface or anything


like this, your experiments are simply Python files to find some sort of a grid search.


And the tool can identify and de duplicate experiments that happen from, I guess, gridding


too much. So it seems to be a simpler alternative to many of the experiments running tools out


there. If for some reason, you're looking for simplicity, you might want to give this


a try. Now being said that it seems simple, the system actually looks really powerful


too. So I have no doubt that you can go up in complexity with this by a lot. For example,


it does interface with scheduling systems such as slurm. Next up habitat lab is a high


level library for development in embodied AI. This is essentially a library that helps


you run our L and robotics tasks in 3d environments. This is not a new library, but there have


been some new developments. First of all, there is a new data set called habitat Matterport


3d data set that brings real world environments into the habitat environment. So these are


real rooms that were scanned by a depth sensor by a depth of where camera and now you can


explore these real environments inside the habitat framework. So if you're into embodied


AI robotics, indoor navigation, anything like this definitely give habitat a try go to toilet.


Good job. And lastly, Google AI announces WIT a Wikipedia based image text data set.


This is supposed to be a very high quality data set connecting images to text. So rather


than scraping the internet and trying to read the alt text from an image, this leverages


Wikipedia. So on Wikipedia, whenever there's an image, there's actually a lot of information


about that image all around it. Not only is there the usual description, but there's also


the page title that usually refers to something inside the image and the data set also grabs


the page description, which very often also relates to an image on the page. And lastly,


the image page itself also usually has something like an attribution description. And the file


name can also give indications about what is in the image. The cool thing about this


is since Wikipedia is so extensive that you not only get image text pairs, but you very


often get a lot of translations for all of these different things into different languages.


So this is an example of one data point that you would get, you get the image along with


URL page title, reference description, attribution, description, and so on. Oh, I said attribute


description before attribution description, sorry. So while this is a smaller data set


than what for example, Dali was trained on, it's definitely a higher quality data set


with lots of more information per data point, it's going to be pretty exciting to see what


people build from it. Alright, this was already it for ml news. This was a long episode, I


realize this, but there's just so much stuff happening. If you have anything happening,


Deep Networks Are Kernel Machines (Paper Explained)

Hi there. Today we're looking at Every Model Learned by Gradient Descent is


Approximately a Kernel Machine by Pedro Domingos. This paper on a high level


establishes a theoretical connection between gradient descent learned models


such as deep neural networks and kernel machines as you might know them from


topics such as support vector machines. The paper interprets its own finding as


meaning that deep neural networks essentially store that training data in


their parameters as a superposition and when a new data point comes in


what it does it is it sort of compares the data point to the stored training


data and then decides with relation to that data what the output should be


which is of course exactly what a kernel machine does. So it is a theoretical


paper and we're gonna go over it. I'm not an entire expert on these things


but the main theorem is fairly easy to grasp and the proof behind it is also


fairly easy so I thought it'd be a good paper to look over. Further Pedro is


coming to our Machine Learning Street Talk podcast in the future and I wanted


to get familiar with his work. So you know if you like content like this too


let me know let me know if you understood it or not or if I just made


it worse. Yeah let's dive into the abstract. The abstract is actually a


pretty good summarization of what the conclusions of the paper are. It says


deep learning successes are often attributed to its ability to


automatically discover new representations in the data rather than


relying on handcrafted features like other learning methods and as you might


know this is the success story of deep learning. Before deep learning we had to


do a lot of hand crafting of features where expert knowledge went into


problems and then we would simply aggregate the handcrafted features with


some sort of linear classifier or you know in some cases a kernel kernel


classifier though the hand crafting of features would also go into kernel


design. Deep neural networks are different because we just feed in the


training data as is and the deep neural network will automatically discover the


features that are important. At least that's the prevailing notion of what's


happening. This paper challenges this view. They say we show however that deep


networks learned by the standard gradient descent algorithm are in fact


mathematically approximately equivalent to kernel machines, a learning method


that simply memorizes the data and uses it directly for prediction via a


similarity function, the kernel. So that's the the main thesis of the paper.


They show that it is equivalent to a kernel machine. If you if you don't know


anything about kernels don't worry there is a good machine learning street talk


episode with Alex Stenlick where I get to ask all the dumb questions about


kernels so you don't have to ask them. So if you are interested in that check that


out as well. That's on the machine learning street talk podcast. They say


this greatly enhances the interpretability of deep network weights


by elucidating that they are effectively a superposition of the training examples.


So saying again that the deep neural networks essentially store the


training data in their weights and then use that to compare new data points too.


Now the conclusion of this paper is interesting. I don't fully agree


like I don't agree with the framing here that it's sort of replacing this


notion. I think this gives rise to sort of a dual view of the problem. It is a


way that you can also look at these deep neural networks. I don't think it kind of


changes like it can both be true that they do discover good representations


and also are a superposition of the training data. I think it's simply a


different way of looking at the problem. However I as I said I'm not a super


duper expert on this and they allude to the fact here that this improved


understanding should lead to better learning algorithms and of course even


though this paper here is has no impact for practitioners down the road this


could actually have some of an impact. So what is a kernel machine? A kernel


machine is this thing right here. So in machine learning we always want to we


have some X and this is our input data and we want to get some Y. Now for the


purposes of this paper think of Y being just a number so think of linear


regression okay not linear but just regression where Y is a number X is a


data point and we want a function f that assigns each data point a number and


then that number is going into a loss function so there is going to be a loss


function that compares that number to the number that we have in the training


data set our true label Y star okay so we have training data XI this gives so


the neural network gives an output YI we compare that to the true label in the


loss function. Now a kernel machine is a particular way of how this f here is


built and usually if you think of this as a neural network you simply say X


goes into layer layer layer layer and at the end you get Y. A kernel machine is


different a kernel machine actually builds a database of all the training


examples so what it would do is it takes your training data set and it would sort


of build a list of all the training data points in here. I'm super


oversimplifying this but it will build a list of all the training data


right here and now when you want to know about a new data point say you want to


classify this X right here what it will do is it'll go to its database and it


will compare X to each of those training data points to each and from each of


those training data points you get a response of how similar is X to that


training data point. So for the first training data point you would get a


score of how similar that is and that score is computed by this kernel


function so X1 and kernel of X with X2 you get kernel of X with X3 so for each


data point you want to know how similar is the data point that you wonder about


to the data points that you've already seen. If we look at this in kind of a


schematic so let's say this is our data space and you have kind of a data point


here and one here and one here and one here in the training data set and you


want to know how should I classify this red data point right here. Your kernel


will tell you and it looks easy if it's on the plane but it's not easy at all in


high dimensions with complicated data like images or structured data. It's not


as easy as simply taking the distance though here it is so here a good kernel


function would simply be the Euclidean distance to these data points and this


says something like the kernel function would tell you that these two data


points right here are very similar to the data point we care about while these


two data points right here are not that similar so when you classify the data


point you consider all the data in your training data set at least in the ground


case so here is your training data set and your kernel will tell you how


similar each one is okay that's the kernel and then you take that similarity


and you aggregate the labels of the training data points since you know and


the labels they are in here so y star it says AI here but yi star so the true


label is usually what gives rise to this A it doesn't need to be the true label


but in the simplest case you will simply aggregate the labels of these data


points in in proportion to how close they are it's it's a bit of a nearest


neighbor classifier okay so that's a kernel machine the important thing is


that there is this kernel this is a function that tells you how close any


two data points are and there is this sum right here so that means that the


your prediction y is going to be it can be a func nonlinear function of the sum


but it's going to contain a sum over the training data okay and each training


data point is measured in its similarity through the kernel function and then the


labels of the training data points are aggregated that's a kernel machine so


you don't you don't need you know any model for this right the learned


parameters here are often the the A's and the the B right here the offset


however the kernel can also be learned but very often the kernel is also fixed


and you can see immediately that choosing the kernel is the name of the


game in kernel machines and before deep learning lots and lots of an expert


engineering has gone into building kernels to measure distances between


data points using kind of expert knowledge it from a field it's probably


still advisable today some people claim we rely too much on neural networks to


lose for us but you know neural networks have been pretty pretty good so what's


gradient descent you might know gradient descent gradient descent means that we


do have a loss function right here and it is differentiable so what we can do


is we can simply calculate the gradient with respect to the loss function and


then change the parameters that we're learning into the direction of that


gradient and we arrive at a new at a new weights and we repeat the process so if


you think of linear regression for example you should simply have X here


and Y here and you might have sort of three data points like this what would


a kernel machine do a kernel machine would do the following if you're trying


to classify a new data point like this one right here the kernel machine will


go look which of the data points that you already have are close this one on


the right here is pretty close this one is kind of close this one is very far


apart and then it would sort of aggregate the labels and it would say


well since you are very close I'm just kind of going to copy your label and


maybe I'll adjust it a bit into the direction of view who are also pretty


close a bit down so I might classify myself as this what would a linear


regression learned by gradient descent do on the other hand you have the same


data points it would start out with a line like like this any you know any


old line will do randomly initialized and then it would calculate sorry it


would calculate the gradient and important in this paper we're always


talking about full batch gradient so no stochastic gradient descent which always


means that we always in every step consider the entire data set so here we


ask this point and this point says well maybe line you should you should come


down a bit to the right and then this data point also says well maybe you


should come a bit to the right and this data point says well maybe you should


come a lot to the right so that line is going to shift to the right and ever so


slightly it will arrive at sort of this optimum right here whereas the data


point on the bottom here says well I'm pretty fine then this data point says


you should probably go up a bit and this one says you'd probably go down a bit so


the line just stays at the same place that's gradient descent now we're going


to connect the two and in order to connect the two we have to introduce


these path kernels right here these are very connected to neural tangent kernels


which I'm an absolute noob at but if you know that you already sort of know


what's coming so we need this quantity right here which is the path kernel as


we said in kernel machines choosing the kernel is the name of the game and the


goal of this paper is to show us that if you choose your kernel like this then a


neural network or any model learned by gradient descent is a kernel machine


with this particular kernel okay so first of all we need to understand what


that kernel is so what does a kernel do a kernel measures how close two


different data points are now you can measure this in in many ways right but


here we need a very particular way of measuring how close two data points are


so what might be a bit special to you is again consider a model that we learn


using gradient descent such as this linear regression example we start out


with a line that's too steep and we slowly come down right to the line that


is the the optimum line so what we've done is we've started with w0 and we


slowly ended up with W and they call it W final right here okay so during that


time the weights took a path if we draw the weights over time right first they


were too high and then they came down and now they are is still positive


but they sort of converge at this level okay that here amounts to a path so the


the weights took a path during learning the interesting thing in this paper is


what we need to do is we need to consider the entire path from beginning


to end so usually models only store you know the the converged optimum but here


we assume right we assume we have a model that's been trained by gradient


descent okay and that model has a history the history of gradient descent


where we start out at w0 and we go a path which is this curve you see right


here to W final so imagine that during gradient descent we have stored along the


way we've stored every single step of gradient descent now in this paper we


consider infinitely small steps but just imagine you know at every step we


actually stored the model during training okay by the way this is not a


training procedure that we're describing here right we assume that we've already


trained the model using gradient descent and now we have the trained model and we


want to see how similar are two data points okay so okay so let's say we have


a we have a data point how do we classify it for that you need to consider


these quantities right here which is the gradient of the function of y with


respect to W so remember before we said X to Y to the loss okay that's everything


now usually usually X to Y is F our neural network and that has parameters


W so usually what we do is we consider the gradient of the loss function with


respect to the weights okay that's what you usually do in gradient descent so it


connects it connects the weights right here with the loss function right here


essentially it says how do I need to change the weights to make the loss


change a certain way okay now this quantity here is different it only


connects the weights it connects the weights to the W right here so if you


see this thing W of X this is the same as F of X right so Y is a function of X


so this quantity essentially says if I change my weights how will the the


output of the neural network change not the loss how will the output change


that's kind of a sensitivity measure okay so imagine you have a neural


network right with with a bunch of weights a bunch of layers how and you


have two data points X 1 and X 2 these are training data points and you have


your new data point X now you want to know is it similar to X 1 or X 2 so what


would you do in this particular case what you do is you forward propagate


both of these data points not to the loss but to their outputs okay so if if


your neural network let's consider this as our linear regression example and


let's consider not the not the beginning not the end but let's consider a model


sort of this model right here okay and you have two data points X 1 and X 2 and


we want to look at not the loss right we don't we want to look at if we use the


model to output the data points as so what's the gradient how how if we change


the weights either in this or in this direction how does the output change now


for this data point right here you can see if we change the line a little bit


the Y value isn't going to shift as much because we're very close to the origin


however for the data point up here the Y value is going to shift more for a


given amount of shifting the line so the this is going to result in a number


right X 1 will have gradient I don't know like 3 and X 2 gradient of so it's


gradient of Y with respect to W will be something like 9 okay and now the


important part is we input X so we input X and we also get a Y from the model no


we never consider the labels here so we have Y right here X right here we also


use it to predict and now we ask if we now consider the same thing we now


consider gradient of the output of this particular X with respect to the weights


what is it and here you can see the point I've drawn also is fairly a lot


away from the origin therefore it's it its output will shift a lot if the


weights shift so maybe that's 8 ok so now you can see that by this number we


can now classify the similarity you can see 8 and 9 are much closer than 3 and


8 ok so two data points in this view are similar if if changing the weights of


the neural network changes their outputs in a similar way right so the outputs


here can actually be vectors and so on if you want and what you what you do is


you consider the inner product between these gradients no sorry it's not that


the output can be vectors actually the weights are vectors right so you want to


know how you need to change the weight to affect a particular change in the in


the output yes I was I formulated it the wrong way and in linear regression it


ends up being the same thing because you only have one parameter but usually you


have lots of parameters that means you get a vector as this gradient and you


consider the inner product of these vectors as your similarity so what does


it mean when two vectors are similar of these gradients it means that if I for


data point X if I change my weights in a certain way how will that affect Y or in


other in other words if I want my Y to go up what way do I need to change the


weights now it's correct so for this data point if I want the the Y value to


go up how do I need to change my weights to achieve this right over here it's the


same right if I want my Y to go up it's just the inverse like I need to change


the weights if I want to go to go up by one unit I need to change the weights by


one ninth and here by one eighth I don't need to change the weights much to make


it go wild because it's so far away from the origin however here I need to change


my weights a lot more like by one third in order to make the output move all


right so if for two data points they need similar changes to the weights in


order to affect the same change in output they are considered similar okay


they they have a similar effect on the neural network dynamics and here you can


see this in action so for a given weight configuration we input all the three


data points into the neural network we evaluate these gradients of the output


not of the loss of the output with respect to the weights and we compare


that gradient of the three data points it the new data point will be closer to


one of them than to the other and that's how we evaluate similarity now what does


this path have to do with this so as I said here we've simply chosen a model


right we can we don't have to do this for the final model we can do this for


any model and in fact what we're going to do is if we have a new data point so


remember that our model evolved from this down here to this if we have a new


data point we're going to rewind time and start out at the beginning with the


first model do this measurement like compare our data point to all the other


data points for this a model then we're going to advance one step and we're


going to do it again and advance one step and we're going to do it again and


we're going to consider the similarity scores over as an average over that path


so that means in order to classify a data point in this way as I said this is


not a practical algorithm in order to classify a data point we're going to


retrace the path of weights that the model took during gradient descent when


it was learned we're going to retrace that along the path and for each step in


the path we're going to compare our data points effect on the neural network so


the neural networks sensitivity to our data point and we're going to compare


that with the neural networks sensitivity to all the data points in


our training example and then we're going to classify our data point by


whichever data points in the training example had a similar effect on the


neural network over the course of training okay so we're not going to


train the network more or anything we're simply going to replay the path we took


during gradient descent and by looking at how the data points affect the


network during that path in terms of their gradients like how much they pull


on the network even though we're not going to do the steps by those poles we


classify how if two data points are similar or not and that is called this


path kernel so we have the most important quantity we have already if


you made it through here good job so here we have the tangent kernel


associated with function f so f is going to be a neural network w our weights x


is a data point and parameter vector V is going to be the inner product of


these two gradients so two data points are close in the tangent kernel if the


gradients of those data points align so if the inner product is high okay and


that's the tangent kernel and the path kernel now is simply the tangent kernel


integrated over the path over any path so this is not even gradient descent


yet we can do any curve but the curve we're going to end up looking is the


curve that gradient descent took during training of the model so I'm going to


look across the whole path of gradient descent we're simply going to integrate


these tangent kernels which gives us sort of an average an average tangent


kernel over the course of training now theorem one is the main theorem it says


suppose the model y equals f w of x and f is a differentiable function of w


that's a neural network fulfills all of that is learned from a training set x i


with y star i right so we have M training data points by gradient descent


so we learn it by full batch gradient descent so each and every step we're


going to consider the whole training data set we're going to consider the


loss with respect as an average over the whole training data set of x i so x i


will give rise to y i through the neural network and that's going to be compared


with y i star and that's going to be our loss and if to differentiate the loss


with it it says right here with a differentiable loss function which can


be in regression it can be the square loss right so the loss function is a


sum here as you can see so this is what the neural network predicts and this is


what you would like to have and the loss function simply compares the two and the


learning rate epsilon then then in the limit of infinitely small steps and


that's that's something you do it in order to be able to do continuous


analysis so it just think if we if you take small enough steps then y equals


this thing right here which is exactly the form of a kernel machine notice that


this and this are now connected ok so that thing here this is f w of x so that


the theorem essentially says that the the neural network can also be


represented as a kernel machine where k is the path kernel associated with f w


of x and the path taken by the parameters during gradient descent ai is


the average loss derivative along the path weighed by the corresponding tangent


kernel and B is the initial model ok so the important thing here is that this K


is going to be this path kernel we just considered and the path that we're


looking at is the path taken by the parameters during gradient descent we


need all of those things


ok so we're going to the proof and the proof as I said it's fairly simple it's


fairly straightforward and it gives sort of an idea of how does connection come


to be so first of all we're going to consider what does gradient descent do


right if we rewrite the equation of gradient descent we can see we can come


to this so this is one step of gradient descent and we're simply considering the


difference between two steps now the difference is exactly going to be the


gradient because that's going to be the steps and here is the step size as we


let the step size go to infinitely small this of course becomes a continuous


function so this is where the gradient descent comes into play we're saying


that the way our weights change over time right this is the way our weights


change over time is always in the direction of the negative gradient of


the loss function right that's that's the continuous form of gradient descent


now it says this is known as gradient flow now we're going to consider a


different quantity namely how do the neural network outputs change over time


so as we already said right


no like we didn't already say this how do the neural network outputs change


over time well I can simply I can simply use the chain rule here to expand this


into the following quantity so how do the neural network outputs change over


time that's the derivative of the output with respect to each of the weights so


this is this is over number of parameters I'm going to sum sorry over


each of the parameters and then how do these weights change over time okay so


how the neural network output changes over time is defined by how the weights


change over time and how the output reacts to those weight changes over time


and it's a it's a sum with with in accordance to the rules of total


differentiation so now we've already seen the quantity on the right here right


how do the weights change over time well they change according to the loss


gradient okay so we're simply going to replace this here by what we established


before so each weight changes according to its derivative from sorry according


to the loss derivative with respect to that weight this is where gradient


descent enters the proof now what we can do is we can apply the additivity of the


loss so we know that the loss is always an addition or a mean or a sum over the


training data so now we're going to bring that in okay so the loss here this


one we're going to split that up into its components since the loss is a sum


over the individual losses that means the gradient of the loss or the


derivative is also a sum of derivatives and again the chain rule we know that X


goes to by means of W goes to Y goes to L you can if you have a gradient of L


with respect to W you can decompose that as the gradient of L with respect to Y


and then the gradient of Y with respect to W you young kids know this as back


propagation so that's exactly what we're going to do right here I'm going to split


that up with the chain rule so now we have two quantities the first quantity is


how does the loss change with respect to the neural networks output right and


that's pretty simple like this is for linear regression this is when where the


loss is the squared norm difference or the squared then this the norm of the


difference of two wise so the derivative is simply going to be something like the


true label minus whatever the neural network outputs and the other quantity


right here is how does the output of the neural network change with respect to


the weights so if I change the weights of the neural network right X if I


change the weights a little bit how does the output change over here this is a


quantity we've already seen I hope I hope so right okay meanwhile we've we've


pulled out the other quantity right here and you might recognize it as the same


quantity note that this here this why I means that it's a particular training


data point whereas this Y is the actual point we are trying to predict for


a given input okay so now we simply rearrange a bunch of terms and look at


that look at what comes out so over here we rearrange this what you see is some


over the number of parameters again that's the number of parameters and


here why won't you see this here is if I incorporate the sum this is the


gradient with respect to the weights of f of X and this here is the gradient


with respect to the weights of f of X I right because it's the IF training data


point and they are multiplied right the sum and the product means that's a dot


product so this is exactly this path is kernel the tangent kernel okay this is


the tangent kernel with respect to a particular set of weights W okay at a


particular time in the algorithm so at some point in this path that's we choose


a bunch of W's and that's what results right this other quantity right here as


we said this is the relatively easy quantity that simply defines how a loss


changes whenever the neural network outputs change and this is also now with


respect to a particular data point so we're going to rewrite a bit right here


so this L prime is going to be defined as that it's just a bit of a rewrite and


here this is this tangent kernel and now what we're going to do is we're simply


going to aggregate all of this so since this says how does Y change over time


during the course what we're going to do is simply we're going to start off


somewhere go along the path and we're going to aggregate all of the Y changes


during this so in this particular case you know Y goes up Y goes up Y goes down


Y goes down if we aggregate all of the changes in Y over the course of the of


this path we're going to end up with the final Y right so we're simply going to


aggregate all the changes in Y over this course which means we're if we start out


with a particular Y we're going to end up at the end so this it's a bit special


but this essentially means that if we look at the neural network at the


beginning of training right we simply if we have a new data point we're simply


going to input it into the W zero neural network right and that gives us Y zero


that is whatever the neural network would have predicted had we not trained


it and then we're going to trace the changes in Y these the dy dt we're going


to trace them over the course of the training that gradient descent has done


we're going to accumulate all of the changes in Y that would have resulted


had we input our data point at each time and what we're going to end up with is


the final Y it's a very complicated way of because we could simply input the


data point into the final model right that that will be so much easier but


we're going to input it into the start model then we're going to consider how


the output changes in each time step and that's how we're going to end up at the


final Y so yeah so as you can see now this is already in the form of kind of a


kernel machine they're going to make it a little bit more like the classic form


by actually averaging over this path kernel such that you end up with this


form right here but essentially what you can see is that this thing here measures


the distance between data points by means of retracing the steps along


gradient descent and then this thing here is the measures the loss derivative


with respect to these data points now in order to actually bring this into a


kernel form what yeah as I said they normalize by this thing but it's


essentially the same so I hope you can see that the connection right here as I


said you always want to you have a one way of measuring distance and then you


want to aggregate the values so you measure distance by how sensitive other


data points are by how sensitive other data points make the network and you see


which of the other data points makes the network sensitive in a similar way to


yours over the course of the gradient descent time and once you have the


similarities you simply aggregate their sort of opinion on the output with


respect with weighted by how similar they affect the network to your data


point all right that's how you come to conclude this proof have a lot of


remarks right here so they say this for example this differs from a typical


kernel machines in that the AIs and B's depend on X which is something that's


not now the AIs and B's are usually kind of learned but here they are actually


functions of X which is a difference to classic kernel machines essentially you


can't like in order to make this a kernel machine right you have to have


the trained neural network already so it's not like this is a new training


algorithm it simply casts these models in the way of a kernel machine and it's


in my mind it's almost like it's a super general statement it also connects it to


to boosting right here I don't even know where but down here in the discussion it


connects it to boosting and it just seems like at some point yet you can


just connect all the learning algorithms to each other because all the learning


algorithms will somehow incorporate the training data into their weights like


otherwise they wouldn't learn and I feel like we're we're rediscovering just


different methods of looking at problems now these different methods the you know


the different way of looking at a problem can give rise to new and better


algorithms because we understand the problem better but yeah it's it's in in


some way it's not a surprise it's not a surprise that neural networks somehow


store the training data because of course any learning algorithm must do so


and that's exactly what this this paper shows and it shows what the exact kernel


is you have to choose in order to make that claim solid so that was the paper I


just want to read the kind of most at some point they say the most important


point for this most significantly however learning path kernels machines


via gradient descent largely overcomes the scalability bottlenecks that have


long limited the applicability of kernel methods to large data sets computing and


storing the gram matrix at learning time with a quadratic cost and the number of


example is no longer required so makes a claim that if you want to build a kernel


machine you might as well I don't actually know what that means does it


mean you might as well find the neural network that is equivalent to the kernel


you want to build it I don't know if that just that just seems to turn out to


to mean that you should build the neural network that you like but they kind of


make the point that neural networks don't discover new representations new


features what they actually do is they discover features that the of how you


compare data points in this gradient space and they do that by means of


gradient descent and the paper it states that this is you know this is very very


dependent on how you choose the architecture so by choosing the


architecture of the neural network you sort of predispose the gradient descent


algorithm to find certain certain features to compare data points as


opposed to other features and the paper again makes this explicit by showing how


how this comparison comes about namely by means of the gradients with respect


to the weights of the output of the neural network which of course is you


know entirely a function of both the architecture and the loss function and


the data set alright so I hope you've enjoyed this let me know what you think


Predicting the rules behind - Deep Symbolic Regression for Recurrent Sequences (w/ author interview)

Hello there, today we'll look at Deep Symbolic Regression for Recurrent Sequences by Stefan


Dascoli, Thier-Alexandre Camieni, Guillaume Lompel and Francois Charton. This is another paper where


the main part will be an interview with the first author, Stefan, and I'll just briefly


introduce the paper right here for 10-ish minutes or so. So if you want to just skip to the interview,


feel free. We'll go over the paper just so that you know what's going on. And there is also an


interactive demo online where you can try it out. And it's a good place to start at what this paper


is trying to do. So in this paper, the authors care about symbolic regression to number sequences,


they have a model for integer and float number sequences. In this case, this is an example for


an integer sequence. So you can enter any sequence right here. You can see that the sequence that is


already entered is the Fibonacci sequence. And you enter as many terms as you want, obviously,


the more you enter, the more success probability the model is going to have. And what the model


will do down here is it will predict an expression, you can see it correctly predicts the expression


for the Fibonacci sequence, saying that the current element is the last plus the last last


element, and it will predict the next terms for you. And it will extrapolate the sequence that


you've input. So you can do any, any that you want. So I'm going to go one, I'm very bad at


coming up with stuff on the spot 2131415. Let's see if it can get that. So as soon as you exit


from the model, it will look at that. So the caution, which is not even sure what that operation


is, but ah, so it divides, it divides the sum of the last elements maybe by the last element,


might figured it out somehow it is, it is not really good at like if conditions. And this is


one thing we're going to talk about in the interview. But you can see it correctly predicts


the next sequence right here. So give that a try. This is this pinpoints exactly what this paper


does. It does symbolic regression for recurrent sequences. Recurrent sequences are sequences


of numbers that can be somehow expressed as a logical rule as a function of the last elements


of the sequence. There is like most sequences can be expressed like this. For example, they give a


bunch of examples right here. 12712471116. So you can see that it's always sort of plus one plus


two plus three plus four plus five and so on. Or this function right here. These are simply the


squares. So the recurrence relation actually isn't a recurrence relation at all. But it is also a


special case of a recurrence relation or this formula right here. It can get very complicated.


They have a bunch of examples right here of recurrence relations. As you can see, they can go


pretty complicated to express something like the final digit of n times n plus one divided by two,


or the final two digits of two to the n, or some maximum or anything like this. So the goal of the


model is that you input a sequence like this, and then the model will output this recurrence


relation. It will not output the numbers directly of the sequence of the following numbers. That's


what they would call a numeric model. And they also train one as a baseline, but the model would


actually output exactly the formula itself. And then you can use the formula to produce the next


elements. Now the good thing is we've all seen what happens if you train a numeric model on like


a bunch of data points. Let's say these are your input data points, you train a numeric model on


that it will perform pretty well on the data you give it. But as soon as you go like outside of


that data, as soon as you extrapolate too much away from the support base of the training data,


without very strong inductive biases, it will sort of do whatever you can't really predict it what it


will do where there is no training data. That's why also deep learning relies on on lots of training


data in covering a lot of the input space, whether that's called extra interpolation or whatnot,


we'll we'll leave it at that. But if you have a symbolic regression, and the symbolic regression


actually predicts the correct formula to match this sequence right here, like, ah, this is just


a sine wave, then you can extrapolate indefinitely, right? And because you have the correct symbolic


formula, you'll be right, you know, in all in all places. So potentially, this is a very, very strong


method for certain types of problems. This paper considers this a sequence to sequence problem.


So it considers transformer stacks. And this is I guess, along the classic transformer stack


of you have an encoder and a decoder stack, the encoder stack gets fed with the input sequence


as numbers. So here, one, one, two, three, five, and so on. That is the input sequence, it is fixed.


And then the output sequence is the formula that you want to predict. And they predict the formula


in reverse Polish notation of the prefix tree of the formula. So they have an example down here.


For example, the cosine of 3x can be expressed as this as cosine of multiplying three by x.


So you would you would sort of load it onto the stack and then work your way down the stack


in in this reverse reverse Polish notation measure. So that would be cosine of mole of three


of x, or whatever that formula is. And then you try to train your transformer to autoregressively


predict first the first token without seeing those tokens. And then once you have the first token,


you want to predict the second token given the input and the first token, there is like,


there's multi head attention in here, like, there is cross attention over here, there's self


attention in here as well, your regular transformer stack. So this is classic sequence to sequence


problem. The only question is how do you obviously encode the input and the output,


the output we've already discussed, and they have a very detailed description of how they produce


the data. So what they do is they take a bunch of operators, you can see them in this table,


in this table, and they make random formulas from those operators, they have a bunch of constraints


on these formulas, but essentially, they make random data set out of just random formulas.


So first of all, they sample the number of operators between one and a maximum number.


In this case, that would be 10. 10 is the maximum number of operators. And then they build a unary


binary tree with that many nodes. So they for example, they would sample two operators right


here, like there are three, a relu, a sub and a mod. And then they would build a unary binary tree.


So relu, then that is a unary thing, right? So it only has one input. So sub, that's a binary


operation. So it needs two inputs. Here, let's say mod, that again needs two inputs. So the second


step is to sample the nodes of the tree from the list of operators. Okay, that's what we've already


done. We've combined steps one and two, sample the recurrence degree between one and d max,


d max is six. So we're maximum allowed to look back six elements into the past. This is kind of


a Markov condition. You can say your recurrence relation can only look back six items. That's kind


of a limit. But most sequences that humans could come up with don't refer back to the seventh last


element, right? There is usually a way to express it in forms of either the current index or the


last few like three or four elements at max, then they sample the leaves of the tree. So the leaves


of the tree are either a constant with probability p constant, these all these probabilities are one


third, and they stress very much that hyper parameter settings are not very crucial in this


way, they sample the leaves of the tree. So either it is a constant or the current index or one of the


previous terms of the sequence. So let's do that. So we'll say here we sample the previous term,


which is u n minus two, here we sample the index, which is n. And here we sample a constant, which


is three. So that would result in the formula relu of u n minus two minus, and then n mod three.


That will be the formula for this. And they need to sample initial terms of the sequence. So in with


the formula, you also need to decide, you know, how the initial terms the initial terms, since we


go back two elements, we probably at least two elements at the beginning of the sequence. So


let's call that one and two. That's we also need to sample that from a distribution, you can see


here, that's just a uniform distribution from negative 10 to 10. And then what's the last


sample the sequence length and compute the next L terms. So now we say, okay, how much leeway do


we want to give the model to infer the sequence, let's say we want to give it five elements. And


now we use the formula to calculate the next three terms right here. Alright, I tried it,


it didn't work out, but it is a rather complicated sequence, I have to say. But now you see how this


stuff is sampled. So you see how the formulas are made, they just define a maximum depth of maximum


length and so on. And then it just sample random data from that they create a data set, the data set


would be this one right here, this would be the input and the output to predict would be the


formula in reverse Polish notation, it's a sequence to sequence task. And that's it. Now during


inference, they can do a beam search, they can input again, the sequence, they can output different


formulas, different, they can start out different formulas, and then they can do a beam search and


check which of the formulas actually match the input sequence that they have already. And they


can discard or rank down formulas that don't match the input sequence on the first few terms.


So that is an additional benefit they have from this symbolic regression. Ultimately, they will


end up with a formula that probably fits the input terms, and hopefully is simple enough.


And the simplicity comes from the data set. Since shorter sequences are more likely to be sampled


and longer sequences, the model is implicitly biased towards easier formulas, which kind of


plays into Occam's razor. So that's it. That's the method they created a data set, massive data set,


they train on random formulas train train to predict them from the initial terms. And then


they evaluate it. As I said, they also have float sequences, but I won't go into that too much.


Notably, they do outperform this numeric model, the numeric model simply tries to learn


the number to number sequence just directly without going to the symbolics. So as you can see,


the symbolic method is better when evaluating on indistribution sequences when evaluating on out of


distribution sequences. And here's a question of how do you even do that? There is this database


of integer sequences. And after a bunch of filtering, you end up with a validation set


of 10,000 sequences. This validation set are human made number sequences like the Fibonacci sequence


or anything essentially that where humans can come up with some sort of logic of how the sequence is


generated. On this data set, they don't perform as well as the numeric model as you can see right


here. So the numeric model outperforms the symbolic model. But there are good reasons why


that might be. And we also discussed this in the interview. Lastly, they also make two experiments


with robustness to noise, which are also very interesting in that they can even suffer from a


bit of noise if they train with the noise. And so the model is even a bit robust, they can still do


symbolic inference, which classically, if you have a symbolic system, these are usually not that


robust to noise because it's more like hit or miss. But if you train appropriately, you can


handle that. Also interesting is that they encode the numbers not as continuous values in the


transformer, but actually as tokens. So at least for the first 10,000 numbers, they are all their


own tokens. So the number 19 and the number 20, they're just two tokens. But it turns out that if


you train the model, then in the embedding space, the tokens will actually form a sort of continuous,


not necessarily line, but a continuous manifold in the embedding space, which is really cool to


see that the model, even though you give the numbers as different tokens, it learns to map


them out according to their numerical values. They also have investigations into the similarities


between embeddings, and they uncover some interesting structures, where similarities


are also according to the numbers like common denominators, and so on. And they give a bit of


evidence that there seems to be kind of a natural base for mathematical operations of multiples of


six and 12. And they say that six is a natural base for reasoning, reminiscent of much earlier


explanation by other people. And you might know this cult of people, I don't even know what they


they're called, but this cult of people that says we should just switch to base 12, because it makes


everything easier. So there might actually be, you know, stuff behind that, or it might just be


a artifact of how we do math, who knows, they experiment a bunch of stuff with expression,


simplification, and so on. But the model seems to be quite robust to any of these modifications.


I think this is really interesting work in that symbolic inference, I believe, can lead us forward


and tackle problems of extrapolation that we aren't necessarily going to be doing with these


numeric models that we currently have. Obviously, this has its own limitations and its own biases


built in. Most notably, how you construct the data set is very, very crucial to how the model


is then going to perform. But it is interesting to see that you can train it like this. And


essentially, it's a, you know, it's a it's a free free training data because you can just generate


it by yourself. So without further ado, I want to jump directly into the interview because we go


over the important aspects of the paper. Again, let me know if you like inter like interview content


like this, I think is super duper helpful. And the interview was very fun. I hope you find that as


well. All right. See ya. Welcome, everyone. Today I have with me right here, Stefan Daskoli, who is


the first author of the paper Deep Symbolic Regression for Recurrent Sequences. Stefan,


welcome. Thank you very much for being here. Yeah, pleasure. Bad timing to have COVID, but I'll try


my best to... Yeah, I hope this goes I hope this goes over relatively smoothly for you. But yeah,


but yeah. So this paper, I have to say it gathered quite some hype online, right? And because


symbolic mathematics is something that is still still even though computers are very good at


math per se at numeric, symbolics is something that has been maybe in the human domain a little


bit more, especially these kind of sequence guessing, right, it seems to be a very, very human


thing, something you would do maybe in high school to try to like figure out some sequence and figure


out the rules behind it. What sort of what prompted you to go into this direction in the first place?


Like, why do you why do you think this is a fruitful direction? Or, you know, what made you


come up with an idea? I know there's some previous work, but you know, why this? Yeah, so as you say,


I mean, this kind of problem is very common, like IQ tests. So that was definitely one of the


motivations. So originally, this project was born from Francois and Guillaume, who have been both


working on papers first, so basically, deep learning for symbolic math for a couple years.


And what they've been exploring is several directions. The first one of them was a paper in


2019, called Deep Learning for Symbolic Regression, where they basically did symbolic to symbolic


manipulations, basically just integrating functions, solving ODEs and stuff. And then more


recently, Francois has been working on a numeric to numeric task involving math, which is basically


doing linear algebra. So taking a matrix and then outputting its inverse or stuff like that.


And so a natural continuation of this was to start from numeric data and go to a symbolic formula.


And that's basically symbolic regression, which means you take a function, you only see its values,


and you have to try and infer the expression of the function. And indeed, it's kind of surprising


that this has been studied quite a lot for quite a few decades, actually, this symbolic issue,


this symbolic regression question, especially with like genetic algorithms and stuff like that. But


there hasn't yet been in the machine learning literature, a paper working on sequences.


And as you said, it's a very common setup for us humans. And so this is originally the motivation.


And so Francois came to discuss with me and Pierre-Alexandre. Pierre-Alexandre is more from the


reinforcement learning background, which is also relevant to sequences because you have basically


sequence of states. And for me, it's because I came from the physics background. And this is also


symbolic regression is useful also for physics for like inferring laws, etc. So yeah, that's kind of


how we got together. Cool. Excellent. And just so we're clear to anyone, the kind of sequences we


talk about, we have a bunch of examples right here. So that would be, for example, here, the final


the final digit of n times n plus one divided by two, that's kind of the formula of all possible


pairwise connections in a group of n points. Or is that n times n minus one? Yeah, the sum of


integers. Okay. And from that, we just want the final digit. So this the sequence here is 0136051865.


That is, it is, it is, I would call it pretty complicated, if you just gave me this as a human,


but there is some kind of a rule behind it, right, that I can figure out. And that's the type of


sequences you would you would get. This one is actually a good example. It's kind of hard to


recognize for us. And if you look at the formula that the model gave us, you can actually figure


out why it predicted that formula, it's un minus one plus n. And the reason for that is that nn plus


one divided by two is the formula for the sum of integers. And so the way it built this formula is


just to take Peter Stern, add n, and then take the modulus respect to 10, because that gives you the


final digit. So it's kind of a clever thing that, you know, would be kind of hard to figure out for


us. Yeah. So if you if you could maybe give the pitch of your model itself, like the pitch of your


paper itself, just before we get into more of the details, it's always super interesting to hear from


the people themselves describing something like a brief pitch of what you did here.


Yeah, so I think our starting point was less ambitious than what it came to. So we originally


just started off from the this sort of thing that that is quite popular for math lovers,


which is the OEIS database. So the online encyclopedia of integer sequences where you


have all sorts of sequences, you can play around with them, you can try and guess next. This is


quite fun to play around with. And the idea was to try and build a model which could complete


the sequences, so sort of understand the logic behind these sequences. So originally, we only


started off with integer models. So we only wanted to predict integer sequences. And we actually


realized that that was pretty easy. Pretty quickly, we managed to get a model working on integer


sequences. And so we then started to think about, can we do the same thing for float sequences,


which are a bit more challenging because you have more freedom in the expressions you can build,


you have more operators, you have cosines and exponentials that come in. And so this is how we


sort of, I'd say it was a lot of serendipity really in this work. We started off with this


integer sequence problem. And then we figured out things as we were going on. So as you can see on


the two tables you have there, the constant approximation thing, which we may discuss a


bit later, was one of the fun side effects of trying to guess sequences. It's that you actually,


the model actually learns to do stuff it wasn't trained for. And so yeah, I'd say the goal of the


paper isn't to provide a model which is useful for real world data. It's not going to be able to


predict stock market or weather forecast, et cetera. It's more of a proof of concept of what


you can do with transformers in terms of math. And you specifically restricted yourself to


recurrent sequences. And I think it's important to point out sort of what kind of inputs does your


model take and what kind of outputs does your model give, right? Because a formula like these,


they are written down in many ways. There's ambiguities and I would guess the inputs are


these numbers right here, right? So our model gets this as an input and then it somehow has to predict


the corresponding formula. So the training data is also like this. How does it take the input


and in what form does it output stuff? Okay. So those are like the two big questions. So maybe


we can start with the inputs. So that's actually quite a tricky question. How do you feed in these


inputs to the model? Because typically deep learning models don't take like, if you think


of a sequence, which is like an exponential, you're going to have very huge numbers if the


exponential has a positive sign and very small numbers if the exponential has a negative sign.


And so if you just feed these kinds of values into a deep learning model, it's not going to learn


much, especially that here we're dealing with a transformer because essentially what we want to


output is a mathematical formula, which is just like basically a language. And so this is why we


use transformers. And so transformers need to take in embeddings. And so we need somehow to represent


our input numbers as embeddings. And that's complicated because of course, integers just like


reals are an infinite set. So you have to somehow find a way to encode them as a fixed vocabulary.


And so this is where we really have to distinguish our two setups. We basically have two different


transformers, one for integer sequences and one for float sequences. So the integer model, what it


does is basically it writes numbers in a base b representation. So for example, for the number like


yeah, exactly like here, 325, you could imagine writing it as 3 to 5, in which case you only need


10 tokens, which is numbers between 1 to 10. Actually, it turns out that it's better to use


a larger base, because if you use a larger base, well, you're going to have a bigger vocabulary,


but you're going to have shorter sequences. And typically, you know, transformers have


quadratic complexity, they struggle a bit with very long sequences, which is why we prefer to


use a large base. Here we use 10,000 as our base. Yeah, so this would be base 30. And obviously


in base 10,000, I think it's important to note that every single number from zero to 9,999


is its own token, right? The model has no inherent knowledge of, you know, three comes after two and


four comes after three and so on. All of this has to be learned. It seems so weird to say, you know,


it is better to make the model learn essentially the entire ordering of 10,000 numbers, rather than,


you know, providing that as some sort of a just to make the sequence a bit shorter, right?


It's funny. Did you ever think of going with continuous values, right? Because the first,


my first intuition would be that I feed the actual number, right? And then it's implicit,


like it's in the number that two is larger than one and three is larger than two.


Exactly. Yes. So that's what's really interesting is that that is one approach. And actually,


we had a couple of discussions on this, like how can we feed in our inductive bias on numbers


directly into the model? And well, I mean, the problem with this is that here we're dealing with


just one dimensional vectors in some sense. Transformers need high dimensional vectors


as inputs. And it's not obvious how you represent these numbers in a high dimension, you know,


because as I was saying just before, the problem is that these numbers have very


vastly different scales. And you know, deep learning models usually take normalized inputs.


And so it's not obvious how you would, so what you want to do is basically map these numbers you


have onto a sphere. And it's not obvious how you would encode, you would put these numbers on the


sphere. And so one very simple way is just to put them randomly on the sphere and let the model


decide all by itself how to put them in the sphere. And this is what we do. And what's


interesting is that when you plot after training what the embeddings look like, you can see that


it has learned in some sense our inductive bias of putting the numbers in order, etc.


So these are t-SNE plots right here. The left would be the integer embeddings. And it sort of


forms this string. What do you make of the t-SNE plots here? Do you think these things are actually,


you know, uniformly on a sphere? Or does the model just use like a tiny part of the sphere where it


can make sort of a continuous path? Well, what's for sure is that the, it's definitely a low


dimensional representation, because you can see that the t-SNE is actually very, really shows a


smooth pattern. Usually when you plot t-SNEs of like word embeddings in NLP, it's going to be a


bit messy, like you're going to get clusters, but it's not going to be as well organized as here. So


clearly the embeddings are lying somehow in a low dimensional manifold. And so then you could think,


okay, so why do we need like 512 dimensions if it's only using a small amount of them?


But that's actually because, you know, the transformer is going to eventually use these


extra dimensions to perform its calculations really. So it's not as if they're wasted,


they're actually going to be used by the model. Yeah. And the float embeddings are very similar,


right? In that you encode them as like a sign, a mantissa and an exponent. And again,


the mantissa, if I understand correctly, same deal that you have a token per number between zero and


10,000. And the exponent, is that correct that you have, you say you have exponent from negative


100 to 100. So one token would be e minus 100. And then another token would be e minus 99,


e minus 98. So these are all different tokens. So now the transformer has to learn kind of two


different embeddings. Both are somehow in sequence. Exactly. Yeah. So just to summarize,


so for the integers, we encode the integer as the sign followed by tokens of the base


B representation of the integer. And so for floats, we also have the sign token. Then indeed,


we have the mantissa token. So here the difference is that we only have one token for the mantissa.


We don't have like a base B representation, which means that we do lose some information in the


discretization process. And then indeed to represent the scale of the number, we use an exponent embedding.


And that indeed goes between minus 100 and 100. And so here indeed, we do plot the t-SNE of the


exponents because they really have a logic to them. For the mantissa, it's less obvious. If you plot a t-SNE


of the mantissas, it would look a bit anarchic. But here the exponents you can... And actually,


just about this plot here. This plot is actually a tiny bit disappointing because we can't see


some of the really interesting features we had with our first models. This is with the very


big, big model with embedding dimension 512. Actually, when we were using a smaller model


with a smaller embedding dimension, we saw a really neat pattern, which was basically the


fact that the model was learning the arithmetic properties of integers. So it was basically


creating a line with 2468, 10, et cetera, then 369, et cetera. And here it's a bit less obvious,


probably because the big model was learning something even more complex that we can't


interpret as easily. If you go in the appendix, you do see actually a figure where we see


that the model learns like a base six representation of integers.


The attention plots you mean?


Actually, not those ones. Yeah, those ones exactly. Like if you zoom in a lot on the left plot,


you see these diagonal lines which are spaced out to every six and every 12,


showing that basically the model is recognizing numbers which have common devices and is


specializing to the base six or 12 representation, which is often considered better than the base 10


representation. So these plots, just to make it clear, these are the cosine similarities between


each of the tokens. So the tokens would be distributed on the axis here. These are tokens


and these are tokens. And then we plot the cosine similarities between every two tokens. So naturally,


obviously every token is going to be very similar to itself, but also very similar to


its immediate neighbors. So it seems to really learn this, the ordering of all the tokens.


But then also, yeah, what I found special, there is this structure of the common factors,


common divisors between the tokens. That's really cool.


One thing also that's hard to see in this big model, but which was much clearer in the small model,


is you could see, for example, the perfect squares would be complete outliers. You would get


9, 16, 25, 49, which would completely stand apart due to their special properties.


I think that here is 49, right? That kind of stands out, right? This gap.


That's something which we haven't really been able to understand. Some guy sent me an email actually,


saying, oh, maybe I have an idea that there's a gap between 46 and 48 because 45 has lots of factors of 5 and 3,


whereas 48 has lots of 2s. There must be some explanation or maybe it's just something due to


optimization. It's very hard to know. I think at this point, it's a bit also important that


we look at the data generation process. You give the model a bunch of options to generate


sequences and these are, where do I have them? So here we have the operators that it can use.


On the left-hand side are the integer operators and then the float operators would be in addition


to the ones on, or sorry, they're repeated in part, but also there are more in the float


formulas. Then you just generate in reverse polish notation, is that correct?


Exactly. You generate reverse polish notation formulas given these things and you can also


have integer prefactors for all the things. Either you sample integers or you sample


the current element index or you sample previous elements of the sequence. So the model could


express, if it's the fifth element, take the current number times the previous element plus


two times the cosine of something, either a constant or again referring to some previous


element or something like this. Is there a logic behind why you chose, why you made these choices


of how you generate these formulas? Actually, if you look at this table, indeed,


there are much more operators for the real case, the floating point numbers, but you do notice that


in terms of binary operators, there are two which you can see in the integer setup, but you don't


see in the float setup, which are integer division and modulus. This really illustrates that we're


trying to learn rather different things in the two setups. Really in the integer setup, we're


focusing on sort of arithmetics and arithmetic properties of numbers, whereas in the float setup,


we're really interested in, let's say, a more classic symbolic regression problem with complex


operators. As you said, our generation process is basically to build a mathematical tree, so a


unary binary tree. This is like previous works by Francois and Guillaume. Then indeed, we fill in


the nodes of these trees, either with operators, so the nodes are filled in with operators, either


binary or unary, and then the leaves of the tree, indeed, as you said, can be either variables or


constants. As you said, the choice of generators is actually basically the hardest part, let's say,


of this problem, because one thing that's nice when you do these kind of symbolic math problems


is that you basically have an infinite data set. Your data is just synthetically generated,


and so you can train as long as you want. You don't have any overfitting issues. You don't have


to regularize that much. Even the hyperparameter choices aren't that important. What is really


crucial here is how you build your formulas. That's what makes the problem, I think, really


quite fun to play around with, because it's a bit like teaching a kid how to learn math. You really


have to figure out what is the best thing to show the model at what time. You want the day set to


be kind of hard so that it can deal with complex cases, but if it's too hard, it's going to learn


more slowly. It's really an interesting problem, how to generate the data. You decided just by


playing around. You do have, as we said, these particular ingredients. You can always say,


why didn't you have more or less and so on, but you have a table of a bunch of operations that


you can do. You decided as well to allow the model to use these sort of recurrence relations,


to allow the model to say, not only I want 5 times n plus 2, but maybe I want 5 times n plus


n plus 2 times the previous or the time step two steps back or something like this. Is there a


reason behind including these recurrence relations? Is that just something you thought would be more


interesting or did you look at the database and see that's a lot of how these sequences are made?


It's true that often people look at the problem they want to solve in order to choose the


parameters of their generation. For example, sometimes people use different weights for


how to sample which operators to sample. They'll put more additions than multiplication.


Here we have, for example, if you go right to the left here, we have these hyperparameters for our


generator. For example, you can see here the probability of choosing a constant leaf or index


leaf, so n or the previous term. Probably we could have tuned these parameters somehow, but here we


really wanted to have the simplest choice possible on the rationale that basically our data set is so


huge that eventually we're going to see all possible formulas at some point. It doesn't


matter that much the specific values we choose and we don't want to tune them to a specific problem.


This is why we really chose very standard, and also for the operators, we didn't use any particular


probabilities with which to sample such and such operator. We just let everything as general as


possible. This is built up as a tree because naturally you can parse these things as a tree.


You can generate them as a tree to have the correct grammar, but ultimately you end up with,


as we said, this reverse polish notation, which is a sequence. This would be one such formula.


You wouldn't have x, but you would maybe have n or something like this. But ultimately this


results in a sequence of tokens. The input to your model is these numbers encoded in tokens,


and the output is a sequence of these symbolic tokens. Did you also investigate the


embedding space of the output vocabulary? Yes, actually a good question. We did look at that,


and actually it didn't have any particular structure. You could have expected maybe cosine


and sine are going to be close to embedding space. I think what's happening is that the


output space is actually much smaller, because in the input space we have a lot of tokens.


Like we have for integers, we have one to 10,000. That's like 10,000 words. So it really tries to


find a structure in the inputs. For the outputs, we only have a very small vocabulary compared to


usual NLP tasks. We only have about 30 operators. Essentially, if you look at the high dimensional


space and you do it in TS and E, you won't see much, because it's just equally spreading


these operators in other spheres or something like that. There isn't much logic to it here.


Let's say, how universal are these sequences? How many sequences that I could come up with


freely would be inside of the scope of your model? Is there a significant class of sequences that


your grammar could not express? With this unary binary true representation,


you can pretty much represent any function. Of course, there are some sequences which don't have


any logic to them, which aren't generated by a recurrence formula. In which case, you can't


represent these sequences. That typically is the case with most of the sequences from the OEIS


database. We had to get rid of quite a lot of them and do some filtering. I did say that you


can represent any function, but there is a limitation. Some functions are very difficult


to express with this tree approach. If you think, for example, of the Collatz sequence,


where basically for odd numbers, you multiply by three, add one, and for even numbers,


you divide by two. That's a rule which is possible to express with a mathematical expression.


Essentially, what you do is write it as n mod 2 times what you do if it's even plus one minus that.


That's an involved way to write it. Generally, the model is going to struggle to output that


because it won't have seen it much during training. That's one important thing also,


which we might discuss a bit more. Our model is kind of biased to the likelihood of the expression


to be generated during training. It's like a hack that we as programmers have for an if condition.


It's just something we learned at some point. If you have an if condition, you can express it as


if you, I don't know, people program NumPy or something like this. That's exactly what you do.


You don't say if. You make your mask with one minus whatever condition and you multiply by this,


and then you have that. I think anyone who programs NumPy or TensorFlow or so on knows,


okay, I can do it like this, and then my stuff is expressible and differentiable as one formula.


But I think that's a hack we learn. If we just generate data at random like you do,


this is not something you come across as often as we come across when we program.


Exactly. It's very unlikely to see this formulation in our data sets. Absolutely.


Okay. Cool. But at the end of the day, you generate a giant data set. You go through


it with transformers and you emphasize transformers. Is there something special