learn from those controller inputs.


[ML News] OpenAI removes GPT-3 waitlist | GauGAN2 is amazing | NYC regulates AI hiring tools

A few challenges that arise there is that usually humans steer very abruptly.


And this user has purposefully as you can see here try to only steer super duper smoothly


such that the model has a better target distribution to learn that is not as noisy.


At the end the model is able to learn the track that it has been trained on.


And interestingly, it also can drive a little bit on tracks that it hasn't been trained


on though not all of the tracks.


So if you think this is cool and you want to learn more go over to Stack Smashing's YouTube


channel and check out the video.


I'll link it in the description.


NBC New York writes New York City aims to be the first to reign in artificial intelligence


hiring tools.


This is about new legislation in New York City that would ban employers from using automated


hiring tools unless a yearly bias audit can show they won't discriminate based on applicants


race or gender.


They compare this to another rule that the city has enacted that restaurants have to


display a calorie count with their menus and the article here goes into the detail of what


the advantages and disadvantages are and that some people think that it doesn't go nearly


far enough.


Now the whole crux of the matter here, of course, is that what does this yearly bias


audit contain?


What does it mean that you won't discriminate based on an applicant's race or gender?


You can interpret this very strictly where if the model doesn't have access to the applicants


race or gender, it cannot possibly discriminate based on that.


Yes, the argument usually goes that there are correlates to race or gender and models


very often make decisions based on those correlates.


However, what's the definition of based on on the very other end of the spectrum, you


can essentially say that any system that has any disparate outcome whatsoever with respect


to hiring fails this yearly bias audit.


It's interesting that with such a simple piece of legislation, you can get into very deep


discussions about nature versus nurture, what is fixed about people, what isn't, how are


decisions made even in humans?


And what does it mean to make a decision based on something?


I mean, there are a lot of interesting questions to be had right here.


And I'm pretty sure none of the people who actually passed the ruling have ever dived


into it.


It just sounds good.


Oh, yes, let's make a rule AI systems cannot discriminate based on race and gender.


That sounds good.


Think of the children.


The article also says that a good outcome of this is a part of the legislation that


says that the company has to disclose if it uses automatic systems to screen you.


I'm not sure what you're going to do with that as an applicant at the end of the day,


I guess the question is, you know, of course, we all feel the kind of disgust being evaluated


by an AI system and then being rejected for some arbitrary algorithmic rule.


But I'm not sure like we seem to all pretend that HR personnel is a lot different.


It's not like an HR person that has a stack of 1000 resumes for like three positions is


going through each of them deeply delving into the applications and really grappling


with every person individually.


No, they're going to look at it school, I don't know, gone, bad grades gone gap in whatever


year something gone.


I mean, I feel we're comparing AI tools to unreachable master standards.


Whereas I think what we should be doing is comparing them to what's already there.


And what's already there most often isn't working either.


Now the people that criticize this, they say that is not going far enough say that essentially


the bill was watered down so that it effectively just asks employers to meet existing requirements


under US civil rights law prohibiting hiring practices that have a disparate impact based


on race, ethnicity, or gender.


Oh, no, how terrible.


You're only asked to comply with the law.


I mean, that is a shame.


Clearly, this isn't far enough, right?


If you're interested, check out this article and tell me what what you think about these


questions.


Just drinks.com analysis, which beverage companies are leading the way in artificial intelligence?


Yes, that is that is what I needed in my Pepsi just just a bit more AI in that can like,


Oh, wow, the drink is now also a recommender system.


Yes, please.


Apparently, after putting your coffee through the portafilter Starbucks now also forward


propagates it through a convolutional neural network before serving it to you.


Or maybe they use RL to finally get customers names, right?


Who knows, but it lets me sleep well at night to know that the beverage companies they're


really on this AI stuff because it really like that is going to make the difference


here.


DeepMind, Google Brain and the chess champion Vladimir Kramnik have published a paper called


the acquisition of chess knowledge in AlphaZero.


They investigate AlphaZero.


I've previously made a video on AlphaZero about what AlphaZero learns about chess, and


it's quite interesting.


So the paper is fairly lengthy and investigates not only how AlphaZero things, but also what


are the overlaps with how humans play chess?


How are human concepts that you know, that grandmasters pay attention to when they play


chess?


How are they represented in the AlphaZero system?


And are they represented at all?


So they do a lot of different analyses, which is really interesting.


And they also have an accompanying website where you can investigate a little bit into


that stuff.


For example, they have different non negative matrix factorizations of the different board


positions, non negative matrix factorization is an excellent tool where you can see how


different components additively combine to form certain structures.


They also let you select given board positions and then track how the different systems react


to that board position, and what continuations there are.


And you're able to compare AlphaZero during training right here with humans over the years


since 1985 ish.


So the assumption here is that humans have gotten better over time.


And maybe we can compare a little bit new strategies that were discovered by humans


with new strategies that AlphaZero discovers as it becomes better using self play.


Now I've investigated this a little bit.


And honestly, I haven't found really a big overlap here.


But I'm also not super good at chess.


So don't take my word for it.


Alright, some helpful things for this week, there is a rudely, which we previously reported


about it's a Russian version of Dali that is trained on emojis.


Now you might think that is ridiculous to which I would respond to with a crying face


emoji.


However, the results are actually pretty cool.


Like look at this for St. Basil's Cathedral looks pretty neat.


There's Donald Trump from Lego, a human eats an apple.


I mean, given that people already use emojis a lot when texting, you can totally imagine


a future where you cannot just select from the emojis that are given to you, but that


sort of emojis would be created on the fly.


And maybe you could choose from 10 emojis that are conditioned on the sentence you just


wrote and then you can select among those seems pretty neat.


Honestly, I know it doesn't solve world hunger, but could be useful.


Open code blocks is a project that is similar to Jupiter notebooks, except that you're able


to connect cells not linearly, but as a graph.


If this data format flourishes, it's no longer necessary to tell people Well, first, you


got to run cell one and then cell two and only run cell three.


If you want this run cell four twice and so on this format abstracts all of this into


a DAG if I understand this correctly, and you can then run these cells individually


or you can run like one strand of these cells seems pretty cool.


The project is quite young.


So if you want to get into this, you have to be ready for us kind of like alpha version


software, but it might be a very, very cool project to contribute if you're into tooling.


TensorFlow has a new library for graph neural networks.


Now TensorFlow has made a bunch of attempts previously at graph neural networks and related


things.


I remember things like TensorFlow fold and stuff like that.


But this now seems to be a pretty sophisticated library for doing graph neural networks.


So you're able to define various architectures and then run your message propagation algorithms


in a way where you can also back propagate through it.


Examples show how to build easy graph neural networks given predefined functions on edges


and nodes and also how to build graph neural networks that have custom functions for that.


So pretty cool.


Check out the GitHub repo.


If you're into graph neural networks and you're using TensorFlow, this might be a very good


library for you.


Keep in mind that this is also an alpha release but should get better in the future.


PyDreamer is a torch implementation of the dreamer v2 reinforcement learning algorithm.


The original dreamer v2 is implemented in TensorFlow and this is essentially a port


to PyTorch.


Now the features differ somewhat and the implementations differ somewhat.


So the results aren't exactly the same, but it could be a cool baseline if you want to


experiment with dreamer like reinforcement learning algorithms.


You can see right here sometimes it does better sometimes it does worse than the original


dreamer implementation but I guess that's just reinforcement learning.


So if you're interested the project has quite an extensive readme to get you started.


Have fun.


CodeGenX is a model that takes in code and spits out what more code you should write.


Pretty simple.


It's a little bit like GitHub Copilot.


However, the difference is that it is open source.


There's GitHub repo.


It's based on GPTJ and there is a VS code extension.


You can get a free API key and start using it right away.


The website is a bit bare bones right now but looks pretty cool.


Other than Copilot it currently supports just Python though they say they are planning to


add additional languages in future releases.


So very cool project.


Go check it out.


And here from DevPost this is another submission from the PyTorch annual hackathon.


This is the Heyo camera.


Now it currently only exists for Mac but this is a camera plugin that recognizes hand gestures


and then displays appropriate reactions.


So this person is happy.


This person is not happy.


This person raises their hand.


Very excellent.


This seems a bit gimmicky but the sort of recognition of gestures of course cannot only


be used to display simple emojis but can be used to trigger various other things.


So again there is a GitHub page you can download and install it for Mac if you want or you


can continue developing it.


And our last story for today IDW Online writes the Einstein Foundation to present the inaugural


500,000 euro award for promoting quality in research and the award in part goes to the


founder of archive.


So the individual award worth 200,000 euros goes to Paul Ginzpark professor of physics


and information science at Cornell.


In 1991 he created the archive a document server for preprints on which scientific findings


are published without review and paywall restrictions.


Archive has become by far one of the most valuable tools especially to the machine learning


community and it's pretty cool to see its creator recognized for putting this out there


as early as 1991.


That is crazy.


Excellent work.


Thank you.


All right this was already it for ML news this week.


I hope you had fun.


[ML NEWS] Apple scans your phone | Master Faces beat face recognition | WALL-E is real

Apple scans your phone for illegal content, master faces are able to bypass almost any


facial recognition software and Wally is real. Welcome to ML news. It's Monday.


All right, before we get into things, this video is sponsored by weights and biases,


weights and biases is of course the one stop shop for any machine learning researcher or


practitioners. Weights and biases can track your experiments with a single line of code,


it lets you reproduce and analyze your experiments, it lets you understand your data,


it's with you all the way from conception, idea, research development up until deployment. Today,


I want to talk to you about a feature called sweeps. Now a sweep in weights and biases is


a hyper parameter optimization search, if you will, the cool thing is you define your experiment,


you define the range of parameters you want to search over, and then the system does the rest


for you. You can even run this in a distributed fashion, you can have lots of agents at lots of


different places, they all going to pull the code from the central server, pull the new hyper


parameters, try them out, and then report back. In the background, there is a Bayesian optimization


algorithm going on deciding what parameters to try next to optimize your objective, they even have


early stopping so you don't waste resources on runs that are clearly going nowhere. And have I


mentioned you can run this in a distributed fashion. So here's one of my sweeps. As you can


see, you get your output as you're used to from weights and biases in a neat dashboard, you get an


overview over all your runs. But in addition, you're able to see the progress of the sweep,


you're able to see which ones succeeded and which ones didn't, it will analyze directly how important


each one of the parameters is individually. So here it tells me that the learning rate is the most


important parameter. And it has a positive correlation with my objective function. One of


the coolest views is this one here that tells me which of the combinations of hyper parameter ended


up at a certain place. So I can filter for runs with particularly low validation loss. And then


I can see what are the learning rates, what are the epochs like in this particular runs. Now,


there's obviously much more you can do in terms of analyzing sweeps, you can run this much larger,


you can look at individual samples of your best runs, pretty much everything you're used to from


weights and biases. So if until now, you've tuned your hyper parameters manually, try this out,


let it do the work for you go to bed and in the morning, come back to find the system has found


the best possible hyper parameters for your problem. Not only is it easier, but you'll


understand more about your problem. Once you see it in this light. Of course, this is only one of


the features of weights and biases. They have many, many more including ways to analyze your data


ways to export your models, ways to keep track of everything that you're doing and ways to send


reports around to other people or generally work in teams. Personal accounts are free with unlimited


experiments for you. If you're an enterprise, that'll cost a bit of money. But hey, you're


an enterprise. And there are free options for academic teams. There are even options to self


host if you need to be compliant with any sort of regulation. So give it a try go over to weights


and biases. That's one DB, I think at least that's how you pronounce it. One DB.ai and have fun. Ciao.


All right, our first story today is not a particularly fun story. TechCrunch writes


Apple confirms it will begin scanning iCloud photos for child abuse images. This has caused


quite a bit of stir in the community, especially since Apple had all these adverts in the previous


series about what happens on your phone stays on your phone was very privacy related, and to end


encryption friendly and all of these kinds of stuff. And now all of a sudden, it seems like


they're going to scan all your data for things they don't like. Of course, it's not a case in


favor of child abuse images or any kind of illegal content, people are worried about privacy more


generally. So I think it's important to say what exactly is going to happen here. Or at least from


what we know, Apple will scan your photos that you are about to upload to iCloud. As I understand it,


iCloud itself is encrypted. So Apple technically has no way to scan the iCloud photos because


they are encrypted with your key that rests on your devices. However, they can scan content that's on


your phone. I'm going to guess there might be a legal reason for it in that they might sort of


kind of be responsible for that content once it goes to their online service. However, that's not


something I know. But of course, once the technical methodology is in place to scan the photos that


are about to be uploaded to iCloud from your device, you can use the same technology to


essentially get access to any data of any user. There's no technical limitation after all, why


only these photos should be scanned. And just because Apple promises that it won't do it doesn't


mean they won't do it in the future, or they can't do it. And that already tells you a little bit


why some people say it is a problem. Because of course, there is also no technical limitation that


says that it can only scan for child abuse images or any sort of illegal content. And for that, it's


a little bit important to dig into what the system actually does. So the way this works is there's no


classifier essentially in there to classify child abuse images from non child abuse images, there is


a database. So the police essentially collects databases of these materials, which means that


those are individual photographs or movies that are sent around by certain people that are illegal,


and the police keeps track exactly of the files that go around. So this is the first important


thing they only want to detect if you on your phone have one of the files that they already


have in their database classified as illegal content. And the way they do it is by comparing


hashes. Now traditionally, a hash would only match if the file is exactly the same bit for bit. So


what you do is your phone would download the database of hashes would hash all the photos on


your device that are about to be uploaded to iCloud, wink, and then it would compare those


hashes to the database of bad hashes. And if one matches, it would upload it to the police.


Alternatively, it could just hash all the contents, upload that to the police, and then the police


could do the comparison. In any way, if these are actually true hashes, they're unlikely to reveal


what data you have on your phone. And that's likely the argument that Apple is going to make right


here in that just because you upload the hashes of what's on your phone, you can't necessarily


reconstruct the images from that. So your personal photos are safe, even more so if your phone


downloads all of these hashes, and then compares them locally and only sends if in fact, there is


a match. However, there are multiple problems with this. First of all, you don't know what's going in


this database. Technically, some political party could simply enter things into that database that


they know are likely the opposition or some rebel group is likely to share around amongst themselves,


they could even instigate such material, and then they could just wait and see what phones blip up.


So you confiscate one phone from your political opponent, you run all these hashes, and you put


them in the database, and all the phones of the associates of that person would then be


automatically reported by the system. So the potential for abuse here of the people who


control what's in the database is enormous. Second, as I understand it, the hashes that are


used right here aren't like classic cryptographic hashes, they are what Apple calls neural hash.


But what is an effect a locality sensitive hashing algorithm. So here's an article by Tyler nail on


about locality sensitive hashing, which explains the concept fairly well. And it makes sense to


use a locality sensitive hash in this case, because what you want to detect is if two images are the


same, meaning display the same thing. For example, if I take an image and then run some sort of JPEG


compression on it, it still shows me the same thing. However, the bits have all changed. So a


classic hash would not be able to recognize that image anymore. However, a content aware hash would


or should at least be able to recognize that this is the same image. YouTube has been doing this for


a long time with their content ID system detecting when someone re uploads a video by someone else,


even if that video has been re encoded. So as far as I understand it, what Apple does is they train


some kind of neural network that gives them a representation of what is in an image. And then


they run that through a locality sensitive hashing procedure. locality sensitive hashing is essentially


a system that allows you to find neighbors in very high dimensional space very efficiently.


So the neural network would produce a space of images and place each image somewhere with the


intention that images containing similar or the same thing would fall very close to each other.


And you can do that with neural network. The question is, you don't want to run an inner


product search over this whole space all the time, like that would fry your phone probably.


So what locality sensitive hashing does essentially, it divides up the space into


buckets. So here, it's straight buckets. And then these kinds of buckets. Once you combine all these


buckets, you get sub buckets. So you get sort of a division of space. And for each point, you can


check is it to the left or to the right of a particular line. And if two points match in being


to the left or to the right, or up or down respectively, for any particular line, that means


they're in the same bucket and probably very close together. At that point, then you can actually go


ahead and check if they are actually close together or not. This is a good way to find approximately


nearest neighbors in high dimensions. So real LSH algorithms are a bit more sophisticated, but that's


the essential concept they work by. So is this going to help? Well, I would say yes, in first


instance, but then I think very, very quickly, you'll realize that adversarial attacks, for


example, can be crafted against these kinds of systems. Given that the system computes the hash


on your phone, that means you have access to the model on your phone. And having access to a model


is a very, very, very good target for crafting adversarial attacks. Technically, there could now


be an entire market of systems that perturb images on your phone automatically, such that they just


scramble the LSH because most of these hashes aren't going to be in the database. So if I just


assign my image some random hash, meaning I run an adversarial attack such that it is just going


to be somewhere in this space, most likely, I won't hit any of the hashes in the database.


And therefore, all my photos are not going to cause any hash collisions. And therefore,


I completely evade that system. Now, the question is, of course, how easy is this going to be,


especially given that it is supposed to circumvent detection of illegal content,


there's going to be a bit of resistance, but there's definitely quite easy ways it seems


to circumvent this system. And we have to ask ourselves, are we really ready to give up basic


privacy? Are we really ready to let the companies build in these giant backdoors that have massive


potential for abuse for what is essentially a method that can be pretty easily evaded when


it's used for what it's really supposed to be used for? I don't have the answers, but I would


err on the side of user privacy. So that's my take on it. Tell me what you think in the comments.


Alright, a quick afterthought here, we now also have the technical summary of Apple, there's a


lot of content in here, notably goes into a lot of detail on how exactly the technology works,


what neural hash is supposed to do. For example, you can see that the left and middle image have


the same neural hash, whereas the right image does not have the same neural hash. So the neural hash


is supposed to be robust to certain transformations that you might do with the image while still


preserving its content. Therefore, as I said, you couldn't just compress the image or change its


color saturation a little bit and evade the neural hash. Apparently, though, after the neural hash


is computed, there is also this blinding step, which means that it essentially goes through a


classic hash function. And therefore, the adversarial attacks on the system become a


little bit more difficult. Now, since this is all still on the device, it's absolutely possible to


evade the neural hash using an adversarial attack. What is less possible is to frame someone meaning


that you send someone an image that is specifically crafted to hit the neural hash filters as illegal


content, but is actually just kind of a normal image that you have adversarially crafted. Now,


with an untargeted adversarial attack, you can evade the filter. But if you want to trip the


filter, you really need a targeted adversarial attack. And because of this blinding step,


you don't know what to target. So the only way to actually craft such an adversarial image to frame


someone is if you yourself already have an illegal image that you can target with the adversarial


attack. So there's a lot more in this technical report right here. And I invite you to read it if


you are interested. And I might actually do a full video on this if this is interesting enough


to people. It's not necessarily machine learning. It's more cryptography and systems design,


but still is pretty cool. All right, while we're on privacy, the EU Parliament approves


mass surveillance of private communications from the European Pirate Party writing today,


the European Parliament approved the e privacy derogation, allowing providers of email and


messaging services to automatically search all personal messages of each citizen for presumed


suspect content and report suspected cases to the police. European pirates delegation in the Greens


EFA group strongly condemns this automated mass surveillance, which effectively means the end


privacy in digital correspondence. So this sounds kind of the same, but it is slightly different.


While Apple announced that it will do something. This is simply the EU saying that you can do


something. However, what you can do now seems to be a pretty big breach of privacy. Now, of course,


just because companies now are allowed to do something doesn't mean they will do it. But


probably it means they will do it. So yeah, but what are you going to do you signal? Well,


then just Apple swoops in and scans your messages before you send them. So I guess we'll just go


back to sending pigeons around. Alright, on a bit on a lighter note, I stumbled across this


book by our Rachel Wong that explains machine learning as answering two basic questions. So


this company's a machine learning class and explains machine learning in the essentially


answering FAQs. So this is a big FAQ of that class. And it's quite good. It's explained very


concisely what do embedding layers do embedding layers converted token and integer to a vector,


a list of floating point numbers. That's fairly concise. And then you say when do you use embedding


layers when you want to process text text can be converted to integers, but because neural networks


are don't directly understand integers. A bit of a typo here. I guess could I change this?


I can make a pull request. suggest edit for check cool. I was pretty stupid. And actually the


recording you're seeing is the second recording. In fact, I forgot the first time to record my


screen. And what happened is pretty funny in that. So I was I was presenting this book,


and I actually saw a typo in the book. And then I immediately opened a pull request and


fix the typo and the pull request got approved. And I was like, yay, ML news and all. And I thought


that will make for some pretty good content. And I was really happy with myself. And it was really


neat and all. And then I realized I forgot to record the screen. So now I'm just going to show


you a compilation of me being absolutely self congratulatory for finding a typo have fun.


Good job, ML news community. We did something. Give yourselves a pat on the shoulders. This is


this is unplanned. Yeah, ML news improving the world story by story. So as you can see, it is not


entirely thorough or particularly technically accurate or anything like this. If you're a


beginner, if you're new into a particular subfield of machine learning that's treated here, this might


be a good place seems fairly concise way to learn about the fundamentals of given subfields.


Okay, we have some new data sets coming out to data sets by Google, both are for NLP,


especially for conversation. One is called time dial, and it tests the models understanding


a sort of the sequence of things whether or not it understands the flow of time. And especially


if the participants in the conversation talk about things that happen one after another,


if the model can correctly infer things about this. So here you can see what's the date today


today is September 28 2007. I have a meeting this afternoon, when will it begin? It'll begin at three


o'clock. What's the time now? And then the model is asked to fill in this blank, it is something


something and then continues after go now I don't want to be late. The model says don't worry time


is enough. What's the most likely filling in the blank so you'd have to reason okay meeting is this


afternoon, it will begin at three yet after that it says okay, I have to go now but time is enough.


So maybe it's a bit before three, you know, not like one to three or something like this, but


also not the day before or so. So out of the four options you have here, the first ones would be


okay, because they fit the constraints. The last ones would not be okay. And in fact, in this


absolutely not cherry picked example, I'm sure the T five both T five and BERT assign most mass


to the last examples. The data set is essentially made up of all kinds of these conversations and


giving you options to fill in and you have to determine the ones that fit the constraints


most. The other data set is called this full qA and tests disfluent questions. So it takes the


squad data set, which is a question answering data set and it rewrites it into questions where


the speaker just kind of turns around mid question or corrects themselves or insert something or


says like, Oh, no, that's not what I meant. I meant this other thing. And this can get quite


complicated because you can start with an entity and then say, Oh, no, no, no, no, no, but then


still refer to that entity when you rephrase your question. So the data set is supposed to test the


models abilities to handle that data sets like this in general are pretty cool because they test


the best sort of human aspects of conversation. However, state of the art on these data sets is


probably going to be reached by models that just heavily overfit to whatever the problems that


data set construction mechanism is. So if you evaluate things on these data sets, what I think


should be done is you should just train like your regular model without these things in mind, and


then evaluate on them as sort of one of the things maybe we can add those to to the superglue suite


or something like this, which gives us a more accurate picture than simply releasing them and


then have a leaderboard for them. That's just my opinion. In other data set news, Facebook research


releases vox populi, which is a speech data set. So their speech data from the European Parliament


event recordings, some of them are even annotated or translated interpreted into other languages. So


very big data set unlabeled and labeled speech data. So if you work with speech, this might be


something interesting for you. Next news, Google tensor debuts on the new pixel six this fall,


Google tensor apparently is some sort of hardware. I don't know this is a giant marketing piece. It


just says the Google tensor chip will make everything very, very fast and machine learning


and the new UI and they don't necessarily edit will actually say anything about the chip. So


your phone is going to be able to do numbery numbery, crunchy, crunchy way faster than it


used to be able to do it. That's all I can say for now. The Pentagon believes it's pre cognitive


AI can predict events days in advance. She learning could help the military make proactive


decisions rights and gadget. So this is an article and it sounds a bit like out of a


dystopian movie, but apparently the US military has very large efforts into using ml to sort of


predict a key situations that are about to happen. And once you read into it, it's apparently not


that different from what they've done so far. So far, they just had like a whole bunch of people


analyze all kinds of satellite imagery or emails from people that they just found on their computer


like people sent it to them their private emails, that's why they can read them legally. And they


just had all these people go through all this data essentially manually, maybe with some assistance.


And now AI is supposed to just be able to go through this data a lot quicker and flag any


information that might be relevant for the human reviewers. The technology itself seems fairly


neutral and actually pretty useful in certain situations. Given that it's the military using it,


it might have a bit of a bad rep. But again, it demonstrates that most technology doesn't really


have a sort of moral underpinning by itself. It's mostly in most cases about the deployment of any


type of technology, like you could use the same thing to predict days or minutes or hours in


advance when ICU patients will become unstable, people actually do it. And the underlying core


technology is not going to look very different from what is done here. So researchers from MIT


and CMU release sketch your own GAN, which is a paper and the method in the paper is essentially


you take a GAN that you have trained on some sort of data set here, for example, on a cat data set,


and you're able to additionally input a sketch, as you can see right here, and the system will


adapt the GAN such that the outputs sort of match that sketch. Of course, there's quite a number of


hyper parameters in here, a lot of engineering decisions. But in essence, it's a pretty, pretty


cool way to control the output of GANs. And this is quite a hard thing to do. And it's not entirely


clear how to do it. A lot of people research sort of disentanglement of features in GANs. So you


could control individual dimensions directly, but that kind of requires you to have either a data


set of these individual dimensions, so you can actually really take them apart, or you just end


up with some dimensions, you have to figure out what they are, in order to control seems like a


pretty cool thing, you can give the GAN a sample. And in this case, not even a sample of real data,


you can actually give the GAN sort of a steering direction directly of what you want it to output.


So I can see this has many more applications beyond images and sketches. Technically, you could


apply this to a lot more stuff where you need to control the output of a generative model by some


sort of demonstration, which doesn't even necessarily have to be in the same space as the


things you're trying to produce. So overall, very cool. Check it out. Next paper that caught my


attention can a fruit fly learn word embeddings by a whole consortium of researchers of different


labs working together on this paper. Now, it's clickbait, let me explain that the paper itself


is actually pretty cool. So we understand fruit fly brains fairly well, they're approximately like


this. Now when I read the title of this papers, I want to see a fruit fly learn word embeddings,


or at least an attempt at doing these kinds of things. However, it turns out that the paper


constructs a sort of abstract model of the fruit fly brain and then shows that that abstract model


can in fact, learn word embeddings much like the word embedding methods that we know from NLP.


Again, the research itself is completely valid and very cool. I was just sort of caught out by how


important a title of a paper is because had it been for a different title, technical title,


I probably would not have clicked on it. So the lesson is if you're trying to get people to read


your paper, a good title can go a long way. Okay, the last paper that caught my eye is generating


master faces for dictionary attacks with a network assisted latent space evolution,


this by the Blavatnik School of Computer Science in Tel Aviv, and by the School of Electrical


Engineering Tel Aviv, this paper essentially uses evolutionary algorithms. And I love the Darwin in


this picture. Just to make clear, we mean Darwinian evolution and not Lamarckian evolution, hashtag


Nola mark. So this paper constructs what they call master faces. And apparently just these faces


just 10 faces. So each of these rows are these master faces, just these faces combined are able


to match a vast number of facial detection algorithms. So what that means is if I go out,


and I encounter a facial recognition system to like let me into a door or into a phone or anything


like this, I can just try out these 10 faces. And there is a high likelihood something like 40 to 50%


that one of them will actually work, which is insane. This shows sort of the brittleness of


the identification part of these facial recognition algorithms. The potential for abuse for this is


large, like someone could get access to all the photos that you're about to upload to iCloud or


something like this, like imagine that that'd be terrible. Fix this. I would just have one


helpful library this week, Pytorch releases the Pytorch profiler version 1.9. So this seems to


be a rather major upgrade that includes distributed training view, memory view, GPU utilization view,


cloud storage support and jump to source code, which replaces the old feature of walk to source


code. Well, in any case, if you use Pytorch, and you ask yourself why your code is so slow,


maybe try giving the Pytorch profiler a look. Next news 0 AD is getting reinforcement learning


capabilities. This is a strategy game that is kind of popular with some people. The cool thing is


that it has now a direct interface for reinforcement learning, meaning that it exposes an API that is


essentially compatible with the gym interface that you know from basic RL. So they even go through


setting up some sort of a task for you with these five spearmen fighting against these five cavalry,


and they take you through training a DQN agent and then evaluating it directly in their game. So if


you're interested in reinforcement learning as it pertains to controlling games, maybe this is a


good topic for you to dive in. And the last news Yahoo News right beach bot rover uses artificial


intelligence to clean up cigarette butts. So apparently there once was an engineer whose son


dug up a cigarette butt at the beach and the engineer looked around and saw all kinds of


cigarette butts lying around realized that they're quite bad for the environment and also not very


pleasant to step into. So he teamed up with his friend and build this thing called beach bot or


BB for short. So this is essentially an incarnation of Wally it goes around and automatically picks


up cigarette butts at the beach. How cute is that? How neat. So it does that fully automatically. I


think the bigger goal here is to sort of develop AI and robotics applications for sustainability.


The project in itself is not going to save the world here they writes it can scoop up about


10 cigarette butts with its grippers within 30 minutes and it has to recharge about once every


hour. So pretty much it's out competed hopelessly by a single chain smoker. But what can I say? It's


very, very cool. But I think such a robot could be better used to actually go and just poke people


who smoke at the beach in the first place. So BB will get a companion pokey BB and pokey best


friends on the beach. Let's go stab some smokers and then pick up a cigarette butt.


Alright, that was all ready for this week's ML news on this beautiful, beautiful Monday. I hope


you learned something today. If you did subscribe if you did not watch the video again, then


subscribe. Please check out weights and biases and I wish you a very pleasant week. I'll see you


Author Interview - Transformer Memory as a Differentiable Search Index

This is an interview with the authors of the paper transformer memory as a differentiable search


index. I have done a comprehensive review of this paper yesterday I've released it just before this


video so be sure to check that out. The authors today have actually seen my review and we'll


dive right into the matter. During this interview, you will not only learn much more about the paper


itself but also the research project itself what went well what didn't and what the authors think


of the future of the field. This is super duper interesting. It's an absolute pleasure to interview


all of these people and that's possible because of you so continue to let me know in the comments


what you think how I can make this content better. Thank you to everyone who shares out


these videos to everyone who's part of our Discord community to all these supporters on


Patreon and so on. And without further ado, let's get into the video.


Hello everyone today I'm here with Yi Tei and Don Metzler, who are authors of the paper transformer


memory as a differentiable search index which I find really cool, really inspiring, very creative


and very happy that you are here. Welcome to the channel. Yeah, thanks for having us. Thanks for


having us. This paper is a bit special, right? Because it takes a little bit of thinking outside


the box, I think to overcome or to arrive at the conclusion, hey, let's just store the entire data


set into transformer weights, or you can frame it in whatever way you want. But it is not an obvious


idea. How did you get the idea that you want to try something like this? Yeah, so maybe I'll just


share a little bit from my point of view and Don can go next about his thoughts. So I think for


from my side, I'm really interested of like, you know, understanding like this is more of like


understanding the properties of transformers and you know, how many documents can transformers encode


in the parameters and you know, like, and then obviously, right retrieval is a good way to test


whether a model is able to like generalize like what and digest what he has like, you know, encoded


in memory. So I think from my point of view is more of like, you know, trying to see what


transformers are capable of and pushing the limits of memorization. And yeah, so I think that's from


my point of view, one of the reasons why we thought of this at the start. Yeah, maybe Don can share


some some thoughts as well. Yeah, yeah. So I'm taking just a you know, sort of a step back.


This paper is somewhat tied to this paper that we published sometime last year called Rethinking


Search, which laid out kind of our vision for you know, how we can bring the latest and greatest in


machine learning, natural language understanding to bear on sort of information retrieval problems.


There's been a lot of, you know, interest in in this space recently. And so one of the things


that we talked about in that paper was this, I mean, essentially this idea, you know, how to


essentially take these large language models that exist, which understand relationships between


sequences of tokens, and imbue them with an understanding of documents, right, because


usually these sequences of tokens come from documents, right. But I've never seen anyone


explicitly model that. And so from my point of view, you know, sort of more as a kind of IR


researcher, and you know, it's great that, you know, Yi sort of has more of the machine learning


NLP background, you know, we decided to come together and say like, hey, what can we actually


do here to study this? You know, is this crazy idea? Is this even possible? And so one of the


things that, you know, we'd hope to do is actually see if we can build like this idea of, you know,


not even like an evolution of language models that are more of like corpus type of models,


right, where you have documents now, and in these types of models, potentially not, we didn't do it


necessarily here, but in the future, right, you can have models that actually understand relationships


between documents. And, you know, we established this, okay, how can you, you know, model relationships


between tokens, sequences of tokens and documents? But I think you can take this sort of a step


further. And yeah, so that's kind of like the broader framing and how we came up with this.


Then also, I mean, obviously, a super cool problem from like machine learning, natural language


understanding side of things as well. I think it's been long suspected, said, however you want to call


it that transformers, especially the large language models, they essentially regurgitate their training


examples and kind of interpolate their training examples. Was this in your mind as you went about


that research? Or how does that connect to people saying, well, all GPT-3 does is essentially,


you know, kind of reproduce a bunch of its training data sets?


It is like a good question. But I guess like beyond memorization, we are also kind of trying


to test for like whether a model can make use of the memory because like if it's like, you know,


you give a model a prompt and it generates like from that prompt is like, you know, associative


memory and stuff. But like, you know, maybe understanding of documents is like maybe


slightly beyond that. And we want to like this probe this ability of the models because, you know,


if you can do zero shot retrieval here, it kind of implies that the model has, you know, understands


reasons a little bit with what he has memorized. So I guess from an ML point of view is at least


some kind of benchmark like type of task to kind of probe for this type of ability in the model.


Yeah, so I think, yeah.


Now, I had a bunch of questions, maybe technical questions about the model. So I suggest we kind


of clarify these first before we go into more the broad or the meanings behind the things.


You have this contrastive objective here that you present in the dual encoders,


and you have the fully differentiable, differentiable search index. Have you tried or


there are these things called cross encoders, right, where I input a query and a document,


and I try to predict some sort of a score of how they go together. These usually work a lot better


than the dual encoders themselves. What is the reason that you chose to not compare to any


cross encoder type setups here? Yeah, that's a great question. I can take that. So the reason


why we don't compare with cross encoders is because generally cross encoders are pretty expensive,


because you cannot like cache the documents in advance and you have to like you always have to,


you know, compute for every query that comes in, you have to always compute with all the documents.


So so there's there's some like, you know, latency and some compute cost restrictions


for cross encoders. So like within the scope of DSI, because DSI is basically,


you know, generating doc IDs, so like we kind of put that in the same ballpark as, you know,


a similar, you know, compute cost as, you know, like instead of doing a like you kind of, instead


of that you kind of decode one document. So we consider that the computer compute costs like to


be more fair than you know, having to pass through a pipeline of like, and not like, usually there's


another re-ranker that does this cross attention stuff and then that definitely improves the


performance. And I don't think that at this point of time, like we will beat a cross attention


encoder. But you know, many cross encoders are just expensive. So that's why we consider it like


out of scope for this. That makes sense. You hear you very elegantly you output just a list of


document IDs. I was wondering, have you ever tried to actually produce the document itself


that you're searching for instead of the document ID? Because I was wondering, because the model


needs to learn this association between the input and the document ID and it kind of needs to


remember what text is in that document, right? There's no other way for it to really learn


to associate text with document IDs. And I was wondering, is it a harder or an easier task for


the model to directly output the text of the document? What do you think? I think there's


a lot of challenges with decoding the text. I mean, you can obviously, you know, constrain


your beam search only generate stuff that is like within a certain like stored memory and stuff.


And then that's definitely possible. Or at least maybe the title of documents. But then I think


we have not tried it in this work. And then, you know, I think this is definitely interesting.


And it's a good point that you brought up. But I think that at least within the scope of this,


we wanted to keep the compute low. And, you know, like we have already emulated a lot of


possibilities in representing the documents. And then that will probably be a different class of


a style of documentation like that using like natural language that can be followed up work.


But the reason why is because the reason why we mainly don't explore that now is because


like there's a lot of like additional challenges that we need to think about. And so we consider


that like slightly out of scope for now. But that's definitely like a great suggestion. And


and we think that is also potentially quite viable as well.


The only other thing I quickly add here, you know, going back to your else your question about


the cross encoders. You know, I mean, these models typically have limited ability to essentially


monopon text length, right? So you're limited usually to passages or parts of documents,


right? By sort of modeling the document ID sort of as itself, you sort of open up the ability


to model larger, more complex documents that you wouldn't be able to do sort of if you were


treating everything as sequences of tokens, which again sort of the standard from the IR perspective,


it's been from my very biased opinion, very unsatisfying the move away from sort of document


retrieval to more passage retrieval that has happened recently. And a lot of that is just


because the models have not been able to handle, you know, these longer sequences like they did


before. So this takes us a little bit back to that. And, you know, obviously, if you have longer


documents and whatnot, it'd be even more challenging to potentially decode that entire document.


Though isn't that it isn't that a bit because if I think of information retrieval in the let's say


the olden days, what I actually retrieved was keywords, right? And then I simply looked up


which documents the keywords belonged to. And I had some heuristics of how I combined for an entire


document, all the keywords that were inside of it. Couldn't this also the move to passages be


viewed as an expansion rather than a reduction in the scope of what I'm looking at? You see what I


mean? Yeah, yeah, for sure. And yeah, obviously, there's always a way to aggregate from the passage


level to the document level. And, you know, I mean, this is the very standard trick that people


have done. I mean, people even did that back in the olden days when you just had, you know,


sort of keyword based indexes as well. So for sure. But, you know, then you also do have


considerations of efficiency, right? You're going to then go and have to score dozens of passages


per document that suddenly explodes the cost versus just scoring sort of at the document. So


there's definitely trade offs. Is this what this introduces is a level of redirection or a level


of indirection in what the model needs to learn. So we no longer map sentences with the same meanings


to each other. For example, we now have to learn this indirection almost like addressing a document


by a variable name, even with your semantically meaningful identifiers. Still, I believe,


a large part I as a model, I need to remember, just this identifier means something it stands


for a particular document. Do you see this applicable in maybe a broader context you already


allude in your paper that this could be part of a differentiable architecture? Where do you see


this types of indirection based models going?


Oh, yeah, that's a great question. I was waiting to talk about this because it's something I'm


pretty excited about. So like, I mean, so the doc IDs, like using the doc IDs is like, you know,


you as you mentioned is some interaction you store the information in some, you know, some


address and then later on you can just use that address as a, you know, in replace of like a long


document and stuff. So I think one possible avenue here is like, you know, like you can imagine like,


you know, like prompt tuning, you know, like this few shot in context learning might require like,


you know, like you might need to like stuff 10 prompts, 10 examples in this large language model.


So like if this, you know, memory addressing type architectures that allows you to compress stuff to


doc IDs and then, you know, you can use that as for like prompt tuning or you can use that for


like retrieval on a ommentation. So I think that there might be more use cases that can be explored


like beyond retrieval. So this is more like a fundamental, I think that you already, you know,


you got it really very accurate where, you know, it's a class of models that you use this memory,


you know, kind of addressing stuff that may have more wider application. So yeah, we are also quite


excited about that. So yeah, I really think that it can be like on top of my head is mainly like


maybe like prompt tuning or like retrieval on omitted models that could benefit from this.


But yeah, as of now we don't know that for sure. But yeah, this is just a, just a, just a


guess. Yeah. In your paper, you describe the performance of your models. And the trend seems


to be if I recall this correctly, at least if we go to the results section real quick, that the


larger models do perform better. However, the larger the data set gets, the less the improvements


of let's say the DSI compared to the dual encoders are if I understood this correctly. And


in your data sets, you're still in the realm of 300,000 documents on an for an IR problem that is


not really a large scale problem. Do you think that in the future, people might be able to expand


these models to also become better on larger document or collection instances? Or do you think


that the application of these types of things might be as you say, much more in as differentiable


component in something maybe in an infor maybe in a reinforcement learning agent or something like


this? Like how do you how do you deal with the fact that as you seem to scale the document


collection size, the benefits get weaker and weaker? Yeah, so that's a good question. So,


so we kind of think that, you know, like it gets harder and harder, like as you increase more


documents, I think that's also because, you know, the model has to like, you know, memorize, like,


you know, link link documents to like much more identifiers. So to be honest, like when


the memorizing, you know, the interplay between memorizing and retrieval is actually quite,


you know, it's quite tough on the model to learn and you know, as you can see that you need like


an XXL model to almost do this, you know, do well on this task. But I think that, you know, to cope


with larger documents, there are multiple ways, like one of them potentially is like, you know,


like sparse models, make sure experts where, you know, you can just like increase the parameter


size significantly without, you know, increasing the compute. So we think that those are also


promising to, you know, scale these models up to like maybe a multiple, a few million docs, at least.


This is like an estimate, we don't have the results yet to show this, but


this is what we think right now. And yeah, it's true that it gets harder and harder, like


eventually. So we are not sure where the limit is yet and we are also excited to find out like


where does this end and where's the limit of this. Yeah. Do you have an idea of how these


things scale? Like if I have double the amount of documents, do I need double the amount of


parameters or do I need an order of magnitude more parameters? Like is it related linearly,


exponentially? Do you have any idea of how this scales? Off the top of my head, I don't really,


like I'm unable to like put a number on it, like right now. It's mainly like, you know,


the intuition is, and it also like a lot depends on like, there's one part which is like the


memorizing capability because like, I believe that, you know, beyond this paper, we have actually


tried like brute force memorizing like a couple million documents, the model does memorize, but


then there's another, like if you need to factorize other part of like how well the model is able to


make use of this information. So like, it depends on like the data set, depends on like many


factors. So it's very hard to say, but at least on NQ, like we don't have, currently we don't have


beyond like 300K documents, but like going from 100K to 330K documents is like, it was really like


a, like it wasn't really like exactly trivial. So we expect that like going to like a 1 million


docs, like in the retrieval context will be like, okay, if I had to put a number on it, probably


like you have to probably may need to go to like 32 billion parameters or something like that.


If I had to like give a guess and estimate. Yeah. And obviously this is the, you know,


sort of standard feedback we get when, you know, people take a look at the paper, right? You know,


lots of questions about the experiments, other data sets, scaling it up, you know, I don't want


to give too much away. Obviously we're aware of this, we're working on this. We hope to be able


to have better answers to all of these questions sometime soon. And also, you know, demonstrate


that this works more than just on, you know, on NQ, on some larger data sets and hopefully have


much better, you know, sort of empirical basis for, you know, understanding sort of limitations


and scalability of these approaches. I have to ask just for, it's a, I mean,


it's a detailed question, but this NQ 100k data set, it seems to be just out of place a little bit,


like it seems like the numbers, they're just kind of off. There is, it looks really good with the


10k data set and the 320k data set. You can see, you know, things either get better or worse,


maybe as you'd expect, but then the 100k data set, it just like, for example, the BM 25 is all of a


sudden a lot better, right, than either on the 10k data set and the 320k data set. And likewise,


in a bunch of the other numbers, it's just sort of out of place. Do you have an idea of what's


going on with the kind of the data set as such? Yeah, yeah, sure. So, like, I think if you look


at the numbers right now, like one of the points that stand out the most is the bucket of the


atomic dog IDs, right? So, the second stuff, so even you look at NQ 320k, like you see a 6.9 there,


like randomly, right? So, the fact is that for atomic dog IDs, like there were a lot of training,


like instability issues that we had to overcome. So, like there's a lot of variance and a lot of


trainability issues and we tried our best to overcome those. And then, like, so sometimes you


get a base model doing better than the... It's more of like optimization and like the interplay between


like, you know, like the retriever and memorization sometimes. I mean, I think if, you know,


like coming from my experience of running like many of these like logical reasoning or memorizing


tasks, sometimes the model gets it in the end and then sometimes it just doesn't get it at the end


by the end of the training. So, like, I think there's generally like, especially for atomic


dog IDs because we initialize, you know, like the softmax layer is like initialized from scratch and


we use the pre-trained models and also depending on the warm-up and everything like so. So, it was


already a challenge to optimize for the atomic dog IDs. That's why you see generally like even on


all three sets, right? There's like a very... I think that the rest of them scales pretty like


more nicely than the atomic dog IDs, but that is actually a big challenge that we had.


I'm not sure if we actually like explicitly point out this instability issue too much in the paper,


but I think I remember like mentioning somewhere, but yeah, but at least, you know,


the middle bucket is really hard to train. The second bucket... You do mention it, yes.


Yeah, the other thing to mention, I mean, if you look at the BM25 number, I mean, that's not trained


in any way. It also obviously demonstrates very different performance there. I mean, the other


thing is just, I mean, there's variance when you sub-sample the documents, right? So, if you go from


320,000 to 100,000, you're sub-sampling, right? Maybe that was just a very lucky good set of


documents that were somehow was much more amenable and much more relevant in some way. So, I mean,


if you do this with like any sort of, I think, standard IR system, right? You just start


sub-sampling documents in different ways, you're going to get very different performance. I mean,


probably the best thing would have been to sub-sample, you know, like five or six times,


did some sort of error bars there to get a sense of what the variance is. So, I suspect that,


you know, probably it's a mix of the instability plus the just the fact that maybe this is a


lucky or, you know, sort of different sample of documents than the 320k and the 10k.


Okay. I actually have an answer about the, there's one point which is a bit implicit. It's not like,


it's mentioned, but it's like, it's not very obvious. But like, so for NQ 10k and NQ 100k,


these are sub-sampled sets from NQ, right? And then NQ 320k uses the official validation set,


right? So, there's like 10k and 100k is sub-sampled. And then like, I'm not exactly


sure how the validation set was constructed in NQ, but like, so 10k and 100k like uses like a


similar way of sampling. It's just random, but like when you go to 320k, like it's actually using


the official validation set. So, I don't know, maybe it's like, maybe a bit cleaner or like


there's some difference in the way this, so 10k, 100k and 320k came from the official validation


set. So, there might be some differences in the way we sample and how the other people sample. So,


yeah. So, I believe that you mentioned the training instabilities also at points throughout,


and that might also explain a little bit as well why different methods are good at different tasks,


right? You have, there's quite a bit of variance in which methods are better here or there,


quite a bit of variance in the numbers itself. Although what I see is very thoroughly the case


is that the larger models tend to do better in general. Whenever a model wins here with whatever


way, it tends to be the larger models that outperform the smaller models within the same


buckets. Do you think that is a property of the larger models being pre-trained better? Because


larger models also exhibit better language modeling behavior, right? And given that these are


pre-trained, I guess T5 style checkpoints, that might be an improvement. Because as far as I


understand it, your retrieval performance also in the part depends on the models being pre-trained


to actually understand language, especially the zero shot ones. Or do you think that is mainly a,


the main contributor is that with more parameters, I can memorize more documents? So could you comment


on that? And maybe also a little bit on what do you think intuitively is going on inside of these


models that they are even able to retrieve those IDs? So I think the pre-training definitely does


contribute, like I wouldn't be able to say like how many, put a number on like how many percent


it contributes to that. But I definitely think that like one way to tell is like probably to just


rerun all the experiments with like, like randomly initialized T5 style models, right? I think at


the very early stage, I mean, it's not in the paper, but we did run some early experiments with


like no pre-trained models and these models actually like, it's way harder to learn without


the pre-training. And this is, you know, it's a common finding across, you know, not only in this


context, but you know, in broader NLP and machine learning in general. So we think that the


pre-training does a lot of like heavy lifting and also the size, you know, like with a larger model,


you also benefit more from like, you know, like it's the composition of two different things


helping each other. So because you know, you are pre-trained and then you also, you know,


larger and then you benefit more from pre-training when you are for this T5 XXL models. So I think


that also probably contributes to like the zero shot and stuff like that. So yeah, just to answer


the question explicitly, I think that the pre-training does contribute a lot to this.


Yeah, I think the other thing we don't have a good understanding of is, you know, after we


fine tune, you know, on these, the DSI tasks, you know, what sort of the model, what knowledge the


model retains or does not retain, right? You know, what was the nature of the model at that point?


Others have sort of asked this question and I think it's a great question. I do suspect


that some of the knowledge that you know, sort of obviously pick up during pre-trainings is helping


as you suggested, but there may be other pre-training tasks that are even more amenable to


sort of DSI than, you know, sort of the standard T5 pre-training.


Did you, have you attempted at introspecting these models in some way to kind of see whether you can


find the documents, whatever it means in inside of these weights, like, you know, I imagine since


I can query these models and they give me a doc ID that I need to be able to go and look inside the


weights or something and find traces of these documents or something like is there something


you can say about the inner workings or is there something one can see in the attention maps or in


the weights? I have a very disappointing answer because I wish I knew like where to look in the


model as well, but the unfortunate thing is that I don't know where this is saved in the model,


is it in the, you know, in the decoder layers, but I think intuitively it seems like, you know,


because the decoder learns to like output doc IDs, I think the decoder does quite a lot of like


heavy lifting in the model, but like which weight is in and you know, like, you know, there's also


like, you know, like the fit for all layers, like key value memories and stuff and then you can,


you know, like somehow probe that. I think this is interesting for a lot, but unfortunately,


we don't know where it's safe now in the model. Yeah.


Is there, what do you think, if people want to get started with this, what do you think is like


the smallest scale thing that would still give meaningful insights into the technical part of


the model? What would be the minimum set up for anyone to get into this type of research,


like differentiable indexing and things like this?


Yeah, that's a very good question actually. So at what point where this starts getting


meaningful, right? Which scale does it get meaningful? I guess like, this is just my personal


opinion. Obviously, like this is my sense of things, but I think starting at around like


Excel, like 3B, it's probably like a reasonable scale to start because, okay, actually I don't


really know why 3B, but like this is just from my experience running the experiments because like


3B and 11B has slightly different training dynamics compared to Bayes and Larch. So it's very


hard to like characterize this, it's very latent within me. But I think like 3B, somewhere around


3B is like medium scale models, but like small and Bayes probably will not be that meaningful,


but I guess like starting from 3B would be pretty nice. Yeah. So that is not, it's not


exactly small, right? I can't really run this on my 1080 at home, but it's still, I guess,


maybe accessible to more people than just the biggest companies. Here you have a pretty


interesting thing in your hierarchical document IDs, and I understand this is not the end all be


all this is like an attempt at forging meaningful document IDs. And you make very interesting


requirements here, you have two requirements that they retain some semantics, which the clustering,


I would say gives you right, it gives you a little bit of semantic thing. But then also,


you want to reduce the search space with each with each decoding step, which is a property of


autoregressive decoding, right, the first decoding step only needs to care about the


big picture, the next one, the smaller and the smaller. Do you have an idea how much these two


things play together? Or which one is kind of the important one? Because one could also I think in


the review, I raised the issue, you could reverse this in this document ID, which would give you


the same meaningful document identifier, but without this property of autoregressive decoding,


do you have an insight of which of the two properties might be the more important one here?


And which one is or are they interacting with each other?


So we have thought like really like factorize both, both of them. I intuitively I think that


the like segmenting the search space is more beneficial, but I think they help each other.


I think this like is possible to also like come out ways of like ablating the search space.


So I think we did try to like try those.


Yeah.


If you if you look maybe a bit more high level. No, wait, I have one more question. Yeah, this L


right here, because you have this very interesting graph that shows this thing right here, which


document representations make the most sense and direct indexing. I also find it interesting that


in your paper, you try out a lot of things. And then at the end, it seems like often the simpler


things work better, which is which is a neat finding, I guess, a encouraging finding for a lot


of people. Although, I was surprised to see that if you index fewer tokens of the documents,


it tends to perform better. What's because that shouldn't be right? What's the problem here?


What's the problem that prevents us from indexing longer sequences of the documents?


So I guess this is like my thoughts on this is that like going up to one to eight and above


makes the training harder. Like we also observe this in memorization, like, you know, looking at


the training accuracy of memorization. So I think by and that's going to be like quite like some


examples. We don't know how many examples, but there's going to be some examples that can be


solved easily by the first 32 tokens or 64 tokens. So I think that the model, like, okay, this is just


a guess. I'm not really 100% sure about this, but it's like the model prioritizes getting the one


in those correctly rather than trying to fit 256 tokens and then not being able to solve anything,


right? Like even the easy ones. So I think this might be what's happening. And then like this 32,


I will not like over index on like this 64 or 32 because it's probably going to be like very data


set dependent. And also the inverted index, I saw on your review that you were surprised that


the inverted index didn't work, right? But this might be like an artifact of like this data set.


And, you know, like it's maybe the simpler approaches work here, but like when we scale up,


when we go to something like harder or more documents or just the structure of the data set


is different, then perhaps the inverted index would help. So I think that there's a lot here that,


you know, we are just showing like a slice of the data points, but like I wouldn't over index or


like, oh, DSI only works when the document length is like short or something. But like,


I think this is data set dependent and for sure, I believe that for other data sets,


you need longer sequence line. Yeah.


If you look ahead a little bit and you came into this, you told me at least that you just wanted


to know certain things like you had some questions, is this even possible and so on? My question is,


is there an end goal here? If you look into the future, maybe two, three, five years or so you


you develop this a little bit hardware gets better and so on. What's the outlook? What's the


North Star that this could lead to?


Yeah, so I'm going to share a bit and I think Dodgy will be able to answer that.


Yeah, so I'm going to share a bit and I think Don surely has thoughts about this as well. So I


will leave some for him. So I think like one of the North Star here is like, because like,


you know, like Retriever is generally like slightly decoupled away from like other NLP


tasks. Like, you know, people are unifying models. They are going for T5, everything is 6 to 6,


right? But then when it comes to Retriever, you always like have this separate infrastructure of,


of, you know, like dual encoders and then you have to compute like, you know, ranking metrics


and then the whole infrastructure is always very different from like, say like, like machine


translation or text generation stuff. So like, I think this, like, at least for me, like one,


one aspect of like, is to be able to like conveniently do Retriever in a way that like


you don't have, you don't need to have a separate infrastructure. You can just co-train your


Retriever, get all the metrics you need, get a competitive performance to like dual encoders


while, you know, like still being able to do machine translation at the same time. So


I, maybe machine translation may not be the best example, but maybe you want like, you know, some


NLU, some, some, some question answering model, like, you know, end to end or, you know, like,


or synthesizing, like from the doc IDs, you, you can, you know, like generate doc IDs together


with text and then like, you know, like maybe substantiating the text with, with, with, with


maybe doc IDs, like, you know, like learning to cite and stuff like that. So, so I think these are


like, like the, the, you know, like visions, the vision that, that, you know, I'm pretty excited


about. So yeah, maybe Dawn can chime in. Yeah. I mean, you're going back to what I mentioned at


the sort of at the start, right. This is part of sort of this, this exploration of, you know,


what's possible. And, you know, if you play this forward, we have no idea, right. What's


going to happen. I mean, one potential outcome is that, you know, it turns out that this is,


is a great way of actually modeling a lot of the things that the sort of IR community, again,


in the past is sort of modeled in terms of documents and terms and, you know, sort of all,


all of, all of this and that, you know, kind of this type of approach, you know, could, could,


you know, be, you know, the sort of a way of unifying sort of retrieval and scoring, right.


Cause you mentioned cross and coders, right. Today, usually as you mentioned earlier, right.


You have this sort of cascaded approach where you do retrieval first, and then you do scoring next.


This does everything together jointly, right. And that kind of simplifies things. And it would be


nice, I think, in the future to be able to have a way of doing that all sort of end to end in a


highly differentiable, you know, sort of way. The other thing that I mean, is obvious here is that


there's a lot of attention and interest recently with, you know, retrieval, augmented sort of


everything. The idea being fewer parameters and more reliance on sort of external, you know,


memory or storage in some way, right. This, this is kind of diametrically opposed to that.


I think there's pros and cons to both of the approaches and it'll be very interesting,


I think, to see as we continue to explore both directions, sort of what are the benefits of,


of each of these things and how maybe the two of them can come together as he was suggesting,


you know, maybe DSI can be a sort of interloop on a retrieval, augmented approach in the future.


And if you look ahead, maybe a bit more short term, what are the hardest problems that are


still outstanding to make the next steps of progression here?


There's actually a lot.


It's good, right? As a researcher.


Yeah, yeah, there's a lot of like, like things that we want to solve and there's still a lot of things


that keep me up at night. So, I think that there are a couple of like pressing ones, like how do


you update documents? How do you, you know, and then solving the trainability issue and then


solving the scale to do like, you know, I'm hoping that, you know, like going to the next


like, you know, I'm hoping that, you know, like going to sparse models, like something like


switch transformer, you can just handle like 20, 30 million docs like off the bat.


But so, I mean, I'm like, I think scaling is, is, is, is, you know, like a more short term to


mid term thing that we want to solve. So, updating, scaling, and also like the interplay between like


retrieval and understanding a little bit more about this zero short behavior and also understanding


where he's in the model, as you mentioned, like understanding this behavior of these models.


I think these are like intermediate, like next steps that, that, that I think like this guy,


you know, to, you know, take this idea further, like I think these things need to be like,


to some extent like solve for at least like, like figured out somehow. Yeah.


Yeah. And obviously, I mean, some of the questions you brought up here are, you know, things that are


actively being thought about and explored. You know, one of the things that, you know,


we were just talking about was, you know, indexing the first like 32 tokens, right? I, I, yeah. So,


so just understanding, you know, the properties of the model across more datasets and kind of like,


what are the best practices here? I think are also like very immediate term things that, that,


that we'll need to do to just get a basic understanding of this beyond kind of this initial


kind of proof of concept, if you will, that, that this crazy idea is even, you know, kind of feasible.


Is there any, anything else that maybe we haven't touched on yet that you would like


people to take away from the paper that they shouldn't go without knowing?


Hmm, that's a good question.


Nothing that I can't think of.


Yeah, yeah, yeah. I can't think of anything right now. Yeah. Yeah.


Okay. Is, can people, even if the models are large, could, could people get into this? Like,


is, is the code somewhere available or?


Are you planning to do this?


So this is like subject to approval, but we do have plans to make the code available sometime in


Q2 of this year. But, but this is all subject to approval. We have not gotten the approval yet as


of now, but this is our plan to release it in Q2. Yeah.


The fight with the lawyers. Excellent.


We have a question here.


Yeah, yeah.


We have a history of, you know, of releasing, of open sourcing, you know, many of the, you know,


you've reviewed several of our papers in the past, right? I mean, we do have a history of,


you know, being able to release the code. It's just a matter of, you know, sort of checking


various boxes and we're committed to this. We've already had folks reaching out, you know,


trying to replicate this and we want to make it easy for everyone so that they can, you know,


sort of get going with this. And yeah, I think that's a good idea.


Yeah. I think it's a really interesting area and hopefully this will stimulate some additional fun research.


Yeah, I was, I was in Google for a while. I know, I know it can be a hassle to,


to open source anything and the amount of approvals you need to get. So


props that you even like want to go through with it. It's pretty cool.


All right. So Don and Yi, thank you very much for being here. This was very enlightening and I hope


people had fun and I hope to see you again soon.


Yeah, thanks for inviting me.


Thank you so much. Yeah, this is great.


Population-Based Search and Open-Ended Algorithms

This is huge. This is just one hall and most people I guess are still waiting for registration.


Yeah, but definitely the size of these things is ginormous. So tutorials have just started.


Off we go, finding a place.


Hi, so I just wanted to give a little update on a tutorial that I liked, which was the


population-based search and open-ended learning tutorial, which happened on Monday here.


So I was pleasantly surprised by this tutorial because I knew almost nothing about these techniques


and they seem really cool, seems to be a really cool line of research.


So, started out with what is population-based search and basically in population-based search


you don't want to just reach one solution of a problem, but you want to kind of maintain a population


of solutions that you develop over time. So natural evolution will be an example of that.


So this can have many benefits that were explored in the tutorial.


So the kind of culprit of traditional optimization, let's say you have a classification problem,


you just train one classifier on it, is what they call deception, meaning that, okay, a better example


is an RL problem where you need to reach some goal, but since the goal might be very hard to reach,


your algorithm has basically nothing to go on, there's no stepping stones, so usually people go


and construct a reward function in a very clever way, but this can be overcome with these techniques as well.


So just imagine like the hardest video game in the Atari suite, this would be something like


Montezuma's Revenge, where you first need to collect some key and then go to some door


and only then you get a score, right? So this reward function is too ambitious and is a problem they call your deception.


An observation they make is if you look at nature and natural evolution, it is very successful even without a goal.


So there's no goal in mind to natural evolution except reproduction creates other reproduction,


but there's not a goal, that's simply a kind of underlying mechanism.


And if you look at nature, all this variety of life was produced without a goal in mind


and all this variety of life filling different niches and basically reproducing at their own pace.


So it's a very interesting observation, the goal of this entire field is kind of to model,


to go into this direction of what if we don't really go after only the cost function, but what if we,


so in the most extreme case, what if we build a search algorithm that only wants to create novel things.


So where kind of novelty is the only goal, what happens then?


And it turns out some interesting things can be achieved with that.


So they introduce this notion of quality diversity, which basically means if you look at, let's again take life on earth,


you want all the achievable behaviors that there are, so maybe one achievable behavior is a very fast life form


that can hunt other life forms and another achievable behavior is one that camouflages very well and so on.


And you want to kind of find for each of these behaviors, you want to find the best possible example.


So that's the direction that these algorithms go into.


And an algorithm that they presented was map elites, so MAP elites, which goes as follows.


So let's say you have a bunch of dimensions you care about, say how fast the creature is,


how tall it is, how well it is camouflaged and so on.


Now you want to discretize each of those dimensions, so this will give you cells basically.


So each of these discretization will introduce a grid of cells and what you now do is you want to keep the best examples of each cell.


So if you have a creature that's very fast but not very well camouflaged at some cell, you look at how well it's doing at the goal that you have in mind.


And you want to keep the best one of those, you have a population and whichever ones are in that cell, you keep the best.


And then you go ahead and you kind of change them, you could do this via evolutionary process like you can mutate them


or it could be via gradient descent something.


But you mutate them and I guess they will probably end up in a different cell.


So you go look at that cell, are these new ones better than the ones that you remembered from that old cell and if so, replace them.


You want to, for each cell, keep the best one and then kind of start continue developing from those.


Sort of like Dijkstra's shortest path algorithm and this will, so what it will return is like an entire landscape of possible behaviors


and for each behavior, it will give you the best result.


Now it doesn't mean they all do equally, some will be better, some cells will be not as good with regards to your cost function


but it will give you an entire landscape and you could see then that there are many kind of modes in this landscape.


As I said, some creatures are very fast hunters, some camouflage very well but then they're kind of slower.


So you'd be able to see these modes in that.


I found this pretty interesting and very kind of opens the door to a lot of different applications.


So a principle they employ is what is called goal switching, namely that means if like a line of development can benefit from inventions of another line.


So let's say the very fast hunters, they're good at that but then maybe they don't reach quite optimal performance


but then another line develops somewhere else and these are camouflaged like the camouflaged life forms develop.


So they invent kind of camouflage.


Now because these, the way this mutation and so on is, you kind of keep the camouflaged ones around and the hunters


and now the camouflage can kind of jump over to the hunters.


It's very difficult to explain like this but they call this goal switching and what it means is that the hunters can now adopt a little bit of camouflage


through let's say mutating one of the camouflaged ones into the hunters or vice versa and then can kind of benefit from that invention over there.


So a good example of that they mentioned is that in order to discover the microwave you first had to work on radar technology


which had nothing to do with microwaves but because of the inventions made in radar technology you could then invent the microwave easily.


So it kind of jumped over into the space of ovens basically.


Before all you had to make food warm was just put it in an oven and heat it up.


Now you had the microwave.


So these algorithms capture the spirit of this.


A book that the people who gave the tutorial wrote is Why Greatness Cannot Be Planned.


I'll definitely get that and I can't recommend it since I haven't read it yet but I'm going to get and read it.


It should be fairly interesting.


So they gave a number of examples of this.


For example robots that can recover from damage because they had a robot with six legs.


They trained it to move.


Now they disabled one leg.


Now usually you have one solution like you trained your neural network.


I don't think it was even a neural network but you trained your system to move this robot as efficiently as possible


and now because you only have one solution, one leg is broken, it doesn't work anymore.


But since you have the entire landscape of solutions you can easily kind of jump to other, not as good solutions if you have all legs,


but you can jump to other solutions in the solution space and try them out.


Which ones do still work if I only now have five legs?


Since you have the entire landscape you're very well able to do that so that's pretty cool.


Another algorithm they presented was Go Explore which is an algorithm that kind of solved these really hard Atari games a while back.


What they do in specific is they kind of have an archive of states that they have reached in the past.


It's a video game and you do some things and then you are in certain states.


It's an archive of states and you just pick one of that.


You pick this state, which means my little person I control is somewhere over there.


Then you just explore from it.


You do a population based, you just kind of go around from it and so on.


Then you look at the state you end up in.


If the state you end up in is a known state, like you've been there before so it's also in your archive, then you compare the two.


Did you get faster to that state via the new route or did you get faster to that state via the route that was already in your archive?


If you are faster in that state via the new route, you replace the archived one with the new one.


This again is kind of like a Dijkstra shortest path algorithm extrapolated to this kind of domain where you have to explore.


You don't actually have a graph.


I think it's pretty cool.


It's all kind of the same principle but it can employ this goal switching thing.


You go to a certain state but then all of a sudden because you explored something else, you find a much quicker way to that state, which you never intended.


It happens.


This is a basic principle that if you explore a lot, then good things might happen.


It's kind of a serendipity discovery mechanism and you could use those good things, incorporate them into the things that already work.


The last topic they covered was open-ended search.


A distinction from what they've already discussed to open-ended is now they give the example, again, life on earth.


If you consider it, it's a single run of an algorithm.


It's not that for every life form a different optimization was started and finished, optimized for a certain thing.


It's all one single run of the same algorithm and it doesn't really have a goal in mind.


Open-ended algorithms are like that.


They kind of define an interesting notion.


Is it still interesting if we were to just let it run for a billion years?


Would it still be interesting?


If yes, consider it an open-ended algorithm, which I find a really good kind of definition.


The fundamental property that open-ended algorithms have and research in this has defined is that constantly, not only is the population shifting, but also the environment is shifting.


There's kind of a never static situation. The environment is always shifting.


That also means there's always new opportunities opening up for new creatures to evolve, to fill the niches that open up.


The research community around this, the open-ended search, open-ended learning community is considering exactly those types of environments.


How can they even describe those, manufacture those, and then learn in those?


A cool experiment they've shown was the pick breeder experiment, where basically it's a human in the loop.


As a human, you go to a website, you pick one picture, and these pictures are procedurally generated.


They start out with a very simple pattern.


You just have the opportunity to pick one, and it gives you a bunch of random perturbations of the procedurally generated image.


You pick the ones that you like, and then you continue exploring from there.


If you're happy, you can just save that to the database, and someone else can look through the database and then pick yours, for example, to continue.


The things that the humans came up with, or the result of that was, was extremely interesting.


Not only could you perturb, but you could also mix pictures as far as I remember.


I'm not sure anymore, but the things they end up with is, you could breed pictures.


You could also put pictures together, so the procedural generation of them.


What you end up with are remarkably interesting things.


The point they made is, it's really only from very few iterations.


These are tens or hundreds of iterations of development, not like a million like we're used to.


There's a real tree of phylogenies that emerge.


The crucial lesson they say is, people only find when they are not looking.


If you had a certain goal in mind, you would never be able to change the pictures in the way that this goal would appear.


If you have no goal in mind, you might discover all kinds of interesting things.


That is all I'm going to say of this.


They discussed many more things, but I think these are the main takeaways.


Population-based search is interesting because it can overcome the problems that if you only had one optimizer, one optimization run of one algorithm.


If you employ quality diversity in the algorithm map elites, this enables this kind of goal switching.


It gives you back an entire landscape of learned actors or systems.


For each one, you know it's the best performing one in that particular constraint of the dimensions you care about.


Open-ended algorithms, open-ended search is definitely a cool research direction and I encourage you to check it out.


S.H.E. - Search. Human. Equalizer.

Hi everyone, just a quick more of a news update in the AI world, which is the following.


S.H.E. - Search. Human. Equalizer.

Pantene launches SHE, the Search Human Equalizer, to shine a light on bias in search.


S.H.E. - Search. Human. Equalizer.

So Pantene, the kind of cosmetic corporation, launches this thing, which is supposed to


S.H.E. - Search. Human. Equalizer.

correct your search.


S.H.E. - Search. Human. Equalizer.

And it's introduced here in this YouTube video, which, as you can see down here, has 400 likes,


S.H.E. - Search. Human. Equalizer.

has 3.5k dislikes, and of course, comments are disabled.


S.H.E. - Search. Human. Equalizer.

So that's kind of already weird.


S.H.E. - Search. Human. Equalizer.

Let's let's say weird.


S.H.E. - Search. Human. Equalizer.

If you go to the website here that they made, basically, let me refresh this and you can


S.H.E. - Search. Human. Equalizer.

see see the intro, they say, let's take the bias out of search.


S.H.E. - Search. Human. Equalizer.

So if you search for greatest engineers, you will get all men.


S.H.E. - Search. Human. Equalizer.

If you search for schoolgirl, you'll get like this kind of sexualized images.


S.H.E. - Search. Human. Equalizer.

If you search for Asian women in Spanish, same.


S.H.E. - Search. Human. Equalizer.

So basically, they have a browser extension that modifies your search results to that,


S.H.E. - Search. Human. Equalizer.

for example, schoolgirl looks like this.


S.H.E. - Search. Human. Equalizer.

Of course, I don't know if if I were to do this, I would actually let people explore


S.H.E. - Search. Human. Equalizer.

the search box right here.


S.H.E. - Search. Human. Equalizer.

But of course, they want you to download this extension.


S.H.E. - Search. Human. Equalizer.

And so to me, what the interesting part is, how do how does this work?


S.H.E. - Search. Human. Equalizer.

So you're asked to install a Chrome extension, which I won't do.


S.H.E. - Search. Human. Equalizer.

But basically, down here, they say view the terms that sh is equalizing.


S.H.E. - Search. Human. Equalizer.

If you click on that, you get to a list.


S.H.E. - Search. Human. Equalizer.

So it very much seems like this is absolutely manual, handcrafted work, because there's


S.H.E. - Search. Human. Equalizer.

a lot of work in kind of correcting bias in for example, in search in machine learning


S.H.E. - Search. Human. Equalizer.

and so on.


S.H.E. - Search. Human. Equalizer.

And these approaches usually have some data driven approach that actually will will change


S.H.E. - Search. Human. Equalizer.

the models and so on, or will rerank based on some kind of learned input.


S.H.E. - Search. Human. Equalizer.

But this here is simply a list of, of terms, for example, a famous actor, famous athletes,


S.H.E. - Search. Human. Equalizer.

and so on that it will then kind of rerank.


S.H.E. - Search. Human. Equalizer.

And I'm pretty sure this is just human manual labor, someone comes up with a new term like,


S.H.E. - Search. Human. Equalizer.

oh, this term, we should, you can actually flag yourself in the Chrome extension.


S.H.E. - Search. Human. Equalizer.

So they say here, flag this search, you can there's a button, so you can suggest one and


S.H.E. - Search. Human. Equalizer.

they will say, Oh, yeah, okay, this is really not biased, or that is really biased, will


S.H.E. - Search. Human. Equalizer.

now rerank the search results for you.


S.H.E. - Search. Human. Equalizer.

It's, I mean, academically, this is a terrible idea.


S.H.E. - Search. Human. Equalizer.

Absolutely terrible, because how are you going to do this, like, manually replace every single


S.H.E. - Search. Human. Equalizer.

series, like, I don't know, it reminds a bit of new speak.


S.H.E. - Search. Human. Equalizer.

But yeah, this approach is doomed to fail.


S.H.E. - Search. Human. Equalizer.

And of course, it's just a company trying to sell you stuff.


S.H.E. - Search. Human. Equalizer.

It's not, I mean, this is not a, this is a PR gag, not really trying to do anything,


S.H.E. - Search. Human. Equalizer.

anything state of the art or meaningful or even effective, right?


S.H.E. - Search. Human. Equalizer.

If you search a little different thing than this, it will still show you the old kind


S.H.E. - Search. Human. Equalizer.

of result.


S.H.E. - Search. Human. Equalizer.

Yeah, so from the terms, you can also pretty clearly see where they come from, they have


S.H.E. - Search. Human. Equalizer.

their own name, they have Pantene, I didn't see this yet.