of abilities of these agents, if you wanted to go further, like what would be questions,


Spurious normativity enhances learning of compliance and enforcement behavior in artificial agents

adjacent questions that you'd like to have answered from such a simulation and what would


need to be added? I'm yeah, I'm thinking of things like maybe a bit of communication between


the agents, some signaling, like I could, I could like signal to others that I'm a good


punisher or something like this or that.


This question, we can go in a few directions. One thing that this is very open is where


do the norms come from the content? Here we just chose like, you know, this is a taboo


very, this other one is taboo very. But what we really want, if we want to have a model


of cultural evolution is a model where norms themselves can emerge from the general training,


general learning of the agents. And so that is one direction that we started to go after


this paper. We have another follow up paper where we have a way for the content of the


norms to evolve within the system. But it's also not perfect. It has, it has continual


learning problems arise because if you have a kind of constantly changing the adaptive


environment for everyone and you can, you can easily break reinforcement learning that


way. So I think the next thing that's going to have to happen in this line before it turns


into like a real model of cultural evolution that feels like you can do the kinds of things


we want cultural evolution to do is we'll have to have some more effort on the continual


learning side. Basically make it so that agents can kind of come up with one norm, society


comes up with one norm and then it can kind of change and then tipping point effects as


it changes, you can see fads and trends and things and none of that can really happen


right now until we solve some continual learning issues.


With respect to, you know, you said something, you know, we have to solve continual learning


issues and so on. What is like, I'm imagining there are quite a bunch of hyper parameters


in this thing, not only reinforcement learning wise, like what's my discount factor, blah,


blah, blah, but also how many points do I give to what, right? I can give, you gave


four points per berry. Like, well, that's just a number. You give 35 points for like


punishing someone correctly. How sensitive are your findings to these things or how sensitive


is the whole system to these parameters?


So I think that's really hard to quantify because a lot of the changes would be really


meaningful, right? If you, let's say, make the berries so valuable that you never care


about the poisoning, where you make the poisoning so weak that you don't have to worry about


it. Any of these things you would expect to make a big difference because you've changed


the balance of all the different things that you need to learn about. The thing that we


tried that I thought was really encouraging was that we just re-implemented the whole


environment and the agent and also tried a different type of learning agent on it and


the results came out very similar. So that kind of made me pretty confident about like


the overall observation that if you have this type of social learning problem where you


learn from the observations of how others treat you, if you get more of those, that


helps. And that can be like a key component in like getting the overall population to


the goal faster.


How does one avoid like confirmation bias in these types of research? Because you probably


have had some sort of idea of what you were going for and you know, like a hypothesis


to show and like Occam's razor is kind of a brutal thing, right? And there is, if you


see these results, you were like, oh yeah, this fits perfectly well with the hypothesis


I had and so on. So what I'm not like, I didn't, not that I see anything wrong here, but I'm


just wondering if you go into this with the hypothesis, kind of what are the steps one


needs to do to avoid sort of falling into confirmation bias?


I mean, this kind of thing is about showing that a particular mechanism exists and is


there and what we don't know is of course, relative to all the other mechanisms that


are supporting silly rules in the real world, how strong is this one versus other things?


And we could talk about some of the other ones as well. And there's no way you could


ever answer that from this kind of problem.


I think though, and Rafa, you may want to say a little bit about this because it was


you and our other co-authors that introduced this idea of testing individual agents at


different points in training to say, can we confirm that that really is what the agents


at these different stages are learning or have learned, right? That, you know, because


otherwise, you know, we're observing just this mess of eight agents interacting in this


complex environment over and over again. I think that was really quite a great insight


and innovation, part of the innovation in the paper. And Rafa, you may want to say a


little bit more about that because I think of that as the psych lab experiment for artificial


agents in this context.


Yeah. So I think you've touched upon this earlier. So one issue of course is with all


the metrics that you just get from the observations from the whole simulation is that it's not


clear if you can take them at face value because there might be indirect effects that like...


I scroll up a little while he talks about this because that's where, I think in the


right above, yeah, right around there.


So if you, for example, observe that they spend less time marked, is that because they


get punished quicker or is it because they get marked less? And also of course the dependence


of more being marked only creates the opportunity for being punished more, which then like creates


pressure to get marked less. So because everything is entangled, it's really hard to know what


do agents actually... what have they learned and how do they actually react to individual


stimuli? What is it that they're actually trying to do? So the way we tried to approach


this is similar to how psychology tries to approach it with humans. That's like try to


give them a controlled experiment, take them out of the complicated world, put them in


like a lab where you just show them individual stimuli and see how they react. Like how quick


are they to pick up the berry?


That's what these pictures are. These are frames from that environment, this like test


environment.


Exactly. And then the results that we uncover are very similar to what you get from the


observations. So, sorry, from the metrics from the whole simulation. So that although


this is a bit of a... like there's some need to do generalization here. This is a bit different


from the world that they actually inhabit. But even if you just show them one stimulus


in isolation, they do pick up... they do start to just not pick up the berry that they have


been punished for frequently. So it is like in that sense like a very clear demonstration


that they have learned the right thing, even if the presentation of it is a bit different.


But I'm not sure if it sort of answers your original question about the concept of...


Yeah, that was my thing.


I think it's more about... I think this is a big question for all modeling papers of


like, what does it take for an economic model or a model of traffic or a model of how a


disease spreads to be so good that you sort of trust it to make decisions based on it?


I think that's sort of a long path that relies on many different papers sort of validating


it.


Calibration as well. I mean, ultimately, if you want to make real world predictions, real


world decisions, you need to get real world data into the model.


I think this is also something that comes from the collaboration between social scientists


and computer scientists on this, because we're seeing more and more computer scientists working


on models that are interested in what's happening in the real world, like analyzing language


models or multi-agent environments. And when you start bringing in social scientists who


think about exactly this point, like, okay, so what's a good experimental design that


allows me to reliably exclude alternative explanations for the phenomenon? And things


like, and you should have a hypothesis before you start. You don't just run the simulation


and say, hey, look at this cool stuff we discovered and report that. You try to craft something.


We spend a lot of time on the experimental design on this one and to exactly be able


to sort of respond to your potential critique of, well, how do we know you're not just giving


us a just so story about what came out of this simulation?


You said something like, to the effect of, we also think work like this is very, very


important towards the direction of AGI. Do you want to explain a little bit what you


meant by this? Because it is quite a different direction AGI currently that the biggest yeehaw


is in the direction of, let's just make one language model really, really, really big.


Where do you sort of where do you come from when you say work like this might be sort


of AGI material?


Yeah, I'll start. So if you start from a place where what you want to do is make human-like


AGI, and you can say, to make a human-like AGI, you need to capture all of the cognitive


abilities that make human intelligence, perception, attention, memory, these kind of things. And


you can have a single agent research program that does that. But from my perspective, and


I think from a spiritual perspective, that's not really what's important about human intelligence.


It's not that we're better at perception or memory or attention or anything like that


than other animals. That's not what's unique to us. It's not the secret of our success.


But what is, the things that are unique by humans are these more collective properties,


things about how we cooperate, things about how we imitate each other, how our cultures


evolve. And that's what you want to capture. So it's not the individual level social cognitive


abilities. It's more like the group level social cognitive mechanisms, some of which


might be ability-like, things like theory of mind. Others might be more like representations.


Or some could even be like motivations, like we talked about this intrinsic motivation


to punish when you see a transgression. Things like that. They're not exactly an ability,


but they, in fact, they're not even things that we think of as terribly smart when you


see an individual engaging in those kind of behaviors. But at a group level, they might


have a effect that influences our cooperation and how we learn from each other and how our


norms work, how our institutions can be built and the way our technology develops and really


contribute to all the things that we're proud of that come out of human intelligence. So


if that's what human-like intelligence is, then it follows that studying these kinds


of issues is what we should be doing. And that's where I see this line of work going


all coming together in the AGI direction.


And normativity in particular is a really important thing, I think. I think it's not


entirely just about if you have a problem that is a social dilemma or something, we


need to cooperate. It's also just about setting up the rules of the game that organize how


we innovate, when we explore and when we don't. And norms broadly construed so that they eventually


include things like institutions are really critical for that. I think we kind of are


that. They set up the game that we're playing. We all work for companies and for universities


and these entities exist and structure our local incentives in ways that cause us to


try to innovate. And I think that's how human intelligence as a collective intelligence


works. It creates local rules of the game for people to play so that intelligence can


be applied in the right direction so we can explore and do things. That's where I come


at with how I come at it. Maybe we should all answer this question within the interactions.


Raphael, you go.


I don't know if I have much to add to that. I think, yeah, there's the perspective of


developing intelligence from cultural evolution of populations of agents. And then as Joel


said, norms are particularly interesting because they are, if you have these multi-agent systems,


it's all about the equilibria of how that the behavior reaches. But the norms are the


ones where you sort of take an active influence on the incentives of others. And that seems


like it's a really important part of like a social structure.


Let me add just one thought here. When I give talks on this, I usually say, look, my favorite


definition of artificial intelligence is the capacity to act with foresight and appropriateness


in a given set of circumstances. Well, that word appropriate in there is normativity.


What in this environment, it's not just a matter of physics, right? There is notion


of how you move a ball, but if you're going to interact with people in a meeting, if you're


going to make decisions together, all of that is the structure that humans have invented.


I think that's, it's really critical to understand that that normative infrastructure is what


allows us to accomplish so much collectively and to share information and learning across


groups, across generations, and to pay attention to the fact that that infrastructure needs


to be generated and maintained by human behavior and perception. So I think this is, to me,


I say artificial general intelligence by definition has to include the capacity to participate


and read this kind of normative information in the environment and participate in supporting


it. So I don't know how we're going to generate artificial general intelligence without paying


attention to normativity. So that's what we're, I think that's the connection for me.


I think the proponents of sort of the scaling hypothesis, they think that models can just


pick it up out of reading stuff or so.


If it's a static environment, right. But if this is dynamic, right?


Is there your research investigates why things exist, why things come to be, what, you know,


why a mechanism might be there? Is there a prescriptive element to what you do? Would


you dare say, well, what we figured out here because of what we figured out here or over


the course of our research, we can give recommendations to specific things in society of what we should


do at some point. Like, you know, hey, how about a silly rule here? Or like, is there


something actually where you could say, you know, here's a recommendation?


I think so. Sorry, I'm on the recommendation side, I think. Yes, actually, this is a really


critical point. And I worry about it a lot when we're thinking about alignment problems


and so on. As we think about norms and values. These, you know, there's this idea where you


you know, if I asked you at the beginning, do you want to imbue your machine with just


the important stuff? Or do you want to give it a bunch of silly stuff as well? silly rules


to follow? Most people would answer that question, but it's well, clearly, just the important


stuff like we don't want the machines to be stupid, like humans and, and worry about haircuts


and and what food you eat and so on. But the point is, is that those silly rules are actually


playing a very important role in this model, they're helping to sustain those behaviors.


In other work that we've done, we've shown how it contributes to robustness and the ability


for the agents to read the state of the system, the enforcement system, like are the rules


being enforced around here? Because if not, I'm leaving, right? I don't want to stay around


and be vulnerable. So I think a recommendation here is, you know, that actually you need


some silly rules, because they're cheap ways for agents to understand the state of the


system. And that's a critical thing to know, to decide, do I continue to cooperate? Or


do I go somewhere else?


Is the scientific method just this is not no longer about RL, I guess, is the scientific


method kind of an antidote to silly rules? Because I figured, you know, at some point,


someone says, Hey, I've actually tested it. And you know, we don't we don't need to avoid


the fish on Friday. It's actually it's actually not doing not doing anything. You know, I


did my randomized controlled trial. Is this sort of like, what percentage of silly rules


that we have is impacted by this more like point 1% 50% 90%?


Like, mostly don't I think we when we when we have a strongly held kind of culturally


cultural belief like this, we don't give up in the face of evidence most of the time.


So the scientific method maybe helps on the margins in some cases. But but most of the


time, the silly rules overwhelmed the evidence, or we feel more strongly about the adherence


of the silly rule enforcing it than we do about scientific method. And yeah, so not


sure. But I'm saying that's what you can do. But there's some some argument here that we


should have we are maintaining so this for a reason. Yeah, papers about course. But it's


not about any particular. And of course, any if a silly rule becomes becomes actually a


harmful rule, then you really do want to have mechanism. Where does the where does the journey


go from here for you? Like in this line of work? What are big? You've already mentioned


a little bit like how do norms appear? What are what are other big unanswered questions


that, you know, maybe other people might want who might want to get into this field might


want to take a shot at?


Another really interesting one that I don't know how we will get to is how do you get


like systems of norms and then institutions? What's the relationship between norms and


institutions? And can we build can we have institutions emerge within our multi agent


systems? And what way would they be different? Maybe like an institution has some kind of


personality to it or something like that. It doesn't matter who the individuals are


or something like that. But we don't know nothing like that has ever emerged in any


institution. But that would be interesting to try.


I think two of the things that I'm really interested in are thinking about robustness


and you know, are these are groups that have have developed these these rule enforcement


and compliance systems better able to respond to shocks and adapt to new information and


changing environments? And then I think also, you know, to what extent does this become


a, you know, is this a more general mechanism for transfer learning across settings, which


is to say, all I need to do when I go into a new environment and a group, particularly


if it's already a stable group, is I need to look around and figure out what do these


people think, you know, what are you going to get punished for around here? What are


we supposed to punish around here? And that can mean you learn a lot very, very quickly,


which is how humans kind of work, right? We we you got dropped down in the in the in the


Arctic and you're lucky enough to land in a you know, among among the Inuit. But the


first thing you would do is say, whatever those folks think is right or wrong to do,


that's what I'm going to do. And fortunately, they'll be punishing you and throwing you


out if you violate the rules. So you even have an added incentive to to not think you


can figure it out better than they can. So I'm interested in that in that the idea that


having this structure in place actually is is part of what makes us so intelligent as


we go down into new into new environments.


Excellent. Is there anything else about this research that you want people to know you


want to shout out? Anything that is important you feel we didn't touch on?


One more thing. So this paper along with all the other papers we've written recently, they


generate both environments and agents, which we also packaged up together in an evaluation


protocol and suite environments that we've released, which is called melting pot. So


it's anyone who wants to do multi agent reinforcement learning research on environments that look


vaguely like this, but on many different topics, melting pot is the place to go. We've put


out a large number of different ones, we're putting out more all the time. And it's a


platform for doing multi agent reinforcement research and having a having benchmarks you


can compare to between others.


Cool. In this case, Raphael, Jillian, Joel, thank you so much for being here. I learned


[ML News] Microsoft combines Images & Text | Meta makes artificial skin | Russians replicate DALL-E

Microsoft trains a universal image language representation model, Facebook gets all touchy


touchy and the Russkies release their own Dali model. Welcome to ML News.


Hello there, this video is sponsored by Weights and Biases Tables. Yes, the video is sponsored by


a feature. That's a new thing. You haven't seen that before. So Weights and Biases Tables is an


interactive way to not only explore your experiments like you would usually do with


weights and biases, but to explore your data as well and the combinations of your data, your


models, your predictions, your experiments, anything you want essentially can go into a table,


you can see they can include pictures, even little sound files that can include videos,


they can include image samples and overlay the models predictions as a mask as you can see here.


And you can compare different models to each other in a single table. This is extremely powerful. And


if the user interface is not enough, they have a special syntax with which you can do pretty much


anything you want. Really cool for visualizing predictions such as this one. Look, here is a


picture and then the overlays of the masks of the model. That's probably my browser that doesn't load


that fast enough. But the effect is a cool one. Let's see that again. Oh, yeah. It's also really


powerful if you want to compute some metrics on the fly like counting false positives, counting


false negatives, area under curve f1 score, anything like this. Very cool. So they have this


example of a data set of Reddit comments. I know red is the most wholesome place on the planet. And


this data set is annotated with all kinds of emotions, whether or not they appear in the


comment by human raters. So you can load this data set directly into a weights and biases table


and then do all kinds of analysis with it. Honestly, it might just be cool to just load the


data set in without even having to do any sort of experiments on it because this is a great viewer.


For example, I can filter all the rows which contain both joy equals one and sadness equals one.


How's that? So apply the filter. And I can immediately see all the comments that match both


joy and sadness. Okay, what are these? Let's see that made me cry tears of sadness and joy at the


same time. Excellent. That's what we're looking for. Another really cool feature is the ability


to group by a certain column. So here I group by subreddit. And then we can analyze all kinds of


stuff across these different groups. For example, let me add a column here that tracks ratio of


sadness inside of each subreddit. Sadness dot sum divided by row dot count should give us that


result. And we have a result. And now we can sort by this and look at that the soccer is in third


place. Who would have guessed though it only has 12 samples. So maybe we want some more complicated


metric. Luckily with weights and biases, you can put all kinds of expressions in the cell expression


tables. And if that is not enough for you, they have a special syntax with which you can create


entire panels and visualizations give weights and biases as a whole a try. It's cool system.


And thanks for sponsoring this video. Hey, how's everyone doing on this wonderful Monday,


let's dive into our first story on the research blog, Microsoft says they have trained a universal


image language representation model called Turing Bletchley. Now Turing is the effort by Microsoft


to go into large scale models, large scale language models, for example. And Bletchley is a


reference I believe to Bletchley Park where Alan Turing cracked the enigma not entirely sure my


concept of these things is based off of Hollywood movies. In any case, this is a model much like


clip that combines text and image modalities. And not only that, but it also combines text from


different languages. So this is really a model that can understand the relationship between


images and text in various languages, all in the same embedding space. They achieve this by


crawling the internet for images that come alongside text in various languages. And then


they have basically two different objectives. One objective is to make the image representation


close to the representations of the various texts that go with the image. And the other loss is to


have the representations of two pieces of text that go with the same image also be close together.


And that means they achieve a representation space where concepts no matter whether they're


expressed in images or in any language cluster together if they mean the same thing. So they


demonstrate this on various different examples right here. For example, the model understands a


Coca Cola ad, irrespective of the languages, it can do a little bit of OCR and recognize words. And


it's not only for natural images. But as you can see right here, it also understands things like


maps. And the multi modality means that you can even mix languages and scripts as you put things


into the model and the model will still understand it. For example, on the left here, it says posing


for a photo at the Great Wall of China, but the Great Wall of China is spelled in Chinese


characters. And as you can see, the nearest neighbors in the embedding space are still


models where people pose for a photo at the Great Wall of China. Yeah, cat programming. This cat


isn't programming. How do you know these cats are programming? This is clearly a gamer cat. They even


have a little demo right here. Now here is where you see the smart PR people and lawyers come in


all of the queries that you're able to do. There are a lot of them, but they are all pre programmed.


So even though you can type here, you can only select one of the things that are already in here.


For example, space needle at night, crazy pants. No, I think this isn't so much because they want


to present you cherry picked examples. It's probably much more so people can't retrieve


things like not safe for work images and even images that might have some copyright associated


with it that ended up in this data set. But there is an interface for English queries,


universal queries, and even image queries. So you can try out what the model thinks which are images


which are sort of close in the space of meaning. Now here's a fatal flaw. If I'm not mistaken,


this here is actually song go Han and not song go coo as all the others. So that changes everything


terrible model. Meta AI Facebook AI meta underscore Facebook AI says today as part of a


larger tactile sensing ecosystem, we're announcing two major advances. Digit, a commercially available


touch sensing hardware produced in partnership with gel site and reskin a replaceable low cost


tactile skin. So Facebook is going into the hardware of touch sensors and general tactile


data. This isn't just hardware, this is sort of a big conglomeration of new advances in hardware


coupled with machine learning advances. So the first one is reskin a versatile replaceable low


cost skin for AI research on tactile perception. So this is really a piece of skin a piece of soft


material that can sense when it touches something. So you can see right here this patch of skin that


the person attached here to the robot hand allows the robot to get tactile feedback as it grabs


things, which is pretty cool because grabbing something like a blueberry is very hard when you


don't want to squish it. And as you saw, maybe up here, one robot simply, you know, does like, no.


So there are several advances right here. And they're not all hardware advances. Notably,


usually you'd have to recalibrate every single individual one of these skin sensors. Because


this being soft material, you can't really manufacture it in such a consistent way that all


the sensors achieve the same accuracy. So you can't just calibrate once you have to recalibrate every


individual thing. And the recalibration in this case, as far as I can read is done using a


self supervised technique rather than supervised calibration, which makes things a whole lot


easier. So there are various applications for this, you can see that not only do you get tactile


feedback of whether you're touching something, you actually do also see where you touch something.


So there are like enormous amounts of applications for this technology. This goes along with another


technology called digits, which is also a touch sensor, but it is a little bit different. Namely,


these are the small sensors that you can see right here. So this isn't necessarily deformable skin,


but this is a very high precision touch sensor, like you might have it in a fingertip. I guess


that's why it's called digit. Also, they say that this is quite low cost, and they have open sourced


the design. Now, as you can see here, the resolution on sensing on these sensors is quite


high, you can see it's able to sense very, very, very detailed things on the things that it grabs.


This goes along with a new pytorch library that they've built called pi touch that is able to take


in this data and transform it in various ways. And also they are open sourcing tacto, which is a


simulator for these types of data. So all in all, Meta Facebook is really making an advance into


this tactile ecosystem reskin deformable skin digit the super high precision touch sensor


tacto the simulator and pytorch the library and they say soon they'll be out with a bunch of


datasets and benchmarks for people very cool. I'm quite excited to see the technologies that are


going to be possible with the sensors and processing tools. anime again is all the rage


right now all timelines of all my social networks are filled with people to defying themselves and


putting their faces and pictures into anime again, and it does look quite cool. So this is a series


of advancements right here starting from classic anime again, improving this to anime gan v2, which


makes various improvements over the classic anime gan by the way, this is a mixture of a style


transfer and generative adversarial network. The code to anime gan was released in TensorFlow, but


has been ported to pytorch. And that again has been released as a space on hugging face that you can


just try out. So here is a picture of me. It looks kind of weird. Here's a picture of the channel


logo. That just looks disturbing. Here's a picture of some industry that looks actually pretty cool


as the output. And here's a picture of Captain Picard and we'll see what happens.


Yeah, that looks pretty sweet. So what I want to highlight besides the fact that this is a cool


model is just the chain of individuals or individual groups that just loosely work together


to achieve something like this from the original research to its improvements, its releases code,


the transformation into various frameworks, and then in the end, the deployment as a really user


friendly interface that you can use for free. This whole ecosystem is quite quite cool and


pretty happy it exists. So I'll link everything you can try it out. Researchers from MIT release


a paper called a system for general in hand object reorientation. And this is pretty cool because it


teaches robot hands here in simulation to reorient any sort of object and it can reorient objects


that are as you can see, very, very tricky from given their form. And it can even do that in a


zero shot fashion. So the trick here is that this is a student teacher model. So the final model,


the student only has access to sort of the sensors in the hands like how the joints are oriented


right now and to the visual input of the camera. However, it turns out that is quite tricky to


learn from you are given the object and you're given a target pose and you need to rotate it


somehow to the target pose. Now the task would be a lot easier if you had access to what they


call privileged data such as the velocity of the fingertips and so on and that you do have access


if you're in a simulator. So the trick here is that they first train a model that gets access


to all that privileged information learns what to do using that information and then teaches the


student model what to do. So the student model doesn't have to learn through reinforcement


learning, but it can instead learn from a very, very good teacher exactly what to do in a supervised


way. And with this method, they achieve very strong even zero shot performance on new object


whether the hand is upright like this or turned around like this, it can even use the table as


as help pretty cool and pretty simple. The Washington Post writes five points for anger,


one for like how Facebook's formula fostered rage and misinformation. And by now, you should be aware


that when you read an article like this, that the journalist here wants to tell some sort of a story.


So what you usually have to do is you have to go to the very, very bottom and read like the last


three paragraphs such that you actually get what's going on. So the whole article is about how


Facebook over the years has changed its algorithm to rank different posts on your page, there seems


to be a sort of a point system. For example, when someone likes your post, that post gets one point,


if someone comments on your post, that post gets whatever 10 points or something like this. And


these points are then used to score your post among all other posts in your friends and followers,


newsfeeds. Now the article here is quite long and details how Facebook evolved this algorithm as


well over the years, especially after the introduction of additional things. So it used


to be just like for a post. And apparently now you can also do love, haha, wow, sad and angry,


I've actually stopped using Facebook except for posting videos even before this was the case. But


you now have various emojis in order to react to content. So the article tries to tell the story


specifically about the angry emoji, people reacting to that, and then the algorithm boosting this


content. And this sort of ties to this notion that what Facebook's trying to do is to make people as


angry as possible, such that it maximizes their engagement and so on. And you know, while there


is truth to the fact that when something makes you angry, it makes you more engaged, the article's


tone and the actual things that happen don't really match up again, this seems to be a recurrent theme


in these articles. So when you read the article, neutrally, you can see that the problem is actually


not that easy. For example, you can see that the title says five points for anger, one for a like,


and you would somehow guess that Facebook intentionally up rated the anger emoji,


which is not the case, they simply operated all of the emojis except the like emoji. And the reasoning


behind it was that in order to use the other emojis, you actually have to do two clicks. And


in order to use the like, you only get to do one click. Therefore, a user doing two clicks is more


effort means they engaged more means this should be operated in comparison to when a post only


receives a like. In addition to that, Facebook was also trying to push these new features of these


new emojis. And that's what platforms often do look at YouTube shorts or YouTube polls or things


like this is that they massively up weigh the new features just to get people to use them. And then


later, they'll down weigh them again. So it was technically true at that particular point in time,


an angry emoji was five times more worth to the algorithm than a like, but do you think that


framing it as the article does here, especially as the title of the article is a fair characterization


of what happened? Well, I don't think so. And the rest of the article essentially goes on in this


tone where you have difficult problems, and you're trying to come up with some sensible solution that


weighs a lot of interests against each other one being profit, but not the only one. And then that


solution not being perfect and having to be refined. That is not the same thing as Mark Zuckerberg


sitting there going like and the kind of sleazy journalism of the Washington Post right here is


just not helping. If you want to give the article a read, see if you can untie the journalists


framing right here from the actual real problems that arise when you program such a recommendation


system algorithm. Demi's husband is tweets thrilled to announce the launch of a new alphabet company


isomorphic labs, our mission is to reimagine the drug discovery process from first principles with


an AI first approach to accelerate biomedical breakthroughs and find cures for diseases.


isomorphic labs appears to be a new company under the umbrella of alphabet, therefore sort of a


sister company to Google and DeepMind. And its goal is to accelerate things like drug discovery


and various other things in biology. Demis himself will be the CEO of isomorphic labs,


but also remain the CEO of DeepMind. Now with DeepMind going into things like alpha fold,


making quite a few advances applying AI to real world things, it's probably makes sense to spin


this off into a single direction business effort right here as isomorphic labs, while probably he


wants to keep DeepMind more on the path of pushing AI research in general, and not that DeepMind


suddenly becomes product implementers for pharma companies or something like this. On the other


hand, maybe it's just some scheme to save taxes, you never know. Sure bank AI releases a Ruda Lee,


which is a Russian version of the Dalai model. The original technical report is available in


Russian, but Google Translate is fairly good nowadays, they detail how they went about


building the model and what they're releasing. So they have two different versions of it, one with


1.3 billion parameters and one with 12. The 1.3 billion parameter model is actually available.


This goes along with various helper models such as their own version of clip and a super resolution


model to do large images. Now I've heard somewhere that they also want to open source the really


large model, but I'm not exactly sure that is super trustworthy. So as I said, both the code


and the models they are released on on GitHub, you can go and look at it. And the outputs of this


model are pretty cool people still figuring out exactly how to prompt them. I think prompting has


come a long way given the whole clip and VQGAN combos, and we'll probably have to learn how to


do the same thing with these Dali based models. So they have a bunch of examples right here. And


they all look very cool. There's also a space on hugging face where you can simply type in something


now this uses a translation engine to translate from English to Russian because you can only


input things in Russian into the model. So if things go wrong, you never really know is it


because of the translation is because of the prompt not being appropriate enough or the model


fails. So here I input a purple tree on top of a mountain is not exactly what I wanted. But people


have gotten quite cool results with it. There are also various notebooks right here that you can try


out. And as I said, there is a technical report and the project website if you're interested in


how all of it was built is quite detailed. And it recounts the engineering challenges that the


researchers had when implementing this. It's pretty cool to see that after open AI has already gotten


a few challengers in the large language model space. Now more and more challengers also appear


in this Dali in this image generation from text space. The business model of not releasing your


models doesn't seem to hold up for too long. I guess if you wanted to do that, you also shouldn't


publish about them. But as soon as you publish, other people are bound to reproduce your efforts,


which is pretty cool for the rest of us. Excellent. This tweet here has gotten a lot of attention


image scaling attacks in the wild. So this is a adversarial attack not on deep learning systems,


but on re scaling procedures. Usually this happens when you get an image you want to input into a


neural network, the neural networks usually have very defined sizes of images that they take in.


So you first resize the image. Now, if you craft an image very smartly, you can craft it such that


the resized version looks nothing like the original version. So you exploit how the resizing algorithm


resizes images in order to achieve this goal. It's pretty unbelievable. But if you do resize the image


on the left right here, you downscale it to the size on the right, then if you input it into the


TensorFlow resizing algorithm, this dog picture will turn out again, there's nothing else you take


the image on the left, you put it through the downscaling algorithm, just downscaling, and the


picture on the right is the output. That's because the picture on the right is sort of like hidden


in the picture on the left in an exact way such that once you down sample all the original picture


essentially cancels out and this new picture appears. Now the picture itself is actually from


quite old work or by old, I mean, like one year, which is ancient in the learning world. But these


image rescaling attacks have been a thing for a while now. So for example, here is a paper about


backdooring and poisoning neural networks with image scaling attacks. There is an interesting


take here from Richard Chung, which says that this is essentially not a property of rescaling itself,


but of faulty implementations of rescaling in various libraries. And there have actually been


papers written about this problem, namely that if you want to calculate things like FID, which is


often used in GAN as a quality metric, then it actually matters how you rescale images. And if


you're rescaling algorithm doesn't do proper anti aliasing, then the rescaled images will have way


too much contributions from certain pixels and way too little contributions from other pixels. So


here, for example, if you ask these libraries to re scale the circle on the left, which is 128 by


128 to 16 by 16, only the pill Python image library does a good job at it. Whereas all the other


libraries you can see right here, they have various under or over contributions of different places


in the image. And this is exactly the weak spots that these image rescaling attacks use in order to


attack these images. So the solution here would be that the frameworks implement proper rescaling


of images, which might cost a little bit of speed. So it's not guaranteed that these will make it to


the final product. Microsoft Azure announces the open AI service, which essentially isn't an API


that you can query GPT three with here, they have an example where GPT three automatically sort of


summarizes sporting events from live feeds. And here is a neat corporate little video about boxes


and things that connect things. Wow. Essentially, you're able to call GPT three in an Azure ecosystem


right now. So if you're an Azure customer, you don't have to go through open AI API, you can go


directly to Azure, this is invitation only right now. But I think it'll be changed in the future.


And you can simply have this as a service on Azure. Here's something cool neural MMO,


I've actually reported about this before, but this has now been published at NeurIPS 21. And there


are continuous updates to the framework, the last commit is 13 days ago. So this is very much a


project that is alive. This is a framework for running reinforcement learning agents in big


worlds with other reinforcement learning agents and that have to live for quite a while. So think


of World of Warcraft, but for RL agents. Now the worlds are still quite simple, because RL is a


data and compute intensive task. So you don't want to make things too complicated. But this is by far


one of the most complicated environments that I've seen so far, especially the introduction of other


agents into the world. So you can have different sort of species of agents, and they'll find


different niches in order to survive and things like this, they do a pretty good job of giving


you various tools to analyze the results of your runs. So this could be used both for researching


reinforcement learning agents, but also researching various sort of population dynamics, if you're


interested in anything like this, I think they do hold competitions, if I'm not mistaken, see,


there is even combat in the game. So if you're into challenges in reinforcement learning that go


beyond just single player Atari games, or something like this neural MMO might be very cool to look


into. Another game that is not meant to be played by machines, but by humans is archive doom. So


Stephen Nicklaus made this little piece of web based doom right here. And the trick is wait,


let me zoom out a little bit that it's doom, but the opponents are sometimes papers, you see,


not only are they papers, but they are as far as I have read recent papers from archive. And once


you shoot them, they get rejected. See, so this is wait, let me show show your face paper, show your


face. Ah, yes, I guess this is so we can scroll down here to see this is attack agnostic detection


of adversarial year rejected. So there are these these other opponents as well. And ah, come on,


you can actually die reject, you can switch your weapon as well. So there's this machine gun right


here. And there's even this blaster. I've never I've never played doom. I'm sorry. If this is


standard, I don't know. Ah, go away. reject. Yeah, if you want to have a bit of fun, give archive doom


a try. It's pretty funny. Next up at the intersection of what machines and humans play


is the arc game. This is by Alex a Borsky and it takes the arc data set and makes it into a little


web based game that you as a human can play. So we want to try just one of these challenge things.


If you don't know what the arc challenges, I've made extensive videos about the measure of


intelligence. So you essentially get three different examples right here. So the top left


is an example. The top right is an example. The bottom middle here is an example, you're supposed


to just figure out the pattern and then complete the pattern at the bottom. So here the pattern is


that I guess every one of these bows here spits out a yellow thing. So from no yellow thing to


yellow thing here as well here as well. So I'm going to take the yellow thing, I'm gonna copy


this over if you click this right and then here we can just we can color in actually whatever we


want. But obviously, this is Yeah, yeah, we got it. We are touring complete. Stay another one. Okay,


so actually, let's do a hard one. Medium part tedious. Now I don't want tedious. Let's just do


hard. Okay, one of the hard ones. Alright, so look at that. So there is this and then there's this,


this. So the blue thing seems to be constant, right? We get four examples right here. Okay.


Um, right. Okay. And then here. Okay, so what's the catch right here? I guess it's whatever


piece can fill from the bottom the holes in the blue thing, such that it's like filled,


but it doesn't matter if it reaches over right, the only it only matters whether you can actually


fill in the hole up until the blue continuous line, you can see why machines would struggle


like this. So let's actually check of whether I'm correct. And then you need to color them red,


like once you figured out the rule, you still need to actually actively color them in red. So let's


do this. Okay, this one here fills that first thing. This one actually doesn't fill it. This


one fills nothing. This one fills it. See, see, this is I'm terrible. What is it? Why not? Why not?


Yeah, yeah, this goes here. This goes here. Yeah, both of these could go there. Yep. Well, come on,


this clearly goes here. This goes on. Ah, the bottom thing could technically go here on the right.


Jeez, I failed the Turing test. Yeah, I mean, give it a try. Definitely.


This is very cute. So this is a Twitter bot that takes memes and puts them through resnext classifier.


This is classified as a skunk, which is super interesting, right? So I'm gonna guess that is


a image net classes which expects there to be a single thing per image, but still skunk.


Zillow has to lay off 25% of its workforce, and they stopped their house flipping service. So


Zillow is this real estate company, they used AI to assess the prices of houses. And then they went


in and bought these houses at what they thought were low prices with the goal to sell them at high


prices. But this didn't work out. These stories are from CBS News and also Business Insider writes


that very often Zillow has their homes at a loss. So they bought them for more than they want to


sell them at. This is I guess, first and foremost, a lesson in what AI can and can't do. It's very


hard sometimes for an AI to just look at data that's available online and make a judgment about


a real life thing such as a house. Two houses might be very different, even though their metadata


looks exactly the same and a local realtor would know where as this sort of worldwide algorithm


maybe doesn't as much. However, it is special that there are other companies doing pretty much


the same thing which are flourishing. So it might simply be a failure of Zillow itself. And it might


be not a lesson in what AI can't do. But in you can't just throw AI at a problem and expect it to


perform well, you have to actually go out and look for good data, you have to program your algorithms


correctly, you have to validate them and so on. And all of this appears to not really have happened


too well with Zillow's algorithm here. So let this be a warning. If you're an ML engineer,


do a good job. Don't make your company bankrupt. Okay, welcome to this week's helpful things. The


first helpful thing is pytorch lightning release 1.5. This is a major release of pytorch lightning,


which if you don't know is a framework around pytorch to make training, saving, loading, etc.


of models much easier. So the new things in pytorch lightning are fault tolerant training.


pytorch lightning can now recognize when a training run abrupt unexpectedly or when one of


the machines in a distributed run aborts and it can restart training from where it left off. This


allows you to use things like preemptible machines without having to worry about you yourself always


making sure that the machine isn't shut down or taken away from you, etc. Also very cool lightning


light is for when you have a pure pytorch model, so not a pytorch lightning model, you can still


use some of the features of pytorch light by simply wrapping the model in this lightning light


module. And you do get almost all of the basic benefits of pytorch lightning such as multi device


training, multi node training, automatic dispatching to accelerators and so on. So there are various


other improvements right here, which I'm not going to mention, you can check them out for yourself,


but I do like pytorch lightning as a framework as cool to see that it's still being improved.


There's a new data set of League of Legends game playing data. This is essentially a recording of


agents in the game, human agents, and you are supposed to learn from them. So this is available


for you. The data set contained 72 games initially, but now has been expanded to contain


987 games. They're all filtered to relatively short games such that the individual episodes


aren't too long. But this is supposed to be a base data set for doing offline reinforcement


learning or imitation learning from teacher demonstrations. If you're into lol and would


like to train agents for it. Maybe this is a cool resource for you. Iris is an open source


alternative to Google Photos. This is submission to the pytorch annual hackathon 21. And it seeks


to provide the functionalities of Google Photos, especially that now Google Photos does actually


count your photos towards your quota. This is a welcome addition to the ecosystem, even though I


don't think that people are going to self host their photos thing in the future. But maybe this


will spur some kind of competition. So this is a framework that essentially ingests your photos,


indexes them does vector descriptions of your images, but also face detection and so on. And


after that, you're able to search for images using text, for example, here, pizza on the left, or you


can recognize what people are in the photos. And you can search by those. I love how the website


design is like exactly like Google Photos. But the icon in the browser is just like the default


react icon. In any case, very cool. open source, check it out. Our liable is a library by Google


Research that is supposed to make evaluation of reinforcement learning agents more reproducible.


So this does things like score normalization, stratified bootstrapping, and calculates various


other metrics that make reinforcement learning algorithms just a bit more comparable than like


a single number on the Atari benchmark. Very cool code is on GitHub, check it out.


Med MNIST v2 is a data set that seeks to be an MNIST like collection of standardized biomedical


images. So these are various data sets 18 to be exact 12 of them are in 2d 828 by 28 pixels and


six of them are in 3d 28 by 28 by 28 voxels. They say everything is available in standard formats


with corresponding classification labels, no background knowledge is required for users. So


if you're looking for an easy entry into biomedical data, this might be for you. I especially love the


papers with code usage graph right here, the histogram number of papers, one excellent.


And lastly, we have an article from fortune saying AI won't break your company's culture,


and it might even boost morale. This goes along with a new report by people associated with the


Boston Consulting Group, as far as I can tell about the cultural benefits of artificial intelligence


in the enterprise. So the article is trying to make the point that introducing AI products or


AI mechanisms into companies might lead to various benefits, especially benefits that people might


not realize initially, but it just sounds like this has been written by an AI to sort of make humans


comply more saying things like every CEO worries that culture will make or break their company's AI


deployment. But few realize that conversely AI can also transform organizational culture,


specifically using AI results in the following more collective learning, greater collaboration,


clearer roles, higher morale, saying things like as many as 79% of the survey respondents reported


an increase in morale after deployment of AI in their companies, like what this is definitely


written by an AI to make us more compliant. Look at all these benefits if you use AI CEO,


but you know, if the carrot isn't working, you also need to get out the stick which the AI authors of


this article definitely understand in the last paragraph saying deploying AI at scale may not


be easy, but CEOs would do well to remember that doing so will not only deliver financial benefits,


but also create high performance cultures CEOs would do well to remember. Excellent stuff right


here. Totally humans who wrote this totally. Thank you. Alright, this is already it for this week's


ML news. Thank you so much for being here listening. Let me know what you think in the


Neural Networks from Scratch - P.2 Coding a Layer

What's going on everybody and welcome to part two of the Neural Networks from Scratch videos.


In this video what we're going to do is just continue building upon what we've got so far.


The first order of business however, I am going to change all these values and moving


forward I'm going to keep doing this to be exactly the same values that we use in the


book.


That way if anybody is following along there, you can, the videos are going to match the


book exactly.


So the first thing that we're going to do is we're going to change these inputs.


These are going to be 1, 2, and 3.


And then the weights, and again these are just totally made up values for now.


Later the Neural Network will randomly initialize weights and then what's going to be tweaking


those weights is really like back propagation.


So for now though we're just making them up but I just want to use the same made up values


as from the book.


So 0.2, 0.8, and negative 0.5.


And the bias used was 2.


So running this we get 2.3, that's what we wanted.


So okay, so now that we've got that, I've got a couple of questions about why is there


only one bias and then some people were asking about weights so I've just kind of re-explained


weights but why only one bias?


So if that's kind of confusing to you let's take a peeksy at what we've coded so far.


So imagine you've got this full Neural Network.


It's fully connected, all that, and then what we've done is we've selected a single neuron


from that network and in this case that neuron had three inputs to it.


Now looking at this you can already see okay we've got three connections but only one neuron.


So in terms of bias there's only going to be one bias associated with this neuron but


because we've got three inputs that means we have three weights.


So every input has its own unique weight but every neuron just has its one bias.


You don't have a bias per weight.


So from here what's happening is yes we've got the exact same inputs that I've coded


here, the weights that we've got here, the bias that we've got here, and then basically


all this just gets added into this massive formula here of inputs times the weights plus


the bias to give us our final output of 2.3.


So now what I'd like to do is let's add one more input in this case.


So before we do that though I want to kind of solidify what exactly are these inputs.


So these inputs in theory could be one of two things.


It could either be truly input as in like values from the input layer of a neural network


which is just going to be a vector of values.


So in theory this could be an example of an input layer.


Now in the example that we showed visually that is exactly what it was.


We had this fully connected neural network and then we kind of singled out that one neuron


that was in that first hidden layer and in that case it was pulling values from the first


true layer which would be our input layer of the neural network.


So what is that input layer?


That input layer is simply whatever values you have that you're tracking.


So let's say you've got an example that you're trying to predict failure or not failure for


servers or computer equipment right?


And so you've got various sensors like heat sensors and humidity sensors and stuff like


that right?


So each value here in these inputs if this is your input layer each unique value here


could be a value from a separate sensor.


Now later when we add four inputs we're going to kind of simulate it's coming from somewhere


in the middle of the neural network.


So your inputs to a neuron could either be from the true input layer that are values


from something like say a sensor or a group of sensors in this case or they could be outputs


from neurons themselves.


So I'll show an example of that when we get there but for now let's say we add one more


input so 1, 2, 3 and then a 2.5.


So now we've gotten there.


So again we have the same fully connected neural network but this time what we're going


to do is we're going to select a different neuron.


So as you can see actually this time we're going to model one of the output layer neurons.


In this case though the calculation is going to remain totally the same.


Now in the future when we start calculating the activation functions you'll see that the


output layer will have a different one but for now the calculation of inputs times the


weights plus the bias that actually stays the same.


So the other thing that's different here is this neuron is taking inputs from not maybe


sensors or whatever it's actually taking inputs from one of the hidden layers but it turns


out it outputs the same stuff right.


So in this case for this neuron with four unique inputs what's going to change?


Well you're just going to have one extra unique weight but do you have any extra biases?


No we're still only modeling a single neuron.


It doesn't matter that four neurons were the input to this neuron because the outputs of


those four neurons become the input of the new neuron and once that neuron has already


generated its output it's already if it was like a hidden layer neuron it's already taken


into account the bias.


So in this case again when we're modeling four inputs to a single neuron we're only


going to have a single bias.


So let's go ahead and code that.


So really all that changes here is yes we've got the 2.5 the weights actually we need one


more unique weight and in this case we used 1.0.


And then the only change here is quite simple right we're just going to copy paste and then


I'm going to just modify this three and then come over here three and then just scrolling


out I think it's pretty easy to follow at this point.


Very simple as part one Harrison would say.


And yeah cool so let's go ahead and run that and we get 4.8 very simple.


So now what happens if instead of modeling so in this case we were modeling only one


neuron but with four inputs and then before that one neuron with three inputs what if


we want to model three neurons with four inputs.


So in the case of three neurons with four inputs what we're looking at is going to be


something like this.


Now in this case what we're doing is we're showing basically the output layer.


As you'll see later on the output layer will have some differences but the first part of


doing the inputs times the weights plus the bias is going to indeed be identical.


Alright so code wise what we're going to need here is if it's four inputs into three neurons


this means the inputs are going to stay the same we already have four inputs defined.


But what we're going to need are since there's three neurons that means there's going to


be three unique weight sets and each weight set is going to have four values because there's


four inputs and then for biases we're also going to need three unique or separate biases.


So let's go ahead and code that in.


I am going to just copy paste paste we're going to call these weights one two and three


and then for bias we're going to copy paste paste and we're going to have bias one two


three.


Now for the people who are like absolutely crying right now we will eventually convert


this to a loop and then eventually we won't even be using raw python anymore we're going


to use numpy to kind of go through this and make this way better but for now we're keeping


it very simple just so nothing is lost on anybody hopefully.


So what we've got here are weights one two three and what I'm going to do is I am going


to again use the exact same values from the book so 0.5, negative 0.91, 0.26, negative


0.5 and then for weights three it is negative 0.26, negative 0.27, 0.17 and 0.87 and that


is actually not a negative.


Cool for bias one is still a two, bias two will be a three, bias three will be 0.5.


Hopefully I got that right.


Okay so that's changed.


Now what is going to change here on the output?


So when we're just modeling a single neuron the output is going to be a single value.


In this case though we're actually modeling three neurons so we're modeling a layer so


to speak so the layer output is going to look an awful lot like the input so in this case


the input again it could have been maybe the input layer but it could also be input from


other neural like another layer in the neural network.


So the output for this should look an awful lot like this only rather than having four


values it should have three.


So let's make that change.


So what we're going to do is basically convert this, this will now be a list and I'm going


to say comma here and I think we should be able to get away with copy and what we'll


do is come down here tab tab paste paste.


Now inputs will always be 0, 1, 2, 3 okay but weights now what we're going to need to


do is change this to 1, 2, 3, 1, 2, 3, 1, 2, 3 and then again here 1, 2, 3 and then bias


needs to also be 1, 2, whoops, 2, 3 and don't forget your comma here and yeah I think that's


relatively acceptable let's go ahead and run that and what we see here for the output for


this these three neurons is 4.8, 1.21, and 2.385.


So again looking at everything you've got your fully connected neural network we've


selected three neurons with four inputs each those four inputs for every neuron will remain


the same but what's different is each neuron is going to have a unique set of weights for


each unique input and each neuron will have its own unique or separate bias.


So when modeling for the first neuron you've got your your weight set one your second neuron


is going to have your weight set two and it's also going to have its own bias and when modeling


for neuron three again its own unique weight set its own unique bias and it will also have


its own unique output same with all of the other neurons and the calculation for that


output is just simply going to be those inputs times those weights plus the bias so because


we've got three neurons it's going to be three unique outputs and each of those again the


calculation is pretty simple it's just inputs times the weights plus the bias for every


single neuron.


Okay that's all the code for now if you have any problem understanding anything up to this


point please ask your questions below if you're a little confused about like what weights


are this should start to become a little more obvious so imagine you want to start changing


these these output values here how would you do it well you have two options right you


could start because the inputs you can't really change the inputs because the inputs are outputs


from like either a previous layer or they're like your actual data from you know your actual


input data from features like sensor data or something like that so you really can never


change the inputs directly but you actually can by changing weights and biases so you


can change because the inputs what are those those are outputs often from other neurons


so how do we tweak those well we do that by tweaking the weights and the bias right so


that is the that is the struggle of deep learning is figuring out how best to tune those weights


and those biases but that's how we can start to have an impact on these output values and


when we get to it that is the whole you know crux of deep learning is the kind of back


propagation calculating gradient and like figuring out how do we adjust weights and


biases to get the output values that we want for like the vast array of inputs that we


might have so anyway if you are confused about weights and biases hopefully now it's starting


to at least become clear that okay to change to actually make an impact on this these outputs


the way that we would do that is by tweaking these weights and these biases now how we


do that that is a mystery that we will be solving much later in the series but for now


I my hope is that all of this is very clear Python code we will be cleaning this up we're


going to convert this well first I'll just show like obviously we can loop through this


we can make this into array data first and then but also when we do that we're going


to have to talk about arrays versus tensors which is a surprisingly heated topic sometimes


matrices all that so we have to cover a little bit more about that and then also some other


operations that we're going to be performing so anyway that's all for now questions comments


concerns feel free to leave them below if you're interested in the book you can get


it at n n f s dot i o eventually the draft will be complete but the the book is even


right now even though it's in draft form is basically everything from starting to code


your neurons the all the activation functions to actually training back propagation actually


getting the neural network to go through a training loop and actually learn things even


we've just released the regularization section as well so if you're watching this on release


that's kind of where we are right now pretty far pretty far along certainly much further


along than this so if you're looking to get a jump on the material that's how you can


[ML News] DeepMind's Flamingo Image-Text model | Locked-Image Tuning | Jurassic X & MRKL

DeepMind releases a system called Flamingo, Google releases a system called LUT and AI21


labs releases a system that's called Miracle. It's a fun week. Welcome to ML news. Have fun.


Hey there, my name is Janek. Welcome to the channel. This is already part two of this week's


news. I've done part one released it a few days ago. So if you haven't checked that out, a lot


has happened in this week. Let's dive into more of it. DeepMind releases a blog post called tackling


multiple tasks with a single visual language model, which details a model that's called Flamingo. So


this model is essentially what GPT-3 was for language but for images and language. This means


that it can take in both images and language and multiple images and multiple pieces of text and


then output a piece of text in response to it. It can also handle all of this in a sort of


conversational mode. And it's pretty powerful. Now one interesting thing is that it builds upon


pre trained and frozen models. As you can see right here on the left, the vision encoder is


completely frozen, which is indicated by the little snowflakey thing, the language model that


everything feeds into is completely frozen. And the entire training of the model is simply adapting


these in between parts, these adapters that connect the two modalities together. That's


pretty cool. Here you can see an example. The first image is a chinchilla along with a description.


The second image is a sheba along with a description. The third image is a flamingo.


And then the model is supposed to complete what this is. And it bases this upon the entire prompt,


which is images and text. This model can be used in a multitude of ways. It can do classification


by simply scoring answers, it can do image captioning, it can do question answering about


images, even about videos, and so you can take in multiple images and those frames of a video.


And it pushes the state of the art in various image language tasks. Here you can see another


bunch of interactions. So what is in the picture, it's a bowl of soup with a monster face on it.


What's the monster made of? And it says it's made of vegetables. The operator says nah,


probably not. It's a fabric. And the model says it's wool, which you know, it's quite a special


image. All right, this is Janek from the future. We have some new results out of flamingo now.


DeepMind seems to follow the similar strategy as OpenAI's Dali, which I'm going to guess they


observed and they learned that it works, namely that you give your employees and a bunch of like


very trusted first data users access to the model, and then you instruct all of them or you let them


tweet about it. So on social media, it looks like it's coming from many people and not just from


like one corporate account makes it seem much more organic. But you know, as long as this group is


very tightly controlled, take all the outputs with a grain of salt. Nevertheless, we do have some


outputs right here. So I've seen this picture going around and variations of it. I don't know


if this is the only one, but they are all very similar in their result. Andre Carpati originally,


I believe, posted this photograph and essentially said this should be one of or this could be one


of the benchmark tasks for visual models. If a visual model could explain why this picture is


funny, that would be, I guess, I don't know, impressive. And people have been trying, as I said,


to post these conversations with flamingo, which is certainly impressive, but you have to lead


flamingo to the point where it kind of tells you why it's funny. You can't just ask why is this


funny and it says, well, there's guy trying to read his weight, but then Obama's foot on it.


So it's a it's heavier than the guy thinks and so on. But the guy doesn't know. So you have to sort


of lead. I mean, you can go find this picture for yourself. Not going to read the whole thing, but


you have to lead it to like, what is he doing? He's looking at the scale. Where is Obama's foot


position? It's positioned on the right side of the scale. What happens as a result? The scale


shows higher. So you have to like carry the questions along. So I don't think that challenge


has been passed quite yet by flamingo. Another cool thing that I've seen this by Alexa Gordage,


the model is very non susceptible to some of the, let's say properties of visual models,


especially visual models. If you train them on classification tasks, they will often interpret


this top picture here as elephant because they focus on textures. Whereas I'm going to guess if


you train with more of a image language focus that tends to focus more on object DNS inside


of images. So flamingo correctly recognize what correctly recognizes this is a picture of a cat,


which I'm going to guess most humans would also see like this, even though the picture is clearly


I guess nonsensical if you want. And then if it doesn't, if there's no shape in it, it says this


is a picture of elephant skin. Another cool one is this one right here. There's this picture with


the this giant cat in it. And flamingo says this is a model of a city. It looks like a tiny city


with a lot of people and cars, which is correct. I don't know, a human would maybe say, well,


it's a tiny city and there's normal sized cat in it. But again, if you kind of lead the model to


what you sort of want to get out of it, it gets it ultimately. So it says, is there anything unusual?


It says, I see a cat in the photo. Is there anything unusual about the cat? The cat is very


big. So ultimately, you get the answer you want. Again, you have to sort of get stuff out of the


model. Yeah, it's not really Turing test passing yet, I would say in this tasks at least, but it's


getting there. Like there's not too much missing, I feel, until this is really at the level of


humans explaining these types of pictures, not AGI, just this task is very impressive. Not quite


there yet, as you often have to lead it. But you know, getting there. Another thing also by Lewis,


which I found quite funny is this one uploading a picture of I guess himself with the drawn on


beard. The model says this is a selfie of a man with a beard. Now there's two possibilities right


here. One is that the model was actually fooled by the beard. And two is that the model realized


that Lewis had put a lot of effort into that beard, but he just isn't very skilled at drawing,


it doesn't want to hurt his feelings. So it's complementing his drawing skills in this way.


We don't know it could be much smarter than you initially think. So you know, who knows. Now back


to Janik in the past. Now there is a paper to go along with it. It's 66 pages long and very detailed


and very cool and probably deserves its own video. Another highlight is that the model builds upon


the perceiver architecture, which means that as you can see here, for example, the image part


on the left here goes into a perceiver resampler model, which is one of these adapter model that


they train during training. And then that is routed into the main part via these learned


latent queries of the perceiver. So a perceiver in this way works a bit different than something like


GPT-3, which just always attends to the entire past. The perceiver has a latent stream of


information that it carries through and then tries to add to that using attention into the things


that come in. So into the pictures from here on out, and then it merges that into the entire


language model transformer. So here you can see one block of that language model transformer,


it is essentially, as you know, a language model, but it every now and then gets these cross inputs


from this gated cross attention dense layer, which is exactly where that visual information comes in


from the perceiver resampler. Another interesting thing here is the collection of data, they sampled


43 million web pages for images and text and used as far as I can tell visual analysis in order to


see which text goes with which image, which text comes before which text comes after, and so on.


And all of this is essentially processed to one stream of information that consists of images,


text and pointers to how they connect. Safe to say this paper is packed with information and


packed with cool ideas. And I think the possibilities of both vision text models together,


but also the combination of learned parts, and then frozen parts and how that all works,


how we can reuse knowledge that we take from single task models, and then fuse them together


is very promising direction, which brings us into the next story.


Google research releases a paper on locked image tuning, abbreviated lit. So this is a model that


combines the advantages of fine tuning, but also the advantages of contrastive pre training. Now,


if you do contrastive training, usually you take two pieces of information in these cases, again,


an image and a text and you try to align their embeddings as much as possible. This results in


representations in which you can do similarity search and all kinds of things over these inputs.


For example, the clip model can do zero shot classification by simply taking an image listing


all the classes and then evaluating the similarity of the representations to each of the classes.


On the other hand, there is the method of fine tuning in which you take some sort of model like


a pre trained image net model and then transfer or fine tune it to your task locked image tuning


combines both of them. So what it does is it has a pre trained image encoder and then it only fine


tunes the text encoder such that the pieces of text the representations match with the


representations that the image encoder gives. Now you might think that is a step back namely


something like clip tunes both the image encoder and the text encoder. However, what this paper


finds is that if you freeze the vision model, then you do get much better performance. There


is something about the contrastive training that almost loses a bit of information that the pre


trained vision encoder would contain otherwise. Therefore, simply fine tuning the text model


helps a lot in retaining a lot of that information. There's also a little bit of an online demo where


you can try out the tool so you can select one of these images and then you can give it a bunch of


classes a bunch of labels and then evaluate that model. What's also cool is that this demo runs


fully in the browser, you only have access to the tiny and small model because otherwise your


browser would catch fire, I guess. However, it's pretty cool that you can do this at all.


AI 21 labs releases Jurassic X in a blog post called crossing the neural symbolic chasm with the


MRKL system. This is the modular reasoning knowledge and language system and is abbreviated


miracle. Now this year is both about a concept for a new system, these miracle systems as well


as a first implementation of it in the form of Jurassic X. Now previously, AI 21 has built a


language model called Jurassic one, which was similar to GPT-3. And now they're using that


language model to interact with non neural systems. So the idea in miracle systems is that


you combine the language model together with a bunch of what they call experts. So experts here


are in form of, for example, a weather API, a currency converter, Wikipedia API, a calendar app,


some sort of calculator, stock databases, anything that you can query, get inputs from or send inputs


to and get some sort of results out of those. Notably, these are black boxes. And for sure,


they're not back probable, they're not differentiable. So the challenge is how do you


make these language models interact with these experts, the way Jurassic X does it is in form of


these input adapters, which will analyze the input query, which is in natural language, in this case,


which green companies had the largest increased share price in the last month, they will then


go out and query all of the available experts with the input that they parse out of the language,


merge all of this in a very discreet and symbolic fashion into a calculator expert that they also


have access to. And at the end, this goes back into a language model, which then gives you a


language answer. This obviously is a more tedious, more manual, more engineering intensive effort


than something like GPT-3 that simply gives you an answer out of the box. However, when it comes to


more complicated stuff, I believe that this approach here might be promising. So so far,


if we want something like GPT-3 to do multi step reasoning or anything like this, we can do it sort


of by prompting it very intelligently. However, it currently has its limits, you can clearly see how


this architecture would help a lot with that. However, on the other hand, the challenge with


this architecture is how do you connect these black box discrete parts with the neural parts,


and probably promising approaches might lie somewhere in the middle, either by making these


black boxes more accessible to the neural things like somehow combining them with back prop, or


maybe combining something like the reasoning prompt engineering from GPT-3 with traces of how people


use these experts. I don't know how it's ultimately going to look, but it's pretty cool. Additionally,


having these experts decouples what the language model needs to do namely language and parsing


things out of language with the abilities of the experts, for example, calculating some functions


or looking up some data online, which is something that frozen language models can't even do no matter


how big they are. Hey, this is Janek from the future. I don't think I've made this really clear


in the original post that I made about this, but you can actually enter your own queries here.


They have three experts available. One is the calculator, one is Jurassic one, the language


model, and one is the Titanic DB, so a database of passengers of the Titanic. So you can enter


any question in here. How much is the fish? And the fish costs $199. This is answered directly


by the Jurassic one model. Now let's let's try to make use of some of these experts. How many


passengers over 30 years old survived on the Titanic? Let's see if it gets it. Okay, select


count survived from Titanic. I like I came up I did not test this before this video. This is quite


impressive. How many meters is the capital of Tibet higher than the capital of India by how many


meters? Okay, well, still the Jurassic language model answered this one, I was trying to get it


to route to the calculator. But what I want to stress is that you can input your own queries


right here and directly try the model. This is very cool and very different from other models


that you don't have access to at all, like Dali or flamingo and where the website you can only


select predefined queries. So I wanted to highlight that here. Yes, they do have predefined


queries to give you an idea. However, you can also definitely go and just enter your own. So try it


out. Alright, and now we're going into a long list of helpful things and cool things that I've just


seen around style gun human is a gun or even multiple guns that are trained on humans, humans


and different clothing in different positions. And it's pretty good, I have to say they have a


nice website and a video to go along with it. If you want to know how it works, you can do things


like interpolation. And every time I see something like this, I'm just amazed at you know, how far


guns have come. This has nice applications for things like fashion design, or if you want to


shop something in an online store to see how it would look on you, or just kind of editing your


style and clothes and look as you can see right here, sleeves are shortened and elongated skirts


and pants and tops are edited all kinds of fun stuff. There's a paper and a GitHub repo and the


models are available to download graph is a data augmentation library for pytorch geometric. So this


brings data augmentation to graph neural networks. Usually data augmentation obviously very widely


used in things like vision. However, graph neural networks are kind of a new thing. And this library


provides some useful augmentation tools for that, especially if you work with pytorch geometric,


give this a try. Burton GPT j six B is a Spanish fine tuned version of GPT j six B. If you want to


generate text in Espanol, check it out. torch dist x is the repository for experimental stuff


in torch distributed. So this is where all the stuff goes that is not yet quite ready for pytorch


distributed. One cool thing is the fake tensor. Now they already have meta tensors and fake tensors


are quite similar, but I didn't know either. So I thought it was cool. This is a tensor that looks


like it's allocated somewhere on a device and you can work with it. However, it isn't so when you


want to access the data or do anything with it, it either fails or then needs to load the data at


that time. This is especially useful in order to do any sort of delayed execution of stuff,


but you already want to build the computation graph or in order to inspect models that are


just too large for you to load them all at once. So you kind of load the graph in but not the weights,


not any of the data and you just look at individual parts and you just load them as needed. Very cool.


Vector flow by Netflix is another neural network library. However, this one is optimized for sparse


data and single machine environments, which is a good use case is not a very standard one. So this


might actually be interesting for you to do some new things on it. I saw this blog post from the


iClear blog track the 37 implementation details of proximal policy optimization. And this is


obviously something that is well known in the domain of reinforcement learning, namely that


these algorithms are not stable, they require a lot and a lot and a lot of tricks to get working.


And that's why people usually regress to taking standard implementations rather than implementing


it themselves. Don't try to implement our algorithms yourself, it is a pain. And it is


almost always better to take a baseline even if you want to invent something new, start from a


baseline good implementation and work your way up from there. This blog post details 37 implementation


details for PPO. So if you want to figure out what kind of work goes into something like stable


baselines three, maybe give this blog post a read. DeepMind tweets about the release of four new


libraries for the JAX ecosystem, I have to say JAX is looking better and better by the day. So


there's MCTX, which does Monte Carlo tree search and JAX. There's KFAQJAX, which is a second order


optimization library. There's DMAUX, which is a sound and audio signal processing library. And


there's TF2JAX, which converts TensorFlow functions and graphs into JAX. MGPT is a 1.3 billion


parameter language model that has been trained on over 60 languages and is available on Hugging


Face. If you do anything in terms of multilingual natural language generation, this might be a good


starting point. CleanLab is an open source library that helps you find and correct labeling mistakes


in data. If you work in the real world and you have messy data and you suspect labeling errors


are a big problem, which they probably are, maybe give CleanLab a shot. They assist you in fixing


mistakes and sometimes they even do it automatically. Shout out to the Efficient Deep Learning book.


This book is in a draft stage and you can already look at the first four chapters of it. From the


table of contents, you can see that it deals with stuff like compression techniques, more efficient


learning techniques, and in the later stages, it takes some deep dives into specific platforms,


for example, TensorFlow or into specific models here, for example, BERT or MobileNet or EfficientNet.


So if you want to be not just a user, but a knower, maybe this book is for you. The MiniHack


level editor is a level editor for MiniHack. If you use MiniHack in any sort of experimentation,


which could be fun, it looks like fun, certainly this level editor will help you make some nice


test cases for your agent at inference time. I mean, you could do it for training time,


but how many are you gonna make? In any case, you can select width and height and you can place the


walls and you can place the lava. How about some lava here? Oh yeah. Mugen is a playground for


video audio text, multimodal understanding and generation. So this is a data set of gameplay


of an open source game. So here's how that looks. So what you saw is a piece of gameplay,


and then they also always showed you the pixel segmentation map. So this is video segments from


a bot playing this game. And the idea is that there are multiple modalities combined. So there's


always the sound, there is the picture that you get, and there is this pixel wise segmentation


map. So this is intended for you to build models that understand what's happening by considering


multiple input streams at once. It's very cool. They have 375,000 of these video clips. And along


the clips, they also have textual descriptions of what's happening in them. Amazon releases the


massive data set. This is a big data set, as the name says, and it is a 51 language data set


containing utterances in the respective languages along with code to get started. So if you're doing


very multilingual natural language understanding, this might be a good data set. OpenFax is an


open source 3d face animation system. So the goal here is to have a system that simulates realistic


facial expression. This is probably one of the hardest tasks you can do because we humans are


excellent at recognizing when a facial expression is not realistic. But the simulator makes a very


good impression and could be the basis of many ml inputs or output pipelines. See, it doesn't like


smile with your eyes. That's what they always say. You're not you're not smart. You're not smiling


with your eyes. Oh, wait, this could be the thumbnail. Avalanche is an end to end library


for continual learning based on pi torch. So this is a library that seeks to just make continual


learning a whole lot easier for you. I can imagine I've never done anything in continual learning,


but I can imagine that it's quite hard because many components need to come together. This stuff


goes on for a long time, even infinite time, right? You have to keep track of stuff, keep track


of checkpoints, be able to roll back have replay buffers, evaluate at regular intervals, yada, yada,


yada. So if there is a library that handles a lot of that in one very cool. And the last thing,


this AI does not exist.com will generate you descriptions and small usage code snippets of AI


projects that do not exist. So what you can do is you can click Next here. For example, this is the


hacker news reply guy AI, which is a bot for the hacker news comment section here you get a little


bit of a code snippet on how that works. If you press Next, then you'll get the next AI. But you


can also enter your own name. So let's try that. How about you only poop once. This is a little bit


of play on Yolo. Let's see what comes out pun intended. Your poll is a neural network that


learns to predict if an image of a dog will result in a poop or not. It was developed by a team at the


University of Bonn and the University of Amsterdam. Great work University of Bonn University of


Amsterdam. This is world changing. You can even see here the snippet says I should load the image dog


dog.jpg. Excellent, excellent. Apparently convnet. So I'm quite relieved. Alright, that was already


it for ML news this week. Thank you so much for being here. If you have news tips, don't hesitate


to come by our discord or if you just want to hang out. I mean, that's cool too. In any case,


Sockets Tutorial with Python 3 part 1 - sending and receiving data

What is going on everybody and welcome to a sockets with python tutorial series


in this video we're going to be working with sockets so we've got a lot of stuff


to cover so let's get into it so I've got a server.py and a client.py


already created so if you haven't done that which probably haven't go


ahead and make two files and we're going to get started in the server.py


so to get started we're going to import socket now socket is part of your


standard library so you won't need to install that or anything like that you


already have it now we want to do is define our socket object so s equals


socket dot socket and the socket family type is af inet and then the actual type


of socket is going to be socket dot sock stream now if inet corresponds to ipv4


and sock stream corresponds to TCP so this will be a streaming socket and if


anybody knows what af inet like why why is it called af inet like why don't we


just say socket dot I don't know ipv4 let me know down below really curious so


anyways it is what it is so that's our socket object now what we want to do is


bind that socket so s dot bind and we want to bind it to a tuple based on


sometimes this will be different but anyway the tuple will be in this case


because of the type of socket an IP and a port now this is our server so we're


actually just going to host the server on the same machine that we have the


code so we're going to bind to socket dot get host name so whatever the host


name is basically localhost and then the port we're going to go with one two


three four you can go with any pretty much four digit port the lower digits


are generally going to be already occupied by some other program and you


just don't want to step on their toes and more likely they're going to step on


your toes so once we've bound that socket also let's just talk real briefly


what even is a socket so a socket is just the end point generally you're


gonna have like two endpoints right to have a communication so that's just the


end point that receives data so with a socket you send and you receive data so


the socket itself is not the communication it's just the end point


that like receives that communication and that end point sits at an IP and a


port so we will s dot bind that socket now what we want to do is s dot listen


this is our server we're gonna be making some connections to the server so this


server has to be prepared for these incoming connections and this server


will prepare and leave a queue of five so if it's under some sort of heavy load


and like multiple connections are coming in so fast that it can't respond quickly


enough and they start stacking up we'll have a queue of five it should be more


than enough for our purposes so then we're gonna say is now we're just gonna


listen basically forever for connections and if we get one we're gonna say client


socket and then address equals s dot except so anybody connects hey we're


happy to see you we're gonna connect and then we're gonna store the client socket


object on into this variable client socket and then address is where are


they coming from so this will be like their IP address basically so so what we


can accept this and then the client socket this is just another socket


object much like our this socket object here we can send information to this


socket we can this socket will receive information as we will eventually


describe in client.py okay cool so once we've got that connection let's just


print some general debugging f strings we're gonna say connection from address


has been established established exclamation mark because this is serious


business okay so now we're gonna do is client socket dot send so we're actually


going to send information to the client socket which I feel like is weird like


the client socket is a socket object but it's like a foreign socket object so


we're going to send information to it but I almost feel like I don't know it


would make more sense to me if like we did s dot send and then we passed in


client socket this feels weird to me but anyways this is like our local version


of the client socket so to send information to that client we're gonna


say client socket dot send what do we want to send to this person so we're


gonna say bytes you also could just do like a string and then dot encode or


whatever anyways we're gonna say here welcome to the server and then what type


of bytes are these these are UTF bytes UTF 8 bytes so now that's pretty much


all we need to do to send some information so I'll save that now this


is not nearly enough for regular use but this a basic example as basic as I could


make it so now we're gonna work on our client.py so again client is much like


server it's gonna use the same exact socket so I'm actually gonna copy paste


that information now this socket rather than binding instead what this socket


really wants to do is connect so we're gonna say s dot connect and then we pass


a tuple again of the IP and the port that we wish to connect to in this case