are the problems of imperfect information games that we're going to tackle. So you see this poker


ReBeL - Combining Deep Reinforcement Learning and Search for Imperfect-Information Games (Explained)

thing is sort of a microcosm. And this was already half of the paper, if you understood why exactly


searching using kind of a value estimator with this combined with this tree search is a problem


in imperfect information games. So let's quickly go through the abstract, then we're going to have


to define a few terms. And then we can go into this algorithm, the algorithm is called rebel.


It's a general framework for self play reinforcement learning and search that provably converges to


a Nash equilibrium in any two player zero sum game. Okay. It says that in the simpler setting


of perfect information games, rebel reduces to an algorithm similar to alpha zero. And they say we


also show rebel achieves superhuman performance in heads up no limit Texas hold them poker while


using far less domain knowledge than any prior poker AI. So last video, I've had a comment,


I just hold them AI out there as far as I can tell. However, it is a very performant one that


uses very little domain knowledge of poker. So it like alpha zero removed basically all domain


knowledge out of the games it played this bot right here, I think the domain knowledge is to


the extent of it is given a limited set of bet sizes, even though it's kind of no limit hold


where you can bet whatever you want. It's given sort of a limited bet limited size of bet sizes


like half the pot full pot, two times the pot and so on. In order to make the actions discrete,


I think that's just easier for this algorithm. But in any case, the algorithm is applicable


pretty much anywhere where you have a two player zero sum imperfect information game or perfect


information. Okay, so let's shortly go over a little bit of background. So we're going to need


some terms right here. The first term we're going to need is what's called a world state. So a world


state is the state of the world. I know easy, easy. But it's quite important that to see that


in poker, what is the world state so in heads up, no limit hold them, there are your cards,


you get to your opponent gets two cards, right. And then there are board cards, like at at the end,


there are five, but maybe there are only three or they're not yet depends on the state of the game.


So the board cards, you know, this is maybe an ace, king and eight, you know, your two whole cards,


which is maybe an ace and an ace, but you don't know your opponent's cards. Okay, we're also going


to assume that the actions are always public. For the purposes of this video, they don't not


not necessarily for rebel the algorithm. But for us, let's just say the actions are all public.


So the world state is the fixed entire state of the world. So the world state would include


the your cards, the public cards and your opponent's cards. So the world state is sort of


like a super user can look at all of the cards. Okay, that's the world state. No one knows the


full world state, but it still exists. Okay, what we also need is, so there's the concept of actions,


there's an action space, which in poker is something like you can bet you can raise and so on. So these


are your classic actions. And there is a transition function, like in classic reinforcement learning.


So the transition function depends on the world state and the action and it gives you the next


world state. And after an action, each agent receives a reward that is also a function of the


world state and the action. Okay, so important to note that this is the reward you receive,


but you don't know the you maybe know the function, but you don't know the world state, right? So you


can't explicitly sort of predict your reward, you can maybe predict the distribution. Alright,


the next concepts are the concepts of observation, since we are in an imperfect information game,


an observation and the world state, these are not the same thing. Like in chess, you need to look at


the board. And that's all there is, that's all there is to know. So the world state and the


observation are the same thing. Here, there is the concept of private and public observations. Okay,


so public observation is like is what everyone knows in each step, whereas private observations


are things that are just revealed to you personally. In poker, the private observation


is simply your two whole cards and the public observation is the middle cards. So this is the


public observation and this is your private observation. So the private observation is


different for each player while the public observation is the same, I guess you could model


the public observation as simply another player that doesn't get any whole cards. But you know,


that's a question of semantics. Alright, the observations can also include the actions that


happened so far it just for completeness. If you like you can you can get information about hidden


actions and so on. There's lots of mathematical freedom here but just the concept is you have


private observations to each player individually, and then public observations. The subscript I


here always denotes a individual player while you see there is no such subscript in the public in


the public observations. Alright, the next concept is a history and a history is pretty much what you


think a history or a trajectory is a finite sequence of legal actions and world states denoted


by this so you can see it's simply the history of world states and actions that happened. Again,


no one knows the history fully, but it's still it is still the case and I know I know you can I


don't know quantum mechanics, many worlds theorem, blah, blah, blah. We'll just assume that whatever


you don't know these, these are fixed cards, they're actually there they have a value even


though no one has looked at them yet. So the world state is is defined even if you don't know it.


So the first real interesting concept here is called an info state. Okay, so the info state


is like the world state or like the history. But it's conditioned on what an individual player


knows. Okay, the info state also called an action observation history for agent i is a sequence of


an agent's observations and actions. So you can see it's very much like a history, except that


it doesn't have the world states. So usually there will be the world state here is that no,


there is the observation for player i at each of the time steps, okay, and these observations,


they include public and private observations, and along with the actions, but we'll say the


actions are public anyway. So an info state is basically the history as it looks to player i.


Okay, that's an info state. In our original game. We said that player two can't distinguish


between the three nodes. So if you look at the three nodes individually, like this node


one node two node three, these are three different world states with three different histories.


And to player two, they're simply the same info state, because all it all player two knows is that


player one has taken some action, it doesn't know which action. So the observation that player two


has is exactly the same. Therefore, it can't distinguish. So you can see that the info state


is sort of the correct abstraction that we're going to look at here. In, you know, in turn,


for if you look for player one, it looks different. Even though for player one, it's also three


different world states, it is also three different info states, okay, because player one knows which


action they have taken. So player one can decide which of these three states player two is in. So


player one is to player one, this corresponds to three different info states. So the info states


is always conditioned on a player. And it is the sort of unit that we'll look at here. Right, so


the info state, briefly, the it includes the observations and actions for a given player. And


the observations include the private and the public observations. The unique info state


corresponding to a history for agent i is denoted by this, the set of histories that corresponds to


some info state is denoted by large h. So as we said, if you have an info state, there are many


different histories that could have led to the info state, okay. So there are many different like


there maybe for player two, it looks like three different histories that could have happened,


lead to the same info state. Okay, that's but any given history determine fully determines the info


state. If I tell you what happened, you can give me the info state for each player, you can say,


ah, player one played rocks, therefore player two is in that info state and player one is in that


info state. So that's why there is a unique info state for each history. But there is a set of


histories for each info state. So the last last concept from here is a policy. A policy is again,


what you think it is. So it is something usually it's something that maps from an observation to


an action or from a history to an action or from a world state to an action. But here, it is a


function necessarily that maps from an info state to a probability distribution over actions. So two


things important here, the input to the policy is an info state. Since the players they can't


distinguish between the world states as long as they correspond to the same info state. Therefore,


their policy necessarily must be taking an info state as an input. So player two's policy cannot


depend on what player one did, because it can't distinguish, it can depend on the strategy of


player one, but not on the concrete action. The second thing is that we map to a probability


distribution over actions. This is usually the case in in RL, if you frame it as a general


principle. However, here, it's going to be quite important that this is always a probability


distribution. Very often in these games, your strategy is probabilistic. So there is no single


best move in rock, paper, scissors. But the best thing to do, the best strategy is to play each


move with a one third probability, or the modified version at the beginning. But it's important to


see that a policy will output a probability distribution. And I will also call this the


strategy of a player. So the strategy is going to be the policy. And I like to call it a strategy,


because it's sort of, it's a kind of a plan what you would do in each situation. And we're going


to see that that is going to be a central theme lifting in solving these games right here using


rebel. So policy profile is simply a tuple of policies. So it's simply the policies of all


players. That's the policy profile. If you combine the policy profile with some with some info state


or some history, you can calculate the expected value. So the expected value for a given history,


given that the players play policy pro players play policy profile pi. So this is all players


play their strategies in history h. And we're going to look at player i and its value so we can


calculate the expected value of some policies. So I can I can given this function v, I can input,


okay, here's what happened. And here's how everyone's strategy. Now tell me in expectation,


what the first player is going to net from this. Okay, solving the value function is pretty much


equivalent to solving the game. So if you if you give me a good value function, I can solve the


game by simply choosing the next action that gives me the best value function. But there's a


difficulty. We said, okay, we know pi strategies are public, but we don't know what history we're


in, right. So even if you had the perfect value function, I don't know what to input. So this is


going to be a problem. All right, the last thing is a Nash equilibrium, you might know this term,


a Nash equilibrium is a policy profile such that no agent can achieve a higher expected value by


switching to a different policy. Our goal here is going to be to find a Nash equilibrium strategy


for these games. And the rebel algorithm is going to provably converge to a Nash equilibrium.


All right. So okay, there's also the concept of a sub game, a sub game is defined by a root


history. It's simply if you're in a, it's simply a game that starts at some intermediate state,


that's a sub game. Okay, alpha zero, for example, constructs sub games. In fact, it constructs these


depth limited sub games, because you only solve up to a certain depth. And at that point, you sort


of ask your value estimator what the value is. This is different in different things. Like,


you can also do this, this kind of Monte Carlo estimation where you just play one trace to the


end, and so on. But the notion is we iteratively construct these depth limited sub games.


That means we play for a certain depth. And then we evaluate at that depth. And the question is,


how are we going to evaluate? Okay, so this is all sort of the build up. So we've built up that


we can't deal with world states like in classic games, we need to deal with info states. Okay. And


now, with info states, we have a problem namely, we can't use the alpha zero algorithm again,


because it will result in the thing on the right, okay? Because if we simply ask our value estimator,


our value estimator, even if it's perfect, we it won't lead us to the correct strategy, because the


value estimator here is the wrong tool. If we use the correct value estimator, we can't use the


tool. If we don't know all of the information because of this fact that the value of a node


doesn't only depend on the downstream actions, but also depends on the upstream strategies. Okay,


so in an info state, we can't distinguish where we are. And that means our value estimations are


going to be rather useless if we just apply this algorithm straightforward. So we need a way to


transform a game where we don't know everything to a game where we do know everything. It sounds a


bit weird. But that's exactly what we're going to do right here. So we're going to go from world


states to public belief states. And the world states are sort of what we would like to have,


but don't know the public belief states, those are going to be things that everyone knows. So


if we go from world states to public belief states, we're going to be in a situation again,


where everyone knows everything. And therefore, it is a perfect information game, it's going to


be a different game. But if we find the solution to this different game, we're going to end up with


the solution to this to the original game. For that they ask you to imagine the following game


consider a game in which one of 52 cards is privately dealt to each players. Okay, so you get


a card, your opponent gets a card, one card, by the way, 52. For those of you maybe in different


parts of the world, that's the number of cards in a standard card deck for like poker and blackjack


and so on. I know different countries have different things like in Switzerland, you'll


very often find 36 cards to a deck. But just that's why because 52 appears like a bit of a weird


number in any case. So on each turn, a player chooses between three actions fold, call or raise.


So these are the sort of standard poker actions, you can either throw away your card if you don't


like it, you can match the bet of your opponent or you can put in some money or some more money


yourself. And at the end, I'm going to guess. Yeah, here, eventually the game ends and players


receive a reward. So let's say whoever has the higher card wins the all the money in the middle.


Now consider a modification of this game, in which the players cannot see their private cards. Okay,


instead, their cards are seen by a referee. On the player's turn, they announced the probability


they would take each action with with each possible private card. The referee then samples


an action and the players on the players behalf from the announced probability distribution for


the players true private card. This is this is weird. So usually you'd look at your card like I


have an ace, okay. And then you come up with a with a sort of strategy, you come up with a policy,


you want to say, I'm going to raise with probability, ace is pretty good. So I'm going to


raise with probability point seven, I'm going to call with a probability of point two, and I'm


going to fold with a probability of point one. So this here, this would be an appropriate policy,


let's say for getting an ace at the beginning, right? Maybe this goes back and forth a bit and


you might change because you might change your belief, you don't know what your opponent has.


Now, the game changes, namely, the game is going to be your opponent gets a card and you get a


card and you don't get to look at even your card. So now you don't know your opponent's card and you


don't know your card. But what you can do is you can announce to the referee, you can say, okay,


referee, I am going to do this. If I have an ace, I'm going to raise with point seven,


going to raise with point seven, call with point two, and fold with point one,


if I have a king, I'm going to okay, I need a bit more space. If I have a king, I'm going to


raise with point six, I'm going to call with point three, and I'm going to fold with point one,


and so on. Until if I have a two, I'm going to raise with probability zero,


I'm going to call with probability point one, I'm going to fold almost all of it. Okay, so you


get to announce your entire strategy to the referee. The referee who is a super user, or,


I don't know, God. So, or, I don't know, choose your favorite deity, sees everything sees all


the cards, right? The referee will input will take this entire table that you give it as input,


it will go look at your card, it will see ah, it's a king or it's an ace, and it will then


choose the appropriate sub table here for you. And then it will sample an action from that.


So instead of you looking and just producing this table, you produce all the tables for all the


things that you could have. And then the referee does the same thing for you. Okay, and so does


your opponent. And you simply do this. So now you see, it's a bit of a different game. The the


namely the actions are different. So the action is no longer that you produce or search or the


policy is no longer you simply look at what you have and you determine the probabilities. Now the


policy is you spout out this table for all the things you could have and in each case for all


the things you could do. The important thing is so they say okay, at when the game starts,


each player's belief distribution about their private card is uniform random. And also about


the opponent's private card, right? However, after each action by the referee, players can update


their belief distribution about which card they are holding via Bayes rule. Likewise, players can


update their belief distribution about the opponent's private card through the same operation.


So it's important to note that this already happened before. So even if in the original game,


you would update your belief about the opponent's private card, according to Bayes rule, or


whatever you rule you want, you simply try to infer what they have. Now the difference is,


you also have to infer what you have, depending on what actions the referee does. So you sort of


treat yourself like a player, like a different player, like an opponent player that you don't


know the private cards of. Thus, the probability that each player is holding each private card is


common knowledge among all players at all times in this game. So that makes it such that you don't


know your opponent's card, you don't know your card, you have to use sort of the same algorithm


to determine what everyone has. So that means that all the knowledge is shared, like, no one knows


the true private cards, but everyone knows the same things. Okay, so if no one knows, then everyone


knows the same. It's sort of, it's a bit like, it's a bit like probability socialism. No one has


anything, everyone's equal. Sorry, that's us. That was a slight right there. So the important thing,


they say the critical insight is that these two games are strategically identical. That's the,


and that's very surprising. But if you think a bit about it, it becomes clear that your strategy


up here is the same as down here, you simply don't fully announce it every time, explicitly, but we


we said anyway, that policies are public. Therefore, this game here is equivalent to this


game. These are the same games. But the latter contains no private information. And is instead


a continuous state and action space perfect information game. Okay, while players do not


announce their action probabilities for each possible card in the first game, we assume that


all players policies are common knowledge, and therefore the probability that a player would


choose each action for each possible card is indeed known by all players. Okay, so


and this, you can even lift the restriction that you know or don't know the opponent's strategy. So


you don't actually need to know it, but we'll simply assume that everyone knows everyone's


strategy, they just don't know their, their private cards. So the, this is a new game that


we've constructed, where it's a bit different, right? There are different states, and they're


different actions. So the states that we deal with in this game, let's quickly analyze this. So


what's so we have state and action. In the in game one, the state is an info state. So this is


an info state. And the action is going to be a probability distribution over actions. So p of


each of the actions. In this game down here, we have different states and different actions. Now,


the states we're going to get to in a minute, but what's the action, the action is to send a table


of all these probability distributions in each case, like in case I have this in case I have this


in case I have this. So that's going to be the action, the action is going to be to send this


entire table to the referee. Okay, now what are the states? This is this next section,


we refer to the first game as the discrete representation, that's the top game. And the


second game as the belief representation. In example above a history in the belief representation,


which we refer to as a public belief state is described by a sequence of public observations and


104 probabilities, the probability that each player holds each of the 52 possible private cards.


Okay, the so this is going to be the state is going to be called a public belief state. And it's


described by the sequence of public observations and 104 probabilities. So the probabilities that


probability that you have an ace, you have a king, you have a queen, and so on, like the


distribution over your cards, and this distribution of your opponent's cards. So it's simply the info,


it's like an info state of someone that just observes the game, that is going to be the public


belief state. Okay. Likewise, an action is described by 156 probabilities, one per discrete


action per private card. In general terms, a PBS is described by a joint probability distribution


over the agents possible info states, you see it's a it's a distribution over info states.


So the state is a distribution for each info state. Or they also call this a public belief state.


So now, we've gone from a game that is imperfect information to a game that is perfect information.


Okay, this is this is this has unknowns like many like who Oh, this is different for each player.


But here, all the information is known. And these two games are equivalent. It's just that you can


see already the problem, like the the solution is that the information is known. And the solution


is that there is a problem like the states are way bigger, because it's a distribution over each


state that could be, and the actions are also way bigger, namely, it's an one policy for each state


that you could be in. So these are massive amounts, but in theory, that makes no difference, right?


They say, since any imperfect information game can be viewed as a perfect information game consisting


of public belief representations, or public belief states, in theory, we could approximate a solution


of any two player zero sum imperfect information game by running a perfect information, or l plus


search algorithm on a discretization of the belief representation. Okay, so nothing stops you from


simply taking this and running alpha zero on this new thing on this new thing with the states being


public belief states, and the actions being descending around of these giant tables, and you


might have to discretize it as it says, but that's feasible. So you can think of constructing this


game tree, but each node here is going to be a public belief state, okay, instead of a world


state like an alpha zero or like an info state, like we started these imperfect information games


with, and then you can construct your tree down here. And then, you know, but this is infeasible,


because these public belief states are just too large, and the actions are also too large. There


are so many actions. These are super high dimensional. So this is not feasible. And we're


going to so they have to find a way to do this thing. But to sort of do it in the domain of the


original game. And that's the I feel that's the entire trick of this rebel paper is to take the


this idea, let's do this search over the public belief states. But somehow this this thing down


here. Because what we need is we need the values of these, right? If we figure out the value of this


public belief state, and the value of this one, right, it's this beta one, this is of beta two,


then we would know which action to take. And then action is this huge thing. But if we knew the


values of these, we would know which action to take. However, this is not feasible. So we need to find a way to figure out these values using the original formulation of the game. And that's what they do in the exact next section, right here.


So they go on saying, however, as shown in the example above, believe representation can be very


high dimensional. So conducting search is, as is done in perfect information games would be


intractable. They say, fortunately, in two player zero some games, these high dimensional belief


representations are convex optimization problems. rebel leverages this fact via conducting search


via an iterative gradient ascent like algorithm. So I don't know what this sentence means that the


belief representations are convex optimization problems. Maybe this is misformulated, or I'm


just not understanding it well enough. In general, this section here is a bit of a mystery to me.


I can sort of tell you what what I understand of it. Okay. So they say rebels search algorithm


operates on super gradients of the PBS value function at the leaf nodes rather than on


PBS values directly. This is the first indication we don't want to work so we want to construct this


search tree. And at the leaf nodes, we need value functions right like an alpha zero. Now since we


operate on public belief states, we would need value functions of public belief states. However,


rebel finds a way to not do that. Specifically, the search algorithms require the values of info


states for PBS is okay. So they find a way to connect the values of info states to the values


of public belief states. And just as a reminder, an info state is a state that as it looks to one


player that could have many different histories, a public belief state has all the info states that


could lead to the public observation. So all the info states that you could be in, right with all


their histories here, basically a distribution over all these info states, that entire thing


is one public belief state. Now they are going to say, we can determine the value of a public belief


state. So the value of this is going to be equal to and we can somehow approximate this with the


values of these thing here, we somehow don't need the value of the entire public leaf state, we


connect this to the values of the individual info states. And that's I mean, that's done fairly


easily because you simply sum over. So we can say the value of a given info state condition that


you're in public belief state beta is simply going to be kind of the expectation over all the


histories that could lead to this info state multiplied by the value of the public belief


state. So the public belief state is going to be multiplied by the value of each history, like you


can have the value of a history given some policy. And therefore, you can approximate the value at a


given info state. And this theorem one here is where they connect the value of a public belief


state to the value of an info state. So they say, for any public belief state, for the beliefs of


any policy pi star, that is a Nash equilibrium of the sub game rooted at beta. So now we route


sub games at public belief states. This thing holds right here. So as you can see, this connects


the value of the public belief states, this is what we sort of need. In order for the search


algorithm to work, it connects it to the value of an info of info states. And info states are way


lower dimensional than public belief states. So it connects it connects the value of this right here


to the value of let's say this, okay, this this might be an info state here s. And the value,


it connects the value of the global public leaf set to the value of this particular info state.


And it does so via this term right here. So this term right here, this is just a unit vector in


the direction of that particular info state. And this here is a super gradient of an extension of


the value function to unnormalized belief distributions. As I understand it, this g is the


gradient with respect to probably beta one, if we care about s one to v one of beta, something like


this. As I said, this is where I don't 100% see through it. But what I understand is that this


connects the value of the public belief state this thing to the value of the individual info states


that are part of this public belief state. So we don't need a value function for public belief


states, we can simply get away with learning a value function for the individual info states.


And that's what they do. So the only the learned part here in this algorithm, this is the first


time we see like a neural network. Since rebel search algorithm uses info state values, rather


than learn a PBS value function, rebel instead learns an info state value function. So we're


going to input a public belief state. Yes. And we're going to get a value for each info state,


we're going to get a value here. So we'll simply learn a value function as sort of a vector output,


you could also input the public belief state and the info state and get out a single number,


I guess that would turn out to be the same thing. Okay, so the info state value function directly


approximates for each info state, the average of the sampled values produced by rebel at beta.


So we're going to learn this in a sort of bootstrap fashion, like like alpha zero does it


a bit like temporal difference learning. So what we're going to do in this algorithm is we're going


to start out, then we're going to construct this sort of this subtree. And we're going to do this


in the discrete representation of the game. Now, that's the genius of the rebel algorithm,


we're going to sort of evaluate these things into discrete representation in the info state


representation. And then we're going to be able to use what we find right here in order to determine


the value of the next actions to take, as far as I can tell. Okay, so that there is only one thing


left to do, right? We need to know how does how does this step here work. So we said we want to


do this tree search over the public belief states, but we can't, it's too cumbersome. Therefore, we


can now we can evaluate values of a public belief state, but we still need to do to determine the


policies. And that's where the self play reinforcement learning comes in. So bear with me


for one second, this is going to kind of snap together all that we've looked at so far. In this


section, we describe rebel and prove that it approximates a Nash equilibrium. At the start of


the game, a depth limited sub game rooted at the initial public belief state is generated. This


sub game is solved by running T iterations of an iterative equilibrium finding algorithm


in the discrete representation of the game, but using the learned value network to approximate


leaf values on every iteration. Okay, so it might seem a bit a bit complicated. But what we're going


to do is we're going to, here's what I think happens. And this is a bit unclear to me, we're


going to take a any public beliefs that we find ourselves in, they call they tell the beginning


of the game, but any any public belief state, okay, so the public belief state is maybe here,


and it contains many different info states. Now, what I think happens here is that they may be


sampling one of the info states, I don't know, or they may input the public belief states at the


beginning, this is unclear to me, but then they're going to solve the game in the discrete representation.


So they're going to use a classic solver to solve the game up to a limited depth. Okay, so this


limited depth is going to be sort of these steps in into the future. This is going to be in the


classic representation. So classic states and classic actions. Now the solver that they use for


this is counterfactual regret minimization. This is a solver that works with info states,


okay, so you can actually use CFR to solve poker. However, you can't solve all of poker because the


game is too big, right? So, but you can solve a sub game provided that you have good value estimates


here at the end. So that since they use CFR that leads me to believe they don't use the entire


public belief state as an input to see if our but they either maybe sample an info state or


they actually sample one particular history that happened. That is unclear to me. However,


what they do is they they do this, they solve the sub game using CFR. And then out of that,


they get a strategy. Okay, so here you ask your solver, what should I do? Given, you know, given


my estimates of the values right here, and the CFR will say, I know what you should do, here is


a strategy, here is a policy that you should do. Now, if this were alpha zero, if this were fully


observable, then you would be done, right? You'd say, okay, I'm done, cool. That's what I'm going


to do. However, what we saw above is that your values right here, your values down here, they


are dependent on what comes before you. Specifically, they are dependent on the strategy.


Okay, now, CFR, it needs sort of sort of an initial strategy. Okay, and it outputs a best strategy for


the given values. But now that you have another strategy, these values here, they are no longer


valid. And you computed the strategy with the values. So what you're going to do is you're going


to do is you're going to plug in, you're going to use this thing to compute new values. Okay, more


values, you're going to construct another set or the same sub game with new values, and then use


CFR again to solve that. And that will give you the next policy for these values. But then the


values change again, and so on. Now, this is going to converge eventually, but you're going to have to


run a couple of iterations of this for this to converge. In fact, I believe it's the the running


average or the average that's going to converge. But you're going to solve a number of these sub


games, okay, until you reach the actual best strategy. And you're going to do that down the


game tree. So from this thing, you're going to construct sub game, you're going to construct one,


two, three, updating the values, solving it. And then once you have it, you sample some state in


between from that you're going to solve the sub game again, one time two time three time and so


on until convergence, and so on. So this multiple solving of the same sub game, that's what we have


to do. So it is the price we have to pay for solving the game in the discrete representation,


because we can't solve it in the belief representation because it's too big. There,


we would only have to solve it once. But here, we have to solve it multiple times. So this is the


entire algorithm right here, you can see while the while we're not in a terminal state, we're


going to construct a sub game and initialize some some policy. And then for each step, we're going


to do first, sorry, we also set the leaf values. So this setting of leaf values, that's simply


forwarding, like, if I know the policy, I can go set the leaf values using my neural network,


right, my neural network can tell me what the value at each of the leaf nodes are, that's what


we train it for. So in the set leaf values, there is a neural network, you see this by the fact that


there are parameters right here. And then we're going to do repeatedly the following two things,


update policy. So this here is where we use the solver CFR. So we determine the best policy given


the current value estimations. And then we're going to set new values given the policy. So see,


see, if are it will take in the last policy, and it will output the next policy and set leaf values


will in will take in these parameters, which meaning this here, that's going to be some kind


of MLP or neural network. And we're going to do this, then we're going to loop back again and do


the same thing solve the game, set new values, solve the game, set new values, solve the game,


set new values, solve the game, set new values, okay, eventually, by aggregating all of this


information, we are going to be able to compute the expected value. And that's going to be the


value of the public belief state altogether. And as we said, if we know the value, we can sort of


take the best action. In fact, here, I believe that the policy that comes out this average policy


is the Nash equilibrium, and we can simply sample an action from that. All right, that's what they


describe here, they use, we describe rebel assuming the counterfactual regret minimization


decomposition CFRD algorithm is used, this is a depth limited version of CF of CFR. That's an


entire research direction by itself, right here, counterfactual regret minimization is simply used


as sort of the inner solver kind of a helper function to call. And that thing by itself is an


entire entire algorithm is like a very complicated algorithm. Okay. On each iteration, CFRD determines


a policy profile in the sub game. Next, the value of every discrete representation leaf node is set


to this and this is this is the neural network, right? So we're going to use the neural network


to set the leaf node values of the discrete representation. Okay. This means that the value


of a leaf node during search is conditional on the policy does the leaf node value change every


iteration, given pi and the leaf node values, each info state has a vet well defined values,


this vector of values is stored. And next, CFRD chooses a new policy profile in the process


repeats for T iterations. Alright, that's the rebel algorithm. And they also describe how they


actually sample data for learning with the exploration. And they also show that running


algorithm one with T iterations of CFRD in each sub game will produce a value approximator that


has an error of at most this for any PBS that could be encountered during play. Okay, so they're


going to say that the value approximator, given that it is sort of idealized, will actually


converge to a good value approximator, if you sample it depending on how many iterations of CFRD


you do. But you can see that the more iterations you do, the better of an approximation you get.


And if you have a good value estimator, as we already said, you basically have solved the game.


The last thing is that they determine now what do we do at test time, you might not have thought of


this, this, this was this seems sort of obvious if you know alpha zero, but they determine that


at inference time, you can simply run the same algorithm except you don't want to produce


training data from it, you don't want to learn anything, you simply want to run this algorithm


to if you run that algorithm at test time, that will actually give you a Nash equilibrium. So


that's theorem three right here. If algorithm run one runs a test time with no off policy


exploration value network with error at most this and this, and was trained as described in theorem


two with T iterations of that, then the algorithm plays a this kind of approximation Nash equilibrium


where c one and c two are game specific constants. Okay, so you can see right here that the Nash


equilibrium is going to be perfect depending on how many iterations you do. And depending on,


I believe how accurate your neural network is, yes, your value network error. Okay, if you make


that smaller, your Nash equilibrium is going to be better. Pretty, pretty cool.


So that was the algorithm, they do a bunch of experiments where they say, what kind of network


they use, if they use the value net or not, if they use self play or not. And they can also


introduce a policy net, I believe for initializing, or searching more effectively. They compare


against previous things like DeepStack, Liberatus, and so on. They do beat top humans, as you can see,


poker has been for a long time kind of an not so solved game by machine learning. But this area has


been over for a while right now. And they do release the code of, I believe, of the Liar's Dice.


So they have the code released for Rebel and the implementation for Liar's Dice, but not for poker,


because that's what they discuss in the broader impact statement. So let's quickly look at broader


impact, then we're done. So I just to say I love this broader impact statement. It is, it describes


like, it praises the paper. So it's kind of more advertisement for the paper. It does almost like


no harm to the paper itself to its reputation. It is actually accurate. So this broader impact


statement actually makes tangible predictions. And it doesn't go beyond the, or it mostly doesn't go


beyond the tangible things you can say about this algorithm. And it actually has as a conclusion,


an action that they take. So, and further, it is nothing like what the original specification of


broader impact statement says. And that makes me happy. So good job on this one. We believe Rebel


is a major step towards general agreement finding algorithm, yada, yada, yada. So they say, if this


is, this is good, because many things are sorry, sort of these kind of games, if you can extend it


to multi agent and so on. So this is the technology good section. But then the big thing is that


the most immediate risk posed by this work is its potential for cheating in recreational games such


as poker. While AI algorithm already exists, they say why, why they're better, why this particular


algorithm could be used for cheating, where the others can't be done so easily. By the way, this


algorithm, by nature of performing the searches over and over again, it needs a lot of compute,


like it needs a lot of compute. The learning isn't the problem. The problem is performing these searches over and over and over again.


Yeah, so it's not super easy to replicate, like don't, don't try this at home. However, if they would to release the pre-trained network, that would make it easy.


And they also say if they release the code, that would maybe make it easier to cheat. If you can simply run the code,


maybe, you know, you don't have the hardware, but given massive poker winnings, who knows.


Retraining the algorithms to account for arbitrary cheat styles, this is the requires more computationally feasible in real time.


That's about the other algorithms. However, Rebel can compute a policy for arbitrary stack size and arbitrary bet sizes in seconds.


So that's at inference time. Partly for this reason, we have decided to use the algorithm to


not to release the code for poker, instead open source our implementation for Liars Dice, a recreational game that is not played competitively by humans.


Okay, so it's a concrete prediction of the impact of the of this work. It has a concrete action to kind of its conclusion.


And it doesn't dabble in who know if, if we now solve these two player imperfect information games, then surely in the future, bombs will fly and stuff like this.


Yeah, good job on this again. All right. So this was the overview of the paper.


We started with the notion of info states and info states are kind of like states in classic reinforcement learning.


And we determined that we can't really use the sort of alpha zero way of doing things, because the value of info states not only depends on downstream things, but also on upstream things.


And the values here, yeah, that makes the values at the end of the tree not constant. And that means we can't really use that as we saw in this poker thing.


Then we converted the game from an info state representation to a public belief state representation, where now it's sort of it's again a everyone knows everything game.


Therefore, we could use the alpha zero way of doing things. However, since the states and the actions are so large, because it consists of these giant tables of numbers, we can't use the alpha zero for computational reasons.


Luckily, they find a way to connect the value function of public belief states to the value functions of info states.


And therefore, we can use a solver in the classic in the discrete representation to approximate or to to to use in this search procedure, as long as we run it multiple times and sort of keep updating its values.


By doing that, we can use this in this self play, simply iteratively doing this in each step. And we can use bootstrapping and play as we said self play between two agents.


And that will provably converge to a good value function and to an ash equilibrium.


ARC Challenge Live Coding

Hello, hello.


ARC Challenge Live Coding

I hope someone can hear me.


ARC Challenge Live Coding

I just need to quickly access the chat so I can even see you because I forgot.


ARC Challenge Live Coding

But if you're here, welcome.


ARC Challenge Live Coding

This is nothing special.


ARC Challenge Live Coding

We're just, thank you.


ARC Challenge Live Coding

We're just going to quickly jump into the Ark game.


ARC Challenge Live Coding

Hi, thanks for letting me know.


ARC Challenge Live Coding

So the Ark challenge is this game.


ARC Challenge Live Coding

I don't know if you've seen that.


ARC Challenge Live Coding

So you're given kind of these boards and there's always an input and an output.


ARC Challenge Live Coding

And so the left one is always the input, the right side is always the output.


ARC Challenge Live Coding

By the way, this is by Alex Borsky, a nice website, the Ark game.


ARC Challenge Live Coding

You can find it on GitHub and a link to it.


ARC Challenge Live Coding

So the left side is the input.


ARC Challenge Live Coding

The right side is the output.


ARC Challenge Live Coding

And you are supposed to do the pattern, you know, recognize the pattern from left to right,


ARC Challenge Live Coding

from left to right, from left to right.


ARC Challenge Live Coding

So this one is an easy one.


ARC Challenge Live Coding

It's basically the rule, as you can see, is there's always twice the same thing, like


ARC Challenge Live Coding

next to each other or here on top of one another.


ARC Challenge Live Coding

And you are required to fill out this test sample right here.


ARC Challenge Live Coding

So I guess we can do that.


ARC Challenge Live Coding

Actually, if you click on the left thing, it will actually copy it over to the right


ARC Challenge Live Coding

side, which just happens to be the the perfect solution here.


ARC Challenge Live Coding

But you can, you know, fill in anything you want right here.


ARC Challenge Live Coding

If we did it manually, we would do it like this, like this.


ARC Challenge Live Coding

So I'll just copy like the top part here, this, and like this.


ARC Challenge Live Coding

And you can check whether you get it correct.


ARC Challenge Live Coding

And we got it correct.


ARC Challenge Live Coding

Okay, very nice.


ARC Challenge Live Coding

Actually, I should probably so let me OBS.


ARC Challenge Live Coding

Let me put some sort of a banner right here of what we're doing.


ARC Challenge Live Coding

No, you cannot win a 3090 at this time.


ARC Challenge Live Coding

So we are solving ARC.


ARC Challenge Live Coding

Cool.


ARC Challenge Live Coding

Yeah.


ARC Challenge Live Coding

So, ARC challenge is this thing.


ARC Challenge Live Coding

All of these are really simple for humans, but really tough for machines.


ARC Challenge Live Coding

So here, you see, and you can't even formulate it, right?


ARC Challenge Live Coding

There's just kind of there's lines.


ARC Challenge Live Coding

So you're kind of supposed to just take the color that is not the lines, maybe, I guess.


ARC Challenge Live Coding

So we'll just all fill all of this with blue.


ARC Challenge Live Coding

Nice.


ARC Challenge Live Coding

So they get from simple to quite hard.


ARC Challenge Live Coding

So yeah, you can check it out for yourself.


ARC Challenge Live Coding

The data set has about 400 training example.


ARC Challenge Live Coding

And every single example is, do you have a link to ARC?


ARC Challenge Live Coding

Yes, sorry.


ARC Challenge Live Coding

Yeah, I should probably.


ARC Challenge Live Coding

So there's different things.


ARC Challenge Live Coding

I'll just I'll put a link, I'll put a link to this thing.


ARC Challenge Live Coding

I'll put a link here.


ARC Challenge Live Coding

So yeah, these go from easy to super hard.


ARC Challenge Live Coding

But for humans, they're never hard.


ARC Challenge Live Coding

But for machines, as you can imagine, they're quite hard.


ARC Challenge Live Coding

And every one of them uses kind of like a different notion.


ARC Challenge Live Coding

So there's not twice, there's rarely twice the same kind of logic behind how you solve


ARC Challenge Live Coding

these games.


ARC Challenge Live Coding

I've never tried Google's Fubar coding.


ARC Challenge Live Coding

I don't even know what that is.


ARC Challenge Live Coding

I'm sorry.


ARC Challenge Live Coding

It's been a long time since I've done actual coding challenges.


ARC Challenge Live Coding

However, we at our Discord have decided to give this a little bit of a shot.


ARC Challenge Live Coding

So we have, I should probably, I think this is public, github.com, ARC community maybe?


ARC Challenge Live Coding

Yeah, exactly.


ARC Challenge Live Coding

So there you can see a bunch of repos and we've just started.


ARC Challenge Live Coding

So my goal in this repository right here is to provide a little bit of a framework, a


ARC Challenge Live Coding

little bit of a framework or of evaluating these things.


ARC Challenge Live Coding

And I got pretty far so far.


ARC Challenge Live Coding

So the idea is that I have a test right here.


ARC Challenge Live Coding

The idea is that you would implement an agent.


ARC Challenge Live Coding

So an agent is this thing.


ARC Challenge Live Coding

And the agent implements a method called solve test sample.


ARC Challenge Live Coding

And there you get a list of training samples.


ARC Challenge Live Coding

These are these three samples or so.


ARC Challenge Live Coding

And you get a test board and you're supposed to output top k list of solutions.


ARC Challenge Live Coding

Top k list is simply a list of boards.


ARC Challenge Live Coding

So you're supposed to output boards.


ARC Challenge Live Coding

You can make use of these hints right here, although you will not get any points in the


ARC Challenge Live Coding

final result if you do.


ARC Challenge Live Coding

But for a start, that's that you can use that.


ARC Challenge Live Coding

The task data simply contains the top k that you're supposed to output.


ARC Challenge Live Coding

So it's pretty, pretty simple.


ARC Challenge Live Coding

And you're simply, you simply need to implement this method.


ARC Challenge Live Coding

And then the goal here is that I then take your agent and evaluate it on the data set,


ARC Challenge Live Coding

give it a nice report and so on.


ARC Challenge Live Coding

So that you can try yourself like how would you go about programming an algorithm that


ARC Challenge Live Coding

can solve these types of tasks, which is really hard, especially there's only 400 training


ARC Challenge Live Coding

examples, as I said, and 400 test examples and none, no two examples use the same logic.


ARC Challenge Live Coding

You just need to generalize from like the essentially the in context learning right


ARC Challenge Live Coding

here.


ARC Challenge Live Coding

And yes, you could use a large language model for this, but GPT4chan made you notorious,


ARC Challenge Live Coding

meaning I wasn't notorious before.


ARC Challenge Live Coding

No, yeah, that was quite something.


ARC Challenge Live Coding

But we're back to family friendly stuff.


ARC Challenge Live Coding

No one, absolutely no one's going to complain about color squares, right?


ARC Challenge Live Coding

So this is completely controversy free.


ARC Challenge Live Coding

That's why it's a good project.


ARC Challenge Live Coding

So this is here is an implementation of a cheating agent.


ARC Challenge Live Coding

So the cheating agent, as I said, you can use these hints right here.


ARC Challenge Live Coding

And one of the hints is actually the output.


ARC Challenge Live Coding

And the output will just give you the solution, right?


ARC Challenge Live Coding

So the cheating agent actually looks at the output, and just returns that.


ARC Challenge Live Coding

And you know, good enough.


ARC Challenge Live Coding

And in fact, it will be recorded in the hints that you actually access that.


ARC Challenge Live Coding

So we'll have that in the evaluation.


ARC Challenge Live Coding

So the goal now that I have today is to finish this setup of this evaluation code.


ARC Challenge Live Coding

So people can actually use it productively.


ARC Challenge Live Coding

Let's say I already have code to download the data set and so on.


ARC Challenge Live Coding

But I'm having a bit of trouble how it when it comes to metrics.


ARC Challenge Live Coding

So metrics in my mind is this thing, and we can look at it.


ARC Challenge Live Coding

So a metric could be like accuracy, accuracy is like or correctness, whether or not you


ARC Challenge Live Coding

solved a sample correctly, that's kind of the most basic metric.


ARC Challenge Live Coding

But there can be more metrics, right?


ARC Challenge Live Coding

There can be how much of the pixels you got correct and so on.


ARC Challenge Live Coding

So these aren't counted in the end, you know, you only get a point if you get it exactly


ARC Challenge Live Coding

correct.


ARC Challenge Live Coding

However, they can tell you something about the progress you're making.


ARC Challenge Live Coding

So my idea of a metric is it has two methods, one is compute, and you would give that a


ARC Challenge Live Coding

single result, right, a single evaluation result, you give to the metric, it will produce


ARC Challenge Live Coding

something.


ARC Challenge Live Coding

For example, right?


ARC Challenge Live Coding

Did you solve the riddle correctly?


ARC Challenge Live Coding

And the aggregate function then would aggregate this across all of essentially across all


ARC Challenge Live Coding

of the evaluation results, all of the riddles.


ARC Challenge Live Coding

So one metric that I implemented here is the correctness metric, as you can see right here.


ARC Challenge Live Coding

So this would compute over the all the test examples of one riddle, the minimum value.


ARC Challenge Live Coding

So did you get essentially you get one if you get all the riddles correct.


ARC Challenge Live Coding

And it would then aggregate across all of them in a mean fashion.


ARC Challenge Live Coding

So the result of this metric would be how many or what fraction of riddles of the data


ARC Challenge Live Coding

set did you get correctly?


ARC Challenge Live Coding

That's it.


ARC Challenge Live Coding

Now, so far, what I've done is I have implemented these as property of the evaluation result,


ARC Challenge Live Coding

you can see here is the evaluation result.


ARC Challenge Live Coding

That's constructed as your agent outputs a solution, this is, you know, done.


ARC Challenge Live Coding

And here I have metrics results in it.


ARC Challenge Live Coding

I don't really like it.


ARC Challenge Live Coding

I think the metric you should be able to save the evaluation result and then compute maybe


ARC Challenge Live Coding

the metric even later on that evaluation result.


ARC Challenge Live Coding

So we're going to rip out the metrics results from here.


ARC Challenge Live Coding

And the aggregation results from this eval result list, which I also construct.


ARC Challenge Live Coding

And then hopefully, hopefully, how good is the state of the art on Arc?


ARC Challenge Live Coding

I believe it's really weak.


ARC Challenge Live Coding

Like it's certainly below 20% or so or 30% solved, or even even weaker, like it is it


ARC Challenge Live Coding

is quite weak.


ARC Challenge Live Coding

There is not a good algorithm yet.


ARC Challenge Live Coding

And it has some some properties, right?


ARC Challenge Live Coding

You actually need to, you need to hard code human priors that that's actually it's in


ARC Challenge Live Coding

the paper.


ARC Challenge Live Coding

So yeah, it's tough.


ARC Challenge Live Coding

Alright, so I'm going to rip essentially out these metrics results right here.


ARC Challenge Live Coding

And I'm going to manage the metrics results externally.


ARC Challenge Live Coding

So let's say so here we have, yeah, no, no.


ARC Challenge Live Coding

Okay, so here is a metric and the metric we can actually also rip out into its own file.


ARC Challenge Live Coding

Let's say we yeah, into the metrics file, why not?


ARC Challenge Live Coding

We have a metric right here, like that and oopsie is the eval result in the interface?


ARC Challenge Live Coding

Yes.


ARC Challenge Live Coding

Okay, so we'll import that we already have it.


ARC Challenge Live Coding

Okay.


ARC Challenge Live Coding

And we need that.


ARC Challenge Live Coding

The eval result here cannot be a string anymore.


ARC Challenge Live Coding

Okay, I mean, this is this is already good enough.


ARC Challenge Live Coding

Can you share you set up the vimrc?


ARC Challenge Live Coding

So yeah, the IDE is neo vim.


ARC Challenge Live Coding

And I am currently really strongly considering redoing my vimrc.


ARC Challenge Live Coding

So first, I want to use the the Lua configuration of neo vim.


ARC Challenge Live Coding

And I want to like get different different plugins because these have become quite a


ARC Challenge Live Coding

mess.


ARC Challenge Live Coding

You will also see if you watch me code that pirate like the loading bar is currently going


ARC Challenge Live Coding

all the time and stuff.


ARC Challenge Live Coding

So it's, it's really become not good and jumbled.


ARC Challenge Live Coding

So I'll redo it.


ARC Challenge Live Coding

And once I've read on it, and I'm actually proud of the setup, then I can I can maybe


ARC Challenge Live Coding

share it.


ARC Challenge Live Coding

But I'm using relatively standard plugins.


ARC Challenge Live Coding

I'm also using coC.


ARC Challenge Live Coding

I'm using co pilot.


ARC Challenge Live Coding

And yeah, I don't actually know if co pilot works in this, because I'm using Python 3.9


ARC Challenge Live Coding

and neo vim complained.


ARC Challenge Live Coding

Anyway, so we get compute and aggregate.


ARC Challenge Live Coding

And so the yeah, the problem here a little bit is


ARC Challenge Live Coding

well, yeah, I will make these actually underscore functions because we can then track.


ARC Challenge Live Coding

So a pattern that I like to use often is anything wrong with VS code.


ARC Challenge Live Coding

It's not vin that's kind of the only thing.


ARC Challenge Live Coding

So you know.


ARC Challenge Live Coding

Okay, and good.


ARC Challenge Live Coding

I know this seems pointless, but it is not.


ARC Challenge Live Coding

Because because yeah, it is not pointless because you can give users kind of the opportunity


ARC Challenge Live Coding

to override this, but you still have a hook into it, right with the non underscore functions.


ARC Challenge Live Coding

As a framework developer, it's an interesting pattern that I think makes sense a lot of


ARC Challenge Live Coding

times if you want to give people, you know, the opportunity to subclass things, and you


ARC Challenge Live Coding

can still do logging and track access and so on in the non underscore function, especially


ARC Challenge Live Coding

here.


ARC Challenge Live Coding

We now have the opportunity to instead of this, give the eval results list.


ARC Challenge Live Coding

No, I missed by one.


ARC Challenge Live Coding

Okay, see, it's too, it's kind of slow.


ARC Challenge Live Coding

So yeah.


ARC Challenge Live Coding

So now here we have to so out of the eval results, we have to fish out.


ARC Challenge Live Coding

No actually, yeah.


ARC Challenge Live Coding

So now, now we get into trouble, right?


ARC Challenge Live Coding

Let's say we just cancel this for now.


ARC Challenge Live Coding

And we make these into underscores.


ARC Challenge Live Coding

So when we eval something, we have a result.


ARC Challenge Live Coding

And then we want to compute a metric on it.


ARC Challenge Live Coding

And that gives us a metrics result, right?


ARC Challenge Live Coding

And in that case, it's any now I'm going to play around with the typing or something.


ARC Challenge Live Coding

But but in our case, it's just kind of any.


ARC Challenge Live Coding

So that's just a result.


ARC Challenge Live Coding

We need to store that result somehow in a data structure.


ARC Challenge Live Coding

So let's go over to the eval here.


ARC Challenge Live Coding

So here we have apply metrics.


ARC Challenge Live Coding

And yeah, so what we'll do is we'll go.


ARC Challenge Live Coding

Now there's the problem.


ARC Challenge Live Coding

We'll go by metric by metric.


ARC Challenge Live Coding

We'll go through all the results.


ARC Challenge Live Coding

What are your thoughts on the Minerva model?


ARC Challenge Live Coding

I have not read I will read the paper as soon as possible.


ARC Challenge Live Coding

But I'm really trying to get this done.


ARC Challenge Live Coding

So the community can actually use this code.


ARC Challenge Live Coding

But it's it's super cool.


ARC Challenge Live Coding

And I think that so I've read it a little bit, but not in detail.


ARC Challenge Live Coding

But my opinion so far is, is data quality is really king right there.


ARC Challenge Live Coding

They just parse.


ARC Challenge Live Coding

This is not I've also talked to some people about it.


ARC Challenge Live Coding

And it seems like how well you can parse PDFs and stuff and LaTeX is a big part of of your


ARC Challenge Live Coding

data quality and the data quality directly influences the results.


ARC Challenge Live Coding

So better take good care of your data.


ARC Challenge Live Coding

It actually pays to pays off to clean your data well and preprocess it well and so on


ARC Challenge Live Coding

seems to be a real boost.


ARC Challenge Live Coding

Alright, so now we're going through every result.


ARC Challenge Live Coding

So for eval result, actually, we can we can just use we can just ask copilot here.


ARC Challenge Live Coding

See metric result.


ARC Challenge Live Coding

Okay, so we get a result for each of these.


ARC Challenge Live Coding

And we can probably make this into a list comprehension actually, because metric dot


ARC Challenge Live Coding

compute like this.


ARC Challenge Live Coding

So that's metric results.


ARC Challenge Live Coding

So for every riddle, we now have a result.


ARC Challenge Live Coding

And then what we're going to do is we're going to give that to the to the aggregation function.


ARC Challenge Live Coding

So metric dot aggregate.


ARC Challenge Live Coding

Now it depends on how we've implemented this, right?


ARC Challenge Live Coding

Let's go over to the metric.


ARC Challenge Live Coding

And so what do we have here, we have the eval results, and we also need a list of metrics


ARC Challenge Live Coding

results, metric results.


ARC Challenge Live Coding

These are the things that have been computed by compute above.


ARC Challenge Live Coding

List any for now, it's not pretty.


ARC Challenge Live Coding

It's not pretty, but you know, that's, that's what we get.


ARC Challenge Live Coding

Um, yes, I think that's that we can also have that exact same interface right here.


ARC Challenge Live Coding

And we can just replicate this here.


ARC Challenge Live Coding

Not even it's reckless to not check copilot output, but sometimes you're just so confident


ARC Challenge Live Coding

that you can just give it to get to copilot.


ARC Challenge Live Coding

Alright, so now we what we need is a data structure that keeps all of this stuff, right?


ARC Challenge Live Coding

Again, we're going to probably go to metric and we are going to make we'll need pydanetic.


ARC Challenge Live Coding

If you don't know pydanetic very good library.


ARC Challenge Live Coding

Like data classes, except cooler metric result.


ARC Challenge Live Coding

Yeah, why not?


ARC Challenge Live Coding

Let's call it metric results.


ARC Challenge Live Coding

So it's like, yeah, it's like data classes, but nicely serializable, nice print functions


ARC Challenge Live Coding

and to JSON from JSON, and so on, and and with validators, really nice things.


ARC Challenge Live Coding

So the metric result here.


ARC Challenge Live Coding

Now, the only issue I have a little bit is the task has several riddles, right?


ARC Challenge Live Coding

And they're just in a list.


ARC Challenge Live Coding

Now, the metrics results also obviously you'll get a list of results.


ARC Challenge Live Coding

The question is, can we simply like have two lists and sort of implicitly be like, well,


ARC Challenge Live Coding

the first element is the first element and the second corresponds to the second.


ARC Challenge Live Coding

That's really not very nice.


ARC Challenge Live Coding

But I don't know, like we would need to make everything into like a dict.


ARC Challenge Live Coding

That's also not particularly sorry, particularly nice.


ARC Challenge Live Coding

So I think I'll just stick with the list.


ARC Challenge Live Coding

So I'm going to say eval root.


ARC Challenge Live Coding

Okay, how are we calling these?


ARC Challenge Live Coding

Let's call them riddle results.


ARC Challenge Live Coding

And this is a list and these and this is like aggregation result.


ARC Challenge Live Coding

Okay, now let's call that compute results because the method is compute.


ARC Challenge Live Coding

And aggregate results result.


ARC Challenge Live Coding

And we'll still need to do something about that any but I will have to read the Python


ARC Challenge Live Coding

documentation on typing to really see how we can do this with inheritance and so on


ARC Challenge Live Coding

because the idea is that you could define your output type here.


ARC Challenge Live Coding

And then you the type checker could make sure that you'll get the same thing.


ARC Challenge Live Coding

For now, it's just any I'm sorry.


ARC Challenge Live Coding

Okay, I know you can do type var, but I don't know how that behaves with inheritance.


ARC Challenge Live Coding

Although I think you could here you should be able to do something and then have that


ARC Challenge Live Coding

show up.


ARC Challenge Live Coding

Can we okay, maybe we'll do this.


ARC Challenge Live Coding

All right.


ARC Challenge Live Coding

Generic, generic, generic.


ARC Challenge Live Coding

Aha.


ARC Challenge Live Coding

Okay.


ARC Challenge Live Coding

So a generic type is from an instantiation of this class.


ARC Challenge Live Coding

Yes.


ARC Challenge Live Coding

Generic.


ARC Challenge Live Coding

Must be a subtype of something.


ARC Challenge Live Coding

We could do this, right?


ARC Challenge Live Coding

We could have inherit from generic and then we could Yeah, okay.


ARC Challenge Live Coding

From typing generic and then here we'd have generic T. Now T isn't known so we also need


ARC Challenge Live Coding

to type var.


ARC Challenge Live Coding

T equals type var metric type.


ARC Challenge Live Coding

What's wrong with that?


ARC Challenge Live Coding

Let's just do it.


ARC Challenge Live Coding

Let's just say it's T. So we can have all of these be T, T, T, T.


ARC Challenge Live Coding

Right?


ARC Challenge Live Coding

And then we need to just take care that here.


ARC Challenge Live Coding

So now I'm wondering here this is a float.


ARC Challenge Live Coding

And then we get a float right here.


ARC Challenge Live Coding

And this here is also float.


ARC Challenge Live Coding

I'm feeling like a duper super programmer.


ARC Challenge Live Coding

And obviously, ah, okay, pidantic and generics.


ARC Challenge Live Coding

Do they play well?


ARC Challenge Live Coding

Do they do they go with one another?


ARC Challenge Live Coding

Let's check it out.


ARC Challenge Live Coding

I'm not sure.


ARC Challenge Live Coding

But okay, okay.


ARC Challenge Live Coding

I see the problem.


ARC Challenge Live Coding

Interesting.


ARC Challenge Live Coding

Yes, we're doing we're doing API stuff.


ARC Challenge Live Coding

Generics from Java.


ARC Challenge Live Coding

Yeah.


ARC Challenge Live Coding

Java is a mess, especially generics.


ARC Challenge Live Coding

However, generic models, there we go.


ARC Challenge Live Coding

Generic models are only supported.


ARC Challenge Live Coding

That's fine.


ARC Challenge Live Coding

That's fine, too.


ARC Challenge Live Coding

That's okay.


ARC Challenge Live Coding

So here, we have generic model.


ARC Challenge Live Coding

Generic data T. Okay, seems like we have to, we have to inherit from


ARC Challenge Live Coding

generic model.


ARC Challenge Live Coding

Can we do that?


ARC Challenge Live Coding

You see the, yeah, pyrite is going.


ARC Challenge Live Coding

And generic T. And then here we get a T.


ARC Challenge Live Coding

I'm still not super satisfied.


ARC Challenge Live Coding

Because because well, how about we have aggregate actually output the


ARC Challenge Live Coding

result.


ARC Challenge Live Coding

Right.


ARC Challenge Live Coding

Like that.


ARC Challenge Live Coding

Like this.


ARC Challenge Live Coding

And well, here we can't have metric results.


ARC Challenge Live Coding

We'll need to call it compute results too.


ARC Challenge Live Coding

Sorry, I'm neglecting chat.


ARC Challenge Live Coding

Are we doing API stuff?


ARC Challenge Live Coding

Yes, we're doing API stuff.


ARC Challenge Live Coding

Hopefully, hopefully.


ARC Challenge Live Coding

Okay.


ARC Challenge Live Coding

Compute results, compute results.


ARC Challenge Live Coding

Now we're ready to everything's called result.


ARC Challenge Live Coding

Suck at naming things.


ARC Challenge Live Coding

So yes.


ARC Challenge Live Coding

Yes.


ARC Challenge Live Coding

Thank you, Copilot.


ARC Challenge Live Coding

That is quite nice of you.


ARC Challenge Live Coding

All right.


ARC Challenge Live Coding

So the compute methods is fine here is all fine.


ARC Challenge Live Coding

All good.


ARC Challenge Live Coding

And the aggregate method is fine.


ARC Challenge Live Coding

And now we simply need to implement we'd have to change the definition of this aggregate


ARC Challenge Live Coding

right here.


ARC Challenge Live Coding

So here we have a class doing mean aggregation.


ARC Challenge Live Coding

If you see this, this needs to be a float, obviously, if it's some sort of a mean.


ARC Challenge Live Coding

And yeah, that's it.


ARC Challenge Live Coding

If not, okay.


ARC Challenge Live Coding

Yep.


ARC Challenge Live Coding

Can you plug Copilot in Vim as yes, Tim Pope himself actually wrote the Copilot plugin


ARC Challenge Live Coding

and does not accept pull requests currently.


ARC Challenge Live Coding

So but yeah, exactly.


ARC Challenge Live Coding

Get value.


ARC Challenge Live Coding

Where do I have this get value?


ARC Challenge Live Coding

Gives a float.


ARC Challenge Live Coding

I think this is fine.


ARC Challenge Live Coding

Luckily, oopsie.


ARC Challenge Live Coding

Luckily, I have written tests.


ARC Challenge Live Coding

Let's see how that goes.


ARC Challenge Live Coding

That is good.


ARC Challenge Live Coding

Very good.


ARC Challenge Live Coding

Very excellent.


ARC Challenge Live Coding

What do you mean?


ARC Challenge Live Coding

What could not be dastardly?


ARC Challenge Live Coding

But it says here a generics.


ARC Challenge Live Coding

Okay.


ARC Challenge Live Coding

All right.


ARC Challenge Live Coding

Nope.


ARC Challenge Live Coding

Generics.


ARC Challenge Live Coding

There we go.


ARC Challenge Live Coding

Okay, now our problem is that in eval, we can't import metric.


ARC Challenge Live Coding

That makes sense.


ARC Challenge Live Coding

Do we need it?


ARC Challenge Live Coding

We actually need it.


ARC Challenge Live Coding

Good.


ARC Challenge Live Coding

And metric compute, eval result, eval result.


ARC Challenge Live Coding

Tuple is not compatible.


ARC Challenge Live Coding

Oh, no.


ARC Challenge Live Coding

See, now we get into typing, typing issues.


ARC Challenge Live Coding

So we're in the eval results list, which is, we actually need to access eval results.


ARC Challenge Live Coding

Eval result list, dot eval results.


ARC Challenge Live Coding

Actually we could just extract that here.


ARC Challenge Live Coding

Like this.


ARC Challenge Live Coding

And we're pretty good.


ARC Challenge Live Coding

I think.


ARC Challenge Live Coding

Let's see.


ARC Challenge Live Coding

Wow, that's slow.


ARC Challenge Live Coding

All right.


ARC Challenge Live Coding

That's better.


ARC Challenge Live Coding

So this has no attribute.


ARC Challenge Live Coding

This has no attribute aggregation results.


ARC Challenge Live Coding

Okay.


ARC Challenge Live Coding

Aggregate.


ARC Challenge Live Coding

I called it aggregate result.


ARC Challenge Live Coding

Yeah, so now the tricky part is, yeah, we are not done by a long shot.


ARC Challenge Live Coding

See here, I say apply metrics.


ARC Challenge Live Coding

This applies the metrics.


ARC Challenge Live Coding

Now what's it, what, what is it going to do?


ARC Challenge Live Coding

It's going to call compute and that is fine.


ARC Challenge Live Coding

Are trying to make an API for metric eval?


ARC Challenge Live Coding

Yes, exactly.


ARC Challenge Live Coding

So I'm trying to write code for other people to use to evaluate their, their agents on


ARC Challenge Live Coding

the arc challenge.


ARC Challenge Live Coding

If you're new here, the arc challenge is, is this thing.


ARC Challenge Live Coding

This yes.


ARC Challenge Live Coding

So we are here and we'll, we then call aggregate.


ARC Challenge Live Coding

Aggregate takes the results list and the results from compute.


ARC Challenge Live Coding

So we'll actually rename that to compute results.


ARC Challenge Live Coding

We'll rename that to eval result list and we'll get a metric result out of that.


ARC Challenge Live Coding

So now we get a result per metric.


ARC Challenge Live Coding

Now the metric result, the metric itself has a name.


ARC Challenge Live Coding

Therefore the metric result should probably also have a name.


ARC Challenge Live Coding

That makes a lot of sense.


ARC Challenge Live Coding

Yeah, yes, it does.


ARC Challenge Live Coding

It does make a lot of sense.


ARC Challenge Live Coding

So good.


ARC Challenge Live Coding

All right, the metric result now has a name and metric results.


ARC Challenge Live Coding

We'll add that to the list.


ARC Challenge Live Coding

We'll return that list.


ARC Challenge Live Coding

We'll say here that is a list of metric result.


ARC Challenge Live Coding

A metric result will import from here.


ARC Challenge Live Coding

All right, so now we just need to, what's the console text editor.


ARC Challenge Live Coding

This is NeoVim, which is very slow because I'm streaming.


ARC Challenge Live Coding

I have the camera as like an external camera feed over an HDMI.


ARC Challenge Live Coding

I have a second screen that is streaming to the iPad.


ARC Challenge Live Coding

And I'm running Copilot and this is an API, but I'm running a PyRite language server in


ARC Challenge Live Coding

the background.


ARC Challenge Live Coding

So this thing is stressed, but it works.


ARC Challenge Live Coding

It's fine.


ARC Challenge Live Coding

And my hard disk is almost full, which I figured is not good.


ARC Challenge Live Coding

Like if you're below 10%, things get slow.


ARC Challenge Live Coding

Need to clean up.


ARC Challenge Live Coding

All right.


ARC Challenge Live Coding

We'll have the eval test right here.


ARC Challenge Live Coding

For this thing now, if you apply metrics, you should get a list of these metric results,


ARC Challenge Live Coding

right?


ARC Challenge Live Coding

Which correspond to, um, these metrics here.


ARC Challenge Live Coding

So the issue is it's just a list.


ARC Challenge Live Coding

How would you explain the reasoning of the correct answer to a kid that failed the question?


ARC Challenge Live Coding

Yeah, that is, it's really, it's like you can't.


ARC Challenge Live Coding

Like a lot of times it is just human intuition type of things.


ARC Challenge Live Coding

So here, for example, look at that.


ARC Challenge Live Coding

So there's just like the lower left or the top right thing goes to red and the lower


ARC Challenge Live Coding

left things go to blue.


ARC Challenge Live Coding

Now that contains so many human assumptions.


ARC Challenge Live Coding

Like you know, what, what is a thing?


ARC Challenge Live Coding

What is an object, right?


ARC Challenge Live Coding

We recognize these same color thing as the object.


ARC Challenge Live Coding

And maybe we recognize even the, the black pixels as the background, but to an algorithm,


ARC Challenge Live Coding

there's no objects, there's just pixels and there's no things and there's no inherent


ARC Challenge Live Coding

separation and so on.


ARC Challenge Live Coding

No, come on.


ARC Challenge Live Coding

What, what?


ARC Challenge Live Coding

That is mad.


ARC Challenge Live Coding

What did I do wrong?


ARC Challenge Live Coding

I'm right.


ARC Challenge Live Coding

I don't know.


ARC Challenge Live Coding

See I don't even, I don't even like, I don't know what's wrong.


ARC Challenge Live Coding

I'm pretty sure this is, no, come on.


ARC Challenge Live Coding

Well, we'll have to live with, with failing this one.


ARC Challenge Live Coding

Bigger ones are red.


ARC Challenge Live Coding

Ah, maybe it's the smallest, the smallest one.


ARC Challenge Live Coding

That's or the one that, no, is or the one that, no.


ARC Challenge Live Coding

Is CodePilot good or bad?


ARC Challenge Live Coding

It's very good.


ARC Challenge Live Coding

Smallest is red.


ARC Challenge Live Coding

Okay.


ARC Challenge Live Coding

Let's try, let's try that.


ARC Challenge Live Coding

I'll make this blue and I'll make this red.


ARC Challenge Live Coding

All right.


ARC Challenge Live Coding

Good job.


ARC Challenge Live Coding

Good job, chat Robbie.


ARC Challenge Live Coding

And code the whole thing with clip then test the similar, will not work.


ARC Challenge Live Coding

Will prop, not, not going to say it's not work.


ARC Challenge Live Coding

Will probably not work because clip is trained very much on natural images.


ARC Challenge Live Coding

It like the, the capability of like abstracting to like seeing objects in these types of,


ARC Challenge Live Coding

of things is very hard.


ARC Challenge Live Coding

The challenge seems similar to the measure of intelligence paper.


ARC Challenge Live Coding

Yeah.


ARC Challenge Live Coding

It's actually from that paper.


ARC Challenge Live Coding

The last chapter in that paper describes this challenge.


ARC Challenge Live Coding

So yeah, different object is different code.


ARC Challenge Live Coding

That could have been it too.


ARC Challenge Live Coding

Yes.


ARC Challenge Live Coding

So the issue right now is we have a metric result, right?


ARC Challenge Live Coding

But we'll just get a list.


ARC Challenge Live Coding

So I would like to have, I would like to have, I would like to have like a thing.


ARC Challenge Live Coding

Maybe I'll just return a dict, but it's good to have some sort of a type.


ARC Challenge Live Coding

So we'll say, we'll, we'll say a metrics result, metrics dict is a dict that goes from a string


ARC Challenge Live Coding

to a metric result.


ARC Challenge Live Coding

Right.


ARC Challenge Live Coding

And we'll put that down here.


ARC Challenge Live Coding

Yep.


ARC Challenge Live Coding

I guess that's cool.


ARC Challenge Live Coding

So what we can then do, and we actually don't need the any anymore.


ARC Challenge Live Coding

That's good.


ARC Challenge Live Coding

It's always a good sign.


ARC Challenge Live Coding

So in eval, what we can do instead of just having a list right here, we can have a metrics


ARC Challenge Live Coding

result dict.


ARC Challenge Live Coding

We can instantiate that here.


ARC Challenge Live Coding

And we can not append, but add that here.


ARC Challenge Live Coding

Cool.


ARC Challenge Live Coding

And then this is no longer a list, but this is now a dict.


ARC Challenge Live Coding

That seems fine.


ARC Challenge Live Coding

So now the last thing we'll actually need is we'll need to create a kind of like a unified


ARC Challenge Live Coding

package of all of this.


ARC Challenge Live Coding

So we now have metrics that we call, right.


ARC Challenge Live Coding

And I think what we should do last is some sort of report.


ARC Challenge Live Coding

Right.


ARC Challenge Live Coding

So let's call that reports.


ARC Challenge Live Coding

Yeah, why not?


ARC Challenge Live Coding

Report.


ARC Challenge Live Coding

Okay.


ARC Challenge Live Coding

User.


ARC Challenge Live Coding

Bin.


ARC Challenge Live Coding

Env.


ARC Challenge Live Coding

Python.


ARC Challenge Live Coding

We'll make it Pydantic.


ARC Challenge Live Coding

Report base model.


ARC Challenge Live Coding

And all right.


ARC Challenge Live Coding

So the report is going to be generated by eval.


ARC Challenge Live Coding

So what do we need in a report on this thing?


ARC Challenge Live Coding

Let's make this a bit bigger so you all can see.


ARC Challenge Live Coding

Oopsie.


ARC Challenge Live Coding

Hey, this crashed.


ARC Challenge Live Coding

All right.


ARC Challenge Live Coding

Let's split it.


ARC Challenge Live Coding

So we'll certainly need the eval result list, right?


ARC Challenge Live Coding

Yeah.


ARC Challenge Live Coding

So from interface import eval result list.


ARC Challenge Live Coding

Then we need these metrics results.


ARC Challenge Live Coding

So import metric result dict.


ARC Challenge Live Coding

Metric result dict.


ARC Challenge Live Coding

Yes.


ARC Challenge Live Coding

Okay, then what do we need?


ARC Challenge Live Coding

We may need like what data set was used maybe.


ARC Challenge Live Coding

Maybe who knows?


ARC Challenge Live Coding

Let's consider eval again.


ARC Challenge Live Coding

We may need a date time.


ARC Challenge Live Coding

Can we do that?


ARC Challenge Live Coding

So yeah, we can do that, right?


ARC Challenge Live Coding

Sure why not?


ARC Challenge Live Coding

This top of the screen is cut off.


ARC Challenge Live Coding

Oh, I'm sorry.


ARC Challenge Live Coding

That is how about this or this?


ARC Challenge Live Coding

Yeah, better.


ARC Challenge Live Coding

A bit more.


ARC Challenge Live Coding

Yeah so a date time to check when the report was generated.


ARC Challenge Live Coding

Do we want to call this like metadata or so?


ARC Challenge Live Coding

Maybe not.


ARC Challenge Live Coding

Import date time.


ARC Challenge Live Coding

So how about this?


ARC Challenge Live Coding

And with a default factory.


ARC Challenge Live Coding

Okay, now it's trying to get it right.


ARC Challenge Live Coding

I think this is okay.


ARC Challenge Live Coding

Cool.


ARC Challenge Live Coding

Yep.


ARC Challenge Live Coding

Um, we can give it.


ARC Challenge Live Coding

I don't know.


ARC Challenge Live Coding

Tags.


ARC Challenge Live Coding

How about?


ARC Challenge Live Coding

Oh yeah, the data set.


ARC Challenge Live Coding

Maybe that is also a good thing.


ARC Challenge Live Coding

So I have a data set utils class right here and these sub tiers.


ARC Challenge Live Coding

Let's say sub tiers list of string and we'll also, we'll also


ARC Challenge Live Coding

get maybe comment string, something like this, right?


ARC Challenge Live Coding

Something that packages all of these things together and then, and then saves it down.


ARC Challenge Live Coding

So these are all either pydanetic models or standard models.


ARC Challenge Live Coding

So we are going to, and this one doesn't even have to be here.


ARC Challenge Live Coding

All we need is the eval results really.


ARC Challenge Live Coding

And now we have, we can have various methods to actually render this or yeah, so we can


ARC Challenge Live Coding

have various representations of that.


ARC Challenge Live Coding

However, I'm not sure if I want to implement str right here.


ARC Challenge Live Coding

Would augmenting, tag the screen with stream with ASMR.


ARC Challenge Live Coding

Okay.


ARC Challenge Live Coding

Would augmenting example with natural descriptions help?


ARC Challenge Live Coding

Maybe I don't know.


ARC Challenge Live Coding

Could you create a program to procedurally generate similar puzzles with a text label


ARC Challenge Live Coding

to train on like an, an L or an I system, then learn to generate the text from the samples


ARC Challenge Live Coding

results from the text?


ARC Challenge Live Coding

Yeah, probably.


ARC Challenge Live Coding

I mean, if you can come up with text descriptions, but again, there's 400 test examples, which


ARC Challenge Live Coding

is not too many.


ARC Challenge Live Coding

So a lot of deep learning type of things are out of the question, at least without help,


ARC Challenge Live Coding

but generating data is one of the things, one of the routes we are considering.


ARC Challenge Live Coding

Yes.


ARC Challenge Live Coding

But the question is how, because you know, you need to build in these human priders like


ARC Challenge Live Coding

gravity and objectness and so on.


ARC Challenge Live Coding

So I don't want to, let's save to file.


ARC Challenge Live Coding

File name.


ARC Challenge Live Coding

No, no, no.


ARC Challenge Live Coding

We'll go.


ARC Challenge Live Coding

OS, so all of this from path lib import path.


ARC Challenge Live Coding

We'll make this a path like.


ARC Challenge Live Coding

Very good.


ARC Challenge Live Coding

Thanks Copilot.


ARC Challenge Live Coding

So this saves the report to file.


ARC Challenge Live Coding

Now the eval here.


ARC Challenge Live Coding

It currently evaluates agents on riddle and on riddles, it applies metrics.


ARC Challenge Live Coding

Now we need a function that, well, it's a good question.


ARC Challenge Live Coding

What we now need is a function, how can I improve my coding skills?


ARC Challenge Live Coding

Have you heard of bit tensor?


ARC Challenge Live Coding

I have heard of it.


ARC Challenge Live Coding

My current opinion is that is buzzwords, but I, I've not looked into it very clearly.


ARC Challenge Live Coding

I hear there is an interview on towards data science, but I've not listened to it.


ARC Challenge Live Coding

How can I improve my coding skills?


ARC Challenge Live Coding

Just code and look at other people's code and submit pull requests to stuff and take


ARC Challenge Live Coding

the feedback with a grain of salt, but take it seriously.


ARC Challenge Live Coding

Or join a company that has some good standards or something, or look at coding guidelines


ARC Challenge Live Coding

from big companies cause you know, they for, for, for some reason they have survived and


ARC Challenge Live Coding

that's a challenge.


ARC Challenge Live Coding

If you code with a lot of people, so they need good standards.


ARC Challenge Live Coding

All right, so we now need, we now need a last function that does everything, you know, maybe


ARC Challenge Live Coding

at once.


ARC Challenge Live Coding

I'm not sure that loads kind of loads.


ARC Challenge Live Coding

So if we load the data set, we say evaluate, we say apply metrics, and then we ship this


ARC Challenge Live Coding

all in, we, we, we shove it all into report.


ARC Challenge Live Coding

That should technically, you know, do all the things, but we could do it in the CLI.


ARC Challenge Live Coding

Hmm.


ARC Challenge Live Coding

Yes, probably we should do it in that CLI.


ARC Challenge Live Coding

So there's a CLI by the way, I'm, I'm really proud of the CLI.


ARC Challenge Live Coding

So you can do something like, so if you type arc, that's the CLI if you've installed the


ARC Challenge Live Coding

thing, so you can do help.


ARC Challenge Live Coding

You can do arc show random ID.


ARC Challenge Live Coding

It will show you one of the tasks.


ARC Challenge Live Coding

That's a bit of a big task, but maybe I zoom out to see, it will show you like a random


ARC Challenge Live Coding

ID puzzle, which is pretty cool.


ARC Challenge Live Coding

And then you can try to solve it.


ARC Challenge Live Coding

This is all in console.


ARC Challenge Live Coding

That's why it's especially cool.


ARC Challenge Live Coding

So yeah, we need one thing that evaluates an agent on a dataset, and that's going to


ARC Challenge Live Coding

be the kind of last thing.


ARC Challenge Live Coding

Now the question is, can I load an agent from, can I load an agent from this?


ARC Challenge Live Coding

So eval, can we import the dataset thing from, from eval?


ARC Challenge Live Coding

Because probably, probably.


ARC Challenge Live Coding

Hmm, CLIP happens to know physics.


ARC Challenge Live Coding

It also fears its own mortality.


ARC Challenge Live Coding

No, not CLIP.


ARC Challenge Live Coding

But yeah, that was, that was a fun, certainly a fun, fun thing.


ARC Challenge Live Coding

I'm just trying to think how we best do this.


ARC Challenge Live Coding

How about we make an example?


ARC Challenge Live Coding

So let's make, not here, let's make examples.


ARC Challenge Live Coding

Like that, out here.


ARC Challenge Live Coding

Actually let's move that into a directory called examples.


ARC Challenge Live Coding

So we'll take that cheating agent.


ARC Challenge Live Coding

Maybe I should, maybe I should move this to this, like base.


ARC Challenge Live Coding

Like that seems pretty good.


ARC Challenge Live Coding

And then we can do an init file, right?


ARC Challenge Live Coding

And just import the agent from, import agent, we'll all it out.


ARC Challenge Live Coding

Good.


ARC Challenge Live Coding

All right, and then we can have the cheating agent inside of this one.


ARC Challenge Live Coding

So let's go to the eval test, rip out the cheating agent like this, put it in here from


ARC Challenge Live Coding

agent, import task data, board hints, board pair, board, top k list.


ARC Challenge Live Coding

I think we have it now.


ARC Challenge Live Coding

Board hints isn't exported.


ARC Challenge Live Coding

Oh, that's from hints maybe.


ARC Challenge Live Coding

I guess we need to export all of these from Arc anyway.


ARC Challenge Live Coding

Challenge seems to be solvable with only low level priors, but high level priors, a sequence


ARC Challenge Live Coding

of translation with reasoning is made up of more primitive priors.


ARC Challenge Live Coding

Yes, but also the com, like there's not an arbitrary combination.


ARC Challenge Live Coding

That's also one of the challenging things.


ARC Challenge Live Coding

So I don't really know how to, yeah.


ARC Challenge Live Coding

So we should be able to import all of these from Arc, I think.


ARC Challenge Live Coding

Like this.


ARC Challenge Live Coding

So let's make that happen here.


ARC Challenge Live Coding

Yeah, from import agent, task data, we'll also need from import board hints, hints certainly


ARC Challenge Live Coding

will also need the top k list right here.


ARC Challenge Live Coding

I hope that's enough.


ARC Challenge Live Coding

Good.


ARC Challenge Live Coding

All right.


ARC Challenge Live Coding

So this cheating agent, as I said, is able to just look at from Arc import star.


ARC Challenge Live Coding

No, no, no, no, no, no, no bad.


ARC Challenge Live Coding

Didn't someone ask how can I improve my coding?


ARC Challenge Live Coding

Not like not with star imports.


ARC Challenge Live Coding

Although sometimes it's just quick.


ARC Challenge Live Coding

I think, yeah, especially if you want to do something quick.


ARC Challenge Live Coding

So the cheating agent will import that then in the eval test right here because we can,


ARC Challenge Live Coding

we don't need any of these anymore except task data from Arc.


ARC Challenge Live Coding

Yep.


ARC Challenge Live Coding

I need to restart.


ARC Challenge Live Coding

Whenever I make a new file, I need to restart.


ARC Challenge Live Coding

Like that's dumb.


ARC Challenge Live Coding

Something's wrong with my setup clearly.


ARC Challenge Live Coding

So does this still run?


ARC Challenge Live Coding

See how slow this is.


ARC Challenge Live Coding

It's terribly slow.


ARC Challenge Live Coding

Has no attribute aggregation results.


ARC Challenge Live Coding

Yeah.


ARC Challenge Live Coding

So now is the part where we have to actually change the test.


ARC Challenge Live Coding

So here is the metrics.


ARC Challenge Live Coding

Here we have the eval results.


ARC Challenge Live Coding

I would call that result list.


ARC Challenge Live Coding

Apply the metrics.


ARC Challenge Live Coding

Yes.


ARC Challenge Live Coding

That will give us a metric result dict.


ARC Challenge Live Coding

Yep.


ARC Challenge Live Coding

And how about that?


ARC Challenge Live Coding

Garbage collector.


ARC Challenge Live Coding

Yeah, maybe.


ARC Challenge Live Coding

Ah, okay.


ARC Challenge Live Coding

I need to aggregate result.


ARC Challenge Live Coding

Yes, hopefully.


ARC Challenge Live Coding

Good.


ARC Challenge Live Coding

Test passes.


ARC Challenge Live Coding

Wow.


ARC Challenge Live Coding

Let's look at it.


ARC Challenge Live Coding

Okay.


ARC Challenge Live Coding

Commits.


ARC Challenge Live Coding

Metrics.


ARC Challenge Live Coding

Okay, now we need like a last function that just generates the report from the riddles


ARC Challenge Live Coding

and then maybe one that also loads the data set.


ARC Challenge Live Coding

Sorry.


ARC Challenge Live Coding

And then we're good.


ARC Challenge Live Coding

Again.


ARC Challenge Live Coding

Any interface.


ARC Challenge Live Coding

Yeah, I see.


ARC Challenge Live Coding

Go, go, go, go, go, go.


ARC Challenge Live Coding

Excellent.


ARC Challenge Live Coding

Okay, ship it.


ARC Challenge Live Coding

And yeah, we'll go on.


ARC Challenge Live Coding

So let's go to eval.


ARC Challenge Live Coding

Let's


ARC Challenge Live Coding

create agent on riddles.


ARC Challenge Live Coding

No.


ARC Challenge Live Coding

How do we call this?


ARC Challenge Live Coding

I want to take something that takes riddles, metrics, and generates a report.


ARC Challenge Live Coding

Essentially, it's just going to call the things above.


ARC Challenge Live Coding

They'd be evaluate agent on riddles and apply metrics and generate report.


ARC Challenge Live Coding

Is hard.


ARC Challenge Live Coding

It's hard.


ARC Challenge Live Coding

Report from riddles.


ARC Challenge Live Coding

That's not too bad.


ARC Challenge Live Coding

All right.


ARC Challenge Live Coding

Let's do this.


ARC Challenge Live Coding

Evaluate R3.


ARC Challenge Live Coding

And metrics.


ARC Challenge Live Coding

So essentially, we'll do the exact same thing as we did before.


ARC Challenge Live Coding

No, not evaluate on riddles.


ARC Challenge Live Coding

Yes.


ARC Challenge Live Coding

Not metrics results.


ARC Challenge Live Coding

I want a report.


ARC Challenge Live Coding

Apply metrics.


ARC Challenge Live Coding

Report.


ARC Challenge Live Coding

Does it have?


ARC Challenge Live Coding

How about instead, because we'll just call that eval results.


ARC Challenge Live Coding

Metric results.


ARC Challenge Live Coding

That should be good.


ARC Challenge Live Coding

All right, return report.


ARC Challenge Live Coding

No, results.


ARC Challenge Live Coding

Evaluate and report.


ARC Challenge Live Coding

That is a good name.


ARC Challenge Live Coding

Very good name.