Building a Quadcopter p.2 - Parts

So and it's the worst kind because it's got a like a relatively weak


Building a Quadcopter p.2 - Parts

Conductivity and so basically the short circuit won't happen immediately you'll plug in the battery


Building a Quadcopter p.2 - Parts

You'll go fly it outside and anywhere from like one to five minutes later your quad is smoking if you don't catch it quick enough


Building a Quadcopter p.2 - Parts

One it's a fire hazard two you'll fry everything and you'll waste like $80


Building a Quadcopter p.2 - Parts

So keep that in mind, so don't don't be like me


Building a Quadcopter p.2 - Parts

Respect the carbon fiber, but anyway, you'll solder basically all the connections here and all that


Building a Quadcopter p.2 - Parts

So it's kind of a pain if you have a basic board like this, but again this board is really cheap


Building a Quadcopter p.2 - Parts

This one's a little more expensive. It has a nice foam padding on the bottom side so for idiots like me


Building a Quadcopter p.2 - Parts

Who think you can just put it right on the carbon fiber?


Building a Quadcopter p.2 - Parts

It's it's basically protected like that


Building a Quadcopter p.2 - Parts

Also foam padding reduces vibration, but it's got these nice connectors everything's already soldered


Building a Quadcopter p.2 - Parts

So it saves you some time


Building a Quadcopter p.2 - Parts

Also, it comes with this XT 60 adapter, so we're going to talk about that now for just a moment so first of all


Building a Quadcopter p.2 - Parts

Batteries for example like this is a venom 2100 11.1 volt battery


Building a Quadcopter p.2 - Parts

But it's a little older, and it uses what's called a dean's plug now this one's you can kind of probably see it


Building a Quadcopter p.2 - Parts

I don't know, but yeah, so it's a dean's plug and as you can see that does not plug into that


Building a Quadcopter p.2 - Parts

Like it just doesn't work, so you want to pay attention to what kind of plugs you're using now


Building a Quadcopter p.2 - Parts

They do make converters, so I have three of these I'm not about to update the battery


Building a Quadcopter p.2 - Parts

But they do make Dean's to XT 60 converters, so you can tell it's an XT 60 by the way that it is


Building a Quadcopter p.2 - Parts

Anyway, it's the shape right the shape and the color and all that you can tell it also says XT 60 on here


Building a Quadcopter p.2 - Parts

But I'm not sure if you'll be able to see that or not, but anyway


Building a Quadcopter p.2 - Parts

That's kind of this is like the new pretty much everybody's using this adapter


Building a Quadcopter p.2 - Parts

But even if you find a dean's adapter they make Dean's to XT 60


Building a Quadcopter p.2 - Parts

Converters, so it's totally fine doesn't really matter


Building a Quadcopter p.2 - Parts

so


Building a Quadcopter p.2 - Parts

That is basically all all the basic things that you're going to


Building a Quadcopter p.2 - Parts

To need you've got the frame the motors ESC's propellers and go ahead and buy like lots of propellers


Building a Quadcopter p.2 - Parts

so you know propellers are really really cheap


Building a Quadcopter p.2 - Parts

Just just buy them and have them around because they'll probably like the first like ten times you fly


Building a Quadcopter p.2 - Parts

You'll break propellers just the way that it is so


Building a Quadcopter p.2 - Parts

So frame motors ESC propellers get lots of them a low voltage warning if it doesn't come with one like this one this board


Building a Quadcopter p.2 - Parts

I'm like 99% certain that that's what this is I'll find out, but I'm pretty sure this comes with one


Building a Quadcopter p.2 - Parts

So just keep that in mind


Building a Quadcopter p.2 - Parts

You might actually have a control board that that literally comes with one, but if you don't have one get one


Building a Quadcopter p.2 - Parts

You need a transmitter and a receiver. This is my receiver, and then this is my transmitter


Building a Quadcopter p.2 - Parts

You can buy separate receivers, so I actually have an extra receiver that I'll be using on this new quad


Building a Quadcopter p.2 - Parts

So transmitter receiver you need and most transmitters come with a receiver


Building a Quadcopter p.2 - Parts

I'm just my point was that you can you can usually buy a transmitter that comes with a receiver


Building a Quadcopter p.2 - Parts

But if you want another receiver you can buy another one


Building a Quadcopter p.2 - Parts

You need some way to charge balance and discharge batteries. That's this tenor GTB 6b


Building a Quadcopter p.2 - Parts

Other than that small things that you might need or you might have around the house or whatever you're going to need a


Building a Quadcopter p.2 - Parts

You're gonna need a soldering iron okay, so cheap out soldering iron will do this one's actually a


Building a Quadcopter p.2 - Parts

Radio shack solder hands this is like vintage man. This is serious. This is like a


Building a Quadcopter p.2 - Parts

Man, this is serious. This is highly valuable


Building a Quadcopter p.2 - Parts

So soldering iron a way to clean the soldering iron


Building a Quadcopter p.2 - Parts

Helping hands for the soldering iron so if you've ever soldered something you realize you want like 30 hands


Building a Quadcopter p.2 - Parts

This will at least get you to four hands, so that's nice


Building a Quadcopter p.2 - Parts

So you want some helping hands there?


Building a Quadcopter p.2 - Parts

You'll obviously need some solder what you're looking for is Rosencour solder. I don't know too much about soldering myself


Building a Quadcopter p.2 - Parts

I just know what the internet tells me to get and that's Rosencour solder


Building a Quadcopter p.2 - Parts

the


Building a Quadcopter p.2 - Parts

Helping hands iron cleaner you'll want to have like foam so like I said like this has like some foam on the bottom you


Building a Quadcopter p.2 - Parts

Can usually just buy like strips of foam and then you cut the foam and you put that under things so like the board


Building a Quadcopter p.2 - Parts

You might want to have some foam this actually has like these little rubber stopper things that are balancing it and that works


Building a Quadcopter p.2 - Parts

Just fine. I've never really had a huge issue with


Building a Quadcopter p.2 - Parts

With using or with vibration in general


Building a Quadcopter p.2 - Parts

So foam


Building a Quadcopter p.2 - Parts

zip ties


Building a Quadcopter p.2 - Parts

Just have a lot of zip ties around as you can see like this has zip ties on it


Building a Quadcopter p.2 - Parts

There's zip ties all over here. There's just some ties everywhere the ties are great get them


Building a Quadcopter p.2 - Parts

electrical tape


Building a Quadcopter p.2 - Parts

There's electrical tape everywhere here like everything is electrical taped


Building a Quadcopter p.2 - Parts

I called it this actually I call ET and it's not because it's not in reference to an alien. It's electrical tape


Building a Quadcopter p.2 - Parts

and then


Building a Quadcopter p.2 - Parts

Velcro strips so double-sided or not double-sided Velcro like Velcro


Building a Quadcopter p.2 - Parts

That's like sticky on one end and then Velcro on the other the reason you have that is because this you put some Velcro here


Building a Quadcopter p.2 - Parts

You put Velcro here, and then you have these little like Velcro straps


Building a Quadcopter p.2 - Parts

Separate it so you've got Velcro straps like that and then to connect the battery you literally just put the battery on there


Building a Quadcopter p.2 - Parts

Strap it connect this to the this is my low voltage reader


Building a Quadcopter p.2 - Parts

Connect that connect this to the actual quad and start your engines


Building a Quadcopter p.2 - Parts

And then when you want to switch it out you just literally pull these apart and switch out the batteries


Building a Quadcopter p.2 - Parts

It's very fast, so I like having it but again if you only have one battery then whatever just you know


Building a Quadcopter p.2 - Parts

Electrical tape the battery there who cares so anyway


Building a Quadcopter p.2 - Parts

There's that so


Building a Quadcopter p.2 - Parts

and then the converters like I said you might want to have some of those and


Building a Quadcopter p.2 - Parts

And I think bullet adapters these are super useful so especially if you have


Building a Quadcopter p.2 - Parts

like this is my power distribution board you take these bullet adapters and you


Building a Quadcopter p.2 - Parts

Basically to the wire you put the male the male end connects to one wire the female end of the other wire and then to connect


Building a Quadcopter p.2 - Parts

Those you just use these little bullet adapters connect them and you're good to go


Building a Quadcopter p.2 - Parts

It's like really quick especially if you need to flip them around when we go to connect the esc's and the motors


Building a Quadcopter p.2 - Parts

You'll see what I mean when you connect in the first time chances are you might have to switch


Building a Quadcopter p.2 - Parts

Positive and negative on a couple of them to make them spin in the other direction


Building a Quadcopter p.2 - Parts

So because some other spin clockwise some others been counterclockwise


Building a Quadcopter p.2 - Parts

Finally before I let you guys go one more thing that you can make the mistake of is on


Building a Quadcopter p.2 - Parts

The the motors and the shaft and the propellers so the way you've got quick


Building a Quadcopter p.2 - Parts

Attach and detach propellers like these are very quick to take on and off


Building a Quadcopter p.2 - Parts

And so for me like the they're literally like threaded on here


Building a Quadcopter p.2 - Parts

and so I just you know detach them in the way that these stay on is


Building a Quadcopter p.2 - Parts

Literally to tighten this you spin the propeller this way


Building a Quadcopter p.2 - Parts

To tighten it right so as it's let's get tight here


Building a Quadcopter p.2 - Parts

So so so it spins this way to get tight so to apply torque that way the motor spins this way


Building a Quadcopter p.2 - Parts

Right so the motor spins this way so as the moat when the motor is on


Building a Quadcopter p.2 - Parts

It's literally always applying a little bit of torque to the propeller so that holds the propeller in place


Building a Quadcopter p.2 - Parts

So we actually don't need like a propeller adapter to sit on top and clamp it on a place


Building a Quadcopter p.2 - Parts

But some motors basically the propeller is totally through and through I'm having trouble finding my


Building a Quadcopter p.2 - Parts

Let's see let's open up the kit so for example when you buy the quadcopter


Building a Quadcopter p.2 - Parts

You can either buy a kit you can buy everything separately you can buy just a frame and put everything together


Building a Quadcopter p.2 - Parts

I recommend getting a kit as best you can


Building a Quadcopter p.2 - Parts

So here's an example of a kit. Let me move this over


Building a Quadcopter p.2 - Parts

Here so here this kit for example came with ESC so these are ESC's motors


Building a Quadcopter p.2 - Parts

motors


Building a Quadcopter p.2 - Parts

ESC motor and then propellers if you can tell these are first of all these are carbon fiber propellers if this is your first quad


Building a Quadcopter p.2 - Parts

Don't use these these are like nice propellers


Building a Quadcopter p.2 - Parts

Use something cheap for a while until you're not crashing anymore, then think about using these but anyway these you can see


Building a Quadcopter p.2 - Parts

They're totally like they're they don't screw on they they don't


Building a Quadcopter p.2 - Parts

Screw on they like just sit on and then you use a propeller adapter like this


Building a Quadcopter p.2 - Parts

Which will screw on top of that propeller


Building a Quadcopter p.2 - Parts

And hold it in place so depending on what kind of motor you have that might be a little different


Building a Quadcopter p.2 - Parts

So just keep that in mind


Building a Quadcopter p.2 - Parts

Anyway, I'll put this all back


Building a Quadcopter p.2 - Parts

So that's that anyway this is a kit this came with basically the frame the motors and the ESC's


Building a Quadcopter p.2 - Parts

And that's it doesn't have a control board as far as I can recall


Building a Quadcopter p.2 - Parts

I don't see it in there anyway some of them will come like with everything you need so really just kind of look around


Building a Quadcopter p.2 - Parts

And make sure because the only thing that you really need to pay attention to is you want to make sure the ESC's and the


Building a Quadcopter p.2 - Parts

Motors are together. That's probably the only thing I would always buy together is motors and ESC's so you can make sure they're like


Building a Quadcopter p.2 - Parts

Compatible with each other and push enough power basically


Building a Quadcopter p.2 - Parts

so


Building a Quadcopter p.2 - Parts

um, I'm pretty sure those are all the requirements and all the like stuff that you're almost certain to need


Building a Quadcopter p.2 - Parts

I might have forgotten something so check the list to make sure now finally


Building a Quadcopter p.2 - Parts

I'm going to talk about some of the optional stuff so that stuff is basically how you could build a quadcopter and fly it around


Building a Quadcopter p.2 - Parts

Um, I don't know what I did. What did I do with the quad? Oh here it is


Building a Quadcopter p.2 - Parts

I couldn't figure out where I put it so that'll get you to this point


Building a Quadcopter p.2 - Parts

Now I'm going to talk about


Building a Quadcopter p.2 - Parts

Uh what we're going to do after that so the next step is quite literally to we want to add a raspberry pi


Building a Quadcopter p.2 - Parts

to our quadcopter


Building a Quadcopter p.2 - Parts

So this is a raspberry pi


Building a Quadcopter p.2 - Parts


Building a Quadcopter p.2 - Parts

now, um


Building a Quadcopter p.2 - Parts

Basically if you're just looking at this you can see like where is the raspberry pi going?


Building a Quadcopter p.2 - Parts

So maybe we could fit a raspberry pi in here, right cool, but


Building a Quadcopter p.2 - Parts

Not only do I want a raspberry pi, but I actually want to add one more thing


Building a Quadcopter p.2 - Parts

And that's going to be the brick pi. So the brick pi just sits on top of the raspberry pi


Building a Quadcopter p.2 - Parts

Possibly, you know connects into the gpio pins there, right?


Building a Quadcopter p.2 - Parts

So that sits on top and so we can still probably stuff that in there, but it's going to be a real tight fit


Building a Quadcopter p.2 - Parts

So actually what i'm going to be doing


Building a Quadcopter p.2 - Parts

Is what's here but covered in a bunch of electrical tape is the power distribution board?


Building a Quadcopter p.2 - Parts

And then these what's stuffed over here are the esc's


Building a Quadcopter p.2 - Parts

If we could move the power distribution board in the esc's this whole space in here would be totally open


Building a Quadcopter p.2 - Parts

To put anything we wanted so ie raspberry pi and a brick pipe and some other stuff even


Building a Quadcopter p.2 - Parts

Because the battery goes on the bottom a lot of people actually have this pretty clean and they put the battery in here


Building a Quadcopter p.2 - Parts

And I will admit if you can put the battery in here because the battery is like the heaviest part


Building a Quadcopter p.2 - Parts

Your balancing is way better. So I may end up putting the battery there, too


Building a Quadcopter p.2 - Parts

But the main crux of the point here is that you want we want to open up the space a little bit more


Building a Quadcopter p.2 - Parts

And the way i'm going to do that is this is another frame, but it's just an I just to kind of show you is


Building a Quadcopter p.2 - Parts

There's this is missing the top part. But in the this is the bottom part and it usually clamps down onto these arms


Building a Quadcopter p.2 - Parts

So what I did was or what i'm planning to do is I bought a bunch of these spacers


Building a Quadcopter p.2 - Parts

Okay, and i'm going to add these spacers because you might say oh you've done it already


Building a Quadcopter p.2 - Parts

But no, I actually haven't because these aren't actually very sturdy because they have spacers here


Building a Quadcopter p.2 - Parts

But this could slide up and down this these arms. So I bought some spacers the power distribution board


Building a Quadcopter p.2 - Parts

Is actually going to sit in between


Building a Quadcopter p.2 - Parts

These two boards the esc's will go on the bottom to save a bunch of space


Building a Quadcopter p.2 - Parts

The raspberry pi goes in here the battery pack goes in here brick pi in here


Building a Quadcopter p.2 - Parts

Or the battery pack might end up going down here. We'll see it. Um along those lines. I know i've mentioned


Building a Quadcopter p.2 - Parts

You know the battery pack and flaming and all that but uh this the most frames come with like


Building a Quadcopter p.2 - Parts

Landing gear that actually aren't these tubes or these are spacers basically


Building a Quadcopter p.2 - Parts

But there it's a landing gear. That's actually just like this little I don't know veeth looking thing


Building a Quadcopter p.2 - Parts

That was not tall enough to protect my battery when I landed. So I actually added these little spacers


Building a Quadcopter p.2 - Parts

Which are basically identical to these um, so keep that in mind. I added those right?


Building a Quadcopter p.2 - Parts

So that's kind of why you want to have a little bit of extra parts laying around


Building a Quadcopter p.2 - Parts

And yeah, so anyway getting the basic quad built relatively similar from there customizing it's a lot


Building a Quadcopter p.2 - Parts

A lot more complex, but those are extra things that you might want to have if you're planning to customize. So anyways


Building a Quadcopter p.2 - Parts

Long list of stuff a lot of topics a lot of stuff to think about and all that like I said the best thing I would


Building a Quadcopter p.2 - Parts

I can suggest is try to find a kit that has almost everything that you need


Building a Quadcopter p.2 - Parts

In the kit i'll do my best to link to some kits for examples


Building a Quadcopter p.2 - Parts

But prices are changing so fast that the stuff that I link to might not always be the best price or it might be


Building a Quadcopter p.2 - Parts

Outrageous or might even be available or whatever. So i'll do my best to help you guys out


Building a Quadcopter p.2 - Parts

And uh, that's that so if you have questions comments concerns, whatever leave them in the leave them below


Building a Quadcopter p.2 - Parts

Otherwise stay tuned the next video. We'll start actually putting a quad together using the kit that I have


Building a Quadcopter p.2 - Parts

We'll start soldering some stuff and all that


Building a Quadcopter p.2 - Parts

Just for the record if you if you didn't get one of the like if you if you have like a board like this that


Building a Quadcopter p.2 - Parts

Requires soldering like this one doesn't require the same soldering basically


Building a Quadcopter p.2 - Parts

I do have at least show a clip or something like that. So if you're interested in that


Building a Quadcopter p.2 - Parts

I do have a video of me soldering the same soldering basically


Building a Quadcopter p.2 - Parts

I do have at least show a clip or something of me soldering one of these boards as well


Building a Quadcopter p.2 - Parts

Just so you can see like both different, you know both boards


Building a Quadcopter p.2 - Parts

Also the setting up of a of a control board like this is different than the setup of a control board like this


Building a Quadcopter p.2 - Parts

So again, we can talk about the differences there. This one has like a nice lcd screen even stuff


Building a Quadcopter p.2 - Parts

This one's really simple to set up too


Building a Quadcopter p.2 - Parts

I actually really like this one. I just wanted to be able to have it turned and this one's just gonna


Building a Quadcopter p.2 - Parts

This one will be compact easier. I can stuff this one in various other locations


Building a Quadcopter p.2 - Parts


TransCoder: Unsupervised Translation of Programming Languages (Paper Explained)

Hi there! So the paper we're looking at today can take the code on the left,


which is written in Python, and can output the code on the right, which is


written in C++. Now the point here is that the code on the right does the same


thing as the code on the left, so it is implementing the same function. The


surprising thing here is that this model that takes the Python as an input has


never been explicitly trained to output C++, so this is an unsupervised


translation model. And that's the the cool thing about this paper is that by


having no target at having no supervised signal at translating source code


languages into one another, it can perform pretty well at the task


nonetheless. So we're going to look at this paper, it's called unsupervised


translation of programming languages by Marianne Lachaud, Baptiste Rozier, Loïc


Chanussault and Guillaume Lempel at Facebook AI Research. As always, if you like


content like this, consider sharing it out and leaving a like, and also leaving


a comment if you have something to say about it. They say a trans compiler, also


known as a source-to-source translator, is a system that converts source code


from a high-level programming language such as C++ or Python to another. They


say trans compilers are primarily used for interoperability and to port code


bases written in an obsolete or deprecated language such as COBOL or


Python 2 to a modern one. So for Python 2 you might know this tool that's called


2to3. So 2to3 is a tool that ships with the Python 3 standard library, I


believe, that allows you to take Python 2 code and produce Python 3 code. And that


is to kind of push people to convert their old code bases of Python 2 to


put to the modern Python 3. Now 2to3 is handwritten program. It has


specific rules built in that the programmers know if we modify Python 2


like this, Python 3 gets out. For example, the print statement in Python 2 requires


no brackets, so we make a rule that whenever there's a print statement with


no brackets, we'll add the brackets such that it's Python 3 compliant. Most of


this code will transfer, transform the source code first into an abstract


syntax tree, modify that, apply specific rules to that, and then output the


language from the abstract syntax tree. Now the problem here is manyfold. First


of all, there can only be so much translation as there are rules. So every


one of these rules has to be coded as a modification to the abstract syntax tree,


and every one of these rules is handcrafted and therefore needs sort of


human ingenuity. Humans need to go and write these rules of how to transform


one language into another. And oftentimes this doesn't, even though it, if you can


write these rules, oftentimes whatever comes out is sort of a bit of a cryptic


source code because you kind of have to make sure that your rules cover all the


possible things and the source code that comes out is oftentimes very cryptic and


a bit hard to understand because it's been sort of expanded and formalized to


make sure that it still does the same thing as the original source code. Now


for Python 2 to Python 3, this is still easy, right? These languages are


extremely similar because it's not that big of a step to Python 3 except if you


use very low-level language language constructs or language features which


have been obsoleted with Python 3. On the other hand, if there's something like


COBOL, so a lot of this old banking code or insurance code or government agency


code whatnot is written in these really old programming languages and they've


been kept alive by these old-school programmers that are slowly but surely


all retiring now and there are just not many new programmers around that can


support these languages and the languages themselves aren't really


updated that much anymore so you would like to transform COBOL into something


like, I don't want to call Java modern programming language but it is used in


modern times. I'd rather not call it modern per se. Java itself is a beast


that's been sort of supported since forever but in any case you would like


to transform something like COBOL to something like Java where you have a


lot of programmers that can develop and further develop your code. This is much


harder. COBOL and Java are much more away from each other than are Python 2 and


Python 3. So what you would like to do is you would like to have a tool that is


like 2 to 3 but humans, because now if you want a tool like this you need


someone that's really proficient in COBOL and Java in order to write a tool


like this and you need lots of them and they need to invest lots of time. What


you would rather like to do is you would like to learn a system that translates


from one language into another such that the meaning is conserved and this of


course is exactly the domain of natural language machine translation except its


source code. So we all know that we've all realized in the last years that


things like Google Translate have become extremely good at translating. So they


say right here although neural models significantly outperform their rule


based counterparts in the context of natural language translation which is


Google Translate is all learned right now. They say their applications to


trans compilation have been limited due to the scarcity of parallel data in this


domain. So what's the problem with just going and saying oh you know we can


build really good neural machine translation models let's just apply them


to source code. The problem is if you build a neural machine translation model


say something that transforms English to German so you have the word hello and


you output hello. You can do it not with just one word but with entire sentences


and so on. These models they usually or in the classical sense what they need


are parallel corpora which means that you have this you have documents that


are written in many languages and you can guarantee that they mean the same


thing. So this is a supervised signal. One example of this is let's say press


releases of the United Nations. So the United Nations will make some press


release and they will then have professional translators translate that


press release into all of the different or into many different languages and so


you can pretty much guarantee that these mean the same thing. So these pairs of


documents or triplets or whatnot they are supervised training data for a


machine translation model that translates from one language into the


other and the neural machine translation models rely heavily on these parallel


corpora. For source code you just don't have that as much. You don't have big


code bases in great numbers where the exact same thing is implemented in one


language and in the other language. There's just not that much data


available. It is the case that sometimes sorry that sometimes let's say in the


case of Torch it started as Lua and then it went to PyTorch and the developers


had to translate the code from Torch to Python but in the same in the same step


they've also made improvements they sort of re-engineered and reinvented the


framework and made it better and so you can't really say these are the same


things and likewise there's not a lot of code available where the same thing is


implemented in two languages. So we just don't have these parallel corpora for


for the source code translation. So rather what this paper does is this


paper goes into unsupervised machine translation. Now what does unsupervised


machine translation mean? Unsupervised machine translation you imagine I have


just a big database of documents and these documents I know they're all


they're all in English okay and I have this other big database and I know that


they are all in German I just know their documents are in German but they don't


correspond to each other they're just German documents and over here they're


just English documents they don't I don't say that these two here are


somehow the same no I just have a bunch of German a bunch of English they don't


even have to correspond they're just text and now what I want to do is I want


to learn a shared embedding space I sort of want to learn a shared space of


embeddings for these two languages such that similar things are mapped to the


similar place so if these two documents just happen to talk about the same thing


I want them to be mapped to similar spaces in this shared embedding space so


I'm gonna have one model a single model where I input the text and it goes in


to this shared embedding space okay now this is unusual because usually in


machine translation if you translate from here from English to German then


you'll have your dedicated model that takes as English as an input and German


as an output and that would be a different model than one that takes even


German as an input in English as an output or French as an input in German


as an output in this case we have this process right here and this process


right here is the same model and then the decoder that translates this of


course now we have the encoder embedding and then the decoder that actually


translates to a language is also going to be the same model so it's the same


model that translates to English then it translates to German so first of all how


do we make the same model let's say let's say we have the perfect encoder


right this is the encoder the same encoder for all languages let's say we


have the perfect encoder and it can map the if whenever a sentence means the


same thing in different languages we can completely map it to the same point in


embedding space irrespective of the language it comes from now how do we


tell the decoder which is also the same model like how does it know what to do


this is a little trick where you basically you take this embedding so you


take the input you put it into your model I don't even know how you know


this and then your output is going to be autoregressive right so you decode one


token at a time so you decode this token and then you feed it back into the model


and then you decode this token you feed that back into the model and so on this


is an autoregressive language model and the trick here is that the first the


very first token is a special token that describes the language you want to


output so here you say I want German and then you let the model decode its its


thing and by conditioning on this token right here it knows it should now


produce German you will simply during training you will simply put the token


here and if it produces something other than German that's a loss right so it


will learn to produce German after you produce this tag alright so what we need


is an encoder and a decoder such that in the encoder we can put in any language


text and it will map to the same space the things that mean the same things and


we need a decoder that you know can produce any language given this first


thing now the decoder should be fairly easy right if we have a shared


vocabulary between the languages and we we always put this token we can the


decoder is not a problem you can just learn the decoder in a straightforward


way but the encoder is going to be a problem so how does the encoder map the


different languages to the same space such that the same things are ending up


in the same place it seems it seems a bit counterintuitive right because it


doesn't know which things correspond to which things now the first thing you


need is a shared vocabulary here since we are in a shared space right here the


what you need is a shared vocabulary so you tokenize all of the text with a


shared vocabulary and this vocabulary is going to consist of sort of word pieces


now if you don't know what word pieces are in a word piece tokenization what


you would you what you would do is you would split words into so-called word


pieces so for example hello right here might be split into two word pieces the


first word piece might be hey and the second word piece might be LLO okay there


is usually like some kind of indicator here that this is the end of a word and


so on but will simplify and hollow right here would be a ha for the first token


and then LLO for the second token and these kind of word piece encodings


since the smallest units are going to be the characters themselves they ensure


that you always have everything is in vocabulary you have no out of


vocabulary tokens but here you can already see that if we tokenize the


languages like this and then we use the same encoder so the same encoder will


pop them into this shared space that means that to the model this and this


looks like the same thing it is the same thing right it's the same input token in


different languages now as you can see this comes from the same word the low at


the end in English and the low at the end in German it comes from the same


word so you know it's fair to assume that since it's the same input it's


going to be mapped into a same embedding space right here or since these things


are usually context dependent we can say in a similar embedding space or a close


embedding space but certainly the the initial vectors are the same that is


already half the task right so by by tokenizing in this way we have already


mapped part of our languages even though they're just a different languages we


have mapped the same word to the same space and this relies on the fact that


in this case for example English and German and for example French they have


a significant overlap in their words as such so the the the word hello and the


word hollow they are almost the same word as letters as word pieces and these


these shared embedding techniques abuse sort of the fact that these languages


are close they'll say ah there will be some word pieces that are going to be


the same in these languages and naturally because they're the same


they'll be they'll end up in the same place in embedding space and because


because that now you have the so what these embedding techniques do is they


simply figure out the statistical relations between the word pieces so if


two things appear together often in the same context they'll be mapped into the


same space as well so it would realize there is a lot of times where ha and


hey appear in front of this low thing so I should probably map the ha and the


hey to the same location in embedding space so the same relation to the low so


they would end up at the same place so now you see even though these word


pieces are different like they get different IDs they'll be mapped to the


same place in embedding space because their relation to the low is the same


and the low themselves are being mapped to the same place because they are


actually the same so you can see that this this partial overlap between word


pieces in the different languages combined with the shared embedding


pre-training results in these token across the languages results in an


alignment of the embeddings so naturally the things that mean the same things are


going to be in the same places in embedding space either because they are


the same or because their statistical relation to the things that are the same


is the same it is it is sort of like these ha and the hey are like synonyms


in this shared language so if you jumble all the English and German text together


the model things ha and hey are synonyms and therefore maps it to the


same space this happens exactly the same in if you only have one language two


true synonyms so true this would exactly be the same thing in this case all right


so now we have different languages we have a single encoder where we can input


any of those languages mapping it to the shared space the decoder can be trained


by simply giving it this indicator token right here to decode the appropriate


language so now the question is how exactly do we train this such that such


that this happens that there's one caveat in programming languages of


course we still have to check whether that's the same or not and we know that


in programming languages a lot of programming language for example the


word if is the same right so if you tokenize Java or Python or C++ the word


if is the same and likewise there is a lot of overlap between the different


programming languages and that exactly is this correspondence to here these


models use the parts that are overlapping either tokens themselves or


this can also be grammatical constructs and so on they can also be overlapping


and therefore map to the same space so if a construct is used in the same way


and this can be in higher layers this can induce the same effect and they'll


use that to map the and sorry they'll use that the result will be that the


div the similar things in the same in these languages will be mapped to the


similar spaces in embedding space now this makes this example of it weaker


because that means this method would work exceptionally well for something


like Python 2 to Python 3 because they of course have like a lot of overlap of


syntax and keywords and constructs whereas something like cobalt to Java


it's more let's say doubtable that they will they will work so well in this


paper here they've chosen C++ Python and Java which do have significant overlap


but especially something like Python to Java of course there is a lot of a


difference or Python to C++ as well Python is not typed and so you can see a


bit of the difficulty is already in the paper here but you have to be aware that


this works less and less the less this shared overlap is given all right so how


do you train these models and remember we don't have parallel corpora we are


simply reliant on having databases from having big repositories of Python code


C++ code and Java code and they don't correspond to each other so the first


thing you do and as I understand it you can do these things in parallel but


there are three different objectives that achieve three different things in


these models the first objective is the cross lingual masked language pre


training now the models here are going to be transformer models with encoders


and decoders and that's comes from the attention is all you need papers and


various other papers like this I've done videos on those if you want to see that


this masked language model pre training however is from the BERT paper so BERT


if you don't know what that is I've also done a video on that this simply trains


the encoder so this is to train the encoder what you would do is you would


input code so usually in masked language model if you train the encoder you input


code with tokens like hello there you would then so this is your input you


would then mask some of the tokens for example here the low and maybe the


entire word there we would scrap that you would put it through your encoder


which is this transformer model like BERT and then BERT is supposed to


reconstruct hello there okay BERT is supposed to reconstruct these two tokens


like it doesn't see them and you ask it what did I cross out and it needs to


reconstruct that so you train the model to reconstruct these masked tokens and


the research on BERT and other things has shown that if you train with this


objective the encoder sort of learns about the structure of code it learns


about it learns which tokens and which constructs often appear together and


therefore it learns something about the the structure of the input and that


means it can create whatever is up here is a good and meaningful embedding for


these things that tells you something about the statistical coexistence of


tokens and of course since we're doing this with all the languages so the


Python goes in here C++ goes in here without telling the model what it is you


just throw it in there right Java goes in there by tokenizing it and as you see


an example right here so if this right here is C++ but in Python this would also


be if and since it needs to learn a single encoder for all of these


languages and since the tokens overlap partially it is going to result in


exactly what we want namely a shared embedding space where even though the


input comes from different languages it is mapped similar things are mapped to


similar places in the embedding space all right so the mask language model


pre-training very quickly as you take a piece of code like here on the left you


mask out some of the tokens here you can see them in this mask and you simply ask


the model the encoder to reconstruct those things okay so this is just for


the encoder as far as I understand it the encoder doesn't it doesn't see the


thing back here it simply sees this and you tell it please reconstruct please


tell me which words or which tokens I I clipped out here and it's supposed to


tell you okay the first one is if the second one is int and the third one is


the I now if you consider what the encoder has to do here so if you were to


see this then that pretty clearly you know you could you could guess that


that is an if of course it's not a hundred percent but this is just


pre-training right so you train it to output if here now here you have to do a


little bit more inference maybe you've seen this for construct a bunch of times


and you can see that this is compared here and this is added so probably it's


an integer and then in the last thing this is even more complicated if you


don't see that the I is here you somehow have to guess that what it is it's not


clear right but you can guess that okay there's a local variable I right here


and probably it's going to be used somewhere in this block now this here


isn't I and I don't see I anywhere else so probably I goes in here which makes


sense because it's an integer and prime is an array and and integers index


arrays so on okay so this is what the model does first the second thing is we


need to train the decoder somehow how do we train the decoder in a very similar


way we make the decoder do denoising auto encoding now before we just had


single tokens we just asked the encoder to reconstruct tokens so the encoder is


this box right here this colors this box is the encoder and the actual part


that's going to predict these words is going to be one sort of one


classification layer on top that is going to predict for each position the


individual word now did this is just for pre training after the pre training you


scrap that and you attach it to a decoder so you attach whatever you got


out of the encoder to a decoder and the decoder will output in an


autoregressive way one token after another righty output a it output a


token right here and it feed that token back into the decoder saying okay here's


what I've produced now produce the next thing it would produce the next token and


so on so it would produce token after token the output and now as I said I'm


not exactly sure I think they're doing all of these things at the same time so


this would still be here but the information would just be routed in two


different ways or maybe they do it one after another it doesn't really matter


but what matters is in this thing here you now train the decoder I mean you


train it jointly with the encoder but you also involve the decoder and you do


this by doing something very similar you corrupt a piece of code and you get


corrupted code now you can see part of this corruption is masking like you did


before but also part of the corruption is like here you scramble some of the


tokens right this was it was this over here you just jumble some of them around


a bit and then you here you also drop a token as you can see that the one is


dropped and you simply so you don't show this to the encoder or the decoder you


input this corrupted code into the decode into the first the encoder and


then you ask the decoder to give you back the original code without showing


it the original code so the the task for the decode for the encoder decoder for


the entire model here is if you're if you see this here is corrupt the code I


have corrupted it in various ways please tell me what I originally had now we can


the masking it does the same as before it sort of infers it this thing here it


says well probably I probably this isn't really correct you don't even tell it


where the errors are right before with the masking you at least told it where


the errors are now you don't even tell it where the where the errors are so it


needs to recognize this here is probably correct this isn't this I'm gonna


rewrite this to that okay and it does this one token at a time so it first


goes in to the data data and it needs to output the correct thing this is I hope


the difference is clear to the masked language modeling which involved the


involve the decoder and here also is the first time where in the encoder you you


prepend this Java token now this as you can see it still goes from the same


language to the same language but this is where you train the decoder to output


a given language so here with the token again this is the same decoder for all


the three languages the only difference here is every time you simply provide it


with the special token at the beginning to tell it which language it should


decode right now so this this now we have an encoder that maps all the


languages to a shared space and we have a decoder that conditioned on a token


like this can output valid code in that thing assuming this here was corrupted


code now since the encoder is shared it should map the same kind of corrupted


code of the different languages to the same place in the embedding space and


therefore this would also this would already be enough to have this model


that we desire we can input some code it doesn't actually have to be corrupted


right we can just input some code in one language and ask the decoder to output


the other language and this works but it doesn't work super well and here the


authors go for another idea from the unsupervised machine translation


literature which is back translation so back translation is a technique where you


can tune an unsupervised machine translation model in a way that you


would tune a supervised one but of course you don't have supervised data so


what's the plan you will produce the data yourself using your own model so


the plan is pretty simple it's actually contained in the back translation name


so if you have a piece of code what you would do is you would first use your


model to translate this to another language any of your choice now you have


no clue whether this thing here is correct or not you have no clue and you


have no way of assessing it because you don't have ground truth what you can do


is use your model again or actually use a second model that you train in parallel


I believe in this case they could use the same model but you could that could


be instable and so on but in any case you can use your system again to


translate it back to your original language your system can do that right


and here whatever you get as an output you know the ground truth it's whatever


you started with so now you can compare what comes out to what you started with


the difficulty of course is if there is a mistake you don't know which of the


two models made a mistake and you so it could be could be that your original


translation model made a mistake and or it could be that your back translation


model made a mistake and you have to find a loss function that kind of


punishes both equally or you simply keep one sort of constant and loss free and


train the other one because there there's going to be a sample where you


have C++ as an input and then the intermediate language is Python so all


of the models sort of get trained once as an as a source to target translator


and once as a target to source translator but I hope the the objective


is clear from the back translation so now with the back translation you


actually you train the models to go from one language to another language okay


and that's the that's the final goal even though you do it without supervised


data you now have a model that can encode things into a shared space that


can decode into a language and that is attuned to translating from one language


to another language so that's that's it how this is all how does this work now


for evaluation the question is of course how do you evaluate models like this for


evaluation they go to this website called geeks for geeks and this is a an


online platform with computer science and programming articles it gathers many


coding problems and presents solution in several programming languages okay so


this is a website that teaches you to code and it will have like an exercise


please do this and then it will provide solutions in the different languages now


why is that cool and they they have an example they have an example right here


why is that cool because not only can you be relatively sure that these


different functions that you have here do the same thing but you can also


relatively be relatively sure that they are implemented in the similar way


right because what this website is trying to do is it's trying to teach the


people how to how to code up an algorithm that they think up in their


head and therefore not only is the solution correct and the same it is


implemented in the in the same way as you can see here the construct there's


this if construct is everywhere the else if is everywhere so even though some of


the languages might have specialty things for implementing some algorithms


these are really the same algorithm the same expression of algorithmic thought


in the different languages so that is a perfect parallel data set the problem of


course is that there is not that many so it is good enough as a test set it is


not good enough as a training set but given that it's a test set you can just


have these as test set and then you can input the C++ and see whether or not the


Java comes out the problem here of course is that even though this is very


clear there are still you know sort of many variations of how you can implement


that to even express the same algorithmic fault so metrics from


natural language processing like blue just aren't going to be very good


because they look at n-gram overlap and you can write this function with very


different n-grams and still be very very valid and correct and also exact match


is not going to be really the the gold standard here so what they do is they


create a set of unit tests where for each of these functions they go they


check their input types they randomly generate input randomly generate a set


of inputs look whatever comes out and if the same thing comes out in all of their


test functions that they consider this a good unit test for that function so


whenever you your model now produces let's say you input Python it produces a


C++ you simply put these unit tests through the C++ function you produce and


if they produce the same output as the Python the original Python function when


on the same inputs then you consider the unit test to succeed and you consider


the function to be correct now this of course this isn't this isn't it is super


duper gold standard especially with random inputs because usually what you


want to do is test sort of corner cases but it's better than anything else so


far I've been a long dis advocate of unit tests honestly because I think


whenever a human writes a unit test then they're probably since they have already


implemented the function itself they're probably going to make the same mistakes


they're probably just going to replicate the code and thinking of the function in


the unit test itself and therefore it doesn't really get you anything I guess


in large organizations you write unit tests so that someone else doesn't screw


up your code but in this case it would actually be cool because now as a human


you could simply write a bunch of unit tests and then let your let your trans


compiler do the heavy lifting and you simply check whether or not the output


is good alright so how does this do here you can see they have some baselines the


C++ to Java as I understand it is a commercial system and the Java to Python


is an open source system both are human experts that make up these rule based


systems on how to trans on how to translate code into other languages now


the if you do what what they have here is transcode or beam one which means a


beam size of one so if you don't know what beam search is very shortly beam


search is like if you decode from your language model you can either always


take the next token that has the highest probability this would be greedy decoding


or a beam size of one or you can sort of always keep the top n hypotheses of what


the of what the most likely output is as you can keep that as a you can keep the


top five in memory and always decode these five on sort of like you have a


mini batch of five sequences and you always keep the top five in memory so at


the end of the decoding you're going to have five different variants of the same


sentence or of the same decoded output and you can then decide which one you


like best and usually what you do is you then output the one that has the highest


probability which is not the same as the greedy because sometimes the next token


will be will look one next token will look very good in a greedy way but you'd


better take the second most likely because the next to next token is going


to sort of make up for that to make the entire sequence even more likely so more


beam size basically means you can keep more hypotheses of the output in memory


until the end so if you just do the greedy decoding you see you already get


fairly close to these baselines it's very very cool very interesting and if


you up the beam size you surpass these baselines now the way they up the beam


size here I find to be a bit let's call it a bit cheaty because when they say


beam five what they mean is they keep the five hypotheses and then at the end


I as I understand it if any of the five hypotheses passes all the unit tests or


the most they keep it right so basically they give themselves the freedom to say


whichever one of the five we output is the best that's the one we count and of


course that's not really a match to the commercial or to the baseline system


because it can output one thing now it is maybe a good practical application to


give the human that you know you input a function you give the human five options


to choose from and it can choose and thereby decide which one the human likes


best but it is sort of it is wonky what I like more is this here the beam 10 top


one this is what you would actually do so we could keep 10 hypotheses during


decoding and at the end output the top one the top likely one and as you can


see that is better than greedy but it is worse than where you you know give


yourself the freedom to output multiple ones of course though they say that most


of the errors that this top one makes come from compilation errors when the


target language is Java or C++ it suggests that the beam and top one


metric could easily be improved we leave this to future work which this again I


find valid right so if you if your method is I'm going to keep the top 10


hypothesis hypotheses until the end and then I'm going from the top and I simply


compile them and I output the first one that compiles that that's not cheating


right that's a valid thing again yeah so in that way I can I can understand what


they're saying right here okay so they give some examples some of which I find


very interesting so the first thing here is that oh yeah by the way I've said in


the I've said that the tokenizer between the natural languages is shared they


make a little tweak here in that they tokenize the different languages with


their language respective tokenizers which will still end up tokenizing


pretty much you know this the print statement in C++ or in Java no actually


the print statement in Python is print and in Java it's println and so on but


it will still like the all the if statements it will still tokenize into


the same into the same word but it's simply not viable to to parse Python


with a C++ parser okay so we have looked at this the results this is one of the


results they look at their shared embedding space and this is a t-sneep


plots a 2d projection of this shared embedding space and you can see that


this is actually happening so the different so null null and none are mapped


to similar locations println and cout are mapped to similar locations in this


space so this is exactly what we want this is sort of a verification that this


method of embedding the different languages into the same space really


turns out such that whatever means the same thing is mapped to the same place


you can see here catch and accept two very different tokens are mapped to the


same place simply because they're used in the same sort of constructs across


the languages very cool one of these examples here is quite impressive and


kind of shows the difference between this and and rule based translation in


this function right here you have a C++ function that takes a character


pointer to that is called STR str in as an input now in C++ strings are at


least in old versions of C++ strings are handled as character arrays so a


string is indistinguishable from a character array and in this case usually


what you do is you don't input the array because that will cause a copy you input


a pointer to the first two input a pointer to the array and that would


define the string ok so if you translate this again this the type of this is


simply character array if you translate this with this transcoder system that


they've built into Java in Java there is a type called string right there's a


native type called string and is that true I think oh yeah that's and then


that's handled really weirdly in the JVM I think yes so there is a at least there


is a type called string so it would map that it would recognize I you mean a


string therefore I'm going to put a string here and it uses all the string


method like string length string character at and so on where in C++ this


is just an array and you just have array accesses now they take this same C++


function and only change one thing they change the name of the parameter


everything else is the same but now the character array is called R okay and


they put it through the same system and that system now outputs a function that


takes in a character array called R instead of a string and it uses you know


here the property length it uses array access instead of this car character at


method so simply by changing the name and this is something where I believe


the the rule based systems can this can be an advantage over rule based system


because what this here does is it simply says oh I've seen a lot of humans in my


code base that use this use like stir as a as a variable name and that usually


means that the constructs here are like the constructs in Java where people use


strings and I've seen other places where people use you know names like this


right here and usually that is used in the same context as in Java people use


character arrays right so it in programming it's not only important what


the the code actually does but a lot of programming goes via naming of things


like other programmers will read your code and by reading stir right here they


will sort of assume that this is a string whereas if they read our right


here they will assume you're a pirate and you are referring to a character


array and they will treat the code the code means something different and these


systems right here these neural machine translation systems can actually


understand that part because they do statistical inference on code that humans


wrote if you change this back to say input then again it goes back to a


string and uses all the string functions so that's fairly impressive in my mind


and it yeah definitely an advantage over rule based systems of course the


disadvantage of a rule based systems is that in rule based systems you can


almost get on like sometimes you can even guarantee that the code does the


same thing here you can't they give some examples of failed translations where so


now you get you run into this problem where the min function in Python is


overloaded it can either give you the minimum of a sequence or it can give you


the minimum of two things now this is translated to Java right here and math


dot min is not overloaded in Java it only gives you the minimum of two things


and not the minimum of an array and it still outputs that now given enough data


probably could learn because these things are all context dependent but


this is one of the this is one of the failure cases of these models of course


all right so this was this paper I I've I've read that the code of this and


the unit tests will be output will be put online at some times they are not


right now if I if I hear about it I can link to it or let you know about it let


me know what you think of this paper in the comments share it out and subscribe


[ML News] GitHub Copilot - Copyright, GPL, Patents & more | Brickit LEGO app | Distill goes on break

An open door. An open window. An open bottle. OpenAI and GitHub invent copilot and everyone


freaks out about copyright. Welcome to ML News.


Greg Brockman writes an AI pair programmer in your editor. It's powered by open AI codecs,


a new AI system which can convert from natural language to code with increasing reliability.


He's talking about GitHub copilot. So copilot is this system that's developed by open AI


and GitHub to be a super duper autocomplete. Basically, what you do is you write the name


of a function or some kind of a class or actually anything you want, maybe along with a little


bit of a doc string, and the system will complete code for you. Now other than classical autocomplete


systems which are rule based and basically suggest to you what's possible, which variables


fit here, which ones are in scope, this system goes much beyond this, it will try to guess


what you're trying to do. And it will write this code for you or it will at least suggest


it so they have a bunch of examples here. For example, this parse expenses statement,


the user writes the function name, and then a few examples in the doc string as you would


write if you were to program it, and then copilot implements the function itself. Now


I've been using tab nine for a while. And I'm pretty happy with its suggestions, especially


if you pair it up with a classic autocomplete, you get the classic autocomplete, which tells


you what you are allowed to do essentially, and you get the AI autocomplete, which is


trying to guess what you want to do. This enables things like if I catch an error that's


called password error, it will already provide a log message for me that says password wrong.


And there are many more examples where it just kind of infers what you want to do. And


that's super helpful at times. Copilot by GitHub is this on steroids, it will implement


entire functions, entire classes from a description or even just from a name of a function. Now,


it's not going to be perfect, of course, whether it actually helps or hurts. And who does it


help? Does it help the experienced programmer because they can write faster and just have


to check for errors because there definitely are errors. If you see right here, in this


expense function, the money is held as a floating point number, which is a big no no when you


handle currency. On the other hand, does it help novice programmers because they see the


implementations of functions they wouldn't know how to implement. However, they're probably


going to not catch the mistakes there are. There's a lot of debate around this, but I'm


pretty excited to see this honestly. Now the issue comes when you talk about the following.


They say it's trained on billions of lines of public code, GitHub Copilot puts the knowledge


you need at your fingertips saving you yada yada marketing, however, trained on billions


of lines of public code. That means they essentially went to all of GitHub all the public repo


and trained a giant language model on it. It's nothing more than this. It's essentially


something like GPT three on code probably augmented by a bit of syntaxing and whatnot,


but it's not much more. It's just lots of data, lots of compute gives you a model of


what people usually do when prompted with some sort of strings. So safe to say this


won't replace programmers exactly anytime soon as you can maybe see from this is even


function implemented to extreme precision, of course, actually, I don't know if that's


even real or a fake because people have definitely been making fakes about Copilot. This is not


going to happen anytime soon. What's more worrisome is for example, open AI Copilot


emitting personal information such as this open SSH private key which someone left in


their repository and now Copilot is just regurgitating it. In fact, on the FAQ page, GitHub Copilot


says yes, they sometimes output personal data not because they do anything wrong, but because


people left that personal data in their repositories and the system is trained on those repositories.


And sometimes it will decide that the most likely output is that training sample. And


that gets us into an interesting topic. So the topic is does GitHub Copilot recite code


from the training set? Now we've been having this discussion for a long time. Do these


large language models actually understand what they're doing? Or are they simply kind


of reproducing the training set? And if they reproduce the training set by which degree


do they integrate maybe multiple training set samples combine them? Or do they just


take one and kind of reformulate it a little bit? Who knows, GitHub did an extensive study


in which they found that only about 0.1% of the outputs are in some way reproductions


from the training set. However, there is a big dispute about what exactly counts as a


copy as a recitation and how different is different enough. And that gets us into the


biggest issue, which is copyright. So the issue here is that GitHub and open AI essentially


take all of this code train their system with it and they don't give you the copilot for


free. Of course not. I mean, how are you going to live up to that name open AI? They're of


course going to sell this. Now fair enough, they did something cool, they want to make


money. However, the code they used in order to train the system isn't always freely available.


At least that's what people think. Now, how would you feel if you wrote some code, you


are the legal owner of the copyright to that code and GitHub simply trains a model on your


code and then sells that model for other people to produce their code and they don't have


to give you anything for it. Also, there is the issue of GPL license code, which requires


that any modifications to it again become GPL license. The question is, if the model


outputs code, that was a result of training on GPL code, does the output of the system


also become GPL licensed or not? And there is even more of an issue when it comes to


patents on code patents are yet another category of intellectual property protection. And we've


seen example of copilot reciting patent protected code. With all of this, I've been reading


into software copyright and whatnot a little bit and I want to give the disclaimer, I'm


not a lawyer, this is not legal advice. This is entertainment purposes only if you want


some actual opinion, go to an actual lawyer and pay them. But also what one can say is


what Lucas buyer here says, with everybody hypothesizing about copilot and GPL license,


let me add another perspective, nobody knows and nothing whatsoever will happen until someone


sues someone. I'm not going to hold my breath, which is true, ultimately, a judge is going


to have to decide case law has to be established, and we'll take it from there. So what follows


is my personal opinion on the matter trying to analyze this a little bit. Here's a bit


of a diagram of what's happening currently in this system, you have the copilot system


as a piece of software that contains a maybe a neural network that has been trained on


some stuff, how did this copilot come to be the copilot is built upon libraries such as


pytorch, which are usually fairly openly licensed, like an MIT license or something like this.


So there's no problem there, then copilot, of course, needs copilot.pi, the thing that


you actually run to do the training and the inference, which also is authored by the copilot


authors, and therefore not an issue in our case. But then one of the inputs to copilot


is of course, the giant data set. Before we even get into licensing of that data, we have


to talk about copyright itself. Everybody's talking about GPL license and whatnot. But


GPL being a copy left license only pulls if copyright law even applies. So first we have


to see does copyright law even say anything about using this code in this way. Copyright


law works differently in different countries, but in general, it protects creative outputs


of people. So if you do something, if you express yourself in some creative way, you


obtain automatically copyright on that artistic expression. So if I write a song, then I am


the owner of copyright for that song, I don't have to register it anywhere, I have it by


default. Now as an owner of copyright, I get certain benefits. For example, I can decide


whether or how my work is reproduced, which derivative works can be made and how they


are treated, how it is distributed to the public, how it is performed, and so on. I


have certain rights to the dissemination, reproduction and modification of my work.


Now notice what's not on this list, enjoying the work reading the book reading the code.


So as a copyright owner, once I've decided to display my work publicly, I can't actually


prevent anyone from looking at it in the public space that I chose to display it. So one place


we actually have to go is the terms of service of GitHub. So under user generated content,


GitHub says you own content you create, but you allow us certain rights to it. And a sub


point they say we need the legal right to do things like host your content, publish


it and share it. This license includes the right to do things like copy it to our database,


make backups, show it to you and other users parse it into search index or otherwise analyze


it. Now you can debate whether or not otherwise analyze it means they can run machine learning


model on top of it given that they say this is in order to fulfill their service. But


certainly you allow GitHub to display your code and anyone can go on GitHub and you cannot


prevent them from reading your code. You cannot prevent them from actually downloading your


code to a private hard drive. In fact, the ideas and algorithms behind code are not copyrightable.


What's copyrightable is only your expression of those ideas. So I can't copy your code,


but I can look at your code, learn from it and then express the same idea in my own code.


If you want to protect an idea, that's the terms of patents. And that's a whole other


game, you actually have to register for a patent, whereas copyright you obtain automatically.


So if I can look at your code, learn from it, and then reproduce it in my own way, why


shouldn't machine be able to and that brings us to the second important point right here,


which is the right to prepare derivative works based upon the work. Now, according to Wikipedia,


a derivative work is an expressive creation that includes major copyrightable elements


of an original previously created first work. Now the article here is mainly concerned with


what copyright exists on the derivative work. But for our purposes, if something is a derivative


work of something else, it is potentially in violation of the copyright of that first


work. And when is something a derivative work if it contains major copyrightable elements


of that original? Now, is this all a bit fuzzy? Yes, absolutely. And there is a giant gray


area of course. So if I look at an algorithm, and I implement that in my own code, what


counts as containing major copyrightable elements of the original if I use the same kind of


indentations, if I use the same variable names, I use the same structure, this isn't really


an exact science, it is for judges to decide, but safe to say, there is a way where I can


learn from other people's code, no matter the copyright situation, and I can then write


something based upon that, and it is not a copyright violation, there is also many situations


where the exact same thing is a copyright violation. And that all depends on how much


of the copyrightable elements, so not the ideas, but the expression of the original


work is contained in the derivative work. And that of course, brings us all the way


back to the discussion, do large language models simply recite the training data and


change it a tiny bit? Or do they integrate the training data, learn from the training


data, learn the patterns behind the training data, and then come up with their own way


of expressing those patterns? The truth is probably somewhere in between, they're not


exactly copying the training data, but it's also not the fact that they understand what's


behind the training data. But safe to say, there is a way where copyright might not even


apply. And then there is actually no problem right here. But let's assume for a moment


that copyright does apply, and things are actually in the realm of derivative works.


Well, then there are still multiple questions right here. For example, here you see that


there are multiple elements in the system. One is copilot itself as a software. Now,


if you argue that somehow the copyrightable elements of the input data end up in the weights


of the neural network, and therefore the neural networks are essentially a derivative work


of the input data, then copilot itself might be in violation of copyright law. But even


if copilot isn't a violation of copyright law, still the output of copilot might be


in violation of copyright law. And that's going to probably have to be decided on a


case by case basis. And it might even be that open AI might not be responsible for this,


but the person actually using the copilot tool to generate output, it's all a bit of


a messy situation. Notice what we haven't talked about so far, GPL, because GPL, as


I said, only applies when copyright applies. Now let's assume copyright applies. So here


is where we get into licenses of code. In general, the training data contains broad


categories of how code is licensed. And I've listed four of them here. There is the boring


code, which is so boring that copyright doesn't apply literally, it's no expression of creativity.


It's just formulaic code writing, maybe even auto generated, not copyrightable, not a problem


there. There is also the open category, which is so openly licensed that it's usable in


any format, like an MIT license, as long as you keep the disclaimers there, you're fine.


Then there is the bunch of code that does not have a license at all. If there is no


license, that essentially means that copyright owner simply gives GitHub the right to publish,


but retains all other copyright and everything we said so far applies. So either copilot


or the output copilot generates or actually both might be a violation of the copyright


of the unlicensed code. And then there is GPL code. So the GPL, the GNU general public


license in this case, version three, but they're all kind of similar. I know, I know TVization,


they are generally known as copy left licenses. Because if a piece of code is licensed under


the GPL, it means that if you were to modify this code, then your modifications also have


to be licensed under the GPL. And being licensed under the GPL means things like if someone


obtains a copy of the software, then also you have to provide a copy of the source code


with that software. So the GPL is a bit like a virus that if it initially applies to a


piece of software, someone else uses that software, maybe modifies it a little bit or


includes it into their system, the whole system has to be under the GPL or they are in violation


of the license. Of course, if copilot is found to be a derivative work of GPL licensed data,


that will mean copilot itself would fall under the GPL and therefore open AI would have to


give us its source. Now what source code is is a bit of a tricky business in the legal


scene, but GPL defines it as the preferred form of the work for making modifications


to it. Now, what is that exactly for open AI pilot? Maybe it's not the weight of the


general network itself, because like, how can I modify them? Maybe it's the training


set plus copilot.pi. Maybe it's even not even the training set, but it's actually the scraper


for the training set as well as the training code. Who knows? Now GitHub and open AI can


save themselves from having to release the source code of copilot if they only make it


available over the network, in which case you don't have to give out the source code


license that would only be in the case of the a GPL. Regardless of that, the bigger


question is what if the output of copilot is a derivative work of GPL licensed code?


In that case, the output of copilot in a case by case basis would also have to be GPL license.


And who's responsible for that? Probably you as a user of copilot. If you ask copilot for


code, you get an output. I don't think it matters whether or not you know that it's


a derivative work of some GPL licensed code. If you then use that code and build upon it


and maybe sell software based on it, that software technically is under the GPL. So


this was my little take on the copyright situation around open AI copilot. I think it's a great


tool, but you can also see it brings a lot of difficulties with it, not necessarily technical


difficulties, but difficulties from the human environment. So let me know in the comments


what you think about the situation about copyright and whether I completely butchered some of


the things. Thanks. Next news speaking of copyright, Facebook AI launches a image similarity


challenge where they want you to figure out where all the memes came from. So the challenge


is essentially figuring out if someone took some photo and modified it in some way. And


of course, the reason behind all of this is going to be to find the original creator of


every meme so we can give them proper credit and glory they deserve. Nothing else, no one


else image matching very limited applications. Don't even worry about it. Next news, Brickett


is a new app that scans your Legos and tells what you can build from them. Pitta pixel


has a good article about it and shows this demo video, the app will scan your collection


of Legos and then tell you what you can do with it. So you can see it gives you a bunch


of suggestions of what to do pretty neat. Now this is a really, really cool app, though


I wonder the things it proposes are often made out of maybe 20 parts and this pile has


at least 500 or so. In any case, if you do have an iOS device, which I don't give it


a try, it looks like a lot of fun. Next news in more sad news, the Distill pub website


is going on a break. So you might know Distill as an online journal which publishes in a


non traditional way they want very interactive articles, they want very visual articles explaining


something. They also publish commentaries, threads, but also peer reviewed science. The


frequency of publication hasn't been too high from them. But the things they have published


generally were super well received. So one reason they cite is volunteer burnout, which


given the high quality standards that they have, I can totally believe this is an enormous


effort to keep this going to keep the quality high and you know, respect for doing it this


long. The article makes another point namely that self publication seems like the future


in most cases. And I think the field generally agrees today's scientific progress is more


made through sharing archive publications and discussing them on social media than it


is through the peer review system of conferences. So even though it's sad, Distill will take


a break what they're advocating for is a better future for science. And that's a great thing.


Okay, next news and gadget writes Amazon is reportedly using algorithms to fire flex


delivery drivers to Amazon being Amazon has this huge fleet of drivers that they don't


necessarily hire. It's kind of like an Uber model where the driver has an app and they


get essentially subcontracted for driving stuff somewhere. And these aren't few drivers,


they're apparently millions of drivers doing this. Now keeping up some sort of HR department


on some sort of human contact with millions of people is a challenge. So Amazon opted


to just not do it. Instead, they use algorithms to track the performance of their drivers.


And if the performance sinks too low, they fire the drivers algorithmically. So the article


states the frustration of some of these drivers saying the system can often fire workers seemingly


without good cause. According to the report, one worker said her rating fell after she


was forced to halt deliveries due to a nail in her tire. She succeeded in boosting it


to great over the next several weeks, but her account was eventually terminated for


violating Amazon's terms of service. She contested the firing but the company wouldn't reinstate


her. Another driver was unable to deliver packages to an apartment complex because it


was closed with the gate locked and the residents wouldn't answer their phones in another building


in Amazon locker failed to open so their own system failed and they punished their drivers


for it. His rating also dropped and he spent six weeks trying to raise it only to be fired


for falling below a prescribed level. If a driver feels they're wrongly terminated, some


feel there's not much recourse either driver must spend $200 to dispute any termination


and many have said it's not worth the effort. Whenever there's an issue, there is no support


said Coke who is 29. It's you against the machine. So you don't even try. Now here you


could try to make a nuanced point that these people aren't employees that it's simply not


a practical solution to manage these as employees that overall the system might be better off


that a lot of drivers are having good experiences that this is just a necessity of managing


so many people but but see, not so long ago, I wanted to get some Amazon gift cards for


my discord admins. They're doing a good job. I wanted to give them some things to try to


buy some gift cards and Amazon locked me out of my account security reasons. So I verified


my identity all good right to buy the gift cards again. They locked me out again verified


my identity tried a third time now they locked me out permanently. So I'm trying to contact


support Guess what you have to do contact support login Oh great. Guess what you have


to do to get a support contact number login Oh great. Try emailing them. Nothing happened.


Tried calling them they say they'll fix it. They haven't fixed it for months now. They


said I should make a new account rate verify the phone number of the new account your phone


is already associated with an account. My old account has all my collection of audio


books and ebooks on it. And this is just splendid. So I definitely feel with this drivers if


it's you against the machine. Amazon ranks just about second to PayPal when it comes


to actual customer support. So I'm not going to make the nuance point here. Screw you Amazon


screw you. You deserve every bit of negative press that you're getting here. At least when


there's an issue have some support for your drivers who get a nail stuck in their tire.


Yes, I'm using a journalistic medium to settle a personal dispute. What are you going to


do about it? Get me my account back. Okay, next we're going to look at some helpful libraries.


We should make this a segment helpful libraries, helpful libraries. Okay, TensorFlow introduces


decision forests, new algorithm never heard of it before. Give it a try decision forests


in TensorFlow. Facebook habitat 3d environment to train your autonomous robot to get you


something from the fridge when you're just too lazy. Have fun with your diabetes. Try


it out. Google Research Falcon trains your game playing agent, you give it a little bit


of a demonstration, it learns how to play your game and test it for you and find bugs.


So now you don't even have to play your game while you don't walk to the fridge. Good job.


And lastly, did you ever want to figure out what the gradient is of your face smashing


against the wall? Well now you can with Google AI's Brax, you can simulate physics in a


differentiable way on a TPU really fast.


And in our last news, TNW writes fake science is getting faker thanks AI journals are retracting


more and more papers because they're not by the authors they claim to be. Now of course,


you always know it's a serious article when there is a very futuristic robot on the picture


in the front. But the article is actually a good article talking about the rise of AI


generated papers and how there is a massive upsurge in retractions among scientific publications.


But besides that, I like the intro they say they say of course, sometimes papers get retracted


because of the authors made an honest mistake in the research in more than half the cases.


However, it's because of academic misconduct or fraud up until a decade ago, this sort


of behavior was more or less limited to researchers falsifying experimental data or skewing results


to favor their theory. The more sophisticated technology has become however, the more things


have gotten a lot more complicated. So the rest of the article talks about how people


add big names to their papers, how people generate fake authors, even how people generate


even fake papers and so on. You know, that's a whole big problem. But I still think that


people being shady with the results of their research is still the biggest problem. There's


just not too many retractions of it in machine learning, because you can never reproduce


someone else's paper. If you didn't get my numbers, you just did it wrong. So what is


the real solution against fake science? It's probably hard to know. But I guess an approach


to a solution would be to have some sort of a distributed checking mechanism where you


can aggregate opinions from all around the world about a given topic and then sort of


look at everything and evaluate for yourself rather than relying on a centralized committee


to do it for you. Be that for fake news or fake science or fake anything. I think that's


the only way forward because any centralized institutions will eventually get either corrupted


or game because they have some sort of scoring system. But I'm interested in what you have


to say all of this is a problem. It's not exactly clear how we go about making this


better. Can we even make it better? Or can we just find better ways to ignore the fake


things? All right, that was it from me for this week's ML news. I hope you had fun. I


hope you don't get replaced by a machine anytime soon. And most of all, I hope I don't get


Agent57: Outperforming the Atari Human Benchmark

Hi there, you're looking at Solaris, which is a game in the Atari benchmark and it is,


has been one of the hardest games for reinforcement learning agents to solve.


What you're seeing is Agent 57, which is a new agent by DeepMind that is the first one


to beat all of the 57 games in the Atari suite to a human or superhuman performance.


So some of these games have been, you know, pretty easy for RL agents, but some of them,


look at this one here, have been pretty hard, mainly because of the reward structure.


Now you see on the top, top hand, top edge, the reward, it's not going up for a long time.


And this kind of games where the reward doesn't go up for a long time is very hard for RL agents.


So Agent 57 builds on a number of previous improvements to the original DeepQ networks


of DeepMind.


And today we'll look into this.


So it's called Agent 57, as I said, because it beats all of these 57 games.


They're quite diverse and it's a cool thing that a single system can beat it.


So they go into this, this is a printout of the website so I can scribble on it.


And this here, it's been cut off, but it should say DQN here.


DQN from 2015.


All right.


So this DQN paper of 2015 was kind of the original paper that popularized this Atari


benchmark and introduced neural networks to reinforcement learning basically, that made


it work.


Since then there have been a number of improvements.


So maybe we'll just go into what DeepQ learning is.


So in reinforcement learning, usually you have an agent and agent here and it, you have


an environment over here, right?


And the environment will give you an observation.


Now the observation in our case would be something like the frame of a game, right?


And you're here, you're a little rocket and there is a bunch of meteors, right?


And then the agent needs to somehow give back an action.


So an action and the actions in the Atari benchmark are always defined.


So you can, in, you know, in Atari, you used to have this kind of a joystick thing.


You can put it up, down, left, right, or you can put it like upright, up left and so on.


And also you have a button, I think one or two buttons.


I don't actually remember, but you do, you can press at least one button.


So these are the actions.


Let's say there is something like 20 different actions.


So all of the directions here, and then you can always press or not press a button with


it.


So you have to give, you send back this action here.


You can say, I want to, you know, put the joystick up and I want to press the button


at the same time.


And then the environment will give you back a, it will say, okay, we'll give you back


a new observation, which would be the next frame of the game.


You know, you've pressed up, so your little rocket is a bit more forward.


You've pressed the button, so you fired a shot and the meteors are still here.


And it will also give you back a reward.


So the reward, different games give different rewards.


For example, in Pacman, you know, every time your Pacman eats a little one of these dots,


you get a reward.


But in other games, most famously games like Montezuma's Revenge, you're in this room and


there are these platforms and ladders and stuff.


And you're here and there are opponents rolling around and there's a door over here.


You need to go down, jump over here, get up, get some key and then go to the door.


And only then will you get a reward.


So games vary in many ways in how you get this reward.


So that's kind of the intrinsic problem here.


So deep Q learning is the following.


We have a neural network taking in the observation.


So we have a neural network, let's designate it as this.


And the observation goes in here, right?


This is O, the observation goes in here.


And also the action goes in here.


Now let's call this AI because we have different actions.


And O, observation at step T. And it will give you a Q value.


So the Q value for the observation at time T and action I.


Now you do this for every single action.


So you put observation with action AJ in the same network, right?


You get an output that is the Q value for observation, same observation with this different


action.


You do this for every action.


And wherever the Q value is the highest, right?


Wherever that's the highest, that's the action you go with.


So what you have to do is you have to train this neural network to predict the Q value


as accurate as possible.


And the Q value is basically the reward that you expect from now until the end of the episode


by performing this action in this situation, right?


That's Q learning.


Simply predicting if I do action I right now, how much reward am I going to get from now


until the end of the episode?


Right, that's basically it.


That's deep Q and deep Q learning simply because you have a neural network doing the learning.


So that was deep Q networks and they work pretty well, but they don't work for these


long time horizons because you always just learn, you just see one observation, right?


And you kind of learn one step at a time and you rely on these Q values propagating through


from your experience.


It doesn't work very well for these long credit assignments.


Now a significant improvement upon that is this R2D2 algorithm that incorporated LSTMs


or GRUs, which are recurrent neural networks.


So not only does your observation go into the neural network, right?


Now your history of observations.


So what happened before?


So not only the current game state, right?


But here you have the observation from step one, the action you did at that step, then


the observation time two, the action you did at time two and so on.


They now all, so this is encoded and this is encoded.


And then you have a recurrent neural network that incorporates all of these things that


happened previously, right, into your current representation.


So now not only does the agency what is happening right now, it also gets the information of


what happened previously, what did it do previously, and it can also now back propagate through


these things and kind of learn a longer range credit assignment.


Credit assignment means it gets to figure out which actions had actually an influence


on the final reward.


If you incorporate the history, right, you can have a direct gradient flow across that


history.


So notably these LSTMs or GRUs, you can let them compute over maybe 10 or 100 steps, right?


And then you get a pretty good idea of which of these actions within those 100 steps led


to which rewards.


And the other thing on R2D2 is of course it is now more distributed.


So this was already here improvements to DQN, but the R2D2 agent is also distributed, meaning


that you have like a central instance.


So this is now engineering, right?


You have a central instance that is called the learner.


And the learner has the main weights, which I'm going to designate with theta here, and


it just takes in experience from all of these workers.


So there's worker one, worker two, worker three, worker four, and so on.


And these, they will all just run episodes.


They will all do work, work, work, work, work, work, work, work independently of each other


and then send back their experience to the learner.


And every now and then, the learner syncs out the weights of the neural networks to


the workers.


So that's kind of distributed RL in this sense.


You have a central learner, then you have many, many workers doing the actual interaction


with the environment.


So one of the main pitfalls of R2D2 is still it has a poor exploration-exploitation strategy,


which I believe it is still just kind of epsilon greedy.


What does it mean?


So in order to understand this, maybe consider again our screen here, right?


So let's say you're here with your space ship, right?


And there are, there's a meteor coming right here, and one right here.


And there is a gold coin right here.


Let's make this gold, right?


So let's say you get a reward for collecting the coin, but you also get a reward for shooting


the meteors, right?


So what happens if you shoot right now?


So if you shoot, then let's say you shoot and this meteor explodes, right?


So you get a reward.


Yeah, so one reward, but then the meteor right behind it will hit you, right?


It's coming toward you, you'll have no way, no time to get out of the way.


So one reward and then death, right?


Let's make a little arrow here, right?


So in total, you get one reward.


Now what happens instead if you move to the right?


So move, right?


So the next, in the next frame, the meteor will fly past you.


You are over here, right?


But the gold coin is here.


Now this has given you so far zero reward, right?


Oops.


It's given you zero reward.


But then in the next frame, you know, these meteors have passed now and you are going


to get that gold coin and that gives you one reward and no death, right?


So you can technically go on here and maybe you'll get five more rewards down the line.


So the, the, this is, here's the exploration exploitation dilemma.


If an agent has for some reason learned that the shooting action in this situation will


give it a one reward and the move action will give it zero reward, but has not learned to


look past this, so this is kind of nebulous here.


It has only experienced, it has only experienced one frame of, here, yeah, it has only experienced


one frame of experience.


It will say, wait a minute, shoot here appears to be like really good.


It gives me one reward and move gives me zero reward.


So from now on, I'll just always do shoot, right?


Shoot, shoot, shoot.


Now what you would like to do, so this is called exploitation, right?


Exploitation.


It has learned something that gives it a reward.


So it will just do that over and over again.


Whereas here you could say, ah, I, I might go this way even though it's zero reward because


I can hope, right?


I don't know yet, but I can hope that I will get more reward down here.


This is exploration.