And the question in, in reinforcement learning is always how to trade off these two, right?


Agent57: Outperforming the Atari Human Benchmark

Ideally, you would want your agent to collect maximum reward that speaks for exploitation


of what it has already learned.


But also you never want to discard the possibility that down the line of things that you don't


yet know, there might be even more reward and that speaks for exploration.


I'm just, this both are abbreviated same.


Exploit, explore.


This is dumb.


So in the original DeepQN formulation and I believe also in R2D2, this is done with


epsilon greedy, which is surprisingly performing well.


So in epsilon greedy, you simply say, I'm going to have a constant epsilon.


This is E epsilon.


This is maybe 5% or something.


I'm going to simply do something at random and the other one minus epsilon, I'm just


going to go with the, with the thing I have already learned.


And this performs pretty well, but you might imagine that there is something smarter to


do.


So never give up.


This algorithm, it kind of goes into this exploration mode where it tries to get, get


to smarter ways to do exploration.


And the keywords here are things like intrinsic motivation.


So intrinsic motivation and curiosity refer to the fact that it is, so in addition to


the reward you get from the environment here, right?


This, this reward right here, you can also interject at this point and say, ah, I'm going


to give some R prime, some reward of myself, right?


To kind of encourage some behavior in the agent.


And this here we call intrinsic, intrinsic.


So that means you add to the reward of the environment, you add some reward of your own


that has nothing to do with the environment or not much, but just encourages certain behavior


in the agent that is now also trying to maximize this intrinsic reward.


And in curiosity and intrinsic motivation formulations, usually you are rewarded for


novelty, novelty, which means the agent is rewarded for finding things that it has not


yet seen.


So you, in this situation over here, you might see why this encourages the agent to go down


this route here, because it says, wait a minute, there's a bunch of stuff, like here I just


die, right?


But there's a bunch of stuff I haven't seen yet down here, so I might want to go explore


that and we give it extra intrinsic reward, R prime, for seeing things it hasn't seen


yet so it will learn if I do things that I have never done, I will get this sweet intrinsic


reward and then it will go explore.


Now, of course, it's a big engineering question of how exactly to set this intrinsic reward


and there are many, many different formulations of that that fall under this term of, let's


say curiosity or something like this.


Nevertheless, this never give up has improved over R2D2 using ideas like that and now agent


57 improves again.


Now how does agent 57 improve again?


And it is mainly in this what I just said, so how exactly do you apply this intrinsic


reward?


How exactly do you navigate the exploration-exploitation trade-off?


That's where agent 57 comes in because what they've realized is that for these different


Atari games right here, some are very easy, some you don't need much exploration, some


you need a lot, some you need it over a large time scale and simply one agent, one never


give up agent with the same settings of this curiosity of how long it looks into the future


is not going to solve all the games.


So agent 57 learns how to modulate this exploration-exploitation trade-off.


So let's jump into the paper a bit more.


I encourage you to read the blog post that is quite thorough and the paper is a bit more


technical.


Sorry, let me switch over.


This is the paper, agent 57 outperforming the Atari human benchmark by Google DeepMind.


Here they say improvements to end you, to never give up.


So the first improvement they do is, so we've already talked about how this is classic Q


learning, right?


So you're trying to learn this function that gives you the Q value of an action and a state.


Now since we're going to deal with intrinsic reward in addition to extrinsic reward, it


makes sense, that's what they argue, to split the Q learning function into two different


parts.


One part that learns the extrinsic reward and one part that learns the intrinsic reward,


right?


And there you have a parameter beta in front of it.


Now beta in this case is the trade-off.


How much do you want to value this intrinsic reward, right?


And here we see our first lever on the exploitation-exploration trade-off.


If an agent gets lots of reward for exploring, right, it might never exploit and exploiting


might actually be a good option in the game that you're in.


So you might want to set beta small, but in other games you might want to encourage exploration


to the max and therefore set beta very high.


Alright, another constant along with that that they modulate is the discount factor,


which is called this gamma here.


So you already see here this beta we've already seen and they also modulate this gamma.


Now what does gamma do?


If I have my state and action we already said, so here is an observation one and I do action


one and that gives me observation two and I do action two and that gives me observation


three and I do action three.


And each time I get a reward, right, an extrinsic reward and an intrinsic reward.


So reward one, reward two, reward three and so on.


Now usually an RL agent will look at these rewards and let's say you are here, you are


at observation one and you're trying to estimate your future rewards.


What will be most important will be the reward that you're getting right now, right, because


that's the most sure because this reward here that you might get two steps from now, you


know, a lot of things could happen, right?


You are pretty sure that if you do action one you're going to get to this state, but


you're not entirely sure you could also get to another state and therefore you had to


do another action and therefore this reward here could be something different.


So these algorithms are having what's known as a discount factor.


That means the value of a state, of a state s is going to be the sum from time zero, let's


say k equals t, that's state at time t, up until some horizon, I think they call it h


in the paper.


You could also think of this as infinity of the reward at step k, but discounted by this


factor and you raise it to the power of k usually or t, t minus, yeah, k minus t.


So basically means that you, this is if t is one, so it's the reward at the, at this


time step plus let's say gamma here is 0.99, right, plus 0.99 the reward at the next time


step plus 0.99 squared the reward of that after that.


And you see that the more, the more into the future you look, the less value these rewards


have.


So little bars here indicate that you're going to value future rewards less and less.


This is called a discount factor right here and it's how to set it is very important because


if you set it very low, let's say you set it to 0.1, that means all that you want to


do is maximize the reward that you're getting in the, like in the next and next, next step.


You're not really looking into the future.


This is very good for games that give you immediate reward for good actions.


But if you, if you set it very high, let's say 0.999, right, that means a reward a hundred


steps from now doesn't, you know, is almost the same to you as a reward one step from


now.


And this is very valuable for games that don't give you a reward immediately or that kind


of trying to trick you as we saw before.


Like if you shoot the meteor now, then you get one reward, but if you don't and pass


on the opportunity, you might get much more later.


So the modulation of the discount factor is also very important to set and really depends


on the game.


So we have two quantities here that really depend on what kind of game it is.


And also they argue it also depends where in the learning process you are.


So if you're at the very beginning of the learning process, you might want to have a


very high goal, the high intrinsic reward to go explore.


And you might want to get, have a very low discount factor in order to learn a good


immediate value function.


But then as time goes on, you might want to bring down the intrinsic reward because now


you really want to actually, because your end goal is to maximize the extrinsic reward


and you want to up this discount factor to look more into the future now that you have


already learned the immediate values very well.


So if I had to summarize and simplify what Agent57 does is it builds a neural network


that adjusts these two quantities across the training, right?


So it adjusts the beta and gamma across the training and it does this in a so-called bandit


setting.


Now there is no real good picture in this paper that I can show you, so I'm just going


to have to draw.


So you have an agent, right, it interacts with this environment here and it always gets


these rewards.


Now what you have here is a meta controller, right?


So the agents, it has two parameters, it has this beta and this gamma and the meta controller


now observes this, it observes this interaction and it outputs values for these two constants


and it does this dynamically as the training progresses, right?


So the agent will kind of learn, the agent will change its behavior over time.


Now this is actually implemented in a slightly different way in that the meta controller


doesn't control the values directly but it has kind of options.


So what you do is you define a bunch of possibilities for beta and gamma.


So you say I have strategy one, strategy one has beta at 0.1 and gamma at 0.9, strategy


two has beta at 0.2 and gamma at 0.8 and so on, right?


And now the meta controller has to choose between one of these, in this case six different


strategies across training.


So it might start off as we said with a high beta which might be over here 0.9, 0.1, it


might start off with a high beta and then transition to the lower ends and it can do


so depending on the game and depending on the progress in the game.


So this is dynamic and this is the improvement over never give up over this other agent because


this other agent simply had these strategies and trained them at the same time and now


this meta controller here controls which strategy is currently trained and which one is used


to generate the experience.


So this is basically, I mean there's a, they also, of course they also say well we also


increase the window of, let me go back, so this LSTM, I've shown you these things here


that incorporate experience over time.


They also say well we increase the window of how long the LSTM, the time window of how


much experience is incorporated and they do a bunch of other things which I always find


kind of annoying because it's always really really hard to see where the improvements


come from that they claim they made.


So but you know barring that basically they built this meta controller to choose the strategies


for the agent over time.


Now of course this meta controller again is trained by the rewards that you get back from


the environment.


So the meta controller as an action has the choice of strategy, right, and the reward


it gets back from the agent environment interaction, right.


So in itself it is a reinforcement learning problem.


Now why, like it to me it seems just shifts the, it just shifts the problem of exploration


exploitation one level higher.


They use a sliding window bandit algorithm to do this but again you have hyper parameters


there like how long is the sliding window and how does the bandit algorithm do the exploration


exploitation trade-off.


So it seems to me you're just shifting it one level higher and it also seems like we're


getting into the region of where we are meta over engineering our approaches to the specifics


of this Atari benchmark because we're kind of observing oh okay these agents do this


wrong, these agents do this wrong, so let's just build an agent that can do both sort


of and then the kind of audastic thing I find that they open with how to measure artificial


general intelligence which I mean come on you're just it's kind of amnest right now


you're just kind of over and over and over fitting on this one benchmark there's not


really a need to make this into a story on artificial general intelligence.


[Interview] Mark Ledwich - Algorithmic Extremism: Examining YouTube's Rabbit Hole of Radicalization

All right, I'm very pleased to have Mark Ledowitch here today.


He's one of the authors of this paper.


It's called Algorithmic Extremism Examining YouTube's Rabbit Hole of Radicalization.


So I've done a video about a topic like this before, actually several,


and this is basically one in a line of research that examines the recommendation algorithm of YouTube specifically,


but also kind of the general social media platforms.


So Mark, thanks for being here.


Could you, maybe for people who do not know anything about this,


could you kind of explain where your work fits into what's been done before


or kind of also what comes out of the mainstream media about this topic?


Because there's been quite a bit of talk.


Yeah, so I'm not a researcher by trade, I'm a programmer,


and the reason why I got into this was because I could see clear bias in the way the YouTube recommendation system's being reported on,


and also in the research.


There's some narratives, I think it might be because there's a lot of people worried about right-wing populism,


and this is a way to explain that.


They're looking for ways YouTube are radicalizing people and finding evidence for that,


but that could be anecdotes or in some of the studies it's sexual quantitative data,


but they're only looking to confirm it, so there's really obvious things I think you covered in your video.


Some of them will just look for movement towards alt-right channels through centrist or alt-light, they call it,


instead of looking for both ways, just really obvious things like that,


calling it an infection that clearly shows they haven't really looked at it like a curious person would.


So I thought I could easily, as a software engineer, just collect all the data,


and without any complicated statistics, just looking at the overall flow of recommendations between videos,


what their political influence is.


Yeah, this was a thing that bugged me of the paper that I made a video about,


is that they claim there's this radicalization pipeline, right?


And with pipeline, everyone sort of understands a different thing,


but I think the general consensus is that the recommendation algorithm itself will steer you towards more extreme content,


and in this case towards the alt-right extremist content.


And the paper actually analyzed and said, okay, we found evidence that there is movement in this direction,


but they never shown that this is significantly more movement than in the other direction.


So in order to justify a pipeline, one would need to show that the movement this way is a lot larger than this way in some notion.


And so I've actually spoken to the author of that paper, and he agrees with that, but obviously doesn't have energy to go into every,


you know, refute everything that comes at them.


They've also been exposed to a lot of criticism, let's say, as have you.


And I think even more when your paper came out, I think for days there was just a giant storm of people attacking your paper, basically.


Just listing every single thing that's wrong with it and why this isn't valid and things like this.


So let's actually jump into what you did specifically.


So if I can summarize and you can then maybe correct so that we can establish what happened.


You basically collected recommendations.


So you scraped these videos on YouTube and you collected these recommendations.


And we can see this.


So in your paper, you then can make such diagrams such as this one or these where in the middle,


the white bar is a channel or a group that you're interested in.


And then to the left, you can see where all the impressions of that channel or group come from.


So what's where basically the views come from through the recommendation system.


And on the right, you can see of all the views the channel has received, where do they go to?


So what is recommended next?


So it basically shows both directions for every group.


And then you've also labeled these by multiple methods so that you can kind of establish these groups.


And what is pretty cool, you've built this website where you can analyze this.


And my computer is a bit overloaded at the moment, but I promise it's really interactive.


All right. So during the interview, my computer crashed.


So I'm doing this in post-production just to show you how this website operates.


So what you have here is an overview over all the channels of what recommendations were collected.


And they're grouped into groups.


For example, here you have the partisan left, center left, social justice, partisan right, and so on.


So you can see for each group or channel where recommendations come from and where they go to.


For example, the large red one here, I happen to know that is Fox News.


You can see Fox News of the daily impression it received from itself, about 36 million impressions,


and it gives to itself 36 million. These numbers have to agree by nature of how the data is collected, of course.


But you can also see it gets 2.7 million impressions from CNN, 2.6 million from the Next News Network, and so on.


And it also gives quite a bit of recommendations to CNN and so on.


So you can go, for example, at some individual channel. Here's the Daily Wire.


The Daily Wire is mainly run by Ben Shapiro, so it's a bit more to the right of Fox News and a bit more on the direction of alternative media.


You can see the Daily Wire gets some, most of its impression count wise from itself, from the Daily Wire, but it gives most of it to Fox News.


So actually, you can see here that itself is a long way down, like in whatever sixth or seventh place.


So actually, if you were to watch the Daily Wire, the recommendation system would most likely steer you towards something like Fox News,


whereas the claim is that the YouTube algorithm would actually steer you towards more radical content.


Actually, in reality, it seems like it would steer you towards more of this mainstream content.


So I actually want to go to this tab. You can see different groupings here.


And the radicalization pathways is the previous paper we have looked at.


So they have all these channels here of this radicalization pathway.


And you can see here, the control group gives very, very, very few impressions to the IDW.


The IDW gives much more impressions to the control group, right?


Again, and the IDW gives very few impressions to the alt-light compared to the amount of impressions the alt-light gives to the IDW and even to the control group.


And if you look at the alt-right, and we're going to zoom in on that here, it's even more so the alt-right, of course, receives most of its impressions from itself,


which you could expect for any kind of group. This is your classic filter bubble situation.


But if we analyze the question of is there a pipeline, you can see that next most likely you are diverted to the IDW and to the control group


much more than you come from the IDW or the control group, right?


Let's look at the alt-light, this is kind of the so-called gateway to the alt-right.


You can see here the alt-light gives most of its impressions next to itself to the control group and the IDW, so de-radicalizing.


If you look at its way to the alt-right, you'll see that it gets about four times as much impressions from the alt-right as it gives to the alt-right.


So basically, it's kind of taking the steam out of a quarter of all of these sessions and gives it to either the control group or the IDW or itself.


So this is exactly the opposite of what you would expect if you were to claim that there is a pipeline.


You would expect their most recommendations to come from more moderate content and go towards more extreme content, but it's exactly the opposite.


And again, these are the exact channels that this original paper used.


Now what this paper finds, the one that we're discussing, if you go to media type here, what you'll be able to see is the division into mainstream media,


YouTube creator, and so-called missing link media, which we'll leave out for a moment.


Let's focus on mainstream versus YouTube creators.


You can see the mainstream media gives most recommendations to itself while giving only very little recommendations to YouTube creators and the missing link media,


while the YouTube creators actually give almost half of their impressions.


Look at that.


They give almost half of their impressions to the mainstream media,


which means that there is a big, big push by the algorithm towards these mainstream media away from YouTube creators.


So in general, and I invite you to look at this website, in general, you can pretty much see that the exact opposite of a radicalization pipeline is happening.


If you, of course, if you look at these recommendations and how they are distributed, actually, most recommendation pathways are towards moderate, centrist content,


and of course, creating filter bubbles, which is a problem by itself, but is not a radicalization pipeline.


Lastly, I want to look at white identitarians, because it's one of the groups that people are referring to when they claim that there are these radicalization pipelines.


So of the white identitarians, they get most of their impressions, of course, from other white identitarian videos, which would be your filter bubble phenomenon.


But they give most, and this is a group, right, the white identitarian channels give most of their recommendations to the partisan right, to the central and left mainstream media, libertarians, and so on,


and themselves are like really, really, really far down.


So to claim that there is this radicalization pipeline, if you look at this data, to me, seems not justified from this data.


And if I look at the other paper that really left out the important analysis of the backwards direction, it seems that given everything, it seems that the claim is not warranted.


All right, back to the interview.


Is that about like what you've done? Is that a good summary of the data collection and analysis?


Yeah, it's a good summary. I can go into detail.


Yeah, please.


So YouTube doesn't make it easy.


So I started this back in November in 2018, and I was using the YouTube API. And to get enough, to get enough quota, because they limit the amount of requests you can actually make to the API.


I created multiple keys, which is against their policy.


And they also asked you delete all your data after 30 days. That's also part of their policy.


So later, I think it was October 2019, they cut off my access because I was doing that.


So I had to move to just scraping the website. So now my collection process actually just loads up the website and gets the recommendations from the actual page like a user would.


And that's difficult because they block access after a couple of hundred requests. They'll stop you, that machine, from actually requesting from the website.


So I need to use a proxy service that's fairly expensive. And what they do is they simulate, well, they have actual residential connections through your home connection like AT&T.


And my requests get tunneled through that, like a variety of locations in the states to get a representative kind of sample.


Cool. So the data collection is, would you say that's the hardest part?


I feel the labeling of channels is also not so easy, but you've managed to kind of do it half automated, also half collecting things from kind of sources that analyze these channels.


But at least for most of the things that I've inspected, I found the labeling to be pretty sane.


I think this is always something you can attack. The original paper was also attacked on how they label.


I find this to be kind of vicarish. Mostly, I think your labels are pretty good as well. The other papers labels are also mostly pretty okay.


Yeah, it's quite subjective. I expected the labeling to be what I get most pushback on. But it turns out it was the anonymous collection.


So what you've actually found here, what would you say are your main results? And I can maybe show.


So you've analyzed a bit where do things come from, where do things go to?


And I found this part here to be one of the, even though it's pretty simple, one of the core things to say about this is that mostly what you found could be said is,


it's simply a recommendation algorithm working as a recommendation algorithm should, which means it creates your typical filter bubbles.


If I watch one makeup tutorial, all of a sudden my site is filled with makeup tutorials and things like this.


But also you found that there is quite some over the top push towards what could be considered mainstream media.


And there is a bit of a draw away from the smaller YouTuber like channels. Is that something that, does that characterize your work?


That's right. Yeah, that's a great way to characterize it. If that chart we're looking at now, if it was a neutral algorithm, the green bars would be the same as the gray ones.


So you'd receive the same amount of recommendations as you give. That would be proportional to the views that you get for your channel organically.


The recommendations you receive be equivalent to that. But we find that it disproportionately recommends mainstream media channels.


That's not even though. So it's not like, it doesn't look like it's consistently doing that. So you can find exceptions to that rule.


I believe one of the main criticisms of your paper has been that you only use data from 2019 onwards.


And I have actually looked at your website and your website a lot of times says that the data has been collected from way earlier than that.


So is it that you've only used 2019 data in your paper? Or what is it?


In the papers just from November and December 2019. And the reason we did that is that we only had 400 channels before that.


And the collection process have changed over time. So this is a clean set of data we could look at. And I thought the most recent was the most relevant.


So what is it doing now? But I've provided, I've got the same analysis over time.


So I've got a gift that I made that everyone can look at, which goes through all the months I've been collecting.


And you can see that chart for where it goes to and has gone through a bunch of changes.


So in about April 2019, that's when they really clamped down on conspiracies and other fringe channels.


Before that, it was much closer to neutral. But it never looked like a rabbit hole. It was never favoring fringe channels.


Yeah, I mean, that has been my experience also personally on YouTube.


I've joined YouTube very early or I've watched YouTube very early when young earth creationism was still active.


And these things were kind of completely discredited by simply having people exposed to other points of view.


And even I find this now, even though YouTube makes it kind of easy to find, let's say niche content,


it also exposes you to a bunch of different views. And I've always found this to be very, very optimistic in the sense of this is probably de-radicalizing much more people than radicalizing.


But you've received, like, as I said, a bunch of criticism.


So if you could, what was the largest criticism, irrespective of whether it was valid or not? What have you found was kind of what most people were criticizing?


Most people criticizing that we were collecting anonymous recommendations. It wasn't the personalized ones.


And it's actually like it is a valid limitation. We had it as a first limitation we talked about in this paper.


And it's still an open question how personalization would affect these aggregate results that we've got.


But I think it's reasonable to assume it will be quite similar once you average it out.


So for any one person, it might be different. But you would expect personalization based on someone's history to even out because it's kind of the algorithms, kind of like the average of all that when it's anonymous.


Yeah, I feel the notion that if because if you're not logged in, the recommendation is like a person with only one video of history.


Right. So it's the same thing, but there's only one point of history instead of multiple.


I find why should why should the behavior be qualitatively different if you have multiple points of history? Like this is a strong claim that you have to you'd have to really show that there is a qualitative difference, not just a more or less accuracy.


And I feel the people making this criticism are it's really on them to show that there is a substantial difference rather than saying that this is a giant limitation of the work.


Yeah, and it's also very hypocritical for a lot of the people saying it, because some of them like Zionette who was mockingly saying that her article, her original article in the New York Times used algo transparency, which is anonymous as well, but she doesn't.


She never looked into that. I think a lot of this is completely motivated reasoning that I don't care about the details.


I've I've seen this one this one Twitter user. She she comment. She said something to the effect of if you've seen this article, please consult someone that works in this space.


Like it's it's please don't don't read article yourself. You must you must get your information through someone.


I've actually I've read the article. I've I find it's pretty straightforward. The limitations are clear, but also the results are pretty clear.


And it's it's actually mostly a boring article. Right. If it I'm sorry, like it's not a criticism. This is good.


Like it's mostly you find that things work as expected. There is a bit of a push towards mainstream, which can be probably explained in that YouTube wants to be advertiser friendly.


Right. And these mainstream channels are already are advertiser friendly, so they probably get bumped a bit.


If what would you say is maybe the most the most valid criticism that you've heard?


Maybe not the biggest, but the most where you where you say, yeah, this is really this is really something that is, you know.


I think I guess what's there was criticism that I'm overclaiming not in the paper so much, but in my tweets and medium.


I guess that's that's fair. But I guess when I tweet and write a medium, those are what I believe in kind of a Bayesian way.


I'm not catching my claims that you would when you're writing a paper. So I guess that's valid.


But I think a lot of people read into what I was saying more than what I was.


So when I say the algorithm has a de-radicalizing influence, I'm just talking about the recommendations, whereas a lot of people consider that to be talking about all things considered.


So even if it isn't doesn't have a bias towards a fringe, maybe sociologically YouTube radicalizes people.


It could be the case. I don't know. But that's what I'm talking about.


I'm talking about just the influence through recommendations. And that's what we can hold Google accountable for.


Or at least it's what we probably all could agree that Google should be held accountable for with its recommendation system.


Yeah. Do you do you expect something to come or have you heard something to come out of YouTube themselves like the company?


Any form of official statement to this? Nothing. Nothing at all.


The only I got a vague a reporter was complaining that YouTube sent them this.


So I think they've read it. But I have no absolutely no contact with them.


OK, cool. Are you doing any anything in follow up or do you have plans for more research?


None of this. I've just gone back to work. I've applied a bunch for a bunch of independent grant money, but I'm not optimistic.


So if I don't get that, I'll keep I'll keep it pattering along. I'll probably reduce the amount of recommendations because I'm spending about five hundred dollars a month at the moment just keeping it running.


So I've got to reduce my costs. Yeah. And you do have a Patreon for people to to chip into that, right?


Yeah. So if you can link to that, that'd be good. So if I'm getting something like like twenty two dollars a month, it doesn't really cover it.


Yeah. All right. So, OK, this has been very, very pleasant. I think we've we've kind of looked at a lot of things.


Is there anything you would like to amend to this that people should know about the research or about this this field?


I just have a I encourage you to have a play digging into data yourself. This if you're in this area, the data is free to use, the code is free to use.


Just consider this a contribution to knowledge. Cool.


Well, thanks a lot, Mark. I wish you a very pleasant evening for you, I guess. And cheers.


Neural Networks from Scratch (NNFS) in Print!

It's here the neural networks from scratch in Python from myself and Daniel Kukiela


It is over 650 pages of pure Python goodness where we essentially are creating our very own deep learning framework in Python


Really with the purpose of learning truly how neural networks actually work


The idea here is not so you can make your own deep learning framework in Python and then use that going forward


Instead it is just simply to get an understanding of how deep learning works so that when you use a framework


You actually know what are these parameters that I'm changing here, what do they actually mean


And when things don't work what might you consider changing to get things to actually begin to work


So the book comes in two physical forms, you've got the hardcover version here that you see


We also have a softcover version if that is what you are after


And then finally we have the ebook version, if you get any version, any physical version of the book you also get the ebook version


But you can also buy the ebook all by itself


The ebook also comes with the ability for you to ask questions or make comments in line with the text of the book


So if you are having problems anywhere along the way you can do that


And I think that is probably the best way to get help with the content of the book


The hardcover and softcover version of the book both come in full color


Which is very useful both for the many charts and diagrams that we have in the book


Also all charts and diagrams or at least most of them wind up having also QR codes which you can scan with your camera on your phone


Most relatively recent phones the camera app will automatically recognize that and allow you to go there


Otherwise you can also download apps from the app store which will use your camera to read QR codes


But these will link to actual videos of animations and so on


So if you are having a hard time understanding or you just want a more full understanding of what we are trying to show


We do have those all through the book, also you can just go directly to those URLs


Now beyond having color for the diagrams and charts we also have color because this is a programming book


And to make it easier to read the actual syntax of the Python language that is why we have syntax highlighting in basically every editor that you might use


So the same thing is true here we have full syntax color highlighting for all of the Python code


Which for one just reading it and understanding it that makes it a whole lot easier than if it had been black and white


And then also it gives us the ability so for example as we make reference to various Python objects


So for example as you are reading this text you get to isinstance what are we talking about


You can find it so much quicker because of the syntax highlighting


So anytime we are referencing Python object we try to use the actual syntax highlighting that would be in reference to that object


Just so it is even that much easier to go back to the code and find it


So now coming to the table of contents for the actual book and what it covers


We obviously have a brief introduction to neural networks what are we actually trying to do here


And very quickly we are getting into coding our very first neurons


We begin to code a layer of neurons and have multiple layers of neurons


Then we get into activation functions not only how do we code them but what are activation functions why do we actually use them


And going over this both conceptually and via code to show how that actually works calculating network error with loss


Again showing all of the mathematics and then the Python implementation of that math


Then we get into optimization which really is the challenge the hard part of neural networks


So first we just kind of introduce the concept of optimization and really why is this so hard


And essentially we are attempting to optimize a very large function with millions of parameters


So that turns out to be quite the hard problem


So then we get into how this is actually done with derivatives more specifically partial derivatives with these huge functions


Then how do we back propagate those things and at least initially the concept of back propagation


At its most basic form is going to be using the stochastic gradient descent optimizer


But there are other optimizers including variations to vanilla stochastic gradient descent


So specifically with something like momentum but then we also get into AdaGrad, RMS Prop and of course the Adam optimizer


After we get through the optimizers we talk about testing out of sample data, validation data, training data


Beyond that we get into regularization so if the network is having a hard time learning


A lot of times regularization techniques can significantly help with generalization


We also kind of up to this point we are doing classification specifically with multi more than two classes


But we also cover binary logistic regression and then also basically up to this point


It's not really a framework until we get to the model object and this is kind of the point where we kind of convert everything


To be very very object oriented programming and pretty much at this point I would say we have a deep learning framework


We cover a realistic data set because up to this point we use a kind of a generated data set


Just to make learning easier, testing quicker and so on. Realistic data sets are usually quite large


We use neural networks with very vast data sets rather than small data sets


We would use different forms of machine learning if the data set in most cases


If the data set isn't very big but at some point we actually want to learn okay we've got a realistic data set


How do we break this data down to become input into a neural network


Then we go through model evaluation as well as saving and loading models so if you have a model that you really like


You want to save it or during training if any point you need to pause and come back later that is how you can do it


Or if you want to save checkpoints along the way you need to be able to save and then if you want to use your model


You need to be able to load the model then we cover prediction and inference and then of course at some point the book must end


So the book ends at a little over 660 pages


So there you have it the Neural Networks from Scratch book now available thank you to all of our pre-orders


And especially the Kickstarter backers who back the book really before there was almost any content available for you guys even to read


So I really appreciate that I appreciate everybody who has taken part in reading the draft document asking all your questions


You might not think the asking a question is actually helpful to us but it really really was


I said it before I'll say it again I think this book is already in like a V2 situation just simply because of


How many people have been through this book how many questions have been asked


All the edits and changes that have been made we have changed so much of the content in this book


In an effort to make it more clear more understandable based on even just the questions people asked


Also all the people who issued edits either grammar mistakes or actual just things got flipped or just outright mistakes that were made


In this much content it's very hard for you know two authors to catch every single mistake


So thank you to everyone who has contributed both in editing as well as just asking questions


Also again thank you to Thomas Halverson and Spencer Churchill who heavily edited the actual text and grammar and English of the book


So anyway that's it the book is now available you can get it at nnfs.io


Again shipping can take anywhere from one to three weeks if not possibly even more


We don't really have too many data points at this point so let us know how long did it take for you to get your book and where you live


Also when you do get your book please share images of your book with us on Twitter or via email or whatever


We want to see your books one for quality control purposes but also two it just put a big smile on both myself and Daniel's face to see these books


So please do share it with us and let us know what you think


Anyway that's all for now thank you again everyone for your support in this project


And I will see you guys in the Neural Networks from Scratch videos which we're going to be starting up very soon


For sure in the month of November we should start seeing some releases of new videos so stay tuned for those


Until next time


Introduction - Deep Learning and Neural Networks with Python and Pytorch p.1

What's going on everybody and welcome to a deep learning with Python and of course


Pytorch tutorial series. This tutorial series is going to start from the very


very basics assuming maybe you don't know anything about deep learning and


then we'll be getting into more complex types of neural networks and


applications for them. So anyways to get started of course you're going to need


Pytorch, you're also going to need Python. I am assuming you already have at least


Python and my hope is that you at least understand the basics of Python as well


as object-oriented programming. So if you don't have either of those two things


probably you should stop and either find someone else's tutorials or I've got


tutorials on both the basics it's a pretty short series it's like 13 parts


or something like that and then also object-oriented programming again it's


not that many videos should be able to get through it pretty quickly. You should


have a basic understanding of Python because I'm just assuming you do I'm not


going to be talking about basic Python concepts and then object-oriented


programming because in general your neural network is going to be a class so


if you don't understand that you're going to be fuzzy about certain things


and then you're also probably fuzzy about neural networks it's just not a


good combination you should be solid on how object-oriented programming works


first and then jump into this. Next up I'm assuming you guys don't at least


some of you guys probably don't know about neural networks or at least don't


know much so I'm going to go really really briefly into how neural networks


work you you can get by here with a pretty high-level understanding of


neural networks and be just fine and then you could if you want to be very


successful you're going to need a pretty deep understanding at least for this


series we're not going to get too deep too quickly because again you don't


really need that but if you're curious or you want to get into more cutting-edge


research and stuff like that you're going to need to so as this series


progresses will be getting a little deeper into things but then also and


we'll be talking about concepts I'm just not going to hit immediately because


there's just no need and then also at some point I'm going to do a neural


networks from scratch video and by scratch I mean we're going to use NumPy


we need something to do like array math or something like that but no helper


functions just because the list would be really slow anyways not important but if


you guys want to get a little deeper at you will be able to both in this series


and in the future anyways assuming you don't know anything about neural


networks I'm going to go briefly over them but then again if you're still kind


of fuzzy about just even sort of how neural networks work you can look up


videos by three blue one brown that there's some great visuals there also


there's just so much information online as far as like how neural networks work


at a basic level but I'll do my best to explain them really quickly so this is a


basic neural network you've got your input data here and let's assume you've


got an initial a situation where you're trying you've got images of dogs cats


and humans and you want a neural network that can differentiate between an image


of dogs cats and humans so over here in your input you've got features and these


are going to have to be numerical valued in some way it might be the case that


they're pixel values and pixel values are already numerical values right zero


to 255 for RGB or if it's grayscale image it's just going to be zero to 255


or some other number it's usually zero to 255 anyways so they'll already be in


numerical form but your features could also be so the input we call that


features this could also be some sort of categorical type of feature so it's like


has four legs has two legs it could be has body of covered in hair or something


like that or what kind of color how tall is this thing that kind of stuff is


going to be like descriptive features and at some point we have to convert


that to numerical so you could just do like a like just simply the first thing


is a zero the second thing is a one and so on and we'll talk much more about


that and how to how to convert things to numbers later but just understand it's


always going to be a number that is being input it has to be a number so


anyways that's your input so let's assume their pixel values because that's


a pretty common task so and it's grayscale let's say so you have this


input and then it gets passed to these things called hidden layers they're


called hidden because we don't have control over those layers the machine


has control over those layers we can read them we can see what's going on but


we're not changing any values there after they come through the hidden


layers they get passed to an output and again in general I'm just going to be


super super vague or not vague but hopefully not vague super general here


let's just say this neuron this is your dog neuron this is your cat's neuron


this is your human neuron and the value that winds up here in general we do an


argmax function that's just going to ask which of these is the greatest so dogs


cats humans let's say dogs is five cats is seven humans is twelve right so that


would mean the largest value is humans so we would say this neural network has


predicted it's a human okay so how does the machine has a neural network even


learn it's not complicated at all we can look at really complicated imagery like


this it's you don't even really have to think of it this way but basically each


of these connections you can see how there's a line drawn from every input to


every neuron here and then every neuron here has a connection to every other one


right as you continue going forward here this is what's referred to as a fully


connected neural network they don't have to be fully connected and we can talk


more about how to do different things later on but in in general this one's


fully connected and each line here each connection is really a weight so you've


got this input value and then it gets multiplied by a weight a bias is


optionally added in and a bias is just an extra you know bias could be three so


it's you know the input times the weight plus three okay and then it becomes a


value in here and then X to the same thing is done and it's in all these are


added together into this one neuron so I've got another image here and like


this is basically what goes on in a single neuron so you've got all your


inputs coming in those inputs are being multiplied by a weight a bias is


possibly added all those are added together so they're just summed together


and they're passed through an activation function so the activation function kind


of mimics a neuron in your brain right is it firing or isn't it and it could be


a firing or isn't it so it'd be like a stepper function but in most cases we


use a sigmoid function so it's actually a range between zero and one and that


again is just kind of to mimic both a neuron firing or not and also it serves


a purpose of keeping values from just exploding in your neural network it just


kind of keeps things again between zero and one one of the key concepts that


just keep coming up time and time again is that in general neural networks


appreciate values between zero and one or negative one and one in some cases so


always keep that in mind so your input usually you're going to do what's called


scaling and scale it down to be between zero and one and then yeah even through


your network when this value start exploding into these gigantic values


something's probably going wrong but way more on that later the one thing all you


have to really think about is what a neural network does all it does is each


of these weights is what we call a parameter so same with the biases so


weights and bias even this really really simple neural network that I've just


hand-drawn here look at all of those parameters every single line is is a


parameter probably two parameters if we consider weights and biases so what the


machine does is it has it has the ability to modify every single one of


those weights independently right so it just keeps tweaking all of those so that


when we pass data when we train a neural network we're passing in this input data


and we're telling it let's say it really is a picture of a human we would say the


desired output the output I wish you would make in this case is a zero zero


one right please make that and it makes its output and then we determine how


wrong that is with loss and then we have an optimizer that goes back and attempts


to update these weights in such a way that it gets closer to predicting the


targeted output and we tend to do this over batches over millions of samples


and slowly over time the neural network learns what exact values to put for each


of these weights to hopefully generalize over your data and predict but at the


end of the day it just it's like this gigantic function with like a million


variables sometimes more like you like I'm not even being I'm not exaggerating


like three million variables is a pretty small neural network 30 million


variables is a regular neural network so it's just so many things that it gets to


tweak and change and of course with that comes things issues like overfitting


stuff like that we will talk about that moving forward but really all in our


network is is this huge function where we allow the machine to slowly tweak


variables and parameters to output desired results ok so now hopefully you


have a general and basic understanding of how neural networks work again I think


it's completely to be expected that you're somewhat vague on how things work


I think one writing it out in code would help but also to there are just vast


resources online as far as depicting how neural networks work and if that's how


you learn I would strongly recommend the three blue one brown just type of like


to YouTube three blue one brown neural network and I am sure you'll find the


series he is wonderful at visualizations way better than my hands are on photos so


anyways moving along we're gonna be using Pytorch now chances are you've


heard of for sure tensorflow which is another machine learning library and


there's a quite a few others I want to use Pytorch one I just start I just


decided one day to learn Pytorch myself and as I was learning it I was like hmm


this would be a really good language to do a you know beginners deep learning


series because Pytorch is like super friendly to the programmer especially a


Python programmer as opposed to tensorflow and in one way it is is with


the graph and the great thing about Pytorch is you don't even need to know


what the heck is the graph what is he talking about you don't need to know


with Pytorch it's great with tensorflow you have to you have to learn about the


graph because you have to work around the graph it's very challenging whereas


in Pytorch you really you just don't need to know about it and so and then


also people call it like more pythonic and like what does that actually mean in


general like with Pytorch you write Python programming like you write the


object you deal with it like it's an object you you it's eagerly executed and


there is tensorflow eager but it's not the main tensorflow and there are


certain issues and limitations with using it also is much slower than


regular tensorflow and and then the next question of course is how does Pytorch


compare to tensorflow and speed and they're pretty close actually but


Pytorch is faster than eager tensorflow and that would be the more fair


comparison because Pytorch is eager and what do I mean by that I mean you can


write a line of code in Pytorch like you can have your network and then you run


like one line or you run through one layer and then you could check the


output you can see it or also with Pytorch you could you can branch out


into different layers and you write pure Python code to do it it's just it's just


easy to make complex networks there are some tedious things about Pytorch and


we'll talk about them when they when we get there but anyways long story short


we're gonna use Pytorch and you're gonna love it so let's click on get started


there's like a million ways you can install this so I'm gonna let you


install it by yourself all this stuff should be super simple for you to


understand except for maybe CUDA if you don't know what CUDA is go over here


click on none if you have a NVIDIA GPU of decent quality you could enable you


could get the CUDA version and put operations on your GPU and when you do


that it's a it's in general somewhere between 50 and a hundred times faster


which makes a big difference in training times to begin this series at least for


the first few videos all the code that I'm gonna write is one I'm gonna run it


I'm gonna be running it on my CPU and it's like totally capable of being run


on a CPU at some point you are going to need access to a mid-range or better GPU


either locally or in the cloud now if you have access to one locally you can


install it we'll be talking about how to do that and then in the cloud there are


lots of options this series is sponsored by Linode who also happens to have the


best prices in terms of bang for your buck performance in the cloud so you can


check them out I've done a video on exactly how to set everything up on them


as well I'm not saying that because they're a sponsor they're a sponsor


because they are simply the best right now anyways right like over time all


these all these cloud providers are duking it out so at some point someone


might beat them on price and I'll let you know when that happens till then


Linode has the best cloud GPU prices by far it's like twice as good as anyone


else right now so anyway check them out but like I said if if you don't know


what CUDA is don't worry about it right now just get the regular CPU version


you're gonna be totally fine you can learn the basics without needing to run


on a GPU but at some point you will need to run on a GPU so anyway once you have


that go ahead and do your install I am going to pause for a moment and assume


you guys when I when I'm done pausing I'm going to assume you have PyTorch


installed great congratulations on your install so the first thing I'm gonna do


is figure out dude I have it oh I do already opened it up nice so I'm gonna


be coding things in the Jupiter lab but you can use whatever editor you want


because remember you guys have done you're at least not basic programmers


anymore so I'm assuming you guys can have your own editor and be fine I'm


going to use Jupiter notebook just because one I can yay for eager


execution but also because it just kind of makes sense at least when you're


trying to learn the very very basics running line by line is just useful also


if you needed to bug things again it's just it's super useful so I'll be


starting in a Jupiter notebook at some point we'll probably leave the notebooks


behind but for now cool so first of all like what is PyTorch right like what is


that so again assuming you know the basics of


Python you're probably familiar with NumPy which is the the numerical


processing library of Python now the problem with NumPy is what I just


explained a moment ago is that it doesn't run on the GPU so whenever deep


learning first came out there was really it was very difficult to to really run


these things because the your CPU the reason why things are so much faster on


your GPU is with a neural network like I said the the thing that is happening is


each of these weights is getting updated so your machine is needing to run


thousands and thousands of operation of small cat like not even that like


millions of small calculations basically and so so the GPU trying to think exactly


want to describe this one order your let's say your CPU so your CPU was made


to do these not really huge it's not like these huge calculations but the CPU


is really there under the assumption that most calculations that are going to


be made are going to be made are going to be large calculations like very much


more challenging calculations than modifying little weights okay your GPU


is your graphics card right so it's computing graphics and graphics there's


a lot of very small calculations so we run on the GPU for that reason so so


running on the GPU is where we want to be for something like deep learning


because what we're trying to do is a lot of simple calculations in your Jeep like


your CPU probably has something between I don't know four and twelve cores okay a


nice GPU has like thousands of cores so it's just it makes a big difference so


let me come over here and so so like I said pi torch is just numpy on the GPU


with some nice helper functions because a lot like like you don't have to write


your optimizer and your back propagation and all these things that probably might


not mean anything to you yet but we'll soon but in its basic core let's go


ahead and import torch and can I probably should have figured this one


out first wouldn't have minded making my font size larger it's got to be a way


besides zooming in but I'll just zoom in for now so import torch and then let's


say you've got two two variables so we're gonna say X equals torch dot


tensor and I mean this could go many ways but let's say like a five and a


three and then you've got Y equals torch dot tensor and we can make this a two


and a one we could then say print X times Y it's gonna take a second to


import torch there but normally that operation would be much faster but yeah


we just we just multiplied these two arrays so a tensor is initially kind of


a scary sounding word but basically a tensor is an array like that's all you


need to think of right and so and it basically a multi-dimensional array so


yes so that's as simple as it gets right and what's nice about torch is we can


put this on the GPU and this could also be on the GPU and then when we multiply


them together the GPU is doing that math but again what just happened just now


was actually on the CPU we weren't really asking very many calculations to


run so it was like no big deal to actually run that the other thing that


we can do is like we can say X equals torch dot zeros and then we can specify


shape let's say say a two by five and let's go ahead and well one thing we can


say is what we could print X then we could also say X dot shape and that


gives you like the size and and before anybody's like it's just really


necessary why are you showing me all these things basically all these things


are things that we are going to use but also if you're already familiar with


numpy one thing you might be starting to realize is hey they've got zeros in


shape and all these calculations that you tend to need to do or you've done in


numpy there's a torch variant so again we could say Y equals torch not rand and


then we could say two five and then we can output Y and there you have just a


random initialization of a two by five tensor so the next thing I want to show


is reshaping so the one thing that for whatever reason they have decided to


call something that you are not familiar with is like when you want to actually


reshape something in numpy you would say dot reshape and I believe in tensor flow


it would be a dot reshape in torch it's not a reshape it's called you use a


function called or really a method called view so this is a two by five so


let's say we want to do what's called a flatten operation so a two by five in


general you're not going to be able to put a two by five into a neural network


so let's say it's an image and the image is two by five pixels it's not going to


be the case but let's just say it was the first thing you have to do before


you feed this image through let's say a basic neural network is you have to


flatten it so it would be a one by ten right because there's ten total elements


if there's two by five if we flatten it it would be a one by ten well how do we


do that though so the way we would do that is with a reshape and like numpy


but here in torch it's not view but one thing to keep in mind I'll show in a


moment we would say one by ten right so there's your one by ten reshape there


one thing to keep in mind is if we print out Y again you'll see oh it hasn't


actually been modified so keep in mind that you would need to reassign so you


would need to say Y equals Y dot view and I guess you know the view sort of


makes sense like it's like show me this view show me why viewed as the shape or


something like I I don't really mind the name but it is different from what we're


used to in terms of reshape but it's honestly not that bad people made a big


deal about that whenever I was reading about updates to to PyTorch people are


complaining about that I'm like really I mean views a good name so anyway okay so


so as you can see it's just doing simple math with arrays basically now again


there are lots of little functions that we can also import to help us


specifically with neural networks but at its core it really is just a library to


help you do array math so anyways I think I'm gonna cut it here and then in


the next tutorial we're gonna start talking about the actual data itself and


then probably that'll consume a tutorial and then we'll talk about building the


neural network and then training the neural network so as strange as it might


sound we're gonna dedicate the entire next video to data alone and even then


it's pretty simple but if you think about it as a machine you know as


somebody who's doing machine learning like your biggest job is like the one


thing you have control over is data and then you know you can you sort of have


control like in other ways like you can change the shapes and sizes of your


neural network you get to dictate what the structure is but in general the


biggest thing for you to do is provide good data it's kind of like with a lot


of things like audio or something where you say garbage in garbage out if you


put in bad data you're probably gonna get bad data out so anyways in the next


tutorial we're gonna discuss data and we're gonna go easy mode on the data


first and then on the next neural network we build will get a little more


complicated but anyways we've we've covered a lot of basic stuff here but


we've covered a lot of stuff if you have questions comments concerns if I said


something wrong feel free to correct me you can leave those below also if a


discord that's discord gg slash syntax if you've got any questions you need


help on something come check out the discord also special shout out to my


channel members who have been with me now for a year


Harsoth angel font Alvo mr. gene genes and Edward McCain thank you guys so much


for your support you guys are amazing if you want to support the channel you can


click on that beautiful blue join button get early access to content shout outs


special rank in the discord that's pretty much covers it I think anyway


Github Copilot: Good or Bad?

What is going on everybody and welcome to a video on github copilot, so there's been a lot of hype and


Github Copilot: Good or Bad?

dangers expressed with copilot


Github Copilot: Good or Bad?

And it's been hard for me because I haven't had access until a few weeks ago


Github Copilot: Good or Bad?

So I finally have had some time to play around with copilot


Github Copilot: Good or Bad?

And I thought I would share my thoughts with you all so if you're been living under a rock


Github Copilot: Good or Bad?

And you don't know what copilot is it's just an in editor extension that makes suggestions to you as you code


Github Copilot: Good or Bad?

Basically it tries to finish your lines or even suggest entire blocks of code


Github Copilot: Good or Bad?

So it's an interesting concept and obviously this spawns all kinds of issues especially because it was trained off of github data


Github Copilot: Good or Bad?

And so there's like privacy concerns security concerns


Github Copilot: Good or Bad?

And then people are worried that their jobs are going to be taking away, so so first off. How does it work?


Github Copilot: Good or Bad?

Does it work well, and then what about all these other little issues with something like this first off copilot was created using


Github Copilot: Good or Bad?

OpenAI's codecs model which is a variant of gpd3


Github Copilot: Good or Bad?

But codecs differs in three what I would call major ways to illustrate things here


Github Copilot: Good or Bad?

We're actually going to use copilot so first off. It's smaller


Github Copilot: Good or Bad?

Gpd3 is a hundred and seventy five billion parameters whereas codecs is 12 billion parameters


Github Copilot: Good or Bad?

This smaller model makes it a bit quicker to run or at least easier to run quickly as well as makes it easier to scale


Github Copilot: Good or Bad?

Out and here you can see just how quickly we made a bar chart with


Github Copilot: Good or Bad?

Copilot to compare these two sizes each time that you see this great text


Github Copilot: Good or Bad?

That's the suggestion from copilot, and I just hit tab to accept them interestingly it even gave it a title and labels the


Github Copilot: Good or Bad?

Y-Label is not quite accurate


Github Copilot: Good or Bad?

We would want to change that but it would be very easy for us to just edit that in and still be way quicker than


Github Copilot: Good or Bad?

If I tried to code this all by hand myself


Github Copilot: Good or Bad?

So gpd3 is indeed far larger and can arguably do a lot more


Github Copilot: Good or Bad?

But it also has a knowledge base of basically everything gpd3 is general purpose it includes concepts of coding


Github Copilot: Good or Bad?

But also law medicine conversation and a whole lot more


Github Copilot: Good or Bad?

Codecs is only concerned with coding so it's much smaller piece of the total pie and a much smaller model can actually be better at


Github Copilot: Good or Bad?

coding than full-scale gpd3


Github Copilot: Good or Bad?

There are also a lot of really little subtle things that you wouldn't expect to copilot to do


Github Copilot: Good or Bad?

But it does so in case you missed it as we adjust to these percentages the suggested percentage that comes next is


Github Copilot: Good or Bad?

Mathematically accurate as a subtraction from a total of 100% so not only does it have to understand that you would max out at


Github Copilot: Good or Bad?

100% it's doing the math which is really cool the final way that codecs differs from gpd3 is


Github Copilot: Good or Bad?

codecs and copilot have a giant context length of


Github Copilot: Good or Bad?

understanding essentially it's a question of how large can your file be before codecs can't


Github Copilot: Good or Bad?

Ingest everything in your file so some of the earlier and further away code might be ignored when generating suggestions


Github Copilot: Good or Bad?

The context size of gpd3 is 4 kilobytes whereas the context size of codecs or


Github Copilot: Good or Bad?

Github's copilot is 14 kilobytes so then the next question you probably have is okay


Github Copilot: Good or Bad?

Well how much code is 14 kilobytes well? I've got a directory with a bunch of Python files from github repositories


Github Copilot: Good or Bad?

So let's just see if we can find one that's about 14 kilobytes and using github copilot


Github Copilot: Good or Bad?

We did find a file and we see that 14 kilobytes is about 400 lines or so which is considerably large


Github Copilot: Good or Bad?

Obviously sometimes your files will be bigger, and it's not like copilot just won't work


Github Copilot: Good or Bad?

It's just that it won't be able to take your entire file into context next


Github Copilot: Good or Bad?

Let's cover an example of some regular expressions


Github Copilot: Good or Bad?

So say we've got a sentence and we want the regular expression to find the price of something in there


Github Copilot: Good or Bad?

We'll let copilot write a regular expression as well as print it out for us


Github Copilot: Good or Bad?

But before we run this let me show you something really cool


Github Copilot: Good or Bad?

What if we write a comment that begins to say what the output will be or should be?


Github Copilot: Good or Bad?

But we let copilot finish the comment


Github Copilot: Good or Bad?

It turns out that copilot knows exactly what the output will be here


Github Copilot: Good or Bad?

This is my own made-up example. It exists nowhere certainly not in the data set it was trained on


Github Copilot: Good or Bad?

This is an example of transformers in their understanding that I've used in the past, but usually transformers aren't too accurate here


Github Copilot: Good or Bad?

I can only sometimes get it to get this right


Github Copilot: Good or Bad?

Copilot gets this right every time I couldn't trip it up


Github Copilot: Good or Bad?

I'm sure it's possible to trip it up


Github Copilot: Good or Bad?

But asking copilot to tell me the expected output at least for me has just plain worked and


Github Copilot: Good or Bad?

This signals a deeper understanding than just simply pattern matching


Github Copilot: Good or Bad?

Notice too that it gets the format right it's going to come in the form of a list


Github Copilot: Good or Bad?

We can change our requested regex to include the dollar sign too and again


Github Copilot: Good or Bad?

We can ask copilot to tell us what's the output going to be now. It's correct


Github Copilot: Good or Bad?

What if we add another dollar amount here again the same thing copilot not only wrote our regex


Github Copilot: Good or Bad?

It actually knows what the output of the regex will be


Github Copilot: Good or Bad?

Now we could keep going here, and this doesn't necessarily have any super important direct use to us


Github Copilot: Good or Bad?

But I hope it drives home that there's more going on here than just simply pattern matching and sort of regurgitating


Github Copilot: Good or Bad?

Things that seen before there's a deeper understanding


Github Copilot: Good or Bad?

This doesn't necessarily have any direct use to us, but it's really cool


Github Copilot: Good or Bad?

Let's do something even more absurd. I'm going to import a non-existent package called sentiment analysis


Github Copilot: Good or Bad?

I'll create an example sentence, and we can keep the suggested string to for now later. We'll try something else


Github Copilot: Good or Bad?

Next I will pretend to this made-up module again has a made-up method


Github Copilot: Good or Bad?

I'm literally just making all of this up right now


Github Copilot: Good or Bad?

Anyway, we're going to say that method is called analyze


Github Copilot: Good or Bad?

Now rather than running. Let's just do the same thing let copilot predict the output


Github Copilot: Good or Bad?

And yeah, that's insane


Github Copilot: Good or Bad?

Okay, so let's go ahead and try our own sentence next not one that copilot came up with for us


Github Copilot: Good or Bad?

This time negative 1.0 that makes sense much of the time. I think you're going to get returned something between a negative one and a positive one


Github Copilot: Good or Bad?

But you might also get a string positive or a string of negative something like that


Github Copilot: Good or Bad?

Trying a few more examples you can see it gets it right every single time and again


Github Copilot: Good or Bad?

There is no sentiment analysis module here. There might be a module called sentiment analysis, but


Github Copilot: Good or Bad?

What's important is it doesn't exist, and I don't think it has an analyze method


Github Copilot: Good or Bad?

It's just copilot, and this is so crazy, and I really want to stress these were all unplanned examples


Github Copilot: Good or Bad?

I just tried them and they worked same thing with the regular expressions examples


Github Copilot: Good or Bad?

I'm really not cherry-picking here even though the performance looks like I'm cherry-picking


Github Copilot: Good or Bad?

Copilot is simply that good and it really is


Github Copilot: Good or Bad?

Staggering to me, and if you have copilot. I really encourage you to try like more


Github Copilot: Good or Bad?

Kind of out there examples like this because it's it's actually really surprising what copilot can


Github Copilot: Good or Bad?

Can actually do so now for some of the problems people have brought up the first one being it's going to steal our jobs and


Github Copilot: Good or Bad?

I don't think that's true. I think eventually you might have an AI that a normal person could


Github Copilot: Good or Bad?

Dictate their wants and needs to and the AI could plausibly code that


Github Copilot: Good or Bad?

But the end of the day, it's just like programming languages or just that their languages


Github Copilot: Good or Bad?

It's our way of explaining to a machine what we want, so I'm not really sure that


Github Copilot: Good or Bad?

Having an AI that you can explain what you want


Github Copilot: Good or Bad?

And it produces it is that far off of programming the leap the difference of using something like copilot


Github Copilot: Good or Bad?

It's almost like an extra layer of abstraction not all that different from


Github Copilot: Good or Bad?

using Python over using something like C++ and a


Github Copilot: Good or Bad?

Perfect example of this and what I would argue is a productivity boost which is Python


Github Copilot: Good or Bad?

I think the same thing is going to be true with stuff like github copilot where


Github Copilot: Good or Bad?

You're able to be far more productive and when you're more productive


Github Copilot: Good or Bad?

There's more of a reason for someone to hire you so it's not really the case that


Github Copilot: Good or Bad?

We'll have less job openings if anything it would be more job openings even if you do remedial stuff as a programmer


Github Copilot: Good or Bad?

This will make you more productive more valuable, so I'm not I'm just not sure I'm buying that it's going to steal anybody's jobs


Github Copilot: Good or Bad?

But you also kind of need to have some sort of contextual understanding at least under the current version of copilot


Github Copilot: Good or Bad?

It again. It's it's totally possible in the future that


Github Copilot: Good or Bad?

You won't and anyone could just dictate to something like copilot. Hey make XYZ for me


Github Copilot: Good or Bad?

But I think you're still going to need people that will still architect


Github Copilot: Good or Bad?

overarching things


Github Copilot: Good or Bad?

It's possible that an AI could do that, but I at least to me


Github Copilot: Good or Bad?

I see this as really more of a layer of abstraction and a productivity boost which


Github Copilot: Good or Bad?

most likely


Github Copilot: Good or Bad?

Is going to be a lot more


Github Copilot: Good or Bad?

And a productivity boost which most likely would just create more jobs more people could afford to hire you because you're far more productive


Github Copilot: Good or Bad?

So I don't know I that's at least the way that I see it


Github Copilot: Good or Bad?

I know people will disagree with with that take, but that's that's kind of how I'm seeing it


Github Copilot: Good or Bad?

Also, I've seen at least for me personally like copilot has taught me things


Github Copilot: Good or Bad?

I've thought about libraries that I haven't had the chance to really look into using and I just kind of started writing code and said


Github Copilot: Good or Bad?

Hey, I want to do this thing with that library


Github Copilot: Good or Bad?

And it just kind of guided me along the way to teach me how to use that library


Github Copilot: Good or Bad?

And I found that to be really really cool


Github Copilot: Good or Bad?

there were also times where I wanted to see if


Github Copilot: Good or Bad?

Copilot could do something that I was familiar with and then it ended up doing it in a different way that I had never really


Github Copilot: Good or Bad?

Thought about or even using a method that I didn't even know existed


Github Copilot: Good or Bad?

So so it's taught me stuff, and I and I also think that's that's really cool


Github Copilot: Good or Bad?

So I also think that copilot does make mistakes right it's got logical errors that it makes it makes


Github Copilot: Good or Bad?

Errors and what it thinks you're going after, but you're not really there's a skill involved in how to prompt


Github Copilot: Good or Bad?

Copilot to give you what you want


Github Copilot: Good or Bad?

So not all that unlike using a search engine for example, so


Github Copilot: Good or Bad?

I think that overall I was personally very very impressed with with what copilot could do for me


Github Copilot: Good or Bad?

I kind of came into this expecting to not like it. I tend to like very simple editors. I don't like autocomplete stuff


Github Copilot: Good or Bad?

So for me. I was almost biased to not like copilot, but I I really enjoy it


Github Copilot: Good or Bad?

I think it's a it's a really cool innovation


Github Copilot: Good or Bad?

Both in terms of how they've applied a transformer model and built it into an editor how fast it is


Github Copilot: Good or Bad?

This is a 12 billion parameter model and it inferences very quickly


Github Copilot: Good or Bad?

especially given the context size, so


Github Copilot: Good or Bad?

Yeah, I think it's a really well integrated thing in the technology and the actual accuracy of what it's actually producing


Github Copilot: Good or Bad?

I find it to be far more useful than not like I said it does sometimes produce silly things


Github Copilot: Good or Bad?

But for the most part it is very helpful very quick and again


Github Copilot: Good or Bad?

I tend to find it most useful in the more remedial task type stuff that I need to do while I'm programming and again


Github Copilot: Good or Bad?

I think that's great. It just all it's done is make me more productive


Github Copilot: Good or Bad?

So anyway at least for now. I really do like it


Github Copilot: Good or Bad?

I hope that the future is not dystopic and that we aren't all replaced by something like copilot


Github Copilot: Good or Bad?

But it's not really how I see it going in closing


Github Copilot: Good or Bad?

I'll share a particularly cool example that I stumbled on by just simply poking around with github copilot


Github Copilot: Good or Bad?

And it is Conway's game of life and then visualizing it with pygame


Github Copilot: Good or Bad?

This particular example worked on the first try and this is the order that I went through to actually produce it


Github Copilot: Good or Bad?

But I have tried other larger projects and more complicated stuff like this to be done completely by copilot


Github Copilot: Good or Bad?

And it really didn't work so of everything I've shown here. This is the most cherry-picked


Github Copilot: Good or Bad?

Example, but still really cool to see


Github Copilot: Good or Bad?

Anyway, that's all for now. I honestly can't wait to see what the future holds with this kind of technology


Github Copilot: Good or Bad?

Let me know what you think down below and with that. I'll see you all in another video.


Github Copilot: Good or Bad?


End-to-End Adversarial Text-to-Speech (Paper Explained)

In this work we take on the challenging task of learning to synthesize speech from normalized


text or phonemes in an end-to-end manner, resulting in models which operate directly


on character or phoneme input sequences and produce raw speech audio outputs.


Okay that wasn't the real model I just thought it sounded really funny.


This is a text-to-speech model and it actually sounds like this.


Okay now you've probably if you have listened to the text and not just how the text sounds


you have gotten what this paper is about.


So the paper is called End-to-End Adversarial Text-to-Speech by Jeff Donaway, Sander Diehlmann,


Mikolaj Binkowski, Erik Elson and Karen Simonian of I believe of mostly DeepMind.


And this paper on a high level it produces speech so sound the sound waves of speech


directly from text or from what they call normalized text or phoneme text and it does


so without any intermediate supervised representations and that's a challenging task and the main


problems here are the alignment problem that they have to solve and actually making this


work in an adversarial manner.


So we're going to look at this paper as always if you like work like this consider subscribing


and sharing it out and if you have any comments leave them in the comment section.


Okay so what's the problem with text-to-speech?


Text-to-speech is basically you take a piece of text like this one modern text-to-speech


synthesis pipelines typically involve blah blah blah and you want to make a model that


takes this and outputs sound waves as if a human would say it right so you know modern


text to speech and so on.


Now I have multiple problems when doing this.


First of all the text here is words let's say we can tokenize the text into words so


you have modern text to speech those are four tokens however sound waves are of course much


much densely sampled so the sound waves they are typically in the order of something like


24 kilohertz sampled so that's the ratio of one token to output samples is super high


so one token will produce many many thousand samples in the speech.


So that's the first problem.


The second problem is that if you have training data so you have data that has a piece of


text and you have the sound wave that a human you know the human read that particular piece


of text you still don't know which word exactly corresponds to which portion of that text


you simply know the entire text corresponds to the entire sound wave you don't know this


word text right here you don't know you don't know where it starts and where it ends in


this sound wave and the last problem you obviously have is that you want to make this in a way


that it generalizes that it sounds like a human but also generalizes to some other text


and this paper here solves all of these problems jointly by doing an adversarial approach to


learning and it does it end to end now adversarial simply means that you have a generator that


takes in the piece of text right here and generates this sound wave and then you have


a discriminator that looks at this sound wave and it looks at the real sound wave so the


real okay the real says let's say this over here is real and this is what the generator


has produced the discriminator tries to discriminate between the two now this is not entirely the


same thing as a as a supervised loss usually in a GAN you do not have the corresponding


samples right you just you simply you simply input a real sample here and the generator


produces a generated sample here you do not necessarily have in a classic GAN the corresponding


sample here you assume that you have the corresponding real sample but it's still different than


supervised learning in that both go through a discriminator and the discriminator tries


to tell the discriminator is a neural network tries to tell which one is real and which


one is generated in fact the discriminator is a set of neural networks that we're going


to go into that shortly so it's adversarial in the sense that there is a generator and


a discriminator and it is end to end in the sense that usually what these pipelines do


is they take the text and we've looked at this for example in the video about this Facebook's


text-to-speech system so they take the text and the first thing they do is they would


produce a set of of of whatever they would call features like textual features call it


textual features so these are these are sort of intermediate features for the text to be


produced and then another model would take these and it would produce something like


spectrograms spectrograms and then another model would take the spectrograms and finally


produce sound or speech so you have usually in these systems you have intermediate representation


and each of these models right here can be trained by itself so that's an advantage that


you can train for example you can train a model that goes from a spectrogram to a sound


wave by itself and you don't need you simply need sound for that the computation from sound


to spectrogram is super easy so you can generate your own training data so you can go from


spectrogram to sound you can train a model like this so in these pipelines usually there


are multiple stages and each of these models has to be trained by itself this paper tries


to do this end to end that means you input the text and you get out the you get out the


sound wave and and there is nothing in between that the I mean of course there are latent


representations but you train it ends to end in one go okay so let's look at the different


systems they employ first of all let's look at the discriminators because that's the easiest


so they have these discriminators and these are adopted from this Gantt TS paper now as


we already said the discriminator try to to differentiate between real and fake sound


and they do it in a sort of so if they were to just look at the entire sound wave then


it would just basically reduce to comparing the two but instead the discriminators they


operate on very small windows so in specific they have five different discriminators and


each of the five different discriminators take a different size window length but all


of these windows are super short so one discriminator might take this long windows another discriminator


might take a bit longer windows and another discriminator might take a bit shorter windows


and that of course from the real end from the fake and the discriminators simply try


to discriminate only in these windows whether it's a real or fake and now we're a bit more


into the Gantt setting where you know you simply have one data point of the real and


one data point of the fake and you have to compare them and this here I believe is one


of the keys why this model generalizes because the discriminators basically they try to assess


whether a short sequence sounds like real or sounds like fake and whether the two samples


sound alike in different scales of time and by only doing this on these short scales you


can you can the loss sort of generalizes otherwise if you compare it on the entire sound wave


you would just reduce to comparing point like point by point right here and if the generator


produces something that's not exactly aligned then of course every point would be wrong


and you you'd run into all sorts of questions so this is a set of five discriminators that


all try to take each takes a different length of sub sound of this wave tries to discriminate


real from fake that's the discriminator loss they have an additional discriminator loss


where they compute spectrograms of these things so spectrograms and compute spectrogram of


this and they have a discriminator another neural network here that it tries to distinguish


which one is real and which one is fake note that this is not the same as down here where


the spectrogram is an intermediate representation here the sound is the output and from the


sound you compute the spectrogram and then you compare the two so this is simply a different


spectrogram is a different feature space for the discriminator to compute the loss it is


not an intermediate representation on the way to produce the sound itself so that the


difference that's the difference here between the classic approach and this approach so


it's end to end adversarial so we got the discriminator the discriminators simply try


to differentiate the sound waves and short scales as well as the spectrograms now the


second part is of course the generator how do we even produce sound and that's this diagram


right here so you have this Gantt TS generator this is a generator that takes in a it takes


in a hidden representation so it takes in tokens let's say token one token two let's


go from the from the one before think of a sentence hello there okay so it takes these


tokens and of course it takes like hidden representations of the tokens and it will


output it will output for each one or for this joint sequence one hidden two it would


output the sound wave okay and this has been a paper before the scan TTS and you also you


condition it on the speaker and on latent variables like how you want the pitch to be


and so on that's not really important for us right here the generator can simply take


these token embeddings and produce sound the problem is in the original paper you had an


alignment you knew which token corresponded to which piece of sound and therefore you


sort of knew so after that you need to compare this to the generator thing and you knew which


token corresponded to which piece of sound right here so the generator knew what it had


to produce from each token how long it should be and so on so this is the generally the


alignment problem or what I call the alignment problem so if you take a piece of text like


this entire paragraph right here let's look at this paragraph this paragraph to read it


out takes like 30 to 60 seconds you can't train really models that output this long


of sound it it would be it would be too big of a sample you want to train ideally on segments


they train segments that are I believe here two second windows from each examples so because


they say if we train on 20 seconds that would just be wasteful and prohibitively expensive


now the problem of course is if I simply take a window here like this one of two seconds


and I have my human that has read this entire paragraph in one go I have no clue again which


which part of the entire sound wave of this paragraph these this sub sequence corresponds


to a good guess would be to go like well this is about 50% in so maybe here to here maybe


who knows okay and even within that and that's what we discussed before you have no clue


how long this word here is going to take up within this piece of sound and that's the


general alignment problem right here so in this entire sound wave where is the piece


and how do these words distribute across the wave of sound the original model had as I


understand it such alignments and therefore this generator could work really well because


you had these alignments without these alignments it doesn't work as well and you can on their


website that I've shown you initially you can listen to samples where they disable each


of these things so this generator is really good at producing sound when it has these


alignments so the challenging task here is how do you compute these alignments how do


you compute this thing if you don't have it if you're if you don't have it in your training


data so it needs to be part of the loss so that's what this entire architecture down


here is all right so the text is down here it goes in and the first thing they do is


they normalize the text and they transform it into phonemes which is you can do this


in a deterministic fashion there are scripts that do this this is the only pre-processing


they do and they can also leave it away in the heaven ablation on their website so this


is like phoneme text cat sat on the mat now this phoneme text goes through these big block


of convolutions and of dilated convolutions and this outputs a 200 hertz representation


length alignment okay I should specify this so for each token here it outputs a length


so this thing predicts the length of each of the tokens these all of this all of this


thing here is to to embed the tokens in hidden space and then predict its length you can


see that right here okay so first we use F to to take X so F is a stack of dilated convolutions


and it takes X and outputs a hidden representation so H so first X goes to H and then H is used


to predict L and L is the length of that token so we embed this into a hidden representation


with this right here and then we use this stack to predict the length of each token


so this could be this could say something like this cat token right here is 20 milliseconds


long or instead of milliseconds you would use something like frames or data points maybe


this is 200 data points long and then sat is a bit shorter so this is 100 long and on


is really short so this is 50 long so for each token it predicts the length all right


so now if we have the length of each we can sort of calculate where the starting point


is so if we want to know if we know that here is the beginning and the beginning of the


sentence we we conservatively assume that there is um so they give some silence buffer


here but roughly you can assume that the beginning of the speech corresponds to the first the


first token right you can simply trace the waveform and whenever it goes up that's where


the first token starts so then if since we know that's where the first token starts and


if we could predict the length of each one correctly we could simply sum those up to


figure out where our word starts so if we want to know where on starts we simply go


from the beginning and go 200 plus 100 milliseconds so or data points 200 plus 100 here is where


on starts okay and if we want to figure out the middle of on we simply add the the half


of this number so plus 25 gets you to the middle so this is this here is the center


of the token on so for each token we predict the length like this and thereby we can just


calculate for each one by summing up from the beginning and then adding half of its


own length where the center of that token in the entire sequence is okay now we do this


we said we take random two second audio but we do this procedure for the entire um for


the entire text okay for the for every single token in the text that we look at so in the


20 second text we do this because then for each token we'll get a token center and now


the aligners job here is to align that to the actual sound so what we also give the


generators here the offset so let's say we have this 20 second of speech and we randomly


sample these two seconds and that's maybe five seconds from the beginning we also tell


it this is five seconds right here so what we can now do is we can calculate back sort


of and say okay here I have I first need to discard five seconds of my signal and I have


a prediction how long each token is so I can just cross out tokens until I have basically


wasted five seconds and then I know okay from here to wherever these things sum up to two


seconds from here to that those are my two seconds that I want to look at now this is


how I figure out where in the big sound wave my fragment is right because I have this offset


where I sampled it and I simply add use this and the predicted lengths to figure it out


now I still need to figure out these tokens that are actually in the span how do they


distribute and that's what this aligner here does since we've already predicted the token


centers we simply assume that if these are correct right then if this is let's say if


this is one second long I assume that the middle is after point five seconds so this


is one second the middle is point five seconds so I think that this token is aligned right


here this is the center of the token now we want to be a little bit a little bit fuzzy


with respect to that so what they do is they sort of use a Gaussian kernel right here so


for each token as you can see here each token has a center which is here so the y-axis is


the time in sound and the x-axis is the token and for each token we say well it doesn't


have to be exactly there it can be so they put a Gaussian kernel like this okay if you


imagine this kernel popping out of the frame they say this is about where the center is


and for this token right for this token right here they say well it's it's probably here


in the middle but it could also be here or here or here or here and we wait this like


this so these are these are the weights and then you simply sum up the weights with these


embeddings so for each token out of this dilated convolution block you get a hidden embedding