Drawing Flowers in Python

so that's how many times we want the petal to be drawn.


Drawing Flowers in Python

So, in range N, run the petal function.


Drawing Flowers in Python

So, we're calling in petal here, within the flower function,


Drawing Flowers in Python

and we want it to, after it's drawn an individual petal,


Drawing Flowers in Python

we want it to turn left 360 degrees divided by N.


Drawing Flowers in Python

And now, remember N was the number of petals that we wanted to draw.


Drawing Flowers in Python

So, why is it 360 degrees divided by N?


Drawing Flowers in Python

Well, because we want our flower to eventually look like a circle.


Drawing Flowers in Python

So, we divide the degrees in a circle by the number of petals we want, aka N,


Drawing Flowers in Python

and then eventually it will turn into a flower with the shape of a circle,


Drawing Flowers in Python

with the number of petals N making up that circle.


Drawing Flowers in Python

And now, we define movement here.


Drawing Flowers in Python

And so, it takes in a T, so the turtle, and length, which is how much we want it to move,


Drawing Flowers in Python

so a number of pixels on the screen.


Drawing Flowers in Python

Now, I put this in here, we'll get to that in a second, to give the background colour,


Drawing Flowers in Python

to give the screen a background colour.


Drawing Flowers in Python

So, window equals turtle.screen.


Drawing Flowers in Python

Again, this could be just T if we wanted it to.


Drawing Flowers in Python

But we'll leave it as turtle for the time being.


Drawing Flowers in Python

Turtle.screen, window.backgroundColour, bgColour, takes in a colour here, argument,


Drawing Flowers in Python

now, parameter, sorry.


Drawing Flowers in Python

Now, this could be blue, but we won't do blue because one of our flowers is blue.


Drawing Flowers in Python

And so, when we run the move function, we want the turtle.pen.pu.


Drawing Flowers in Python

So, what does pu stand for?


Drawing Flowers in Python

So, if we come back to here, pu is short for pen up, so it picks the turtle's pen up.


Drawing Flowers in Python

So, when the pen's up, it won't draw.


Drawing Flowers in Python

So, what's similar to pen down?


Drawing Flowers in Python

Puts the turtle's pen down.


Drawing Flowers in Python

Because turtle, essentially, remember, is just a drawing function, and it's a pen.


Drawing Flowers in Python

So, pd, pen down, drawing, pu, pen up, not drawing.


Drawing Flowers in Python

So, we want to pick the pen up, to move the turtle forward a specified length,


Drawing Flowers in Python

and then to put the pen down and start drawing.


Drawing Flowers in Python

So, here's where we define our turtle.


Drawing Flowers in Python

Now, I've called mine Sam.


Drawing Flowers in Python

This could be whatever you want.


Drawing Flowers in Python

It could be Bob.


Drawing Flowers in Python

It could be Charlotte.


Drawing Flowers in Python

The name doesn't really matter.


Drawing Flowers in Python

We're just giving the variable a name.


Drawing Flowers in Python

And now, Sam equals turtle.turtle.


Drawing Flowers in Python

And what does this mean?


Drawing Flowers in Python

Well, we're calling in the turtle function up here from the turtle library to say that


Drawing Flowers in Python

the variable Sam will be a turtle, a.k.a. a drawing function.


Drawing Flowers in Python

And now, we can adjust some of this.


Drawing Flowers in Python

This is where we get the turtle or the pen to start drawing.


Drawing Flowers in Python

And now, we can find the speed of the drawing.


Drawing Flowers in Python

And the speed, I think, I'm not sure.


Drawing Flowers in Python

I tried it at different speeds.


Drawing Flowers in Python

I tried 100, and it goes very fast.


Drawing Flowers in Python

So, we'll try that in a second.


Drawing Flowers in Python

But, essentially, one is the fastest, slowest.


Drawing Flowers in Python

Zero won't move at all.


Drawing Flowers in Python

But, for this sake, to make it fast, we'll go 100.


Drawing Flowers in Python

And now, to draw our first flower, we've got Sam.color green.


Drawing Flowers in Python

Now, you can probably guess that this means that this flower is going to be green,


Drawing Flowers in Python

because we're calling the turtle, we're giving it a value of green.


Drawing Flowers in Python

And now, this Sam.shape turtle, this could, if we delete this,


Drawing Flowers in Python

let me just show you what that comes out as.


Drawing Flowers in Python

So, see how there was an arrow just drawing?


Drawing Flowers in Python

That went really fast, because I had it on 100 speed.


Drawing Flowers in Python

If we go back to having turtle there, it'll use a turtle to draw the flowers, not an arrow.


Drawing Flowers in Python

Now, I'm not sure why this keeps crashing.


Drawing Flowers in Python

But, let's get out of there.


Drawing Flowers in Python

Alright, so move Sam, negative 150.


Drawing Flowers in Python

Now, that means that, from the start, I'm calling the move function here,


Drawing Flowers in Python

and I'm putting in Sam, which is the turtle, and then the amount I want it to be off-center.


Drawing Flowers in Python

So, it's going to move along the x-axis, negative 150 pixels from the center.


Drawing Flowers in Python

So, what does that look like?


Drawing Flowers in Python

Well, when we draw our green flower, see how the green one's not in the center,


Drawing Flowers in Python

the red one's in the center?


Drawing Flowers in Python

Because we're shifting it across from the start.


Drawing Flowers in Python

So, we're moving it from the start, because the turtle, when it starts by default,


Drawing Flowers in Python

will start in the center.


Drawing Flowers in Python

And then, why are these two, we have a red one that's drawn with outline,


Drawing Flowers in Python

a blue one that's drawn with outline, and then a green one that's drawn with fill.


Drawing Flowers in Python

Well, because on the green one, we have these two commands here,


Drawing Flowers in Python

begin fill, end fill, but we'll get to those in a second.


Drawing Flowers in Python

Let's talk about this flower command here.


Drawing Flowers in Python

Flower function more, would be more correct term.


Drawing Flowers in Python

So, if we go up here, def flower, define flower function, takes in a parameter turtle,


Drawing Flowers in Python

and an angle.


Drawing Flowers in Python

So, here's what we're doing here, we're entering these values.


Drawing Flowers in Python

We've got flower takes in the T value, Sam, with an N value of seven,


Drawing Flowers in Python

with an R value of 60, and an angle value of 60.


Drawing Flowers in Python

So, see there, T, Sam, N, seven, R, 60.0, angle, 60.0.


Drawing Flowers in Python

And so, let's have a look how these relate to our drawings.


Drawing Flowers in Python

So, if we go over here, seven will be the N.


Drawing Flowers in Python

So, essentially, it's going to draw, it's going to run this petal function here, N amount of times.


Drawing Flowers in Python

So, what do you think if we were to run the petal drawing function seven times, we should result with?


Drawing Flowers in Python

We should end up with seven petals, right?


Drawing Flowers in Python

So, one, two, three, four, five, six, seven.


Drawing Flowers in Python

Perfect, that's what we want.


Drawing Flowers in Python

Now, what happens here?


Drawing Flowers in Python

What's the R?


Drawing Flowers in Python

So, that's the radius, so that's 60 pixels.


Drawing Flowers in Python

So, the radius here of a flower petal is 60 pixels.


Drawing Flowers in Python

And then the angle it turns is also 60.


Drawing Flowers in Python

And so, that's how we get this curvature up here, rather than it being a circle.


Drawing Flowers in Python

Because remember up here, we're taking in R and angle.


Drawing Flowers in Python

And why does it complete, why all the petals are even spacing around each other?


Drawing Flowers in Python

Well, because remember, when we finish drawing each individual petal,


Drawing Flowers in Python

we run T.left at an angle of 360 degrees divided by N.


Drawing Flowers in Python

So, what is that?


Drawing Flowers in Python

So, if we go here, 360 divided by seven, 51.


Drawing Flowers in Python

So, each one of these petals is at an angle of 51 degrees.


Drawing Flowers in Python

And then we repeat the process here for the red flower and the blue flower.


Drawing Flowers in Python

Except the red flower has 10 petals and the blue flower has 14 petals.


Drawing Flowers in Python

And they have, the red flower is at a radius of 40 pixels,


Drawing Flowers in Python

and the blue flower has a radius of 70 pixels.


Drawing Flowers in Python

So, what happens if we change this number to like 100?


Drawing Flowers in Python

And let's say this one to like 25 on the red flower.


Drawing Flowers in Python

Let's have a look what happens.


Drawing Flowers in Python

There we go.


Drawing Flowers in Python

So, now the flower has 25 petals, as you can see,


Drawing Flowers in Python

and each of them has a radius of 100 pixels.


Drawing Flowers in Python

And, but the angle is still at 80.


Drawing Flowers in Python

So, what if we change this to maybe 30 and let's see what happens.


Drawing Flowers in Python

Ah, there we go. It turns a lot tighter.


Drawing Flowers in Python

Let's change this to 30.


Drawing Flowers in Python

Maybe I got the angle and radius mixed up.


Drawing Flowers in Python

Ah, it's the width of the petal.


Drawing Flowers in Python

There we go. That's cool.


Drawing Flowers in Python

So, across. So, rather than from here to here, it's from here to here.


Drawing Flowers in Python

Now, have we covered everything?


Drawing Flowers in Python

Oh yes, the fill. So, why is this filled in and this is not?


Drawing Flowers in Python

Well, with the turtle function, there's a begin fill and an end fill command,


Drawing Flowers in Python

as you'll find over here on this summary of turtle methods.


Drawing Flowers in Python

So, begin fill, remember the starting point for a filled polygon,


Drawing Flowers in Python

a.k.a. where did you start drawing, end fill, close the polygon,


Drawing Flowers in Python

and fill with the current fill color.


Drawing Flowers in Python

So, the end fill, and why do we run it,


Drawing Flowers in Python

why do we run these functions above and below the flower function?


Drawing Flowers in Python

Well, because begin fill is going to remember where the turtle initially started.


Drawing Flowers in Python

So, let's turn this right down to, let's say, 1,


Drawing Flowers in Python

and then the end fill is going to remember where, let's go a bit higher,


Drawing Flowers in Python

let's go to 2, we don't want to be waiting too long,


Drawing Flowers in Python

and then the end fill is going to be,


Drawing Flowers in Python

is going to remember the final point of where the turtle was,


Drawing Flowers in Python

and then it's going to backfill the entire drawing with the color you had been using.


Drawing Flowers in Python

So, in our case, green.


Drawing Flowers in Python

Let's, for demonstration's sake, just change this to blue,


Drawing Flowers in Python

just so you can see that the color can change.


Drawing Flowers in Python

So, nice and slow, the turtle started off in the middle point,


Drawing Flowers in Python

it's going to draw seven petals here, because we've got seven here,


Drawing Flowers in Python

it's going to draw each of them with a radius of 60,


Drawing Flowers in Python

oh, there we go, the end fill, so right on the last point,


Drawing Flowers in Python

it remembers where it finished and where it started,


Drawing Flowers in Python

and fills the rest of the shape with the color you were drawing in.


Drawing Flowers in Python

So, let's do the same for the other two.


Drawing Flowers in Python

sam.beginFill.end


Drawing Flowers in Python

See how it's going very slow?


Drawing Flowers in Python

It's because we've changed our speed to 2,


Drawing Flowers in Python

so for this next demonstration, I'll make sure I put it up to like 100 or something like that again.


Drawing Flowers in Python

All right, now, if this all works correctly,


Drawing Flowers in Python

we should have all of our flowers filled up with their specific color.


Drawing Flowers in Python

Let's change this one back to green, and let's run for one last time.


Drawing Flowers in Python

Nice and fast, oh, look at that, pretty pattern, there we go.


Drawing Flowers in Python

So, that is a quick and easy way of drawing some shapes in Python


Drawing Flowers in Python

using the turtle feature, defining a whole bunch of functions,


Drawing Flowers in Python

so we've got one, two, three functions here, and then we're calling them,


Drawing Flowers in Python

flower we're calling and adding in the parameters here,


Drawing Flowers in Python

and within the flower function, we're calling the petal function,


Drawing Flowers in Python

which we've defined up here, and then before the flower function starts to go,


Drawing Flowers in Python

we're calling the move function, and so everything works from top to bottom,


Drawing Flowers in Python

so it's going to start off by defining sam, because if you imagine there's a line here,


Drawing Flowers in Python

we define all of our rules first, and then we start to write what we want to happen.


Drawing Flowers in Python

So we define sam as the turtle, aka the drawing parameter,


Drawing Flowers in Python

we give it a speed, and then we give it a color, give it a shape.


Drawing Flowers in Python

Now, this shape can be a number of different things, turtle is a fun one.


Drawing Flowers in Python

We call the move function, so rather than the green flower starting in the center,


Drawing Flowers in Python

we move it across a little bit, and then we begin the fill in the position


Drawing Flowers in Python

where the turtle ends up, so negative 150 pixels from the center,


Drawing Flowers in Python

and then it starts to draw the flower with these parameters that we've defined up here,


Drawing Flowers in Python

and then it ends the fill and colors the flower in green.


Drawing Flowers in Python

Then we repeat the process for the red and the blue flower, and we're done.


Drawing Flowers in Python

We get some pretty shapes.


Drawing Flowers in Python

So that's my brief explanation, or well, not fairly brief,


Drawing Flowers in Python

I'm finding this video is going to last 20 minutes now,


Drawing Flowers in Python

but that's essentially how I did this project on Udacity.


Drawing Flowers in Python

And how did I do it? Well, I did a whole bunch of Googling.


Drawing Flowers in Python

You can see I've got some websites here with Python code,


Drawing Flowers in Python

example of how to use turtle, and then I just chopped and changed the code to suit me,


Drawing Flowers in Python

but the project asked me to draw one of these shapes,


Drawing Flowers in Python

and I ended up with wherever they've gone.


Drawing Flowers in Python

It's lagging out.


Drawing Flowers in Python

But yeah, hope you gained some value out of this.


Drawing Flowers in Python

If you haven't, well, then I'll keep making more videos


Drawing Flowers in Python

and hope you get more stuff out of that.


Drawing Flowers in Python

But I'm rambling now, so I'm going to get back to the next project,


Drawing Flowers in Python

and I'll record that when I'm doing it, and then we'll see where we end up.


Drawing Flowers in Python


10-minute Lower Body Bodyweight Workout for Beginners

Oh, I'm building this hat. Look how good it is. Airbnb amenity detection. This is part of my Airbnb project


But that's not what you're here for you're there to move


We'll put another video up when that's finished and in the meantime check out streamlit.io. Amazing. That's a real thing


Let's get some light in here. I've been coding in basically a


Dungeon it is day 18 for Revsterona


And I'm so excited because we have my upper body's a little bit sore, you know


After doing these body weight workouts. That's why it's important


body weight workouts might seem a little bit more easier than


Say a weighted routine or something like that, but that means they're easier to overtrain


So it's important that if you are doing the 10 minutes per day of body weight


Exercises while you're inside if you've been following along with the series if you're just watching this randomly


Be careful beware with body weight exercises


Yes, they may seem easier


But they also are easier to over train which means your muscles will get too fatigued short story is rest when you need to rest


Let's get 10 minutes on the clock lower body focus today


We have a body weight squat a split squat a side lunge a jump squat and a chair lunge


I actually need a chair. We'll be back in a second


We got the chair now don't make the mistake


I did start this workout off with the chair, but before we do get started


I just want to give a big big big shout out to the man behind the camera


It big wham it is his birthday today, April 3rd. He turns 20 years old born in the year 2000


I always tell him this story. I remember when he was a little baby. No one could get him to stop crying


Except for me. I remember I was about seven years old holding him on my chest and he stopped crying


Even though mum couldn't get him to stop crying. I felt really special that day. So happy birthday big wham


Love you, and thank you for your help. All right, let's get started


You know this this app puts little quotes there


The reason I don't carry a mobile phone is I don't want people to know where I am


That's the reason why my mobile phone lives in the drawer most of the day


distraction device


Let's get started. You know the drill 30 seconds on 30 seconds off. We're starting with a body weight squat. Oh


Yes


This feels good already


Good already


This is one of my favorite times of the day. Yeah, I'm doing rest for Rona


So again, we're not overexerting ourselves


We're just getting 10 minutes of movement in every day


whether it be in a walk


whether it be in stretching or whether it be


What we're doing right now with some body weight exercises, let's take that off body weight squat


And you might be wondering


what is


The theme of today's workout


Is a work versus life?


So you often hear a lot of the time


Work life balance and so what's happening in the world at the moment is


It's kind of all being merged into one and that's why all the time


That's why oh, what are we on?


Split squats. So this one


Just nice and small like this nice and small split left foot forward for the first one


It's like a lunge but not as not as far forward as a lunge


So it's just your knees coming down to meet your ankle your back knee meets your front ankle. There we go


See how your balance is and goes faster as slow as you want


I'm just going at a steady pace here


There we go, so right now a lot of people working from home maybe you are


but life and work have kind of merged so you might be at home your whole family's at home your your kids at home and


You're trying to work at the same time and that's you often hear


People discuss work life balance or maybe I'm working too much. Maybe I'm I'm living too much


I don't think there's such a thing right because how can you how can you live too much?


You see I've tried to take a perspective or


Side lunge so this one we're gonna go start here. We'll go left leg actually because that way we'll keep consistent


That's it. So start standing up vertical


Out to the side back up out to the side. I feel that stretch. I'm feeling that stretch right in my right hip


and oftentimes life is separated from work and


Sure, I see it. I see it a little bit differently or at least I try to to trick myself into seeing it differently


One more good luck as we start late


There we go


They're both the same thing. You see work kind of has a negative connotation to it. Whereas it was like, oh I got work


busy


You know, you know that feeling people talk about work. Well, it's like no if you treat work as play


Whatever you're doing if you're learning something hard, you could go every time you learn a new thing. It's like you're dancing


You're dancing from one point to the other


What do we got?


Jump squats. Let's go down here. Throw your hands up like this


That's it


And you turn you turn it into a game. It can be whatever


Kind of work there is and of course there are going to be times where you have to work on things


You don't really want to work on


And that's going to just be a grind but the same philosophy


applies


You can turn it


Into a game, you know what that's how do you get a group of kids to do anything they want anything you want


Have you ever had a group of nine-year-olds for example had you free at a jiu-jitsu class and you want them to?


To all do some kind of drill


What do you do you turn it into a competition?


Because then everyone sees it sees the work of the drill as play


So what I go next


chair lunge


left leg forward


We're going right leg on the chair come forward find your balance


Kneel right down feel that stretch. That's it


Up


Down yes, that's it


Hold it up one for a beautiful walk at sunrise this morning


Absolutely amazing to get out in nature


Every day


Every day with some kind of walk is a good day every day with some kind of


Movement is a good day every day when it's your little brother's birthday. It's a good day


Now we're back to bodyweight squats we've been through it through through one time, you know, I was saying to someone


I know we're gonna mention their name or


Or whatever


Just leave it at that. We're having a conversation how oh


Oh, we wanted to do this thing. Oh, what do I squat?


Let's go here we wanted to do this thing keep moving I'm watching you let's go bodyweight squat


Where it was a live event put it that way


Is I oh, no, I got I got work now this life event wasn't too big. It could have just been a half an hour and


Comes to the realization


It's like no, I got work it it takes priority over this life event


Which this life event didn't happen or doesn't happen very often you could probably guess what doesn't happen very often and is today


And I said


Or I came to the thought


Why is work allowed to get in the way of life?


But life isn't allowed to get in the way of work


Except for now because there's an absolute health crisis


crisis


No team crisis


What do we got split squats this time right leg for?


There we go. Keep it up


Let's go


We're moving we're philosophizing


Yeah, I thought why is the why is it generally okay for?


Work we just someone says


You're trying to keep up with me man. Well, there we go


Like the example could be why can't you say it's a Wednesday and it's like 2 p.m.


And you wanted to watch a movie for two hours, but you're like, oh no, I've got work for two hours


But then it's like Friday night and it's 5 p.m. And you're supposed to be finished work


or whatever time you finish and then it's like oh you're supposed to have some dinner at on Friday evening


It's like oh no works going overtime and the work going overtime would be seen as okay


We're into side lunges we're going to the right


The work going overtime would be like people like yeah, that's perfectly fine


Well, maybe you won't but if you if you don't that's a good thing


But if you just wanted to blow off work on a Wednesday afternoon just go. Yeah, I'm gonna watch a movie and chill out


Why is that what's the difference there, you know


Sips in the health crisis where not many people a


Lot of people losing their jobs. It's now a big time to reconsider what's valuable


I said in a little post I did yesterday


I've been thoroughly thoroughly enjoying the the dinners


we've been having together as a family on the on the back deck and sort of a


It wouldn't happen as often as it's happening now and we got up the conversation. It's like


What's what's like most important in this time and as always?


Number one came out as what Sam?


Family family. Oh, that's great. Wasn't what I was thinking of but it's very close


health and but they're again running the same just like life and work can be the same


family and health same thing


What happens when a family member gets ill


What can you think of nothing except that keep it up let's jump


Be a beautiful how good is jumping now when a family member gets ill what can you think of


Nothing, but that family member getting better when you get sick


What do you think of a sick person wants one thing a healthy person wants?


A healthy person wants


a healthy person


wants one thing a healthy person wants


10,000 things


So let's go. We got chair lunges to finish off


This time we're going right foot forward


We're putting our right foot forward getting our reps to Rona in


That's it


I can't believe it. My littlest brother is 20 years old


Making me feel old


And not so little actually


Guess what?


It's 18 days in a row


18 days of


moving


Working up a little sweat keeping our hands


Hands nice and clean getting rid of that virus and staying indoors doing our part


Let me let me just pause this


Doing our part helping out where we can so thank you for joining in today was a lower body session


We started off with the bodyweight squat did the split squat side lunge jump squat chair lunge


Again, you don't have to do these things day to day if you need to rest take rest if you want to do a little


Bit more of an extension, of course you can but thank you so much for tuning in


remember


Life first work. Is there such a thing as work-life balance or are they are they just the same thing remixed?


Ask a Machine Learning Engineer Anything (live) | March 2019

All right, I think we're live. I hope we're live. I just updated the stream key actually


if you're just tuning in. Oh Bob, we're getting green lights. If you're in a chat somewhere,


if this is live somewhere, let me know. But what's happening team? It's March, the end


of March. 2019 is flying by. This month has been hectic. I need to readjust my chair.


So what we're going to do, the purpose of this stream is for me to just say hello and


for me to answer any questions that you may have. So if you do have questions, let me


know in the chat. I'll stick around for the next, what's the time? 6.52. So I'll stick


around for the next half an hour to 45 minutes. Let me know if you have anything that's been


burning on your mind, what your thoughts have been. What are you working on? Share with


me. I want to hear it. We can talk about it. And hold on. I've got a bunch of windows open


here that I need to fix. It's kind of weird at the moment. I'm like watching myself and


trying to talk to the camera and I've got another monitor over here. I've got a lot


going on. Still getting this live stream business happening. But a big shout out to everyone


who's joined the channel in the last month. That is massive. Thank you all for joining


in and of course, anyone who's been here for a while. Thank you so much. I really appreciate


it guys. I love making videos. I love seeing your comments. I love hearing what's happening.


Switch account stream is lagging. All right, I'm going to close some windows and let me


know in the chat. Thank you for the heads up by the way. Let me know if you can hear


me. Let me know if you can see me. Thank you for that Leon. Hello, morning a lot. Shayk,


what's happening brother? Good to see you man. What are you been working on? It's not


better. All right, what's happening? Maybe my internet. Please tell me. Maybe I pause


my own stream. I just look at the chat. Okay, how's it now? Morning Michael, what's happening?


Okay, I've got bare bones internet running on my computer right now. So hopefully it


should be working. All I have is the streaming system. Leon, thank you so much. Good afternoon


Nitesh. Now it's smooth. Okay, all right, we're slowly working out all the kinks. All


I'm going to be looking at is the chat window and trying to go back to the webcam now and


then. Thank you for everyone for hitting me up. Shayk has a CS exam tomorrow. Going to


crush it. Yeah, that's what's up. Good work man. I love the attitude. I love the attitude.


All right, so I'm going to jump straight into questions. If you're all happy for me to do


that for the next half an hour to 45 minutes, I'm going to try to answer as many as you


can. Otherwise, if you don't necessarily have a question, but you want me to talk about


something, let me know as well and we can go but I'm sure we'll have plenty of times


here. All right. And bear with me. I'm going to get to them as as all relatively quickly.


So we have Swastik Nayak says, where can I get a real world data set? My friend, there's


a lot a lot on the internet and it depends what you mean by real world, you know, so


Kaggle is the first place that comes to my mind. Otherwise, it's just about researching


the specific problem that you want to work on. For me, I like working on health projects.


So I would try and find a health data set somewhere. And now there are some privacy


issues with health data, of course, because people's health I mean, it's a private issue.


So it really depends on what you want to work on. But I'd start on Kaggle, they're pretty


close to what you would get to a real world data set. As a machine learning engineer working


at Max Kelson, which is a Brisbane based company. The data sets that you can find on Kaggle


are quite similar to some of the ones I've worked on in actual businesses. But otherwise,


researching for a proper data set is all part of the part of the fun of being a data scientist


slash machine learning engineer. Shushant Pratikar asks, Hey, I'm second year doing


engineering computer science and I started self learning ML. I'm loving it, but I'm kind


of stuck between my passion and university. What should I do? Yeah, this is a tough one


Shushant. I'm assuming that your passion is is is machine learning. And now of course,


everyone you asked this question is going to have a different piece of advice. But I


would weigh it up and saying, depends how long your computer science degree is. Do you


think that where do you want to be in three to five years? This is what I like to do.


This is what I think of myself is where do you want to be in three to five years? And


will a will studying computer science at university get you closer to that? Or B will leaving


university studying machine learning and using those skills for something get you closer


to that three to five year period? That's something I do with myself. Will it work for


you? Not entirely sure. So it's probably going to take a little bit of a little bit of time


away from from other people. And just really thinking about, okay, what is it that I want


to do in the next three to five years? Writing it down is really helpful because that way,


it's in front of you, right? So I write a lot of things down. And then going, okay,


what do I need? What actions do I need to take over the next 12 months to get to where


do I want to go? That that's always my advice with that. And again, ultimately, you're going


to be half, you're going to be the one to have to make that type of decision. I never


went to university for computer science. So I can't talk about that. But I self taught


myself well, self taught myself machine learning online. And I can say it's difficult. But


everything worth learning is always difficult. Michael, Michael, Michelle and Michelle or


Michael or Michelle. Michael, I'm going to pronounce names wrong, but bear with me. And


how's it going with the Google Cloud Data Engineer certificate? Ah, update on that I


got the certificate. They gave me a hoodie. But that's a spoiler. I'm making I'm going


to make a video explaining how I exactly got certified. Just the study I did and what I


think is best if you want to go for Google Cloud Data Engineer certification. But that's


that's coming in a future video. But I passed the exam first try is pretty difficult exam.


Not gonna lie. I'm not a fan of exams. Leon goes, thoughts on the crowdedness on Kaggle.


It depends what you mean, Leon, like crowdedness. I'm, I'm guessing you mean crowdedness. Hold


on one second. I'm guessing you mean crowdedness as in so many people doing different competitions.


Now, that that's a good point. There are lots of people doing different competitions. However,


if you treat Kaggle as a resource to start developing your own skills, I think it is


great because you can see what other people are doing. You can see how well your results


go compared to leaderboards. Maybe you won't get the best in the world because yes, you're


right. There are a lot of teams now working towards getting really good results. But you


can certainly you can certainly increase your knowledge on how to work on a wide variety


of problems, right. And I mean, at the end of the day, to me, learning those sort of


skills is more important than winning competition. Sure, if your goal is to win the competition,


then win the competition. But for me, I use it as a way to practice my skills to get better


at things to see what other people are doing. Hurry. Hey, man, what's up? Good to see you,


my friend. Thanks for joining in. And by the way, thank you everyone for joining in. Yeah,


it's good. It's Sunday, Sunday, fun day. We're gonna have a good hangout session here. Nitesh


asks, from where you completed deep learning.ai course? From where did I complete it? I completed


it from this room. Where you see me here? I did that in about probably a year ago now.


But I've even gone back to some of the lectures actually last week. We're doing a speech to


text problem where I work. And I wanted to go back to some of the lectures from Andrew


Ong. They're really great. And that just shows how much of a good course it is that I'm revisiting


a year later. These are the sort of things I want to do right is, is not necessarily


go through things really fast. I'll go through lots of lots of lots of different things.


I think it's valuable. But once you've got a foundation, I think it's helpful as well


to to revisit what you've already done. That way, it's kind of like looking at it with


a different set of eyes. If you've ever read a book more than once, and you know, the second


time you go back through it, you go, Oh, wow, I didn't notice that the first time. Same


with going through a movie. So that's what I'm trying to do. If I find a good resource,


not not continually piling on new resources, but if I find a good one, trying to revisit


it every so often. And I mean, it's a hard balance, right? G H KKK. G H J J J H V us.


How are the machine learning opportunities in Australia? Are there many companies working


on it? Okay. Ah, straight up. I I Oh, Leon, his updated his question. I'll get to that


in a second. How are machine learning opportunities in Australia? Are there many companies working


on it? Well, I can only really talk about Brisbane. Because I haven't I don't know too


much about Sydney and Melbourne. Yes, there are companies working on it. But not many.


It's still a it's still in terms of what we offer. Max Kelson, where I work is a machine


learning consultancy. So we, we partner with other businesses and go, Hey, if you have


data, we can help you find more out of that data using data science, machine learning


techniques. And we had a meeting with with Amazon Web Services the other day. And they


actually talked they they said, I'm paraphrasing here that there aren't many companies offering


what we offer in Brisbane. So that was a good thing in terms of business opportunities.


So I would say, yes, there are many machine learning opportunities. But there aren't as


many companies working on it. But depends what what you say many, right? Australia only


has about 25 million people us has 300 plus million. But yeah, so opportunities. Yes,


there are there are opportunities machine learning everywhere. Many companies working


on it, I couldn't give you an exact figure. But from my intuition, it's saying not too


many. Tenzil asks, How do I improve my mathematics skills and logic encoding any good resources


in Tenzil? I've been using for mathematics Khan Academy and three blue one brown as well


as just practicing code just working on projects. I mean, they're, they're, they're the three


resources. To be honest, in terms of how do I say this? In terms of math, like mathematics,


I haven't had to do too much in my machine learning role. A lot of what I've been doing


is applying code. So my workflow will usually be some extensive exploratory data analysis,


which is exploring a data set from scratch and just trying to figure stuff out about


it. And then running some models on it, and then doing up a presentation for a client,


and then figuring out where they want to take it to next. So that's a lot of what my work


flow is, is not not too much hardcore math involved in that. I'm more so taking code


that exists or writing my own and applying it to a data set. Leon, I don't think Kaggle


is good for data sets because they're cleaned already. It's not the case in real life. Leon,


you're right. Kaggle data sets are relatively clean to some you'll find in real life. But


I think especially when you're just starting out, without actually partnering with like


a business who hasn't done machine learning or data science before, I think Kaggle is


great to sort of start practicing your exploratory data analysis skills, which are going to be


very, very valuable with whatever data set you start working with. Of course, there's


going to be worse data sets when you get into the real world. But as a kind of training


ground, it's probably the best that I can think of online. If you have a better one,


I'd love to hear it. Ray Lowe asks, morning, Daniel. Hey, morning, Ray. Thanks for joining


in. Do you work on the research side of machine learning algorithms or more on the implementation?


Which path would you recommend beginners to go for? Ray, I work on the implementation


of the algorithms. I do some research in terms of reading papers, like machine learning papers,


and then trying to build out what the paper has talked about. That's not as much as using


existing algorithms that aren't sort of, I don't want to say, how do I explain this?


Okay, I'll give two project examples I can talk about. So one, I was working with tabular


data, which was structured data. And it was trying to predict a binary variable. Well,


I made it to predict a binary variable, binary classification on over $1,000 or under $1,000.


It's a commerce venture. So this company bought and sold things. And I wanted to see, is it


over $1,000 or under $1,000? Because $1,000 was about their median price of profit. So


would this item make us $1,000 if we bought it? And so for that one, I used the CatBoost


algorithm. If you go to catboost.ai, really, really great team, really great algorithm


there. I applied that. And for another problem, so that was using existing machine learning


algorithms that are really easily accessible, like CatBoost. And another one was a natural


language processing with multi labels. So multiple labeled pieces of text. And I read


a paper called extreme multi labeling, I believe XML, XML, CNN was the architecture that I


took from the paper, coded up in Keras, and then applied it to that problem. And that


ended up getting state of the art compared to the existing NLP that we were using. So


that's one example. I do a bit of both, but I would say mostly implementation of the algorithms.


And where would you begin? Where do you be? Where do I recommend beginners go? It really


depends on what you want to get into. If you've got some Python foundations, great. Start


with the fast AI ML course, I really like that one. Or if you don't have Python foundations,


start on data camp. So I usually recommend it at the moment, my brother is learning data


science. I've told him data camp, and then Coursera for a little bit. And then start


working on your own projects. And then fast AI ML can kind of be alongside Coursera. Path


Dave, man, I love your videos. Thank you so much. Well, I really appreciate it. I deal


I'm Ed. Hey, Daniel. Hey, how you going to deal? I've got Oh my goodness, I'm going so


slow through all these. But thank you everyone for joining in. I really appreciate it. So


hey, Jim, what websites do you use for your self learning resources? Coursera, Coursera,


Udacity, Medium, Archive, they're there for YouTube. Yeah, I said most of them. Coursera,


Udacity, Medium, Archive, and then a whole bunch of different blogs. But if you want


just Google Daniel Burke AI master's degree, I've listed them all there. Shabam, please,


can you tell me I have no idea how to learning Python and ML from scratch Shabam, if you


have never started before, I highly recommend Data Camp. Otherwise, if you can't go on Data


Camp, I would just maybe start with YouTube and go Python for beginners. Seriously, there's


a lot of good resources on YouTube. And data science stuff on there. Maybe if you've got


no ML, if you got a little bit of Python, check out my video that I did on Kaggle, the


Titanic data set. I want to start, let me know if you want me to start making more of


those Kaggle type videos that I did like Kaggle from scratch. I'm thinking of that to be my


next series like going from zero, maybe like 10 videos, and it would be like six to eight


minutes long each. And just step by step of how I will work through a problem. This is


what I'm thinking of. Let me know in the chat if you'd like to see something like that.


What do we have here? Leon puts in a great resource, Cracking the Coding Interview, the


first six chapters. I've got that book as well. I might start making videos on that


book. I don't know if I'm allowed to. I might have to change the problems. There's so many


video ideas. I have too many, not enough time. Trying to get here. Path Dave, one question,


one question only. How to focus on your body and staying lean and on your brain? So Path


Dave asks, how to focus on your body, staying lean and brain? MLDLAI more specifically.


How do I do it? Okay. How do I maintain it? Well, here's the thing. Health and technology


are my two favourite things in the world. Oh, actually health, technology and creating


things and relationships. All right. So there's about four, four different things. But I think


about what I want to, what's the, the three words that I use is what's the alternative.


Right? That's, that's what I say to every, every single problem that comes up in my head.


This is how I literally deal with, with my problems. I'll write down what's the alternative


and then answer it. So let's say for example, I have an option of being healthy. What's


the alternative to being healthy? Being unhealthy. I don't want to be unhealthy. So that's simple.


I spend time to, I dedicate time to being healthy. And then we got the, how do I focus


on my brain? What do I want to learn? Machine learning, deep learning, AI, programming,


mathematics, health, nutrition. I have a lot of things I want to learn. What's the alternative


to that? Not learning them. It's simple. All right. I, this is how I deal with it. Will


it work for you? I'm not entirely sure, but that, those, those three words are literally


how I deal with many, many big altercations in life is what is the alternative. And if


I don't want the alternative, I do the thing. So maintain my health and try to learn as


much as I can every day. After all this, this world is exciting. Learning can be hard, right?


I'm not going to lie. It can be tough, right? When, especially when you're learning new


stuff because the brain cells literally don't exist yet. So you need to make them and that's,


that's hard to do. But the alternative is to not learn it and just keep doing what you're


always doing. And my personality doesn't align well with doing the same thing over and over


again. Except if it comes to maintaining health and learning, but whatever. What's the alternative?


Ask yourself those three words. Hari, are you excited? Hari says, am I excited for the


research going on about using DNA for data storage? Hari, to be honest, I don't know


too much about what the research is going on for DNA for data storage, but I'm pretty


sure it's the, the longest running data store in the universe that we have aside from, well,


maybe you can store data in atoms and other things, but in terms of life or biologically


based, I think DNA is the longest. You might want to correct me on that, but if it has


anything to do with amazing ways to use DNA, DNA is already fascinating. So I'm really


excited. Michael Swain says, as someone who's transitioning from a non-CS background, do


you have any tips to make prospective employees take you seriously? Michael, my best advice


for that is to work on, work on things which utilize your skills. So I don't have a computer


science background, but if you're starting to learn machine learning and data science,


have a portfolio of things that you can say or projects that you can go, here's what I've


done and show someone. And another thing is being able to communicate them really well


is, is a, is a skill that is definitely worth working on, right? So rather than just going,


dumping them and just saying, here, here's what I've done. Maybe you could write, I'm


talking about uploading things to maybe GitHub or something like that. And I need to do this


for my latest one. So don't use me as an example. Maybe in the read me file, you write a little


bit about that project. So I'm in the process of doing that for my latest one, but that


is the best way to get prospective employees to take you seriously. And now it's going


to depend on which employee you talk to, right? Like some will require, it's just a thing


about their business that they require you to have a computer science degree. Others,


not so much, right? So, so it's, it's always going to be, how do I say it? It's always


going to be a balance. Yeah. You can say that to anything, Daniel, but it's about using


the skills you've got and then creating things with, with the skills that you have and then


being able to showcase those skills. So whether that's through a GitHub or a website or something


like that, if you can do that, well, that's a way to, to get employees to take you seriously,


but just saying, Oh, I can do this is not as good as being able to go, Hey, here's what


I've done. Have a look at this. Um, Aravind, Nitesh, can high school students start learning


ML? You definitely can. If you Google, what is machine learning and then start from there.


There's a lot out there. So be ready for it. Uh, Ilken, hello. Do you think, what do I


think about IBM data science professional certificate? Uh, I, I I've never done the


IBM data science professional certificate, so I'm not entirely sure, but IBM put out


some great work. So, uh, we use some of their services at Max Kelson. So I mean, if they're,


if they're putting out a data science professional certificate, I'd say it might be worth checking


out. Um, Kamsky J is spending much time on Kaggle necessarily. No spending time on anything


is not necessarily, but I think it is valuable if you want to start working on projects which


aren't necessarily contained in a course environment. So here's what I would say. Here's like the


learning, the learning array of things. Courses are like level one, right? So you start on


a course and they can be hard, right? Level one can be hard, but once you've done enough


courses, you sort of get good at doing courses and then level two would probably be using


what you've, you've learned in the courses in a Kaggle competition. And then level three


would be using what you've learned in the courses, using what you've learned in the


Kaggle competition and then creating your own project, which you really don't know what's


going to happen. You can go, Oh, I want to use data to, to figure out this. But if it's


your own project, you might not know what the outcome is going to be, right? There's


a lot of value in that. That's one of the, what's what I sent out in my recent newsletter


is the value of working on projects that might not work, right? So Kaggle is 100% not necessarily


necessary. I just think it's a useful resource. So level one would be courses, level two would


be Kaggle and level three would be working on your own projects. That's, that's my opinion.


Um, Envy says, do I really need a master's degree or computer science to get a job in


machine learning? I don't have a master's degree or a computer science degree. I have


a science degree with a major in food science and nutrition. I created my own AI master's


degree. You can Google that. Daniel Burke, AI master's degree, which is literally just


a medium article of, of different courses and why I'm doing them. Um, so no, you do


not need to get a master's degree to get a job in uni. I don't have, I mean, to get a


job in machine learning. I don't have one. However, that's not to say for every job you


don't need a master's degree. Some jobs may require one. Um, but if you don't have one,


you can still start to develop the skills and work towards ones that don't. And who


knows a lot of job listings, right? This is what I've seen with job listings will list


requirements that you need to be practically the best person in the world at, at five different


things, which first of all, no one is. So it's worth applying for the role, even if


you're not 100% ready, um, because requirements are often just like the pie in the sky. It's


like you need to have 10 years experience. You need to have built your own projects.


You need to, uh, worked on teams before that have, that have solved gravity, all these


things, right? Requirements are usually set the bar pretty high, but if you've got skills,


if you can communicate those skills, I'd argue that it might be worth you trying to apply


anyway. Um, Neha, Neha Gandhi says, what all frameworks should we know to do a computer


vision project? Well, there's open CV, Kerris, TensorFlow, glue on, Pytorch, Python, straight


Python. I mean, there's a lot of frameworks, but I think you can get very far using things


like just Kerris and open CV. Neha, I highly recommend the blog pie image search.com, um,


for, for anything to do with computer vision. I've learned a lot from there. Uh, Adrian


is the guy. Adrian is, is awesome dude. Um, Leon says, oh, hold on. Aravind, Leon says


any experience with sequential analysis, um, to be honest, Leon, I, I'm not entirely sure


what you mean by sequential analysis is in, um, sequence. We've been working on sequence


to sequence models in terms of speech to text at that Max Kelson lately. Um, so if that's


what you mean from sequence to sequence, um, that, that it is, or if you mean something,


or actually I'm not going to try and guess what you mean, but yeah, if you, if you, if


you mean something different to what I just said, let me know. Chris Highland says, love


your videos from Sydney. Thank you, Chris. Homegrown Aussie, Aussie, Aussie. That's the


power of the internet guys. This goes, uh, uh, this stuff, these it's crazy where how


far the internet can take your content. Like it's, it's actually amazing. I have people


from Germany, uh, from, from Africa, from Brazil, from America, from New Zealand, from


Asia, from India, of course. Um, it really, I really recommend if you're, if you're thinking


about making things online, you get to meet so many amazing people like yourself. So that's


just a little, a little sidetrack. I'm really, it really humbles me to be able to see like


interactions with, with my work from all over the world. It's actually amazing. I can't


believe we're growing up in a world we're living in such a time. Um, Ilken says, I am


working on my data science skills, but I want to get a certificate for abroad companies,


maybe Germany or America, because I don't have master's degree. Do you think this certificate


is enough or not? Ilken, I'm going to be honest. I'm not entirely sure, um, what requirements


are going to be enough for anything, right? It's hard to, it's hard to answer that. I


could say yes, get a certificate. Yes, get a degree, but it really depends on what you're,


what you're looking to get into. Um, America, I know you need a degree for some things,


to get a visa, but other than that, I haven't looked into it enough. So, um, I'm not very,


I'm not too much help on that question. Leon, did I write a paper already? No, I haven't,


I haven't written an official technical paper. Um, to be honest, it interests me, but it,


at the same time, it, it's, uh, I dunno if I'm, I don't know if it interests me enough


to want to do that. Like writing a paper is definitely very valuable, but, um, to me,


I like using the latest state of the art stuff to work on my own projects. Maybe eventually


I'll write a paper, but for now I'm happy to write, um, I want to start writing more


technical blog posts and, and just publishing code to, to GitHub and, and medium and, and


all these places online that are easily accessible. I think to me, writing a paper personally


is not the style of writing I enjoy. Um, that's just, that's just my, um, my take on it. Uh,


Swashstick says some of the dimensionality reduction technique like t-SNE has got fairly


advanced mathematics. Where can I get core mathematics for this advanced technique? Uh,


Swashstick Nayak, I, I don't have a great resource for, for learning the mathematics


of t-SNE. Um, I'm sure someone has done a pretty good blog post on it somewhere. So


best to, best to check out, maybe just Google that. Um, here we go. We're slowly getting


through these, slowly getting through these guys. I'll hang around. How long we've been


going for? I dunno. It doesn't really matter. I'll hang around for a, for about 10 to 15


minutes more. Otherwise then I got to go help, help with dinner with the fam, Sunday night,


family dinner. Um, how much software engineering did you have to learn on the job and how much


did you prepare for beforehand for the interviews from Lumina? Thanks for the question Lumina.


Uh, software engineering. Okay. I don't really have the traditional interview, um, story,


right? So I bought the book, Cracking the Coding Interview. I actually didn't read it


too much, uh, just because I was working on too many things. So I've got the book, but


I, I honestly didn't go through it much. Um, but what did I do? Here's, here's how it literally


went. I actually wrote this in a, um, I want to learn machine learning. How do I start?


I met with someone who saw my post on LinkedIn. I was posting about AI machine learning and


what I was working on. And then she actually, she messaged me and then I met with Mike.


Mike said I should meet Cameron. I met Cameron who works at Max Kelson. We still, we work


together now and cam. I met with Cameron on a Monday, told him what I was working on and


then he said, Hey, do you want to come in on Thursday and, and meet the team and, and


work on some projects? I went on on Thursday. I met Athen, who's the machine learning lead


at Max Kelson. I started coding some pandas, working on a, uh, just some exploring some


data that they had. Um, I did that on Thursday and then the next Thursday I went in again.


So this is like starting an internship, did a similar thing again on the next Thursday.


And on the Thursday after that, uh, Nick, who's the CEO of Max Kelson asked me, do I


want to, do I want to start? So I didn't really have a traditional, uh, software engineering


interview beforehand, but I've learned a lot since being on the job. And honestly, this


is why it's so hard to sort of say, um, I'm, I'm not really the greatest person to say


how to improve for how to prepare for interviews. All I can really say is what the skills I've


learned in the past year and has mostly been, uh, exploratory data analysis, writing reports


on what you found in terms of dealing with a client and just, yeah, then learning how


to best apply models to the, to the code you've, you've explored and cleaned and gone through.


Um, so hopefully that sort of shed some light on my story. I'm not going to claim that I,


I'm the best person to, to know how to prepare for interviews because my interview was kind


of like, um, just working, working through data with some pandas. Um, and then they just


said, do you want to start? And I said, yes. Um, Leon, there is a book on Amazon called


pro deep learning with TensorFlow and 50% of the book is math. Um, actually there's


a, if you, if someone Googles deep learning, the straight dope, I've seen that book seriously


get deep learning, the straight dope. And I'm, I'm, I might actually go through that.


There's too many resources I want to go through at the moment actually. Um, all right, here


we go. Let's get back to some questions. So Ab is there any scope of bioengineer or AI


of all machine learning? Um, well, I'm not sure what the question means, but in terms


of bioengineering, that's definitely something I want to get into. If you mean the crossover


of biology and, and machine learning engineering, I haven't quite worked it out how to properly


yet, but I'm all I know is that health and technology interests me a lot and I plan on


intersecting them at some point. Hold on. I got a book. There's this book. I'm really


excited to read this. Maybe you didn't hear that, but anyway, I'm really excited to read


this. It's called deep medicine. Um, I haven't started yet, but it just arrived. So this


is apparently how artificial intelligence can make healthcare human again. I'm actually


really excited to get into that. I'm going to start like reading books and doing book


reviews on the channel. Let me know if there's anything you'd like to see or if you'd like


to actually see any book reviews at all. But I plan on at least reading through that and


sharing what I learned from it. Lumina says, let me re-ask how good of a software engineer


do you have to be to be a part of a good way? How good of a software engineer do you have


to be to be a good machine learning engineer in your team? Uh, in my personal team, if


you can code and, um, if you've worked on some machine learning projects, you can, okay,


this is our latest hires. We hired, uh, people with maths and physics backgrounds. Um, a


couple of, a couple of interns, uh, studying quantum physics. So they're good with math


and whatnot. Um, so they're kind of learning code on the fly, but at the moment we're starting


to, we've, we've kind of grown a lot in last year. So we're, we're starting to more look


towards, I don't, I don't have a best way of explaining this, but if you've, if you've


got some good, good experience or good skills behind you, it's a likely fit. It's a likely


chance that you'll be good enough for our team because the way we sort of hire is by


hiring people who don't, maybe not necessarily have direct machine learning experience, but


have the ability to learn concepts in machine learning and apply them, um, in, in multiple


different ways because we deal with so many different industries. So that's kind of what


we're, where we look for at Max Kelsen is just people who have some kind of technical


ability already. Um, who knows what that might be. It might be a math background. It might


be a, a pure software engineering background. Uh, but what it, what's your ability to learn


that, that's what we're most, mostly interested in. Um, Sam says, I did Andrew Ong machine


learning course on Coursera. How much work do I need to do before I'm ready for an entry


level machine learning position? Um, Hassan, this is again, very hard to, to answer. I


mean, every entry level position will be, will be different. I would say start applying


and what you've learned in the Andrew Ong machine learning course into some projects.


Um, if you haven't got quite the skill to start building your own project, um, I would


argue that you probably do, but if you don't start with Kaggle, maybe do one to two, maybe


three like pipelines on Kaggle and then start on something completely, completely of your


own accord. Um, and then once you start to do that, you'll probably find that you, you're


probably starting to get ready for an entry level position. Make sure to share your work


to like put it somewhere where you can go, Hey, job recruiter for machine learning. Here


are the things I've done and then point them to your GitHub. Um, and don't just dump the


code, put some information. What are you working on? Why'd you work on that? Et cetera, et


cetera. A blog post on medium is great. You don't have to do all these things, but at


least somewhere where you can go, Hey, here's what I've done. Um, Bryce cheers from France,


man. Hey, awesome from France. Uh, I'm not even going to try to say something in French.


I kind of thought of saying something in French, but I don't want to, you know, those people


you say, Oh, I'm from, from here. And then they say something in your local language


and it's terrible. Right. Um, thanks Daniel. Appreciate a lot. Cheers, Ray. I really appreciate


it. My friend Lumina, by the way, cheers from New York city. It's 5 a.m. Oh, 5 a.m. New


York. I love New York so much. I'll be back. Um, uh, we have Albert from your video. I


see that you do a number of course in parallel, like ML, PI and deep learning. I know I can't


do that. What course do you recommend where to start? Let's say after Python, what next?


Okay. Um, if you have some foundation Python skills and a little bit of data, maybe not.


If you don't have any data, that's okay. I recommend if you've got some foundation Python


going to the applied data science with Python course on Coursera or, um, data camp and working


through data camps curriculum there. That's what I recommend my brother and he's, he's


learning data science at the moment. He's learning it from complete zero. He doesn't


have any Python experience, but I think they're two great resources to, to start getting into.


Um, and then after that it's, there's a, there's a bunch out there, but I would say that's


what I'm actually trying to dial back on is doing too many things at once. So I'm really


trying to sort of stick to one thing and it's hard for me because my personality, I'm just


excited by too much. Um, Kaggle series would be great. Yes. Thank you so much. I made videos


in ML too, but it slowed me down a lot. Um, Hey, Leon makes videos on machine learning.


So Leon Nelsden, that's awesome. Leon, go check out, go check out Leon's videos. Um,


but it slowed him down a lot. Yeah, you're right. Um, making videos takes a while, but


I enjoy it so much. I kind of see, this is the problem, right? It's a trade off. I go


through this a lot. It's like, I love making videos. I love writing. I love doing all these


things, but I'm trying to think, is it, does it take away making this stuff? Does it take


away from me actually getting better on the thing that matters? And that's a question


I'm dealing with a lot. So the thing that matters to me is getting really good at, at,


at the crossover of health and technology, um, or just, just health education for people


in general. That's, that's really where I want to go just cause fundamentally like I'm


interested in myself and making videos sometimes feels like to me it slows down the progress,


but I also love sharing what I learn. So that's how I sort of deal with that is that I love


learning to create and creating to learn. If there's a Japanese word for that somewhere


and someone knows it, cause I'm, I bet there will be a Japanese word for learning to create


and creating to learn. That is kind of my reason for existing. All right. Drew Stables,


how impactful was social media for you when getting your first job? I got my first job


through a message on LinkedIn. So, um, I would say LinkedIn is definitely the best, uh, for,


for anything to do with job markets, but, um, otherwise that was it. So LinkedIn, I


just posted a lot on LinkedIn. I still post a lot on LinkedIn. Um, type Norbert Kraut


on YouTube and watch the healthcare video with us. This is a future of healthcare and


many options to apply ML. Norbert Kraut. Leon, you're amazing. Thank you so much for all


these resources. Um, I got to type Norbert Kraut on YouTube. I might not watch it on


stream. I might watch it later and I'll, I'll talk, talk about in a second. I'm going to


YouTube. Maybe you can, you can watch this with me. Norbert Kraut. Um, maybe send me


that in a message. Uh, Leon, I'm not sure if I got the right video. Maybe it's this.


Is it this? I'm not sure. All right. Back to the stream. Okay. Um, Leon giving out a


lot of value. Uh, Shayik Assad. Sure. I have a lot of time these days. We didn't talk for


quite a while on Slack. Leon. Yeah. Leon's a legend. Yeah. Thank you for, for, for your


comments and your questions and your sharing Leon. I really appreciate it. Um, hey, Daniel


Burke. Ahmad says, Hey, Ahmad. Nice to meet you, my friend. I'd like to know how we can


deal with in-balance data. All right. That's really going to depend on the, on the data


set you have. Um, and I don't have any good examples, um, off the top of my head. So send


me a message, uh, Ahmad on telegram, t.me slash mrdberk and we can, we can talk about


in-balance data. Um, Manoj Dave. I'm learning ML since one month and everything is fine,


but I'm struggling with EDA. Any tips to improve EDA? Uh, Manoj check out my resource, um,


exploratory data analysis. If you just Google Daniel Burke, exploratory data analysis. I


wrote a, um, a blog post, a pretty in-depth one on it. Uh, I did a video as well with


exploratory data analysis. I'm going to do more on this cause I think it's such a valuable


step. Um, otherwise Daniel Thumoso has a great notebook on end to end, uh, exploratory data


analysis. Um, that the link to that will be in my exploratory data analysis resource.


So if you Google Daniel Burke exploratory data analysis, you should find some good stuff


there. I link some stuff. Also, my opinions, uh, sorry, Lumina says also any opinions about


data engineer. It doesn't seem like it's really taught well in YouTube. Must let uni. It doesn't


look so sexy of a job either. Um, it depends on what you mean by data engineer. Like if


I'm honest, I group artificial intelligence, machine learning, data science, data engineer


kind of into the, the one bucket that just, uh, there's otherwise there's too many things


going on for me. Um, reason being, and I know, I know, I know they're not exactly all the


same thing, but this is, um, a data engineer and a, uh, machine learning engineer to me


a kind of almost the same thing because if you're working with data and you're doing


say exploratory data analysis, you're a data engineer and then applying models and some


of your models, uh, have machine learning in them. Um, well, you're kind of machine


learning engineer and a data engineer. Um, so yeah, I, my opinions, it doesn't look so


sexy of a job. I would say, um, anyone who's going to get into the data science or machine


learning field has to have at least some data engineering skills on them. Um, Safi says,


Hey Daniel, have you taken Google cloud data engineering certification or any certificate


from GCP? Thanks. Um, Safi, I had the exam for the Google data professional certification.


Is that what you call it? Yes. I, I passed, I passed the exam. Um, I don't really like


exams, but I passed. It was quite difficult. I'm going to make a video slash blog post


on what I did to pass that. So stay tuned for that. Um, um, the first, the first, watch


it now. Far out. I've, I've, I've, I've done a terrible job at keeping up with this chat.


You've asked so many great questions. Um, I'm, I'm way back. My scroller is like halfway


through the chat. All right. I've lost where I was at. Um, all right. We'll probably do


three more questions cause otherwise I really got to go for dinner. If you've asked, um,


if you've asked something and I haven't got to it, thank you. Thank you so much. But we


might have to, uh, save that for another time. Um, I'm just going to start at a random place


of the chat. Sorry if I missed your stuff. Praful says, I've been self learning machine


learning for a month now. Awesome work Praful. Uh, I've completed machine learning course


from Coursera and the first two courses from deep learning specialization from Coursera.


Oh, that's not a question. You're just saying what you've done. That's amazing. Keep going,


keep learning. Um, so I'd like to know how you spend up the learning process. Praful,


I spend my learning process. If you check out the Coursera learning, how to learn video


I do, I did, um, that I use those tips for, for, for how I learn anything. Drew staples,


how do you solve the problem of people living longer, prolonging cell life, cybernetics,


cryo preservation, embedding with androids or other. Drew, that is an amazing question.


And I not going to even claim to know that I know, um, how do I solve the problem of


people living longer, prolonging cell life, cybernetics cryo observation embedded with


androids. Drew, I'm definitely going to be spending a large portion of the next 10 years


trying to figure things like that out. Um, at the moment I don't know, but stay tuned.


I will share as I find more. I actually listened to a great podcast on Joe Rogan with David


Sinclair who's trying to do similar things to what you just said. Um, if you Google Joe


Rogan episode one, two, three, four, I know so easy to remember. Um, it's a great podcast


on a researcher from Harvard who's actually working on a lot of those things and creating


companies out of that. Um, is your field only machine Frank says is your field only machine


learning applied to food science. Um, Frank, I'm not quite, quite sure what that means.


I'd love to start looking to use machine learning for things to do with food. That is on my


very, very long to do list. Uh, pro full says, how do you speed up the learning process?


Uh, before you don't trying to speed up the learning process. All right, here's, here's


how I think of it. How do I think of it in the short term, act quick in the long term,


be patient. This is how I, I have to remind myself a lot, right? So don't try to speed


it up too much. Otherwise you'll get frustrated. Right? What, what you have to do is be patient


learning the skills you want to learn because good skills take time to learn. But in the


short term, so day to day is don't be distracted by, by things that, that hinder your learning.


So in the short term, be fast at working on what you need to work on, but in the long


term, be patient because being great at something takes 10 plus years. Um, what do we have here?


Uh, Abhilash, Abhilash says, have I contributed to something like open source projects like


ML libraries? How do we get started for it? I haven't properly. Well, yeah, I haven't,


I haven't contributed to open source projects like ML libraries, but I really, I've got,


I asked myself the same question as you. How do I start? Because a lot of the stuff I use


has, has most, most of the functionality that I need, but I, you're right. I do need to


start contributing to open source ML libraries. Um, but it's just one of those things I haven't


got round to doing yet. Um, here we go. Lumina, thanks Daniel. You're really inspiring personally,


especially since you didn't come from a software background. Thank you Lumina. I really appreciate


that. Um, opinions, Atharava, Athavar, Athavar asks opinions on deep reinforcement learning.


Really really interesting, but I haven't quite seen it applied, um, to any like real world,


uh, uh, projects yet. So I really, really like it. Like I love, um, I love deep mind


stuff. I love open AI stuff. There's a lot of deep reinforcement learning there. Haven't


quite seen it quite yet. I think it has a lot of potential, but I haven't just quite


seen it assigned to a practical problem yet. I do sit next to a team of quantum. We call


them the quantum boys because they're, um, they're studying quantum physics at university


and we've sort of, we've taken them on and now they're working on using deep reinforcement


learning to, I don't know anything about quantum computing apart from what they've said to


fix error correcting in quantum computing. Now maybe I'm explaining it wrong, but, um,


you could probably send me a message on telegram, T.me or send me an email. You can find my


email online wherever you want. Um, I can send you their work, but they're using deep


reinforcement learning for quantum computing. Insane. Yash, what's up my brother? You're


late. No worries. Don't be sorry. Thank you for coming. I really appreciate it. Um, all


right, two more questions and then we got to go. Um, tips on creating brand online.


Ah, afar of us. Start making things and putting them on the internet. That's your brand. What


you do is your brand. Don't worry too much about the specific word brand. Just worry


about keeping doing what you enjoy doing and make good things and put them online. That


is your brand. Um, that's, that's the advice I have. Um, Leon says I should have watched


that video back then. Leon, I'm sorry. I'll watch it another time, but thank you so much


for the resource. Uh, all right. I'm right down to the bottom here. Here we go. Actually,


I think I'm down the bottom. Yeah, Leon, that sounds like a great idea actually. If you


want information about my resources, you could interview me if you want. Um, Leon, I'd, I'd


love that. Let's get an interview on the, on the channel of us, us having a conversation.


Um, if anyone, if anyone wants to see that, let us know. Leon has been a amazing, amazing


help in this stream. I really appreciate that. Um, Leon, I'd love to talk more. Um, do I


play games? I love games. I don't play them very, very much at all. Um, all right. One


more question. And then I got to go. I really got to go for dinner. I'm holding the rest


of my family up. Uh, here we go. Aravind, my boy says, Daniel Burke, bro, most of us


start with AI but end up in machine learning. Please do mention your opinion. Um, I think,


I think it's hard. My opinion kind of changes day to day, but at the moment it's like, okay,


AI, there's a lot of hype about it because it's, it's the media does a great job of making


it seem like this be all and end all thing, which I do think it is that, uh, which it's,


it's amazing. Um, but at the end of the day, a lot of these changes have come fairly quickly


and we're not, I don't think we're, it's at the moment it's, we've got a lot of solutions,


like a lot of powerful computing stuff to use, but not really, uh, we haven't worked


out yet. And I say yet because I think, I think we will write, this is the great challenge


is working out how to use these amazing technologies of AI to apply to problems in, in the world.


Right? So at the moment there's a lot of awesome work and a lot of awesome stuff being done,


but it's like having that awesome work, right? Is different to say having that accessible


to billions of people, which is a really hard thing to do, which is why companies like Google,


Facebook, and sort of that sort of stuff. Um, uh, I'm, I'm missing the exact way I want


to describe this, but it's why these companies, um, sort of keep getting bigger and bigger


and bigger what they're working on because they're the only real ones who have the ability


to deal with such a scale at the moment, right? At the moment. But as we learn more and as,


as companies like where I work and where I want to create in the future, as we sort of


learn how to apply these technologies properly to, to, to different problems, um, I think


that's why, where did, where did this end up? I think that, uh, that is where we're


heading, right? It's kind of like at the moment there's this big giant cannon, but we have


nowhere to shoot it at. Um, maybe that's a better analogy. Um, but as otherwise machine


learning, uh, has been around for a long time and is more easily applyable to existing problems.


Uh, maybe that doesn't, um, doesn't really answer your question, but that's my thoughts


on it at the moment. Um, Leon, absolute legend. Otherwise guys, thank you so much, um, for,


for coming around. This has been amazing. Let me know if you, um, have any questions.


I tried to answer as many in the last hour or so. If you have any questions that you'd


like answered that didn't get answered on stream, send me an email, Daniel at mrdberg.com.


I'll try my best to get back to you as fast as I can, but otherwise thank you so much


for, for your support. I really appreciate it. Um, as I keep learning more, I'll of course


be sharing what I learn. Um, yeah, I'm having a good time guys and I hope you are too. Um,


if you have any advice on how I can improve in the future, please let me know. I'm all


ears, but otherwise happy Sunday. Keep learning.


All right. Now I just need to figure out how to end this stream. Oh, by the way, my, my


telegram for those people who are asking is T dot me. You can message me here straight


to telegram T dot me slash mrdberg. I'm going to put it in the chat and then I'm going to


end the stream. T dot me slash mrdberg. Let me, let me type it into, to the website for


you. T dot me dot mrdberg. If you want to message me, this is my telegram, right? This


address here. Look at that picture. All right. Send me a message if you, if you want to talk,


um, offline about whatever you want, I'll do my best. Otherwise email daniel at mrdberg.com.


The most valuable skill I've learned

Yo, what's going on everyone? Welcome back to day 98 of the Daniel Burke show. Thank you so much for tuning in whatever you're up to


The most valuable skill I've learned

What are we doing today? So this morning? I got up. I did some study again doing this machine learning course on Coursera


The most valuable skill I've learned

It's free. It's by Stanford University. I highly recommend it


The most valuable skill I've learned

Give it a go if you're looking to get into the field of machine learning


The most valuable skill I've learned

I know it's an extreme niche at the moment


The most valuable skill I've learned

But if a lot of what I've been reading the world is being taken over by software because there's so much tech out there and software


The most valuable skill I've learned

is just taking over the world every every industry has been dramatically changed by by software and


The most valuable skill I've learned

machine learning is


The most valuable skill I've learned

Eating up software, so I think I know if the internet was a massive boom


The most valuable skill I've learned

if


The most valuable skill I've learned

websites were a massive boom I think


The most valuable skill I've learned

Artificial intelligence and machine learning is going to be the next massive boom and virtual reality like this is just all my intuition


The most valuable skill I've learned

by the way, I might be completely wrong, but I'm


The most valuable skill I've learned

Thoroughly enjoying it and I've been reading some articles on what AI can actually do right now


The most valuable skill I've learned

One of the headlines I haven't read the full article


The most valuable skill I've learned

But the headline was if it takes a human about less than a second or around a second to do


The most valuable skill I've learned

A computer can probably do it so any task that takes a human being a bit longer than a second


The most valuable skill I've learned

I


The most valuable skill I've learned

Probably can't do it just yet, so that's what I've been up to most of today


The most valuable skill I've learned

I spent fully at least six hours studying


The most valuable skill I've learned

I find once I hit a certain point like it just doesn't stick anymore like there was a quiz actually actually I failed three


The most valuable skill I've learned

Times today, I need to try again in about in tomorrow morning. I can only try


The most valuable skill I've learned

Three times every eight hours. I think so I failed this quiz on my machine learning course


The most valuable skill I've learned

It's only five questions, and I need to get 80% I got three out of five three times


The most valuable skill I've learned

So I don't know. I don't know what's going on. I failed but


The most valuable skill I've learned

Every time I think I got a different question wrong or right. I'm not sure


The most valuable skill I've learned

I'll work it out. I know I can pass pass it tomorrow sometime


The most valuable skill I've learned

but


The most valuable skill I've learned

Yeah, I failed I found it was actually is actually it sucked


The most valuable skill I've learned

but it's a good feeling at the same time because I feel like when when I used to do a


The most valuable skill I've learned

University exams and I failed I never got really feedback on on why I failed whereas I feel like an


The most valuable skill I've learned

to me an online course is I


The most valuable skill I've learned

get instant feedback and the fact that I can chat in the forums and and learn where other people have gone wrong and and


The most valuable skill I've learned

Ask questions and whatnot and get instant feedback. That's such a valuable education resource


The most valuable skill I've learned

There's so many reasons online to learn right now


The most valuable skill I've learned

Coursera you to me Udacity all these all these places a lot of the courses are free or cheap


The most valuable skill I've learned

I paid I think ten dollars for a


The most valuable skill I've learned

70-hour course like that is that is incredible and


The most valuable skill I've learned

What what are my plans? What do I want to do with this?


The most valuable skill I've learned

Well as I want to help the world move more I'm building an artificial intelligence based messenger bot to do that


The most valuable skill I've learned

Through Facebook messenger


The most valuable skill I've learned

You can find out more details at my Facebook page facebook.com slash mr. D Burke


The most valuable skill I've learned

Really because I feel like a lot of the the massive communication technologies like a lot of the things a


The most valuable skill I've learned

Lot of the problem we have problems. We have our communication problems, and if you can you can nail communication


The most valuable skill I've learned

Between two people between education or something like that if you can tailor


The most valuable skill I've learned

If you can tailor tailor education to someone if you can tailor communication to individuals


The most valuable skill I've learned

Then then you're in a good place like what what Elon Musk is trying to do with neural link is is reduce the bandwidth


The most valuable skill I've learned

On I start increase the bandwidth of communication


The most valuable skill I've learned

so


The most valuable skill I've learned

if you think about communication, I think of a thought and


The most valuable skill I've learned

For me to communicate that to you guys. I have to think of that thought


The most valuable skill I've learned

Translated into speech in my mind and then speak this speech with my mouth you


The most valuable skill I've learned

the sorry the microphone records that speech and then it comes out through your speakers or however you're listening or watching this this piece of


The most valuable skill I've learned

media then your eyes or your ears


The most valuable skill I've learned

absorb that communication into your brain and then it


The most valuable skill I've learned

decipher that and then you create a thought of what I I


The most valuable skill I've learned

Communicated to you and so what I'm thinking in my head may be totally different to what you're thinking


The most valuable skill I've learned

after I communicate this and so


The most valuable skill I've learned

imagine if that communication bandwidth that it was just


The most valuable skill I've learned

completely opened up and so if I thought of a thought you could tune into that thought straight away and


The most valuable skill I've learned

And you could think the exact same thing as me you could experience the exact same thing as me look


The most valuable skill I've learned

It's incredibly hard thing to do, but I believe with enough hard work with enough. I know


The most valuable skill I've learned

Advances and technology and whatnot that's that's where we're going one day that may be even within my lifetime that I'm able to


The most valuable skill I've learned

Have a thought and then someone else on the other side the globe even because we're all connected through satellites or something like that


The most valuable skill I've learned

be able to see that thought as well and


The most valuable skill I've learned

it's it's


The most valuable skill I've learned

One of the what are the implications that what are the ethics?


The most valuable skill I've learned

How can what what are the chances of someone else getting inside my head well?


The most valuable skill I've learned

I think it'll almost be the same as


The most valuable skill I've learned

If someone was already talking to you can you can you block them out or something like that?


The most valuable skill I've learned

I don't know. I don't know I need to learn more about this myself


The most valuable skill I've learned

But where's all this going? I think


The most valuable skill I've learned

Communication number one thing it's like the root of the root of many of the problems in my life have been because because of communication


The most valuable skill I've learned

Whether it be relationship issues friendship issues


The most valuable skill I've learned

Everything like that. It's because it's come down to some sort of communication issue


The most valuable skill I've learned

I forgot to ask for something I forgot to to talk to someone about something


The most valuable skill I've learned

I didn't say the right thing at the right time. It's it's all I know this is this is what I just keep


The most valuable skill I've learned

This is what I've been finding. I don't know if I'm right, but it's just just a recurrent pattern in my life


The most valuable skill I've learned

It's like whenever I forgot to ask for something or whenever I didn't ask for anything or whenever I didn't


The most valuable skill I've learned

Say something I thought I should have said or I said something wrong like that's that's that's what's gotten me into trouble


The most valuable skill I've learned

And that's what's caused issues in my life


The most valuable skill I've learned

so I think if we can solve communication and we solve a lot of things but


The most valuable skill I've learned

Question is how do you do it who knows so what's the day's daily challenge?


The most valuable skill I've learned

Think about a way that you can


The most valuable skill I've learned

More effectively communicate with someone else so is there I know your spouse your husband your your spouse your husband. It's it's late


The most valuable skill I've learned

I'm sorry


The most valuable skill I've learned

That's no excuse by the way no excuse and


The most valuable skill I've learned

So whether whoever it is in your life


The most valuable skill I've learned

Could you say out hello to them if you haven't talked to someone in a while could you say hello to them or even if?


The most valuable skill I've learned

It's just maybe once a week you you ask your mom


The most valuable skill I've learned

How are things going what not and I need to do this too. I think I need to do this too


The most valuable skill I've learned

So maybe this is the daily challenges me sublimely telling me what I should actually be doing


The most valuable skill I've learned

That's crazy right well. I am in a car park and recording a podcast and recording this video so maybe that's crazy, but


The most valuable skill I've learned

oh


The most valuable skill I've learned

I'm having fun, but whatever you're doing have an awesome day have an awesome night


The most valuable skill I've learned

Have an awesome day have an awesome night. What if you're listening to this on the treadmill or driving in the car?


The most valuable skill I've learned

I thank you so much for tuning in thank you so much for watching call someone telling you love them, and I'll catch you tomorrow


The most valuable skill I've learned

It's day 99. I believe we have two more episodes to the experiments over, but don't worry. We're gonna keep going with this


The most valuable skill I've learned

We're gonna keep going with this it won't be daily, but I will have a podcast not sure what it's about


The most valuable skill I've learned

Maybe it'll have a rebranding. Maybe it won't be the Daniel Burke show. I I need to work that out, but that's a problem for future me


The most valuable skill I've learned


FNet: Mixing Tokens with Fourier Transforms – Paper Explained

Hello there! Did you really believe that attention is all you need?


According to the papers of the last weeks, not anymore.


In today's video we showcase the Fnet paper replacing the self-attention in BERT with Fourier transforms.


Fourier what?


Miss Coffee Bean, did you know it all along that self-attention is doomed?


Then why didn't you tell me? Is it because I left you in the coffee roaster for such a long time?


Here we go.


The Fnet proposed in this paper does one modification to the transformer architecture,


but this one thing has quite the impact.


For a bit of a context here, the transformer, composed of self-attention, followed by fully connected layers,


and not to forget about the residual connections and layer normalization,


have been proven to be extremely successful on multiple data modalities.


It started with text, became mainstream on both image and text, then on image only,


watch Miss Coffee Bean's videos for all of this, then it moved on to speech and even further than that.


But powerful as it is, the transformer has a problem.


The self-attention mechanism scales quadratically in time and memory with increasing sequence length.


In other words, there is a limit to what the length of your text might be.


For images, this size or resolution also determines the sequence length,


and with usual transformers you can reach the sequence length limit quite fast.


So far, methods to reduce this quadratically scaling attention to a linearly scaling operation


have left the concept of attention untouched.


Methods like Linformer, Performer and others have cleverly approximated self-attention matrices,


sped up computations and enabled much longer sequences,


but the concept of attention stayed mostly as it was.


Well, so far. Because now heretic papers are coming out and rebel against attention.


No, it's not attention that we need anymore.


Miss Coffee Bean is machine learning now a teenager and doesn't want attention anymore?


Because this is exactly what Fnet also does.


It replaces the self-attention layers through Fourier transforms.


But the most important question here is how dare the authors make such a move and not call their paper


Fourier is all you need.


The audacity of these authors is unbelievable.


CERKASM


Okay, so let's see how this replacement of attention works and what it brings.


The Fnet authors take the classical transformer block and modify it by replacing the self-attention layer


containing learnable parameters with an unparameterized, thus not-aligned transformation being the Fourier transform.


So before, for each token, self-attention layers were learning to compute importance scores for the token given all other tokens.


So to emphasize, the computation of the importance is learned from data, is task specific.


And the new representation for each token was the sum of all tokens weighted by their importance to the original token.


Now with the Fourier transform, the new representation for a token is not learned, but it is a predefined mixing scheme of tokens.


This makes this whole thing faster because this layer doesn't perform any gradient updates, but is also unchanged from task to task or one training scenario to another.


The mixing scheme of the tokens is following the rules of the Fourier transformation.


Now, what exactly is this?


This is where the math gets a little complicated, but it does not need to scare you.


Here it is just written that each token is multiplied by sines and cosines because this e to the power of something we can write as a superposition of sines and cosines, hand-wavy, like this.


So for example, the first token is a weighted sum of all other tokens, but here the weights are modulated by sines and cosines.


But because we are at the first token, where k equals zero, the frequency of the sines of cosines hidden in this exponential is zero.


So the first token is just the sum of all other tokens.


For the second token, the modulation is more interesting by superimposing waves that increase in frequency as we add more tokens.


For the third token, the frequency of each wave we add increases even more, like this.


To put it as an analogy, the new representation of each token is like a cake, where the base frequency is still the first token, then the second token adds some detail and becomes the cream on the cake,


then the third token becomes the even more detailed icing, the fourth token is the super detailed nuts and so on, and then the whole cake is the new token representation, because we take the summation of all of this.


And to be fair, all of this has real and complex components, but forget about it, because after all the operations are computed, the authors delete the imaginary component.


Anyway, the important messages here are, firstly, the Fourier transform applied to tokens mixes the tokens among themselves.


It's kind of similar to what self-attention does, but self-attention learned the weights by which these things need to be mixed.


Also, the same thing is done to the hidden dimensions, so that the features of the tokens are also mixed among each other.


And until here, the transformation that was done is reversible by the inverse Fourier transform.


In other words, the components of the cake can be recovered by decomposing after each frequency.


It is the next step, the feed-forward layers that mix things even more and usually not in an invertible way.


The second important message is that the mixing scheme or pattern is not learned, it is fixed and always follows the same rule, making the architecture simpler and easier to compute.


It decreases the number of the learnable parameters of the transformer, which now is technically Fnet and not the transformer.


Thirdly, a more advanced take-home message, also mentioned in the paper, which Ms. Coffee Bean found super interesting, is the following.


You maybe know, or if not, just take Ms. Coffee Bean's word for it.


Convolutions in the real-slash-time domain correspond to simple multiplications in the Fourier-slash-frequency domain.


So the first Fourier layer of the architecture moves the representation from the real domain to the Fourier domain and multiplications in these layers happening afterwards are now actually convolutions in the real space.


So by adding more and more transformer layers, each one with one Fourier layer, Fnet is jumping back and forth from multiplications to convolutions.


In other words, convolutions are somehow here and perhaps now are taking revenge on the transformer and crash their party.


Are convolutions all you need? I actually cannot hear all you need anymore, but Ms. Coffee Bean makes me say it.


Okay, but one question remains. Why exactly the Fourier transform?


Reading the paper, it becomes clear that the first thing the authors tried was simple linear mixing schemes, but then went with something more structured like the Fourier transform.


But believe it or not, the other paper getting rid of attention, the MLP Mixer, learns mixing patterns that look very much like Fourier.


Just look, here is the base frequency, here is the double of that frequency with its negative counterpart.


The MLP Mixer found something like a Fourier scheme on itself, but on images.


Fnet has been applied to text and has the Fourier transformation hardcoded and it seems to work also well.


So the Fnet basically is a BERT variant without attention, but Fourier transformations and achieves 92% of the accuracy of BERT on Glue, an important NLU benchmark.


Which is not exactly 100% of BERT, but good enough, and especially because there is no quadratically scaling attention anymore.


It runs 7 times faster on GPU and 2 times faster on TPUs and you can increase the sequence length.


So self-attention was praised as being the recipe to success for quite a while.


But now papers like Fnet, MLP Mixer and GMLP relativize this by a bit, showing that if there is enough data and compute, the architecture does not matter much.


So it seems like back in the days, when MLPs were a thing, we discarded MLPs too early, when perhaps the only thing that was missing then was the data and compute scale we have now.


So we preferred CNNs or RNNs because of their inductive biases. Always a good idea with little data.


But to say this is very unfair, because architectures like Fnet or MLP Mixer are not exactly like a vanilla MLP from back then.


The structure of these MLPs follows the trend set by transformers. So Ms. Coffee Bean sees this as an evolution and not really as a backwards involution or something.