How I Designed My Own Full-Stack ML Engineering Degree
By: Logan Yang
Tags: Machine Learning, Artificial Intelligence, Full Stack, Data Science, Machine Learning Engineer

Earlier this year I came across a viral Twitter thread by Randall Kanna about how to create one’s own computer science degree with free online content. It was not only excellent for people who don’t have prior knowledge in computer science, it’s also valuable for new software engineers who didn’t major in CS in college. That’s when I thought about creating my own full-stack machine learning engineering degree. After being in the tech industry and having learned it the hard way, I believe a custom-designed curriculum is going to be valuable to myself and others who have similar goals like mine. I curated a collection of the best resources I found, wrote down a curriculum, shared it on Github and Twitter where I called it “My CS Degree” with a focus on Full-Stack ML Engineering. Since then, a lot of people have starred and forked it. I’m glad that they found it helpful.

In this article, I will explain the philosophy behind this curriculum, who can benefit from it, and how to reason about what and how to learn given your personal goal and the exploration-exploitation dilemma in the vast world of knowledge. If you are only interested in the actual curriculum, check it out on Github.

Disclaimer

This is not trying to be a one-size-fits-all machine learning degree. There are different career paths in machine learning. Everyone has their own background, career goals, and interests. The diversity in the field is healthy and should be encouraged. What shouldn’t be encouraged is the wrong organizational separation of responsibility. But more on that later.

The Target Audience

Before you read on, this question must be answered first: what is the goal of this curriculum, and who is it for?

If you want to build and ship machine learning powered applications end-to-end, from problem formulation to production, this curriculum is for you.

You can apply the skills learned here to find a job, or bootstrap your machine learning startup.

Why Should You Care

The field of data science and machine learning engineering is enormous and fast-evolving, nobody can learn everything. Traditional universities are too slow to design new curriculums. At the same time, there are tons of world-class educational resources online. The advantage of designing such a customizable online degree is easy to see. Even so, it is very hard for beginners to design such a degree for themselves because what they often see is stuff like this:

119 easy steps to becoming a data scientist

The wrong way to start is to try to cover as many points as possible in that subway map. In addition, if you want to become end-to-end, you need a similar map but for software engineering as well! A sane person would immediately see through the complexity on the surface and identify the real question:

What is your goal and how to determine relevance and priority based on it?

Having worked as a software and machine learning engineer for several years, I have developed my philosophy of learning in this field. I decided to write about my experience and share some advice with newcomers on what not to do, and what to do instead.

The Controversial Separation of Responsibility between Data Scientists and Engineers

Why the emphasis on “end-to-end” and “production”? Some data scientists build models and hand them off to engineers to productionize later. This is how data science organizations operate in many companies now and it is gradually recognized as an anti-pattern for those who need models in production. In Eugene Yan’s excellent article Unpopular Opinion — Data Scientists Should Be More End-to-End, he explained why it is the case eloquently. I completely agree and won’t repeat it here.

One major reason that this separation exists is the distribution of expertise in the talent market. Internally, it becomes perceived expertise based on job titles and descriptions. The average data scientist has a degree in statistics or other math-heavy fields with less knowledge in computer science. An engineer usually has a CS degree but lacks modeling expertise. Do you see the difference? It is a legacy issue caused by traditional universities. For most companies, there aren’t enough talents who learned both well enough in school or from previous experience. Even if the talents exist, a wrongly structured organization tends to underutilize them, or even pit data scientists and engineers against each other. It is time to rectify it with a set of more up-to-date expectations.

Nowadays, it is not unreasonable to ask for both software engineering and machine learning skills from one person. Since both are very big fields, it is unreasonable to expect deep knowledge in every domain, but the person should have a functional set of skills where they can handle end-to-end workflows in their strongest area reasonably well.

This isn’t to say we don’t have skill specialization anymore.

Think about it this way: each person has a relatively constant skill capacity that can be spread out to different topics. Traditionally, people’s skillsets form a software engineering cluster and a modeling cluster. As the space gets more complex, the boundary between these two clusters doesn’t make sense anymore. A person can be deeply specialized in a collection of topics in both clusters. The total volume they cover in this space is still roughly the same as traditional university-educated people, but the possible combinations of topics and depths increase significantly.

This increased diversity of expertise is a very good thing for creativity as well. Steve Jobs famously said:

If you’re gonna make connections which are innovative … you have to not have the same bag of experiences as everyone else does.

To be more in-demand in the current job market by being end-to-end, and not subject yourself to the limits imposed by traditional educational systems, you need to be result-driven. Recognize your personal interests and the trend in technology, create our own path that leads to value creation.

My Background

A little digression before I dive into the structure and philosophy of this degree. I have an undergraduate degree in physics and used to study machine learning in grad school before deep learning became the hottest thing. I didn’t study computer science systematically but did take courses in algorithms and data structures. So you may see me fall into the “data science” camp mentioned above in terms of education. As for real-world software engineering and web development, I was clueless when I graduated.

With a short period of intense preparation and some luck, I started my career as a software engineer at a Silicon Valley company, working on full-stack web development with some of the top engineers in the field.

Nobody had a bigger imposter syndrome than mine.

I went into the office every day feeling extremely insecure, grinding my way through each line of code, and somehow made it work without understanding the big picture how each component talked to each other.

It was initially painful but ultimately rewarding. I was lucky to have worked with an amazing team and learned a ton. During that period I also tried to make contributions to machine learning projects that were outside my team’s area. I began to realize the importance of software and data engineering. It is the most important aspect of any project, machine learning or not. Ask any machine learning professional, they will tell you that building models is just a tiny part of the job. Building something that people can actually use is almost an entirely different problem.

After that, I joined an early-stage startup to experience what it’s like to be one of the founding members of a machine learning team. This is the period when I realized that formulating the right problem based on business use cases is the hardest of all tasks. The next hardest is how you collect and engineer your dataset. In all cases, modeling is a relatively minor factor of success, which might be very counterintuitive to beginners who came from classrooms where datasets and problems are well-defined and handed to them.

The road to a full-stack machine learning engineer may not look as glamorous as becoming a rockstar researcher. The current machine learning education online and offline often don’t even mention these factors that are more important than modeling in real-world ML applications:

Computer science fundamentals and software engineering

Understanding how to formulate the problem correctly

Designing a system that works reasonably well in production with constraints

So if your goal is a career in building production-ready machine learning applications and not academic research or Kaggle competitions, the priority is to strengthen these areas and not to follow the next shiny model or tool every couple of weeks. This leads to the philosophy of how I structured my curriculum.

The Structure of the Curriculum

There are 2 main ways to categorize the courses in this degree.

1. Group by Bottom-up or Top-down Learning

There are two types of courses in this full-stack ML curriculum: general knowledge courses (bottom-up) and project-based courses (top-down).

General knowledge courses are for indexing knowledge in the brain into an organized, connected, and easy-to-search system. This is bottom-up learning and it is best when limited to the basics.

Project-based courses are where the real learning is. Come up with a small product you want to build, do some research on the necessary parts to learn, learn them on-demand as you build it up. When you are done, deploy it and try to get some users. Write an article teaching others by explaining the process in detail. It could become more useful than your resume in the future! This is top-down learning, an approach that is most effective in learning anything practical but is usually not adopted by traditional schools.

2. Group by Domain

Another way to group the courses is by knowledge domain. My degree has these major domains: computer science fundamentals, deep learning fundamentals, software engineering (including MLOps), and natural language processing.

Being end-to-end usually means you need to write production services. This requires you to understand the basics of computer science:

How does the computer work?

Courses: Computer Architecture, Operating Systems

How does the Internet work?

Courses: Computer Networking, Distributed System Design

How do databases work?

Course: Intro to Databases

How do some common algorithms and data structures work?

Course: The Coding Interview

There is no need to understand every topic in-depth, but the key ideas are needed to be effective as an engineer. After going through these courses, at least you know what relevant info to look for when facing a new problem at work.

For machine learning and deep learning, I highly recommend fast.ai. It is the best top-down teaching on the internet and it’s free! Jeremy packs rich content in the lectures, you’ll have to watch the lectures more than once to get the most out of them. It all pays off when you build your own deep learning project using fastai techniques to achieve state-of-the-art results.

For technical interviews, I recommend the book Elements of Programming Interviews. The goal is not quantity but quality. Once you understand the common algorithms, most problems are just their variants.

For production machine learning / MLOps which is the core of this degree, there is an online course currently being released (as of Nov 2020) Applied ML in Production created by MadeWithML. This is a rare gem that lets you take a close look into how a real production ML system is built from scratch. Check out other MLOps courses in the curriculum to complement your learning.

On top of these domains, you can pick your specialization. My interest is in natural language processing and sequence modeling, so I added a project course for it. You can come up with your own and use it as a capstone project, showcase your work to the world once you are done.

This is just a quick recommendation for each domain, there are a lot of other great courses listed on my Github page. The curriculum is open-source so you can fork it, trim down the ones not relevant to you, or add more as you wish.

Bottom-up or Top-down? — My Learning Philosophy

Neither bottom-up nor top-down is the “best way of learning”. They serve different purposes.

Top-down learning is usually not adopted in schools. The textbooks and lectures always have an organized flow of logic. Necessary knowledge points are introduced before more advanced topics, and it truly had an effect on me — it made me feel uncomfortable dealing with a complex task in which I had gaps in knowledge. It made me naturally want to understand everything before I’d start the task. Ironically, almost all real-world problems are complex and you often don’t understand every underlying mechanism. But you don’t have to know everything to do it! The stark contrast between school and reality shows that top-down learning needs much more attention. Without it, a lot of potentially productive people don’t even start.

On the other hand, there are cases where bottom-up is necessary. For instance, if you learned to code without learning computer science basics, you quite possibly will never encounter a problem that explicitly requires you to learn them.

You may become a somewhat competent web developer or data scientist without knowing how the computer works, why a bunch of transistors become programmable, what it takes to make the computer understand your code, or what makes the code efficient, despite it being your workhorse and the greatest machine mankind has ever created.

This is harmless to an extent. Many coders who didn’t major in CS made it in the tech industry without needing to master the basics, myself included. Yet, I can tell you that it does make a difference long term. It limits the things you can do and how good you can get. So if you are like me, taking the time to learn the basics is worth it.

Another example that favors bottom-up learning is math — the core of machine learning. Math is the subject where you work with a set of axioms, operations, and theorems as building blocks to solve new problems you’ve never seen before. The magic about math is that you can pretty much solve any problem once you master the logical building blocks, it’s just a matter of how to use the building blocks to get there (this statement has a caveat, Google “computability theory” if you are a nerd).

Algorithm is another form of math. When you learn the likes of depth-first search or breadth-first search, you can generalize and solve problems that are difficult to solve otherwise.

Enough with the philosophy, but when to use which?

When dealing with a complex system, it is not always possible to decompose it into clear building blocks that follow a strict line of logic. In this case, top-down is better. You try what works and what doesn’t in order to get a high-level feel of how to do it correctly. Examples include learning to play tennis, to swim, to recognize an object in an image (computer vision), or to build a new product (entrepreneurship).

When preparing for a coding interview, however, internalizing and generalizing the algorithms and data structures is key. It needs a more bottom-up approach.

Avoid the Traps

When there is an abundance of great resources, it’s easy to fall into the traps of the learning process. Here are some points I frequently need to remind myself of.

Trap 1. Learning it as a University Degree

Don’t do this as a university degree. You don’t need to finish one course after another religiously in order to graduate. Think about it, what are the things you can still remember from your school days? It could just be a handful of mental models and a personal way of learning new things. The reality is, you won’t remember much from this curriculum either. So do this instead:

Distill knowledge into mental models in short notes that are easy to remember. Be result-driven. Which domain is the one you most urgently need to work on? Pick one relevant general knowledge course and one project course at a time. Ignore others.

Trap 2. Lack of Positive Feedback

When you are starting out, set some humble goals to feel good about your progress. If you get discouraged too often, you won’t be able to make it through this long journey. You have to enjoy it to go on.

Trap 3. Chasing after Trendy New Things

When you are used to curating good resources, you may get distracted every now and then just to find new things to collect. In the field of machine learning, there are countless new papers, new tutorials, new tools coming out every day. If you constantly chase them, you will feel insecure forever.

Learning a new thing is an investment. It is a much better investment to first learn the fundamentals well because they stay the same and won’t be outdated for a very long time.

For example, there’s no need to feel insecure if you are not familiar with Spark when you don’t use it in your job. Instead, learn the basics of distributed computing. It will supercharge your future learning of any big data framework.

We need to ground ourselves every day, to cool our heads in the era of information overload and focus on a small set of truly important things.

Trap 4. Not Showcasing Your Work

If you are like me, we always feel insecure about the quality of the work we do. There are brilliant people out there who are more experienced and knowledgeable, what if my work is not good enough and people will laugh at me?

The truth is, people don’t care. We shouldn’t take ourselves too seriously either. Everybody is a student. If someone doesn’t think they are a student, they are not growing.

We don’t need to compare ourselves to others, only compare with our old self.

Another reason that showcasing your work is crucial is that the current ML hiring process is broken. Many companies can’t find qualified candidates, and candidates can’t stand out just with their resume. With a portfolio, you not only show that you are competent in what you do, you also show your passion and dedication in this field. All significant events in life are unpredictable. It may surprise you with exceptional opportunities someday!