The Best Way to Learn Data Science
By: Jahnic Beck
Tags: Machine Learning, Learning, Data Science, Editors Pick, Education

The Best Way to Learn Data Science

From a non-traditional background

Image by Arek Socha from Pixabay

Not too long ago I started my journey, transitioning from a non-traditional background into data science. Particularly at the very beginning, the path ahead appeared daunting. Linear algebra, statistics, probability, programming; the requirements seemed endless. Where to start? What resources to use? I had to find a path forward through trial and error, repeatedly adjusting my priorities according to new incoming information. Looking back, there are a few things that I would have done differently. In this article, I am going to share some of my key insights that I would have liked to have access to at the beginning of my journey. I will provide resources that I would use if I had to start all over again and establish a framework to develop technical skills in the most effective and most enjoyable way that I came across.

Image by birgl from Pixabay

Curiosity is key

Before I move on with the topic of the post, I would like to briefly discuss the importance of curiosity. In my experience, it is an often-overlooked tool for learning. Difficult subjects require critical appraisal. Without going through the process of posing questions and seeking out answers, any learning is doomed to be shallow. You risk memorizing rather than understanding, which will hamper your ability to apply the newly learned information appropriately. If you do not understand the meaning of a p-value, you risk accepting meaningless or fraudulent results based on statistical significance.

There is a litany of scientific studies that link curiosity to enhanced learning performance, memory formation, and creativity. If you are interested in the topic, “The Business Case for Curiosity” in Harvard Business Review would be a good starting point. Lastly, delving into any vast topic such as data science requires continuous and long-term effort. Without motivation you are likely to give up somewhere down the road. If you remain curious, you will continue looking for answers, stay motivated and keep on track for success.

Easier said than done

Now, the question becomes, how to optimize your learning path. How can you ensure not to squander your cautious curiosity in the face of inevitable opposition? Your learning path should be tailored towards your individual preferences. However, there are some general rules that apply.

1. Take a top-down approach to learning. This is something I had to learn the hard way. I find mathematics fascinating, but this is not where you should start. Data science is an applied discipline, familiarity with the tools that you will be using should be your first objective.

2. Establish the basics within a short time frame. We learn best when we connect new information to prior knowledge. Establishing a basic understanding of data science and machine learning early will boost your learning trajectory. To do this, you can use any popular language such as R, MATLAB, or Julia. However, Python is the gold standard in the industry and will give you the greatest advantage. An excellent resource that I came across far too late are the Kaggle micro-courses. Short, to the point, and entirely free, they will get you up to speed in no time.

3. Start working on personal projects early on. This is what I will focus on for the rest of the article and I will make practical references to my most recent project.

Photo by Daria Nepriakhina on Unsplash

Explore data relevant to you

Once you establish the fundamentals it is time to pick a personal project. Create insights into a topic that you are interested in or build an application that does something useful for you. If you work on issues that are relevant to you, you will be motivated and curious about the results. It is also likely that you will have some domain knowledge that you can use in your analysis. Lastly, if you solve a problem in your life, there is a good chance that others will be interested in your solution as well.

The key insight that I was missing and that was preventing me from starting to work on personal projects was how to obtain pertinent data. There are many free data suppliers that you can use such as Kaggle or various governmental sites. However, collecting your own data is infinitely more interesting and gives you more options. You can use resources that are more relevant to the problem you want to solve. Once you learn about web-scraping the sky’s the limit. You can scrape data from forums, news-sites, e-commerce platforms, sport-sites, finance resources and many more. Not all websites allow scraping, but you can always find alternatives and should be able to build something interesting on almost any topic.

A secondary benefit of collecting your own data is that you will need to go through a cleaning and feature engineering process. Over 80% of a data scientist’s time is spent on data collection and wrangling. If you only work on clean data, you will miss out on these two integral steps.

Start your project

For my most recent project, I decided to create a model to predict real estate prices from my home city of Montreal. Predictions considerably higher than the actual price would imply that comparable listings tend to be more expensive. Therefore, by exploring over-predictions undervalued properties may be discovered. My resource of choice was a popular Canadian real estate website. Consequently, I had to employ web scraping. The two most popular python packages for obtaining data from websites are Beautiful Soup and Selenium. Beautiful Soup is faster and easier to work with, but Selenium has more capabilities. If you need to interact with java-script objects, you will have to opt for Selenium. The real estate websites I came across were packed with clickable buttons and other interactive objects. Selenium was, therefore, inevitable.

Once you collected your data into a nice tabular form it’s time to explore the fruits of your hard work. The goal should be to familiarize yourself with the data and to look out for potential issues. Some recurring problems that you are likely to encounter are duplicate data points, non-numeric data, and columns with many missing records.

Duplicate and non-numeric data are relatively straightforward to deal with, for the missing records things might be trickier. The following are some questions worth exploring for each column.

How many records are missing? How relevant is the feature in relation to its purpose? Can you substitute missing data with information from other columns or from a different data source?

To answer question two, you could either draw from domain knowledge or try to infer the answer from the data. Scatter plots, correlation matrices, and permutation feature importance are great techniques to help you out. This is comparable to the steps you would take during feature selection. Should your analysis reveal an important feature, with many missing records (~5–40%) and no means of substitution, you may resort to imputation. This is best done by creating a predictive model to estimate and fill in the missing values. You can click this link for more information on the topic and on how to implement various methods in python.

Additional steps you will likely undergo are feature engineering, and model optimization but this would go beyond the scope of this article. Kaggle is generally a great source that you can use for further guidance.

Photo by Anne Nygård on Unsplash

Beware of shortcomings

Regardless of cleaning, the data you obtained will likely not be perfect. It is necessary to be aware of any limitations to prevent reaching wrong conclusions. I will illustrate this with a few examples from my real estate project.

The Data contains records of condominiums from Montreal. Consequently, results are not generalizable to other housing types or even different cities. There is no information on building materials. This is an unaccounted source for variation. The model won’t differentiate between condos built with cheap and those built with expensive materials. This is expected to be particularly problematic for the small number of high-end condominiums. Neighborhood clusters are relatively large. Less general neighborhood identities would be a valuable addition. This could allow detection of more specific signatures of high potential. Population growth of the past 5 years is a sub-optimal metric for assessing neighborhood potential. There are several issues with this metric. First, population growth says nothing about why it is growing. Second, we don’t know how the growth was distributed over the 5 year period. And last, a metric better suited for optimizing profits would detect potential for growth prior to large population increases.

Keeping the shortcomings of your data in mind will shield you from reaching false interpretations. Furthermore, you can use this information to seek out improvements. Engineering a metric that better reflects the potential of a neighborhood would be immensely valuable. To do this effectively, additional information would be required, such as data on construction and infrastructure projects. Another option would be to collect data on neighborhoods over time. This may allow the detection of characteristic changes in neighborhood attributes that are typical of promising up-and-coming locations.

Deployment

Creating an interactive web application is the best way to showcase your results. It allows you to easily share your work and show off your accomplishments. There are a multitude of tools that you can use. If you want a nice-looking application while minimizing your time investment, I suggest Streamlit. Founded in late 2018, the company is relatively new but has already made major waves in the world of data science.

Keep the user in mind while developing your application. Your goal is to create something of value that can be used by yourself and others. To give you some context, I will go over the functionalities of my real estate project.

As mentioned, one of the main goals was to generate a predictive model to detect undervalued real estate. Required model parameters can be found on any real estate website and used to generate price predictions. Information on how to use the web application is given at the very beginning and before every new section. Clear communication is necessary if you want others to use the tools that you created.

The immediate environment of real estate substantially impacts its value. To generate insights into neighborhoods, I aggregated data using PCA and K-means clustering. Most users of your application will not have a background in data science. Breaking down technical concepts into plain English is essential. An example of this can be found in the section on how to interpret clusters.

To facilitate the process of identifying leads, I supplied a table of the top 200 over-predictions found in the data. Differences are given as predicted minus actual prices. Both the absolute and percent difference are relevant. For example, condominiums in the upper price range may have large absolute differences in combination with small percent differences. Showing only the absolute difference would be misleading. Condominiums can be filtered by price to remove any listings beyond a given budget.

IDs of listings are shown on the very left of the over-prediction table above. These can be used to gain full access to data on condominiums of interest. The table below shows the complete data for the condo with the highest absolute difference and ID 2023.

I highly recommend the use of interactive figures in your projects. They are great for exploring data and easy to use. A large amount of information can be packed into a small area without overwhelming. An industry standard is Plotly, which was used to generate the geoplot below. Colors represent the clusters of listings, while the size corresponds to percent over-predictions. The larger the dot the higher the potential for profit. Hovering over the data will give you some rudimentary information such as the ID of the listing, population growth of the neighborhood and the actual price of the condo. As before, the IDs can be used to access more information on the listing.

If you want to explore the rest of my web application you can click here.

Last thoughts

In summary, when devising your learning path, give your curiosity the spotlight it deserves. Gain familiarity with the fundamental tools first, then shift towards working on personal projects. Collect your own data to include all steps of the data science workflow and be aware of its limitations. Once your project is finalized, deploy it as a web application and broadcast your accomplishments. Finally, rinse and repeat the magic for subsequent undertakings.

By choosing projects that solve a personal pain point or pursue questions you are curious about, you can leverage your inquisitive nature for optimal learning. As you progress, it will become increasingly important to further expand your skill set. Particularly, a firm understanding of SQL, statistics and machine learning concepts are integral cornerstones of the data science toolbox. At the end of this article I included a list of excellent, entirely free resources that you can consult to take your data science skills to the next level.

Like and share the love

Thank you for taking the time to read through my article! As a writer, it means the world.

Top learning resources for data science mastery

Statistic and machine learning concepts

All things data science

Coding Practice