How to use Kaggle to Master Data Science
By: Sharan Kumar Ravindran
Tags: Data Scientist, Kaggle, Data Science, Programming

How to use Kaggle to Master Data Science

Kaggle is one of the world’s largest community of data scientists and machine learning specialists. This platform is home to more than 1 million registered users, it has thousands of public datasets and code snippets (a.k.a. notebooks), more importantly, this platform is actively used by some of the world’s best data scientists. I am not sure how many other fields offer something equivalent to this. It provides a unique opportunity for aspiring data scientists to learn from the world’s best for free. All you need to do is very simple: follow them!

In this article, I am going to explain to you about getting started with kaggle and making use of it to master your data science skills. The approach discussed in this article is not the only way of getting started with kaggle, but it is something that I have seen works based on my mentoring experience. Feel free to modify it to match your preferences. If you have a preference for video format, check out the video version here — https://www.youtube.com/watch?v=9u4zkLoF4DI

Equip yourself with the basic skills

Kaggle is a great place to learn and master data science skills, but it could easily become overwhelming if you don’t have strong knowledge of the basics. So first do a gap analysis on your skillset, understand your current level of competency and check what would require for you to reach a level of competency where you are comfortable with the below:

Basic programming in any one programming language. Python and R are the most popular programming languages for data science. A lot of notebooks available in kaggle will also be in either Python or R. Having basic programming knowledge would be very helpful in reviewing and understanding the notebooks available

Comfortable with the libraries and packages offered by the programming language you have chosen to work on data analysis, numerical operations, statistics operations, and data visualizations. The data analysis notebook would use a lot of libraries and having sufficient background will be very helpful

Have a basic understanding of the different kinds of algorithms and broadly about the different use-cases that can be solved using them

When you have these basic skills then it becomes easy for you to learn further topics with ease and you would be able to appreciate some of the techniques or methods used by experienced data scientists.

If you are very new to data science and looking forward to learning the basics, check this youtube playlist on mine about learning data science in 100 Days.

Explore the datasets

If you are new to data science then begin with the dataset explorations. Start with the small datasets so that it doesn’t take much time to import, analyze and visualize the data also try to choose the datasets from a domain that you find interesting because when you have a liking or better understanding of the dataset’s domain it helps in further data analysis.

Check the description of the datasets, here usually details about how the data were collected and the time period to which the data belong and other details would be provided, this would help in framing your questions for the exploratory data analysis.

Start exploring the dataset and captures the insights. For further ideas on analysis, check out the “Tasks” tab, this is a recently launched feature where people can add interesting things that can be done using the data and others can submit their solutions to it.

Try exploring different kinds of data, slowly move out of your comfort zone and get familiarize with datasets from domains which you haven’t worked. You can even submit your analysis and see how the community reacts to it.

Learn from the EDA code snippets

It’s time to learn data exploration from the best people. For the datasets which you have been working on, go to the Notebooks tab and look for the analysis code snippets with a high number of upvotes and those that come from highly qualified users. Explore the analysis that is being done and try to compare it with what you have done. Identify the gaps or the analysis that you have missed, this retrospective analysis will ensure that the learning will come a long way.

Similarly, check the other datasets and the notebooks with the analysis scripts and understand the kinds of analysis that have been done by some of the experienced data scientists.

Now having learned from some of the experts it’s time to put them into use. Pick a new dataset and start analyzing the data and I am sure your analysis will be much better now and try to incorporate the other standards like documenting the scripts and formatting to ensure they are easy to read and understand.

It is important to spend some quality time in these steps because your data analysis quality will have a direct impact on the quality of the model/solution you are building, so ensure you spend enough time to explore and learn from the experts on data analysis

Explore and re-execute the data science notebooks

Having sharpened your data analysis skills now it’s time to move into building predictive models and other data science solutions. Check out the notebooks solving use-cases and try to understand the logic line by line by re-executing them. Try to explore different kinds of solutions like there are notebooks on building predictive models both regression and classification, also there are notebooks on building solutions like a recommendation engine so going through a range of these solutions and understanding them will be very helpful.

Next focus on the competitions, start with the knowledge competition it helps a lot in understanding the methodology followed in solving competition problems and these knowledge competitions provide exposure to feature engineering as well as model building. Some of the knowledge competitions to start with are below, the first one is good to learn about the classification algorithm and the second one is good to get started with NLP

On being comfortable with the knowledge competition next explore the closed competition and give an attempt in solving them and check where you stand in terms of ranking and accuracy. In many cases, the winning solution would be shared with the participants through the discussion forum in those cases try to understand them and see if there are any learning that you can pick that can be applied in other competitions.

Pointers to get started with Kaggle

Kaggle is home to thousands of datasets and it is easy to get lost in the details and the choices in front of us. Below examples can be considered as a pointer to get started with Kaggle.

The housing price dataset is a good starting point, we all can relate to this dataset easily and hence it becomes easy for analysis as well as for learning. Below is a link to the housing dataset from kaggle

This housing dataset can also be used to learn about building a regression algorithm to predict housing prices. The notebooks available in this dataset would include a variety of algorithms and approaches to building the algorithm, exploring and trying them would help in better understanding the approach to build a predictive model.

Having covered a dataset suitable for the regression problem then next one is to learn about a classification problem and a few good kaggle datasets that can be used for this are below,

Both the credit card fraud and heart failure dataset are something that we can relate to easily. The above credit card fraud dataset is a transformed data and hence you will notice the details are encoded into numerical columns and hence might not be intuitive but once you become comfortable with the datasets then you can start exploring the credit card fraud dataset.

For the above regression and classification examples, try to first understand the dataset, make use of the available exploratory data analysis notebooks to better understand the data, and then also try to learn about the model-building part, there should at least a few notebooks with model implementations using a variety of algorithms

Having covered the regression and classification problem of supervised learning the next one would explore a dataset related to an unsupervised learning problem. The below Groceries dataset is a good example and easy to relate to as well. This dataset can be used for Market Basket Analysis and also for recommendation algorithms

Finally, explore NLP related dataset attached below. This dataset can be used to learn about the sentiment analysis and other NLP related use-cases like topic modeling and summarizations.

Participate in competitions and follow the discussions

Now you are ready to jump into a live competition, choose something that interests you because these competitions are like a marathon it goes on for weeks and it takes continuous effort and hard work to stay on the top on the leadership board and choosing something you like will help you keep motivated.

Don’t try to participate in too many competitions at one time. Stick to one ideally or just a few if you have time. But participating in a lot at one time will not be helping you

While participating in competition always keep an eye on the discussion forums as the data issues and other issues faced by the fellow participants would be discussed here and suggestions about solving them will also be discussed and shared. Hence it is important to stay connected with the discussion groups

Know about what you don’t learn as well

Kaggle is a great platform it provides a lot of exposure to the best performing model and techniques like cross-validation and others packages that can be used to improve the performance of the model but in reality, these modeling phase accounts to just 10–20% of a data science project whereas there is a tremendous amount of effort that goes into formulating the business problem, understanding about the data requirement and identifying the data sources, transforming them to the requirements, featuring engineering and finally comes the model building and deployment.

In the case of kaggle, the problem statement and the dataset are provided to us upfront where in reality, the problem statement needs to framed based on the discussions with the stakeholders, and depending on it, the data requirements will be then identified. Also, mostly the data required for the analysis would be spread across multiple platforms and across public sources and 3rd party websites so I would take a huge effort in consolidating them. So while Kaggle offers a great learning experience on a number of things it will be good to acknowledge the areas where it doesn’t help so that focus can be given to learning those as well.

Other Benefits of using Kaggle

There are some many people will similar interests and you could find a good teammate for your next competition as well

These competition general have a monetary prize attached to it and there are recruitment competition too so you could potentially find your next employer

They also have a job portal so easy to apply for jobs as well

There are courses offered in Kaggle these courses are generally short and useful for brushing up your skills and knowledge

Kaggle is quite famous among the data science community and hence your achievements here will be well received and recognized in the industry

About Me

I am a Data Science professional with over 10 years of experience and I have authored 2 books in data science, they are available for sale here. I have a YouTube channel where I teach and talk about various data science concepts. If interested, subscribe to my channel below.