5 Steps to Develop Unique Data Science Project Ideas
By: Julia Nikulski
Tags: Machine Learning, Tds Narrated, Getting Started, Data Science, Programming

5 Steps to Develop Unique Data Science Project Ideas

A guide that helps you identify new and unique data projects that are worth your while

Arguably the most difficult part of any project is coming up with a novel but executable idea. You can easily be trapped in the dataset rabbit hole. You spend hours looking through existing datasets and try to come up with new and interesting ideas. But here’s the problem with that approach: When you focus only on existing datasets — on Kaggle, Google Datasets, FiveThirtyEight — you limit your creativity to a small set of problems that this dataset was designed for.

At times, I enjoy exploring a dataset that sounds interesting to me. The one-millionth replication of a Kaggle prediction task may benefit my learning progress. But as data scientists, we strive to create something new and unique and to generate useful insights.

So how can you come up with a new idea? I combined insights from creativity research with my own experience. This resulted in 5 key questions including project idea examples. These questions guide your idea generation process and enable you to leverage the full potential of your creativity. This will lead to new and unique data science project ideas.

Image by Danielle MacInnes on Unsplash.

1. Why do I want to start a project?

When you think about starting a new project, you will have an intention or goal in mind. Why do you want to do another data science project in the first place? A broad categorization of your goal can help you focus your search for an idea. So think about whether you want to create

a portfolio project to show to potential employers,

to show to potential employers, a blog post on things such as a concept, model, or an exploratory data analysis,

on things such as a concept, model, or an exploratory data analysis, a project that allows you to practice a skill — such as natural language processing, data visualization, data wrangling, or a particular machine learning algorithm,

— such as natural language processing, data visualization, data wrangling, or a particular machine learning algorithm, or whether you have something completely different in mind.

Image by Free-Photos from Pixabay.

2. What are my areas of expertise and interest?

Thinking about this question is important for three main reasons. First, remember the Venn diagrams of data science. Domain expertise is an important asset every data scientist should have. You can only solve problems with data if you also understand the underlying problem. Otherwise, you implement algorithms, visualizations, and create predictions that are irrelevant to any practitioner in the field. And if your work will be irrelevant, why should you do it in the first place?

Second, it is important to be interested in the idea and dataset you are dealing with. You don’t want to force yourself to spend your free time on a project you don’t care about. If you are interested in an area, you don’t need to be an expert in this domain. But you need to be willing to spend time on extra research and understand the problem beyond the data.

Third, researchers found that introducing constraints to the creative process leads to better results. This means focusing on a particular area or a combination of areas yields better results than trying to come up with an idea without any limitation to the search process. As an example, my areas of interest and expertise are sustainability, finance, popular culture, and natural language processing. Focusing on these topics will allow me to leverage my existing knowledge when determining whether inspiration can be turned into an executable and new idea.

Image by Markus Spiske on Unsplash.

3. How can I find inspiration?

The most important thing you can do to find inspiration is to read. Various sources can help you identify interesting topics during your idea generation process:

News, opinion pieces, blog posts : Anecdotal evidence — in the form of a story — is a great way to come up with ideas. WIRED published this story on the political bias in the autocomplete function of Google. You could use this to investigate bias in language models. Or you could investigate whether it is possible to predict a person’s location based on the search query entered in Google.

: Anecdotal evidence — in the form of a story — is a great way to come up with ideas. WIRED published this story on the political bias in the autocomplete function of Google. You could use this to investigate bias in language models. Or you could investigate whether it is possible to predict a person’s location based on the search query entered in Google. Scientific literature : Academic papers very often contain paragraphs discussing open research questions related to the investigated topic. This paper introducing the language model GPT-2 mentions that the model is not performing better than random on certain tasks — such as answering questions — without fine-tuning. Why not write something on the intricacies of fine-tuning this model?

: Academic papers very often contain paragraphs discussing open research questions related to the investigated topic. This paper introducing the language model GPT-2 mentions that the model is not performing better than random on certain tasks — such as answering questions — without fine-tuning. Why not write something on the intricacies of fine-tuning this model? Data science posts: Looking through blog posts presenting data science topics and projects can lead to new ideas. I saw this NLP exploration of The Office and immediately regretted not coming up with this first. But what about exploring another TV show? Or several movies to determine patterns? You could use GPT-2 to write scripts of your favorite TV show.

Aside from reading, be open in your everyday life for inspiration. Every time you are wondering about something, consider whether you could answer this question with data. For example, I recently stumbled upon a trailer for the TV show The Boys and saw lots of positive reviews on IMDb. So I asked myself, is there a proven trend that popular TV shows become more violent? And is there an increasing audience who likes violent shows? Use these moments of curiosity and explore these questions with data.

What options do you have to create ideas from the inspiration above? Neuroscientists identified three distinct psychological processes that are connected to idea generation:

You can combine existing ideas and create a new outcome ( combinational creativity ) → Various projects analyzed Airbnb listings, while others analyzed the housing market. Combine these ideas to investigate whether Airbnb increases housing prices in a city.

) → Various projects analyzed Airbnb listings, while others analyzed the housing market. Combine these ideas to investigate whether Airbnb increases housing prices in a city. You can explore an existing idea and look for new problems to be solved ( exploratory creativity ) → Take the existing debate around university-educated vs. self-taught data scientists and investigate who is more successful.

) → Take the existing debate around university-educated vs. self-taught data scientists and investigate who is more successful. You can take an existing idea and change something about it that completely changes its rules or meaning (transformational creativity) → This is the rarest form of creativity and operates outside of the established conceptual space. It is difficult to grasp and describe. An example could be that instead of predicting that something will happen, try to predict something that won’t happen.

Image by Diana Parkhouse on Unsplash.

4. Where can I find relevant data?

Once you have a general topic, look for data to determine how you can implement the idea in a data science project. This is crucial in determining whether your idea is executable. Consider existing databases as well as data that still needs to be collected and is more difficult to find:

If you cannot find any data that could help you in addressing your project idea, rephrase it. Try to derive an idea that can be answered with the data you have. At the same time, ask yourself why the data you want is not available? What is the issue in the area you focus on and how could it be solved? Answering these questions could lead to a data science project on its own.

Image by Erol Ahmed on Unsplash.

5. Can I execute my final idea?

You have a fantastic idea! But is it implementable? Recap your idea generation process. Think about what you wanted to achieve (Question 1), whether you have an interest or expertise in this area (Question 2) and whether you have the data you need (Question 4) to execute the idea. Now you need to determine whether you have the skills to implement this project and achieve your goal.

An important factor to consider is the time you are willing to spend on this particular project. You do not want to do your Ph.D. on this topic. So your final project might only be one part of your idea or it might be learning the skill you need to achieve your idea in the future. At the end of this idea generation process should be a research question you can and want to answer in the time you are willing to spend on it to achieve your goal.

Image by Sharon McCutcheon on Unsplash.

Final thoughts

Manage your expectations : Coming up with a novel idea that is executable will take more than a few hours. It is a continuous process of inspiration and you should write down any ideas that come to you. Open a note on your phone and track your ideas. You might be able to combine several of your ideas into one strong project.

: Coming up with a novel idea that is executable will take more than a few hours. It is a continuous process of inspiration and you should write down any ideas that come to you. Open a note on your phone and track your ideas. You might be able to combine several of your ideas into one strong project. Speak to someone about your idea : Discussing your project idea with someone can help you. Maybe a related question is more interesting. Maybe they can point you to additional data sources. Maybe you need a sounding board to realize whether your idea makes sense.

: Discussing your project idea with someone can help you. Maybe a related question is more interesting. Maybe they can point you to additional data sources. Maybe you need a sounding board to realize whether your idea makes sense. Don’t be afraid to start over: Every experience teaches you something. Every time you write a line of code you practice and you expand your skills. When you realize your idea is not bringing you closer to your goal or the idea is not executable, don’t be afraid to leave it and move on. The time you spent was not wasted. It is important to understand when your effort will not yield at least as much reward.

This guide reliably leads to new and unique data science project ideas for me and I hope you will find it as useful as I do. Let me know what your process is and what you think is missing from these steps.