Top Data Science Misconceptions I Learned in 2021 | Towards Data Science

IntroductionAB Testing is Only About SignificanceOnly Looking at Model Error & AccuracyYou Won’t Use SQLSummaryReferences
Introduction
AB Testing is Only About Significance
Only Looking at Model Error & Accuracy
You Won’t Use SQL
Summary
References
Data scientists have a lot of expectations from companies as well as expectations for the role itself. There are things we expect that turn into misconceptions of the position itself, which can cause stress and confusion down the road. In this article, we will look at five misconceptions that I have personally experienced that you may also experience, and what you should expect instead; intended for any data scientist, from beginner to advanced. With that being said, let’s dive deeper into five misconceptions of 2021 that you can learn from for your data science career in 2022.
Data scientists can get wrapped up in statistics and lose sight of the company goal. AB testing is one of those areas that data scientists may employ. While understanding and executing proper statistics are usually a part of the job and are expected, focusing too much on p-values itself, for example, might lead you away from the overall goal.
What not to do:
When comparing a test group against a control, only look at signifiance/p-value
What to do:
When comparing a test group against a control, look at the p-value, the sample size, and the duration of the test. It may be the case where you reach significance (or not), but it is not held steady over time.
When you assess more than just the p-value, you can make sure that your test is fully processed and will stand the test of time. In addition to the test characteristics, you will want to focus on if the metrics make sense to test in general. For example, if you are incorporating a new product, you will want to look at metrics that can show an increase in business success, like:
Revenue
Retention
Engagement
You will also want to make sure that you do not harm the business, or cause negative effects to the business with a product change from a data science model.
In a similar way to the above point, this misconception summarizes that just because you have an accurate model, it does not mean that you have a useful model.
What not to do:
A. The model has 94% accuracy for classifying good users from bad users
(identifying users as good or bad is most not likely as useful because you cannot perform actionable processes from it, and it is not well defined)
B. The model has an MAE of 4 min for bus arrival time
(the MAE might be low, but if it tends to sway in one direction, then it could not be as useful for customers who do not want to stand at the bus stop for longer and would prefer to be there earlier)
What to do:
A. The model has 94% accuracy for classifying customers who buy products more versus who buy no products at all, or very few
(this way, you can identify characteristics for people who spend more money, and can see if those features can be applied to the users who do not spend as much money, for example, if people tend to click on an iPhone notification versus an Android notification, perhaps the UI experiences are different, and that needs to be addressed)
B. The model has an MAE of 4 min but now the metric is optimized for more values to be on the earlier side rather than the later side
(this way, the model is just about as accurate overall, but it helps the use case more in the real world)
As you can see, in addition to accuracy or error, you will need to identify the direction of the metric, as well as if what you are predicting is usable to make improvements or not.
Because education programs in data science focus more on data science itself as well as machine learning, SQL can often be neglected or not taught at all. In order to be a more well-rounded data scientist, you will want to learn SQL so you do not have to learn it on the spot in your career.
What not to do:
Expect that all the SQL will be done for you or the data you use for your model is already created
(you will most likely need to query your company database using SQL for obtaining your training data)
What to do:
Understand SQL while learning data science, and take more than just one course on it
(you will also want to learn SQL to query your results from your models, as usually, it is the case that your predictions are stored in a table that is queryable)
There are some jobs where you will not need to perform SQL for data science, but usually, you do, and it is not as much of a focus when you are trying to learn about machine learning algorithms. Learn it over time so that you do not have to rush into it, especially in your first data science position.
As you can see, there are a few misconceptions about a career in data science. I have learned about these in the role itself this year, rather than from studying about it before entering a role. Of course, there are plenty more, but I hope this article has given some the opportunity to fix or improve upon your data science misconceptions moving forward.
To summarize, here are some of the common data science misconceptions that you can learn from as well:
* AB Testing is Only About Significance* Only Looking at Model Error & Accuracy* You Won’t Use SQL
I hope you found my article both interesting and useful. Please feel free to comment down below if you agree or disagree with these misconceptions included. Why or why not? What other misconceptions do you think we could include that are just or more important? These can certainly be clarified even further, but I hope I was able to shed some light on some more unique misconceptions that I have personally experienced. Thank you for reading!
I am not affiliated with any of these companies.
Please feel free to check out my profile, Matt Przybyla, and other articles, as well as subscribe to receive email notifications for my blogs by following the link below, or by clicking on the subscribe icon on the top of the screen by the follow icon, and reach out to me on LinkedIn if you have any questions or comments.
Subscribe link: https://datascience2.medium.com/subscribe
Referral link: https://datascience2.medium.com/membership
(I will receive a commission if you sign up for a membership on Medium)
[1] Photo by Isabela Kronemberger on Unsplash, (2021)
[2] Photo by Markus Winkler on Unsplash, (2020)
[3] Photo by Nicolas Horn on Unsplash, (2021)
[4] Photo by Caspar Camille Rubin on Unsplash, (2017)