Trends in data science with O’Reilly Media’s Chief Data Scientist
By: Jeremie Harris
Tags: Future Of Work, Data Analytics, Towards Data Science, Tds Podcast, Data Science

Editor’s note: This is the ninth episode of the Towards Data Science podcast’s “Climbing the Data Science Ladder” series, hosted by Jeremie Harris, Edouard Harris and Russell Pollari. Together, they run a data science mentorship startup called SharpestMinds. You can listen to the podcast below:

When I started grad school, data science was only just becoming a thing. At the time, a couple of jupyter notebook projects with pandas and sklearn were all you needed to land some of the most lucrative jobs in tech. By the time I dropped out in 2016, that wasn’t enough — employers were already asking for more in terms of code quality. Today, companies are looking not only for people who are good at coding, statistics and machine learning, but who are good software engineers as well.

The trend towards model deployment, engineering and just generally building “stuff that works” is just the latest step in the evolution of the now-maturing world of data science. It’s almost guaranteed not to be the last one though, and staying ahead of the data science curve means keeping an eye on what trends might be just around the corner. That’s why we asked Ben Lorica, O’Reilly Media’s Chief Data Scientist, to join us on the podcast.

Not only does Ben have a mile-high view of the data science world (he advises about a dozen startups and organizes multiple world-class conferences), but he also has a perspective that spans two decades of data science evolution. He’s seen how trends come and go, and stayed on top of the field as a whole through all that time. Here were our biggest take-homes from the chat:

The latest shift in the data science scene is an emphasis on putting models into production. Understanding how to work with tools like Docker, Git, AWS/GCP or other cloud compute platforms, and even web frameworks like Flask is becoming increasingly important — and even expected.

Data scientists are professional automators: their entire job is to automate decision-making processes that previously would have been carried out by humans. As a result, data science tasks are some of the first that tend to get automated with each successive wave of innovation in the space, because data scientists are staring at them all the time. That’s why data science roles change so fast: they’re constantly being automated away in pieces, leaving data scientists more time to think at increasingly high levels of abstraction.

It sounds cliché to say this, but one consequence of this last point is that soft skills (communication and empathy) are make-or-break today, and look set to become even more important in the future. As data scientists spend less time on tedious tasks like hyperparameter tuning or model selection, they have more and more time to think in big-picture terms. Figuring out business and product development priorities then become increasingly important, and you can’t do either of those things without having the communication skills you need to talk to users and clients, and advocate for specific courses of action within your own company.

Being well-rounded can be a huge advantage, and promises to become an increasingly decisive edge in the future. We saw in our last episode with George Hayward how even law degrees can be powerfully leveraged in data science roles, and Ben’s position is that this breadth of knowledge is important not only at the individual level, but at the level of teams. As managers and VPs are increasingly looking to pull talent from different backgrounds, the ability to pitch your unique story and perspective will become increasingly critical.

You can follow Ben Lorica on Twitter and Medium via his handle @bigdata, and check out the O’Reilly data science conferences that he helps run and organize.