My experience as a data scientist in a startup
By: Eryk Lewinson
Tags: Machine Learning, Education, Editors Pick, Data Science, Office Hours

My experience as a data scientist in a startup

What are the main things to expect when working as a data scientist in a “smaller” company?

We live in times of rapid data expansion, as basically any activity we perform using technology (and not only) generates some kind of data, which can be then further analyzed and used to draw insights. No wonder that Harvard Business Review named “data scientist” the sexiest job of the 21st century!

Given this brief introduction, it only makes sense for companies to use the available data to make better, more informed business decisions, while at the same time improving the customers’ experience (at least in principle). That is where the role of data analysts/scientists comes into play.

However, the definition of the data scientist — or rather the tasks a data scientist actually handles — differs a lot depending on the company, industry, etc. In this article, I wanted to provide my perspective by describing what it looked like to work as a data scientist for a FinTech startup in the Netherlands.

Photo by Danielle MacInnes on Unsplash

A product data scientist

My position could be more accurately described as a product data scientist, both because of how the teams were structured within the company and what tasks I used to work on. My role was deeply embedded within the product team, which consisted of a product owner, designers, front- and back-end developers, and more. You could say that each data scientist worked in two teams: the general Business Intelligence Team and then in the respective Product Team. In practice, that kind of a hybrid-approach to team structure worked really well and enabled efficient collaboration, while keeping everyone in the loop about what is currently happening in the teams.

After this short intro, I wanted to touch on some of the points that stood out during my time there and which I believe can be helpful for other people searching for a job in startups.

Being close to the decisions

Definitely one of the highlights of working as a data scientist for a smaller company is that you are closer to the decisions and stakeholders. This means that your projects— be it a single Jupyter Notebook, a multi-sheet dashboard, or a deployed model — can be directly used for decision-making or shaping the product (in my case, it was a mobile app). That can be especially rewarding for data scientists, as they see that their work is actually used (as opposed to some report buried in an avalanche of emails) and provides clear value-added to the business.

Domain knowledge and stakeholder management

Being that close to decisions enables the data scientists to understand the business logic and gain valuable domain expertise. This way, they can better understand the potential requirements and expectations of the stakeholders, which makes the cooperation easier and more fruitful.

As an example, it happens that stakeholders do have some goal in mind, but instead of expressing it, they already describe the means to achieve that goal, which they came up with. While such an initiative is always welcome and can be very helpful, it also happens that another approach should be followed, either because the suggested one has some disadvantages or maybe there is no such data available, etc. Because in the end, this is exactly the data scientist’s job to point the best way to achieve the goals of the stakeholders.

Data scientist = jack of all trades

When you hear the term data scientist, you might associate it with a person who is mostly training machine learning models, constantly tuning the hyperparameters to improve the accuracy/precision/recall/another score in order to impact the business. While this may be the case for bigger companies or companies with dedicated machine learning products, in my company (and similarly for other startups I heard about) the role was more of a jack of all trades.

In practice, this meant that we were working on a variety of different projects. Naturally, most of them were related to typical data scientists’ work, but some of them could easily fall under the data or software engineering label.

Regarding the diversity of the projects, while some of them did indeed involve building predictive models (for example, a conversion prediction model), others could be substantially different. Sometimes, we had to build and deploy an importer that downloaded marketing spend from social media platforms and stored it in the internal database for further reporting,

Another time, we built a detailed onboarding funnel, which could be used for optimizing the user journey, localizing the bottlenecks, and reducing the drop-off. We also planned and designed experiments and analyzed the results of multiple A/B tests. And those projects were just the tip of the iceberg.

I would say that working for a startup enables a significantly higher project diversity than in a dedicated team, for example, focusing on customer segmentation or time series clustering. Whether this is something that you would like to experience — the potential width vs. depth trade-off — is entirely up to you. For me, it was a lot of fun and I definitely learned a lot from different areas of data science. And then I documented some of those learnings in the form of Medium articles :)

Pragmatic approach

I believe that startups are one of the best places to learn the pragmatic approach to data science. That is because as young companies, they need to quickly grow, develop, and gain traction. To achieve these goals, it is often best to focus on the low-hanging fruits. Let’s say you have a conversion prediction model indicating whether a given customer is likely to convert this week or not. You can probably spend months trying to improve the metric of interest, either by creating new features, trying the latest state of the art classifier, or tuning the hyperparameters using any of the ten available libraries. Yes, that number is made-up and probably seriously underestimated.

But the real question is, will this extra percentage point or two above a perfectly good Random Forest classifier bring lots of value to the company? Assuming the company is not a behemoth like Google or Facebook, probably not. That is why achieving satisfactory results in a reasonable amount of time is so crucial for smaller companies and — contrary to what you might think right now — it is a thing that data scientists (myself included) often struggle with. When to call it quits, settle for the current solution, and proceed to another project. That is exactly when advice from an experienced manager with a bird’s-eye view can really help.

Not just the shiny things

Connected to the above, but in a bit broader context, for many people, training and tuning models is the place where all the fun is. Also, to take it one step further, everyone heard that a data scientist spends 80% of their time cleaning and wrangling the data and 20% on actual modeling.

But in practice, every day’s work might not even involve training models. Sometimes there is already a lot of value to be extracted from doing some smart aggregations in pandas or prepping a quick and efficient dashboard in Tableau to tell your story. Personally, for some ad-hoc requests, I often found that actually just working with SQL was faster and just as good as loading the data to Python for further analysis.

Is it the most glorious thing to do as a data scientist? For many, probably not. But I would argue that a lot depends on the actual definition of a data scientist. Most likely, in just a tiny % of data scientist positions out there the person only works on training and tuning the models. And for many companies, especially smaller ones, a data scientist is a generic umbrella term for a person who takes data, does some magic, and generates crucial insights for the business. For me, it was rewarding to be presented with a problem, or a task, and then having the possibility to decide which approach (simple aggregate statistics, a fancy model or somewhere in-between) would be best suited to solve it.

The data

It might seem obvious, but there are no data scientists without the data. Before starting actual work, many students and aspiring data scientists mostly play with clean datasets available online (be it the UCI Machine Learning Repository or Kaggle) and unfortunately, this is quite far away from the truth. In my company, we already had a great data engineering pipeline built by my manager, which removed many potential pain-points.

However, we still encountered quite a few issues when we wanted to add some more data (like user session info or data from external parties, like Firebase events for A/B testing). An example could be a weird sequence of events happening for some users (which in theory should not be possible), that only came out while trying to create a linear funnel or applying process mining techniques.

But I would not say that this was a bad thing in the end. Of course, it added some extra work in order to locate those weird patterns. But first, it was satisfying to actually spot the issue (sometimes it required cooperation with developers to narrow down the issue). Secondly, each case left us smarter and we knew what to look out for in the future and how to prevent similar issues from happening again. So all in all, definitely a very educational experience.

Photo by Headway on Unsplash

Fostering creativity

While I already mentioned the diversity of the projects in startups, my company was great when it came to supporting your own initiatives. When there was no pressing sprint work to be done, every now and then we could spend a day playing around with a new library or researching some concepts that might be applicable in the company. In the end, it’s a win-win situation, in which the data scientists grow and develop new skills, while the company can benefit from using a novel approach. Oftentimes, such small exploratory “things” turned into bigger and more significant projects.

In my opinion, it is great to find a company with a high focus on personal development. So you not only complete your tasks and learn on the way, but you can also try to find something worth pursuing on your own. And then, the company supports you by providing you with time or resources to follow it through.

Responsibility and ownership

Connected to the points above, in my previous company the data scientists have quite a lot of freedom in how they can approach the tasks, what tools they use, or even coming up with new ideas and exploring them to provide an MVP or proof of concept.

A person working directly on the project might have some insights into why a certain approach might be better or what kinds of analyses might be beneficial for the company as a whole. Ultimately, the company trusts that the data scientist will always do their best and provide deliverables of the highest quality. This atmosphere of trust further fostered creativity and made you want to put your best effort into each project. Because it was not only one of the twenty projects in the pipeline assigned to you this half a year but something that you could actively shape, build, and deploy.

Tech Stack

What is definitely more common for startups than big companies (especially those under lots of regulations such as banks) is using the latest and greatest tech available. Because startups often start from scratch, without years of legacy, they can quickly adopt the new standards and iterate on them.

That was also the case for my data science team. We had quite a lot of freedom in choosing the libraries which we wanted to use, even if they were just released. But something you should definitely take into account though is the library’s/framework’s maturity, as it matters a lot, especially if you want to put something into production or make a crucial decision based on it. Also in terms of software (Python vs. R, etc.), we followed the pragmatic approach and choose the best tool for the job, even if it sometimes lied outside of our comfort zone.

Conclusions

In this article, I described my experience of working as a data scientist in a startup. Of course, those opinions are subjective and are based on a sample size of 1, so naturally, there will be a lot of variability from startup to startup. But even so, probably some/most of the patterns will be applicable to many companies at a relatively early stage of growth.

For me personally, I learned a lot over the 3 years I worked in a startup, not only the skills strictly connected to data science but also other ones such as pragmatism, stakeholder management, responsibility for the deliverables, time management, and probably some more. And I am sure they will be important in my future career.

If you are interested in finding out more similarities/differences between working as a data scientist for different kinds of companies or you are just starting out your adventure, I can definitely recommend reading Build a Career in Data Science.

Did you have a vastly different experience? I would be happy to read about that in the comments, or you can reach out to me on Twitter.