What does it mean to scale your python powered ML pipeline?
Tags: python, machinelearning, datascience
Published: 2022-01-03 16:22:44

---
title: What does it mean to scale your python powered ML pipeline?
published: true
description: Tools you can use to scale python
tags: python, machinelearning, datascience
//cover_image: ![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nvvwzgqupp7shx98gpq3.png)
---

> Scaling, in general, is starting out with a small solution and growing it into a bigger solution.

Different people mean different things when they talk about scaling. Do you want to scale a mountain, a web app, or an ML pipeline?

Even when talking about machine learning, scaling can mean different things:

- Processing more data
- Using a more complex model
- Running more often

### Scaling comes at a cost

Scaling requires resources and cost to execute. The cost can be in the form of:

- Developer time required for code refactoring and optimization for performance
- Performance optimized hardware for accelerated computing
- Distributed processing requires more computers
- Longer wait time for processing more on the same infrastructure

### Think before you scale

There are two important things to consider before you invest in scaling:

- **What are your goals?** You must decide beforehand, what is good enough? Over-engineering is a trap, all engineers are familiar with. Don't over-engineer!

- **Measure your code?** To treat a patient, a doctor has to determine what the illness is? Likewise, you need to measure your code for bottlenecks before deciding on a scaling strategy.
There are many tools that can use to profile your python code. [Scalene](https://github.com/plasma-umass/scalene) is one of the most complete measuring tools available at the moment.

### How to scale?

1. **Increase code efficiency:** Python is designed for ease of use and easy extension, but not performance. As a developer, the onus is on you to do more work so that the application executes less code. Whenever possible use vectorized library functions instead of loops.
Python is successful in data science because of the pre-compiled code offered by data-appropriate libraries in the pydata stack such as [pandas](https://pandas.pydata.org/) and [numpy](https://numpy.org/).

2. **Adding parallelism:** You can achieve parallelism on a single machine or by using multiple machines. In principle, you want to stay on a single machine as much as possible as distributed computing increased complexity because of the following challenges:
  - network transfer
  - task management
  - cluster management

  2a. **Hardware accelerators:** You can leverage GPUs and 
  TPUs for your ML workflows. Libraries such as CuPy and 
  RAPIDS leverage GPUs. There are non-GPU accelerators also 
  like [TPUs offered by Google](https://cloud.google.com/tpu/docs/tpus) and [Apple Neural Engine](https://developer.apple.com/machine-learning/core-ml/).

  2b. **Distributed Computing:** Going distributed is much 
  harder for the aforementioned reasons but if you have to 
  then you can look at the following projects:
   - [Dask](https://dask.org/): Distributed data frames, machine learning and more
   - [Ray](https://www.ray.io/ray-core): Parallel python framework
   - [Horovod](https://eng.uber.com/horovod/): Uber's framework for distributed deep learning
   - [PySpark](https://spark.apache.org/docs/latest/api/python/index.html): Python wrapper for Apache Spark ecosystem

Whichever direction you take on your scaling journey, be sure to pause before you commit, and solve one problem at a time.

P.S. I'll cover I/O bottlenecks in a separate post.