Design Pattern for Machine Learning training work flow | by Saurav Chakravorty | Towards Data Science

I am sure all my readers would have tried multiple training algorithms on their datasets to find the best performing model. I have done this on multiple occasions and at the end of this whole exercise, my workspace would look like this —
Eventually I forget what I did in each of these files and even to remember the best model, I will have to go through each of the individual notebook files review the performance. When I invite collaborators to this project, the system become much more complex. My collaborators and I will have our own training code and each one of us will create a Python file/notebook with their own code. This whole setup becomes a maintenance headache.
I have learned to reduce the complexity by implementing a design pattern called “Strategy”. I set this pattern as soon as I start the project and improve the maintainability and re-usability of my project code.
Before I elaborate the strategy, I need to introduce the concept of higher order functions and decorators. If you already know about this, please skip the next two sections.
Python treats functions as first-class objects. This idea is deep and nuanced. I cannot do justice to this complex topic in this post. Instead, I will urge all of you to learn from the gurus out there who have written multiple articles about the “first-class”. Instead, I will show 2 examples of the implications of treating functions as first-class objects.
In this example we create a function to compute the factorial and Python interpreter registers this at 0x103d6b430. The first class nature allows me to-
Assign factorial to a new variable called fact and use that variable to compute factorial. And,Pass factorial as a parameter to the function accumulate.
Assign factorial to a new variable called fact and use that variable to compute factorial. And,
Pass factorial as a parameter to the function accumulate.
The function accumulate is an higher order function as it takes a function as an input. Higher order functions are popular in data science (and Python overall). I am sure all data scientists have used patterns such as df.apply(np.mean) to their pandas Dataframes. np.mean is a function and since apply takes it as an in input, apply is a higher order function.
Decorators are also higher order functions that accept another function (called the decorated function) as an argument and return or replaces that function. I will focus on the simplest case of an decorator.
def decorator(func):	'''do something'''	return func@decoratedef my_function():	'''Interesting code'''# The above is equivalent to - decorate(my_function)
Decorators leverage the first-class nature of functions in Python.
A design pattern describes a problem that has a pattern and then proposes a solution to that problem. In my case, the problem is to reduce the clutter when collaboratively doing rapid experiments in machine learning.
Quoting from WikiPedia -
In computer programming, the strategy pattern (also known as the policy pattern) is a behavioral software design pattern that enables selecting an algorithm at runtime. Instead of implementing a single algorithm directly, code receives run-time instructions as to which in a family of algorithms to use.[1]
The strategy pattern that I use involves using a decorator to keep track and get the models. The implementation looks like the image below.
I have initialized and empty list to store all the models I am interested in.I create a register decorator function to register my models. The decorator takes any function and adds that function to the list modelsI use the @register keyword to register my modelsI write each model training code as an independent function. But all of them implement a standard return type. In this example the F1 Score and the trained model.Now best_model(data) takes all the registered models and returns the best performing model.
I have initialized and empty list to store all the models I am interested in.
I create a register decorator function to register my models. The decorator takes any function and adds that function to the list models
I use the @register keyword to register my models
I write each model training code as an independent function. But all of them implement a standard return type. In this example the F1 Score and the trained model.
Now best_model(data) takes all the registered models and returns the best performing model.
Whenever my collaborator adds a new model training function, she decorates that function with @register keyword and the new model is integrated into my training runs.
This pattern allows us me to develop ML model in a flexible and easy way. This pattern selects the best model at runtime. I can add a new model training code and I can run all or a specific subset of model training functions. To remove any function (model training code) I can just comment out the @register decorator.
Collaborative Machine Learning gets complex and confusing as many people try out different tricks and your development environment becomes a complicated mess of stale code and notebook files. The pattern that I recommend help developers flexibly design new code and allows maintainers to keep the code base clean. The strategy enforces discipline among developers to write standard reusable code. This strategy has help me and my team to improve code development time and reduce bugs in code.
No rights reserved
 by the author.