Systems Engineering for Machine Learning Applications
By: Pallab Maji
Tags: Machine Learning, Deep Learning, Systems Thinking

Being in the field of Machine Learning (ML) and working on most advance and ‘cool’ technology is amazing. However, many fail to gauge the complexity of these algorithms. Mostly because over a period of time, experts have done extremely well in making beginners feel comfortable with the subject, efficiently abstracting the complexities, and rightly so. Coupled with this, the introduction of efficient libraries have made abstraction much opaque. Thereafter, using neatly defined data (most of them are meant to converge the model) with minimal coding effort, one can achieve human-like efficiency on tasks that seemed difficult previously.

But in real life, data is chaotic with various uncertainties due to noise in training data, model, and other training processes. Thereby, modeling ML solutions become quite difficult and need deeper understanding. This is just like the pretty walls of our home that is visible, hiding all the complex electrical wirings, plumbings and pipings underneath. If someone gives a drawing of the entire layout, it is easy to develop. But doing it from scratch requires real skill.

Without much focus on the actual algorithm, machine learning is, unfortunately, becoming a science of the blind. It is becoming something, forefathers of this elegant field never imagined; a black-art or magic. It is been used by many developers across the world and only few care to delve deep to understand how a model actually treats the data and more importantly, what face of reality does the dataset show the model. Many treat it as a black box by using an enormous amount of data to train a system for a long time and eventually, the system starts producing some output which makes sense. All sounds good and everything is green until you want to make it into a product.

I feel with the ease of prototyping, Machine Learning is perceived by many as a black box that can replicate any task with human-like accuracy. Not to be critical, but many of these professionals have brought glories to the corporations and have achieved marvels in respective fields. However, unlike traditional software development where logic is coded, here logic is learnt from the data. Thereby, it’s imperetive to have knowledge of the algorithm and the data, togther that forms a model. Otherwise, it is impossible to design a system for any application that is optimized, efficient and productive.

I strongly feel that the following questions needs to be answered at some point of product development (sooner the better):

How accurately does the dataset represents the application? Does the training dataset have enough diversity of real world capturing all requirement specifications? How accurately is that data representation learned by the model? Is the model built rightly? Are they developed on/with tools which are tested throughly? How are the models tested and with what data? Could it be possible to encapsulate the data with a model less complex? Is there a way to reduce the complexity of the algorithm (parameter of the model, model operating precision) without reducing accuracies. How the noise in the training data handled? What are the parameters in the data that has been and may be affected by noise. What kind of accuracy measure actually reflects the performance of the model? What are the KPIs (evaluation metrics) used for the testing and how are they derieved from the requirements?

And similarly, there could arise many more questions when dived deep.

I perceive ML solutions to be more work of art than science. Knowing how a system works is different from making the system work efficiently and accurately. Thereby, taking cues from the theory and applying in real datasets will actually help in gaining the right perspective. I agree that presently there are some aspects of machine learning, specifically deep learning, which cannot be explained completely, but that does not mean that it cannot understood completely, nor there cannot be a better scientific approach towards ML based Application Development.

At last, I would like to say that “beating the $hit out of your data will not help, undrstanding them will.”

Happy MLing!!!

Train, Test and Validate…