How to Structure Your AI Consulting Service
By: Pedram Ataee
Tags: Data Science, Consulting, Machine Learning, Artificial Intelligence, Technology

In recent years, I offered artificial intelligence or AI consulting services to several companies. Most of them suffered from a lack of AI knowledge. The lack of knowledge may create opportunities for experts to provide AI consulting services; however, it creates many challenges to defining project requirements and deliverables. I went through most of these challenges, and I want to share my experience dealing with them. If you want to offer or need an AI consulting service, I highly recommend reading this article.

Business Understanding — Step 0

In this step, you must determine the objectives and requirements of the project. You must identify what the customer wants to accomplish and how the customer wants to measure its success. Plus, you must define the required resources and project requirements. In short, you must aim to answer “What does the business need?”.

After defining the business objectives, you must determine technical goals, i.e., what success looks like from a technical perspective. A technical requirement plan must be created in this step, describing the required technologies and tools.

Then, you must conduct a cost-benefit analysis. I designed a framework to conduct the cost-benefit analysis in AI projects called the expectation-complexity framework. You can learn more about this framework here: If You Consider Using AI In Your Business, Read This.

Data Understanding — Step 1

In this step, you must identify the data fields needed to train machine learning or ML models. Then, you must write a plan for the customer to collect an initial set of data. The initial data is needed to explore and analyze. Many unknowns such as quality and sufficiency would be clarified during this process. In short, you must aim to answer, “What data do we have/need? Is it clean? Is it enough?”

You must identify required data, create a data collection plan, and provide tips to analyze the data. Anything that can help the customer to accomplish the project goals.

Then, you must help the customer to create a plan for large-scale data collection. You must document all the quality issues and surface properties of data in this step. For example, the relationships among the data must be determined. Visualization always helps to dig deeper into the data. You can read more here: On the Path to Conducting a Large Scale Data Collection.

Data Preparation — Step 2

In this step, you must decide on the required data to conduct large-scale data collection. The quality of data must also be recorded during the data collection. A large-scale data collection is an expensive process; so, you must make a plan to start conducting it. In short, you must aim to answer, “How does the company collect and organize the data?”

The final dataset must become clean before going to the next step. A common practice to clean or cure data is to correct, impute, or remove erroneous values. This is often the lengthiest step in the process. Without proper data curation, the project will encounter with “garbage-in, garbage-out” scenario.

Without a proper data curation, the project will encounter with “garbage-in, garbage-out” scenario.

The AI team may need to drive new attributes or features from the raw data to construct new data, a.k.a., feature engineering. According to model architecture, you may also need to combine the existing data and reformat them. For example, in many applications, string values are converted to numeric values. That helps utilize mathematical operations on textual data. A famous example of data reformatting is the Word2Vec model that is regularly used in text processing. You can read more about Word2Vec models here: Word2Vec Models are Simple Yet Revolutionary.

Model Training — Step 3

In this step, you must train and assess various ML models with different algorithms (e.g., random forest, XGBoost, or deep learning). That helps determine the best modeling techniques for the problem. The selected models will need further tunning and evaluation anyway. In short, you must aim to answer, “What modeling techniques should be used?”

No model can solve all the problems. An ML model that fits problems with tabular data may not work for those with image data and vice versa. Plus, an ML model that fits problems with small datasets may not work for problems with large datasets. And, many more!

Many people think model training or building is the most important part of an AI project. This is not true, at least, anymore.

Many people think model training or building is the most important part of an AI project. This is not true, at least, anymore. Using a large set of tools and libraries, such as SciKitLearn library, the model building is summarized into a few lines of code. The AI team must compare multiple models against each other and interpret their results based on domain knowledge and performance metrics. You can learn more about the story of an ensemble classifier that wins its competition here: The Story of an Ensemble Classifier Acquired By Facebook.

Model Evaluation — Step 4

In this step, you must extensively evaluate models and identify a model that meets the business requirements. Step 3 includes a series of evaluation tasks; however, its focus was mostly on standard performance metrics. Most industrial problems require problem-specific metrics, and standard performance metrics are no longer sufficient for them. In short, you must aim to answer, “Which model best meets the business objectives?”

Many industry problems are constrained by business and technical requirements other than machine learning criteria. For example, a business case that needs responses in less than a second must ensure using low-computation techniques.

You must constantly review the work accomplished using an experiment management system that helps you summarize results and identify mistakes if any.

To build an AI product, you must train a large number of models with different parameter configurations. These models are trained using a training dataset that evolves. The performance metrics can also be changed according to various business requirements. Nevertheless, you must manage this complex process and identify the best model that meets the business objectives. You must use an experiment management system to manage the evaluation process. You can learn more about the experiment management system here: Why Experiment Management is the Key to Success in Data Science.

Model Deployment — Step 5

In this step, you must create a thorough plan for deployment, monitoring, and maintenance. Most companies and experts look down at this step; however, that may generate issues during the operational phase. The best model must be deployed on the cloud to let customers access it. In short, you must aim to answer, “How do stakeholders access the results?”

A model is not useful unless the customer can access its results, it becomes updated to address unpredicted issues, and it would comply with the customer's technical and business requirements. For example, cloud computing services can become costly especially for ML models that are computationally intensive.

A model is not useful unless the customer can access its results, it becomes updated to address unpredicted issues, and it would comply with the customer’s technical and business requirements.

In the end, you must review the whole process by conducting a project retrospective. You must constantly review the process by answering these questions: [1] what went well, [2] what could have been better, and [3] how to improve in the future. If you want to be successful in building AI products, you must learn about common mistakes. You can learn about four common mistakes to build an ML product here: Build An ML Product — 4 Mistakes To Avoid.

The Last Words

We offer turnkey solutions for AI projects. If you need a consulting service for your AI projects, we are more than happy to help you. You can reach me through LinkedIn, Quora, and Twitter.