Top 5 Issues with Prescriptive Analytics and how to Overcome Them | by Martin Schmitz, PhD | Towards Data Science

Prescriptive analytics is somewhat like predictive analytics on steroids. As pointed out in an earlier post on this topic, prescriptive analytics is a powerful addition to your usual predictive analytics. You don’t just predict future events, but leverage this prediction to optimize your decisions. Prescriptive analytics presents unique challenges as you dive into a melting pot of machine learning and optimizations techniques to understand and implement advanced techniques. It is also a challenge to communicate this to the end user. Here is the list of the top 5 issues you encounter while implementing such a system, and ways to get around them:
Prescriptive Analytics requires you to define a fitness function. This fitness function should reward good optimization results. To define this fitness function, you need to have a good understanding of the business. Getting this equation can sometimes be tough because it requires a close cooperation with the business from the get-go. In my experience, it is beneficial to set up the full pipeline of preparation, modelling and prescriptive analytics first. This pipeline might be simplistic in the beginning. The whole prototype can quickly be deployed e.g. into a BI System or Excel. From there you can have a way better conversation with your end users because they can already experience the results.
Fitness functions which depend on predictive modelling are usually not differentiable. Exceptions are of course simple models like GLMs. Thus, you usually must optimize your fitness function with algorithms which do not require a Gradient or a Hesse matrix. This excludes usual optimizers used in Neural Networks.
In general, the feasible solutions split up into two broad categories. The first is the evolutionary optimization family, which is always a great contender. Evolutionary optimizations always have fan-boys and haters. The main issues for all optimization problems is to converge to an optimum. While finding local optima is relatively simple, evolutionary methods try to overcome local optima and find good solutions in the complete solution space. The tradeoff for that possibility is, that they have much more starting parameters, which can largely effect the quality of the result. That’s why runtime can become an issue with evolutionary methods.The second category deals with approximate-gradient methods. These methods calculate a numerical approximation of the gradient internally. Afterwards they use a variant of the usual gradient descent idea. Well known methods are Simplex or Powell, but there are of course more methods around.
In many cases you will have constraints on your parameters. This can happen when your solution can not be realized. An example for this is would be a negative length. Another reason for a constraint can be a business rule — maybe don’t want to offer prices lower than $10.You can handle these constraints in two ways — you can name them explicitly and make them known to the optimizer, or you can code them into your fitness function.
Some algorithms can incorporate constraints in their methods. Examples for this are BYOBA or Evolutionary Optimization. Others can’t.If your desired algorithm supports the constraints, it’s generally better to use this. In some cases, it will be very tough to do so. An example for this might be the case where your constraint depends on the prediction. In this case you can hardly use this in normal optimizers.on the prediction. In this case you can hardly use this in normal optimizers.
Another way of doing this would be to build in a penalty factor for solutions you do not want to have. You should avoid using threshold based penalties like:
#Fitness should be maximized.def my_fitness(y_pred):    fitness = y_pred    if (y_pred > threshold):  fitness-=1e6 return fitness
If you do it this way, you get discontinuities in your fitness function. This can be very bad if you use this in a gradient-based method.
Alternatively, you can also use penalties on the fitness like this:
def my_fitness(y_pred): fitness = y_pred if (y_pred - threshold > kickin):  fitness-= factor*abs(y_pred - threshold) return fitness
The downside is, that this fitness value is not understandable anymore. Before it was the absolute difference to the threshold with a penalty. Now it’s a combination of a fitness and a penalty function.The downside here is that you lose interpretability. Previously you could interpret your fitness as gain. Now it’s skewed.
So far, we have only discussed constraints which can be derived on a row by row basis. These constraints are constraints which are defined per run of the machine or per customer. You might encounter constraints which need information from other examples to be used. Examples for these are:
Find the price with the maximum profitability, while ensuring you have 1000 Customers a month.
This might mean that you need to hunt for customers that are not profitable to satisfy this business constraint. Another example in manufacturing:
Find the best setting for this run of the machine, but the settings are not allowed to differ too much from last run setting
In the scenario above, you may take a local minimum next to last run’s solution even though other solutions have a higher fitness.
One way to achieve Global Optimization is by optimizing not a single customer or machine run individually, but all customers at the same time. This allows you to define a global penalty which you can include in the fitness. That way you can optimize on the local and global constraints at the same time.One roadblock is that you need to vary all parameters at the same time. A simple 2-dimensional optimization problem can turn into a 20000-dimensional problem if you have 10000 customers a month. This makes the numerical optimization way harder.Another issue is the real-time deployment for this. Think about Predictive Pricing. You cannot run this optimization in real time. What you can do however is overcome this by optimizations for previous’ data, and using this as an input for yet another round of machine learning.
In some use cases it might also be feasible to do a post-processing to satisfy the global constraints. One way of doing this would be to export not just the optimal fitness, but also other sub-optimal data points. Afterwards you try to replace your optimum with one of these candidate points to satisfy the global constraint.
Prescriptive Analytics has some important use cases in manufacturing. One common use case is to control the parameters of the machine or plant for optimal quality. A common issue here is that you only have data for “in-control situations”. If you apply prescriptive analytics on this data, you will encounter problems like moving into hyper-space areas with a low number of data points — making your prediction unreliable.
One idea to overcome this issue with limited input data would be to use an anomaly detection method to figure out how dense the hyper-space is in this particular data point. If the chosen data point is in an area where you don’t have enough points to predict, you would reject the data point. Examples for these methods are Local Outlier Factor (LOF), One-Class SVM or Gaussian Mixture model.
While it’s sounds logical to remove these “outliers” at first glance, it’s not necessary easy to prove. Your prediction performance might not depend on this “sparseness”. Not taking these examples might yield to unnecessary rejection and thus to worse performance. It also might add another bias towards “usual” and thus maybe old-fashioned solutions.
Some plants have a digital twin. The concept of a digital twin arguably deserves its own blog post. The concept is that you have a complete numerical simulation of your entire plant. This twin can be used to generate data points for sparse regions in the hyper-space. You can then use this to fill up the hyper-space with more examples and still predict reliably.