## 1 Introduction
### Occam's Razor and Simplicity in Optimization  
Occam's razor is a problem-solving principle stating that \"other things being equal, simpler explanations are generally better than more complex ones\", an idea originally attributed to [PERSON], an English Franciscan philosopher who lived in the 14 th century. He used a preference for simplicity to defend the idea of divine miracles. In other words, \"The simplest solution is most likely the right one if results match\". Alternatively, \"When presented with competing hypotheses that make the same predictions, one should select the solution with the fewest assumptions\". In science, Occam's razor is used as an abductive heuristic in the development of theoretical models. Simpler theories are in general preferable to more complex ones because they are more testable and easier for people to understand, remember and use.  
How does that apply to optimization algorithms? We will investigate this through an example. Expert A proposes an elegant, simple and clever optimization algorithm OA. Then researcher B in an effort to make a contribution of his/her own, takes this algorithm and proposes a new version that is usually named as \"Improved OA\" (IOA) or similarly \"Enhanced AO\", \"Better AO\", \"Modified AO\", etc. To make this \"improvement\", [PERSON] has no hesitation to add a number of extra parameters to the original algorithm, essentially adding extra complexity to it. In the work published by [PERSON], it is \"proven\" that IOA works better than the original OA in the problems examined. However, in the end, only rarely is IOA acknowledged and recognized as generally better than OA by the scientific community as a whole. Why is that? Assuming that [PERSON] is not trying to cheat and truly believes in the superiority of IOA over OA, there are three main reasons that this can happen:  
* **Test problems bias**. IOA has better performance than OA in the test examples published by [PERSON], but not in other problems. In other words, there is some bias as the problems chosen and presented are in favor of IOA in comparison to OA.
* **Parameter bias**. The extra parameters introduced in IOA work good for some problems, such as the ones chosen and presented by [PERSON], but not in others. And of course, extra knowledge and experience is needed in order to find the right values of the extra parameters for every single problem.  
* **Author bias**. A combination of the two above-mentioned points. Author [PERSON] knows the IOA better than anyone else and has also experience in optimization problems. As a result, [PERSON] can fine-tune the algorithm in such a way that he/she can achieve optimal performance, but the same is not true for other users of the algorithm facing different optimization problems. Thus, the \"improved\" algorithm works good only if it can be fine-tuned by an expert, such as [PERSON].  
In some extreme cases, we have seen authors adding so much complexity and so many extra parameters to an algorithm that literally a new optimization problem needs to be solved in order to fine-tune the parameters of the optimization method to achieve good results. As such, the question is always if it is worth it to add complexity to a method that simply works well. In the opinion of the authors, adding some complexity is not bad in itself, but it should always be justified by managing to achieve clearly better results in the majority of the problems, while also keeping things clear, simple and elegant.