TUTORIALSEVOLUTIONARY LARGE-SCALE GLOBALOPTIMIZATION: AN INTRODUCTION
Mohammad Nabi Omidvar, Xiaodong Li

Many real-world optimization problems involve a large number of decision variables. Thetrend in engineering optimization shows that the number of decision variables involved in a
typical optimization problem has grown exponentially over the last 50 years [7], and thistrend continues with an ever-increasing rate. The proliferation of big-data analytic
applications has also resulted in the emergence of large-scale optimization problems at theheart of many machine learning problems [1, 8]. The recent advance in the area of machinelearning has also witnessed very large scale optimization problems encountered in training
deep neural network architectures (so-called deep learning), some of which have over abillion decision variables [2, 4]. It is this “curse-of-dimensionality” that has made large-scale
optimization an exceedingly difficult task. Current optimization methods are often illequipped in dealing with such problems. It is this research gap in both theory and practicethat has attracted much research interest, making large-scale optimization an active field inrecent years. We are currently witnessing a wide range of mathematical and metaheuristics
optimization algorithms being developed to overcome this scalability issue. Among these,
metaheuristics have gained popularity due to their ability in dealing with black-box
optimization problems.
In this tutorial, we provide an overview of recent advances in the field of evolutionary
large-scale global optimization with an emphasis on the divide-and-conquer approaches(a.k.a. decomposition methods). In particular, we give an overview of different approachesincluding the non-decomposition based approaches such as memetic algorithms andsampling methods to deal with large-scale problems. This is followed by a more detailed
treatment of implicit and explicit decomposition algorithms in large-scale optimization.
Considering the popularity of decomposition methods in recent years, we provide a detailed
technical explanation of the state-of-the-art decomposition algorithms including thedifferential grouping algorithm [5] and its latest improved derivatives, which outperform
other decomposition algorithms on the latest large-scale global optimization benchmarks

Tutorial Chair
Qingfu Zhang · qingfu.zhang@cityu.edu.hk

IEEE Conference on Evolutionary Computation
Donostia – San Sebastián, June 8-5
www.cec2017.org · info@cec2017.org

TUTORIALS[3]. We also address the issue of resource allocation in cooperative co-evolution and provide
a detailed explanation of some recent algorithms such as the contribution-based
cooperative co-evolution family of algorithms [6]. Overall, this tutorial takes the form of a
critical survey of the existing methods with an emphasis on articulating the challenges inlarge-scale global optimization in order to stimulate further research interest in this area.
[1] L. Bottou, F. E. Curtis, and J. Nocedal. Optimization methods for large-scale machinelearning. arXiv preprint arXiv:1606.04838, 2016.
[2] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neuralnetworks. Science, 313(5786):504–507, 2006.
[3] X. Li, K. Tang, M. N. Omidvar, Z. Yang, and K. Qin. Benchmark functions for the CEC’2013
special session and competition on large-scale global optimization. Technical report, RMITUniversity, Melbourne, Australia, 2013.
[4] J. Ngiam, A. Coates, A. Lahiri, B. Prochnow, Q. V. Le, and A. Y. Ng. On optimization
methods for deep learning. In Proceedings of the 28th International Conference on Machine
Learning (ICML- 11), pages 265–272, 2011.
[5] M. N. Omidvar, X. Li, Y. Mei, and X. Yao. Cooperative co-evolution with differential
grouping for large scale optimization. IEEE Transactions on Evolutionary Computation,
18(3):378–393, 2014.
[6] M. N. Omidvar, X. Li, and X. Yao. Smart use of computational resources based oncontribution for cooperative co-evolutionary algorithms. In Proc. of Genetic and Evolutionary
Computation Conference, pages 1115–1122. ACM, 2011.
[7] G. N. Vanderplaats. Very large scale optimization. National Aeronautics and Space
Administration (NASA), Langley Research Center, 2002.
[8] Z.-H. Zhou, N. V. Chawla, Y. Jin, and G. J. Williams. Big data opportunities andchallenges: Discussions from data analytics perspectives. IEEE Computational intelligencemagazine, 9(4):62– 74, 2014.

2/2