RAAL: Resource Aware Active Learning for Multifidelity Efficient Optimization

No AccessTechnical NotesRAAL: Resource Aware Active Learning for Multifidelity Efficient OptimizationFrancesco Grassi, Giorgio Manganini, Michele Garraffa and Laura MaininiFrancesco Grassi https://orcid.org/0000-0002-2360-0354Raytheon Technologies Research Centre, Cork City T23 XN53, Republic of Ireland*Senior Research Scientist, Autonomous and Intelligent Systems Department.Search for more papers by this author, Giorgio ManganiniRaytheon Technologies Research Centre, Cork City T23 XN53, Republic of Ireland*Senior Research Scientist, Autonomous and Intelligent Systems Department.Search for more papers by this author, Michele GarraffaRaytheon Technologies Research Centre, Cork City T23 XN53, Republic of Ireland*Senior Research Scientist, Autonomous and Intelligent Systems Department.Search for more papers by this author and Laura Mainini https://orcid.org/0000-0002-5969-9069Raytheon Technologies Research Centre, Cork City T23 XN53, Republic of Ireland†Chief Technologist, Applied Research and Technology, Collins Aerospace, Cork City T23 XN53, Republic of Ireland; . Associate Fellow AIAA.Search for more papers by this authorPublished Online:8 May 2023https://doi.org/10.2514/1.J061383SectionsRead Now ToolsAdd to favoritesDownload citationTrack citations ShareShare onFacebookTwitterLinked InRedditEmail About References [1] Jones D. R., Schonlau M. and Welch W. J., “Efficient Global Optimization of Expensive Black-Box Functions,” Journal of Global Optimization, Vol. 13, No. 4, 1998, pp. 455–492. https://doi.org/10.1023/A:1008306431147 CrossrefGoogle Scholar[2] Queipo N. V., Haftka R. T., Shyy W., Goel T., Vaidyanathan R. and Tucker P. K., “Surrogate-Based Analysis and Optimization,” Progress in Aerospace Sciences, Vol. 41, No. 1, 2005, pp. 1–28. https://doi.org/10.1016/j.paerosci.2005.02.001 CrossrefGoogle Scholar[3] Eldred M. and Dunlavy D., “Formulations for Surrogate-Based Optimization with Data Fit, Multifidelity, and Reduced-Order Models,” 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA Paper 2006-7117, 2006. https://doi.org/10.2514/6.2006-7117 LinkGoogle Scholar[4] Robinson T., Eldred M., Willcox K. and Haimes R., “Surrogate-Based Optimization Using Multifidelity Models with Variable Parameterization and Corrected Space Mapping,” AIAA Journal, Vol. 46, No. 11, 2008, pp. 2814–2822. https://doi.org/10.2514/1.36043 LinkGoogle Scholar[5] Forrester A. I. and Keane A. J., “Recent Advances in Surrogate-Based Optimization,” Progress in Aerospace Sciences, Vol. 45, Nos. 1–3, 2009, pp. 50–79. https://doi.org/10.1016/j.paerosci.2008.11.001 CrossrefGoogle Scholar[6] Bhosekar A. and Ierapetritou M., “Advances in Surrogate Based Modeling, Feasibility Analysis, and Optimization: A Review,” Computers and Chemical Engineering, Vol. 108, Jan. 2018, pp. 250–267. https://doi.org/10.1016/j.compchemeng.2017.09.017 CrossrefGoogle Scholar[7] Kennedy M. C. and O’Hagan A., “Predicting the Output from a Complex Computer Code When Fast Approximations Are Available,” Biometrika, Vol. 87, No. 1, March 2000, pp. 1–13, https://www.jstor.org/stable/2673557. https://doi.org/10.1093/biomet/87.1.1 CrossrefGoogle Scholar[8] Forrester A. I., Sóbester A. and Keane A. J., “Multi-Fidelity Optimization via Surrogate Modelling,” Proceedings of the Royal Society of London, Series A: Mathematical, Physical and Engineering Sciences, Vol. 463, No. 2088, 2007, pp. 3251–3269. https://doi.org/10.1098/rspa.2007.1900 CrossrefGoogle Scholar[9] Fernández-Godino M. G., Park C., Kim N.-H. and Haftka R. T., “Review of Multi-Fidelity Models,” Preprint, submitted 23 Sept. 2016, https://arxiv.org/abs/1609.07196. https://doi.org/10.48550/arXiv.1609.07196 Google Scholar[10] Park C., Haftka R. T. and Kim N. H., “Remarks on Multi-Fidelity Surrogates,” Structural and Multidisciplinary Optimization, Vol. 55, No. 3, 2017, pp. 1029–1050. https://doi.org/10.1007/s00158-016-1550-y CrossrefGoogle Scholar[11] Peherstorfer B., Willcox K. and Gunzburger M., “Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization,” SIAM Review, Vol. 60, No. 3, 2018, pp. 550–591. https://doi.org/10.1137/16M1082469 CrossrefGoogle Scholar[12] Beran P. S., Bryson D., Thelen A. S., Diez M. and Serani A., “Comparison of Multi-Fidelity Approaches for Military Vehicle Design,” AIAA AVIATION 2020 Forum, AIAA Paper 2020-3158, 2020. https://doi.org/10.2514/6.2020-3158 LinkGoogle Scholar[13] Brochu E., Cora V. M. and de Freitas N., “A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning,” Preprint, submitted 12 Dec. 2010, https://arxiv.org/abs/1012.2599. https://doi.org/10.48550/arXiv.1012.2599 Google Scholar[14] Frazier P. I., “A Tutorial on Bayesian Optimization,” Preprint, submitted 8 July 2018, https://arxiv.org/abs/1807.02811. https://doi.org/10.48550/arXiv.1807.02811 Google Scholar[15] Williams C. K. and Rasmussen C. E., Gaussian Processes for Machine Learning, MIT Press, Cambridge, MA, 2006. https://doi.org/10.7551/mitpress/3206.001.0001 Google Scholar[16] Viana F. A., Haftka R. T. and Watson L. T., “Efficient Global Optimization Algorithm Assisted by Multiple Surrogate Techniques,” Journal of Global Optimization, Vol. 56, No. 2, 2013, pp. 669–689. https://doi.org/10.1007/s10898-012-9892-5 CrossrefGoogle Scholar[17] Kushner H. J., “A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise,” Journal of Basic Engineering, Vol. 86, No. 1, 1964, pp. 97–106. https://doi.org/10.1115/1.3653121 CrossrefGoogle Scholar[18] Hennig P. and Schuler C. J., “Entropy Search for Information-Efficient Global Optimization,” Journal of Machine Learning Research, Vol. 13, No. 6, 2012, pp. 1809–1837, https://jmlr.csail.mit.edu/papers/volume13/hennig12a/hennig12a.pdf. Google Scholar[19] Hernández-Lobato J. M., Hoffman M. W. and Ghahramani Z., “Predictive Entropy Search for Efficient Global Optimization of Black-Box Functions,” Advances in Neural Information Processing Systems, Curran Assoc., New York, 2014, pp. 918–926. Google Scholar[20] Mockus J., Bayesian Approach to GlobalOptimization: Theory and Applications, Vol. 37, Springer Science and Business Media, New York, 2012. https://doi.org/10.1007/978-94-009-0909-0 Google Scholar[21] Forrester A., Sobester A. and Keane A., Engineering Design via Surrogate Modelling: A Practical Guide, Wiley, Hoboken, NJ, 2008. https://doi.org/10.1002/9780470770801 CrossrefGoogle Scholar[22] Holicki T., Scherer C. W. and Trimpe S., “Controller Design via Experimental Exploration with Robustness Guarantees,” Preprint, submitted 19 March 2020, https://arxiv.org/abs/2003.08613. https://doi.org/10.48550/arXiv.2003.08613 Google Scholar[23] Negoescu D. M., Frazier P. I. and Powell W. B., “The Knowledge-Gradient Algorithm for Sequencing Experiments in Drug Discovery,” INFORMS Journal on Computing, Vol. 23, No. 3, 2011, pp. 346–363. https://doi.org/10.1287/ijoc.1100.0417 CrossrefGoogle Scholar[24] Packwood D., Bayesian Optimization for Materials Science, Springer, New York, 2017. https://doi.org/10.1007/978-981-10-6781-5 CrossrefGoogle Scholar[25] Majda A. J. and Gershgorin B., “Quantifying Uncertainty in Climate Change Science Through Empirical Information Theory,” Proceedings of the National Academy of Sciences, Vol. 107, No. 34, 2010, pp. 14,958–14,963. https://doi.org/10.1073/pnas.1007009107 Google Scholar[26] Lizotte D. J., Wang T., Bowling M. H. and Schuurmans D., “Automatic Gait Optimization with Gaussian Process Regression,” IJCAI ‘07, International Joint Conferences on Artificial Intelligence Organization, Morgan Kaufman, San Mateo, CA, 2007, pp. 944–949. Google Scholar[27] Cutler M., Walsh T. J. and How J. P., “Reinforcement Learning with Multi-Fidelity Simulators,” 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE, New York, 2014, pp. 3888–3895. https://doi.org/10.1109/ICRA.2014.6907423 Google Scholar[28] Snoek J., Larochelle H. and Adams R. P., “Practical Bayesian Optimization of Machine Learning Algorithms,” Advances in Neural Information Processing Systems, Curran Assoc., New York, 2012, pp. 2951–2959. Google Scholar[29] Chakraborty S., “Transfer Learning Based Multi-Fidelity Physics Informed Deep Neural Network,” Preprint, submitted 19 May 2020, https://arxiv.org/abs/2005.10614. https://doi.org/10.48550/arXiv.2005.10614 Google Scholar[30] Lam R., Allaire D. L. and Willcox K. E., “Multifidelity Optimization Using Statistical Surrogate Modeling for Non-Hierarchical Information Sources,” 56th AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper 2015-0143, 2015. https://doi.org/10.2514/6.2015-0143. LinkGoogle Scholar[31] Viana F. A., Simpson T. W., Balabanov V. and Toropov V., “Special Section on Multidisciplinary Design Optimization: Metamodeling in Multidisciplinary Design Optimization: How Far Have We Really Come?” AIAA Journal, Vol. 52, No. 4, 2014, pp. 670–690. https://doi.org/10.2514/1.J052375 LinkGoogle Scholar[32] Guo Z., Song L., Park C., Li J. and Haftka R. T., “Analysis of Dataset Selection for Multi-Fidelity Surrogates for a Turbine Problem,” Structural and Multidisciplinary Optimization, Vol. 57, No. 6, 2018, pp. 2127–2142. https://doi.org/10.1007/s00158-018-2001-8 CrossrefGoogle Scholar[33] Meliani M., Bartoli N., Lefebvre T., Bouhlel M.-A., Martins J. and Morlier J., “Multi-Fidelity Efficient Global Optimization: Methodology and Application to Airfoil Shape Design,” AIAA Aviation 2019 Forum, AIAA Paper 2019-3236, 2019. https://doi.org/10.2514/6.2019-3236 LinkGoogle Scholar[34] Kontogiannis S. G., Demange J., Savill A. M. and Kipouros T., “A Comparison Study of two Multifidelity Methods for Aerodynamic Optimization,” Aerospace Science and Technology, Vol. 97, Feb. 2020, Paper 105592. https://doi.org/10.1016/j.ast.2019.105592 Google Scholar[35] Le Gratiet L. and Garnier J., “Recursive Co-Kriging Model for Design of Computer Experiments with Multiple Levels of Fidelity,” International Journal for Uncertainty Quantification, Vol. 4, No. 5, 2014, pp. 365–386. https://doi.org/10.1615/Int.J.UncertaintyQuantification.2014006914 CrossrefGoogle Scholar[36] Cutajar K., Pullin M., Damianou A., Lawrence N. and González J., “Deep Gaussian Processes for Multi-Fidelity Modeling,” Preprint, submitted 18 March 2019, https://arxiv.org/abs/1903.07320. https://doi.org/10.48550/arXiv.1903.07320 Google Scholar[37] Huang D., Allen T. T., Notz W. I. and Miller R. A., “Sequential Kriging Optimization Using Multiple-Fidelity Evaluations,” Structural and Multidisciplinary Optimization, Vol. 32, No. 5, 2006, pp. 369–382. https://doi.org/10.1007/s00158-005-0587-0 CrossrefGoogle Scholar[38] Zhang Y., Hoang T. N., Low B. K. H. and Kankanhalli M., “Information-Based Multi-Fidelity Bayesian Optimization,” 31st Conference on Neural Information Processing Systems (NIPS 2017), Curran Assoc., New York, 2017. Google Scholar[39] Takeno S., Fukuoka H., Tsukada Y., Koyama T., Shiga M., Takeuchi I. and Karasuyama M., “Multi-Fidelity Bayesian Optimization with Max-Value Entropy Search,” Preprint, submitted 24 Jan. 2019, https://arxiv.org/abs/1901.08275. https://doi.org/10.48550/arXiv.1901.08275 Google Scholar[40] Sacks J., Schiller S. B. and Welch W. J., “Designs for Computer Experiments,” Technometrics, Vol. 31, No. 1, 1989, pp. 41–47. https://doi.org/10.1080/00401706.1989.10488474 CrossrefGoogle Scholar[41] D’Ambrosio C., Nannicini G. and Sartor G., “MILP Models for the Selection of a Small Set of Well-Distributed Points,” Operations Research Letters, Vol. 45, No. 1, 2017, pp. 46–52. https://doi.org/10.1016/j.orl.2016.11.004 CrossrefGoogle Scholar[42] Rosenbrock H., “An Automatic Method for Finding the Greatest or Least Value of a Function,” Computer Journal, Vol. 3, No. 3, 1960, pp. 175–184. https://doi.org/10.1093/comjnl/3.3.175 CrossrefGoogle Scholar[43] Bryson D. E., “A Unified, Multifidelity Quasi-Newton Optimization Method with Application to Aero-Structural Design,” Ph.D. Thesis, Univ. of Dayton, Dayton, OH, 2017. Google Scholar[44] Mainini L., Serani A., Rumpfkeil M., Minisci E., Quagliarella D., Pehlivan H., Yildiz S., Ficini S., Pellegrini R., Di Fiore F. and Bryson D., “Analytical Benchmark Problems for Multifidelity Optimization Methods,” Preprint, submitted 16 April 2022, https://arxiv.org/abs/2204.07867. https://doi.org/10.48550/arXiv.2204.07867 Google Scholar[45] Grassi F., Manganini G., Garraffa M. and Mainini L., “Resource Aware Multifidelity Active Learning for Efficient Optimization,” AIAA SciTech 2021 Forum, AIAA Paper 2021-0894, 2021. https://doi.org/10.2514/6.2021-0894 LinkGoogle Scholar[46] Paleyes A., Pullin M., Mahsereci M., Lawrence N. and González J., “Emulation of Physical Processes with Emukit,” Second Workshop on Machine Learning and the Physical Sciences, NeurIPS, 2019. Google Scholar[47] Mitchell S., O’Sullivan M. and Dunning I., “PuLP: A Linear Programming Toolkit for Python,” Univ. of Auckland, Auckland, New Zealand, 2011, https://optimization-online.org/?p=11731 [retrieved March 2020]. Google Scholar[48] Lougee-Heimer R., “The Common Optimization INterface for Operations Research: Promoting Open-Source Software in the Operations Research Community,” IBM Journal of Research and Development, Vol. 47, No. 1, 2003, pp. 57–66. https://doi.org/10.1147/rd.471.0057 Google Scholar Previous article Next article FiguresReferencesRelatedDetails What's Popular Volume 61, Number 6May 2023 CrossmarkInformationCopyright © 2023 by Francesco Grassi, Giorgio Manganini, Michele Garraffa, Laura Mainini, United Technologies Research Centre Ireland. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission. All requests for copying and permission to reprint should be submitted to CCC at www.copyright.com; employ the eISSN 1533-385X to initiate your request. See also AIAA Rights and Permissions www.aiaa.org/randp. AcknowledgmentsThis work was supported by the Industrial Development Agency’s Center of Excellence in Cyber Physical Systems (grant no. 176474) under the Industrial Development Agency (Ireland) program.PDF Received19 October 2021Accepted26 March 2023Published online8 May 2023