Mathematical Foundations of Complex System Science
Black-Box Optimization
Research
We have developed a surrogate-based black-box optimization method, termed Polynomial-model-based optimization (PMBO).
PMBO alternates polynomial approximation with Bayesian optimization steps, using Gaussian processes to model the error between the objective and its polynomial fit. PMBO is demonstrated to outperform the classic Bayesian optimization and is robust with respect to the choice of its correlation function family and its hyper-parameter setting, which, on the contrary, need to be carefully tuned in classic Bayesian optimization, Fig.1.
Remarkably, PMBO performs comparably with state-of-the-art evolutionary algorithms such as the Covariance Matrix Adaptation — Evolution Strategy (CMA-ES). This finding suggests that PMBO emerges as the pivotal choice among surrogate-based optimization methods when addressing low-dimensional optimization problems. Hereby, the simple nature of polynomials opens the opportunity for interpretation and analysis of the inferred surrogate model, providing a macroscopic perspective on the landscape of the objective function, Fig.2, a path we are currently following.
Papers:
Schreiber, J.,Wicaksono, D., and Hecht, M. Minimizing Black Boxes due to Polynomial- Model-Based Optimization. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation (GECCO ’23 Companion). Association for Computing Machinery, New York, NY, USA, 759–762, 2023.
Schreiber, J., Wicaksono, D., and Hecht, M. Polynomial-Model-Based Optimization for Black-box Objectives. arXiv:2309.00663., 2023