Package: pflacco
pflacco: The R-package flacco in native Python codeFor people who are not comfortable with R.SummaryFeature-based landscape analysis of continuous and constrained optimization problems is now available in Python as well.
This package provides a python interface to the R packageflaccoby Pascal Kerschke in version v0.4.0.
And now it also provides a native Python implementation with additional features such as:Features for exploiting black-box optimization problem structure.Ruggedness, funnels and gradients in fitness landscapes and the effect on PSO performance.Global characterization of the CEC 2005 fitness landscapes using fitness-distance analysis.Analysing and characterising optimization problems using length scale.Fitness Landscape Analysis Metrics based on Sobol Indices and Fitness-and State-Distributions..Local optima networks for continuous fitness landscapesThe following is the description of the original flacco package:flacco is a collection of features for Explorative Landscape Analysis (ELA) of single-objective, continuous (Black-Box-)Optimization Problems. It allows the user to quantify characteristics of an (unknown) optimization problem's landscape.Features, which used to be spread over different packages and platforms (R, Matlab, Python, etc.), are now combined within this single package. Amongst others, this package contains feature sets, such as ELA, Information Content, Dispersion, (General) Cell Mapping or Barrier Trees.Furthermore, the package provides a unified interface for all features -- using a so-called feature object and (if required) control arguments. In total, the current release (1.7) consists of 17 different feature sets, which sum up to approximately 300 features.In addition to the features themselves, this package also provides visualizations, e.g. of the cell mappings, barrier trees or information contentThe calculation procedure and further background information of ELA features is given inComprehensive Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems Using the R-Package flacco.FAQFor some (very few) features the values for the same sample differ between pflacco and flacco:
This is a known occurence. The differences can be traced back to the underlying methods to calculate the features. For example,ela_metarelies on linear models. The method to construct a linear model in R is based on qr (quantile regression) whereas theLinearModel()in scikit-learn uses the conventional OLS method. For a large enough sample, there is no statistical difference. However, to keep this consistent between programming language this issue will be addressed in future version.What is the difference between 0.* and 1.* version of pflacco?
The 0.* version of pflacco provided a simple interface to the programming language R and calculated any landscape features using the R-package flacco. While this is convenient for me as a developer, the downside is that the performance of pflacco is horrendous. Hence, the >=1.* releases of pflacco offer an implementation of almost all features of the R-package flacco in native python. Thereby, the calculation of features is expedited by an order of magnitude.Is it possible to calculate landscape features for CEC or Nevergrad?
Generally speaking, this is definitely possible. However, to the best of my knowledge, Nevergrad does not offer a dedicated API to query single objective functions and the CEC benchmarks are mostly written in C or Matlab.
Some CEC benchmarks have an unofficial Python wrapper (which is not kept up to date) likeCEC2017. These require additional compiling steps to run any of the functions.PrerequisitesFor a stable (and tested) outcome, pflacco requires at leastPython>=3.8SetupEasy as it usually is in Python:python-mpipinstallpflaccoQuickstartfrompflacco.samplingimportcreate_initial_samplefrompflacco.classical_ela_featuresimportcalculate_ela_distributionfrompflacco.misc_featuresimportcalculate_fitness_distance_correlationfrompflacco.local_optima_network_featuresimportcompute_local_optima_network,calculate_lon_features# Arbitrary objective functiondefobjective_function(x):returnx[0]**2-x[1]**2dim=2# Create inital sample using latin hyper cube samplingX=create_initial_sample(dim,sample_type='lhs')# Calculate the objective values of the initial sample using an arbitrary objective function (here y = x1^2 - x2^2)y=X.apply(lambdax:objective_function(x),axis=1)# Compute an exemplary feature set from the convential ELA features of the R-package flaccoela_distr=calculate_ela_distribution(X,y)print(ela_distr)# Compute an exemplary feature set from the novel features which are not part of the R-package flacco yet.fdc=calculate_fitness_distance_correlation(X,y)print(fdc)# Compute a Local Optima Network (LON). From this network, LON features can be calculated.nodes,edges=compute_local_optima_network(f=objective_function,dim=dim,lower_bound=0,upper_bound=1)lon=calculate_lon_features(nodes,edges)print(lon)It is also possible to include objective functions provided by other packages such asCOCOandioh.Note that these packages do not always pandas dataframes as input. Hence, sometimes it is necessary to transform the initial sample X to a numpy arrayCOCO ExampleIn order for the following code snippet to work, you have installcocofirst (which isnotpossible via pip/conda).
This code snippet calculates the specified landscape features for the well-known single-objective noiseless Black-Box Optimization Benchmark (BBOB).
The optimization problems are comprised of all 24 functions in dimensions 2 and 3 for the first five instances.importcocoexfrompflacco.classical_ela_featuresimport*frompflacco.samplingimportcreate_initial_samplefeatures=[]# Get all 24 single-objective noiseless BBOB function in dimension 2 and 3 for the first five instances.suite=cocoex.Suite("bbob",f"instances:1-5",f"function_indices:1-24 dimensions:2,3")forprobleminsuite:dim=problem.dimensionfid=problem.id_functioniid=problem.id_instance# Create sampleX=create_initial_sample(dim,lower_bound=-5,upper_bound=5)y=X.apply(lambdax:problem(x),axis=1)# Calculate ELA featuresela_meta=calculate_ela_meta(X,y)ela_distr=calculate_ela_distribution(X,y)nbc=calculate_nbc(X,y)disp=calculate_dispersion(X,y)ic=calculate_information_content(X,y,seed=100)# Store results in pandas dataframedata=pd.DataFrame({**ic,**ela_meta,**ela_distr,**nbc,**disp,**{'fid':fid},**{'dim':dim},**{'iid':iid}},index=[0])features.append(data)features=pd.concat(features).reset_index(drop=True)IOH ExampleSimilar to the example above, this code snippet calculates the specified landscape features for the well-known single-objective noiseless Black-Box Optimization Benchmark (BBOB).
The optimization problems are comprised of all 24 functions in dimensions 2 and 3 for the first five instances.
In constrast tococo,iohcan be installed via pip/conda and offers other benchmark problems. See the respectivedocumentationfor more details.frompflacco.classical_ela_featuresimport*frompflacco.samplingimportcreate_initial_samplefromiohimportget_problem,ProblemTypefeatures=[]# Get all 24 single-objective noiseless BBOB function in dimension 2 and 3 for the first five instances.forfidinrange(1,25):fordimin[2,3]:foriidinrange(1,6):# Get optimization problemproblem=get_problem(fid,iid,dim,problem_type=ProblemType.BBOB)# Create sampleX=create_initial_sample(dim,lower_bound=-5,upper_bound=5)y=X.apply(lambdax:problem(x),axis=1)# Calculate ELA featuresela_meta=calculate_ela_meta(X,y)ela_distr=calculate_ela_distribution(X,y)ela_level=calculate_ela_level(X,y)nbc=calculate_nbc(X,y)disp=calculate_dispersion(X,y)ic=calculate_information_content(X,y,seed=100)# Store results in pandas dataframedata=pd.DataFrame({**ic,**ela_meta,**ela_distr,**nbc,**disp,**{'fid':fid},**{'dim':dim},**{'iid':iid}},index=[0])features.append(data)features=pd.concat(features).reset_index(drop=True)CitationIf you are using pflacco in any capacity, I would appreciate a citation. You can use the following bibtex:@article{10.1162/evco_a_00341,
    author = {Prager, Raphael Patrick and Trautmann, Heike},
    title = "{Pflacco: Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems in Python}",
    journal = {Evolutionary Computation},
    pages = {1-25},
    year = {2023},
    month = {07},
    abstract = "{The herein proposed Python package pflacco provides a set of numerical features to characterize single-objective continuous and constrained optimization problems. Thereby, pflacco addresses two major challenges in the area optimization. Firstly, it provides the means to develop an understanding of a given problem instance, which is crucial for designing, selecting, or configuring optimization algorithms in general. Secondly, these numerical features can be utilized in the research streams of automated algorithm selection and configuration. While the majority of these landscape features is already available in the R package flacco, our Python implementation offers these tools to an even wider audience and thereby promotes research interests and novel avenues in the area of optimization.}",
    issn = {1063-6560},
    doi = {10.1162/evco_a_00341},
    url = {https://doi.org/10.1162/evco\_a\_00341},
    eprint = {https://direct.mit.edu/evco/article-pdf/doi/10.1162/evco\_a\_00341/2148122/evco\_a\_00341.pdf},
}DocumentationA comprehensive documentation can be foundhere.ContactI endorse and appreciate every comment and participation. Feel free to open an issue here on GitHub or contact me underraphael.prager@uni-muenster.de