Package: fedot-example-test
FEDOTpackagetestsdocslicenseThis repository contains Fedot - a framework for automated modeling and machine learning. It can build composite models for the different real-world processes in an automated way using an evolutionary approach.Composite models - the models with heterogeneous graph-based structure, that can consist of ML models, domain-specific models, equation-based models, statistical, and even other composite models. Composite modelling allows obtaining efficient multi-scale solutions for various applied problems.Fedot can be used for classification, regression, clustering, time series forecasting, and other similar tasks. Also, the derived solutions for other problems (e.g. bayesian generation of synthetic data) can be build using Fedot.Core.The project is maintained by the research team of Natural Systems Simulation Lab, which is a part of the National Center for Cognitive Research of ITMO University.InstallationCommon installation:$ pip install fedotIn order to work with FEDOT source code:$ git clone https://github.com/nccr-itmo/FEDOT.git
$ cd FEDOT
$ pip install -r requirements.txt
$ pytest -s testFEDOT featuresThe generation of high-quality variable-shaped machine learning pipelines for various tasks: binary/multiclass classification, regression, clustering, time series forecasting;The structural learning of composite models with different nature (hybrid, bayesian, deep learning, etc) using custom metrics;The seamless integration of the custom models (including domain-specific), frameworks and algorithms into pipelines;Benchmarking utilities that can run real-world cases (the ready-to-use examples are provided for credit scoring, sea surface height forecasting, oil production forecasting, etc), state-of-the-art-datasets (like PMLB) and synthetic data.How to useThe main purpose of FEDOT is to identify a suitable composite model for a given dataset.
The model is obtained via optimization process (we also call it ‘composing’).Firstly, you need to prepare datasets for fit and validate and specify a task
that you going to solve:task=Task(TaskTypesEnum.classification)dataset_to_compose=InputData.from_csv(train_file_path,task=task)dataset_to_validate=InputData.from_csv(test_file_path,task=task)Then, chose a set of models that can be included in the composite model, and the optimized metric function:available_model_types,_=ModelTypesRepository().suitable_model(task_type=task.task_type)metric_function=MetricsRepository().metric_by_id(ClassificationMetricsEnum.ROCAUC)Next, you need to specify requirements for composer.
In this case, GPComposer is chosen that is based on evolutionary algorithm.composer_requirements=GPComposerRequirements(primary=available_model_types,secondary=available_model_types,max_arity=3,max_depth=3,pop_size=20,num_of_generations=20,crossover_prob=0.8,mutation_prob=0.8,max_lead_time=20)composer=GPComposer()Now you can run the optimization and obtain a composite model:chain_evo_composed=composer.compose_chain(data=dataset_to_compose,initial_chain=None,composer_requirements=composer_requirements,metrics=metric_function,is_visualise=False)Finally, you can test the resulted model on the validation dataset:roc_on_valid_evo_composed=calculate_validation_metric(chain_evo_composed,dataset_to_validate)print(f'Composed ROC AUC is{round(roc_on_valid_evo_composed,3)}')Extended examples:Credit scoring problem, i.e.binary classification taskTime series forecasting, i.e.random process regressionAlso, several video tutorials areavailable(in Russian).Project structureThe latest stable release of FEDOT is on themaster branch. Make sure you are looking at and working on the actual code if you’re looking to contribute code.The repository includes the following directories:Packagecorecontains the main classes and scripts. It is acoreof FEDOT frameworkPackageexamplesincludes severalhow-to-use-caseswhere you can start to discover how FEDOT worksAllunit testscan be observed in thetestdirectoryThe sources of documentation are in thedocsAlso you can checkbenchmarkingrepository that was developed to
show the comparison of FEDOT against the well-known AutoML frameworks.Basic ConceptsThe main process of FEDOT work iscomposingleading to the production of the composite models.Composeris a block that takes meta-requirements and the evolutionary algorithm as an optimization one
and get different chains of models to find the most appropriate solution for the case.The result of composing and basic object user works with is the Chain:Chainis the tree-based structure of any composite model. It keeps the information of nodes relations
and everything referred to chain properties and restructure.In fact, any chain has two kinds of nodes:Primary nodesare edge (leaf) nodes of the tree where initial case data is located.Secondary nodesare all other nodes which transform data during the composing and fitting, including root node with result data.Meanwhile, every node holds theModelwhich could be ML or any other kind of model.The referenced papers:Kalyuzhnaya A. V. et al. Automatic evolutionary learning of composite models with knowledge enrichment //Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. вЂ“ 2020. вЂ“ P. 43-44.Kovalchuk S. V. et al. A conceptual approach to complex model management with generalized modelling patterns and evolutionary identification //Complexity. вЂ“ 2018. вЂ“ V. 2018.Nikitin N. O. et al. Deadline-driven approach for multi-fidelity surrogate-assisted environmental model calibration: SWAN wind wave model case study //Proceedings of the Genetic and Evolutionary Computation Conference Companion. вЂ“ 2019. вЂ“ РЎ. 1583-1591.Vychuzhanin P., Nikitin N. O., Kalyuzhnaya A. V. Robust Ensemble-Based Evolutionary Calibration of the Numerical Wind Wave Model //International Conference on Computational Science. вЂ“ Springer, Cham, 2019. вЂ“ P. 614-627.Nikitin N. O. et al. Evolutionary ensemble approach for behavioral credit scoring //International Conference on Computational Science. вЂ“ Springer, Cham, 2018. вЂ“ P. 825-831.Current R&D and future plansAt the moment, we execute an extensive set of experiments to determine the most suitable approaches for evolutionary chain optimization, hyperparameters tuning, benchmarking, etc.
The different case studies from different subject areas (metocean science, oil production, seismic, robotics, economics, etc) are in progress now.
The various features are planned to be implemented: multi-data chains, Bayesian networks optimization, domain-specific, equation-based models involvement, model export and atomization, interpretable surrogate models, etc.Any support and contribution are welcome.DocumentationThe documentation is available inFEDOT.Docsrepository.The description and source code of underlying algorithms is available inFEDOT.Algsrepository and itswiki pages(in Russian).Also, FEDOT API inRead the Docs.Contribution GuideThe contribution guide is available in therepository.AcknowledgementsWe acknowledge the contributors for their important impact and the participants of the numerous scientific conferences and workshops for their valuable advice and suggestions.Supported byNational Center for Cognitive Research of ITMO UniversityCitation@inproceedings{kalyuzhnaya2020automatic,title={Automatic evolutionary learning of composite models with knowledge enrichment},
author={Kalyuzhnaya, Anna V and Nikitin, Nikolay O and Vychuzhanin, Pavel and Hvatov, Alexander and Boukhanovsky, Alexander},
booktitle={Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion},
pages={43–44},
year={2020}}