Package: ionbench
ionBenchA benchmarking tool for comparing different parameter optimization algorithms for ion channel models.Project StructureThe tree structure of this project is outlined below.├───docs
├───ionbench
│   ├───modification
│   ├───benchmarker
│   ├───data
│   │   ├───loewe2016
│   │   ├───moreno2016
│   │   └───staircase
│   ├───optimisers
│   │   ├───external_optimisers
│   │   ├───pints_optimisers
│   │   ├───scipy_optimisers
│   ├───problems
│   ├───uncertainty
│   └───multistart.py
└───testThedocsdirectory contains information and guides on how to use the benchmarker, the test problems and the optimisation algorithms.Theionbenchdirectory contains the majority of the code, including the benchmarker and problems classes and the different optimisation algorithms. This is what is installed using pip.Themodificationsubdirectory contains the modification classes, generalised settings for handling transformations and bounds.Thebenchmarkersubdirectory contains the mainBenchmarkerclass that the test problems all inherit from and defines the core features of the benchmarkers. It also contains theTrackerclass which contains the functions to take performance metrics over time.Thedatasubdirectoy is split up into the available test problems. Each subdirectory contains the Myokit.mmtfiles, the voltage clamp protocols stored as.csvfiles where relevant, and output data to train the models, also stored as a.csv.Theoptimiserssubdirectory contains all of the optimisation algorithms that are currently implemented. These are then further subdivided into three directories, containing the optimisers frompints, fromscipy, and other optimisation algorithms used in fitting ion channel models that have been implemented specifically forionBench.Theproblemssubdirectory contains the classes for the available benchmarking problems. This features the problems from Loewe et al 2016 and Moreno et al 2016. In addition to these previously defined problems, we have introduced two further test problems, a Hodgkin-Huxley IKr model from Beattie et al 2017 and a Markov IKr model from Fink et al 2008.The final subdirectory,uncertainty, contains functions for determining uncertainty and unidentifiability in the problems, such as calculating profile likelihood plots and Fisher's Information Matrix.multistart.pyprovides a tool for rerunning an optimiser to derive average performace metrics.Thetestdirectory contains unit tests for ensuring changes do not break previous functionality.InstallationionBenchcan be installed using pip.pip install ionbenchNote thationBenchusesmyokitto do its simulations, which relies on CVODES (from Sundials). For Linux and Mac OS users a working installation of CVODES is required. For Windows users, CVODES should be automatically installed withmyokit.Getting StartedIf you want to useionBench, check out theintroductionandtutorialin thedocsdirectory.WorkflowThe intended workflow for using the benchmarker is to generate a benchmarker object, setup the optimisers modification and apply it to the benchmarker, and pass the benchmarker into the optimisation algorithm to evaluate. All optimisers should accept a single benchmarker as input with all other inputs being optional.import ionbench
bm = ionbench.problems.staircase.HH_Benchmarker()
modification = ionbench.optimisers.pints_optimisers.cmaes_pints.get_modification()
modification.apply(bm)
optimisedParameters = ionbench.optimisers.scipy_optimisers.nelderMead_scipy.run(bm)Future FeaturesBounds - Current only bounds on the parameters can be included in the benchmarker but it would be nice to have bounds on the rates. Additionally, it would be good to include barrier function style bounds to allow them to work nicely with gradient based methods.Parallelisation - Its not clear yet how well the benchmarker would handle being run in parallel (specifically for the tracker) but it is something that would be worth looking into.Real data - Both Moreno et al 2016 and Loewe et al 2016 include real data in the papers. It would be nice to see how the algorithms handle fitting to real data but its not clear how to best implement the performance metrics, two of which rely on knowing the true parameters.