Package: black-it
*Black-box abm calibration kitBlack-it is an easy-to-use toolbox designed to help you calibrate the parameters
in your agent-based models and simulations (ABMs), using state-of-the-art
techniques to sample the parameter search space, with no need to reinvent the
wheel.Models from economics, epidemiology, biology, logistics, and more can be dealt
with. The software can be used as-is - if your main interest is the ABM model
itself. However, in case your research thing is to, e.g., devise new sampling
strategies for ginormous search spaces and highly non-linear model, then you can
deploy and test your new ideas on a solid, reusable, modular foundation, in a
matter of days, with no need to reimplement all the plumbings from scratch.InstallationThis project requires Python v3.8 or later.To install the latest version of the package fromPyPI:pip install black-itOr, directly from GitHub:pip install git+https://github.com/bancaditalia/black-it.git#egg=black-itIf you'd like to contribute to the package, please read theCONTRIBUTING.mdguide.Quick ExampleThe GitHub repo of Black-it contains a series ready-to-run calibration examples.To experiment with them, simply clone the repo and enter theexamplesfoldergit clone https://github.com/bancaditalia/black-it.git
cd black-it/examplesYou'll find several scripts and notebooks. The following is the script namedmain.py, note that copying and pasting
the lines below will not work in general as the script needs to be inside the "examples" folder in order to run correctly.importmodels.simple_modelsasmdfromblack_it.calibratorimportCalibratorfromblack_it.loss_functions.msmimportMethodOfMomentsLossfromblack_it.samplers.best_batchimportBestBatchSamplerfromblack_it.samplers.haltonimportHaltonSamplerfromblack_it.samplers.random_forestimportRandomForestSamplertrue_params=[0.20,0.20,0.75]bounds=[[0.10,0.10,0.10],# LOWER bounds[1.00,1.00,1.00],# UPPER bounds]bounds_step=[0.01,0.01,0.01]# Step size in range between boundsbatch_size=8halton_sampler=HaltonSampler(batch_size=batch_size)random_forest_sampler=RandomForestSampler(batch_size=batch_size)best_batch_sampler=BestBatchSampler(batch_size=batch_size)# define a model to be calibratedmodel=md.MarkovC_KP# generate a synthetic dataset to test the calibratorN=2000seed=1real_data=model(true_params,N,seed)# define a lossloss=MethodOfMomentsLoss()# define the calibration seedcalibration_seed=1# initialize a Calibrator objectcal=Calibrator(samplers=[halton_sampler,random_forest_sampler,best_batch_sampler],real_data=real_data,model=model,parameters_bounds=bounds,parameters_precision=bounds_step,ensemble_size=3,loss_function=loss,random_state=calibration_seed,)# calibrate the modelparams,losses=cal.calibrate(n_batches=15)print(f"True parameters:{true_params}")print(f"Best parameters found:{params[0]}")When the calibration terminates (~half a minute), towards the end  of the output
you should see the following messages:True parameters:       [0.2, 0.2, 0.75]
Best parameters found: [0.19 0.21 0.68]DocsBlack-it calibration is initiated via theCalibratorwhich,
when called, performs three main steps.First, aSampleris summoned to suggest a set of promising
parameter configurations to explore.Second, themodelto be calibrated is simulated for
all the selected parameters.Third, a specificloss function, measuring the goodness of fitness
of the simulation data with respect to the real data, is evaluated.These steps are performed in a loop, and this allows the samplers to progress towards better parameter values
by exploiting the knowledge of previously computed loss functions.A more detailed explanation of how Black-it works is availablehere, while the full documentation -complete with examples
and tutorials- is availablehere.CitingBlack-itA description of the package is availablehere.Please consider citing it if you found this package useful for your research@article{black_it,title={Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models},journal={Journal of Open Source Software},publisher={The Open Journal},year={2022},volume={7},number={79},pages={4622},doi={10.21105/joss.04622},url={https://doi.org/10.21105/joss.04622},author={Marco Benedetti andGennaro Catapano andFrancesco {De Sclavis} andMarco Favorito andAldo Glielmo andDavide Magnanimi andAntonio Muci}}DisclaimerThis package is an outcome of a research project. All errors are those of the authors. All views expressed are personal views, not those of Bank of Italy.* Credits toSara Corbofor the logo.