# Experimenting with Agent-Based Model Simulation Tools
## 5 ABM Tools Evaluation
### Hands-on Developer Experience  
A critical quality of each ABM tool is the perceived hands-on experience. To evaluate this characteristic, we considered the following elements. _Installation and setup_. Perceived difficulty in installing and configuring the tool. * _N/A_: installation guide not available. * _Easy_: an installation script or installer is provided, or the tool can be used as a standard library. * _Hard_: information about the installation procedure is vague or it requires technical expertise. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & & \multicolumn{2}{c}{_DECLARED_} & _EASE_ & _OF_ & _USE_ \\ \cline{3-6}  & & **Very low** & **Low** & **Medium** & **High** & **Very high** \\ \hline  & & & & & & Insight Maker \\  & **Very low** & & & & & NetLogo \\  & & & & & Mesa & GAMA \\ D & & & & & & \\ E & & & & & & \\ E & & & & & & \\ E & & & & & & \\ F & & & & & & \\ F & & & & & & \\ F & & & & & & \\ I & & & & & & \\ C & & & & & & \\ I & & & & & & \\ E & & & & & Agents\_jl & \\ N & & & & & & \\ C & & & & & & \\ Y & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Trade-off between the declared ease of use and efficiency of the ABM platforms. Documentation and examples.Extensiveness of the documentation and presence of examples and tutorials. * N/A: no documentation and/or tutorials are available. * Basic: most functionalities are not documented or only essential information is provided; few or no examples are given. * Good: all functionalities are documented; some examples are given. * Extensive: all functionalities are extensively documented; several comprehensive examples exist. Effort.Effort required to implement a model, measured in hours.Problems.Errors and issues that prevented the tool's installation or usage. Hands-on evaluation process.The hands-on programming experience was evaluated by two developers with a strong background in computer science, relevant expertise in the ABM field, and some experience with most of the ABM tools considered. Both developers independently installed each ABM tool reviewed in this work and tried to solve any arisen problems before moving on to the model development phase. At the end of this process, both developers rated each tool according to (i) its installation and setup procedures, (ii) the comprehensiveness of its documentation and examples, (iii) the effort required to develop a single model, and (iv) documented whether they encountered any issues in the process (as detailed above). Finally, both developers filled in Table 6, discussing any discrepancy until agreement. The upper part of the Table includes all tools correctly installed and used; the lower part lists the remaining tools, clarifying the problems encountered in the installation phase or when running examples. The number of hours describing the effort required refers to the average time needed to develop a single model since each has different peculiarities. In the case of ready-to-use ABM models (e.g., Flockers in NetLogo), the developers estimated the effort required to adapt them to the experiment's requirements (see Section 5.2). \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Tool** & \
\begin{tabular}{l} **Installation and** \\ **Setup** \\ \end{tabular} &
\begin{tabular}{l} **Documentation** \\ **and Examples** \\ \end{tabular} & **Effort** & **Problems** \\ \hline
**ActressMAS** & N/A & Basic & \(\sim\)4 h & No installation information provided; only works with \\  & & & VisualStudio. \\ \hline
**AgentPy** & Easy & Good & \(\sim\)2 h & \\
**Agents.jl** & Easy & Extensive & \(\sim\)2 h & \\
**CppyABM** & Easy & Basic & \(\sim\)4 h & \\
**GAMA** & Easy & Extensive & \(\sim\)2 h & \\
**krABMaga** & Easy & Good & \(\sim\)3 h & \\
**MASON** & Easy & Extensive & \(\sim\)3 h & \\
**Mesa** & Easy & Extensive & \(\sim\)2 h & \\
**NetLogo** & Easy & Extensive & \(\sim\)3 h without & \\  & & & VSL & \\ \hline
**Repast** & Easy & Good & \(\sim\)4 h & \\ \hline \hline \end{tabular}
\end{table}