Package: networkunit
ASciUnitlibrary for validation testing of spiking networks.InstallationpipinstallnetworkunitConceptThe NetworkUnit module builds upon the formalized validation scheme of the SciUnit package,
which enables the validation ofmodelsagainst experimental data (or other models) viatests.
A test is matched to the model bycapabilitiesand quantitatively evaluated by ascore.
The following figure illustrates a typical test design within NetworkUnit.
The blue boxes indicate the components of the implementation of the validation test, i.e.,
classes, class instances, data sets, and parameters.
The relation between the boxes are indicated by annotated arrows.The basic functionality is
shown by green arrows.  The difference in the test design for comparing against experimental
data (validation) and  another  simulation  (substantiation)  is  indicated  by  yellow  and
red  arrows,  respectively.  The  relevant  functionality  of  some  components  for  the
computation  of  test  score  is  indicated  by  pseudo-code.  The  capability
classProducesPropertycontains  the  functioncalc_property(). The testXYTesthas a functiongenerate_prediction()which makes use of this capability, inherited by the model class,
to generate a model prediction. The initialized test instanceXYTest_paramZmakes use of itsjudge()function to evaluate this model prediction and compute the scoreTestScore.
TheXYTestcan inherit from multiple abstract test classes (BaseTest),
which is for example used with theM2MTestto add the functionality of evaluating multiple model classes.
To make the test executable it has to be linked to a ScoreType and all free parameters need to be set
(by aParamsdict) to ensure a reproducible result.Showcase examples on how to use NetworkUnit can be foundin this repositoryand interactive reveal.js slides are
accessible via the launch-binder button at the top.Overview of testsClass nameParent classPrediction measuretwo_sample_test--firing_rate_testtwo_sample_testfiring ratesisi_variation_testtwo_sample_testinter-spike-intervals, their CV, CV2, LV, or LVRcovariance_testtwo_sample_testcovariancescorrelation_testtwo_sample_test-correlation_dist_testcorrelation_testcorrelation coefficientscorrelation_matrix_testcorrelation_testcorrelation coefficient matrixavg_std_correlation_testcorrelation_testavg or std of correlation coefficientsgeneralized_correlation_matrix_testcorrelation_matrix_testmatrix of derived cross-correlation measureseigenvalue_testcorrelation_testeigenvalues of the correlation coefficient matrixpower_spectrum_testtwo_sample_testpower spectral density valuesfreqband_power_testpower_spectrum_testpower spectral density values in frequency bandtimescale_testtwo_sample_testspike train timescalegraph_centrality_helperclasssciunit.Testgraph centrality measures of given adjacency matrixjoint_testtwo_sample_testjoining other test classes for multivariate evaluationInheritance order in case of multiple inheritance for derived test classes:classnew_test(sciunit.TestM2M,graph_centrality_helperclass,<base_test_class>)Overview of scoresClass nameTest nameComparison measurestudents_tStudent’t testsample meanks_distanceKolmogorov-Smirnov testsample distributionkl_divergenceKullback-Leibler divergencesample entropymwu_statisticMann-Whitney U testrank sumlevene_scoreLevene’s testsample varianceeffect_sizeEffect sizestandardized meanbest_effect_sizeBayesian estimation effect sizestandardized meanwasserstein_distanceWasserstein Distancemultivariate score distanceeigenangleEigenangle Testeigenangle similarityOverview of model classesModel nameCapabilityParent classPurposeloaded_spiketrainsProducesSpikeTrainssciunit.RunnableModelloading simulated spiking datastochastic_activityProducesSpikeTrainssciunit.RunnableModelgenerating stochastic spiking datanest_simulationProducesSpikeTrainssciunit.RunnableModeltemplate for NEST modelsOther validation test repositoriesNeuronUnitHippoUnitBasalUnitMorphoUnitCerebellumUnitAcknowledgmentsThis open source software code was developed in part or in whole in the Human Brain Project, funded from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under Specific Grant Agreements No. 720270, No. 785907 and No. 945539 (Human Brain Project SGA1, SGA2 and SGA3).