Package: metrics-gen
metrics-gendummy metrics generatorGetting StartedMetrics generator is built upon three main components:Deployment: The indexes of the table, for example:symbol in stock market.(data_center, device_id) for devices in data centersStatic Data: Static data regarding the deployment, for example:model_number for a devicescore for a modelMetrics: Continuous metrics to generate about the deployment, for example:cpu_utilization of a deviceprice of a stockThe first step in setting up the generator is creating a deployment.  Then using the deployment, you can generate static data or continuous stream of metrics.Create a deployment from configurationTo create a deployment from configuration you need to provide ayamlfile containing the following:deployment:<level_name>:faker:<faker_type>num_items:<num_items in the level>Wherelevel_namewill be the name of the index,faker_typeis the name of thefaker generatorandnum_itemsis how many keys to create for this index.Each provided level will create anothernum_itemsinstances for each entry in it's previous levels.Example: Given the following configuration yaml file:deployment:device:faker:msisdnnum_items:2core:faker:msisdnnum_items:2and running the following command:frommetrics_gen.deployment_generatorimportdeployment_generatordep_gen=deployment_generator()deployment=dep_gen.generate_deployment(configuration=configuration)Will generate the following example deployment:devicecore041202719116776950611701382141202719116772255426557707241202719116777717168891372322601580028863213635322383422601580028864007792940086522601580028863720953132595Noticethat each extra level, multiplies the number of items created bynum_item, thus we got 2 * 3 = 6 items created.Create Static DataTo create a static data generator you need to supply a deployment dataframe and a configuration yaml.The static data generator knows how to generator from two kinds of feature configurations:rangeandchoicewhich should be specified in the yaml.static:<feature_name>:kind:rangemin_range:<min_feature_range>, defaults to 0max_range:<max_feature_range>as_integer:<int or float>, defaults to False<feature_name>:kind:choicechoices:<list of possible choices>Each provided feature will generate a new feature column in the generated dataframe.Example: Given the following yaml:static:models:kind:rangemin_range:10max_range:15as_integer:Truecountry:kind:choicechoices:[A,B,C,D,E,F,G]And the previous deployment:frommetrics_gen.static_data_generatorimportStatic_data_generatorstatic_data_generator=Static_data_generator(deployment,static_configuration)generated_df=static_data_generator.generate_static_data()Will generate the following dataframe:devicecoremodelscountry04120271911677695061170138213A14120271911677225542655770714C24120271911677771716889137214G32260158002886321363532238314G42260158002886400779294008611G52260158002886372095313259514DCreate Continuous MetricsTo create a continuous metrics stream you need to provide a deployment dataframe and metrics creation configuration yaml.errors:rate_in_ticks:< ~ticks between errors>length_in_ticks:< ~length of error mode>timestamps:interval:<time between samples in seconds>stochastic_interval:<create random intervals (around interval)>metrics:<metric name>:accuracy:<decimals to produce>distribution:normaldistribution_params:mu:<mean>noise:<noise>sigma:<std>is_threshold_below:<True to produce max when in error mode, False for min>past_based_value:<True to add the latest metric to the last result (like in daily stock market), False to replace normally)produce_max:<True for candles-like presentation>produce_min:<True for candles-like presentation>validation:distribution:# per-sample validationmax:<max value for individual sample>min:<min value for individual sample>validate:<True to activate validation>metric:# metric level validationsmax:<max value for overall-metric sample (only applicable to past-based-values)>min:<min value for overall-metric sample (only applicable to past-based-values)>validate:<True to activate validation>Each configured feature will generate additional metric for your deployment.Example: Given the following yamlerrors:{length_in_ticks:10,rate_in_ticks:5}timestamps:{interval:5s,stochastic_interval:true}metrics:cpu_utilization:accuracy:2distribution:normaldistribution_params:{mu:70,noise:0,sigma:10}is_threshold_below:truepast_based_value:falseproduce_max:falseproduce_min:falsevalidation:distribution:{max:1,min:-1,validate:false}metric:{max:100,min:0,validate:true}throughput:accuracy:2distribution:normaldistribution_params:{mu:250,noise:0,sigma:20}is_threshold_below:falsepast_based_value:falseproduce_max:falseproduce_min:falsevalidation:distribution:{max:1,min:-1,validate:false}metric:{max:300,min:0,validate:true}And the previous deployment:frommetrics_gen.metrics_generatorimportGenerator_dfmetrics_generator=Generator_df(metrics_configuration,user_hierarchy=deployment)generator=metrics_generator.generate(as_df=True)df=next(generator)Will generate the following dataframe:timestampcoredevicecpu_utilizationcpu_utilization_is_errorthroughputthroughput_is_erroris_error2022-01-31 19:20:21.00708721133098316734469221325973100.0True0.0TrueTrue2022-01-31 19:20:21.00708721159336860874469221325973100.0True235.0679405785135FalseFalse2022-01-31 19:20:21.0070870175482390171446922132597370.26657388732976False208.34378630077305FalseFalse2022-01-31 19:20:21.0070871626403145660403889087842659.932750968399404False217.4335871243806FalseFalse2022-01-31 19:20:21.0070877247058922310403889087842683.98361382584898False265.3476318369042FalseFalse2022-01-31 19:20:21.00708770302391280614038890878426100.0False225.16604191632058FalseFalseTo generate new samples all we need to do is callnext(generator)and a new sample will be generated.