Package: edc-metadata
edc_metadataedc-metadataputs a “metadata” layer on top of your data collection forms, namely CRFs and Requisitions. The “metadata” is used on the data entry dashboard (see alsoedc_dashboard). The metadata may be queried directly by a data manager to review the completion status of CRF and Requisition forms.Metadata is stored in two models,CrfMetaDataandRequisitionMetaData. One metadata record is created per form per visit. Metadata is only created for the data collection forms of a visit as defined in thevisit schedule.Metadata model instances are created for each visit when thevisitmodel is saved.edc_metadatareads from thevisit_scheduleto decide which data collection form metadata model instances to create for a visit. (Note: Seeedc_visit_schedule)Metadata is guaranteed to exist for every form defined in a visit after the visit form has been submitted.metadatamodel instancesEachmetadatamodel instance,CrfMetadataorRequisitionMetadata, is managed by
an actual CRF or REQUISITION model listed in thevisit_schedule.CrfMetadata` model instances are created for each CRF listed in the visit schedule. That is, if the visit schedule schedules a CRF for 5 different visits, 5 ``CrfMetadata` model instances will eventually be created. Metadata model instances are created when the ``visitmodel for a
timepoint is saved.
When yousavea CRF within a visit, theentry_statusof the the metadata instance`s
it manages is updated fromREQUIREDtoKEYED.The same applies toRequisitionMetadatafor REQUISITIONS.Entry statusBy default theentry_statusfield attribute is set toREQUIRED. You can change the default of each CRF toNOT_REQUIREDin your declaration in the visit schedule.  Seevisit_schedule.crf.The same applies to REQUISITIONS.metadata_rulesmanipulatemetadatamodel instancesmetadata_rulesare declared to manipulatemetadatamodel instances. The rules change theentry_statusfield attribute fromREQUIREDtoNOT_REQUIREDor visa-versa.
If the manager of the metadata instance, the CRF or REQUISITION model instance, exists, the entry status is updated toKEYED``andthe ``metadata_rulestargeting the metadata instance are ignored.metadata rulesare run on eachsaveof the visit and managing model instances.
If a value on some other form implies that your form should not be completed, your form`s metadata “entry_status” will change from REQUIRED to NOT REQUIRED uponsaveof the other form.
Metadata isupdatedthrough apost_savesignal that re-runs themetadata rules.See alsoedc_metadata_rulesGetting startedModels: Visit, Crfs and RequisitionsLet`s prepare the models that will be used in the scheduled data collection. These models are your visit models, crf models and requisition models.Your application also has one or moreVisitmodels. Each visit model is declared with theCreatesMetadataModelMixin:classSubjectVisit(CreatesMetadataModelMixin,PreviousVisitMixin,VisitModelMixin,RequiresConsentModelMixin,BaseUuidModel):appointment=models.OneToOneField(Appointment)classMeta(RequiresConsentModelMixin.Meta):app_label='example'YourCrfmodels are declared with theCrfModelMixin:classCrfOne(CrfModelMixin,BaseUuidModel):subject_visit=models.ForeignKey(SubjectVisit)f1=models.CharField(max_length=10,default='erik')classMeta:app_label='example'YourRequisitionmodels are declared with theRequisitionModelMixin:classSubjectRequisition(RequisitionModelMixin,BaseUuidModel):subject_visit=models.ForeignKey(SubjectVisit)f1=models.CharField(max_length=10,default='erik')classMeta:app_label='example'metadata_rulesAs described above,metadata_rulesmanipulate theentry_statusof CRF and Requisitionmetadata.metadata_rulesare registered tosite_metadata_rulesin modulemetadata_rules.py. Place this file in the root of your app. Each app can have onemetadata_rules.py.See alsoedc_metadata_rulesautodiscovering metadata_rulesAppConfig willautodiscoverthe rule files and print to the console whatever it finds:checking for metadata_rules …registered metadata_rules from application ‘edc_example’Inspect metadata_rulesInspectmetadata_rulesfrom the site registry:>>>fromedc_metadata.rules.site_metadata_rulesimportsite_metadata_rules>>>forrule_groupsinsite_metadata_rules.registry.values():>>>forrule_groupinrule_groups:>>>print(rule_group._meta.rules)(<edc_example.rule_groups.ExampleRuleGroup:crfs_male>,<edc_example.rule_groups.ExampleRuleGroup:crfs_female>)(<edc_example.rule_groups.ExampleRuleGroup2:bicycle>,<edc_example.rule_groups.ExampleRuleGroup2:car>)Writing metadata_rulesmetadata_rulesare declared in aRuleGroup. The syntax is similar to thedjangomodel class.Let`s start with an example from the perspective of the person entering subject data.
On a dashboard there are 4 forms (models) to complete. The “rule” is that if the subject
is male, only the first two forms should be complete. If the subject is female, only the
last two forms should be complete. So the metadata should show:Subject is Male:crf_one - REQUIRED, link to entry screen availablecrf_two - REQUIRED, link to entry screen availablecrf_three - NOT REQUIRED, link to entry screen not availablecrf_four - NOT REQUIRED, link to entry screen not availableSubject is Female:crf_one - NOT REQUIREDcrf_two - NOT REQUIREDcrf_three - REQUIREDcrf_four - REQUIREDARulethat changes themetadataif the subject is male would look like this:crfs_male=CrfRule(predicate=P('gender','eq','MALE'),consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['crfone','crftwo'])The rule above has apredicatethat evaluates to True or not. Ifgenderis equal
toMALEthe consequence isREQUIRED, elseNOT_REQUIRED. For this rule, for a
MALE, the metadataentry_statusforcrf_oneandcrf_twowill be updated toREQUIRED. For a FEMALE both will be set toNOT_REQUIRED.Rules are declared as attributes of a RuleGroup much like fields in adjangomodel:@register()classExampleRuleGroup(CrfRuleGroup):crfs_male=CrfRule(predicate=P('gender','eq','MALE'),consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['crfone','crftwo'])crfs_female=CrfRule(predicate=P('gender','eq',FEMALE),consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['crfthree','crffour'])classMeta:app_label='edc_example'RuleGroupclass declarations are placed in filemetadata_rules.pyin the root of
your application. They are registered in the order in which they appear in the file. All rule
groups are available from thesite_metadata_rulesglobal.IMPORTANTIf the related visit model (e.g. SubjectVisit) has a differentapp_labelthanMeta.app_label, aRuleGroupErrorwill be raised because theRuleGroupassumes
the app_labels are the same. To avoid this, specify the related visit modellabel_loweronMeta.For example:@register()classExampleRuleGroup(CrfRuleGroup):crfs_male=CrfRule(predicate=P('gender','eq','MALE'),consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['crfone','crftwo'])classMeta:app_label='edc_example'related_visit_model="edc_visit_tracking.subjectvisit"InheritanceWhen using single inheritance, set Meta classabstracton the base class:classExampleRuleGroup(CrfRuleGroup):crfs_male=CrfRule(predicate=P('gender','eq','MALE'),consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['crfone','crftwo'])classMeta:abstract=TrueclassMyRuleGroup(ExampleRuleGroup):classMeta:app_label='edc_example'related_visit_model="edc_visit_tracking.subjectvisit"More on RulesThe ruleconsequenceandalternativeaccept these values:fromedc_metadata.constantsimportREQUIRED,NOT_REQUIREDfromedc_metadata.rules.constantsimportDO_NOTHINGREQUIREDNOT_REQUIREDDO_NOTHINGIt is recommended to write the logic so that theconsequenceis REQUIRED if thepredicateevaluates toTrue.In the examples above, the rulepredicatecan only access values that can be found
on the subjects`s currentvisitinstance orregistered_subjectinstance. If the
value you need for the rulepredicateis not on either of those instances, you can
pass asource_model. With thesource_modeldeclared you would have these data
available:current visit model instanceregistered subject (seeedc_registration)source model instance for the current visitLet`s say the rules changes and instead of refering togender(male/female) you wish
to refer to the value field offavorite_transporton modelCrfTransport.favorite_transportcan be “car” or “bicycle”. You want the first rulepredicateto read as:Iffavorite_transportis equal tobicyclethen set the metadataentry_statusforcrf_oneandcrf_twoto REQUIRED, if not, set both to NOT_REQUIREDand the second to read as:Iffavorite_transportis equal tocarthen set the metadataentry_statusforcrf_threeandcrf_fourto REQUIRED, if not, set both to NOT_REQUIRED.The field for car/bicycle,favorite_transportis on modelCrfTransport. The
RuleGroup might look like this:@register()classExampleRuleGroup(RuleGroup):bicycle=CrfRule(predicate=P('favorite_transport','eq','bicycle'),consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['crfone','crftwo'])car=CrfRule(predicate=P('favorite_transport','eq',car),consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['crfthree','crffour'])classMeta:app_label='edc_example'source_model='CrfTransport'Note thatCrfTransportis acrfmodel in the Edc. That is, it has aforeign keyto the visit model. Internally the query will be constructed like this:# source model instance for the current visitvisit_attr='subject_visit'source_obj=CrfTansport.objects.get(**{visit_attr:visit})# queryset of source model for the current subject_identifiervisit_attr='subject_visit'source_qs=CrfTansport.objects.filter(**{'{}__subject_identifier'.format(visit_attr):subject_identifier})If the source model instance does not exist, the rules in the rule group will not run.If the target model instance exists, no rule can change it`s metadata from KEYED.More Complex Rule PredicatesThere are two provided classes for the rulepredicate,PandPF. WithPyou
can make simple rule predicates like those used in the examples above. All standard opertors
can be used.For example:predicate=P('gender','eq','MALE')predicate=P('referral_datetime','is not',None)predicate=P('age','<=',64)If the logic needs to a bit more complicated, thePFclass allows you to pass alambdafunction directly:predicate=PF('age',func=lambdax:Trueifx>=18andx<=64elseFalse)predicate=PF('age','gender',func=lambdax,y:Trueifx>=18andx<=64andy==MALEelseFalse)Rule predicates as functionsIf the logic needs to be more complicated than is recommended for a simple lambda, you can
just pass a function. When writing your function just remember that the rulepredicatemust always evaluate to True or False.The function will be called with:visit: the related_visit model instanceregistered_subject: the instance for the current subjectsource_obj: the model instance who triggered the post_save signalsource_qsdefmy_func(visit,registered_subject,source_obj,source_qs)->bool:ifregistered_subject.age_in_years>=18andregistered_subject.gender==FEMALE:returnTruereturnFalseThe function is then called on the RuleGroup like this:@register()classExampleRuleGroup(RuleGroup):some_rule=CrfRule(predicate=my_func,consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['crfone','crftwo'])classMeta:app_label='edc_example'source_model='CrfTransport'Grouping rule predicate functions withPredicateCollectionIf you have manyRuleGroupsand predicate functions, it is useful to collect your predicate functions into a class:classPredicates:household_head_model="edc_he.healtheconomicshouseholdhead"patient_model="edc_he.healtheconomicspatient"@propertydefhoh_model_cls(self):returndjango_apps.get_model(self.household_head_model)@propertydefpatient_model_cls(self):returndjango_apps.get_model(self.patient_model)defpatient_required(self,visit,**kwargs)->bool:required=Falseif(self.hoh_model_cls.objects.filter(subject_visit__subject_identifier=visit.subject_identifier).exists()andnotself.patient_model_cls.objects.filter(subject_visit__subject_identifier=visit.subject_identifier).exists()):required=hoh_obj.hoh==YESreturnrequiredthen you might do something like this in yourmetadata_rulesmodule:pc=Predicates()@register()classExampleRuleGroup(RuleGroup):some_rule=CrfRule(predicate=pc.household_head_required,consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['healtheconomicshouseholdhead'])some_other_rule=CrfRule(predicate=pc.patient_required,consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['healtheconomicspatient'])classMeta:app_label='edc_he'source_model="edc_he.healtheconomics"related_visit_model="edc_visit_tracking.subjectvisit"Setting a customPredicateCollectionfor a RuleGroup using MetaIf aRuleGrouphas its ownPredicateclass you can declare it on theMetaclass. Set thepredicateattribute to the name of the function to call.@register()classExampleRuleGroup(RuleGroup):some_rule=CrfRule(predicate="household_head_required",consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['healtheconomicshouseholdhead'])some_other_rule=CrfRule(predicate="patient_required",consequence=REQUIRED,alternative=NOT_REQUIRED,target_models=['healtheconomicspatient'])classMeta:app_label='edc_he'source_model="edc_he.healtheconomics"related_visit_model="edc_visit_tracking.subjectvisit"predicates=Predicates()Rule Group OrderIMPORTANT: RuleGroups are evaluated in the order they are registered and the rules within each rule group are evaluated in the order they are declared on the RuleGroup.Updating metadataIt is a good idea to updata metadata after code changes and data migrations. To do so just
run the management command:pythonmanage.pyupdate_metadataTestingSince the order in which rules run matters, it is essential to test the rules together. Seetestsfor some examples. When writing tests it may be helpful to know the following:the standard Edc model configuration assumes you have consent->enrollment->appointments->visit->crfs and requisitions.rules can be instected after boot up in the global registrysite_metadata_rules.all rules are run when the visit  is saved.More examplesSeeedc_examplefor working RuleGroups and how models are configured with theedc_metadatamixins. Thetestsinedc_metadata.rulesuse the rule group and model classes inedc_example.Notes on EdcThe standard Edc model configuration assumes you have a data entry flow like this::consent->enrollment->appointment->visit (1000)->crfs and requisitions
                     appointment->visit (2000)->crfs and requisitions
                     appointment->visit (3000)->crfs and requisitions
                     appointment->visit (4000)->crfs and requisitionsYou should also see the other dependencies,edc_consent,edc_visit_schedule,edc_appointment,edc_visit_tracking,edc_metadata, etc.SignalsIn thesignalsfile:visit model ``post_save``:Metadata is created for a particular visit and visit code, e.g. 1000, when thevisitmodel is saved for a subject and visit code using the defaultentry_statusconfigured in thevisit_schedule.Immediately after creating metadata, all rules for theapp_labelare run in order. Theapp_labelis theapp_labelof the visit model.crf or requisition model ``post_save``:the metadata instance for the crf/requisition is updated and then all rules are run.crf or requisition model ``post_delete``:the metadata instance for the crf/requisition is reset to the defaultentry_statusand then all rules are run.