Package: edc-data-manager
edc-data-managerData manager administrative models and classes.edc-data-manageradds models and functionality to the Edc that compliment the role of the clinical trial data manager.
TheData Queryform guides the data manager in describing missing, incomplete or incorrect participant data. TheData Queryis then made available to research staff on the participant’s dashboard and administrative pages. Additionally, the data manager
can defineQuery Rulesthat scan the dataset for participant data that match the rule’s criteria. For each match found,edc-data-managerautomatically creates aData Query.For automated queries, those created when aQuery Ruleis run,edc-data-managerwill re-run aQuery Ruleupon updates
to the participant data. If the criteria is no longer met, theData Queryis automatically closed.User Rolesedc_data_manageradds theDATA MANAGERand theDATA_QUERYuser groups.
Members of theDATA_QUERYgroup can respond to existing data queries by completing  the “site response” section of the form.
They do not have permissions to change the criteria of the data query. Research staff responsible for submitting participant
data are typically given membership to theDATA_QUERYuser group.Members of theDATA MANAGERuser group can add/change/delete anyData Queryform andQuery Ruleform.
Data managers, those who oversee data collection but do not submit data themselves, are typically members of theDATA MANAGERuser group.The Data Query ModelThe central model ofedc_data_manageris theData Querymodel. AData Queryis either created manually by a data manager
or automatically when aQuery Ruleis run. Thedata querydescribes an issue for the attention of the research staff.
A data query might be general, describing the issue with nothing more than a free text comment, or specific. To allow for a specific
data query, theQuery Ruleform has questions where the data manager (orQuery Rule) can select the relevant timepoints, form questions,
and timing.Data query statusTheData Querystatus is split between two fields, one managed by the research staff and one by the data manager. Initially the
research staff status is set toNewand the data manager’s status is set toOpen.The available states of aData Queryare similar to those used on ticketing systems; namely, New, Open , Feedback, Resolved, Closed.
Only a data manager can close a data query.Members of theDATA_QUERYgroup can:Open the query – indicating they are working on the issue. (Open)Request feedback – indicating they need assistance from the data manager .(Feedback)Resolve the query – indicating the issue is resolved. (Resolved)This status field, managed by members of theDATA_QUERYgroup, is in a seperate section of theData Queryform.Members of theDATA_MANAGERhave their own section on the form and can:Resolve the query (but only if the site status is alsoResolved)Resolve with an action planSince data managers have full permissions to the form, they can override the status set by the research staff.Query RulesQuery Rules define criteria to be used to scan the dataset for data problems. In a query rule you can define the following:CRF modelRelated requisition panel name (if applicable)CRF questionsTimepoints (visits)Timing (when to run the query, e.g. 48 hours after the visit report is submitted)PriorityResearch staff contactData manager contactAs mentioned above, aData Querycan be automatically created by aQuery Rule. SimpleQuery Rulesare defined using theQuery Ruleform.Query Rulesare run by a “handler”. The default handler is sufficient in most cases. If not, a custom handler can be written, registered withedc_data_manager, and selected on theQuery Ruleform.When are Query Rules run?Query rules can be run as an Admin action from the QueryRule Admin page;A Query rule is run when a model is modified;Query rules can be scheduled ifdjango-celeryis installed.Data Queries trigger action itemsWhen a data query is created, a supporting action item is also created. As with all action items, this means an “alert”
shows on the participant dashboard and users can be notified by email and/or SMS. Seeedc_action_item.QueryRuleHandler – the default rule handlerFor each timepoint specified in theQuery Rule, the handler:checks to see if the visit report has been submitted. If not, the rule is ignored.checks if the requisition has been completed (if a requisition panel is linked to the query rule). If the requisition has not been completed, aData Queryis created immediately.gathers each value specified in the list of CRF form questions and callsinspect_model.ifinspect_modelreturns False, theData Queryis created or re-opened.ifinspect_modelreturns True, no data query is created or the existingData Queryis resolved.Custom rule handlersThe default rule handler,QueryRuleHandler, already does a lot, but it cannot satisfy all cases. The defaultinspect_modelmethod
does most of the form specific work ofQueryRuleHandler. In the default implementation ofinspect_model, a blank field value is
considered invalid andinspect_modelreturnsFalse. This may be fine if theQuery Ruleis just looking for just a single field value
but not for a combination of values. When looking for a combination of field values, a blank field value may be valid. In such cases
you can override theinspect_modelmethod and specify the correct logic for the desired data condition.For example:# data_manager.pyfromambition_subject.constantsimportAWAITING_RESULTSfromedc_constants.constantsimportNOT_DONE,YES,NOfromedc_data_manager.handlersimportQueryRuleHandlerfromedc_data_manager.site_data_managerimportsite_data_managerclassLumbarPunctureHandlerQ13(QueryRuleHandler):name="lumbar_puncture_q13"display_name="Lumbar Puncture (Q13, 15, 21, 23, 24)"model_name="ambition_subject.lumbarpuncturecsf"@propertydefinspect_model(self):"""Lumbar Puncture/Cerebrospinal Fluid 13, 15, 21, 23, 24.
        """valid=Falseifself.get_field_value("csf_culture")==AWAITING_RESULTS:passelifself.get_field_value("csf_culture")==NOT_DONE:valid=Trueelifself.get_field_value("csf_culture")==YES:if(self.get_field_value("other_csf_culture")andself.get_field_value("csf_wbc_cell_count")andself.get_field_value("csf_glucose")andself.get_field_value("csf_protein")and(self.get_field_value("csf_cr_ag")orself.get_field_value("india_ink"))):valid=Trueelifself.get_field_value("csf_culture")==NO:if(self.get_field_value("csf_wbc_cell_count")andself.get_field_value("csf_glucose")andself.get_field_value("csf_protein")and(self.get_field_value("csf_cr_ag")orself.get_field_value("india_ink"))):valid=Truereturnvalidsite_data_manager.register(LumbarPunctureHandlerQ13)Note the use ofget_field_valuemethod instead of directly accessing the model instance. This is not absolutely necessary but
avoids confusion by ensuring you only access fields defined in theQuery Rule.Registering custom rule handlersedc_data_managerhas a site registry thatautodiscoversmoduledata_manager.pyin the root of each app inINSTALLED_APPS.For example:# data_manager.pyfromedc_data_manager.handlersimportQueryRuleHandlerfromedc_data_manager.site_data_managerimportsite_data_managerclassMyCustomHandler(QueryRuleHandler):name="my_custom_handler"display_name="My Custom Handler"model_name="my_app.somecrf"@propertydefinspect_model(self):valid=Falseifself.get_field_value("field_one")==1:...somemorecodethateventuallysetsvalidtoTruereturnvalidsite_data_manager.register(MyCustomHandler)Dumping and loading a QueryRule fixturepythonmanage.pydumpdataedc_data_manager.queryrule--natural-foreign--natural-primary--indent4-oqueryrule.jsonpythonmanage.pyloaddataqueryrules.jsonUpdating query rulesQuery rules can be triggered manually to run from the admin action under theQueryRuleadmin page.Ifceleryis enabled, theupdate_query_ruleswill try to send proccessing to the MQ.See alsoupdate_query_rules,update_query_rules_action.Rerun form validationYou can use theFormValidationRunnerto rerun form validation on all instances for a model.You could do this:runner=FormValidationRunner(modelform)runner.run()You could also run for every model in your EDC deployment by getting theModelFormclass
from theadminregistry and runningFormValidationRunner:fromdjango.appsimportappsasdjango_appsfromedc_data_manager.form_validation_runnersimport(FormValidationRunner,FormValidationRunnerError,get_modelform_cls,)forapp_configindjango_apps.get_app_configs():ifapp_config.name.startswith("edc_"):continueformodel_clsinapp_config.get_models():ifmodel_cls==Appointment:continueprint(model_cls._meta.label_lower)try:modelform=get_modelform_cls(model_cls._meta.label_lower)exceptFormValidationRunnerErrorase:print(e)else:print(modelform)try:runner=FormValidationRunner(modelform)exceptAttributeErrorase:print(f"{e}. See{model_cls._meta.label_lower}.")else:try:runner.run()except(AttributeError,FieldError)ase:print(f"{e}. See{model_cls._meta.label_lower}.")You could also create a customFormValidationRunnerfor your model to add extra fields and ignore others.For example:classAppointmentFormValidationRunner(FormValidationRunner):def__init__(self,modelform_cls:ModelForm=None,**kwargs):modelform_cls=modelform_clsorAppointmentFormextra_fieldnames=["appt_datetime"]ignore_fieldnames=["appt_close_datetime"]super().__init__(modelform_cls=modelform_cls,extra_formfields=extra_fieldnames,ignore_formfields=ignore_fieldnames,**kwargs,)