Package: imdm
imdm- Python data models for images (and other stuff!)What's the pointThis module contains a few helpful and flexible data models which can be easily extended for other applications. Here, I focused on making some (very) specific validators focusing on MRI data. These can be used to test DICOM files, SITK-readable images or numpy-loadable arrays.InstallationThe project should be readily instalable withpoetry(recommended) by runningpoetry install.UsageThree fundamental abstract classes are used here:Check, which focuses on checking whether specific conditions are true given an input data,DataValidator, which focuses on validating a specific type of data, andDataModelwhich focuses on validating a sample (composed by multiple types of data).CheckACheckis a generic class that can be used as the base for data checks. These data checks, after being defined (check = Check(target="hello")), can then be called with a given input argument (check("hello)) and the output (True) tells us whether the check has passed (Trueif the check has passed,Falseotherwise).The check above considers only simple comparison - this is hardly useful when we want to perform more complicated comparisons. For example, we may want to know if our data is within a given range. To do this, we can define novel dataclasses fromTestthat performexactlythese comparisons, by redefining theunpackandcomparemethods:fromimdmimportTest@dataclassclassCheckRange(Test):target:Tuple[Union[int,float],Union[int,float]]def__post_init__(self):name=self.get_name()self._success_msg=f"Target shape{name}contains input values"self._fail_msg=f"Target shape{name}does not contain input values"defunpack(self,x:Any)->Any:returnnp.min(x),np.max(x)defcompare(self,unpacked_x:Any)->bool:within_range=Trueifself.target[0]isnotNone:ifunpacked_x[0]<self.target[0]:within_range=Falseifself.target[1]isnotNone:ifunpacked_x[1]>self.target[1]:within_range=Falsereturnwithin_rangeBy redefining ourunpackandcomparemethods, we can ensure that the correct checks are performed. These methods are then executed in the__call__method of theTestabstract class:def__call__(self,x:Any)->bool:"""Performs the comparison using the input data and defines the``self.msg``.Args:x (Any): input data.Returns:bool: whether the comparison was successful or not."""ifself.targetisNone:returnself.targetunpacked_x=self.unpack(x)result=self.compare(unpacked_x)ifresult==True:self.msg=self._success_msgifresult==False:self.msg=self._fail_msgreturnresultAmsgattribute is defined with either_success_msgor_fail_msg, depending on whether the result isTrueorFalse. This is helpful in case verbosity is important for your applications.DataValidatorDefining aDataValidatoris relatively easy:fromimdmimportDataValidatordata_validator=DataValidator(type=str,length=11,shape=None,range=None)and running data validations is just as easy:output=data_validator.validate("test_string")print(output)>>>{"type":True,"length":True,"shape":None,"range":None}TheDataValidatormethod automatically checks fortype,length,shapeandrange(if specified). If necessary, users can also add their own methods. For example, if you would to check whether a given path exists:importosdata_validator.add_check(key="path",check_fn=os.path.exists,data_stage="raw")output=data_validator.validate("test_string")print(output)>>>{"type":True,"length":True,"shape":None,"range":None,"path":False}Easy! All arguments are relatively clear, butdata_stageis somewhat more ellusive; for this reason I introduce here the concept of three data stages:raw- the input exactly as it is. This is useful to check whether a file exists.preprocessed_data- if apreprocess_fnis specified in theDataValidatorconstructor, checks can be applied to these functions. For instance, thetypecheck is automatically ran on thepreprocessed_datastage.value_data- some files (SITK-readable files, for instance) require some non-obvious wrangling before one can actually use their values asnumpyarrays, which is the assumed format for checking therange. This function (value_fn) is applied to the output ofpreprocess_fn.If nopreprocess_fnorvalue_fnare supplied, thenpreprocessed_dataandvalue_fnwill be identical to the input data.DataModelADataModelis simply a structure ofDataValidators, i.e.fromimdmimportDataValidator,DataModeldata_model=DataModel(structures={"a":DataValidator(type=str,length=11,shape=None,range=None),"b":DataValidator(type=int,length=None,shape=None,range=[-10,10])})Thisdata_modelcan then be applied to any given data input that follows a structure similar todata_model.structures.MRI- and image-specific data validatorsAn easy-to-use data validators have been implemented specifically for image data (ImageFile). I work with images, so these were especially useful for me.Additionally, since I work with a lot of MRI data, specific methods for MRI data were also implemented (DicomFileandSitkFile). A more generic method fornumpyfiles has also been (NumpyFile).pprintpprintis a simple function that allows you to more easily inspect the output ofDataValidatorandDataModel. It comes with colours!Unit testingTests for the data validators and models are available inimdm/data_models/testing. Test images were collected from:https://www.rubomedical.com/dicom_files/(.dcm)https://github.com/neurolabusc/niivue-images(.nii.gz)