Package: medpicpy
MedPicPy is a Python library to simplify ingesting medical imaging
datasets for feeding in to deep learning pipelines.Medical imaging datasets can be difficult to read in with many
different file formats and dataset structures. MedPicPy
allows you to quickly get up and running with a dataset
so you can get initial results and see how changes affect
them.Table of ContentsWhy use MedPicPy?InstallationBasic UsageCode Examplemini-MIAS Breast Cancer ClassificationFull ScriptAPI ReferenceContributeWhy use MedPicPy?Turns data straight into numpy array format which to be fed
into a machine learning model.Streamlines reading in data so you can focus on the model.Simple functions that work with 2D, 3D or higher dimensional data.InstallationUsing pip:pip install medpicpyThe pip version should be up to date
but if you are desperate to download
and install from the repo then:git clone https://github.com/cdmacfadyen/MedPicPy
python -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r MedPicPy/requirements.txtBasic UsageGenerally for a machine learning dataset the
metadata about the image will be stored in a csv, in the
directory structure, or a combination of those two things.
This package has functions for obtaining paths to
images by searching a dataset for paths containing
certain strings (e.g. "CT" or "DX"). These paths can
then be passed into a MedPicPy image loading function
which takes the paths and returns the image data in
numpy format, ready to be used in a machine learning model.
See the examples below.Code ExampleThewiki pagecontains
several examples of how this can be used with different kinds of dataset.
Here is an example of how to ingest the mini-MIAS dataset for
breast cancer segmentation.mini-MIAS Breast Cancer ClassificationYou can find this dataset atthis link. It's small (~100Mb) so its a good place to get started with medical imaging data. Once you download it the metadata is contained in the README, so open that and copy it into a new file. For this example it has been moved into a file calleddata.txt.Import pandas and medpicpy and read the data using pandas.importmedpicpyasmedimportpandasaspddescription=pd.read_csv("mini-MIAS/data.txt",header=None,delim_whitespace=True)# delim whitespace because the data is space separatedNext we need to format the data so that we can feed it into our csv reading function. Currently the image names in the metadata do not match the actual image names. The lambda function appends ".pgm" to the end of all of the image names in this dataframe.description[0]=description[0].apply(lambdax:"{}.pgm".format(x))# append .pgm to image namesNow we can load in all the images withload_images_from_csvwhich takes the dataframe, where the image names are in the dataframe, the path to the image directory, and the desired output shape of each image (which will depend on the model you are using). This loads all of the images in to memory.images=med.load_images_from_csv(description,0,"mini-MIAS/",(224,224))The mini-MIAS data also has class and bounding box information and we can read those in too. First the data needs cleaned. We also resized the images in the last step so we need to move the bounding boxes by the right amount. We know from the metadata that the images were originally 1024 x 1024 so we can find the scale factor by finding our output image size over our input image size.classes=description[3]classes=classes.fillna("N")# convert bounding box data to numeric data typedescription[4]=pd.to_numeric(description[4],errors="coerce")description[5]=pd.to_numeric(description[5],errors="coerce")x_scale_factor=224/1024y_scale_factor=224/1024xs,ys,widths,heights=med.load_bounding_boxes_from_csv(description,4,5,6,6,x_scale_factor=x_scale_factor,y_scale_factor=y_scale_factor)You will probably want to convert your class data to a one-hot array,
sklearn'sOneHotEncoderis
useful for this.Full ScriptSee the full script here with a some simple visualisation code at the end.importpandasaspdimportmatplotlib.pyplotaspltimportmatplotlib.patchesaspatchesimportmedpicpyasmeddescription=pd.read_csv("mini-MIAS/data.txt",header=None,delim_whitespace=True)# delim whitespace because the data is space separateddescription[0]=description[0].apply(lambdax:"{}.pgm".format(x))array=med.load_images_from_csv(description,0,"mini-MIAS/",(224,224))description[4]=pd.to_numeric(description[4],errors="coerce")description[5]=pd.to_numeric(description[5],errors="coerce")classes=description[3]classes=classes.fillna("N")# N for normalx_scale_factor=224/1024y_scale_factor=224/1024xs,ys,widths,heights=med.load_bounding_boxes_from_csv(description,4,5,6,6,x_scale_factor=x_scale_factor,y_scale_factor=y_scale_factor)print(classes)image=array[0]fig,ax=plt.subplots()ax.imshow(image,cmap="gray")bbox=patches.Circle((xs[0],ys[0]),widths[0],linewidth=1,edgecolor="r",fill=False)ax.add_patch(bbox)plt.show()API ReferenceThere is an API reference on thepages site. It contains detailed documentation and examples of how functions can be used.ContributeFeel free to contribute! Check out the issues if you
want to find something to do, or add an issue if you think the
package could be extended. Pull requests will be accepted provided
they don't break anything and the feature is easy to use.Please ensure that all modules/functions added have docstrings, ideally
with an example usage and then run./scripts/makedocs.shto add the
documentation to the pages site.