Package: histoprep
HistoPrepPreprocessing large medical images for machine learning made easy!Description•Installation•Usage•API Documentation•CitationDescriptionHistoPrepmakes is easy to prepare your histological slide images for deep
learning models. You can easily cut large slide images into smaller tiles and then
preprocess those tiles (remove tiles with shitty tissue, finger  marks etc).InstallationInstallOpenSlideon your system and then install histoprep withpip!pipinstallhistoprepUsageTypical workflow for training deep learning models with histological images is the
following:Cut each slide image into smaller tile images.Preprocess smaller tile images by removing tiles with bad tissue, staining artifacts.Overfit a pretrained ResNet50 model, report 100% validation accuracy and publish it
inNaturelike everyone else.WithHistoPrep, steps 1. and 2. are as easy as accidentally drinking too much at the
research group christmas party and proceeding to work remotely until June.Let's start by cutting a slide from thePANDAkaggle challenge into
small tiles.fromhistoprepimportSlideReader# Read slide image.reader=SlideReader("./slides/slide_with_ink.jpeg")# Detect tissue.threshold,tissue_mask=reader.get_tissue_mask(level=-1)# Extract overlapping tile coordinates with less than 50% background.tile_coordinates=reader.get_tile_coordinates(tissue_mask,width=512,overlap=0.5,max_background=0.5)# Save tile images with image metrics for preprocessing.tile_metadata=reader.save_regions("./train_tiles/",tile_coordinates,threshold=threshold,save_metrics=True)slide_with_ink: 100%|██████████| 390/390 [00:01<00:00, 295.90it/s]Let's take a look at the output and visualise the thumbnails.jopo666@~$treetrain_tiles
train_tiles
└──slide_with_ink├──metadata.parquet# tile metadata├──properties.json# tile properties├──thumbnail.jpeg# thumbnail image├──thumbnail_tiles.jpeg# thumbnail with tiles├──thumbnail_tissue.jpeg# thumbnail of the tissue mask└──tiles[390entriesexceedsfilelimit,notopeningdir]That was easy, but it can be annoying to whip up a new python script every time you want
to cut slides, and thus it is recommended to use theHistoPrepCLI program!# Repeat the above code for all images in the PANDA dataset!jopo666@~$HistoPrep--input'./train_images/*.tiff'--output./tiles--width512--overlap0.5--max-background0.5As we can see from the above images, histological slide images often contain areas that
we would not like to include into our training data. Might seem like a daunting task but
let's try it out!fromhistoprep.utilsimportOutlierDetector# Let's wrap the tile metadata with a helper class.detector=OutlierDetector(tile_metadata)# Cluster tiles based on image metrics.clusters=detector.cluster_kmeans(num_clusters=4,random_state=666)# Visualise first cluster.reader.get_annotated_thumbnail(image=reader.read_level(-1),coordinates=detector.coordinates[clusters==0])I said it was gonna be easy! Now we can mark tiles in cluster0as outliers and
start overfitting our neural network! This was a simple example but the same code can be
used to cluster all severalmilliontiles extracted from thePANDAdataset and discard
outliers simultaneously!CitationIf you useHistoPrepto process the images for your publication, please cite the github repository.@misc{histoprep,
  author = {Pohjonen, Joona and Ariotta, Valeria},
  title = {HistoPrep: Preprocessing large medical images for machine learning made easy!},
  year = {2022},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {https://github.com/jopo666/HistoPrep},
}