TITLE: HTAN Multiplex IHC Image Cytometry V0.1
AUTHORS: Shamilene Sivagnanam, Courtney Betts, Lisa Coussens

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block">Understanding immune complexity within the tumor microenvironment provides valuable insight to tumor-immune composition, spatial interactions, and the immune system's reaction to therapy allowing us to better understand response and resistance at the interface of the tumor and host. </div><div class = "text-block"><span>Multiplex IHC allows for the simultaneous detection of multiple Ab targets in the same tissue section using published methods previously described</span><span style = "vertical-align:super;">1,2</span><span>.This protocol contains detailed instructions on how to prepare files and run the image processing pipeline for mIHC SVS images in a GUI or command line, making it accessible to biologist and labs without computational specialty. </span></div><div class = "text-block"><span style = "vertical-align:sub;">REFERENCES</span></div><div class = "text-block"><ol style = "list-style-type: decimal;"><li style = "counter-reset:ol0;"><span>Tsujikawa, Takahiro, et al. "Quantitative multiplex immunohistochemistry reveals myeloid-inflamed tumor-immune complexity associated with poor prognosis."</span><span style = "font-style:italic;">Cell reports</span><span>19.1 (2017): 203-217</span></li><li style = "counter-reset:ol0;"><span>Banik, Grace, et al. "High-dimensional multiplexed immunohistochemical characterization of immune contexture in human cancers."</span><span style = "font-style:italic;">Methods in enzymology</span><span>635 (2020): 1-20.</span></li></ol></div></div>

[STEPS]
?. [File prep and Folder Structure]
FOLDER AND FILE NAMING CONVENTIONMake sure folder and file naming convention is consistent throughout the entire study. Filenaming convention should be used when saving images during acquisition.
NAMING SLIDE FOLDERS AND IMAGE FILESEach slide folder in a cohort batch should maintain some consistent format of letters and numbers:Example slide folder name: 1 letter, 6 numbersH342253H342259Each newly scanned image from one slide should be named with the following file naming convention at acquisition:Initials_StudyID_UniqueSlideID_PanelName_CycleRound_Marker.SVSExample file name:GG_HTAN1_H342253_D18_C01R1_PD1.svsGG_HTAN1_H342253_D18_C01R2_PDL1.svs
?. [File prep and Folder Structure]
SET UP FOLDER STRUCTUREThe pipeline is optimal for batches of slides that run from a 'Parent' folder, that is the study or batch directory.The study folder structure should be organized as the cohort/study/batch of slides as the parent folder, with each slide within the study containing all marker files with no duplicate scans of markers. There should be an initial hematoxylin, a second hematoxylin after antigen retrieval, and a third hematoxylin at the end of the panel - see mIHC wet lab protocol for details: Below is an example of the folder structure of a study called HTAN1 that contains 1 slide (H342253), 17 marker images, and 3 hematoxylin stained images.
Example study folder structure:├── HTAN1 (Parent Study)│   ├── H342253 (Slide 1)│   │   ├── GG_HTAN1_H342253_D18_C00R1_HEM.svs│   │   ├── GG_HTAN1_H342253_D18_C01R1_PD1.svs│   │   ├── GG_HTAN1_H342253_D18_C01R2_PDL1.svs│   │   ├── GG_HTAN1_H342253_D18_C01R3_HEM.svs│   │   ├── GG_HTAN1_H342253_D18_C02R1_HLADPDQDR.svs│   │   ├── GG_HTAN1_H342253_D18_C02R2_DCLAMP.svs│   │   ├── GG_HTAN1_H342253_D18_C03R1_CD3.svs│   │   ├── GG_HTAN1_H342253_D18_C03R2_CD45.svs│   │   ├── GG_HTAN1_H342253_D18_C04R1_CD8.svs│   │   ├── GG_HTAN1_H342253_D18_C05R1_GRZB.svs│   │   ├── GG_HTAN1_H342253_D18_C05R2_CD20.svs│   │   ├── GG_HTAN1_H342253_D18_C06R1_CD68.svs│   │   ├── GG_HTAN1_H342253_D18_C07R1_CD66B.svs│   │   ├── GG_HTAN1_H342253_D18_C07R2_CD11C.svs│   │   ├── GG_HTAN1_H342253_D18_C08R1_CD163-001.svs│   │   ├── GG_HTAN1_H342253_D18_C09R1_EOMES.svs│   │   ├── GG_HTAN1_H342253_D18_C10R1_FOXP3.svs│   │   ├── GG_HTAN1_H342253_D18_C11R1_KI67.svs│   │   ├── GG_HTAN1_H342253_D18_C11R2_PANCK.svs│   │   ├── GG_HTAN1_H342253_D18_C11R3_HEM.svs
?. [Get Code and Required Software ]
DOWNLOAD SOFTWARE AND CODEDownload or clone the MultiplexIHC_Code.zip file that contains all the functions and code for image processing. Save and unzip this folder somewhere other than your Downloads directory.git@github.com:Sivoyer/MultiplexIHC_Code.git├── AEC_CMYK_ColorDecon.ijm├── CP_DiscoveryPanelTemplate.cpproj├── CommandLine│   ├── AEC_CMYK_ColorDecon.txt│   ├── CL_Registration│   │   ├── get_image_sets.m│   │   ├── parse_xml.m│   │   ├── register_SURF.m│   │   ├── run_mIHC_registration.m│   │   └── sort_regions.m│   └── Nuclei_BinaryMask.txt├── Nuclei_BinaryMask.ijm├── README.md└── Registration├── get_image_sets.m├── mIHC_Registration.m├── parse_xml.m├── register_SURF.m└── sort_regions.mIn addition to this code, you will also need the following software installed:Registration is performed in Matlab R2018b or higher:Additional toolboxes required: Computer Vision ToolboxColor Deconvolution and Nuclei Segmentation is performed in FIJI:Additional files required: To install this .class file, drag and drop into FIJI bar, then close and restart the program.Single Cell Quantification is performed in Cell Profiler 3:
?. [Pipeline Overview]
PIPELINE OVERVIEWOnce file names and folder structure is properly formatted for mIHC image processing, batch analysis can be performed. For every slide, multiple regions can be selected, avoiding regions of tissue deformation, necrosis, or other artifact or trouble tissue regions. The image processing workflow utilizes steps commonly found in biological image analysis that have been optimized for brightfield RGB multplexIHC images with AEC chromogen and hematoxylin. Although image processing can be completely automated, it is highly recommended that each step is checked to ensure data is as expected and is of high enough quality to continue processing. The modular steps allow for QC and adjustments at each major step. Brief Description of Image Processing Workflow Each region is registered to the final stained hematoxylin image from the same area, aligning each image such that major shifts due to repetitive scanning are adjusted for. Color deconvolution separates the AEC conjugated Ab signal from background. Single cell nuclei segmentation identifies cell areas using hematoxylin stain. Each identified cell is measured for mean intensity of every signal extracted marker.
?. [Region of Interest Selection]
SELECT REGIONS OF INTEREST Regions are annotated in Aperio ImageScope, available for Windows only.Annotating Regions on final hematoxylinRegions of interest are annotated on the last hematoxylin image to ensure that only cells maintained throughout all staining cycles and rounds are evaluated in downstream analysis. Tissue areas that are lost, deformed, folded, or of poor quality should be excluded from analysis.
DRAWING REGIONS IN IMAGESCOPE1. Open the last cycle hematoxylin image in ImageScope.         Example:  GG_HTAN1_H342259_D18_C11R3_HEM.svs2. Select the rectangle tool for annotating boxes.3. Draw boxes on areas of tissue that are of good quality (no folds, tissue loss, tissue deformation       - Box sizes can be uniformly sized in settings, but only square/rectangle regions accepted.       - Boxes should consist of tissue area with          (constraint of ROI cell classification downstream if using FCS Express Image Cytometry) 4. When ROIs have been annotated, you can view the list of them on the Annotations - Detailed View in ImageScopeFor more instructions on using Aperio ImageScope please see the manual:   5. Close the SVS file and you will be prompted to save annotations. Select yes, and ImageScope will save an XML with the same filename as the image it was drawn on (last hem). Do not rename or move this file out of the slide folder containing the SVS images.
?. [Workflow Options]
Pipeline Run OptionsThis can be run using the software interfaces (code free), or by terminal. Instructions for both methods are listed below:
?. [Image Registration]
IMAGE REGISTRATIONHow registration works:Image registration is performed in Matlab. Each marker image is registered to the last cycle target hematoxylin image. Keypoints are identified on the blue channel of the hematoxylin image and the blue channel of the AEC marker image.If not enough matching keypoints are found, other single RGB channels are checked for more keypoints, and the channels with the highest strongest matching indexed keypoints are used. A similarity based geometric transform is generated and applied to each channel of the marker image and merged. The registered RGB image is saved as a TIFF inside an automatically generated 'Registered_Regions' folder created in the slide/region folder.Replace  with the path to your matlab application. For macOS, it is likely: /Applications/MATLAB_R2018b.app/bin/matlabReplace  with the path to the location where you saved the MultiplexIHC_RegistrationCode directory.Replace  with the path to your Parent directory containing all slides to be registered.Replace  with the path to the location where you saved the MultiplexIHC_RegistrationCode directory.Replace  with the path to your Parent directory containing all slides to be registered.Example:"/Applications/MATLAB_R2018b.app/bin/matlab" -nodesktop -nosplash -r "cd('/Users/sivagnan/MultiplexIHC_Code/CommandLine/CL_Registration');run_mIHC_registration('/Users/sivagnan/PROJECTS/Cohort1');exit;"
?. [Image Registration Quality Control]
REGISTRATION QCWhen registration is complete, you will see a Registered_Regions directory created inside each slide folder with an ROI folder for each region box annotation. Each ROIxx folder contains all marker images for that region with reg_ prepended to the filename.A new folder called 'Registration_Check' will be created in the Parent directory that contains low resolution images of all registered regions for every slide in the batch. Each low res image will have 'regck_' prepended to the filename. This image set can be used to quickly check the images for gross misregistration or images that were were false registered (passed but should have failed). Regions or stains that did not adequately register well enough for quantification due to tissue deformation or loss are removed from further analysis.If registration failed for a marker or region, you will see a 'Redo_ROI' folder inside the slide folder containing the failed marker region and the hematoxylin region that failed registration. Images that failed registration can be excluded from further downstream analysis, or manually registered. See Troubleshooting guide ### for how to register images - 'Manual Registration' XXX.
?. [Signal Extraction]
AEC COLOR DECONVOLUTIONColor deconvolution separates the AEC signal from the tissue background to quantify signal.AEC signal is extracted using an RGB to CMYK conversion which utilizes a maximum gray component replacement to subtract the lowest gray level from all channels. AEC chromogen signal is separated into the ‘Y’ and 'M' channels and the 8-bit image channels are added together. Pixel results greater than 255 are set to 255. The middle 90% of pixels are rescaled to 0-255 (Pham et al. 2007). EXAMPLE:Java -jar /Applications/Fiji.app/jars/ij-1.52p.jar -ijpath /Applications/Fiji.app -batch /Users/sivagnan/MultiplexIHC_Code_072020/CommandLine/AEC_CMYK_ColorDecon.txt /Users/sivagnan/PROJECTS/Cohort1/The script runs on the Parent directory and creates a directory in each slide called 'Processed' with ROI folders inside; same folder structure as 'Registered_Regions'.  Each generated image will have 'V_' prepended to the filename.
?. [Image Registration]
IMAGE REGISTRATIONHow registration works:Image registration is performed in Matlab. Each marker image is registered to the last cycle target hematoxylin image. Keypoints are identified on the blue channel of the hematoxylin image and the blue channel of the AEC marker image.If not enough matching keypoints are found, other single RGB channels are checked for more keypoints, and the channels with the highest strongest matching indexed keypoints are used. A similarity based geometric transform is generated and applied to each channel of the marker image, then merged back into an RGB image. The registered RGB image is saved as a TIFF inside an automatically generated 'Registered_Regions' folder created in the slide/region folder.Registration Steps:1. Open Matlab and set working path by selecting the 'choose folder' icon and set the path to the location you saved the MultiplexIHC_Code directory. On the left panel called 'Current Folder' you will see the files listed in gray. 2. Right click on the MultplexIHC_Code directory in the 'Current Folder' panel and select 'Add to Path' > 'Selected Folders and Subfolders'. They will turn from gray to black indicating they are available.
RUNNING MATLAB REGISTRATION1. Double click mIHC_Registration.m file from the Current Folder panel and open in the editor.2. In the EDITOR tab, click 'Run' to start the script3. At the file prompt, select the Parent folder containing your slide folders.
?. [Image Registration QC]
REGISTRATION QCWhen registration is complete, you will see a Registered_Regions directory created inside each slide folder with an ROI folder for each region box annotation. Each ROIxx folder contains all marker images for that region with reg_ prepended to the filename.A new folder called 'Registration_Check' will be created in the Parent directory that contains low resolution images of all registered regions for every slide in the batch. Each low res image will have 'regck_' prepended to the filename. This image set can be used to quickly check the images for gross mis-registration or images that were were false registered (passed but should have failed). Regions or stains that did not adequately register well enough for quantification due to tissue deformation or loss are removed from further analysis.If registration failed for a marker or region, you will see a 'Redo_ROI' folder inside the slide folder containing the failed marker region and the hematoxylin region that failed registration. Images that failed registration can be excluded from further downstream analysis, or manually registered by selecting matching keypoints from both images. See Troubleshooting guide ### for how to register images - 'Manual Registration' XXX.
?. [Signal Extraction]
AEC COLOR DECONVOLUTIONColor deconvolution separates the AEC signal from the tissue background to quantify signal.AEC signal is extracted using an RGB to CMYK conversion which utilizes a maximum gray component replacement to subtract the lowest gray level from all channels. AEC chromogen signal is separated into the ‘Y’ and 'M' channels and the 8-bit image channels are added together. Pixel results greater than 255 are set to 255. The middle 90% of pixels are rescaled to 0-255 (Pham et al. 2007). The script runs on the Parent directory and creates a directory in each slide called 'Processed' with ROI folders inside; same folder structure as 'Registered_Regions'.  Each generated image will have 'V_' prepended to the filename.
Running Color Deconvolution in FIJI1. Open FIJI2. On the menu bar, select Plugins > Macros > Run...3. At the file prompt, select the AEC_ColorDeconCMYK.ijm file located in the MultiplexIHC_Code directory.4. At the next file prompt, select the Parent folder of your study.
?. [Nuclei Segmentation]
NUCLEI SEGMENTATIONSegmentation is performed on the last cycle hematoxylin to ensure that only cells maintained throughout all staining cycles are evaluated downstream. If using the first hematoxylin, tissue and cells lost during the staining process will result in false negatives.The FIJI macro is optimized to identify nuclei from hematoxylin only stained tissue. The macro uses the Color_Deconvolution [H AEC] vector in FIJI to first isolate hematoxylin OD, followed by additional filtering steps for background subtraction and signal cleaning. Objects are segmented by finding point maxima within a given *noise tolerance limited to pixels within Otsu thresholding.  Segmented objects go through a series of EDM watershed and erosion-dilation steps to generate the final binary mask.*Noise Tolerance: The expected difference in signal to noise. Maxima are ignored if they do not stand out from the surroundings by more than this value. A threshold is set at the maximum value minus noise tolerance and the contiguous area around the maximum above the threshold is analyzed.
Running nuclei segmentation in FIJI1. Open FIJI2. On the menu bar, select Plugins > Macros > Run...3. At the file prompt, select the Nuclei_BinaryMask.ijm file located in the MultiplexIHC_Code directory.4. At the next file prompt, select the Parent folder of your study.5. At the Noise parameter prompt, input your expected noise tolerance or try the default setting.
?. [Single Cell Measurements]
SINGLE CELL MEASUREMENTSSingle cell measurements are performed in Cell Profiler. Although there are many methods to measure mean intensity values for masked objects, this workflow utilizes Cell Profiler due to the output compatibility with FCS Express Image Cytometry for single cell classification. Measurements necessary for single cell gating are mean intensity for each marker, object area, object index, and object centroid X,Y location coordinates. There are specific nomenclature that is required in order for the Cell Profiler output file to work properly in FCS Express Image Cytometry.See the attached document on how to use the template Cell Profiler project, which can be found in the MultiplexIHC_Code directory.
?. [Signal Extraction]
CONVERT TO OME.TIFFThe 'V_' prepended images are your registered, signal extracted, single channel 8-bit final image. Each set of images within an ROI can be converted into a single ome.tif using bioformats.