TITLE: Quantitative whole-slide image analysis of CD3 and CD8 immunohistochemical stains
AUTHORS: Seungyeon Yoo

[DESCRIPTION]
Given the virtual slide images of CD3 and CD8 immunohistochemical stains, this protocol provides the following quantitative measures of tumor immune microenvironment:
Density of intraepithelial/stromal tumor-infiltrating lymphocytes (TIL)
Proportion of stroma within an annotated tumor area
Ratio of intraepithelial and stromal TIL 
Ratio of CD8 and CD3 TIL 

Once you annotate the tumor area manually, it segments the area into 1000 x 1000-pixel tiles. This enables you to assess not only the overall density of TIL and stroma, but also their mean, minimum, median, maximum and variance over the tiles. 

 
 


[STEPS]
SECTION: Create a new project
1.  
 

 
 

After creating the project, open 'code' folder and execute 'run.bat' by double-clicking. It will automatically create the script files according to your directory setting. 


 

This script do the following substitution and move all the above files to Project/Scripts (this /Scripts subfolder is generated by creating a new project): for non-windows users who might not be able to run .bat file, you can just do that by yourself. 


_REPLACE_DIR_ → <Your work folder>/Data/ (i'll call it 'data folder' from now on) 
_REPLACE_PROJECT_DIR_ → <Your work folder>/Project/ (i'll call it 'project folded' from now on)




SECTION: Populate the project with your virtual slides
2. File > Project > add images 
 
 
 
 

SECTION: Annotation of the tumor area
3.  
After selecting the polygon tool, clik one point and drag along the contour of the tumor area while not releasing the button. When the contour is finished, double-click the finishing point. 

I don't think there is a general rule of tumor area selection, but I tried to keep the following rules: 
I tried to have one uninterrupted field of tumor area per each image, but there were some cases with scanty tumor which inevitably forced me to select several separated pieces. 
 
 
I tried to exclude the adenomatous area, but there were some ambiguous cases. 
 
 
In the case of mucinous carcinoma, the area with acellular mucin pool was also included. 
The area of normal mucosa, necrotic debris and artifacts such as tissue folding, speckles were avoided. 
Normal lymphoid structures such as peritumoral lymph node and mucosa-associated lymphoid tissue near the mucosa were avoided. 
 
 

You don't need to be too precise at this point, as tumor area can be more precisely defined in the following step. 



SECTION: Segmentation
4. Analyze > region identification > create tiles 
 
 

Once the tiles are created, it is free to delete or move them to elaborate the tumor area. 
 
 

SECTION: Designate the invasive margin (IM)
5. While pushing the control key (Ctrl), click the tiles that constitute the IM. 
 
 
Again, I don't think there is no general definition of the IM, but I tried to keep the following rules: 
When the mucosa-to-serosa axis is evident, the IM is defined as the "frontmost" files. In other words, the tiles approaching the mucosa are not considered as IM.  
 
 
When the axis is not evident, any border with the non-neoplastic tissue was considered as IM. 
 
 
The area bordering with abscess, necrosis or lymph node is NOT invasive margin. 
The IM has to contain tumor cells. As drawing tumor area in step 3 produces somewhat crude outline, some tiles at the border of tumor and normal tissue might not contain tumor cells. In that case, such tiles need to be deleted. 


SECTION: Run 1.groovy
6. This script saves the screenshot of tiles with IM highlighted (just like the one you've just seen in the previous step) and saves tile objects for later use. 

 
 

Select Automate > Show script editor, then you'll see a new window.

 
 

Select File > Open, go to your project folder's script folder then you'll find script files. Select 1.groovy and open it. 

  
 
To run the script, select Run > Run or just push Ctrl+R. 



SECTION: Repeat step 3 through 6 for all the slide files
7. The annotation of tumor area is the only step human intervention is needed. All the other steps can be done automatically. I thought it was better for me to stay in front of the computer only when I was needed and let the computer do all the other jobs by itself while i was away, since the other jobs would be time-consuming. That was the reason I separated 1.groovy from others. 

So suppose you started with N slide files. If you have successfully finished the step 7, your data folder would have the two subfolders as follows, each of them containing N files. 
 

and your project folder's subfolder named [data] would also have N qpdata files. 
 
 

SECTION: Run 2.groovy
8. Check any image using project image list and make sure you see a image with tiles overlaid, like the one below. Since this is what qpdata file stored at /project/data does, any image should be seen with tiles overlaid if you have proper N qpdata files generated. 
 
 

Then go to the script editor and open 2.groovy. Since N files are ready, you'd want the computer to process all the files by itself. To do so, 

 
 

Select Run > Run project (without save), then a new window will appear. 

  
 

Push >> to select all the images in the project to be processed and click [OK]. 
It will take some time to finish... for example, processing 1158 files took more than one day. The program might seem to be halted, but you will see it's not by checking the data folder. There will be four subfolders (Cell_before_data, Cell_before_qpdata, SP_before_data, SP_before_qpdata) created and new output files being generated. 
 
When each of the four subfolders is populated with N files, step 8 is over. 


Take time to check contents of SP_before_data and SP_before_qpdata to see whether there are some output files with oddly small size. It happened because superpixel segmentation sometimes fails for unknown reason. Since there's no superpixel, the output file containing information of superpixels should be small. In my case, the problem disappeared by repeating step 3 through 6 and running 2.groovy only for those files. 

SECTION: Run 3.py
9. In fact, 2.groovy identified putative cells and segmented images into superpixels. Then the next script, 3.py, filters true lymphocytes and classifies superpixels into tumor or stroma. It should be run outside QuPath because I used machine learning classifier implemented by R and used python to process QuPath output files ready to be classified. QuPath does have machine learning classifier, which can be replaced with this step :)

Folded tissue fragments or ink particles can be falsely recognized as lymphocytes, so try to avoid the artifacts as possible. I tried to make classifiers to filter them out, but realized that so many false negatives happen by doing so. 

As mentioned in my article, lots of my CD8 images suffered varying degree of nonspecific membranous staining. As a result, I had to apply a lymphocyte classifier for CD8 images to filter out artifacts generated by the nonspecific staining. For CD3 images, no filtering was needed. So my script was designed to classify every cell as "positive" for CD3 stains and to retrieve a classifier for CD8 stains as follows: 
 
  
 
You might have different kinds of problems on your images. In that case, you might want to create your own classifiers and modify the above part of classifyCell.R to selectively apply the classifier. 


After do your modification, run 3.py. 

For the script to run with no problem, two things to be done.
1) the directory of R belongs to the the system environmental variable
2) package readr, caret and randomForest have to be installed in your R. 


For Windows uers who have installed Anaconda, you can run the script by executing Anaconda Powershell Prompt and type as follows: 

> python <path to the script file>/3.py 

  
 

for me, it took several hours (12-24h) to process 1158 files. 


SECTION: Run 4.groovy
10. Go back to QuPath and check any image using project image list. If what you see is an image with tiles and putative cells like the one below, that means everything so far has been fine.
 
 

Then open 4.groovy and Run > Run for project (without save). 
This script applies the result of cell filtering to QuPath.

SECTION: Run 5.groovy
11. Check any image using project image list and make sure you see a image like the one below. These are superpixels!
 
 
Then open 5.groovy and Run > Run for project (without save).  
This script applies the result of superpixel classification and retrive the filtered cells to obtain intraepithelial (true lymphocytes in tumor superpixels) and stromal (true lymphocytes in stroma superpixels) TIL. 

To see the result after the execution is finished,  select File > Open > go to the data folder > select 'ES_Tilewise_qpdata' subfolder  > select any qpdata file 
 
 


SECTION: Run 6.py
12. This is the final step. It produces a file named count.csv, which contains values of 207 measures for each case. 
 
 


For Windows uers who have installed Anaconda, you can run the script by executing Anaconda Powershell Prompt and type as follows: 
> python <path to the script file>/6.py