TITLE: RasberryPi-computer based phenotyping for side view image process
AUTHORS: Li'ang   Yu, Magdalena M Julkowska

[DESCRIPTION]
Imaging of plants can be done in an inexpensive way, using Raspberry Pi based setup. Here - we describe how we are processing the side-view images taken from 7 different sides - using the PlantCV pipeline. 

[STEPS]
SECTION: Data preparation
1. Download the data from RaspberryPi computer, where the images are being saved into one folder. If you have multiple stands for side-view image collection - download all the images into one folder entitled as your experiment. 

NOTE - please make sure that the RapberryPi's were recording the timestamp correctly (i.e. were continuously connected to the internet router - that ensures the right time and date being timestamped on each image).

If your RaspberryPi computer was not connected to the internet throughout the experiment - it is better to save each day of imaging in a separate folder. It will be slightly more work to analyze them but will ensure that your timestamp is correct.
SECTION: Script development
2. On your desktop/laptop - make sure to have Conda, Jupyter Notebook and PlantCV installed, including all the dependencies. Please see the links for the installation instruction, and follow the instructions by installations through Conda.  

IF you are not sure about whether conda, jupyter or plantcv are already installed at your computer - you can check it through typing the following command in your terminal:

 

To check PlantCV version - search through the list of packages that were delivered to you when prompting conda list. PlantCV should be one of the packages listed alphabetically therein - similar to the screenshot below:

  

SECTION: Script development
3. Make sure to download this example Jupyter Notebook - which has been optimized for cowpea/tepary/tomato imaging using the PhenoCage at BTI, and save it in your current working directory.

  

Please also see pdf version of the same notebook of how individual steps SHOULD look like

  

Just as a reminder - to know what is your current working directory use command 
 
whereas to change the directory use the command

 
to go one directory up (out of your current folder) use:

 

SECTION: Script development
4. Open the notebook by typing two commands into your terminal window
 

SECTION: Script development
5. Navigate to the folder that contains example jupyter notebook - and change theimage directory in the 3rd chunk of the notebook. This is how it looks like:

  

SECTION: Script development
6. Make sure that you load one example image, and that it shows up in 4th chunk of Jupyter notebook. It should look like on the image below, but with your plant. 

  

SECTION: Script development
7. Once the image is loaded - you can try and run the entire notebook for your image - to see if the default parameters are working for your imaging setup. You can run the entire notebook by choosing "run all" from the Cell menu in Jupyter Notebook - as on the screenshot below:

  

SECTION: Script development
8. After you have run throughout the jupyter notebook - please make sure that:
1) the MASK is covering all of the plant parts (chunk #11) - i.e. all of your plant bits are covered with the PINK mask
        IF that is not the case - please adjust the threshold values in v_thresh and a_thresh channels 
        (chunks #6 and #8 respectively)

  

2) if the ROI (Region of Interest) is at least partially covering your plant (chunk #12)

  

3) if there are any other elements of the picture included in the final measurement - in the last chunk. 
        It is possible that some stems are going to be too thin to be recognized by the PlantCV. For our purpose - estimating the biomass from 7 different side-angles - as the major plant parts are included in the measurements - we should be good to go. 

  

SECTION: Script development
9. Re-run the pipeline on afew example images from your dataset - including young and old plants - and plants that differ in their coloring (e.g. lime green vs. dark green) - by selecting different images in chunk #3.

Please note that you can also use a file name as "my_picture" to select the files that you want to use for testing:

  

SECTION: Script development
10. Once you have the correct thresholds - please adjust them in the python code - for which the example is attached below. This is the code that you will be used for batch processing of all images. 

NOTE: please adjust the python code only using the code editing software - such as Atom - to make sure that the editing software won't replace your symbols, yielding the code non-functional. 

  

SECTION: Batch processing of the side images
11. For processing the images - we need to prepare a json file that will contain the information on:
- the path to the repository containing ALL of your images - indicated as "input_dir" in the 2nd line of the json file
- the path to the json output file - indicated as "json" in the 3rd line of the json file
- the way that your images are named - the order of timestamp - plant barcode - id - camera - indicated as "filename_metadata" in 4th line of the json file
- the path to the python code to be used - indicated as "workflow" in the 5th line of the json file
- the path to the repository where all the PROCESSED results will be deposited - indicated as "img_outdir" in the 6th
- delimiter between the individual elements of the file name - we usually use underscore - indicated as "delimiter" in the 11th line of the json file
- the timestamp format - indicating the general organization of your time-metadata in the file name - indicated as "timestampformat" in line 13 of the json file

NOTE: Please adjust json file ONLY in the code-processing software - such as Atom. Otherwise, you are running a risk that the individual symbols are going to be changed by your text editing software - and damage the json file.  

NOTE: the output directories should exist PRIOR to going on with the next steps. Please make sure to generate the correct folders. 

  
The example json file is attached below

  
SECTION: Batch processing of the side images
12. PROBLEMS with file naming? - Check out our SOLUTION here. NOTE - this step is OPTIONAL - and only applicable if you have oddly-named files:

Recently - we were recording the images with the error in their names - i.e. the timestamp was separated by a "." but the Raspberry Pi was also separated by a ".", instead of an "_". If something similar happens to you - that you have to rename your files prior to batch processing - follow this step:

1) determine the UNIQUE part of your name that you would like to change - such as ".Raspi" - rather than just "." or "_" - 

In our case - we had to rename ".RaspiY" to "_RaspiY" for correct delimiter between "timestamp" and "plantbarcode" - like on the screenshot below:

   
2) in the terminal - navigate to your directory containing the wrongly named images - and run the following command line - which will take ALL images containing in their name "RaspiY", and replace a specific part of the file name indicated in the "sed" part of the command. This code will generate a file called "rename.txt" that will be saved in the same folder as your original images.

 

3) open the rename file which should contain ALL the "move" commands (mv) for all the files within your directory with your wrongly named images. Should look something like the screenshot below:

  
4) copy ALL the contents of the rename.txt file - and paste them into your terminal window. This will run ALL the rename commands - that will replace the plant name in your repository with the wrongly made images. 

5) check the repository if the names are indeed changed to the correct names.

   

SECTION: Batch processing of the side images
13. Now - you have prepared ALL the files - make sure to also have plantcv-workflow.py and plantcv-utils.py codes in your directory - it does NOT have to be the same directory as your experiment/images. Just have them somewhere with the identifiable path to them
    
SECTION: Batch processing of the side images
14. Run the following command - adjusting the path to your "plantcv-workflow.py" file and path to your "json" file prepared in step 11 of THIS protocol. 

 

NOTE: if you are running it on your Desktop - this step might take some time (it took me +/- 20 minutes to process 980 images of cowpea) 
SECTION: Batch processing of the side images
15. Once the script is finished working - you should see the output files in the output directory specified in the json file generated in Step 11 of THIS protocol. 

Please take time to go through the output files and make sure to go through the processed images - possibly identifying any problems that the python code had with identifying the plant parts in your images.

  

SECTION: Batch processing of the side images
16. The script will also create a json file with the results in the output folder. It will look something like the screenshot below:

  


In our lab, we typically prefer to work with the .csv files rather than json files. To transform the json file into csv files - please run the below code adjusting the path to your "plantcv-utils.py" file and path to your  "json" file containing the results - produced by Step 15 of THIS protocol. 

 

The script will result in two files being produced:
- Results-multi-value-traits.csv
- Results-single-value-traits.csv 

NOTE: if you are processing individual days separately - you might want to add a DATE identifier to these files before moving on with the next day processing - otherwise they will be renamed when you will run the next itteration of the code. If that happens - you can always re-run this step for the specific json file - as that will NOT be renamed - since the names are going to be unique (if you change them in the json file for each day in STEP 11!)  

SECTION: Analysis of the data
17. Processing the data further to calculate the cumulative plant area from all 7 angles as well as growth rates can be done in R. Please follow the example pipeline developed by Li'ang and Hayley in R, which can be accessed here.

NOTE: it is a good practice to run a correlation analysis between your cumulative area and the fresh weight (if recorded) on the final day of the experiment.