Package: pytcherplants
pytcherplantsGeometric trait and color analysis for top-down images of pitcher plants, built on top ofilastik,OpenCV, andDeep Plant Phenomics.Pot SegmentationColor AnalysisGrowth Point EstimationGeometric TraitsTODOProject layoutApproachSetting up a development environmentRequirementsInstalling dependenciesvenvAnacondaDockerRunning the codeJupyter notebooksPython CLIProcessing (image analysis)Post-processing (aggregations)Detecting growth point labelsDeveloped for images obtained from an experiment performed byMason McNairat the University of Georgia. Segmentation strategy (viaIlastikpixel classification) adapted fromPeter Pietrzyk'sDIRTmuproject. Color and trait analysis adapted fromSMART(Speedy Measurement of Arabidopsis Rosette Traits) bySuxing Liu.Project layoutJupyter notebooks detailing methods are innotebooks. A few test photos are included indata. A Python CLI exposing commands for image analysis and postprocessing/aggregations is provided inscripts.ApproachThis repository currently provides functions for color distribution analysis. A few more features are under development, but not yet complete:growth point counting via density estimationpitcher counting, semantic segmentation, & traitsColor analysisFirst each image is preprocessed and pots are segmented. The number of pots per image can be provided on the command line, otherwise it is automatically inferred. A series of preprocessing steps are first applied including Gaussian blur and an adaptive threshold, followed by contour detection and a hue filter. Once individual pots and the plants within have been distinguished, each is cropped for individual analysis. K-means clustering is used to assign each pixel to a centroid in RGB-space (corresponding to the nearest color cluster). This yields a reduced image with k distinct colors. Pixels are counted and frequencies recorded, grouped by plant, timestamp, and fertilizer treatment. Next, another round of k-means clustering is applied separately to the (RGB) pixel distribution corresponding to each treatment. Pixels are also binned according to hue.Growth pointsThis feature is still in developement.Several methods are compared for estimating growth point locations, including:convex hull centroidskeletonization, root node estimationdensity estimation via convolutional neural netGrowth points are useful in turn for estimating average pitcher length per plant/pot, even in the absence of semantic pitcher segmentation: the average distance from growth point to convex hull boundary doesn't seem an unreasonable proxy.Centroid of convex hullA convex hull is computed around the plant after background and pot pixels have been removed, then its centroid is interpreted as the likeliest growth location on the assumption that pitcher length tends toward a uniform distribution. With this method we also assume each pot has just 1 growth point.Root node of skeletonized graphA binary skeletonization is obtained for each plant after background and pot pixels are removed, then an estimate of its graph structure is extracted and we attempt to determine the root node's location.CNN density estimation: heatmap object countingA convolutional neural net, modeled on the Deep Plant Phenomics project's example, is used here for density estimation. This produces a 2-dimensional Gaussian distribution over the cropped image of each pot: a heatmap showing locations the model considers the likeliest growth points.Pitcher segmentation & traitsThis feature is still in development.TODOSetting up a development environmentClone the repo withgit clone https://github.com/w-bonelli/pitcherplants.git.RequirementsPython3.6+ with the packages inrequirements.txt. A few options for setting up an environment:venvAnacondaDocker/SingularityInstalling dependenciesAnaconda or Python3's built-in venv utility can be used to create a virtual environment. Alternatively there is an image available on Docker Hub:wbonelli/pytcher-plants.venvFrom the project root, runpython3 -m venvto create a virtual environment, then activate it withsource bin/activate. Dependencies can then be installed with pip, e.g.pip install -r requirements.txt. The environment can be deactivated withsource deactivate.AnacondaCreate an environment:condacreate--name<yourenvironmentname>--filerequirements.txtpython=3.8anacondaAny Python3.6+ should support the dependencies inrequirements.txt. The environment can be activated withsource activate <your environment name>and deactivated withsource deactivate.DockerThere is a preconfigured Docker image available on the Docker Hub atwbonelli/pytcher-plants. From the project root, run:dockerrun-it-p8888:8888-v$(pwd):/opt/dev-w/opt/devwbonelli/pytcher-plantsbashThis will pull the image definition (if you don't already have it) and start a container, mounting the project root into the container's working directory. It also opens port 8888 in case you want to use Jupyter.Running the codeJupyter notebooksA Jupyter server can be started withjupyter notebookfrom the project root. (If you're using Docker, add flags--no-browser --allow-root.)This will serve the project atlocalhost:8888. Then navigate to thenotebooksdirectory, open a notebook, and refer tothe Jupyter docsif unfamiliar.Python CLIThe Python CLI can be invoked withpytcher_plants/cli.py. This script includes commands for processing one more image files as well as post-processing/aggregations after images are analyzed.Image namesThe various CLI commands expect image file names to conform to the schemedate.treatment.name.ext, where dates are_-delimited integer triplesmonth_day_year. For instance:10_14_19.Calmag.5V4B9763.jpg1_14_19.bloomboost.5V4B3121.JPGPreprocessing (pot segmentation, initial color clustering, pixel counting)pythonpytcher_plants/cli.pypreprocess-i<inputfileordirectory>-o<outputdirectory>By default JPG and PNG files are supported. You can select one or the other by passingpngorjpgto the--filetypesflag (shorthand-ft).You can also specify the number of plants per image by providing an integer--count. If this argument is not provided, the software will keep the top $n$ largest contours, of those with area greater than a threshold value--min_area(if this value is not provided, an area equivalent to a (w/5)x(h/5) square is used).Color analysisAfter processing all image files individually, thepostprocesscommand can be used on the files produced by theprocesscommand to compute color distributions and geometric traits.pythonpytcher_plants/cli.pycolorsanalyze-i<inputfileordirectory>-o<outputdirectory>Growth pointsA command is provided to detect manually labeled growth points, find their coordinates, and write them to CSV for ingestion into aheatmap-based counting dataset for use with a model trained with Deep Plant Phenomics. For instance, the following will produce thelabels.csvfile included indata/labels/growth_points, run on the images included in that folder:pythonpytcher_plants/base.pygpointsload_labels-i<inputfileordirectory>-o<outputdirectory>-c'#ea57f5'TODO: train command