Current approaches for accurate identification, classification, and quantification of biotic and abiotic stresses in crop research and production are predominantly visual and require specialized training. However, such techniques are hindered by subjectivity resulting from inter-and intrarater cognitive variability. This translates to erroneous decisions and a significant waste of resources. Here, we demonstrate a machine learning framework's ability to identify and classify a diverse set of foliar stresses in soybean [Glycine max (L.) Merr.] with remarkable accuracy. We also present an explanation mechanism, using the top-K high-resolution feature maps that isolate the visual symptoms used to make predictions. This unsupervised identification of visual symptoms provides a quantitative measure of stress severity, allowing for identification (type of foliar stress), classification (low, medium, or high stress), and quantification (stress severity) in a single framework without detailed symptom annotation by experts. We reliably identified and classified several biotic (bacterial and fungal diseases) and abiotic (chemical injury and nutrient deficiency) stresses by learning from over 25,000 images. The learned model is robust to input image perturbations, demonstrating viability for high-throughput deployment. We also noticed that the learned model appears to be agnostic to species, seemingly demonstrating an ability of transfer learning. The availability of an explainable model that can consistently, rapidly, and accurately identify and quantify foliar stresses would have significant implications in scientific research, plant breeding, and crop production. The trained model could be deployed in mobile platforms (e.g., unmanned air vehicles and automated ground scouts) for rapid, large-scale scouting or as a mobile application for real-time detection of stress by farmers and researchers. plant stress phenotyping | machine learning | explainable deep learning | resolving rater variabilities | precision agriculture C onventional plant stress identification and classification have invariably relied on human experts identifying visual symptoms as a means of categorization (1). This process is admittedly subjective and error-prone. Computer vision and machine learning have the capability of resolving this issue and enabling accurate, scalable high-throughput phenotyping. Among machine learning approaches, deep learning has emerged as one of the most effective techniques in various fields of modern science, such as medical imaging applications, that have achieved dermatologist-level classification accuracies for skin cancer (2), in modeling neural responses and population in visual cortical areas of the brain (3), and in predicting sequence specificities of DNA-and RNA-binding proteins (4). Similarly, deep learningbased techniques have made transformative demonstrations of performing complex cognitive tasks such as achieving human level or better accuracy for playing Atari games (5) and even beating a human expert in the game Go (6). In this paper, we build a deep learning model that is exceptionally accurate in identifying a large class of soybean stresses from red, green, blue (RGB) images of soybean leaves (see Fig. 1 ). However, this type of model typically operates as a black-box predictor and requires a leap of faith to believe its predictions. In contrast, visual symptom-based manual identification provides an explanation mechanism [e.g., visible chlorosis and necrosis are symptomatic of iron deficiency chlorosis (IDC)] for stress identification. The lack of "explainability" is endemic to most black-box models and presents a major bottleneck to their widespread acceptance (7). Here, we sought to "look under the hood" of the trained model to explain each identification and classification decision made. We do so by extracting the visual cues or features responsible for a particular decision. These features are the top-K high-resolution feature maps learned by the model based on their localized activation levels. These features-which are learned in an unsupervised manner-are then compared and correlated with human-identified symptoms of each stress, thus providing an inside look at how the model makes its predictions. Leaf Sample Collection and Data Generation. We started with the collection of images of stressed and healthy soybean leaflets in the field. The labeled data were collected following a rigorous imaging protocol using a standard camera (see SI Appendix, Sections 1 and 2 for details). Over 25,000 Plant stress identification based on visual symptoms has predominately remained a manual exercise performed by trained pathologists, primarily due to the occurrence of confounding symptoms. However, the manual rating process is tedious, is time-consuming, and suffers from inter-and intrarater variabilities. Our work resolves such issues via the concept of explainable deep machine learning to automate the process of plant stress identification, classification, and quantification. We construct a very accurate model that can not only deliver trained pathologist-level performance but can also explain which visual symptoms are used to make predictions. We demonstrate that our method is applicable to a large variety of biotic and abiotic stresses and is transferable to other imaging conditions and plants.
Recently, REF proposed an explainable deep machine vision framework for plant stress phenotyping by selecting the top-K high-resolution feature maps.
An explainable deep machine vision framework for plant stress phenotyping
Proceedings of the National Academy of Sciences of the United States of America
Proceedings of the National Academy of Sciences of the United States of America
Computer Science Medicine