Patent ID: 12190501

DETAILED DESCRIPTION

FIG.1illustrates an environment in which one or more selected aspects of the present disclosure may be implemented, in accordance with various implementations. The example environment includes a plurality of client devices1061-N, an agriculture knowledge system102, and one or more sources of vision data1081-M. Each of components1061-N,102, and108may communicate, for example, through a network110. Agriculture knowledge system102is an example of an information retrieval system in which the systems, components, and techniques described herein may be implemented and/or with which systems, components, and techniques described herein may interface.

An individual (which in the current context may also be referred to as a “user”) may operate a client device106to interact with other components depicted inFIG.1. Each component depicted inFIG.1may be coupled with other components through one or more networks110, such as a local area network (LAN) or wide area network (WAN) such as the Internet. Each client device106may be, for example, a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device of a vehicle of the participant (e.g., an in-vehicle communications system, an in-vehicle entertainment system, an in-vehicle navigation system), a standalone interactive speaker (with or without a display), or a wearable apparatus that includes a computing device, such as a head-mounted display (“HMD”) that provides an augmented reality (“AR”) or virtual reality (“VR”) immersive computing experience, a “smart” watch, and so forth. Additional and/or alternative client devices may be provided.

Each of client devices106and agriculture knowledge system102may include one or more memories for storage of data and software applications, one or more processors for accessing data and executing applications, and other components that facilitate communication over a network. The operations performed by client device106and/or agriculture knowledge system102may be distributed across multiple computer systems. For example, agriculture knowledge system102may be implemented as, for example, computer programs running on one or more computers in one or more locations that are coupled to each other through a network.

Each client device106may operate a variety of different applications that may be used, for instance, to view information about individual plants and/or groups of plants that is generated using techniques described herein. For example, a first client device1061operates an image viewing client107(e.g., which may be standalone or part of another application, such as part of a web browser). Another client device106Nmay take the form of a HMD that is configured to render two-dimensional (“2D”) and/or three-dimensional (“3D”) data to a wearer as part of a VR immersive computing experience. For example, the wearer of client device106Nmay be presented with 3D point clouds representing various aspects of objects of interests, such as fruits of crops.

In various implementations, agriculture knowledge system102may include a digital image engine112, a plant recognition engine116, and/or a user interface engine120. In some implementations one or more of engines112,116, and/or120may be omitted. In some implementations all or aspects of one or more of engines112,116, and/or120may be combined. In some implementations, one or more of engines112,116, and/or120may be implemented in a component that is separate from agriculture knowledge system102. In some implementations, one or more of engines112,116, and/or120, or any operative portion thereof, may be implemented in a component that is executed by client device106.

Digital image engine112may be configured to receive, directly or indirectly from data sources1081-M, a plurality of two-dimensional 2D images captured by one or more 2D vision sensors. In various implementations, the 2D images each may capture an individual plant among a populations of plants (e.g., fields of plants). In some implementations, each digital image may capture an individual plant surrounded by some environmental context, e.g., that includes other visual features such as sprinkler heads, rocks, or other artificial or natural features that can be used as visual landmarks. In other implementations, the vision data received from robots1081-Nmay include 3D data generated using 3D vision sensors such as light detection and ranging (“LIDAR”) sensors, stereographic cameras, etc.

2D/3D vision data may be obtained from various sources. In the agricultural context these data may be obtained manually by individuals equipped with cameras, or automatically using one or more robots1081-Mequipped with 2D/3D vision sensors (M is a positive integer). Robots108may take various forms, such as an unmanned aerial vehicles1081, a wheeled robot108M, a robot (not depicted) that is propelled along a wire, track, rail or other similar component that passes over and/or between crops, or any other form of robot capable of being propelled or propelling itself past crops of interest. In some implementations, robots1081-Mmay travel along lines of crops taking pictures at some selected frequency (e.g., every second or two, every couple of feet, etc.), or whenever a whole plant is detected within a field-of-view of the vision sensor.

Robots1081-Mmay provide the vision data they capture to agriculture knowledge system102over network(s)110. Digital image engine112may be operably coupled with a database114that it uses to store vision data (e.g., digital images) captured by any number of sources (e.g., robots108). In some implementations, a user may interact operate a client device106to interact with user interface engine120. During this interaction the user may request that particular sets of vision data be processed by agriculture knowledge system102using techniques described herein to allow the user to view information about individually-recognized plants.

Plant recognition engine116may be configured to perform various aspects of the present disclosure on vision data captured from various sources, such as people, robots108, etc., to be able to recognize, in vision data such as 2D digital images, individual plants in distinction from other individual plants (including other individual plants of the same genus or species). For example, in some implementations, plant recognition engine116may be configured to obtain a digital image that captures at least a first plant of a plurality of plants. This digital image may have been previously captured and provided to digital image engine112, or plant recognition engine116may obtain the digital image directly from the source, such as one of robots1081-M.

Plant recognition engine116may also determine additional data indicative of an additional attribute of the first plant. As noted above, this additional data may take various forms. In some implementations, the additional data may be determined from a sensor signal generated by a sensor of a device used to capture the digital image. For example, many robots108are equipped with position coordinate sensors, such as inertial measurement units (“IMU”) sensors, global positioning system (“GPS”) sensors, sensors that obtain position from triangulation of wireless signals, etc. Position coordinates may be generated using sensors such as these while a vision sensor captures a digital image of a plant. The position coordinate may then be associated with the digital image of the plant, e.g., as metadata.

This additional data may take other forms in addition to or instead of position coordinates. For example, in various implementations, a bounding shape may be calculated for the first plaint. Various techniques such as edge detection, machine learning (e.g., a convolutional neural network, or “CNN”), segmentation, etc., may be employed to detect a bounding shape that encloses at least a portion of the first plant, e.g., a minimum bounding shape that encloses an entirety of the first plant. In some such implementations, various aspects of such a bounding shape, such as its height, width, diameter, shape, etc., may be used as an additional attribute of the first plant.

Additionally or alternatively, in some implementations, plant recognition engine116may calculate a bounding shape that captures not only the plant, but some portion of an environment or area that surrounds the first plant. In this way plant recognition engine116is able to determine an environmental context of the first plant. For example, the first plant's spatial relationship to one or more neighboring plants may be useful in recognizing first plant in distinction from other individual plants. As another example, artificial or natural landmarks near the first plant, such as rocks, weeds, flowers, moss, sprinkler heads, other irrigation equipment such as hoses/pipes or valves, indicia or fiducial markers placed near the first plant (e.g., as a standalone sign, a tag on the plant, or on irrigation equipment), natural formations, etc., may for part of the first plant's environmental context, and may also be used to recognize the first plant in distinction from other individual plants.

In some implementations, temporal data associated with a digital image of the first plant may also be used, e.g., by plant recognition engine116, in addition to or instead of other attributes of the first plant and the digital image, to recognize the first plant in the digital image. For example, a timestamp associated with the digital image may be used to calculate a time interval since some reference milestone in the first plants' life, such as its planting, its being transferred to a new location, its most recent digital image, etc. In some implementations, milestones of multiple plants, e.g., crops in a field, may be aligned, e.g., because all the plants were planted or last photographed at or around the same time. Thus, a time interval since such a milestone may be useful for distinguishing one individual plant from another.

This temporal data may be particularly useful when used by plant recognition engine116in combination with other plant attributes. As a non-limiting example, suppose a particular plant has a bounding shape of a particular width five weeks after being planted. Suppose the same plant is photographed two weeks later. A new bounding shape determined for the new digital image will likely be larger than the previous bounding shape, unless the plant is diseased or malnourished. An association between the time interval since the last photograph (two weeks) and an expected growth rate of the plant (individual or as part of a population) may have been effectively baked into the machine learning model during training. Consequently, the larger bounding shape, and in some implementations, a delta between the smaller and larger bounding shapes, may be used in conjunction with the new digital image to identify the plant in distinction from other plants.

Once plant recognition engine116has obtained the digital image and the additional data indicative of one or more attributes of the first plant, plant recognition engine116may process these data to recognize the first plant, e.g., in association with previous images of the first plant. In some implementations, plant recognition engine116may apply these data as input across a machine learning model to generate output. In some implementations, the output may include a latent space embedding. In some such implementations, the embedding may be close to, and therefore similar to, other embeddings in the latent space that were generated from other digital images of the same plant. Additionally or alternatively, in some implementations, the output may take the form of a unique identifier that is also associated with previously captured images of the plant.

Various types of machine learning models may be employed by plant recognition engine116to recognize individual plants. These may include, for instance, CNNs, other types of neural networks, sequence-to-sequence networks such as encoder-decoder networks, etc. In some implementations, a database118may be provided to store a plurality of machine learning models that may be applied by plant recognition engine116under different circumstances. In some implementations, a different machine learning model may be trained for each genus and/or species of plant. For example, one CNN may be trained to distinguish between individual strawberry plants, another may be trained to distinguish between individual tomato plants, another may be trained to distinguish between individual soy plants, etc. In other implementations, a single machine learning model may be trained to distinguish between individual plants across multiple species or genera.

Based on the output generated using the machine learning model, plant recognition engine116may store in database114, or may cause digital image engine112to store in database114, an association between the digital image that captures the first plant and one or more previously-captured digital images of the first plant. In some implementations, the first plant may have been previously assigned a unique identifier, such as a string of numbers, characters, symbols, or any combination thereof. The latest digital image capturing the first plant may then be assigned to or otherwise associated with this unique identifier, e.g., in database114. In other implementations, the latest digital image and the previous digital image(s) of the same plant may be associated with each other in other ways, such as using hash functions, links, pointers, etc.

In this specification, the term “database” and “index” will be used broadly to refer to any collection of data. The data of the database and/or the index does not need to be structured in any particular way and it can be stored on storage devices in one or more geographic locations. Thus, for example, the databases114and118may include multiple collections of data, each of which may be organized and accessed differently.

Once multiple digital images are associated in database114with a particular plant, user interface engine120may be interacted with by one or more client devices106to perform a variety of different agricultural applications. As one example, a client device106may provide a graphical user interface (“GUI”) that is operable by a user to select individual plants, or groups of plants (e.g., a row of plants in a field, a section of plants, etc.). Once plant(s) are selected, the GUI may provide the user with various tools to learn more about the selected plant(s), such as their growth histories, disease statuses, pest infestation statuses, projected yield, experienced weather patterns, and so forth. Some tools may allow for the extraction of phenotype and/or genotype information of plant(s) from images of the plant(s). These phenotypes and/or genotypes may indicate, for instance, whether the plant is growing well (compared to other similar plants), is diseased or is susceptible to disease or is pest resistant, etc.

FIG.2depicts a row of six plants2301-6at fifteen days after planting on the left and twenty-five days after planting on the right. It can be seen that each plant230grew by some amount during this ten-day time interval, with the amount of growth varying based on the initial size of the plant230. Also visible inFIG.2is irrigation equipment in the form of a pipe232that includes two sprinkler heads,2341and2342, that are positioned, respectively, in between first and second plants2301and2302and in between fourth and fifth plants2304and2305.

In various implementations, a vision sensor such as a 2D camera may be traversed along and/or above the row of plants2301-6, e.g., by a robot108(not depicted inFIG.2) so that digital images can be captured of the individual plants230. In some cases the robot108may have wheels that touch the ground on one or both sides of plants2301-6, such that the wheels flank the row of plants2301-6. In other examples, the robot108may be an aerial drone that flies over plants2301-6. Whichever type of robot is used to capture the digital images, the distances between individual plants inFIG.2is likely less than the amount of inherent error of a position coordinate sensor of the robot108. Consequently, and as mentioned previously, the position coordinate associated with a digital image (e.g., added to the image's metadata based on the robot's position) may not be sufficiently accurate to conclusively recognize an individual plant230in distinction from other individual plants.

Accordingly, in various implementations, the content of the digital images themselves, which capture numerous features of individual plants, may be used, e.g., in combination with one or more of the previously-described plant attributes, to recognize individual plants230. Although individual plants230likely would appear similar to a human, subtle visual features of those individual plants that are captured in the digital images may be used by computers to effectively establish a “fingerprint” for each plant230. Based on this fingerprint, which may be associated with a unique identifier for the plant230, it is possible to recognize individual plants230across time in distinction from other individual plants230.

For example, the particular configuration of leaves of first plant2301, including their size, orientation, number, color, leaf texture, angle, arrangement relative to each other, etc., may not appear substantially different to a person from configurations of leaves of other plants2302-6. However, using techniques described herein, a machine learning model may be trained to distinguish between individual plants depicted in images based on these subtle differences that are not readily noticeable to a human. And in some implementations, other attributes of the plant230aside from its digital image may be used to distinguish it from other plants.

FIG.3depicts the same row of plants2301-6asFIG.2at the same times, namely, fifteen and twenty-five days. However, various annotations are added to various individual plants230ofFIG.3in order to demonstrate how additional plant attributes, beyond digital images of the plants, may be used to aid in recognizing individual plants in distinction from other individual plants.

For example, inFIG.3, a first bounding shape3361is detected around third plant2303at fifteen days. While bounding shape3361and other bounding shapes depicted inFIG.3are rectangular, this is not meant to be limiting. Other bounding shapes are contemplated herein, such as various polygons, triangles, circles, ovals, etc. In some implementations, the particular bounding shape used may be determined based on the type of plant. For example, some plants may be readily captured within a hexagon shape because that shape most closely tracks their outer contours. Other plants may be better captured using a pentagon, a circle, an elongate rectangle, etc.

First bounding shape3361is a minimum bounding shape that encloses an entirety of third plant2303.—that is, first bounding shape3361has the smallest size possible that captures the outer extremities of third plant2303. In other implementations, a bounding shape may be detected that captures, for instance, some predetermined percentage of the plant, or a portion of the plant that is identified with at least a threshold amount of confidence. For example, the tips of leaves of a first plant may overlap with a neighboring plant. Accordingly, those overlapping portions of the first plant may not be identified as being part of the first plant with as much confidence as the middle of the first plant, and therefore may not necessarily be captured by a bounding shape. Similar to first bounding shape3361, a second bounding shape3362is detected around third plant2303at twenty-five days. As is depicted inFIG.3, second bounding shape3362is larger than first bounding shape3361. This is unsurprising given that third plant2303grew during this time interval.

In various implementations, various aspects of bounding shapes3361-2may be used as additional plant attributes to recognize third plant2303as distinct from other plants2301-2and2304-5. For example, a width and/or height of bounding shapes3361-2may be used as proxies for dimensions of third plant2303. These additional attributes may be applied, e.g., by plant recognition engine116, as additional inputs to a machine learning model, along with the digital images in which the bounding shapes3361-2were detected. The output of the machine learning model may distinguish the plant from other plants based at least in part on these additional attributes.

For example, a particular type of plant may be expected, absent extenuating circumstances and under conditions that are well-understood, to grow by a particular amount or percentage in a particular time interval. Accordingly, when matching later digital images of those plants to earlier digital images of those plants, this expected amount of growth may be detected in the dimensions of the later-detected bounding shape relative to the dimensions of bounding shapes of the earlier images of the plants.

In some implementations, this growth over a particular time period may be captured in the machine learning model during training. For example, for a training example, two images of the same plant may captured at two points in time separated by a predetermined time interval, such as ten days, two weeks, etc. The training example may signal (e.g., be labeled with data) that the two images depict the same plant. In some such implementations, aspects of bounding shapes around the plants in the two digital images may be determined and applied as additional inputs to the machine learning model. To the extent the machine learning model fails to identify the plants in the two training images as the same plant, the machine learning model may be trained, e.g., using techniques such as gradient descent and/or back propagation, to more accurately classify the images as being of the same plant in the future. With enough similar training examples, the machine learning model may be trained to accurately match two temporally-distinct digital images of the same plant to each other, especially when bounding shape dimensions are also used as inputs.

Referring back toFIG.3, as another example, a third bounding shape3363is depicted surrounding first plant2301at fifteen days. Unlike bounding shapes3361-2, bounding shape3363does not tightly fit around first plant2301. Rather, bounding shape3363captures at least some area around first plant2301, e.g., to obtain an environmental context of first plant2301. This is especially useful for first plant2301because first plant2301is adjacent first sprinkler head2341. Consequently, the environmental context captured by bounding shape3363is relatively rich with information beyond first plant2301itself. Ten days later, a similar, if larger, bounding shape3364is detected around first plant2301. Similar to bounding shape3363, bounding shape3364captures, in addition to the various subtle visual features of first plant2301itself, the rich environmental context of first plant2301, including first sprinkler head2341.

Another bounding shape3365is detected around fourth plant2304. Similar to first plant2301, fourth plant2304is also near a sprinkler head, in this case second sprinkler head2342. Consequently, bounding shape3365around fourth plant2304captures a similar environmental context as bounding shapes3363-4around first plant2301. Likewise, another bounding shape3366is detected around fourth plant2304at day twenty five, and also captures second sprinkler head2342. However, even though the environmental contexts of first plant2301and fourth plant2304are similar, these plants can still be distinguished from each other (and recognized individually) using other attributes of those plants, such as the unique visual fingerprint of each plant. And, first plant2301and fourth plant2304are relatively spaced from each other, likely more than the error of a robot's position coordinate sensor. Consequently, it is likely that position coordinates associated with images of these plants, when used as additional inputs for the machine learning model, will be very influential in distinguishing these plants from each other.

FIG.4schematically depicts an example of how a machine learning model such as a convolutional neural network (“CNN”)440can be trained to recognize individual plants in distinction from each other. In the example ofFIG.4, a loss function used to train CNN440is triplet loss. Each training instance438takes the form a triplet or 3-tuple that includes (i) an anchor image4421of a plant4301under consideration, (ii) a positive image4422that also captures plant4301under consideration (e.g., at a different time and/or from a different angle), and (iii) a negative image4423that depicts a different plant4302. In some implementations, CNN440generates a latent space embedding based on input that includes one of the three digital images4421-3and data indicative of additional attribute(s) of the plant depicted in the one of the three digital images4421-3.

To train CNN440, in some implementations, the three images4421-3of training triplet438are applied as input across CNN440to generate three respective latent space embeddings4461-3. As depicted inFIG.3, the latent space embeddings4461-2generated from the anchor image4421and positive image4422—both depicting the same plant4301—are fairly similar, and therefore should be close together in latent space. By contrast, the latent space embedding4463generated from the negative image4423, which depicts a different plant4302, is different from the other two embeddings4461-2, and therefore should be distanced from them in latent space. Accordingly, CNN440may be trained by minimizing a distance in the latent space between embedding4461generated from anchor image4421and embedding4462generated from positive image4422. Meanwhile, a distance between embedding4461generated from anchor image4421and embedding4463generated from negative image4423may be maximized. In other implementations, the machine learning model may be trained using other techniques, such as using surrogate losses followed by separate metric learning steps.

FIG.5depicts an example GUI550that may be provided by user interface engine120to a client device106. In some instances, GUI550may be provided as a website that is accessible via a web browser of a client device106. In other instances, GUI550may be provided as part of a mobile app that operates on a mobile client device106(e.g., smart phone, tablet, etc.) using data received/obtained from agriculture knowledge system102. In yet other instances where client device106is an HMD106N, HMD106Nmay operate a VR or AR application that receives/obtains data from agriculture knowledge system102and provides a user with an immersive experience.

GUI550may be operable by a user to conduct various types of analysis on individual plants that are recognized using techniques described herein. In some implementations, a portion552of GUI550may depict an overhead view of a plurality of plants, e.g., in a field, along with some farm equipment (“F.E.”) such as rain collection barrels, silos, etc. This view may be a single digital image captured from a vantage point above all the plants, a stitched-together image (e.g., a mosaic) generated from a plurality of digital images that each captures an individual plant, or even a simulated image of the field generated from, for instance, 3D point cloud data captured of the plants. In some implementations, GUI550may first provide the user with an opportunity to select a region, field, etc., and this selection may cause portion552to present an overhead view of the plants.

In various implementations, a user may be able to operate an input device such as a mouse or touchscreen to, for example, select individual plants depicted in portion552, select groups of plants, zoom in/out from one or more plants, etc. By selecting plant(s), the user may then be able to view various information about those plant(s). InFIG.5, for instance, various statistics are presented about the selected plant(s). These statistics included a health status of the plant(s) (e.g., diseased, healthy, malnourished, etc.), an estimated fruit volume (assuming the depicted plants are of a type that produces fruit, such as strawberry plants, etc.), an average fruit volume, a projected yield in kilograms, an average leaf size, an average leaf orientation (e.g., relative to a plane defined by the ground), various genotypes about the fruit (e.g., pest resistant, color, shape, etc.), and a number of branches in the plant(s). If multiple plants are selected, many of these numbers may be averages among the multiple selected plants.

A user may be able to perform other actions with selected plants as well. For example, at bottom of GUI550, a selectable element (“CLICK HERE”) is provided that a user can select to view a time-lapsed sequence of images of the selected plant(s). As another example, where such data is available, a user can view 3D data generated for the plants, such as point clouds of the entire plants, or of selected portions of the plants (e.g., fruit, leaves, etc.).

As yet another example, in some implementations, a user may be able to select a particular plant and a particular time during the plant's crop cycle, e.g., two weeks ago, three weeks in, two months into a crop cycle, three weeks in the future, etc. Based on these selections, the user may be presented with predictions about estimated plant growth and/or evolution of the plant. In some such implementations, these predictions may be determined using a time-sequence of images of the same plant, which may be associated with each other (e.g., in database114) and/or with a unique identifier of the plant using techniques described herein. In some such implementations, these predictions may be based on other signals as well, such as known plant attributes/genotypes/phenotypes (e.g., pest resistance, drought resistance, etc.), and/or exogenous factors such as irrigation, precipitation, sunlight, temperature, available/applied nutrients, applied fertilizers/pesticides, etc.

The GUI550ofFIG.5, and more particularly, the ability of the user to select and view information about individual plants over time, is made possible from techniques described herein. Without being able to recognize individual plants over time, in distinction from other plants, it might not be possible without extraordinary effort to track individual plants' progress, disease states, growth trajectories in view of environmental conditions, applied pesticides/fertilizers, etc.

In some implementations, a user may be able to search for individual plants and/or images of individual plants based on their attributes. For example, a user could provide the search query “show me plants infested with mites” at a GUI similar to550. The search query may be provided to agriculture knowledge system102. In response, agriculture knowledge system102may search database for plants in a particular field known to be infested with mites. The results may include a list of individual plants that were observed, e.g., in their respective time-sequence of images, to be infested with mites. A user may select any one of these results to view statistics about the plant associated with the selected result, to view a time-sequence of digital images of the plant associated with the selected result, etc.

In some implementations, a user may operate GUI550to view genotype and/or phenotype information about a selected plant. For example, the user may select a particular plant that appears to be growing well. The user can then see what genotype attributes and/or phenotype attributes are present in the selected plant. These genotype/phenotype attributes may be determined, for instance, by performing image processing on the digital images of the plant, or from known attributes of those plants. Once the user knows which genotype/phenotype attributes are present, the user can, for instance, search on those attributes to identify other individual plant(s) having similar attributes, or find other individual plant(s) that lack those attributes. In the latter case, the user may decide to cull the plant(s) lacking those attributes, e.g., to make room for new plants and/or to conserve nutrients and/or other resources for those remaining plants having favorable genotype/phenotype attributes.

FIG.6is a flowchart illustrating an example method600of performing selected aspects of the present disclosure, in accordance with implementations disclosed herein. For convenience, the operations of the flow chart are described with reference to a system that performs the operations. This system may include various components of various computer systems. Moreover, while operations of method600are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted or added.

At block602, the system may obtain a digital image that captures at least a first plant of a plurality of plants. As noted previously, this digital image may be obtained in real time, e.g., as it is captured by a robot108, or it may be pulled by digital image engine112from database114. At block604, the system may determine, based on a sensor signal generated by a sensor that is relevant to the plant, additional data indicative of an additional attribute of the first plant. This sensor may be, for instance, the same vision sensor that captured the digital image (e.g., when the additional attribute is a bounding shape capturing, or an environmental context of, the plant), a tensiometer (to measure soil moisture), a position coordinate sensor of the robot108that took the picture, a thermometer, light sensor (e.g., to measure sunlight exposure), a clock, etc. The additional data indicative of the additional attribute of the plant may be, for instance, dimension(s) of a bounding shape, environmental context of the plant, a position coordinate associated with the digital image, a timestamp, a soil moisture measurement taken in closest temporal proximity to the digital image, a temperature reading, or a posteriori knowledge such as applied fertilizer/pesticides, genetic traits, etc.

At block606, the system may apply the digital image and the additional data as input across a machine learning model to generate output. This output may be, for instance, a unique identifier associated with the plant, e.g., in database114or even118, a latent space embedding that is clustered near other embeddings generated from other digital images of the same plant, etc. Based on the output generated at block606, at block608, the system may store, e.g., in database114, an association between the digital image that captures the first plant and one or more previously-captured digital images of the first plant. For example, these images may be indexed in database114by a shared unique identifier.

Later, at block610, the system may receive user selection, e.g., at a user interface such as GUI550, a speech interface, etc., of a user interface element that corresponds to the first plant. For example, the user may select the first plant in portion552of GUI550. Based on the user selection received at block610, at block612, the system may cause first plant information to be output. This first plant information may be determined based on, among other things, the digital that captures the first plant and the previously-captured digital image(s) of the first plant.

FIG.7is a block diagram of an example computing device710that may optionally be utilized to perform one or more aspects of techniques described herein. Computing device710typically includes at least one processor714which communicates with a number of peripheral devices via bus subsystem712. These peripheral devices may include a storage subsystem724, including, for example, a memory subsystem725and a file storage subsystem726, user interface output devices720, user interface input devices722, and a network interface subsystem716. The input and output devices allow user interaction with computing device710. Network interface subsystem716provides an interface to outside networks and is coupled to corresponding interface devices in other computing devices.

User interface input devices722may include a keyboard, pointing devices such as a mouse, trackball, touchpad, or graphics tablet, a scanner, a touchscreen incorporated into the display, audio input devices such as voice recognition systems, microphones, and/or other types of input devices. In general, use of the term “input device” is intended to include all possible types of devices and ways to input information into computing device710or onto a communication network.

User interface output devices720may include a display subsystem, a printer, a fax machine, or non-visual displays such as audio output devices. The display subsystem may include a cathode ray tube (CRT), a flat-panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for creating a visible image. The display subsystem may also provide non-visual display such as via audio output devices. In general, use of the term “output device” is intended to include all possible types of devices and ways to output information from computing device710to the user or to another machine or computing device.

Storage subsystem724stores programming and data constructs that provide the functionality of some or all of the modules described herein. For example, the storage subsystem724may include the logic to perform selected aspects of the method ofFIG.6, as well as to implement various components depicted inFIG.1.

These software modules are generally executed by processor714alone or in combination with other processors. Memory725used in the storage subsystem724can include a number of memories including a main random access memory (RAM)730for storage of instructions and data during program execution and a read only memory (ROM)732in which fixed instructions are stored. A file storage subsystem726can provide persistent storage for program and data files, and may include a hard disk drive, a floppy disk drive along with associated removable media, a CD-ROM drive, an optical drive, or removable media cartridges. The modules implementing the functionality of certain implementations may be stored by file storage subsystem726in the storage subsystem724, or in other machines accessible by the processor(s)714.

Bus subsystem712provides a mechanism for letting the various components and subsystems of computing device710communicate with each other as intended. Although bus subsystem712is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple busses.

Computing device710can be of varying types including a workstation, server, computing cluster, blade server, server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, the description of computing device710depicted inFIG.7is intended only as a specific example for purposes of illustrating some implementations. Many other configurations of computing device710are possible having more or fewer components than the computing device depicted inFIG.7.

While several implementations have been described and illustrated herein, a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantages described herein may be utilized, and each of such variations and/or modifications is deemed to be within the scope of the implementations described herein. More generally, all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the teachings is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific implementations described herein. It is, therefore, to be understood that the foregoing implementations are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, implementations may be practiced otherwise than as specifically described and claimed. Implementations of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, if such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the scope of the present disclosure.