Patent ID: 12207582

Throughout the drawings and the detailed description, unless otherwise described, the same drawing reference numerals will be understood to refer to the same elements, features, and structures. The relative size and depiction of these elements may be exaggerated or adjusted for clarity, illustration, and/or convenience.

DETAILED DESCRIPTION

In the following description, specific details are set forth in order to provide a thorough understanding of the various example embodiments. It should be appreciated that various modifications to the embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the disclosure. Moreover, in the following description, numerous details are set forth for the purpose of explanation. However, one of ordinary skill in the art, upon reading the following disclosure, will readily understand that embodiments may be practiced without the use of these specific details. In other instances, well-known structures and processes are not shown or described in order not to obscure the description with unnecessary detail. Thus, the present disclosure is not intended to be limited to the embodiments shown but is to be accorded a scope consistent with the principles and features disclosed herein.

In accordance with one or more embodiments, a system and method are provided for an autonomous vehicle or system for detecting and identifying insects or other pests or diseases (or symptoms of diseases) that could harm or otherwise damage crops. Pursuant to some embodiments, the systems and methods may be used for other detection actions as well (such as, for example, the detection or identification of attributes of plant growth, the detection or identification of weeds present among the crops, etc.). An autonomous vehicle may comprise a wheeled vehicle and a frame which travels along a row of crops so that the wheels are located on a dirt path on opposing sides of a row of crops and a body of the autonomous vehicle travels over the row of crops. The autonomous vehicle is configured to perform one or more detection actions while traversing a field. For example, one type of detection that will be described herein as an illustrative example are detection activities related to detecting the presence of diseases or pests.

Pursuant to some embodiments, the presence of pests, or the attributes of diseases that may cause crop damage may be identified from photographs or video (“images”) captured while observing the crops. For example, a common type of disease is caused by fungi. If a crop is being affected by a fungal disease, there may be spots visible on the leaves of the crop. For example, certain fungi which are attacking a plant may result in the presence of certain spots on the leaves of the plant, such as white, red, or brown spots in some instances. Moreover, the shape and size of the spots may be markers indicating the type of fungus attacking the crop and how long or how much damage has been caused by the fungus. The presence of pests such as insects, as well as the type of pests and the amount of pests present, may be identified through the analysis of images (including both still images or photographs as well as videos) captured by the autonomous vehicle. For example, pursuant to some embodiments, one or more machine learning models may be deployed on an autonomous vehicle for use in performing inferencing processing to process images captured by the autonomous vehicle to identify and classify any pests or diseases present in the images. In some embodiments, the inferencing may be performed by one or more remote systems in communication with the autonomous vehicle. Those skilled in the art, upon reading the present disclosure, will appreciate that a number of different types of machine learning models may be used with the present invention to perform other detection tasks as described herein.

Pursuant to some embodiments, a group of autonomous vehicles may be programmed to follow a set of mission plans which defines a pre-defined path through a field of crops. The mission plan may be transmitted to the autonomous vehicle from a remote server or control system, and may specify both the pre-defined path as well as detection tasks or activities to be performed along the path. The autonomous vehicle, following the mission plan, may be configured to stop or slow down at pre-defined location(s) along the path in order to take one or more images from one or more cameras disposed on the body of the autonomous vehicle. For example, the autonomous vehicle may include various cameras disposed at different heights of the frame of the robot to capture images of a crop located in a horizontal or lateral direction from the frame. In some embodiments, there may also be one or more cameras disposed on an underside of the frame to take one or more photos of crops beneath the frame of the autonomous vehicle. The autonomous vehicle may also include selectively placed lighting devices, such as light emitting diodes (LEDs) or other illumination devices to ensure that optimal images are captured, even in dark or low-lighting conditions. In some embodiments, these lighting devices may be automatically controlled by the autonomous vehicle to operate in low or no-light conditions to ensure that images captured by the cameras are of high quality. Captured images may be associated with certain metadata or other information such as a geolocation indicating a location at which each image was captured as well as a timestamp indicating the date and time when each image was captured or acquired

The autonomous vehicle may also include a computer processing capability to automatically analyze images to determine whether crop damage from any pests or diseases is detected or identified in the captured images (e.g., by providing the images as inputs to one or more machine learning models trained to classify or detect the presence of pests, diseases, crop damage, or to perform other detection actions). If, for example, such crop damage is detected, one or more messages may be transmitted from the autonomous vehicle to a server or other processing device to inform the server of the detected disease or pest, a location at which the pest or disease (or other attribute to be detected) was detected, an identification of the pest, disease or other attribute, which may include the type of disease detected and/or the type of pest detected, and the captured image to serve as proof to the detection action. The remote system may include one or more computer processors or systems to process messages received from an autonomous vehicle and formulate a response which may include spraying certain pesticides or other chemicals on the crops at the identified locations in order to control detected diseases and/or pests. The timing of spraying of pesticides, other chemicals or biologicals may be of critical importance because delays in such spraying may adversely affect crop growth, possibly permanently. Accordingly, by periodically transmitting messages from the autonomous vehicle to the server as the autonomous vehicle moves through one or more rows of crops along a pre-defined path, detected diseases and pests may be relatively quickly addressed.

In one or more embodiments, an autonomous vehicle may travel at a relatively fast speed along a path through a field of crops. For example, the autonomous vehicle may travel at a speed of 12-15 km/hour through a row of crops. In part, this is made possible by the mechanical design of the autonomous vehicle (which, as described further below, allows the vehicle to travel through crops without damaging the crop). This is also made possible by the autonomous nature of the vehicle which allows it to traverse rows in fields by following waypoints established in a mission plan in addition to relying on navigation autonomy capabilities for crop row and obstacle detection and recognition informing travel path. Further, the autonomous vehicle may perform operations at high speeds at any hour of day, as the vehicle is capable of operating in both light and dark conditions.

In some implementations, the autonomous vehicle may employ a sampling technique for capturing or acquiring images. For example, the autonomous vehicle may stop at periodic intervals, such as once every 100 m to take photos or video. In one implementation, the autonomous vehicle may include five cameras. Each of the cameras may capture one or more images (both while the vehicle is moving and while the vehicle is stopped). In some implementations, all of the cameras may capture images at the same times, whereas in other implementations, the cameras may capture images one at a time, such that a first camera captures an image and then a split second later, a second camera captures an image, and so forth. To ensure that optimal images are acquired and that a view of one or more of the cameras is not obscured, partially or fully, while capturing images, the autonomous vehicle may automatically be operated to advance a short distance, such as one meter, and take another series of images and then advance one additional meter and take a third set of images before advancing another 100 m to the next image acquisition site of the predefined path. By moving the autonomous vehicle at a relatively fast pace and stopping at defined locations to capture images, a number of detection tasks (such as detecting the presence of pests and diseases affecting crops) may be performed and addressed quickly.

Pursuant to some embodiments, a control system (such as a mobile control vehicle) may be in communication with one or more autonomous vehicles to provide mission plans to the autonomous vehicles and to receive the detection results from those vehicles. The control system may perform processing to aggregate and analyze the detection results from multiple passes on a field (from one or more autonomous vehicles) to generate a heat map or analysis of problem areas in the field. In this manner, embodiments allow the performance of a number of different detection tasks. The detection tasks are performed substantially automatically by one or more autonomous vehicles that are capable of operating for extended periods of time even in low or no-light conditions. Further, the autonomous vehicles of the present invention can perform such detection processes quickly and without damaging plants.

For convenience and ease of exposition, a number of terms are used herein. For example, the term “autonomous” is used to refer to a vehicle that is capable of operation without active physical control or monitoring by a human operator. As used herein, the term “autonomous” may also refer to semi-autonomous operation of a vehicle (e.g., where some human intervention may be required or possible during operation of the vehicle).

The term “image” or “images” is used to refer to pictures or videos obtained by a camera mounted on the autonomous vehicle of one or more embodiments.

The term “machine learning model” or “model” may be used to refer to a model trained to classify or detect patterns in one or more images. For example, a model may be a so-called “classification” model that is configured to receive and process image data and generate output data that “classifies” the image data (e.g., as including a type of pest or disease). As used herein, the term “classification model” can include various machine learning models, including but not limited to a “detection model” or a “regression model.” Embodiments may be used with other models, and the use of a classification model as the illustrative example is intended to be illustrative but not limiting. As a result, the term “model” as used herein, is used to refer to any of a number of different types of models (from classification models to segmentation models or the like).

The term “mission plan” refers to a plan of operation that may be executed by the autonomous vehicle of the present invention. The “mission plan” may be provided to the vehicle in a file or other data structure that defines a number of geographical locations and actions to be taken by the autonomous vehicle. In some embodiments, the mission plan (as well as data collected during the execution of a mission by the autonomous vehicle) might be configured as robot operating system (“ROS”) bagfiles compatible with the ROS™ by Open Robotics. Other data file configurations and structures might be used in other embodiments.

Further, while specific examples are provided herein describing the operation of the autonomous vehicle to perform tasks associated with the detection of the presence and location of pests or disease, embodiments are capable of performing a number of different tasks. For example, in some embodiments, the autonomous vehicle may be configured to capture information associated with one or more of: (i) the density of planting in an area, (ii) the morphology of planting in an area (e.g., to determine information such as leaf area on plants, height of plants, number and size of healthy fruits, etc.), and (iii) the quantity and quality of a crop in an area (e.g., to predict the economic yield of the crop). Pursuant to some embodiments, an autonomous vehicle of the present invention may execute each or any of these tasks based on information provided in a mission plan. For convenience, these tasks (including the tasks of detecting the presence and location of pests and diseases) will be referred to herein as “detection tasks”.

FIG.1is a first elevated perspective view102of an autonomous vehicle100according to an embodiment. As discussed above, autonomous vehicle100may travel through an agricultural field, such as through one or more rows of crops and may acquire images (including, for example, photos or videos) of the crops to perform one or more detection tasks. For example, continuing the examples introduced above, the autonomous vehicle100may be controlled to substantially automatically traverse a field pursuant to a mission plan and perform operations to detect the presence of disease, such as fungi, and/or pests, such as insects, which may adversely affect the growth and health of the crops. The autonomous vehicle100may be programmed, under the direction of a mission plan, to travel a particular route through the field or a portion of a field. As will be discussed further below, the autonomous vehicle100is shaped such that it may travel along rows of crops while minimizing or substantially eliminating any damage to the crops. The plants of the rows of crops pass through a center portion of the autonomous vehicle100while wheels of the autonomous vehicle100travel along the rows on either side of the plants.

In some embodiments, programming that defines the route and specific detection tasks to be taken by the autonomous vehicle100is defined (at least in part) by the mission plan delivered to the autonomous vehicle100from a central control system or vehicle (e.g., such as the control system1480ofFIG.14, or the communications vehicle240ofFIG.2). For example, a mission plan may specify a defined path or route that may be comprised of a series of waypoints. A “waypoint,” as used herein, refers to defined location along a path. Each waypoint, for example, may be identified by a geographic location. The geographic location may be specified by a global positioning system (“GPS”) location or the like. For example, a defined path may include a series of waypoints which are located several meters or more apart. In some embodiments, a remote system (such as the control system1480ofFIG.14or the communications vehicle240ofFIG.2) creates a mission plan by mapping a field and identifying one or more waypoints. Pursuant to some embodiments, one or more navigation cameras and other sensors may be provided on the autonomous vehicle100. These devices may provide data to aid in navigating the autonomous vehicle100through a field. For example, in some embodiments, one or more stereoscopical depth cameras are deployed on the autonomous vehicle100to provide data that is used by a control system (shown as item1402ofFIG.14) of the autonomous vehicle100to assist the navigation of the autonomous vehicle100. For example, these cameras and other sensors (such as LiDAR sensors) may provide fine tuning for row navigation and object detection for collision avoidance while executing a mission plan. In this manner, embodiments utilize GPS waypoints for coarse navigation and sensor data (such as from navigational cameras) for fine tuning of the navigation and collision avoidance. This allows the autonomous vehicle100to adjust to the actual field conditions, providing additional robustness and navigational accuracy, and further reducing crop damage.

The autonomous vehicle100may be programmed to travel to or through each of the waypoints. At certain waypoints, the autonomous vehicle100may stop (or slow down) and to perform one or more detection tasks. For example, the autonomous vehicle100may control the operation of one or more cameras or sensors to perform one or more detection tasks (e.g., such as controlling one or more cameras to capture images of plants proximate the autonomous vehicle100). In some embodiments, the detection task may include both the operation of one or more cameras to take one or more images as well as inputting those images into one or more machine learning models to determine whether a pest, disease or other item of interest is present in the one or more images. In some embodiments, the detection task may further include the operation of one or more lighting devices in conjunction with the one or more cameras to compensate for any low lighting condition that may presently exist.

In some implementations, the autonomous vehicle100may operate the one or more cameras to take images at a waypoint, then travel a certain distance, such as one additional meter, operate the one or more cameras to take more images, and then travel an additional distance, such as one more meter, at which point one or more cameras may be operated to acquire one or more additional images. After operating the one or more cameras to take one or more images as specified by the mission plan, the autonomous vehicle100may subsequently travel to the next waypoint along the predefined path. Some waypoints may indicate that autonomous vehicle100is to change direction, such as to make a 90 degree turn. For example, if the autonomous vehicle100has reached the end of a row of crops, there may be a waypoint at which the autonomous vehicle100is to change direction of travel in order to reach the next row of crops. There may also be some waypoints at which the autonomous vehicle100is to continue travelling without stopping or changing directions. In some embodiments, the waypoints may be used to control both the movement and direction of the autonomous vehicle100as well as to indicate which detection tasks are to be performed at different locations.

While not shown inFIG.1, the autonomous vehicle100includes a number of sensors and control systems allowing the autonomous vehicle100to be operated in an autonomous or semi-autonomous manner. For example, the autonomous vehicle100may include one or more navigation modules (shown as item1412ofFIG.14) which may be operated to capture information such as position data. For example, the sensors may include one or more satellite positioning sensors and/or inertial navigation systems such as GNSS/IMU. A Global Navigation Satellite System (GNSS) is a space-based system of satellites that provide the location information (longitude, latitude, altitude) and time information in all weather conditions, anywhere on or near the Earth to devices called GNSS receivers. GPS is the world's most used GNSS system. An inertial measurement unit (“IMU”) is an inertial navigation system. In general, an inertial navigation system (“INS”) measures and integrates orientation, position, velocities, and accelerations of a moving object. An INS integrates the measured data, where a GNSS is used as a correction to the integration error of the INS orientation calculation. Any number of different types of GNSS/IMU sensors may be used in conjunction with features of one or more embodiments of the present invention. The data collected by each of these sensors may be processed by a vehicle controller (shown inFIG.14as item1408) to generate control signals that control the operation of the autonomous vehicle100. For example, the vehicle controller may generate control signals to control the operation of one or more drive motors (shown inFIG.14as item1436) and one or more steering motors (shown inFIG.14as item1434) thereby causing the autonomous vehicle100to move along a path defined by a mission plan. As discussed above, additional sensors and devices may be provided which enhance the navigational accuracy of the autonomous vehicle100. For example, in addition to the GNSS and IMU sensors, in some embodiments, the absolute encoders from the steering motors (shown as item1434ofFIG.14) may provide information such as wheel turning angles, and encoders from traction motors (giving wheel speed information, e.g., from the drive motors1436ofFIG.14) may provide information about the vehicle's speed. This data may be processed by the control system1402ofFIG.14and may be processed using sensor fusion to further enhance the navigational control and accuracy of the autonomous vehicle100. Some or all of the sensors and control systems may be mounted on the autonomous vehicle100within structural walls or other compartments of the autonomous vehicle100as will be discussed further below.

The autonomous vehicle100may include two walls (or legs, or also referred to as “structural walls”) that extend downward from a roof resulting in a shape of the autonomous vehicle100that allows plants of a row of crops to pass between the two walls as the autonomous vehicle100travels along the row. As will be described further below, each wall is configured to reduce or substantially eliminate any damage to the plants as the autonomous vehicle100travels along the row (even when traveling at a high speed). As shown inFIG.1, the two walls include a first structural wall105and second structural wall110. Each structural wall105,110may include two or more wheels disposed along the bottom of the structural wall105,110. A structural top120or roof may be disposed on a top end of the autonomous vehicle100, extending between the first structural wall105and the second structural wall110. The structural top120may be formed at least partially of a sturdy and strong material, such as a metal material to create a strong connection between the first structural wall105and the second structural wall110. In some embodiments, the structural top120may be formed of panels which may be removed from the structural walls105,110and replaced with panels of different sizes, allowing the effective width of the autonomous vehicle100to be modified. For example, the width of the vehicle100(e.g., the distance between structural wall105and structural wall110) may be increased to execute a mission plan that involves larger plants or crops, and the width may be reduced to execute a mission plan that involves smaller plants or crops. As an illustrative, but not limiting example, the width of the autonomous vehicle100may be configured to allow the autonomous vehicle100to traverse crop rows that are 46 cm, 76 cm or 90 cm apart.

As shown inFIG.1, the two walls105,110have a distinctive shape that is wider at the bottom and narrower at the top (e.g., a leading and trailing edge of the walls extend diagonally up to the structural top120). This shape minimizes any damage to plant foliage as the vehicle100passes along a row of crops. Other features of some embodiments which reduce or eliminate plant damage during operation of the autonomous vehicle100will be described further below.

Pursuant to some embodiments, the autonomous vehicle100is designed to minimize or substantially eliminate any damage to crops as the autonomous vehicle100traverses rows of a field. For example, as shown inFIG.1, when viewed from one of the sides, the autonomous vehicle100has a shape that generally slopes upwards from the bottom of the autonomous vehicle100to the top of the autonomous vehicle100. This shape enables the autonomous vehicle100to quickly travel through both immature and mature crops without snagging, snapping or otherwise damaging the limbs of the crop (as the shape of the autonomous vehicle100causes the limbs to be pushed upward, away from the wheels of the autonomous vehicle100). Both the leading and trailing surfaces of the autonomous vehicle100have a similar shape, allowing the autonomous vehicle100to move in either direction without damaging crops. Further, as shown inFIG.1, the leading and trailing surfaces of the autonomous vehicle100are covered by a bumper175which is formed with very few edges that may cause a branch, leaf or limb of a plant to be caught or snagged as the autonomous vehicle100passes through a field. In some embodiments, each bumper175may be formed using 3-D printing or other techniques that produce a rounded face that is not susceptible to snagging or catching crops. Each bumper175may be formed, for example, of a nylon material with carbon fibers or other materials.

As will be described further below, the autonomous vehicle100is further configured to reduce or eliminate damage to crops through the use of a steering and suspension system in which the wheels of the autonomous vehicle100can be turned and operated without catching or snagging plant leaves, limbs or branches. The result is an autonomous vehicle100that can quickly and efficiently traverse a field while capturing images or performing other tasks without damaging crops. As will be described further below, the autonomous vehicle100can perform such operations at all hours of the day and night.

Each structural wall105,110may include wheels115disposed near a front end and a back end thereof and which enable movement of the autonomous vehicle100. One or more of the wheels115may be partially encapsulated by a wheel cover125and a bumper175. For example, the wheel cover125and the bumper175may encapsulate a majority of the surface area of the corresponding wheel115. Each wheel115may have a particular tread suitable for operating autonomous vehicle robot100through relatively bumpy and rough agricultural fields. If the soil of a particular agricultural field is known to be relatively rocky a different wheel tread may be desirable versus use on another agricultural field known to have a high amount of clay soil, for example. In some embodiments, each wheel115may be approximately 40 cm in diameter, although different sizes may be used in different environments. Each wheel cover125may be formed of a sturdy material such as a hard plastic or metal and may extend below a midpoint of an axis of the wheel115. The wheel cover125and bumper175may offer protection to a wheel115by, for example, preventing sticks, leaves, or other portions of a crop or other plant from being entangled around the wheel115, such as around an axle thereof. Each wheel cover125may include a pin126or other component to secure the wheel cover125to an axle of wheel115to ensure that the wheel cover125has an ability to radially change direction in tandem with radial movement of wheel115. For example, if wheel115rotates radially by 45 degrees in order to change direction, wheel cover125and bumper175may also rotate radially by 45 degrees. The wheel cover125may extend from a connection point132located above the top of wheel115to a position near the bottom of wheel115, such as to a few inches above a bottom surface of wheel115. For example, the amount of wheel115that is exposed may be less than 20 cm or about 17 cm to reduce potential damage to crops. The wheel cover125and bumper175(as well as other panels of the present invention) also provide protection to the internal wiring and hydraulic systems.

The rotation of a wheel115as well as the wheel cover125and the bumper175are shown inFIG.16. When a wheel115is turned (e.g., to operate the autonomous vehicle100through a turn), the wheel115, the wheel cover125, and the bumper175all turn, while the wheel assembly cover170remains fixed. Further, the wheel assembly cover170is spaced slightly apart from the bumper175, allowing the bumper175to rotate within the wheel assembly cover170. At all times during a turn, this configuration reduces the possibility of branches, leaves or limbs of plants being damaged or caught on the autonomous vehicle100. Further, the bumper175, the wheel115and the wheel cover125are able to move independently of the rest of the autonomous vehicle100in a vertical direct. For example, when the autonomous vehicle100moves across obstacles, the suspension of the autonomous vehicle100(shown inFIGS.7and8) allow the bumper175, the wheel115and the wheel cover125to move up and down without exposing any gaps or edges that could snag or otherwise damage a plant. In part, this independent movement is achieved by mounting the bumper175, the wheel115and the wheel cover125to pivot points (shown as items815ofFIGS.7and8) that also are attached to the suspension of each wheel assembly.

In some embodiments, each structural wall105,110is formed around a substantially rectangular shaped chassis frame (shown as item704ofFIGS.7and8). The structural walls105,110have a number of removable panels mounted on the chassis frame which protect electronics and other components that may be mounted within the walls105,110(which will be described further below). In some embodiments, the first structural wall105may be comprised of two or more portions, such as a first structural wall upper portion128and a first structural wall lower portion130. Second structural wall110may similarly be comprised of two or more portions. First structural wall upper portion128may include an upper panel135which may be secured to first structural wall105by screws, bolts, or any other suitable securing mechanism. Upper panel135may include an emergency button140or a hole through which emergency button140may be accessed. In some embodiments, for example, if the autonomous vehicle100experiences a malfunction or other issue, a human worker or operator may manually depress the emergency button140to stop movement of autonomous vehicle100and/or to power down the autonomous vehicle100. The autonomous vehicle100provides a number of modularity and expansion benefits. In some embodiments, for example, if relatively tall crops are to be analyzed, first structural wall portion128may be removed from first structural wall105and replaced with a replacement structural wall portion having a larger height. Similarly, second structural wall110may be comprised of two or more portions, one of which may be replaced with a replaced portion in order to increase (or decrease) a height of second structural wall110. For example, portions of first structural wall105and second structural wall110may be removed and replaced with replacement portions having different heights in other to change a distance between structural top120of autonomous vehicle100and the bottom of the wheels115thereof. Further, each of the panels of the structural walls105,110may be replaced with different panels that are configured to hold different sensors or devices.

Various circuitry may be disposed within first structural wall upper portion128and may be protected from environmental elements, for example, by upper panel135. For example, circuitry for implementing movement of the autonomous vehicle100, performing computer vision to enable the movement across various terrain and around obstacles, processing images and/or video captured of crops to identify pests and/or diseases may additionally be performed by the circuitry. For example, the circuitry may include one or more processors, such as a Central Processing Unit (CPU), a Vision Processing Unit (VPU), various signal processing devices, one or more memory or storage devices, and various input/output devices (e.g., as shown and described in conjunction withFIG.14below).

The first structural wall105may include a number of removable panels. For example, a lower panel145may be provided which forms a cavity in the first structural wall105which houses one or more power sources, such as batteries. Such batteries may power movement and other circuitry of autonomous vehicle100(such as shown inFIG.14). Each of the removable panels on the structural walls105,110(as well as the structural top120) may be secured by screws, bolts, or any other suitable securing mechanism.

In some embodiments, operation of the autonomous vehicle100may be aided by the use of a computer vision system or an embedded LIDAR system able to generate a 3D point cloud of plants and obstacles. For example, a computer vision system may include one or more navigation cameras150to capture video or other images of terrain in front of the autonomous vehicle100to ensure that the autonomous vehicle100is able to traverse from waypoint to waypoint along a predefined path while passing or avoiding driving into obstacles in the path.

A number of different lighting devices, such as light emitting diodes (LEDs) may be disposed on the chassis of the autonomous vehicle100. For example, a row of LEDs may be disposed on a bottom side of structural top120to illuminate portions of a crop disposed below structural top120. Such illumination may be particularly useful for circumstances when autonomous vehicle100is acquiring photos or video at night or when the conditions are otherwise relatively dark so that clearer images and video may be acquired. As will be described below in conjunction withFIG.5, one or more lighting devices may also be provided in conjunction with one or more cameras to improve the quality of images captured in low light or dark conditions. Each of these lighting devices are powered by power modules of the autonomous vehicle100and controlled by a control system of the autonomous vehicle100(e.g., as shown inFIG.14).

When in operation, autonomous vehicle100may be operated (such as under control of the autonomous vehicle controller1408shown in system1400ofFIG.14) travel along an axis155as indicated by arrows illustrated along axis155. The autonomous vehicle100may also travel in a reverse direction along axis155and the autonomous vehicle controller1408shown inFIG.14may cause the operation of wheels105to change direction by controlling the operation of the drive motors1436.

The shape of the body or structure of autonomous vehicle100is designed to reduce drag from plants or crops being observed. For example, the shape of the body of autonomous vehicle100is designed to be sufficiently wide and sufficiently tall to reduce or minimize the occurrences of any portion of autonomous vehicle100knocking into portions of plants or crops which may slow movement of the autonomous vehicle100.

FIG.2illustrates an overhead view202of an autonomous vehicle100operating in an agricultural field200according to an embodiment. The field200may include one or more rows of crops such as a first row205, second row210, third row215, fourth row220, fifth row225, and sixth row230. Although six rows of crops are shown inFIG.2, it should be appreciated that any number of rows of crops may be disposed in an agricultural field200in which the autonomous vehicle100operates. There may be dirt disposed in the space between each row of crops.

The autonomous vehicle100may traverse the first row205with the wheels115of first structural wall105disposed on the dirt on one side of the first row205and the wheels115of second structural wall110disposed on the dirt on the other side of the first row205. As the autonomous vehicle100traverses a row, the crops of the first row105are disposed on the space formed between first structural wall105, second structural wall110, and below an underside surface of structural top120. As discussed above, the mission plan being executed by the autonomous vehicle100may cause the autonomous vehicle100to travel a certain distance along first row205and to periodically perform one or more detection tasks (e.g., such as operating one or more cameras to capture one or more images at different waypoints to detect a presence of disease and/or pests).

In some embodiments, the autonomous vehicle100may be transported to agricultural field200via a communications vehicle240(also referred to as a base station). For example, the communications vehicle240may be driven to the end of agricultural field200with the autonomous vehicle100in the back or trunk of communications vehicle240. The autonomous vehicle100may drive down a ramp out of the back of communications vehicle240or may otherwise be wheeled down the ramp. The communications vehicle240may wirelessly communicate with one or more autonomous vehicles100. For example, the communications vehicle240may transmit one or more messages with a mission plan or other instructions defining one or more paths for the autonomous vehicle100to travel. The mission plan or other instructions may also define the waypoints at which one or more detection tasks are to be performed (e.g., such as the locations at which one or more cameras are to be operated to capture one or more images). In some embodiments, the mission plan or other instructions may also define waypoints at which other sensors of the autonomous vehicle100are to acquire other types of samples, such as soil samples, crop leaf samples, or inspect samples via use of automated insect traps that can detect and classify insect species, as discussed in more detail below with respect toFIG.7. In some embodiments, during the execution of a mission plan, autonomous vehicle100may similarly communicate with communications vehicle240if it gets stuck, tips over, is running low on battery power, or experiences any other type of malfunction, for example. In some embodiments, the communications vehicle240acts as a mobile base station and is deployed in or near an agricultural field to support one or more autonomous vehicles100in the execution of one or more mission plans associated with the field. The communications vehicle240may also be equipped with equipment to support the operation of the autonomous vehicles100(e.g., such as spare batteries, tires, maintenance tools and equipment, etc.).

FIG.3illustrates a front view300of the autonomous vehicle100in an agricultural field200according to an embodiment.FIG.3illustrates how the wheels115of the first structural wall105and the wheels115of the second structural wall110of autonomous vehicle100are positioned on opposite sides of a first row205of crops during operation.

Referring back toFIG.1, the autonomous vehicle100may include one or more cameras, such as camera510to capture images of crops. Although only one camera510is shown inFIG.1, any number of cameras may be included, such as those shown in the embodiment illustrated inFIG.5.

The autonomous vehicle100may include an antenna165to enable communication between the autonomous vehicle100and a server or other electronic devices of communication vehicle240or other control system (e.g., such as control system1480ofFIG.14). For example, the autonomous vehicle100may receive a mission plan or other instructions via one or more messages received through use of antenna165. Similarly, the autonomous vehicle100may transmit the results of a detection task to a control system while the autonomous vehicle100is traversing a field or executing a mission plan. For example, the autonomous vehicle100may perform actions to capture images, analyze the images and transmit results to a control system via an antenna165substantially in real time as the autonomous vehicle100traverses a field. Information such as locations of disease and/or pests and, in some implementations, the type of disease and/or pests detected as well as the density or amount of the disease and/or pests detected may be transmitted as well as images associated with each detection.

FIG.4is a second perspective view402of an autonomous vehicle100according to an embodiment. For example,FIG.4illustrates an obverse view of autonomous vehicle100relative to the first perspective view shown inFIG.1. For example, in the second perspective view402shown inFIG.4, the second structural wall110is in front of the first structural wall105, whereas in the first perspective view shown inFIG.1, the first structural wall105is shown in front of the second structural wall110. In some embodiments, the interior facing surface of one of the structural walls105,110is colored and formed to provide a contrasting surface for images captured by cameras mounted on an interior facing surface of the other of the structural walls105,110. For example, inFIG.4, the interior facing surface of the structural wall105does not have cameras mounted thereon and is provided with a contrasting surface for images taken by cameras mounted on the interior facing surface of the structural wall110. In some embodiments, either or both of the structural walls105,110may have interior facing cameras mounted thereon to capture images for processing pursuant to the present invention.

In some embodiments, the second structural wall110may be comprised of two or more portions, such as a second structural wall upper portion405and a second structural wall lower portion410. The second structural wall upper portion405may include a removable upper panel415which may be secured to the second structural wall110by screws, bolts, or any other suitable securing mechanism. The upper panel415may include an emergency button425or a hole through which the emergency button425may be accessed. The emergency button425of the upper panel415may be the same as or similar to the emergency button140of the upper portion128of the first structural wall105as shown inFIG.1. If the autonomous vehicle100experiences a malfunction, for example, a human worker or operator may manually depress the emergency button425to stop movement of the autonomous vehicle100and/or to power down the autonomous vehicle100in some implementations. Various circuitry may be disposed within the first structural wall upper portion405and may be protected from environmental elements, for example, by the upper panel415. For example, circuitry for implementing movement of the autonomous vehicle100, performing computer vision to enable the movement across various terrain and around obstacles, processing images and/or video captured of crops to identify pests and/or diseases may additionally be performed by the circuitry. As an example, one of the walls105,110may contain an MCU, VCU and other control electronics (as shown inFIG.14), while the other of the walls105,110may contain a hydraulic system (as shown inFIG.7as item720). Both walls may have power distribution systems and network connections.

The upper panel415may include a removable subpanel430and one or more additional subpanels in various implementations. The subpanel430may be secured to the upper panel415via use of screw, bolts, or any other suitable fastening mechanism. The subpanel430may be removed and reattached to, for example, replace a circuit board, processor, or some other item of circuitry.

The second structural wall110may include one or more removable panels of its own, such as a first lower panel420and a second lower panel435. One or more power sources, such as batteries may be disposed in a cavity behind the first lower panel420. Such batteries may at least partially power movement and other circuitry of the autonomous vehicle100. The second lower panel435may include a power switch440. The power switch440may be used to manually turn on or off power to the autonomous vehicle100, for example. The first lower panel420and the second lower panel435may each be secured to the second structural wall lower portion410of the second structural wall110by screw, bolts, or any other suitable securing mechanism, for example.

As discussed above with respect toFIG.1, the embodiment shown inFIG.4illustrates axis155. When in operation, the autonomous vehicle100may travel along an axis155as indicated by arrows illustrated along axis155. The autonomous vehicle100may also travel in a reverse direction along axis155and may turn wheels115to change direction, for example.

FIG.5is a perspective view502of the autonomous vehicle100according to an embodiment.FIG.5illustrates various details of the autonomous vehicle100which are not visible in first elevated perspective view102or the second elevated perspective view402of the autonomous vehicle100. For example, the perspective view502shows various cameras for capturing or capturing images for use in one or more detecting tasks (e.g., such as capturing images to detect the presence of disease and/or pests). For example, a first camera500, a second camera505, a third camera510, and a fourth camera515may be disposed at various locations along second structural wall110. While four cameras are shown inFIG.5, any number of cameras or imaging devices may be provided. Each of these cameras may be positioned to capture images of portions of a crop disposed in a space between the first structural wall105and the second structural wall110at a predefined or predetermined location or time, such as when autonomous vehicle100has stopped at a waypoint in accordance with a predefined path as defined by a mission plan. Several cameras located at different heights may be employed, for example, because certain diseases and/or pests may only be visible or may be more easily detected if photos or video is taken from a certain height, for example.

For example, as shown in perspective view502, the autonomous vehicle100may also include a top camera520disposed on an underside of the structural top120. The top camera520may be positioned to face approximately directly down in a direction orthogonal to a plane formed by an underside of the structural top120. By using top the camera520in such a location, images may be acquired of a top portion of a crop as the autonomous vehicle100is positioning in a row of crops with underside of the top portion120being directly above the top of such a crop.

Although only five cameras are shown in perspective view, it should be appreciated that in some implementations, more or fewer than five cameras may be employed. Moreover, an underside of the structural top120may employ more than one top camera520in some implementations. Moreover, although four cameras are shown on second structural wall110in the perspective view502, it should be appreciated that in some implementations, one or more cameras may be disposed on the first structural wall105instead of on the second structural wall110, or in addition to the cameras shown on the second structural wall110.

The perspective view502shows two light panels525. Each light panel525may be coupled to an underside of the structural top120. For example, each light panel525may include one or more LEDs to illuminate a crop and/or an area around the crop to provide an additional level of illumination which may be beneficial for acquiring useful images or video from the various cameras disposed on autonomous vehicle100. Although two light panels525are shown approximately along a center line of an underside of structural top120, it should be appreciated that in some implementations, a single light panel525or more than two light panels525may be disposed in different locations. Moreover, in some implementations, one or more additional light panels may be employed, such as an additional light panel disposed on the first structural wall105and/or on the second structural wall110. As shown by the dotted lines530, different light panels530may be positioned proximate one or more cameras. These light panels530may be configured and positioned to illuminate an area at which each camera is focused. In some embodiments, a light panel530may be formed as one or more square or rectangular LED light panels positioned proximate one or more cameras (such as the light panels530shown proximate cameras515,510). In some embodiments a light panel530may be formed as a circular or ring-shaped LED light panel (e.g., such as the light panels530shown proximate cameras500,505). Other shapes and configurations of light panels may be provided to increase the quality of images captured by the cameras. Further, the light intensity and wavelengths of each light panel530may be selected based on the nature of each camera. In this manner, embodiments allow the autonomous vehicle100to operate and perform detection tasks in a wide range of lighting conditions (including at night).

Different types of cameras may be employed within a body of autonomous vehicle100. For example, the cameras may be capable of capturing Red Green Blue color model (RGB) photographs and/or video. However, in some implementations, one or more of the cameras may be capable of capturing images other than RGB images and/or video, such as near-infrared (NIR), Red Edge, and/or thermal images, to name just a few examples among many. In some implementations, a single camera may be capable of capturing RGB, NIR, Red Edge, or thermal images. However, in other implementations, an RGB camera may be removed from the body of the autonomous vehicle100and replaced with a different type of camera, such as an NIR camera. In some embodiments, LIDAR cameras and sensors may be provided for the detection of plant morphology attributes. In some embodiments, hyperspectral cameras may also be provided for other detection tasks. The ability to replace such cameras as desired provides a modularity or customizability benefit to autonomous vehicle100, for example.

FIG.6illustrates a front view600of autonomous vehicle100according to an embodiment. As illustrated in the front view600, two light panels525may be disposed approximately along opposite sides of a midline of the underside of the structural top120in accordance with an embodiment. Further, the front view600illustrates the shape of each structural wall105,110. As shown the lower portion130is wider than the upper portion128. This allows the autonomous vehicle100to operate without damaging larger crops, as plants are typically wider at their tops. In some embodiments, the lower portion130of each structural wall105,110is approximately 20 cm wide and the upper portion128is approximately 10 cm wide (although different widths may be used). This relatively narrow structure allows the autonomous vehicle100to easily traverse rows in a causing minimal or substantially no damage to the crops.

FIG.7illustrates a first perspective interior view700of autonomous vehicle100according to an embodiment. The view700is similar to the first elevated perspective view102shown inFIG.1, but with the removal of various external panels, a top surface of structural top120, and wheel coverings125, for example. As depicted in the view700, various cavities may be disposed within the structural top120, the first structural wall105, and the second structural wall110. The structural integrity of each structural wall105,110is provided by a generally rectangular chassis frame704. The removable external panels are mounted on the rectangular chassis frame704, and control systems and other electronics are mounted within the rectangular chassis frame704. The view700shows one or more batteries705disposed behind panels on each of the first structural wall105and the second structural wall110within the rectangular chassis frame704. The batteries705are sized to hold enough charge to power movement of the autonomous vehicle100as well as to operate the electrical components and detection equipment mounted on the autonomous vehicle100. For example, each battery705may comprise a 12 volt, 100 Ah battery, such as a lithium battery. During operation, panels of the first structural wall105and the second structural wall110may be removed from the rectangular chassis frame704to replace the batteries705stored in cavities therein. For example, during operation, the batteries705of the autonomous vehicle100may be periodically replaced with fully charged batteries as prescribed or needed. A human operator may manually remove and replace used the batteries705with fully charged batteries in a particular implementation. Pursuant to some embodiments, the autonomous vehicle100may be configured to automatically perform processing to replace a depleted or otherwise unusable battery705. For example, in some embodiments, the autonomous vehicle100may automatically navigate itself to a battery replacement location which then is operated to replace one or more batteries705.

The first structural wall105may include one or more cavities710in which various circuitry, such as circuit boards, processors, storage devices, sirens, or other components of circuitry may be disposed. The rectangular chassis frame704may carry wiring to one or more ports715to which cables or connectors of one or more components of circuitry may be connected. The wiring may deliver control signals, data and power to electronics connected to the ports715and mounted within the rectangular chassis frame. Certain components, such as a greenhouse gas sensors or soil chemistry sensors or soil physics sensors or detectors may be integrated with the autonomous vehicle robot100by being connected to the one or more ports715. Such sensors or devices connected to any of the ports715may be compatible with a software platform employed by autonomous robot system100, for example.

The autonomous vehicle100may comprise a fully electric vehicle which does not require use of a combustion engine, for example. One or more electric drive motor or steering motors may be provided. In some embodiments, each wheel115is associated with a drive motor820and a steering motor830that controls the operation and movement of the wheel115. Each drive motor820and steering motor830may be coupled to the rectangular chassis frame704at a number of pivot points815which allow the components to pivot with respect to the rectangular chassis frame704(e.g., such as when a wheel traverses a bump or other obstacle). The components are also coupled to the rectangular chassis frame704via a shock absorber730or other suspension system. Power and control signals are transmitted to the drive motor820and steering motor830via wiring routed through the rectangular chassis frame704. The drive system associated with each wheel115(including the drive motor820, steering motor830, fork840, pivot points815and shock absorber830) are concealed by the wheel assembly cover170and bumper175(not shown inFIG.7).

Each wheel115may be powered by an electric drive motor820and a steering motor830. The electric drive motor820may impart a force to cause a particular wheel115to advance forward or backward, and/or accelerate. The steering motor830may impart a force to change a direction (e.g., in a clockwise or in a counterclockwise direction) on movement of the wheel115, for example. A brake (not shown inFIG.7) may slow or stop movement of the wheel115. In some embodiments, the autonomous vehicle100may utilize regenerative breaking to charge one or more batteries705while decelerating the vehicle. Because an autonomous vehicle100may perform multiple such decelerations while executing a mission plan, such regenerative breaking can result in significant improvements to the range of a vehicle. Shock absorbers730may form a portion of a suspension system and may absorb shocks from uneven terrain, for example, as wheels115are in motion.

In some embodiments, a cavity740within the structural top120may include a UV light emitter735to attract insects. Insects attracted to the UV light emitted by the emitter735may be electrically zapped when they come in contact with or come into close proximity to the UV light and remain in cavity740for subsequent analysis. For example, the number and type of insects may be counted and categorized by the autonomous vehicle100or by a human operator at periodic intervals. In some embodiments, other sensors or devices, such as a pherome emitter may be provided which emits one or more pheromes to attract insects. For example, the pheromone emitter may release pheromones and insects may fly into an opening of the structural top120to get close to the pheromone emitter. A sticky trap may be disposed adjacent to the pheromone emitter to trap any insects which come in contact with a surface of the sticky trap. For example, the number and the type of insects which are trapped within the sticky trap may be counted and categorized by the autonomous vehicle100or by a human operator at periodic intervals. For example, a human operator may periodically remove and replace a used sticky trap and may count and categorize the insects trapped on the removed sticky trap. In some embodiments, a soil sampling module may be included in a cavity located adjacent to one of the batteries705illustrated in the first perspective interior view700. For example, such a soil sampling module may include a hole which may open on a side of a panel or from a bottom of the panel. In some embodiments, a robotic arm may extend down into the soil below the soil sampling module to acquire a sample of soil. For example, such a robotic arm may extend down one or two inches into the soil to scoop out or otherwise extract a relatively small sample of soil for analysis. The robotic arm may subsequently retract into the soil sampling module and analyze the physical and chemical characteristics of the soil to determine whether there is a lack of certain macro and micro nutrients in the soil and/or a presence of certain bacteria, fungi, nematoids and viruses which adversely affect plant growth or contribute to it and/or measure the soil carbon sequestered in the soil and the greenhouse gas (“GHG”) emissions of the soil. In one example, the soil sampling module may include sensors to perform such analysis directly. Alternatively, the soil samples may be collected and the soil may subsequently be extracted and analysis may be performed after the soil samples have been removed from the soil sampling module. In some embodiments, one or more leaves may similarly be extracted from a crop via use of a robotic arm for analysis.

FIG.8illustrates a second perspective interior view800of the autonomous vehicle100according to an embodiment. For example, second perspective interior view800ofFIG.8illustrates an obverse view of the autonomous vehicle100relative to the first perspective view700shown inFIG.7. For example, in the second perspective view402shown inFIG.4, the second structural wall110is in front of the first structural wall105, whereas in the first perspective view102shown inFIG.1, the first structural wall105is shown in front of the second structural wall110. The second perspective view800is similar to the second perspective view402shown inFIG.4, but with the removal of various external panels, a top surface of structural top120, and the wheel coverings125, for example.

The second perspective view800shows certain details not visible in the first perspective view700ofFIG.7. For example, the second perspective view800shows a power switch440, which may be used to manually turn on or off power to autonomous vehicle100, for example. The power switch440may be associated with the emergency button140ofFIG.1. The second perspective view800also shows a first circuit board802and a second circuit board805. Each of first circuit board802and second circuit board805may include various circuitry components, such as processors for controlling the computer vision features of the autonomous vehicle100, analyzing photographs and/or video captured from any of the cameras embedded within a body of the autonomous vehicle100to detect diseases and/or pests, and/or generating messages to transmit to a server or other device to indicate areas of interest to spray with chemicals such as pesticides and/or biopesticides, for example. Second perspective view also includes one or more relays810which are operable to cut the main battery power when necessary.

FIG.9illustrates a top view900of the structural top120according to an embodiment. The structural top120may include various components which provide a rigid or sturdy support to the structural top120. For example, one or more frame bars905may be disposed on outer ends of the structural top, with one frame bar905disposed above first structural wall105and an opposing frame bar905disposed over the second structural wall110. Several expansion bars910may be disposed between the frame bars905. For example, the expansion bars910may be positioned orthogonal or perpendicular to an axis along which the frame bars905extend and the expansion bars910may be coupled to the frame bars905by either connectors925or support beams930. In some embodiments, the expansion bars910may be expandable. For example, a distance between first structural wall105and second structural wall110may be adjusted by increasing or decreasing the width of structural top120. In order to change the width of structural top120, each expansion bar910may be expanded or contracted

In some embodiments, the width of the structural top120may be modified by detaching the structural top120from fasteners mounting the structural top120to the walls and reattaching the structural top120at fastening locations that are a desired width apart. This allows the width of the autonomous vehicle100to be narrower or wider to accommodate different crop row widths or different crop and tree/shrub sizes.

Structural top120may also include end support beams915, which couple a front end935of structural top to an expansion bar910, or a back end940of the structural top120to an expansion bar910, for example. A central support beam920may also be included which extends between approximate center points of expansion bars910to provide additional structural stability in accordance with an embodiment. The structural top120may also include cable routing paths that route network or power cables from one structural wall to the other and which electrically connects the antenna165to other components of the autonomous vehicle100.

FIG.10illustrates a perspective view1000of the underside of the structural top120according to an embodiment. As illustrated, two light panels525may be disposed on an underside of the structural top120. For example, each light panel525may include a plurality of lighting elements, such as LEDs, and may extend between the front end935and the back end940of the structural top120. As discussed previously above, light emitted by the light panels525may illuminate a portion of a crop under observation or an area around or near the crop.

FIG.11illustrates a map1100showing a visual representation of a mission plan with three distinct passes to control an autonomous vehicle100to travel along while performing one or more detection tasks pursuant to the present invention (e.g., such as acquiring images, taking soil samples or capturing insect samples, etc.). The map1110shows a first pass1105, a second pass1110, and a third pass115for the autonomous vehicle100to travel through rows of crops of an agricultural field1102. Each pass includes various waypoints1120, each of which is denoted with an “x” in the map1100. During each pass, the autonomous vehicle100is operated to travel from waypoint1120to waypoint1120. At some of the waypoints1120, the autonomous vehicle100is operated to perform one or more detection tasks. For example, the autonomous vehicle100may be operated to stop or slow down, acquire images from each of the cameras, such as first camera500, second camera505, third camera510, fourth camera515, and top camera520, such as is shown in perspective view502ofFIG.5. Video may also be acquired from one or more of the cameras. In some implementations, a light emitter735may be operated to emit attractive light wavelengths to attract insects which may be captured via use of a trap740, such as is shown in the view700of autonomous vehicle100depicted inFIG.7.

The mission plan may define one or more waypoints1120at which the autonomous vehicle100is to change directions, such as to make a 90 degree turn to the left or to the right, for example. The mission plan may further define one or more waypoints1120at which the autonomous vehicle100is to continue travelling straight forward in the same direction, for example.

The map1100indicates three different passes. For a relatively large field, it may take several hours for the autonomous vehicle100to complete a pass, such as first pass1105. After completing first pass1105, autonomous vehicle100may proceed to a battery swapping area1125, where batteries705within the first structural wall105and/or the second structural wall110may be removed and replaced with fully charged or fresh batteries705. In some embodiments, such a battery replacement or swapping operation may be performed by a human operator within the span of a few minutes. For example, a panel behind which batteries705are disposed may be removed, such as by unscrewing screws, bolts, or otherwise unfastening a fastening mechanism. After removing such a fastening mechanism, the batteries705may be accessed and physically removed and replaced. In accordance with an implementation, a communication vehicle240such as shown inFIG.2may be positioned in a battery swapping area1125and a human operator inside the communication vehicle240may perform the battery705removal and replacement. After replacing one or more batteries705, the batteries may be placed in a battery charging mechanism if they are rechargeable batteries, so that they are fully charged for subsequent use, for example. As discussed above, in some embodiments, the autonomous vehicle100may automatically navigate to a battery swap location if the vehicle detects that one or more batteries require replacement. In some embodiments, a battery swap location may be configured to automatically replace one or more batteries705.

By performing multiple passes as shown in map1102, a relatively dense plot of information captured by the detection tasks may be produced. For example, information from a number of passes by the autonomous vehicle100(or from multiple autonomous vehicles100) may be aggregated and used to generate a “heat map” or plot depicting areas of a field where problems have been detected. For example, a plot or heat map of problem areas where crops are affected by pests and/or disease may be identified and used to determine where to spray pesticides or other chemicals to address pest and/or disease issues. The map1100is a visual representation of a mission plan that may be delivered to an autonomous vehicle100for execution. In practical application, the actual mission plan that is delivered to an autonomous vehicle100will include plain text or other instructions which, when processed by processing devices of the autonomous vehicle100, will cause the operation of the autonomous vehicle100to follow the mission plan and execute any detection tasks specified therein. In some embodiments, a control system such as the control system1480ofFIG.14may cause the generation of one or more additional mission plans to be delivered to one or more autonomous vehicles100for execution of tasks to remediate or respond to issues detected by the performance of other mission plans. For example, an autonomous vehicle100may be configured to perform pesticide applications. That vehicle may receive a mission plan that is configured with waypoints and other information that causes the vehicle to apply pesticides to locations detected by other vehicles.

FIG.12Aillustrates a plot1200of an agricultural area having detected locations of concern1205according to an embodiment. For example, the plot1200may have a shape representative of an agricultural area being monitored. If a location having pests and/or disease is identified, such as from images acquired by cameras on the autonomous vehicle100, a square may be positioned on the plot1200which indicates a location of concern1205. Plot1200indicates that there are six different locations of concern1205in the agricultural field.

FIG.12Billustrates a heat map1210generated for an agricultural area based on the plot1200showing areas of concern1205according to an embodiment. The heat map1210again illustrates locations of concern1205depicted in the plot1200ofFIG.12A. The heat map1210includes a heat area1215disposed around areas of concern1205. A radius of the heat area1215disposed around each area of concern1205may be dependent upon many factors, such as the type of crop, the type of disease and/or pest, and a magnitude of the amount of disease and/or pests detected, for example.

FIG.12Cillustrates a pesticide application map1220generated for an agricultural area based on the heat map1210according to an embodiment. For example, the pesticide application map1210may show a bounded application area1225for which pesticide is suggested to be sprayed or otherwise applied. The boundaries or outline of bounded application area1225may encapsulate all the heat area1215shown in the heat map1210ofFIG.12B. The pesticide application map1220may show a person or computing device in charge of applying pesticides where such pesticides are to be applied in order to address a disease or pest issue.

Pursuant to some embodiments, the maps or plots shown inFIG.12may be generated by a control system such as the control system1480ofFIG.14based on information received from one or more autonomous vehicles100which have executed one or more mission plans. The maps or plots may be transmitted from the control system1480to one or more user devices or other systems for analysis and to determine a course of action to resolve any areas of concern.

FIG.13illustrates an embodiment 1300 of a method for acquiring and processing images or video by an autonomous vehicle100. Embodiments in accordance with claimed subject matter may include all of, less than, or more than blocks1305through1320. Also, the order of blocks1305through1320is merely an example order. For example, a method in accordance with embodiment 1300 may be performed by various components of the autonomous vehicle100, such as the components ofFIG.14.

Processing begins at1305where a mission plan is received by the autonomous vehicle100. The mission plan may be received by the autonomous vehicle100via a communication link between the autonomous vehicle1000and a control system (such as the control system1480ofFIG.14). The mission plan may be a file or other data which specifies information to be processed by the autonomous vehicle100in order to control the operation of the autonomous vehicle100. For example, the mission plan may identify a path for autonomous vehicle100to travel through an agricultural field. The path may be defined by information specifying a plurality of waypoints (such as GPS locations). The mission plan may also include information specifying one or more detection tasks to be performed at various locations along the path. For example, a detection task may include operation of one or more cameras to capture one or more images and to process those images to determine whether the images indicate the presence of a pest or disease. Processing at1305may include operating the autonomous vehicle100to receive and store the details of the mission plan in a memory or storage (such as item1410ofFIG.14) and initialize components of the autonomous vehicle100to execute the mission plan.

Processing continues at1310where the autonomous vehicle100is controlled using a control system1402to travel along the path defined by the mission plan and to execute the detection tasks. This processing includes operating the controller1408to activate and operate components1430of a drive train of the autonomous vehicle100(e.g., such as the drive motors1436, the steering motors1434, etc.) to cause the autonomous vehicle100to travel along the path specified in the mission plan. The processing also includes operating the control system1402to activate and operate one or more detection components1416of the autonomous vehicle100to perform detection tasks specified in the mission plan. For example, one or more cameras1420may be operated to capture one or more images of a crop area.

Processing continues at1315where the autonomous vehicle100is operated to perform processing to process the information obtained from the detection task. For example, if the detection task performed at1310was to capture images of a crop area, processing at1315may include processing to analyze the images to detect the presence (or absence) of a pest or a disease. In some embodiments, this processing may include providing the images as inputs to one or more machine learning models to classify the image or to otherwise detect the presence or absence of a pest or disease. In some embodiments, the processing may further classify or identify the type of pest or disease. Other detection tasks may include processing to identify plant attributes such as leaf area, height, size of fruits, etc. Processing at1315includes associating each image with one or more items of meta data (such as the geographical location where the image was taken, a timestamp, etc.). Processing continues at1320where the results of the detection tasks are transmitted to a control system1480for further processing. For example, the control system1480may aggregate information from one or more mission plans executed by one or more autonomous vehicles100and produce one or more plots or heat maps (such as shown inFIG.12). In some embodiments each time a detection task is performed, processing at1315is performed to process the information. In some embodiments, processing at1315may be performed as a background task while the autonomous vehicle100is completing a mission plan. In some embodiments, processing at1315may be performed after completion of a mission plan. In some embodiments, portions of, or all of the task of processing information associated with the detection tasks may be performed by a remote control system1480.

FIG.14illustrates a system1400pursuant to some embodiments. The system100may include a control system1480which is in communication with one or more autonomous vehicles100thru existing or fit for purpose telemetry systems. For example, the control system1480may comprise one or more servers and associated communication services that are in communication with one or more autonomous vehicles100. The control system1480may be deployed in a communications vehicle240such as shown inFIG.2or it may be deployed in another location (e.g., such as a cloud service such as Amazon Web Services or equivalents). The control system1480is in wireless communication with each autonomous vehicle100and information is transmitted over a wireless communication link. In some embodiments, the control system1480transmits a mission plan to an autonomous vehicle100for execution by the autonomous vehicle100. The autonomous vehicle100then transmits the results of the performance of the mission plan to the control system1480for further processing. While not shown inFIG.14, the control system1480may be in communication with one or more user devices allowing users to create mission plans, review results of mission plans, etc.

The autonomous vehicle100includes a number of components, including the mechanical components shown and described in conjunction withFIGS.1-10, as well as the components shown inFIG.14. For example, the autonomous vehicle100has a number of control components1402which operate to control the operation of the vehicle as well as to perform processing to analyze the results of any detection tasks. These control components1402include one or more processing or control units1404,1406, controllers1408, data storage systems1410, navigation modules1412communication modules1414, and power modules1416. Each of the components or modules may be in communication via one or more buses and network connections routed through the autonomous vehicle100. The processing or control units1404,1406may execute computer software code to implement one or more machine learning models to perform detection tasks as described herein.

The autonomous vehicle100also includes a number of detection components1416which are operable (under control of the control components1402and the mission plan) to capture information for use in detection tasks. For example, a number of cameras1420, sensors1422, and lighting modules1424are provided as discussed elsewhere herein. A number of different types of cameras1420, sensors1422and lighting modules1424may be provided to support and perform detection tasks of the present invention. For example, a number of different types of cameras1420may be provided, including, for example, still or video capture, RGB, thermal band, multispectral, hyperspectral, LIDAR, etc.) For example, in some embodiments, sensors1422may be provided for detecting different odors (e.g., using volatile organic compound sensors), sound detection devices (e.g., to detect flight patterns of insects through ultra sound sensors), sampling devices (e.g., to obtain and analyze soil samples, leaf samples, or the like), etc. The cameras1420, sensors1422and lighting devices1424may be used to support detection tasks as well as to enhance navigation as discussed elsewhere herein. The modular construction of the autonomous vehicle100allows these sensors, cameras and lighting devices to easily be installed, replaced and maintained through the removal of the exterior panels and use of the ports and power system of the present invention.

The autonomous vehicle100also includes a number of drive components1430which allow operation of the autonomous vehicle100under control of the control system1402. The drive components1430include, for example, one or more suspension systems1432, steering motors1434and drive motors1436.

FIG.15illustrates an example computing system1500which may represent or be integrated in any of the above-described components, etc.FIG.15is not intended to suggest any limitation as to the scope of use or functionality of embodiments described herein. The computing system1500is capable of being implemented and/or performing any of the functionality set forth hereinabove. For example, a computing system such as shown inFIG.15may be implemented or deployed in the control system1480ofFIG.14, the communications vehicle240ofFIG.2, or any of the processors or systems of the autonomous vehicle100(e.g., such as the components ofFIG.14).

The computing system1500may include a computer system/server, which is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use as computing system1500include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, tablets, smart phones, databases, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, distributed cloud computing environments, databases, and the like, which may include any of the above systems or devices, and the like. According to various embodiments described herein, the computing system1500may be a tokenization platform, server, CPU, GPU, or the like.

The computing system1500may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. The computing system1500may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

Referring toFIG.15, the computing system1500is shown in the form of a general-purpose computing device. The components of computing system1500may include, but are not limited to one or more processors or processing units1505, a network interface or I/O1525, which may include a port, an interface, etc., or other hardware, for outputting a data signal to another device such as a display, a printer, etc., and a storage device or memory1510which may include a system memory, or the like. Although not shown, the computing system1500may also include a system bus that couples various system components including system memory to the processor1505.

The memory1510may include a variety of computer system readable media. Such media may be any available media that is accessible by computer system/server, and it may include both volatile and non-volatile media, removable and non-removable media. System memory, in one embodiment, implements the flow diagrams of the other figures. The system memory can include computer system readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory. As another example, memory1510can read and write to a non-removable, non-volatile magnetic media (not shown and typically called a “hard drive”). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a “floppy disk”), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected to the bus by one or more data media interfaces. As will be further depicted and described below, memory1510may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of various embodiments of the application.

As will be appreciated by one skilled in the art, aspects of the present application may be embodied as a system, method, or computer program product. Accordingly, aspects of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present application may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Although not shown, the computing system1500may also communicate with one or more external devices such as a keyboard, a pointing device, a display, etc.; one or more devices that enable a user to interact with computer system/server; and/or any devices (e.g., network card, modem, etc.) that enable computing system1500to communicate with one or more other computing devices. Such communication can occur via I/O interfaces. Still yet, computing system1500can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via network interface (such as a receiver1515and a transmitter1520). Although not shown, other hardware and/or software components could be used in conjunction with the computing system1500. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

As will be appreciated based on the foregoing specification, one or more aspects of the above-described examples of the disclosure may be implemented using computer programming or engineering techniques including computer software, firmware, hardware or any combination or subset thereof. Any such resulting program, having computer-readable code, may be embodied or provided within one or more non-transitory computer readable media, thereby making a computer program product, i.e., an article of manufacture, according to the discussed examples of the disclosure. For example, the non-transitory computer-readable media may be, but is not limited to, a fixed drive, diskette, optical disk, magnetic tape, flash memory, semiconductor memory such as read-only memory (ROM), and/or any transmitting/receiving medium such as the Internet, cloud storage, the internet of things, or other communication network or link. The article of manufacture containing the computer code may be made and/or used by executing the code directly from one medium, by copying the code from one medium to another medium, or by transmitting the code over a network.

The computer programs (also referred to as programs, software, software applications, “apps”, or code) may include machine instructions for a programmable processor and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus, cloud storage, internet of things, and/or device (e.g., magnetic discs, optical disks, memory, programmable logic devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The “machine-readable medium” and “computer-readable medium,” however, do not include transitory signals. The term “machine-readable signal” refers to any signal that may be used to provide machine instructions and/or any other kind of data to a programmable processor.

The terms, “and”, “or”, “and/or” and/or similar terms, as used herein, include a variety of meanings that also are expected to depend at least in part upon the particular context in which such terms are used. Typically, “or” if used to associate a list, such as A, B or C, is intended to mean A, B, and C, here used in the inclusive sense, as well as A, B or C, here used in the exclusive sense. In addition, the term “one or more” and/or similar terms is used to describe any feature, structure, and/or characteristic in the singular and/or is also used to describe a plurality and/or some other combination of features, structures and/or characteristics. Of course, for all of the foregoing, particular context of description and/or usage provides helpful guidance regarding inferences to be drawn. It should be noted that the following description merely provides one or more illustrative examples and claimed subject matter is not limited to these one or more illustrative examples; however, again, particular context of description and/or usage provides helpful guidance regarding inferences to be drawn.

While certain exemplary techniques have been described and shown herein using various methods and systems, it should be understood by those skilled in the art that various other modifications may be made, and equivalents may be substituted, without departing from claimed subject matter. Additionally, many modifications may be made to adapt a particular situation to the teachings of claimed subject matter without departing from the central concept described herein. Therefore, it is intended that claimed subject matter not be limited to the particular examples disclosed, but that such claimed subject matter may also include all implementations falling within the scope of the appended claims, and equivalents thereof.