Patent Description:
The business of lawn maintenance, which may be an example of vegetation maintenance, has proven to be a lucrative one. However, many of the practices of the lawn maintenance crews are based on experience and intuition and may not always be the most effective practices to efficiently maintain healthy, well-groomed lawns and other vegetation. For example, practices associated with simply determining a mowing pattern for a lawn can have substantial impacts on the health of the lawn, the quality of the cut, and the efficiency (e.g., time to completion) of the cut. In some instances, with respect to work site efficiency, the quoting process for determining the number of man-hours (and thus the cost) needed to perform a regular vegetation maintenance on a residential lawn or other worksite may be quite inaccurate using conventional approaches, which can lead to lost time and profits. As such, there continues to be a need to innovate in the area of worksite analysis and workflow optimization with respect to, for example, vegetation maintenance and similar worksite operations.

<CIT> discloses a system that aligns with property trimming and cutting tools for autonomous vehicles. The system is linked to superimposed design overlays giving augmented reality guidance. Further, an interactive imaging apparatus is provided for an improved viewing method.

<CIT> discloses controlling of an unmanned aerial vehicle to collect and forward information pertaining to a forestry worksite area. The UAV is controlled to fly to a first location and capture image information at the first location and to send the captures image information to a communication system.

<CIT> discloses a number of autonomous outdoor devices and one or more service vehicles. The service vehicles are configured to selectively load and unload one or more of the autonomous outdoor power devices at a jobsite based on instructions received by a controller of the one or more service vehicles.

A system according to the invention comprises an autonomous vehicle comprising a camera and a position sensor. The autonomous vehicle is configured to operate the camera and position sensor to capture image data associated with a worksite. The image data comprises images of the worksite with corresponding position coordinates. The system also comprises a worksite analysis engine comprising processing circuitry. The processing circuitry is configured to receive the image data of the worksite captured by the autonomous vehicle, generate a virtual layout of the worksite based on the image data, receive equipment data comprising a list of equipment available to be deployed at the worksite with corresponding equipment attributes, receive crew data comprising a number of crew members available to be deployed at the worksite, and generate a workflow based on the virtual layout, the equipment data, and the crew data. The workflow comprises workflow assignments for each crew member at the worksite, each workflow assignment indicating a task, equipment to perform the task, and an equipment path for the task.

A method according to the invention comprises capturing image data associated with a worksite. The image data is captured by an autonomous vehicle comprising a camera and a position sensor. The autonomous vehicle is configured to operate the camera and position sensor to capture the image data with corresponding position coordinates. The method according to the invention also comprises receiving the image data of the worksite captured by the autonomous vehicle by processing circuitry of a worksite analysis engine, generating a virtual layout of the worksite based on the image data by the processing circuitry, receiving equipment data comprising a list of equipment available to be deployed at the worksite with corresponding equipment attributes, receiving crew data comprising a number of crew members available to be deployed at the worksite, and generating a workflow based on the virtual layout, the equipment data, and the crew data. The workflow may comprise workflow assignments for each crew member at the worksite, each workflow assignment indicating a task, equipment to perform the task, and an equipment path for the task.

Having thus described some example embodiments in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:.

Some example embodiments now will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all example embodiments are shown. Indeed, the examples described and pictured herein should not be construed as being limiting as to the scope, applicability, or configuration of the present disclosure. Rather, these example embodiments are provided so that this disclosure will satisfy applicable legal requirements.

As used herein the term "or" is used as the logical or where any one or more of the operands being true results in the statement being true. As used herein, the phrase "based on" as used in, for example, "A is based on B" indicates that B is a factor that determines A, but B is not necessarily the only factor that determines A.

According to some example embodiments, a system is provided that is configured to perform worksite analysis in effort to increase efficiency in consideration of a number of factors. In this regard, according to some example embodiments, an autonomous vehicle, such as an aerial or land-based drone may be employed to capture position-based images of a worksite (e.g., a residential or commercial property) for provision to a worksite analysis engine to generate a model of the worksite in the form of a virtual layout. According to some example embodiments, the autonomous vehicle may be configured to capture perspective images of the worksite (as opposed merely overhead images) that can be leveraged to generate the virtual layout with topology information. The worksite analysis engine may leverage this virtual layout with other sources of information to generate, for example, an efficient equipment path to be used when performing vegetation maintenance activities (e.g., mowing, edging, trimming, blowing, aerating, seeding, leaf collection, fertilizing, or the like).

According to some example embodiments, the worksite analysis engine may implement such generated equipment paths in the context of a crew workflow. In this regard, the virtual layout may be analyzed in association with equipment data and crew data to generate a workflow as a type of sequential crew task list for efficiently and effectively performing worksite maintenance. The equipment data may include a list of available equipment for use at the worksite with corresponding equipment attributes (e.g., mowing deck width, turning radius, speed, slope limitations, clipping catch capacity, fuel consumption rate, fuel capacity, and the like). The crew data may include a number of available crew members and, for example, crew member experience data. Using this information, the worksite analysis engine may be configured to generate a workflow for each crew member, where a workflow is comprised of a sequential list of work assignments. Each work assignment may include a task to be performed, the equipment to be used to perform the task, and the equipment path to be used when performing the task. As further described below, the worksite analysis engine may also be configured to perform workflow compliance analyses to determine if the workflows are being properly executed by the crew members.

<FIG> illustrates an example system <NUM> for performing worksite analysis. According to some example embodiments, the system <NUM> may comprise a worksite analysis engine <NUM> and an autonomous vehicle <NUM>. Additionally, the system <NUM> may comprise an equipment transportation vehicle <NUM>, equipment <NUM> and <NUM>, and crew devices <NUM> and <NUM>. Further, the system <NUM> may also comprise a GIS (geographic information system) database <NUM>, a topology database <NUM>, and an equipment attribute database <NUM>.

In short, the worksite analysis engine <NUM> may be configured to gather information from a number of sources to perform various functionalities as described herein. In this regard, the worksite analysis engine <NUM> may comprise a number of sub-engines, according to some example embodiments, that may be stand-alone engines that need not be bundled into the worksite analysis engine <NUM> as shown in <FIG>. In this regard, the worksite analysis engine <NUM> may comprise a virtual layout generation engine <NUM>, an equipment path generation engine <NUM>, a crew workflow generation engine <NUM>, and a workflow compliance engine <NUM>. These engines may be configured to perform various functionalities as further described below by employing configured processing circuitry of the worksite analysis engine <NUM>.

With respect to the structural architecture of the worksite analysis engine <NUM>, referring now to the block diagram of <FIG>, the worksite analysis engine <NUM> may comprise processing circuitry <NUM>, which may be configured to receive inputs and provide outputs in association with the various functionalities of, for example, the virtual layout generation engine <NUM>, the equipment path generation engine <NUM>, the crew workflow generation engine <NUM>, and the workflow compliance engine <NUM>. In this regard, one example architecture of the worksite analysis engine <NUM> is provided in <FIG>, wherein the worksite analysis engine <NUM> comprises the processing circuitry <NUM> comprising a memory <NUM>, a processor <NUM>, a user interface <NUM>, and a communications interface <NUM>. The processing circuitry <NUM> may be operably coupled to other components of the worksite analysis engine <NUM> that are not shown in <FIG>. The processing circuitry <NUM> may be configured to perform the functionalities of the worksite analysis engine <NUM>, and more particularly the virtual layout generation engine <NUM>, the equipment path generation engine <NUM>, the crew workflow generation engine <NUM>, and the workflow compliance engine <NUM>, as further described herein.

Further, according to some example embodiments, processing circuitry <NUM> may be in operative communication with or embody, the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM>. Through configuration and operation of the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM>, the processing circuitry <NUM> may be configurable to perform various operations as described herein. In this regard, the processing circuitry <NUM> may be configured to perform computational processing, memory management, user interface control and monitoring, and manage remote communications, according to an example embodiment. In some embodiments, the processing circuitry <NUM> may be embodied as a chip or chip set. In other words, the processing circuitry <NUM> may comprise one or more physical packages (e.g., chips) including materials, components or wires on a structural assembly (e.g., a baseboard). The processing circuitry <NUM> may be configured to receive inputs (e.g., via peripheral components), perform actions based on the inputs, and generate outputs (e.g., for provision to peripheral components). In an example embodiment, the processing circuitry <NUM> may include one or more instances of a processor <NUM>, associated circuitry, and memory <NUM>. As such, the processing circuitry <NUM> may be embodied as a circuit chip (e.g., an integrated circuit chip, such as a field programmable gate array (FPGA)) configured (e.g., with hardware, software or a combination of hardware and software) to perform operations described herein.

In an example embodiment, the memory <NUM> may include one or more non-transitory memory devices such as, for example, volatile or non-volatile memory that may be either fixed or removable. The memory <NUM> may be configured to store information, data, applications, instructions or the like for enabling, for example, the functionalities described with respect to the virtual layout generation engine <NUM>, the equipment path generation engine <NUM>, the crew workflow generation engine <NUM>, and the workflow compliance engine <NUM>. The memory <NUM> may operate to buffer instructions and data during operation of the processing circuitry <NUM> to support higher-level functionalities, and may also be configured to store instructions for execution by the processing circuitry <NUM>. The memory <NUM> may also store image data, equipment data, crew data, and virtual layouts as described herein. According to some example embodiments, such data may be generated based on other data and stored or the data may be retrieved via the communications interface <NUM> and stored.

As mentioned above, the processing circuitry <NUM> may be embodied in a number of different ways. For example, the processing circuitry <NUM> may be embodied as various processing means such as one or more processors <NUM> that may be in the form of a microprocessor or other processing element, a coprocessor, a controller or various other computing or processing devices including integrated circuits such as, for example, an ASIC (application specific integrated circuit), an FPGA, or the like. In an example embodiment, the processing circuitry <NUM> may be configured to execute instructions stored in the memory <NUM> or otherwise accessible to the processing circuitry <NUM>. As such, whether configured by hardware or by a combination of hardware and software, the processing circuitry <NUM> may represent an entity (e.g., physically embodied in circuitry - in the form of processing circuitry <NUM>) capable of performing operations according to example embodiments while configured accordingly. Thus, for example, when the processing circuitry <NUM> is embodied as an ASIC, FPGA, or the like, the processing circuitry <NUM> may be specifically configured hardware for conducting the operations described herein. Alternatively, as another example, when the processing circuitry <NUM> is embodied as an executor of software instructions, the instructions may specifically configure the processing circuitry <NUM> to perform the operations described herein.

The communication interface <NUM> may include one or more interface mechanisms for enabling communication with other devices external to worksite analysis engine <NUM>, via, for example, a network, which may, for example, be a local area network, the Internet, or the like, through a direct (wired or wireless) communication link to another external device, or the like. In some cases, the communication interface <NUM> may be any means such as a device or circuitry embodied in either hardware, or a combination of hardware and software, that is configured to receive or transmit data from/to devices in communication with the processing circuitry <NUM>. In some example embodiments, the communications interface may comprise, for example, a radio frequency identification tag reader capable of reading tags in close proximity to the communications interface to gather information from the tag (e.g., identification data) and to determine a proximity of the tag to the communications interface. The communications interface <NUM> may be a wired or wireless interface and may support various communications protocols (WIFI, Bluetooth, cellular, or the like).

The communications interface <NUM> of the worksite analysis engine <NUM> may be configured to communicate directly or indirectly to various components of the system <NUM> of <FIG>. In this regard, via the communications interface <NUM>, the worksite analysis engine <NUM> may be configured to communicate directly or indirectly with the autonomous vehicle <NUM>, the equipment transportation vehicle <NUM>, the equipment <NUM> and <NUM>, the crew device <NUM> and <NUM>, the GIS database <NUM>, the topology database <NUM>, and/or the equipment database <NUM>.

Referring back to <FIG>, the user interface <NUM> may be controlled by the processing circuitry <NUM> to interact with peripheral devices that can receive inputs from a user or provide outputs to a user. The user interface <NUM> may be configured to provide the inputs (e.g., from a user) to the processor <NUM>, and the processor <NUM> may be configured to receive the inputs from the user interface <NUM> and act upon the inputs to, for example, determine and output a result via the user interface <NUM>. For example, according to some example embodiments, a user may interact with the user interface <NUM> to input a stripping pattern for mowing an area of the worksite <NUM> and indications of the stripping pattern may be provided to the processor <NUM> for analysis and determination of a path as further described herein. In this regard, via the user interface <NUM>, the processing circuitry <NUM> may be configured to provide control and output signals to a device of the user interface such as, for example, a keyboard, a display (e.g., a touch screen display), mouse, microphone, speaker, or the like. The user interface <NUM> may also produce outputs, for example, as visual outputs on a display, audio outputs via a speaker, or the like.

Referring now to the block diagram of <FIG>, a structural architecture of the autonomous vehicle <NUM> is provided. As mentioned above, the autonomous vehicle <NUM> may be an aerial or land-based drone configured to capture image data as part of a drone-based worksite survey. The autonomous vehicle <NUM> may comprise processing circuitry <NUM>, which may include memory <NUM>, processor <NUM>, user interface <NUM>, and communications interface <NUM>. The processing circuitry <NUM>, including the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM> may be structured the same or similar to the processing circuitry <NUM> with the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM>, respectively. However, the processing circuitry <NUM> may be configured to perform or control the functionalities of the autonomous vehicle <NUM> as described herein. In this regard, for example, the communications interface <NUM> of the processing circuitry <NUM> may be configured to establish a communications link with the worksite analysis engine <NUM> to provide the worksite analysis engine <NUM> with image data. According to some example embodiments, the image data may be provided via the communications interface <NUM> indirectly from the autonomous vehicle <NUM> to the worksite analysis engine <NUM> via for example a removable memory stick or jump drive.

In addition to the processing circuitry <NUM>, the autonomous vehicle <NUM> may also comprise a camera <NUM>, a position sensor <NUM>, and a propulsion and navigation unit <NUM>. The processing circuitry <NUM> may be configured to control the operation of the camera <NUM>, the position sensor <NUM>, and the propulsion and navigation unit <NUM>.

The camera <NUM> may be configured to capture images of a selected area around the autonomous vehicle <NUM>. In this regard, the camera <NUM> may be a digital imaging device configured to receive light to capture an image and convert the light into data representative of the light captured by the camera <NUM> as a component of image data as described herein.

According to some example embodiments, the camera <NUM> may be controlled by the processing circuitry <NUM> to capture images as requested by the processing circuitry <NUM>. In this regard, the processing circuitry <NUM> may be configured to cause images to be captured such that the images may be combined (e.g., overlapping images) to generate a larger image or model from the component captured images. The camera <NUM> may be stationary or moveable relative to the autonomous vehicle <NUM> to which the camera <NUM> is affixed. In example embodiments wherein the camera is stationary, the autonomous vehicle <NUM> may move into different physical positions to capture a desired image. Alternatively, if the camera <NUM> is moveable, the processing circuitry <NUM> may be configured to aim the camera <NUM> at a target area to capture an image using a motorized pivot or turret. Possibly with the assistance of the position sensor <NUM>, an angle of perspective (e.g., relative to the ground) may be stored in association with a captured image. In this regard, considering an autonomous vehicle <NUM> that is an aerial drone, the camera <NUM> may be configured to capture images at different perspectives (i.e., not simply overhead images aimed straight down). Such perspective images may be combined and leveraged to generate geospatial models that include topological data indicating terrain slopes and the like.

The position sensor <NUM> may be circuitry configured to determine a current position of the autonomous vehicle <NUM> and may generate position data indicative of the position of the autonomous vehicle <NUM>. The position of the autonomous vehicle <NUM> may be defined with respect to a coordinate system (e.g., latitude and longitude). Further, the position sensor <NUM> may be configured to determine an orientation of the autonomous vehicle <NUM> with respect to, for example, parameters such as pitch, roll, and yaw. The position and orientation of the autonomous vehicle <NUM> as determined by the position sensor <NUM> may be components of position data for the autonomous vehicle <NUM>. The position sensor <NUM> may, for example, include circuitry (including, for example, antennas) configured to capture wireless signals that may be used for determining a position of the position sensor <NUM> and the autonomous vehicle <NUM> based on the signals. In this regard, the position sensor <NUM> may be configured to receive global positioning system (GPS) signals to determine a position of the autonomous vehicle <NUM>. In this regard, according to some example embodiments, real-time kinematic (RTK) positioning may be employed to assist with correction of GPS positioning. Additionally, the receipt of wireless signals may also be leveraged to determine a position based on locating approaches such as received signal strength indication (RSSI), time-difference-of-arrival (TDOA), and the like. Additionally or alternatively, the position sensor <NUM> may be configured to determine a position of the autonomous vehicle <NUM> using locating techniques such as received signal strength, time of arrival, or the like.

Additionally, the autonomous vehicle <NUM> may include a propulsion and navigation unit <NUM>. The propulsion and navigation unit <NUM> may include the mechanisms and components configured to move the autonomous vehicle <NUM>. In this regard, in an example embodiment where the autonomous vehicle <NUM> is an aerial drone, the propulsion and navigation unit <NUM> may comprise motors and controllable rotors to fly and steer the drone. In an example embodiment where the autonomous vehicle <NUM> is a land-based drone, the propulsion and navigation unit <NUM> may comprise motorized wheels, tracks, or the like configured to assist with moving the drone on land. The propulsion and navigation unit <NUM> may also include the power source for powering the motors. The propulsion and navigation unit <NUM> may also include navigation circuitry configured to permit the processing circuitry <NUM> to steer the autonomous vehicle <NUM> into desired locations and positions.

Additionally, the autonomous vehicle <NUM> may include one or more sensors <NUM> which may take a variety of different forms. The sensor <NUM> may be configured to take one or more measurements of the worksite <NUM> under the control of the processing circuitry <NUM>. The measurement information may be coupled with position data to indicate a position or location within the worksite <NUM> where the measurement was taken. The measurement information gathered by the sensor(s) <NUM> may be provided to the worksite analysis engine <NUM> (e.g., possibly coupled with the respective position data) in the form of sensor data and integrated with the image data to be used as input component for the determinations made by the worksite analysis engine <NUM> or the sub-engines thereof.

In this regard, according to some example embodiments, the sensor <NUM> may be configured to gather additional information to assist with topographical mapping. The sensor <NUM> may be configured to use RADAR (radio azimuth direction and ranging), LiDAR (light detection and ranging), or the like to make measurements and capture information regarding, for example, changes in elevation and contours of the surface of the worksite <NUM> to be provided to the worksite analysis engine <NUM>.

According to some example embodiments, the sensor <NUM> may additionally or alternatively be configured to measure characteristics of the soil in the worksite <NUM> to be provided as sensor data. In this regard, the sensor <NUM> may be a type of imaging sensor that detects, for example, temperature variations (e.g., via infrared light) across the worksite <NUM>. Additionally, or alternatively, the sensor <NUM> may detect a hydration level in the soil at the worksite <NUM>. In some example embodiments, hydration levels may be detected via imaging techniques at certain electromagnetic wavelengths. However, according to some example embodiments, the sensor <NUM> may include a probe that may penetrate the surface of the worksite <NUM> (e.g., extend a desired depth into the soil) to take hydration measurements (e.g., at selected locations across the worksite <NUM>). Additionally or alternatively, such a sensor <NUM> may be configured to take other measurements of the soil, such as, for example, pH, color, compaction, organic content, texture, or the like.

Referring now to the block diagram of <FIG>, a structural architecture of the equipment transportation vehicle <NUM> is provided. As mentioned above, the equipment transportation vehicle <NUM> may be a truck, van, trailer, or the like that is configured to transport equipment to a worksite. The equipment transportation vehicle <NUM> may comprise processing circuitry <NUM>, which may include memory <NUM>, processor <NUM>, user interface <NUM>, and communications interface <NUM>. The processing circuitry <NUM>, including the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM>, may be structured the same or similar to the processing circuitry <NUM> with the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM>, respectively. However, the processing circuitry <NUM> may be configured to perform or control the functionalities of the equipment transportation vehicle <NUM> as described herein. In this regard, for example, the user interface <NUM> of the processing circuitry <NUM> may be configured to establish a communications link with the worksite analysis engine <NUM> to provide the worksite analysis engine <NUM> with data, such as, position data for the equipment transportation vehicle <NUM>.

In addition to the processing circuitry <NUM>, the equipment transportation vehicle <NUM> may also comprise a position sensor <NUM> and a propulsion and navigation unit <NUM>. The processing circuitry <NUM> may be configured to control the operation of the position sensor <NUM> and the propulsion and navigation unit <NUM>. In this regard, the position sensor <NUM> may be structured and configured in the same or similar manner as the position sensor <NUM>.

Additionally, the equipment transportation vehicle <NUM> may include a propulsion and navigation unit <NUM>. The propulsion and navigation unit <NUM> may include the mechanisms and components configured to move the equipment transportation vehicle <NUM>. In this regard, in an example embodiment, the propulsion and navigation unit <NUM> may comprise motorized wheels, tracks, or the like configured to assist with moving the equipment transportation vehicle <NUM>. In this regard, the propulsion and navigation unit <NUM> may include a user interface for driving the equipment transportation vehicle <NUM> by a crew member.

Referring now to the block diagram of <FIG>, a structural architecture of the equipment <NUM> is provided. Note that the other equipment in <FIG> (e.g., equipment <NUM>) may be structured similar to equipment <NUM> with the exception of the working unit <NUM>, but otherwise the block diagram architecture may be the same or similar. As mentioned above, the equipment <NUM> may be a tool or device that has utility in the context of the worksite <NUM>. According to some example embodiments, the equipment <NUM> may be vegetation maintenance equipment. In this regard, if vegetation maintenance is to be performed at the worksite <NUM>, the equipment <NUM> may be a ride-on or push mower, a trimmer, a blower, an aerator, a fertilizer spreader, a pruner, or the like. According to some example embodiments, the equipment <NUM> may comprise processing circuitry <NUM>, which may include memory <NUM>, processor <NUM>, user interface <NUM>, and communications interface <NUM>. The processing circuitry <NUM>, including the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM>, may be structured the same or similar to the processing circuitry <NUM> with the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM>, respectively. However, the processing circuitry <NUM> may be configured to perform or control the functionalities of the equipment <NUM> as described herein. In this regard, for example, the communications interface <NUM> of the processing circuitry <NUM> may be configured to establish a communications link with the worksite analysis engine <NUM> to provide the worksite analysis engine <NUM> with data, such as, position data for the equipment <NUM>.

In addition to the processing circuitry <NUM>, the equipment <NUM> may also comprise a position sensor <NUM>, an operation sensor <NUM>, a propulsion and navigation unit <NUM>, and working unit <NUM>. The processing circuitry <NUM> may be configured to control the operation of the position sensor <NUM>, operation sensor <NUM>, the propulsion and navigation unit <NUM>, and the working unit <NUM>. In this regard, the position sensor <NUM> may be structured and configured in the same or similar manner as the position sensor <NUM>. However, the position sensor <NUM> may be configured to generate position data for the equipment <NUM>.

The operation sensor <NUM> may be a single sensor or a plurality of sensors that monitor and log data regarding the operation of the equipment <NUM>. In this regard, the operation sensor <NUM> may be configured to monitor and log rotation per minute (RPM) data, fuel quantity and utilization data, gear usage data (e.g., high gear, low gear, reverse), idle time data, and the like. Such data may be collectively referred to as equipment operation data. According to some example embodiments, the equipment operation data may be communicated to the worksite analysis engine <NUM> for use in compliance analyses by the workflow compliance engine <NUM>.

Additionally, the equipment <NUM> may include a propulsion and navigation unit <NUM>. The propulsion and navigation unit <NUM> may include the mechanisms and components configured to move the equipment <NUM>. In this regard, in an example embodiment, the propulsion and navigation unit <NUM> may comprise motorized wheels, tracks, or the like configured to assist with moving the equipment <NUM>. The propulsion and navigation unit <NUM> may operably couple with the user interface <NUM> for driving the equipment transportation vehicle <NUM> by a crew member. According to some example embodiments, the equipment <NUM> may include a display <NUM>, which may be, for example, an LCD display. According to some example embodiments, information may be provided to a crew member operating the equipment <NUM> via the display <NUM>. Such information may be rendered by the processing circuitry <NUM> on the display <NUM> in the form of, for example, a determined equipment path for the operator/crew member to follow when using the equipment <NUM> at the worksite <NUM>.

The equipment <NUM> may also include a working unit <NUM>. The working unit <NUM> may be the component or components of the equipment <NUM> that perform a work action (e.g., cutting, blowing, aerating, spraying, or the like). In this regard, for example, if the equipment <NUM> is a ride-on lawn mower, the working unit <NUM> may comprise cutting blades and a deck for mowing turf and the associated control and power systems. If the equipment <NUM> is a blower, the working unit <NUM> may comprise a fan, an air-directing nozzle, and the associated control and power systems to support operation of the fan.

Referring now to the block diagram of <FIG>, a structural architecture of crew device <NUM> is provided. Note that the other crew devices in <FIG> (e.g., crew device <NUM>) may be the same or similar to crew device <NUM>. The crew device <NUM> may be a device that worn or carried by a crew member and is configured to track a position of the crew member. Additionally, according to some example embodiments, the crew device <NUM> may be configured to communicate with or read a tag on a piece of equipment (e.g., equipment <NUM>) to determine a proximity of the equipment and determine that the crew member is operating the equipment. As such, the crew device <NUM> may clip to a crew member's belt, be affixed to a lanyard, or the like.

The crew device <NUM> may comprise processing circuitry <NUM>, which may include memory <NUM>, processor <NUM>, user interface <NUM>, and communications interface <NUM>. The processing circuitry <NUM>, including the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM>, may be structured the same or similar to the processing circuitry <NUM> with the memory <NUM>, the processor <NUM>, the user interface <NUM>, and the communications interface <NUM>, respectively. However, the processing circuitry <NUM> may be configured to perform or control the functionalities of the crew device <NUM> as described herein. In this regard, for example, the user interface <NUM> of the processing circuitry <NUM> may be configured to establish a communications link with the worksite analysis engine <NUM> to provide the worksite analysis engine <NUM> with data, such as, position data for the crew device <NUM> and the associated crew member.

In addition to the processing circuitry <NUM>, the crew device <NUM> may also comprise a position sensor <NUM>. The processing circuitry <NUM> may be configured to control the operation of the position sensor <NUM>. In this regard, the position sensor <NUM> may be structured and configured in the same or similar manner as the position sensor <NUM>. However, the position sensor <NUM> may be configured to generate position data for crew device <NUM> and the associated crew member.

Having described the structures of the components of the example system <NUM>, the following provides as description of the functionalities that may be employed by the components of the system <NUM> while referring to <FIG>. In this regard, the autonomous vehicle <NUM> may be deployed near a worksite <NUM> and may be configured to operate the camera <NUM> and the position sensor <NUM> to capture images of the worksite <NUM> in association with corresponding position coordinates. The propulsion and navigation unit <NUM> of the autonomous vehicle <NUM> may be configured, via the processing circuitry <NUM>, to maneuver into positions to capture images to obtain a comprehensive survey of the worksite <NUM>. In this regard, the autonomous vehicle <NUM> may be configured to capture overlapping images to facilitate matching of the edges of the images by the worksite analysis engine <NUM> and more specifically the virtual layout generation engine <NUM> of the worksite analysis engine <NUM> to generate a virtual layout as further described below. Additionally, the position data corresponding to each of the captured images may also be used to match content of the images when building the virtual layout of the worksite <NUM>.

According to some example embodiments, the autonomous vehicle <NUM> may be configured to capture images of the same space from different perspective angles. By capturing the images in this manner three-dimensional information may be extracted from the collection of images to determine the size, shape, and placement of objects, other items of interest, and the spatial geography of the items of interest by the virtual layout generation engine <NUM>. Further, topology data may be determined indicating slopes within the landscape of the worksite <NUM> based on the perspective angles of the captured images.

As such, whether on land or through the air, the autonomous vehicle <NUM> may navigate the worksite <NUM> to collect image data comprising images of the worksite <NUM> with corresponding position coordinates (e.g., a form of position data) for the images. Further, according to some example embodiments, the position coordinates may include orientation coordinates indicating pitch, roll, and yaw, as well as altitude, to be able to define a perspective and perspective angles for the images captured. Additionally, according to some example embodiments, the autonomous vehicle <NUM> may also collect sensor data (e.g., captured by sensor(s) <NUM>). According to some example embodiments, the image data and/or the sensor data may be provided by the autonomous vehicle <NUM> for receipt by the worksite analysis engine <NUM>. In this regard, the autonomous vehicle <NUM> may be configured to wirelessly transmit the image data and/or the sensor data via a network to the worksite analysis engine <NUM> or, according to some example embodiments, the autonomous vehicle <NUM> may be configured to store the image data and/or sensor data on, for example, a removable memory (e.g., memory <NUM> or a component thereof) that may be delivered to the worksite analysis engine <NUM> for upload.

As mentioned above, the worksite analysis engine <NUM> may be configured to generate a virtual layout of the worksite <NUM> based on various data (e.g., image data and sensor data) and generate workflows to optimize maintenance work at the worksite <NUM> based on the virtual layout, possibly in combination with other data retrieved by the worksite analysis engine <NUM>. In this regard, according to some example embodiments, the worksite analysis engine <NUM> may be configured to generate the virtual layout via the processing circuitry <NUM>.

In this regard, the virtual layout generation engine <NUM> may be configured to receive data and generate the virtual layout of the worksite <NUM> based on the received data. According to some example embodiments, the received data may include image data and/or sensor data captured by the autonomous vehicle <NUM>. Additionally or alternatively, the received data may include geographic data received from the GIS database <NUM>. In this regard, the GIS database <NUM> may be, for example, a government maintained database of property records indicating surveyed meets and bounds of property plots and associated satellite imagery. Additionally or alternatively, the GIS database <NUM> may be a commercial database (e.g., a real estate business database) that includes property boundary lines and satellite imagery. According to some example embodiments, the GIS database <NUM> may include satellite imagery that may be received by the virtual layout generation engine <NUM> for use in developing the virtual layout. Further, the virtual layout generation engine <NUM> may also receive data from a topology database <NUM>. Again, the topology database <NUM> may be a government or commercial database indicated property elevations and topographic contours. The topology database <NUM> may include data provided as satellite topography.

Accordingly, the virtual layout generation engine <NUM> may be configured to generate a virtual layout in the form of a geospatial model of the worksite <NUM> based on one or more of the image data, sensor data, data from the GIS database <NUM>, or data from the topology database <NUM>. With respect to the image data, the virtual layout generation engine <NUM> may be configured to match edges of the captured images using the content of the images and the corresponding position data to generate the virtual layout in the form of a three-dimensional geospatial model. The virtual layout generation engine <NUM> may include functionality to identify and classify areas and objects within the virtual layout. To do so, the virtual layout generation engine <NUM> may evaluate colors, textures, and color and texture transitions within, for example, the image data to identify objects and area boundaries against a comparison object database.

In this regard, the virtual layout generation engine <NUM> may be configured to identify and classify lawn or turf areas and define boundaries for the lawn or turf areas. Further, the virtual layout generation engine <NUM> may be configured to identify and classify planting beds and define boundaries for the planting beds. Further, the virtual layout generation engine <NUM> may be configured to identify and classify structures (e.g., houses, buildings, fences, decks, etc.) and define boundaries for the structures. Additionally, the virtual layout generation engine <NUM> may be configured to identify and classify pavement areas (e.g., roads, driveways, sidewalks, etc.) and define boundaries for the pavement areas. Also, with respect to vegetation, the virtual layout generation engine <NUM> may also be configured receive vegetation data and analyze coloration and shapes of, for example, leaves and other vegetation characteristics to identify and classify the types of vegetation (e.g., trees, bushes, turf, annuals, etc.) on the worksite <NUM> based on the received vegetation data and indicate the placement of the vegetation within virtual layout.

According to some example embodiments, the virtual layout generation engine <NUM> may also consider human survey information that may be provided to the virtual layout generation engine <NUM> relating to the worksite <NUM>. The human survey information may indicate spatial information such as the placement of planting beds, structures, pavement areas, and the like. The human survey information may also indicate vegetation types and locations within the worksite <NUM>. According to some example embodiments, the human survey information may be entered into a separate terminal or directly into the worksite analysis engine <NUM> to be received via the communications interface <NUM> or the user interface <NUM>, respectively.

Accordingly, the virtual layout may be formed as a geospatial model comprising the topography of the worksite <NUM> that can be analyzed to assist with equipment path determinations and workflow generation as further described herein. In this regard, the virtual layout may be used to determine distances between the identified and classified objects. As such, the virtual layout may provide a digital representation of the physical worksite <NUM> at the time that the images used to generate the virtual layout were captured.

According to some example embodiments, the virtual layout may also be generated based on historical virtual layouts for the worksite <NUM>. In this regard, according to some example embodiments, a virtual layout may include a temporal element and the virtual layout may describe the state of the worksite <NUM> over time. In this regard, snapshot or time captured virtual layouts may be combined to identify changes that have occurred at the worksite <NUM>. For example, a virtual layout that incorporates historical information may indicate vegetation growth (e.g., tree growth or turf growth). Additionally, such a virtual layout may show differences in the landscape of the worksite <NUM> due to, for example, erosion or degradation of ground cover (e.g., degradation of mulch). Further, the virtual layout may also show differences due to the presence of movable objects such as debris or toys that may be moveable prior to performing worksite maintenance.

As mentioned above, the worksite analysis engine <NUM> may also include an equipment path generation engine <NUM>. In this regard, the equipment path generation engine <NUM> may be configured to analyze the virtual layout in combination with other data to determine an efficient and effective equipment path for performing a worksite maintenance task. Data in addition to the virtual layout that may be evaluated and incorporated when determining an equipment path. Such data may include equipment data and crew data. According to some example embodiments, the equipment path may be defined as a direction or pattern of movement for equipment use in an area. However, in some example embodiments, the equipment path may indicate a specific route indicating exact positions for the equipment as the equipment is utilized to complete a task.

The equipment data that may be used to generate an equipment path may include a list of equipment available to be deployed at the worksite <NUM>. Such a list may be an inventory list of the equipment that is present on the equipment transportation vehicle <NUM>. The equipment data may also include equipment attributes for the equipment on the inventory list. Such attributes may indicate, for example, for a ride-on mower, turning radius, deck width, deck height, maximum slope, speed, clipping catch capacity, and the like. For such a ride-on mower, as well as other equipment, the equipment attributes may also include fuel capacity, fuel consumption rate, equipment category (e.g., wheeled, wheeled-motorized, ride-on, hand-carry, or the like), and a work unit action (e.g., mow, trim, blow, aerate, spread fertilizer, hedge trim, saw, or the like).

The crew data may indicate a number of available crew members that may be utilized at the worksite <NUM>. Crew data may also indicate certain qualifications or experience of the individual crew member. For example, the crew data may indicate equipment that a crew member is qualified to use or that the crew member has proven to have a relatively high effectiveness using. Further, the crew data may indicate a classification or rank of a crew member as, for example, a supervisor, a senior crew member, a junior crew member, or the like.

Accordingly, based on the virtual layout and in some instances, the equipment data and the crew data, an equipment path may be generated by the equipment path generation engine <NUM>, via the processing circuitry <NUM>, as an efficient and effective path for implementing selected equipment within the worksite <NUM>. Further, the equipment path generation engine <NUM> may be configured to generate the equipment path based on the virtual layout, where the virtual layout includes topographic information for analysis in determining the equipment path. Additionally or alternatively, according to some example embodiments, the equipment path may also be based on desired path parameters, such as, for example, a desired striping pattern (e.g., a user-defined striping pattern) for the turf, a desired hedge height or the like. Additionally or alternatively, the equipment path may be generated based on recent weather data. Such weather data may comprise precipitation data and sun exposure data. In this regard, the weather data may, for example, indicate that there has been little precipitation and high sun exposure, and therefore only the shaded areas within the worksite <NUM> may require mowing and the equipment path may be generated accordingly. Further, for example, the weather data may indicate that substantial precipitation and low sun exposure has occurred recently and therefore low areas of the worksite <NUM> may be removed from the equipment path for a ride-on mower to prevent ruts in the turf. Additionally or alternatively, the equipment path generation engine <NUM> may be configured to generate the equipment path based on the virtual layout and work zones defined within the worksite <NUM>, as further described below. In this regard, for example, the equipment path may be generated for work within a particular work zone, and thus, the equipment path may be, in some instances, limited to routing the crew member and the associated equipment within the work zone.

If, for example, the equipment is a ride-on mower, the equipment path may indicate the path that the mower should move from the equipment transportation vehicle <NUM> to the worksite <NUM>, through the worksite <NUM> to perform mowing, and return to the equipment transportation vehicle <NUM>. The equipment path may be determined based on the equipment data to determine areas from the virtual layout where, for example, a ride-on mower may not have access because of sloped terrain, a small gate, an area being smaller than the deck width, turning radius limitations, or the like. Similarly, for example, if the equipment is a trimmer, the equipment path generation engine <NUM> may indicate a path that a crew member may move from the equipment transportation vehicle <NUM> to each area that needs to be trimmed and return to the equipment transportation vehicle <NUM>. According to some example embodiments, some equipment paths may be dependent upon other equipment paths or the capabilities of other equipment. In this regard, the equipment path for the trimmer may be dependent upon the accessibility of the ride-on mower to all areas of the worksite <NUM>, and there may be areas that are not accessible to the ride-on mower, and therefore the equipment path for the trimmer may include some or all of those areas that are not accessible to the ride-on mower. Further, according to some example embodiments, the equipment path may also be based on a requirement to return to a location during completion of a task. In this regard, for example, if mowing is being performed such that yard clippings are collected and removed, then the equipment path may be defined to return to the equipment transportation vehicle <NUM> to empty the clipping catch at an efficient point in the equipment path based on, for example, the clipping catch capacity of the equipment.

According to some example embodiments, the equipment path may be provided (e.g., transmitted or otherwise delivered) to, for example, the equipment <NUM>. Upon receiving the equipment path generated by the equipment path generation engine <NUM>, the equipment <NUM> may be configured to store the equipment path in the memory (e.g., memory <NUM>) of the equipment <NUM>. When the crew member is prepared to undertake the task associated with the equipment <NUM> (e.g., mow the turf portions of the worksite <NUM> or trim determined areas of the worksite <NUM>), the crew member may retrieve the equipment path for output via the user interface <NUM>, or, more specifically, via a display of the user interface <NUM>. As such, the equipment path may be output to the crew member to enable the crew member to follow the determined equipment path during execution of the task.

According to some example embodiments, the worksite analysis engine <NUM> may also be configured to implement a crew workflow generation engine <NUM>. In this regard, the crew workflow generation engine <NUM> may be configured to generate a workflow for the crew members servicing the worksite <NUM>. The workflow may comprise a list (e.g., a sequenced list) of workflow assignments to be performed by a crew member when servicing the worksite <NUM>. A workflow assignment may comprise a task, equipment to perform the task, and an equipment path (as described above) for performing the task. In this regard, for example, a workflow assignment may include a task of mowing, equipment for the task may be a ride-on mower, and the equipment path may be defined as provided by the equipment path generation engine <NUM>. Additionally, according to some example embodiments, a workflow assignment may also indicate a work zone for the task.

As mentioned above, the crew workflow generation engine <NUM> may be configured to analyze the virtual layout to determine work zones within the worksite <NUM>. To determine a work zone, the crew workflow generation engine <NUM> may be configured to determine sub-boundaries within the worksite <NUM> where, for example, topology changes (e.g., areas with increased or decreased slope), access changes (e.g., a fenced in area), pavement boundaries, worksite boundaries, or the like. Work zones may also be defined based on the equipment needed to service, for example, the vegetation within the work zone. For example, a work zone may be defined by an area that has a steep grade because a ride-on mower may not be able to mow the area and a push mower may be needed to mow that area. In another example, a work zone may be defined in association with a densely treed area where only a trimmer can be used to maintain grasses that may grow in such an area. The crew workflow generation engine <NUM> may therefore define the work zones as piece-wise geographic regions within the worksite <NUM>. As such, for example, boundaries of the work zones may be determined based on physical changes indicated in the virtual layout (e.g., a change from turf to pavement), a need for a different piece of equipment to maintain the area, or the like.

Whether the workflow is defined with or without work zones, the workflow may be a maintenance execution plan for each member to complete, for example, in unison upon beginning the maintenance effort at a worksite <NUM>. The workflow and the workflow assignments therein may be determined based on the virtual layout, the equipment data, and the crew data. Additionally, the workflow and the workflow assignments therein may, according to some example embodiments, be based on the defined work zones for the worksite <NUM>. Additionally, the workflow and the workflow assignments therein may also be based on the weather data (e.g., including precipitation data, sun exposure data, or the like) as described above, or sensor data. According to some example embodiments, the workflow and the workflow assignment therein may be defined based on safety criteria such that crew members may be located, for example, in different work zones at the same time to reduce interaction that increases the likelihood of a safety incident. As mentioned above, the equipment selected for a task within the workflow may be determined based on the type of task and the type of, for example, vegetation being maintained.

Additionally, for example, a mower provided on the equipment list of the equipment data may be selected for use when maintaining turf. However, according to some example embodiments, if the task could be completed more efficiently by a piece of equipment that is not on the equipment list, the crew workflow generation engine <NUM> may be configured to recommend purchase of a new piece of equipment, based on the equipment data and the virtual layout, that could more efficiently complete the task. Such information regarding equipment that is not in the equipment list may be retrieved, for example, from other sources of information such as websites and databases of equipment information provided by equipment sellers. According to some example embodiments, the crew workflow generation engine <NUM> may be configured to determine an efficiency payback associated with the purchase of the new equipment that indicates when use of the new equipment at the worksite <NUM> (and elsewhere) may increase profits due to the efficiency increase resulting in payback in the amount of the purchase price over a determined period of time.

According to some example embodiments, the crew workflow generation engine <NUM> may also analyze the virtual layout to determine an efficient location to park the equipment transportation vehicle <NUM>. The determination of the location of the equipment transportation vehicle <NUM> may also be a factor when generating equipment paths as described above. According to some example embodiments, the determined location of the equipment transportation vehicle <NUM> may be a location that minimizes travel distances of equipment to the worksite <NUM>. As such, the workflow assignment and tasks of the workflow may also be factors evaluated by the crew workflow generation engine <NUM> when determining a location for the equipment transportation vehicle <NUM> and for the generation of equipment paths.

Additionally, the worksite analysis engine <NUM> may also include a workflow compliance engine <NUM>. The workflow compliance engine <NUM> may be configured to evaluate actual execution of the workflow by the crew to determine compliance with the workflow. In this regard, according to some example embodiments, a workflow compliance score may be calculated based on the crew's execution of the workflow.

Workflow compliance may be performed based on tracked data (e.g., equipment operation data and equipment position data) regarding the utilization and location of the equipment by the crew with respect to the workflow. To track the actual activities of the crew, the workflow compliance engine <NUM> may receive position data from the equipment position sensor <NUM> and the crew device position sensor <NUM>. Additionally, the workflow compliance engine <NUM> may collect data regarding operation of the equipment from data captured by the operation sensor <NUM> of the equipment <NUM>.

Based on the position data and operation data captured by the equipment <NUM> and the crew device <NUM> and received by the workflow compliance engine <NUM>, workflow compliance analyses may be performed, for example, with respect to the determined equipment path indicated in the workflow. In this regard, equipment position data captured by the equipment <NUM> may be compared to the generated equipment path to determined differences between the actual path taken and the proposed equipment path. Such differences may be a factor in a compliance score. Additionally, compliance analysis may also be performed with respect to the type of equipment being used for a task within the workflow. For example, the workflow may indicate that a push mower is to be used for mowing a particular work zone, but the operation data and the position data of the ride-on mower may indicate that the push mower was not used and the ride-on mower was used, which would be out of compliance with the workflow.

Having described various aspects of some example embodiments, the following describes an example implementation of the system <NUM> in the context of an example worksite <NUM> that is a residential worksite for vegetation maintenance. In this regard, with reference to <FIG>, an overhead view of a worksite <NUM> is shown. Image data of the worksite <NUM> may be captured by the autonomous vehicle <NUM> as indicated by image captures <NUM> across the entirety of the worksite <NUM>. While <FIG> shows images captured in a two dimensional plane above the worksite <NUM>, it is understood that the autonomous vehicle <NUM> may be configured to capture image data at a number of different perspectives to facilitate generation of a virtual layout of the worksite <NUM> in three dimensions as a geospatial model that includes topographic information.

Now referring to <FIG>, the worksite <NUM> is shown as an example virtual layout that may be generated by the virtual layout generation engine <NUM>. In this regard, a worksite boundary <NUM> may be generated to define the extents of the worksite <NUM>, for example, based on GIS data or the like as described herein. Additionally, the virtual layout includes areas identified and classified as planting beds <NUM>, which may include plants, shrubs, trees, or the like. Additionally, the virtual layout includes an area identified and classified as a structure <NUM> in the form of the house. Further, the virtual layout includes an area identified and classified as pavement <NUM> which includes the areas of the driveway and the sidewalk. The virtual layout also includes contour lines <NUM> indicating sloped areas of the worksite <NUM> that have been determined based on topographic data.

Now referring to <FIG>, equipment path generation engine <NUM> has analyzed the virtual layout with equipment data and determined equipment paths. In this regard, the equipment paths may be determined for different areas of the worksite <NUM> based on, for example, the type of equipment to be used and the topography of the area. In this example scenario, the equipment paths <NUM>, <NUM>, <NUM>, and <NUM> are defined. The equipment paths <NUM>, <NUM>, <NUM>, and <NUM> may be defined directions or patterns of movement for use by a crew member operating, fro example, a ride-on mower in accordance with the equipment paths <NUM>, <NUM>, <NUM>, and <NUM>. Alternatively, <FIG> illustrates a more specifically defined equipment path <NUM>. In this regard, the equipment path <NUM> may also be for a ride-on mower, but the equipment path <NUM> indicates the exact location for movement of the ride-on mower throughout the mowing task. Additionally, the location of an equipment transportation vehicle <NUM> is shown. In this regard, the crew workflow generation engine <NUM> may have analyzed the virtual layout and determined an efficient location for parking the equipment transportation vehicle <NUM> for beginning and ending the equipment path for the task of mowing using a ride-on mower, as well as other tasks in the workflow.

As shown in <FIG>, the worksite <NUM> may be divided by the crew workflow generation engine <NUM> into a plurality of work zones. In this regard, the work zones <NUM>, <NUM>, <NUM>, and <NUM> have been defined, in addition to a work zone associated with the paved area <NUM>. As can be seen, the work zones have been defined with boundaries based on the boundaries of the worksite <NUM> and pavement boundaries in some instances. The boundaries between work zone <NUM> and <NUM>, and work zone <NUM> and <NUM> may be based on, for example, the presence of a structure in the form of a fence.

Additionally, as described above with respect to the work zones, equipment paths may be defined within the context of the work zones individually, as shown in <FIG>. In this regard, equipment paths <NUM>, <NUM>, and <NUM> may be defined within each of the work zones <NUM>, <NUM>, and <NUM>, respectively, as directions or patterns of movement, for example, for a ride-on mower completing the task of mowing within each of the work zones <NUM>, <NUM>, and <NUM>. However, in an example scenario, due to the slope of the terrain in work zone <NUM>, a push mower is designated as the equipment for completing the task of mowing in the work zone <NUM> in accordance with the equipment path <NUM>.

Based on the work zones <NUM>, <NUM>, <NUM>, and <NUM> defined in <FIG> and <FIG>, an example workflow may be generated by the crew workflow generation engine <NUM> as provided in Table <NUM> below. The example workflow of Table <NUM> includes work assignments described with respect to <FIG>.

As shown in the workflow of Table <NUM>, the crew workflow generation engine <NUM> has generated a workflow for the worksite <NUM> using two crew members (i.e., crew member <NUM> and crew member <NUM>). The work assignments in the same row are scheduled to be performed at the same time and are planned to require a similar amount of time to complete. As shown in the Table <NUM>, each workflow assignment within the workflow may be defined by a task, equipment, work zone, and equipment path.

With reference to <FIG>, the equipment path <NUM> for workflow assignment 1a is shown. Additionally, in <FIG>, the crew workflow generation engine <NUM> has also determined an efficient location of for parking the equipment transportation vehicle <NUM>, as shown. Again with respect to workflow assignment 1a, crew member <NUM> is assigned to a task of mowing with a clipping catch using the equipment being a ride-on mower in work zone <NUM> using equipment path <NUM>. As shown in <FIG>, the equipment path <NUM> begins and ends at the equipment transportation vehicle <NUM> to provide for emptying the clipping catch at the equipment transportation vehicle <NUM>. Meanwhile, crew member <NUM> is assigned workflow assignment 1b (to be performed at the same time as workflow assignment 1a) of trimming, using the trimmer, in work zone <NUM>. Notably, crew member <NUM> and crew member <NUM> are not assigned to work in the same work zone at the same time for safety purposes. While the equipment path generation engine <NUM> may have generated an equipment path for trimming, in this example workflow the equipment path for the trimming tasks are not shown.

Subsequently, and now referring to <FIG>, crew member <NUM> is assigned to workflow assignment 2a, which is to mow with a clipping catch using the ride-on mower in work zone <NUM> using equipment path <NUM>. As shown in <FIG>, the equipment path <NUM> again begins and ends at the equipment transportation vehicle <NUM> to provide for emptying the clipping catch at the equipment transportation vehicle <NUM>. Meanwhile, crew member <NUM> is assigned workflow assignment 2b (to be performed at the same time as workflow assignment 2a) of trimming, using the trimmer, in work zone <NUM>.

Now referring to <FIG>, crew member <NUM> is assigned to workflow assignment 3a, which is to mow with a clipping catch using the ride-on mower in work zone <NUM> using equipment path <NUM>. As shown in <FIG>, the equipment path <NUM> again begins and ends at the equipment transportation vehicle <NUM> to provide for emptying the clipping catch at the equipment transportation vehicle <NUM>. Meanwhile, crew member <NUM> is assigned workflow assignment 3b (to be performed at the same time as workflow assignment 3a) of trimming, using the trimmer, in work zone <NUM>.

Now referring to <FIG>, crew member <NUM> is assigned to workflow assignment 4a, which is to mow with a clipping catch using the push mower in work zone <NUM> using equipment path <NUM>. As shown in <FIG>, the equipment path <NUM> again begins and ends at the equipment transportation vehicle <NUM> to provide for emptying the clipping catch at the equipment transportation vehicle <NUM>. Meanwhile, crew member <NUM> is assigned workflow assignment 4b (to be performed at the same time as workflow assignment 4a) of trimming, using the trimmer, in work zone <NUM>.

Now referring to <FIG>, crew member <NUM> is assigned to workflow assignment 5a, which is to blow using the blower in the pavement work zone defined at <NUM> using equipment path <NUM>. As shown in <FIG>, the equipment path <NUM> again begins and ends at the equipment transportation vehicle <NUM> to provide for removing and returning the blower to the equipment transportation vehicle <NUM>. Meanwhile, crew member <NUM> is assigned workflow assignment 5b (to be performed at the same time as workflow assignment 5a) of pruning, using the pruners, in work zone <NUM>.

Now with reference to the flow chart of <FIG>, an example method for generating a workflow is provided in accordance with some example embodiments. In this regard, the example method may include, at <NUM>, capturing image data associated with a worksite, where the image data is captured by an autonomous vehicle (e.g., autonomous vehicle <NUM>) comprising a camera and a position sensor. The autonomous vehicle may be configured to operate the camera and position sensor to capture the image data with corresponding position coordinates. According to some example embodiments, sensor data may also be measured and otherwise captured by the autonomous vehicle. The example method may further include, at <NUM>, receiving the image data (and in some cases sensor data) of the worksite captured by the autonomous vehicle by processing circuitry (e.g., processing circuitry <NUM>) of a worksite analysis engine. Additionally, at <NUM>, the example method may include generating a virtual layout of the worksite based on the image data (and in some cases sensor data), by the processing circuitry. The example method may also include, at <NUM>, receiving equipment data comprising a list of equipment available to be deployed at the worksite with corresponding equipment attributes, and at <NUM>, receiving crew data comprising a number of crew members available to be deployed at the worksite. Further, at <NUM>, the example method may include generating a workflow based on the virtual layout, the equipment data, and the crew data. In this regard, the workflow may comprise workflow assignments for each crew member at the worksite, and each workflow assignment may indicate a task, equipment to perform the task, and an equipment path for the task.

According to some example embodiments, the image data may include perspective angles corresponding to the images captured, and the example method may further comprise generating the virtual layout as a geospatial model of the worksite including topographic data based on the image data comprising the perspective angles. Additionally, the example method may comprise generating the equipment path based on the virtual layout comprising the topographic data.

Further, according to some example embodiments, the example method may, additionally or alternatively comprise determining a plurality of work zones within the worksite based on the virtual layout, the equipment data, and the crew data, and generating the workflow based on the work zones. In this regard, each workflow assignment may also indicate a work zone for a task. Additionally or alternatively, the example method may further comprise generating the equipment path based on the plurality of work zones. Additionally or alternatively, the equipment attributes for the equipment data may include information indicating a deck width and a turn radius. Additionally or alternatively, the example method may comprise generating the virtual layout based on vegetation data indicating types of vegetation within the worksite. Additionally or alternatively, the example method may further comprise generating the workflow based on weather data comprising precipitation data and sun exposure data, or sensor data. Additionally or alternatively, the example method may further comprise generating the virtual layout based on historical image data. In this regard, the example method may further comprise identifying moveable objects within the virtual layout based on differences between the historical image data and the image data captured by the autonomous vehicle.

Claim 1:
A system comprising:
an autonomous vehicle (<NUM>) comprising a camera (<NUM>) and a position sensor (<NUM>), the autonomous vehicle (<NUM>) being configured to operate the camera (<NUM>) and position sensor (<NUM>) to capture image data associated with a worksite (<NUM>), the image data comprising images of the worksite (<NUM>) with corresponding position coordinates;
and
the worksite analysis engine (<NUM>) comprising processing circuitry (<NUM>), the processing circuitry (<NUM>) being configured to:
receive the image data of the worksite (<NUM>) captured by the autonomous vehicle (<NUM>);
generate a virtual layout of the worksite (<NUM>) based on the image data; receive equipment data comprising a list of equipment available to be deployed at the worksite (<NUM>) with corresponding equipment attributes; receive crew data comprising a number of crew members available to be deployed at the worksite (<NUM>);
generate a workflow based on the virtual layout, the equipment data, and the crew data, the workflow comprising workflow assignments for each crew member at the worksite (<NUM>), each workflow assignment indicating a task, equipment to perform the task, and an equipment path for the task;
characterized in that the system further comprises: at least one equipment (<NUM>, <NUM>) comprising processing circuitry (<NUM>) configured to establish a communications link with a worksite analysis engine (<NUM>) of the system to provide the worksite analysis engine (<NUM>) with position data and operation data for the equipment (<NUM>, <NUM>);
a crew device (<NUM>, <NUM>) configured to track position of a crew member and configured to communicate with or read a tag on the equipment (<NUM>, <NUM>) to determine a proximity of the equipment (<NUM>, <NUM>) and to determine that the crew member is operating the equipment (<NUM>, <NUM>) and comprising processing circuitry (<NUM>) configured to establish a communications link with the worksite analysis engine (<NUM>) to provide the worksite analysis engine (<NUM>) with position data of the crew device (<NUM>, <NUM>);
and wherein the processing circuitry (<NUM>) is further configured to:
evaluate actual execution of the workflow by the crew based on the position data and operation data captured by the at least one equipment (<NUM>, <NUM>) and the at least one crew device (<NUM>, <NUM>).