SYSTEMS AND METHODS FOR SURVEYING A MANUFACTURING ENVIRONMENT

A method for surveying an environment includes segmenting a plurality of nodes into a plurality of communities. The method includes identifying a plurality of community centroids based on the plurality of communities, where each community centroid from among the plurality of community centroids is associated with one community from among the plurality of communities. The method includes determining, for each community centroid from among the plurality of community centroids, a community type of the community centroid, where the community type is one of a primary type and an auxiliary type, and where the community type is based on a distance between the community centroid and one or more charging stations. The method includes generating a plurality of intercommunity routes based on the plurality of community centroids, the one or more charging stations, the community type, and one or more performance constraints.

FIELD

The present disclosure relates to systems and methods for surveying a manufacturing environment.

BACKGROUND

A digital twin provides a virtual model of components and/or existing conditions of an environment. To generate the digital twin, an operator may traverse the environment with sensing systems (e.g., cameras and/or light detection and ranging (LIDAR) sensors) to capture scene information at each location of the environment. Once captured, the operator processes the data to stitch together individual areas to construct the digital twin. As another example, robotic systems equipped with sensing systems may traverse the environment to capture the scene information.

However, dynamic changes to the layout and conditions of the environment may compromise the accuracy of the digital model. Furthermore, obstacles of the environment may inhibit the robot from traversing the environment and accurately collecting the scene information to generate the digital twin. These issues with generating digital twins, among other issues, are addressed by the present disclosure.

SUMMARY

A method for surveying an environment includes segmenting a plurality of nodes into a plurality of communities, where each node from among the plurality of nodes corresponds to a location of the environment, and where each community from among the plurality of communities includes a set of nodes from among the plurality of nodes. The method includes identifying a plurality of community centroids based on the plurality of communities, where each community centroid from among the plurality of community centroids is associated with one community from among the plurality of communities. The method includes determining, for each community centroid from among the plurality of community centroids, a community type of the community centroid, where the community type is one of a primary type and an auxiliary type, and where the community type is based on a distance between the community centroid and one or more charging stations. The method includes generating a plurality of intercommunity routes based on the plurality of community centroids, the one or more charging stations, the community type, and one or more performance constraints.

In one form, the method further includes segmenting the plurality of nodes into the plurality of community centroids and the plurality of communities based on an asynchronous label propagation routine. In one form, the method further includes connecting the plurality of nodes via a plurality of edges based on one or more edge constraints. In one form, the one or more edge constraints are based on a threshold travel distance, a radial distance associated with each node of the plurality of nodes, and one or more traversability scores indicating a traversability of one or more locations of the environment corresponding to the plurality of nodes. In one form, the method further includes segmenting the plurality of nodes into the plurality of communities based on the plurality of edges.

In one form, the method further includes selecting a first charging station and a second charging station from among the one or more charging stations, determining a first distance between the first charging station and the community centroid, and determining a second distance between the second charging station and the community centroid, where the community type of the community centroid is further based on a ratio between the first distance and the second distance. In one form, the method further includes determining one or more additional distances between the community centroid and one or more additional charging stations from among the one or more charging stations and selecting the first charging station and the second charging station when the one or more additional distances are greater than the first distance and the second distance.

In one form, generating the plurality of intercommunity routes based on the plurality of community centroids, the one or more charging stations, the community type, and the one or more performance constraints further includes: selecting a first charging station from among the one or more charging stations, selecting a first set of community centroids from among the plurality of community centroids, where each community centroid from among the first set of community centroids is the primary type, and where a distance between the first set of community centroids and the first charging station is less than a distance between the first set of community centroids and a second charging station from among the one or more charging stations; selecting a first community centroid from among the first set of community centroids, where a distance between the first community centroid and the first charging station is greater than a distance between each remaining community centroid of the first set of community centroids and the first charging station; and assigning the first community centroid to a first intercommunity route from among the plurality of intercommunity routes when an energy consumption of the first community centroid and an intracommunity time of the first community centroid satisfy the one or more performance constraints. In one form, the method further includes selecting a second community centroid from among the first set of community centroids, assigning the second community centroid to the first intercommunity route when an energy consumption of the second community centroid, an intracommunity time of the second community centroid, and an intercommunity time of the first community centroid and the second community centroid satisfy the one or more performance constraints, and assigning the second community centroid to a second intercommunity route from among the plurality of intercommunity routes when at least one of the energy consumption of the second community centroid, the intracommunity time of the second community centroid, and the intercommunity time of the first community centroid and the second community centroid do not satisfy the one or more performance constraints. In one form, the first intercommunity route originates from the first charging station.

In one form, the method further includes selectively assigning an auxiliary set of community centroids from among the plurality of community centroids to the plurality of intercommunity routes, where each community centroid from among the auxiliary set of community centroids is the auxiliary type. In one form, selectively assigning the auxiliary set of community centroids from among the plurality of community centroids to the plurality of intercommunity routes further includes, for each community centroid from among the auxiliary set of community centroids: determining whether one or more intercommunity routes from among the plurality of intercommunity routes are available based on an energy consumption of the community centroid, an intracommunity time of the community centroid, and an intercommunity time of the community centroid, and in response to the one or more intercommunity routes being available, assigning the community centroid to a given intercommunity route from among the one or more intercommunity routes based on a distance between the community centroid and the one or more intercommunity routes. In one form, the method further includes generating an additional route based on the community centroid and the one or more charging stations in response to the one or more intercommunity routes not being available.

The present disclosure provides a system for surveying an environment including a processor and a nontransitory computer-readable medium including instructions that are executable by the processor. The instructions include segmenting a plurality of nodes into a plurality of communities, where each node from among the plurality of nodes corresponds to a location of the environment, and where each community from among the plurality of communities includes a set of nodes from among the plurality of nodes. The instructions include identifying a plurality of community centroids based on the plurality of communities, where each community centroid from among the plurality of community centroids is associated with one community from among the plurality of communities. The instructions include determining, for each community centroid from among the plurality of community centroids, a community type of the community centroid, where the community type is one of a primary type and an auxiliary type, and where the community type is based on a distance between the community centroid and one or more charging stations. The instructions include generating a plurality of intercommunity routes based on the plurality of community centroids, the one or more charging stations, the community type, and one or more performance constraints.

In one form, the instructions include selecting a first charging station and a second charging station from among the one or more charging stations, determining a first distance between the first charging station and the community centroid, and determining a second distance between the second charging station and the community centroid, where the community type of the community centroid is further based on a ratio between the first distance and the second distance.

In one form, the instructions for generating the plurality of intercommunity routes based on the plurality of community centroids, the one or more charging stations, the community type, and the one or more performance constraints further include: selecting a first charging station from among the one or more charging stations, selecting a first set of community centroids from among the plurality of community centroids, where each community centroid from among the first set of community centroids is the primary type, and where a distance between the first set of community centroids and the first charging station is less than a distance between the first set of community centroids and a second charging station from among the one or more charging stations; selecting a first community centroid from among the first set of community centroids, where a distance between the first community centroid and the first charging station is greater than a distance between each remaining community centroid of the first set of community centroids and the first charging station; and assigning the first community centroid to a first intercommunity route from among the plurality of intercommunity routes when an energy consumption of the first community centroid and an intracommunity time of the first community centroid satisfy the one or more performance constraints. In one form, the instructions further include selecting a second community centroid from among the first set of community centroids, assigning the second community centroid to the first intercommunity route when an energy consumption of the second community centroid, an intracommunity time of the second community centroid, and an intercommunity time of the first community centroid and the second community centroid satisfy the one or more performance constraints, and assigning the second community centroid to a second intercommunity route from among the plurality of intercommunity routes when at least one of the energy consumption of the second community centroid, the intracommunity time of the second community centroid, and the intercommunity time of the first community centroid and the second community centroid do not satisfy the one or more performance constraints.

In one form, the instructions further include selectively assigning an auxiliary set of community centroids from among the plurality of community centroids to the plurality of intercommunity routes, where each community centroid from among the auxiliary set of community centroids is the auxiliary type. In one form, the instructions for selectively assigning the auxiliary set of community centroids from among the plurality of community centroids to the plurality of intercommunity routes further include, for each community centroid from among the auxiliary set of community centroids: determining whether one or more intercommunity routes from among the plurality of intercommunity routes are available based on an energy consumption of the community centroid, an intracommunity time of the community centroid, and an intercommunity time of the community centroid, and in response to the one or more intercommunity routes being available, assigning the community centroid to a given intercommunity route from among the one or more intercommunity routes based on a distance between the community centroid and the one or more intercommunity routes. In one form, the instructions further include generating an additional route based on the community centroid and the one or more charging stations in response to the one or more intercommunity routes not being available.

DETAILED DESCRIPTION

The present disclosure provides a survey system that performs a bi-level routing problem for efficiently surveying an environment using a carrier robot and a drone. Specifically, the survey system is configured to partition one or more nodes representing various points of interests of the environment based on an asynchronous label propagation routine. The survey system then assigns the detected communities of nodes to a generated intercommunity route based on the location of the charging stations, energy constraints, time constraints, and performance constraints associated with the carrier robot and the drone. Subsequently, the survey system generates one or more subgraphs based on the intercommunity routes and performs various vehicle routing problem routines to define subgraph routes within the individual subgraphs, such as a traveling salesperson problem with drone energy management (TSP-D-EM) routine or a moving horizon heuristic formulation routine. Accordingly, the survey system of the present disclosure provides a path planning solution that can satisfy various performance and time constraints in a continuous manner.

Referring toFIGS.1A-1B, an environment1including a survey system5according to one or more forms of the present disclosure is provided. In one form, the survey system5includes a carrier robot10, a drone20, charging stations30, and a central controller40. While the central controller40is illustrated as part of the environment1, it should be understood that the central controller40may be positioned remotely from the manufacturing environment1. In one form, the carrier robot10, the drone20, and the central controller40are communicably coupled using a wireless communication protocol (e.g., a Bluetooth®-type protocol, a cellular protocol, a wireless fidelity (Wi-Fi)-type protocol, a near-field communication (NFC) protocol, an ultra-wideband (UWB) protocol, among others).

In one form and as shown inFIG.1B, the carrier robot10is a wheeled carrier platform robot that includes a docking station11disposed thereon that is configured to physically support and/or accommodate the drone20. As an example, the docking station11includes hardware/systems for attaching/mounting the drone20to the docking station11when the carrier robot10is traversing the environment1. In one form, the drone20is a legged robot or an unmanned aerial vehicle robot configured to access places in the environment1that the carrier robot10may not access. It should be understood that the carrier robot10and/or the drone20may be implemented by other types of autonomous robots and are not limited to the examples described herein.

In one form and as shown inFIG.1A, the carrier robot10includes a sensor system12, a carrier controller14, and a charging system16, and the drone20includes a sensor system22and a drone controller24. In one form, the carrier robot10and the drone20are partially or fully autonomous and configured to autonomously move to various locations of the environment1based on data from one or more autonomous navigation sensors disposed thereon (e.g., a global navigation satellite system (GNSS) sensor, an imaging sensor, a local position sensor, among others). Specifically, the carrier controller14, the drone controller24, and the central controller40are configured to process the data from the autonomous navigation sensors and perform known autonomous navigation routines to navigate the environment1.

In one form, the sensor systems12,22include image sensors configured to capture image data when performing a surveying routine of the environment1. Example image sensors include, but are not limited to: an image sensor, a two-dimensional (2D) camera, a three-dimensional (3D) camera, an infrared sensor, a radar scanner, a laser scanner, among other imaging devices. Accordingly, the carrier controller14and the drone controller24may provide the captured image data to the central controller40, which may generate a digital twin of the environment1using known digital twin generation routines.

In one form, the charging system16is disposed on the carrier robot10and includes electrical components that are configured to charge the drone20while it is supported by and/or attached to the carrier robot10. In one form, the carrier robot10may have a battery life that is greater than a battery life of the drone20. As described herein in further detail, the central controller40may selectively deploy the drone20to scan a given area of the environment1based on a surveying routine.

In one form, the central controller40includes a robot control module50and a path planning module60. In one form, the robot control module50is configured to control the deployment and operation of the drone20and the operation of the carrier robot10. As an example, the central controller40is configured to broadcast a command to the drone20to deploy and/or detach from the docking station11and autonomously navigate the environment1to initiate the surveying routines described herein in further detail.

In one form, the path planning module60is configured to define intercommunity and intracommunity paths in which the carrier robot10and the drone20traverse to survey the environment1and capture the image data. Additional details regarding the functionality of the path planning module60is provided below in further detail with reference toFIGS.2-12.

To perform the functionality described herein, the carrier controller14, the drone controller24, and the central controller40may each include one or more processor circuits that are configured to execute machine-readable instructions stored in the one or more nontransitory computer-readable mediums, such as a random-access memory (RAM) circuit and/or read-only memory (ROM) circuit. Furthermore, the carrier controller14and the drone controller24may each include movement drivers and systems, transceivers, routers, and/or input/output interface hardware to perform the functionality described herein.

Referring toFIG.2, the path planning module60of the central controller40includes a partition module61for generating subgraphs and a subgraph route and agent (SRA) module81for generating subgraph routes. It should be readily understood that any one of the modules/components of the path planning module60can be provided at the same location or distributed at different locations (e.g., via one or more edge computing devices) and communicably coupled accordingly. In one form, the partition module61includes a node generation module62, a node connection module64, a community detection module68, an intercommunity route assignment module70, a subgraph module72, and a map74.

Referring toFIGS.2-3, in one form, the node generation module62is configured to generate a plurality of nodes100corresponding to various locations of the environment1. As an example, the node generation module62is configured to generate a plurality of nodes100within an image110representing the environment1based on known image processing routines and/or an input from an operator/technician indicating a selection of the location as a point of interest. In one form, the plurality of nodes100is defined based on a layout, physical boundaries, and/or other characteristics of the environment1. As described below in further detail, at least one of the carrier robot10and the drone20are configured to perform the surveying routine at each of the nodes100.

Referring toFIGS.2and4-5, in one form, the node connection module64is configured to connect the nodes100via a plurality of edges120and based on one or more edge constraints. In one form, the edge constraints correspond to traversable and/or allowable travel paths of the carrier robot10or the drone20between the nodes100. In one form, the edge constraints are based on obstacles130and the layout of the environment1, minimum travel distances between the nodes100, and/or a radial distance indicating a communication range of the carrier robot10and the drone20. In one form, the edge connection module64may perform known minimum travel distance routines to determine the minimum travel distance, such as Dijkstra's routine.

As an example and as shown inFIG.5, the edge constraints may inhibit the edge connection module64from connecting node100-1and nodes100-2,100-3,100-4since the nodes100-2,100-3,100-4are beyond a communication range of the carrier robot10and the drone20(indicated by radial distance RMAXinFIG.5). Additionally, the edge constraints may cause the edge connection module64to connect the node100-1with nodes100-5,100-6via edges120-1,120-2since a minimum travel distance between the nodes100-1,100-5and the nodes100-1,100-6is less than the communication range. As another example, the edge constraints may prevent the edge connection module64from connecting node100-1and node100-7since the minimum travel distance between the nodes100-1,100-7(shown by dotted line122inFIG.5) is greater than the communication range of the carrier robot10and the drone20(RMAX). That is, the obstacle130inhibits the navigation of the carrier robot10and the drone20between the nodes100-1,100-7. It should be understood that the edge connection module64iteratively repeats the above-described node connection routine for each node100.

In one form, the node connection module64assigns one or more traversability scores for each edge120that indicates a traversability of at least one of the carrier robot10and the drone20along the edge120. In one form, the traversability score is based on a threshold travel distance between the nodes100(e.g., a minimum travel distance), obstacles of the environment1, and/or the geometry and/or mobility characteristics of the carrier robot10and the drone20. In one form, the node connection module64assigns, for each edge120, a separate traversability score for the carrier robot10and the drone20. As an example, the edge connection module64may assign a larger traversability score for each of the carrier robot10and the drone20along the edge120-1based on a relatively smaller minimum travel distance, the lack of obstacles130between the nodes100-1,100-2, and the lack of any geometry/mobility characteristics that inhibit the carrier robot10and the drone20from traveling between the nodes100-1,100-2. As another example, the edge connection module64may assign a larger traversability score for the drone20along the edge120-2, but the edge connection module64may assign a lower traversability score for the carrier robot10along the edge120-2since the geometry of the carrier robot10inhibits travel between the adjacent obstacles130.

Referring toFIGS.2and6, in one form, the community detection module66is configured to segment the plurality of nodes100into a plurality of communities140-1,140-2, . . .140-n(collectively referred to herein as “communities140”) and a plurality of community centroids150-1,150-2, . . .150-n(collectively referred to herein as “community centroids150”). In one form, each community140includes a set of the nodes100(e.g., one or more nodes100), and the community centroid150corresponds to a centroid location within the respective community140. In one form, the community detection module66may employ known label propagation routines, such as an asynchronous label propagation routine, to segment the plurality of nodes100into the communities140and the community centroids150. In one form, the community detection module66may segment the nodes100into the communities140based on the traversability scores of the edges120between the nodes100(e.g., one or more nodes100may be assigned to another community140if the edge120therebetween has a traversability score that deviates from a median, mean, or other arithmetic representation beyond a threshold amount within the given community140).

Referring toFIGS.2and7, in one form, the community type module68is configured to determine a community type of each community centroid150as one of a primary type and an auxiliary type based on a distance between the community centroid150and the one or more charging stations30. In one form, the community type module68determines, for each community centroid150, the distance between each of the charging stations30and the community centroids150and selects the two charging stations30(or other predefined number of charging stations30) that are nearest to the community centroid150. Subsequently, the community type module68determines a ratio between the distances of the identified charging stations30and the community centroid150and identifies the community centroid150as one of the primary type and the auxiliary type based on the ratio.

As an example and as shown inFIG.7, the community type module68determines a distance between the community centroid150-3and charging station30-1(i.e., the first distance), the community centroid150-3and charging station30-2(i.e., the second distance), and the community centroid150-3and charging station30-3(i.e., the third distance). Moreover, the community type module68selects the charging stations30-1,30-2since the first and second distances are less than the third distance for determining the community type. Subsequently, the community type module68determines a ratio of the first distance and the second distance to identify the corresponding community type. Specifically, the community type module68determines the community centroid150-3is a primary type since the ratio of the first and second distances is less than a threshold value. The community type module68may iteratively determine the community type for each community centroid150and determine that, for example, community centroids150-4,150-5,150-6,150-7are auxiliary-type community centroids150, and the remaining community centroids150are primary-type community centroids150. The primary-type community centroids150may be collectively referred to herein as “primary-type community centroids150a,” and the auxiliary-type community centroids150may be collectively referred to herein as “auxiliary-type community centroids150b.”

In one form, the community type module68is configured to group the primary-type community centroids150abased on a minimum travel distance to the charging stations30. As an example, the community type module68identifies a first set160-1of primary-type community centroids150athat is nearest to the charging station30-1, a second set160-2of primary-type community centroids150athat is nearest to the charging station30-2, and a third set160-3of primary-type community centroids150athat are nearest to the charging station30-3.

Referring toFIGS.2and8, in one form, the intercommunity route assignment module70is configured to generate a plurality of intercommunity routes based on the community centroids150, the one or more charging stations30, the community types, and/or one or more performance constraints. Example performance constraints include, but are not limited to, an energy consumption required to survey the community140associated with the community centroid150and/or an amount of time required to survey the community140associated with the community centroid150, which is hereinafter referred to as the “intracommunity time.” In one form, the intracommunity time may be based on the time to travel between the nodes100of the community140and the amount of time needed to scan the environment1at each node100. In some forms, the performance characteristics may also include an amount of time required to travel between communities140, which is hereinafter referred to as the “intercommunity time,” and/or an energy consumption required to travel between communities140.

In one form, the intercommunity route assignment module70generates the intercommunity routes by iteratively assigning routes to the primary-type community centroids150aand then iteratively assigning routes to the auxiliary-type community centroids150. In one form, the intercommunity route assignment module70generates the intercommunity routes such that they originate from at least one of the charging stations30.

In one form, the intercommunity route assignment module70selects the first charging station30-1and selects the first set160-1of primary-type community centroids150aassociated with the first charging station30-1. The intercommunity route assignment module70selects a community centroid150from the first set160-1of primary-type community centroids that is furthest from the first charging station30-1(e.g., the community centroid150-9). Subsequently, the intercommunity route assignment module70generates a first intercommunity route170-1that originates from the first charging station30-1and ends at the selected community centroid150provided that the energy consumption/intracommunity time of the corresponding community140is less than a respective threshold value.

When the intercommunity route assignment module70generates the first intercommunity route170-1, the intercommunity route assignment module70selects the next remaining community centroid150from the first set160-1of primary-type community centroids150athat is furthest from the first charging station30-1(e.g., the community centroid150-8). It should be understood that the intercommunity route assignment module70may select any one of the remaining community centroids150from the first set160-1of primary-type community centroids150aand is not limited to the example described herein.

Subsequently, the intercommunity route assignment module70assigns the first intercommunity route170-1to the selected community centroid150provided that the energy consumption/intracommunity time of the corresponding community centroid150(e.g., the community centroid150-8), the energy consumption/intracommunity times of other community centroids assigned to the first intercommunity route170-1(e.g., the community centroid150-9), and the intercommunity time between the community centroids assigned to the first intercommunity route170-1are each less than a respective threshold value. If one of the performance constraints is not satisfied when assigning the first intercommunity route170-1to the selected community centroid150, the intercommunity route module70generates a new intercommunity route170(e.g., intercommunity route170-2) for the selected community centroid150. As used herein, the intercommunity route170may refer to at least one of intercommunity routes170-1,170-2,170-2,170-3,170-4,170-5,170-6.

As a specific example, the intercommunity route module70assigns the community centroid150-8to the first intercommunity route170-1provided that the communities140associated with community centroids150-8,150-9can be completed within a predetermined period of time and without further charging of the carrier robot10at the charging station30-1. Furthermore, the intercommunity route module70assigns the community centroid150-8to a second intercommunity route170if the intercommunity time, intracommunity time, and/or energy consumption constraints are not satisfied.

In one form, the intercommunity route assignment module70iteratively performs the route assignment routine such that each primary-type community centroid150ais assigned to a given intercommunity route170. When each primary-type community centroid150ais assigned to a given intercommunity route170, the intercommunity route assignment module70selectively assigns the auxiliary-type community centroids150bto one of the given intercommunity routes170. In one form, the intercommunity route assignment module70determines whether the one or more intercommunity routes170are available based on an energy consumption of the auxiliary-type community centroid150b, an intracommunity time of the auxiliary-type community centroid150b, and/or an intercommunity time between the auxiliary-type community centroid150band one or more of the primary-type community centroids150aalong the intercommunity routes170. The intercommunity route assignment module70assigns the nearest available intercommunity route170to the auxiliary-type community centroid150bin response to the determination that the performance constraints are satisfied when it is added to the intercommunity route170. If one of the performance constraints is not satisfied when assigning one of the intercommunity routes170to the auxiliary-type community centroid150b, the intercommunity route module70generates an additional intercommunity route170for the selected auxiliary-type community centroid150b.

Referring toFIGS.2and9, in one form, the subgraph module72is configured to generate one or more subgraphs180-1,180-2,180-3,180-4,180-5,180-6(collectively referred to herein as “subgraphs180”) based on the intercommunity routes170and store it as the map74. In one form, each of the subgraphs180originates from one of the charging stations30, thereby enabling the carrier robot10and the drone20to collectively perform the surveying routine at the given nodes100within a subgraph180in a continuous manner and without charging the carrier robot10and the drone20at the charging stations30.

Referring toFIG.2, in one form, the SRA module81includes a node number module82, a traveling salesperson problem (TSP) module84, and a moving horizon formulation (MHF) module86. In one form, the node number module82is configured to determine the number of nodes100within each of the subgraphs180. The node number module82is configured to selectively activate one of the TSP module84and the MHF module86to generate the subgraph routes within the subgraphs180based on the number of nodes100. As an example, the node number module82activates the TSP module84for determining the subgraph route of the given subgraph180in response to the number of nodes100being less than a threshold value (e.g., ten nodes). Additionally, the node number module82activates the MHF module86for determining the subgraph route of the given subgraph180in response to the number of nodes100being greater than or equal to the threshold value. Accordingly, the node number module82inhibits latency and reduces the computational resources needed for generating the subgraph routes of the subgraphs180.

Referring toFIGS.2and10, in one form, the TSP module84performs the TSP-D-EM routine, which may be a mixed integer linear routine that includes an objective function configured to reduce the total time needed to survey the nodes100. In one form, the objective function is based on the time in which the carrier robot10and the drone20move between nodes and the time required for the drone20to detach from the carrier robot10and survey the environment1at a location associated with the node100. In one form, the mixed integer linear routine includes one or more operation constraints, such as flow conservation equations of the drone20to detach from the carrier robot10, rules defining the number of instances each node100can be visited (e.g., one time), rules defining which nodes100are to be surveyed, and/or upper and lower bounds, among other operational constraints. It should be understood that the operational constraints may include various other operational constraints and are not limited to the examples described herein.

In one form, the mixed integer linear routine includes energy management constraints, including, but not limited to: energy consumption limits when surveying routine at each node100, energy consumption limits when traveling between nodes100, charging events in which the charging system16charges the drone20when the drone20is attached to the carrier robot10, the battery life of the drone20, and upper and lower bounds, among other energy constraints. It should be understood that the energy constraints may include various other energy constraints and are not limited to the examples described herein.

As an example implementation of the TSP-D-EM routine of subgraph180-7shown inFIG.10, the TSP module84instructs at least one of the carrier robot10and the drone20to survey node100-10, to collectively travel to node100-11(the carrier robot10path is shown with a solid line, and the drone20path is shown with a dashed line inFIG.10), and to have at least one of the carrier robot10and the drone20to survey node100-11. When the surveying routine is complete at the node100-11, the drone20detaches from the carrier robot10and travels to node100-13to perform the surveying routine, the carrier robot10travels to node100-12to perform the surveying routine, and the carrier robot10and the drone20meet at the node100-13when the surveying routines are completed. The drone20attaches itself to the carrier robot10via the docking station11at the node100-13, and the carrier robot10and the drone20collectively travel to nodes100-14,100-15to perform the surveying routine at the respective nodes. When the surveying routine is complete at the node100-15, the drone20detaches from the carrier robot10and travels to node100-16to perform the surveying routine, the carrier robot10travels to node100-17to perform the surveying routine, and the carrier robot10and the drone20meet at the node100-17when the surveying routines are completed at the respective nodes. The drone20attaches itself to the carrier robot10via the docking station11at the node100-17, and the carrier robot10and the drone20collectively return to the node100-10.

Referring toFIGS.2and11A-11E, in one form, the MHF module86performs the MHF routine, which may include performing known TSP routines for one of the carrier robot10and the drone20among the nodes100, as shown inFIG.11A. Additionally, and as shown inFIGS.11B-11C, the MHF routine includes selecting a set of nodes100from subgraph180-8based on a preview horizon value (e.g., selecting nodes100-20,100-21,100-22,100-23,100-24when the preview horizon value is equal to five) and performing the TSP-D-EM routine described above for the selected set of nodes100. The preview horizon value may be selected based on computing resources and/or capabilities of the central controller40. In one form and as shown inFIGS.11D-11E, the MHF routine iteratively selects additional sets of nodes100(e.g., nodes100-25,100-26,100-27,100-28,100-29) based on the preview horizon value and performs the TSP-D-EM routine for the selected additional sets of nodes100until the subgraph route is defined.

Referring toFIG.12, an example routine1200for surveying the environment1is shown. At1204, the central controller40segments the plurality of nodes100into a plurality of communities140. At1208, the central controller40identifies a plurality of community centroids150based on the plurality of communities140. As described above, the central controller40may perform the asynchronous label propagation routine to segment the nodes100into the communities140and identify the community centroids150. At1212, the central controller40determines a community type of each community centroid150, and the central controller40generates a plurality of intercommunity routes170based on the plurality of community centroids, the one or more charging stations, the community type, and one or more performance constraints at1216.

At1220, the central controller40generates a plurality of subgraphs180based on the intercommunity routes170and determines the number of nodes100within each subgraph180at1224. At1228, the central controller40generates the subgraph routes by selectively performing one of the TSP-D-EM routine and the MHF routine for each subgraph180based on the number of nodes100. At1232, the central controller40selectively controls the operation and deployment of the carrier robot10and the drone20to initiate the surveying routine based on the intercommunity routes and subgraph routes.