Cooperative perimeter patrol system and method

A method of patrolling a perimeter of a geographic area, using two or more unmanned vehicles having means for locomotion along a perimeter path. Each vehicle is equipped with at least the following systems: a navigation system operable to autonomously navigate the unmanned vehicle, an anomaly detection system, a communications system, an anomaly tracking system, operable to track, visually or by following, a detected anomaly, and an alert evaluation system. Each vehicle travels the path on a predetermined route, and is operable to broadcast an alert message to all other vehicles if that vehicle detects an anomaly, to perform an evaluation of any received alert message to determine if it will travel to an anomaly based on stored evaluation rules, and to respond to an alert message based on the evaluation.

TECHNICAL FIELD OF THE INVENTION

This invention relates to patrolling the perimeter of a secured (protected) region, and more particularly, to patrolling with unmanned ground vehicles under decentralized control.

BACKGROUND OF THE INVENTION

For various reasons, it is often desired to patrol the geographic perimeter of an area. Examples of perimeter patrol applications are prisons, airports, schools, sea ports, and military installations. Larger-scale applications include patrol of geographic borders.

A common application is for an area where there is a particular need for security and protection against “anomalies”. Examples of perimeter anomalies are objects blocking a perimeter path, fence or wall breaches, or unauthorized persons crossing the perimeter.

Conventional methods for perimeter patrol use human operators of small to mid-sized vehicles. This approach places the human operators in a dull environment, where repetition can lead to complacency and where detection of anomalies requires human perception and attention. Funding is often a limiting factor as to how many vehicles and personnel can be devoted to the patrol system.

A more modern approach is using unmanned ground vehicles to autonomously (without human intervention) perform the patrol tasks. The technology for an automated perimeter patrol system can be expensive, but such a system can significantly improve overall reliability in detecting anomalies.

DETAILED DESCRIPTION OF THE INVENTION

The following description is directed to a perimeter patrol system that uses unmanned ground vehicles (UGVs) with cooperative control. The term “unmanned ground vehicle” is used herein in a broad sense, and includes robots and other devices having varying degrees of autonomy and means for locomotion across the ground.

More specifically, as used herein, an “unmanned ground vehicle” or “UGV” is a vehicle that operates while in contact with the ground and without an on-board human presence. Generally, the vehicle will have a set of sensors to observe the environment. The vehicle is capable, to varying degrees of sophistication, to either autonomously make decisions about its own behavior or to pass the information to a human operator at a different location who may exercise additional (non autonomous) control of the vehicle through telecommunications. For the patrol method and system described herein, the autonomous capabilities of the UGV's may vary from vehicle to vehicle and are discussed below.

As explained below, the system enables multiple unmanned ground vehicles to share currently perceived information about their environment and to coordinate a response without the need for a central controller. Various algorithms discussed herein enable this cooperative sensor sharing and decentralized response coordination to detected anomalies. The algorithms are not dependant on the specific vehicle type, available sensors, path planning capabilities, or communications hardware, and are scalable and extensible for any number of vehicles.

The system allows the integration and relative positional correlation of disparate sensor data from two or more separately moving unmanned ground vehicles. A communication message protocol is used to exchange information and to coordinate a system response.

FIG. 1illustrates an example of a protected region, enclosed and defined by a perimeter fence or other barrier10. A path11, suitable for unmanned ground vehicles runs along and just inside fence10. For purposes of example, path11generally conforms to the perimeter geometry, but as long as the patrol objectives described herein are achieved, path11may vary from this exact geometry.

The travel of vehicles on path11is referred to herein as “on-road” travel. Additional paths (not shown) may be added across the protected region.

The smoothness and levelness of path11are appropriate for the particular type of unmanned ground vehicle. In general, the more rugged the path11, the more sophisticated is the locomotion system of the vehicles. As explained below, however, a feature of the invention is that less expensive and complex vehicles may be used for on-road patrol, with additional special vehicles used for “off-road” travel, that is, travel not on a paved or otherwise groomed path.

In the example ofFIG. 1, the protected region also includes rugged areas inside path11. The fleet of unmanned vehicles includes a first type of vehicle that travels only path11. These vehicles are referred to herein as “on-road” vehicles13.FIG. 1shows four such vehicles, referenced as vehicles13a-13d.

The fleet of unmanned patrol vehicles also includes a second type of vehicle that is capable of traversing all or some of the interior of the protected region. These vehicles are referred to herein as “off-road” vehicles14.FIG. 1shows two such vehicles, referenced as14aand14b. Typically, a vehicle14that is capable of off-road travel is also capable of on-road travel.

In other embodiments, additional types of vehicles, especially suited for different terrains within the protected region may be included. As explained below, the system is easily scalable. The number of vehicles and the various locomotion mechanics of the vehicles may vary depending on the size and terrain of the protected region.

As further explained below, not all vehicles have the same capabilities. A feature of the invention is that vehicles communicate with each other to determine which, and how many, vehicles are to respond to an anomaly.

In the example scenario ofFIG. 1, a burning vehicle15has been detected on path11. The detection has been achieved by sensors on-board the nearest vehicle13a.

The burning vehicle15may be generalized as an “anomaly”. The vehicles13are configured to detect anomalies of at least four different types. A “path blockage” anomaly is an object that is blocking the path normally traversed during the perimeter patrol mode of an on-road vehicle13. A “static object” anomaly is similar to a path blockage anomaly, but may be further refined to differentiate a downed tree from a human or animal, which could be detected using thermal sensors. A static object could further include a breach in the perimeter fence10or other barrier. A “dynamic object” anomaly is an object that changes position over time. An “evasive object” anomaly is an object that actively seeks to avoid detection by the vehicle through changes in position over time.

The patrol system further has a control station19, which may be at, or may be remote from, the protected region. Typically, control station19is monitored by a human operator. Control station19is programmed to receive the various alerts and other messages discussed herein. It has appropriate hardware and software for performing the tasks described herein. It displays a control interface that allows the human operator to intervene in operation of the patrol vehicles13and14if desired.

FIG. 2illustrates various autonomous systems on-board vehicles13and14. The illustrated systems include equipment and processing for basic tasks performed by each on-road vehicle13and by each off-road vehicle14(in solid outline), as well as for optional tasks that may be performed by only some vehicles (in dashed outline). Each system has mechanical and/or electrical equipment and corresponding processing hardware and software appropriate for performing its tasks.

As explained below, the systems of different vehicles may vary. That is, different vehicles may be equipped to perform different tasks within one of the systems illustrated inFIG. 2, or may perform certain tasks to different levels of complexity.

A feature of the invention is the specialization of vehicles and coordination of responses among vehicles so that the vehicle(s) best equipped to a particular anomaly are the vehicles that autonomously decide to respond. A basic “generic” patrol vehicle can be designed, and individual vehicles equipped with special equipment. Vehicles can travel and respond according to their capabilities, which avoids the need to equip each vehicle with all features.

Each vehicle is assumed to have a locomotion system21. A vehicle's mechanical equipment for locomotion is referred to generally herein as its “engine”. Thus system21also includes various control processes for controlling the vehicle's engine.

Each vehicle further has a navigation system22. The navigation system allows the vehicle to navigate autonomously, that is, without human or other external navigation assistance. Any coordinate system can be used, but typically GPS type coordinates will be used. As explained below, the navigation system of on-road vehicles13allows them to travel along path11. The navigation system of off-road vehicles14allows them to travel on more difficult off-road terrain.

Each of the on-road vehicles13has an anomaly detection system24. The detection equipment can be any one or more of various types of sensors. Any kind of imaging, proximity, or other type of sensor for detecting the presence of an object near the detecting vehicle can be used. Off-road vehicles14may or may not have detection capabilities; as explained below, an off-road vehicle may be specialized for anomaly resolution.

The anomaly detection system24further includes programming or other means for classifying the anomaly into one of the above-described anomaly types. Also, if an anomaly is detected, the detecting vehicle assigns it an identification (ID) number that is used in subsequent messages.

The sensing system on-board a particular vehicle may be specialized for a certain type of anomaly. Some vehicles may be equipped for only one type, whereas other vehicles may be equipped for more than one type. Examples of sensor types are imaging, proximity, and ultrasonic sensors. Sensors can also have varying configurations. For example, a vehicle equipped to detect objects on the path may have forward-sensing sensors. A vehicle equipped to detect breaches in the perimeter fence or other barrier may have side-sensing sensors. A vehicle equipped to detect humans may have infra-red sensors. These are just a few examples of the different types, configurations, and tasks of sensors.

Vehicles13or14having an anomaly detection system24also have an anomaly tracking system25. As described below, a vehicle that detects an anomaly sends out an alert message to all other vehicles. The detecting vehicle also begins to track the anomaly.

Each patrol vehicle13and14has a communication system26. As explained below, messages communicated among vehicles have a specific message structure. As used herein, the term “broadcast” means that a vehicle transmits a message that will be received by all other vehicles and by the control station19.

Each vehicle13and14further has an alert evaluation system27. As explained below, a vehicle's evaluation system27has stored data representing features of that vehicle. It receives data about an anomaly, and uses received data and stored data to determine whether it will respond (physically travel) to the anomaly.

An anomaly resolution system28is on-board at least one vehicle. As explained below, the system28is operable to deal with at least one type of anomaly. A feature of the invention is that certain vehicles may be designated as “anomaly resolution specialists” and have a system28especially designed for a certain type of anomaly.

FIG. 3illustrates a method of perimeter control, using the system of vehicles13and14described above. Vehicles13and14may also be collectively referred to herein as “patrol vehicles”.

Step301is using on-road vehicles13to follow path11. Each on-road vehicle13is in a continuous anomaly detection mode using system24. Each patrol vehicle13and14is also in a receive mode, using system23, ready to receive alert messages from other vehicles13that an anomaly has been detected.

FIG. 4illustrates Step301. The perimeter10of a protected region is defined by a barrier, such as a fence10. A path11, suitable for on-road vehicles13, conforms generally to the perimeter and the barrier10. In the example ofFIG. 4, there are three on-road vehicles13and one off-road vehicle14.

Each on-road vehicle13is programmed to follow at least a portion of path11. The patrol route traveled by each vehicle13may vary, but typically, each vehicle will traverse the entire path in a single direction. However, many other route strategies are possible, such as one in which each vehicle travels back and forth over a portion of the path.

Along path11are one or more observation points41. An observation point41is a location along path11at which a vehicle may stop for a period of time. The length of the stop may be predetermined and stored within the vehicle's navigation system22.

Observation points41are particularly useful at locations that are high security, such as a weapons store. Or an observation point may be a location that is geographically advantageous, such as at a point having an elevated altitude.

Depending on the nature of the observation point41, the vehicle may perform one or more of various tasks. It may “stop and stare”. Or it could inspect an object closely and report to the control station19.

In general, when vehicles13are not responding to an anomaly, they are in a “patrol mode”. This means that they follow the perimeter path11and stop at various observation points41for a set period of time.

Referring again toFIG. 3, Step302is performed by a vehicle13that has detected an anomaly. This vehicle broadcasts an “alert” message to all other vehicles. Data in the alert message represents at least the location of the detecting vehicle, the location of the detected anomaly, the ID of the detecting vehicle, the anomaly type, and an anomaly ID.

In Step304, the detecting vehicle tracks the anomaly. For example, if the anomaly is in motion, the detecting vehicle physically follows the anomaly according to that vehicle's navigation capabilities. Alternatively, the detecting vehicle tracks the anomaly by maintaining sensor contact, visual or other.

Step306is performed if the anomaly is resolved. In this case, the detecting vehicle broadcasts an “anomaly clear” message. Step306may also be performed by other vehicles that have arrived at the location of the anomaly.

Step320is performed by any vehicle13and14that has received an alert message. Vehicles that receive an alert message then calculate the distance to the anomaly. This distance calculation is based on the anomaly position in the received message and on the location of the receiving vehicle. The calculation is thus reliant on data from the locomotion system21and is part of the anomaly alert evaluation system27.

The evaluation of an alert-receiving vehicle of its own response (whether or not to physically navigate to the location of the anomaly) involves evaluating if it is appropriately equipped to travel to the anomaly and to respond to the anomaly type. For example, the vehicle may determine that it is too far away, is not properly equipped to respond, cannot handle the terrain in the area of the anomaly, or that other vehicles are better equipped or closer. This evaluation may depend on the type of the anomaly that has been detected.

Referring again toFIG. 2, evaluation of an alert is performed by the alert evaluation system27in accordance with response rules and thresholds. These rules and thresholds may be predetermined and stored. Alternatively, thresholds may be modified by a human operator at control station19in real-time relative to the anomaly or at any other time. Alternatively or in addition, the vehicles themselves may be programmed to modify the thresholds through a collective decision-making process.

Various additional evaluation rules and criteria can be used by a vehicle's programming to determine whether that vehicle has on-board features that make it suitable to respond. For example, only certain vehicles may have the sensors suitable for detection and/or for following the particular anomaly. Or, a vehicle may be faster or have other features of its locomotion system21that enable it to reach the anomaly. Or, a vehicle may have special disarming or capturing equipment as part of its anomaly resolution system28.

If a vehicle has the appropriate on-board features to respond, at least two additional rules determine whether that vehicle responds to an anomaly alert. First, the vehicle responds only if the number of responding vehicles is less than a “response crowding” threshold. Second, the vehicle responds only if the distance to the anomaly is less than a “response distance” threshold. As stated above, a user interface of control system19displays the activity of the vehicles in real-time. A human operator may monitor the display and adjust thresholds.

Referring again toFIG. 3, Step322is performed by vehicles that have received an anomaly alert, have evaluated the alert, and are responding by following path11to the anomaly.

FIG. 5illustrates a scenario in which an anomaly51has been detected along path11by a detecting vehicle13(D). This vehicle13has broadcast an alert message. Receiving vehicles13and14have evaluated the alert message (Step320). One receiving vehicle is an on-road vehicle13and one receiving vehicle is an off-road vehicle14. Both vehicles have determined that the appropriate action is to travel to the anomaly (Step322). A fourth vehicle13has determined that it is not equipped, or is too far away, or is superfluous to respond.

All or some vehicles13and14are programmed with “path planning” capabilities. Referring toFIG. 2, this programming is an optional task of a vehicle's navigation system22. Some vehicles must remain on their existing patrol route to travel to the anomaly. Other vehicles can plan a path that is not part of its normal patrol route or is an “off-road” path to the anomaly.

FIG. 5is also an example of a screen display at control station19. As inFIG. 4, vehicles and anomalies are graphically represented. Vehicles that have arrived at, or are en-route to, an anomaly are also depicted, such as by the solid oval outlines inFIG. 5.

FIG. 5further depicts each vehicle's sensor horizon. A “sensor horizon” is the farthest effective range of a given sensor type to detect an anomaly. This range is based on the sensor type and placement on the vehicle. InFIG. 5, the dotted oval outlines represent sensor horizons, and are displayed only when a vehicle has detected an anomaly. The shape of the outline can be indicative of the direction of the range, such as a semi-circular shape for a forward-looking sensor.

Step324is performed by responding vehicles, such as vehicles13and14ofFIG. 5. These vehicles may at any time, or on a periodic basis, send an update request to the detecting vehicle13. The update requests are particularly significant in the case of a mobile anomaly type, because such an anomaly may have moved since the last alert message was received. The movement of an anomaly over time may alter a responding vehicle's navigation behavior, as determined by that vehicle's alert evaluation system27and navigation system22.

If the detecting vehicle13receives an update request, in Step326, it delivers an “update” message. This message has data representing the anomaly type, ID, and location, and the detecting vehicle's location and ID. Typically, because the detecting vehicle13has been following the anomaly, the detecting vehicle's location will be near the anomaly location.

As responding vehicles are en route to anomaly51, they may continue to detect additional anomalies. In Step328, if additional anomalies are detected, the detecting vehicle(s) broadcast a “multi-alert” message. Like other alert messages, a multi-alert message contains the vehicle location and ID and the anomaly type, ID, location. Additionally, a multi-alert message contains an anomaly list, which contains the ID and type of all detected anomalies.

To prevent too many or too few responding vehicles at the location where an anomaly has been detected, the vehicles use two strategies. The processing for these strategies may be part of the anomaly evaluation system27.

A first strategy is that each detecting vehicle periodically sends out an update to all other vehicles (Step326). This update message may include a count of arriving vehicles. Responding vehicles can use this update to reassess their individual response. If enough vehicles have already arrived at the anomaly, a vehicle that is en-route to respond will switch back to the perimeter patrol mode. If not enough vehicles have decided to respond, a vehicle may choose to respond.

A second strategy to prevent overcrowding is that if too many vehicles have arrived at the location of the anomaly, through their normal perimeter patrol routes, a vehicle will, if possible, reverse its direction of travel. The determination of overcrowding is accomplished through the calculation of a local vehicle density, defined as the number of vehicles within a proximity threshold, and is a configurable parameter.

Step330is determining whether the anomaly has been or can be resolved. The anomaly may require no action. Or, upon arriving at the anomaly, if appropriately equipped, a vehicle will resolve the anomaly such as by disarming or destroying it. As illustrated inFIG. 2, one or more vehicles may be equipped with an optional anomaly resolution system28.

LikeFIGS. 4 and 5,FIGS. 6 and 7illustrate examples of user display screens at control station19. As inFIG. 5, the status of a vehicle as “detecting” and “responding” is graphically depicted, such as by the solid outlines.FIG. 7also shows sensor horizons with dotted outlines.

As stated above, a human operator may monitor the display. The display has features that allow the operator to set the above-described evaluation thresholds. The operator may intervene in a vehicle's decision making to whatever extent is programmed into the patrol system.

InFIG. 6, too many vehicles have responded to an anomaly. The operator may redirect one or more of the vehicles. InFIG. 7, more than one anomaly71has been detected. If an appropriate number of vehicles to respond is limited, the operator may determine priorities and intervene accordingly.

An advantage of avoiding overcrowding at an anomaly is that the patrol system cannot be circumvented by a diversion anomaly that attracts so many vehicles that additional anomalies can arise undetected and undeterred.