Patent Description:
It is often desirable to maintain situational awareness of a force protection area to mitigate hostile actions against military personnel, the general public, resources, facilities and critical information, such as a military base, airport, stadium, transit system, smart city, power plant or detention facility, and autonomously react to newly sensed information in these locations. In today's command and control environment, rapid reaction to newly acquired information is necessary so an operator can decide on actions to quickly changing circumstances.

An operator can quickly make decisions by having the system recommend predefined rules of engagement based on the newly sensed situational awareness information. Further, an autonomous system may be able to support decision-making actions independent of an operator if the automated responses are properly controlled and potential consequences of improper responses are carefully considered and the associated potential risks are deemed to be warranted.

Sensor fusion systems demonstrate the efficacy of combining diverse perspectives in an operational area to improve decision making processes. Such systems may provide insight to facilitate object detection, identification, classification, geospatial positioning, geospatial navigation, and many other contextual data metrics that can be derived from data of heterogeneous sensors. With a potential data volume greater than what can be analyzed by unaided human intellect, computer-based support can be necessary, or at a minimum facilitate, the processing and analysis of new data. Further, desired insights may require additional data or analytic processing- such as making sensor adjustments, performing rapid analytics of the data, or real-time reaction to immediate threats that are identified as a result of data analysis. Document <CIT> describes a real-time system embodied in a navigation-capable vehicle (in particular: an autonomous vehicle) for multi-modal <NUM>-D geospatial mapping, object recognition, scene annotation and analytics. The operation of the system is based on data fusion and integration of data received from multiplication sensors. In particular, a 3D visualization of a relevant environmental area of the vehicle is created. Document <CIT> describes a system for generating multidimensional maps of a scene using a plurality of sensors of various types. At least one high-density image of the scene is acquired using at least one passive sensor and at least one new set of distance measurements of the scene acquired using at least one active sensor. The latter are merged with the previous set of distance measurements in order to detect changes. In particular, points of interest (POI) generated based on an analysis and priority levels are set for them. For example, each pixel may be given a priority level, or the point of interest may correspond to rectangular regions.

The present invention is defined by the features of the appended claims.

Exemplary embodiments are best understood from the following detailed description when read in conjunction with the accompanying drawings. Included in the drawings are the following figures:.

Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description of exemplary embodiments is intended for illustration purposes only and is, therefore, not intended to necessarily limit the scope of the disclosure.

Exemplary embodiments of the present disclosure can provide real-time three-dimensional (3D) situational awareness of morphological objects in an operational area and support command, control, communications, collection of intelligence, analysis of data regarding the operational area, surveillance, and reconnaissance (C4ISR). In addition, the disclosed exemplary embodiments can instantiate efficient means for processing heterogeneous sensor data and control dynamic sensor data collection without human intervention. The system and methods described herein extend beyond common sensor fusion methods by interrogating objects after initial sensor data ingestion occurs. For example, the disclosed system and methods can be implemented in the control of servo motors (e.g., optical pan/tilt/zoom), steering electromagnetic antennas (e.g., phased array antennas, servo motor-controlled directional antennas, operating plasma antennas, or importing interferometry of laser analysis of cesium atoms), software defined radios (e.g., controlled adjustment of spectral monitoring), and relocation of unmanned vehicles equipped with sensors to other positions within and outside of the operational area.

Additionally, the exemplary embodiments disclosed herein may provide capabilities for autonomously targeting, exploiting and defeating real-time threats to an operational area if those threats and respective rules of engagement have been confirmed. Exploiting or defeating targets may include soft kill weapons such as electronic warfare tools or hard kill weapons such as directed energy equipment and traditional kinetic systems.

The exemplary embodiments of the present disclosure provide several advantages including: Emergency Response for situational awareness in locating and tracking resources, threats, and military personnel or civilians in need of assistance; automated robotics surrounding awareness including use cases such as warehouse operations and self-driving vehicles; and in assistance with off the grid navigation services in the event satellite, triangulation, or other means of identifying spatial coordinates are unavailable
<FIG> illustrates a system for dynamic three-dimensional command and control in accordance with an exemplary embodiment of the present disclosure. <FIG> illustrates an exemplary computing device and edge detection devices in accordance with an exemplary embodiment of the present disclosure.

As shown in <FIG> and <FIG>, an exemplary system <NUM> for monitoring an operational area <NUM> is disclosed. The system <NUM> includes a computing system <NUM> having a communication interface <NUM>, a processor <NUM>, an input/output (I/O) interface <NUM>, a memory device <NUM>, and a display device <NUM>. The components of the computing system <NUM> can include a combination of hardware and software devices. The computing system <NUM> can be configured to: receive plural data streams via the communication interface <NUM>, each data stream including a different spatial characteristic of the operational area <NUM>. The processor <NUM> can be configured to generate a three-dimensional (3D) virtual visualization of the operational area for display on the display device <NUM> based on observation perspectives associated with the data streams and the spatial characteristics. The processor <NUM> can also be configured to dynamically prioritize operational sub-regions within the operational area <NUM> based on the spatial characteristics and generate a signal encoded with data for verifying the three-dimensional virtual visualization of the operational area <NUM> including the prioritized operational sub-regions.

The communication interface <NUM> can include a receiving and transmitting device configured to connect to a network <NUM>. The communication interface <NUM> can be encoded with program code to receive and transmit data signals and/or data packets over the network <NUM> according to a specified communication protocol and data format. During a receive operation, the communication interface <NUM> can identify parts of the received data via the header and parse the data signal and/or data packet into small frames (e.g., bytes, words) or segments for further processing at the processor <NUM>. During a transmit operation, the communication interface <NUM> can receive data from the processor <NUM> and assemble the data into a data signal and/or data packets according to the specified communication protocol and data format of the network <NUM>. The communication interface <NUM> can include one or more receiving devices and transmitting devices for providing data communication according to any of a number of communication protocols and data formats as desired. For example, the communication interface <NUM> can be configured to communicate over the network <NUM>, which may include a local area network (LAN), a wide area network (WAN), a wireless network (e.g., Wi-Fi), a mobile communication network, a satellite network, the Internet, optic fiber, coaxial cable, infrared, radio frequency (RF), or any combination thereof. Other suitable network types and configurations will be apparent to persons having skill in the relevant art. The communication interface <NUM> can include any suitable hardware components such as an antenna, a network interface (e.g., an Ethernet card), a communications port, a PCMCIA slot and card, or any suitable processing devices for performing functions according to the exemplary embodiments described herein.

The processor <NUM> can be a special purpose or a general purpose hardware processing device encoded with program code or software for performing the exemplary embodiments disclosed herein. The processor <NUM> can be connected to a communications infrastructure including a bus, message queue, network, multi-core message-passing scheme, etc. The processor <NUM> can include one or more processing devices such as a microprocessor, central processing unit, microcomputer, programmable logic unit or any other suitable hardware processing device as desired.

The I/O interface <NUM> can be configured to receive the signal from the processor and generate an output verifying the 3D virtual visualization of the operational area. The I/O interface <NUM> can include a combination of hardware and software for example, a processor, a circuit card, or any other suitable hardware device encoded with program code, software, and/or firmware for communicating with a peripheral device such as the display device <NUM>.

The memory device <NUM> can be configured to store the plural data streams. The memory device <NUM> can include one or more memory devices such as volatile or non-volatile memory. For example, the volatile memory can include random access memory, read-only memory, etc. The non-volatile memory can include a resident memory device such a hard disk drive and a removable storage drive (e.g., a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory, or any other suitable device). The non-volatile memory can also or in the alternative include an external memory device connected to the computing device <NUM> via the I/O interface <NUM>. Data stored in the computer system <NUM> (e.g., in a non-volatile memory) may be stored on any type of suitable computer readable media, such as optical storage (e.g., a compact disc, digital versatile disc, Blu-ray disc, etc.) or magnetic tape storage (e.g., a hard disk drive). The data may be configured in any type of suitable database configuration, such as a relational database, a structured query language (SQL) database, a distributed database, an object database, etc. Suitable configurations and storage types will be apparent to persons having skill in the relevant art.

The display device <NUM> can include high-definition multimedia interface (HDMI), digital visual interface (DVI), video graphics array (VGA), or any suitable display device or display type as desired. The display <NUM> may be any suitable type of display for displaying data transmitted via the I/O interface <NUM> of the computer system <NUM>, including a cathode ray tube (CRT) display, liquid crystal display (LCD), light-emitting diode (LED) display, capacitive touch display, thin-film transistor (TFT) display, etc..

The operational area <NUM> can include a sensor arrangement <NUM> configured to generate the plural data streams by observing the operational area. The sensor arrangement <NUM> can include a plurality of heterogeneous sensors DS1-DSn of various types such as sensors any one or more of an acoustic, radio frequency, electro-optical, thermal, chemical, biological, radiological, nuclear, explosive, temperature, mechanical, etc. Each sensor DS1-DSn can be attached, affixed, or integrated into an edge device <NUM> to establish a dynamic sensor (DS) or edge device that can be moved between two or more locations within the operational area <NUM>. The sensor data is dynamically acquired, meaning one or more of the dynamic sensors D1-DSn may appear or disappear on the network <NUM> at any given time. The sensor data is then stored in a memory device <NUM> or real-time memory-based database <NUM>, such as Redis or any suitable database as desired, for later processing and analysis operations. The edge device <NUM> with an attached or integrated dynamic sensor DSn can include a communication interface <NUM> configured to receive the plural data streams of other sensors. The edge device <NUM> can include one or more processors <NUM> configured to generate a 3D virtual visualization of the operational area <NUM>, dynamically prioritize operational sub-regions within the operational area <NUM>, and generate a signal encoded with data for verifying the 3D virtual visualization of the operational area <NUM>. Each dynamic sensor DSn is configured to transmit a data stream periodically or non-periodically to the computing device <NUM> over the network <NUM>. For example, one or more dynamic sensors DS1-DSn can be configured to transmit data to the computing device <NUM> when a change in a spatial characteristic of an operational sub-region is detected. Each data stream has data of a sensor type and is generated by the corresponding sensor observing the operational area, and a 3D geometry of each operational sub-region is created using one or more characteristics of the sensor arrangement. For example, if the dynamic sensor DSn of an edge device <NUM> is an image sensor or camera, the 3D geometry of the operational sub-region <NUM> can be determined by a field of view (e.g., a cone or triangle) of the image. According to another exemplary embodiment, the dynamic sensor DSn can be an acoustic sensor wherein the sensing field can be represented by a circular shape. According to another exemplary embodiment, the 3D geometry of each operational sub-region <NUM> can be dimensioned according to spatial properties of the operational area <NUM>. For example, one or more operational sub-regions <NUM> can include a 3D geometry having sides and/or a shape limited or constrained by the bordering features (e.g., bodies of water) or physical features (e.g., mountainous or rocky terrain) of the operational area.

<FIG> illustrates a 3D virtual visualization of an operational area in accordance with an exemplary embodiment of the present disclosure. The 3D virtual visualization of the operational area <NUM> generated by the computer system <NUM> and the one or more processors <NUM> of the edge devices <NUM> includes "3D grids" that define plural sub-regions <NUM> within the operational area <NUM> extended to a preferred ceiling height <NUM> above the terrestrial floor. For example, each of the computer system <NUM> and the one or more processors <NUM> of the edge devices <NUM> is configured to generate the 3D virtual visualization of the operational area by dividing a space of the operational area <NUM> into a plurality of 3D grid spaces <NUM>. Each operational sub-region <NUM> includes one or more 3D grid spaces <NUM>. Each 3D space <NUM> is bounded by ground-level altitude <NUM> (e.g., the height above ground level), a user-defined ceiling altitude (e.g., the highest altitude at which the edge device <NUM> or dynamic sensors DS1-DSn can reach in the operational area <NUM>), and the jurisdictional terrestrial boundary requiring protection/monitoring (e.g., land-based geographical areas). This physical space may dynamically change if one or more of the dynamic sensors DS1-DSn detect a new activity or a change in a spatial characteristic of an operational sub-region <NUM> (e.g., a vehicle's presence and its on-board sensors may define a new geographic area to have relevant importance).

The computer system <NUM> and the one or more processors <NUM> of the edge devices <NUM> can be configured to: assign a real-time weighting value to each operational sub-region based on one or more of: an importance to a specified operation, a spatial or temporal relevance to a specified operation, time since last sensor observation, available granularity of the spatial characteristics in the data streams, observational perspectives of sensors from which the data streams are generated with respect to an operational sub-region, corresponding sensor-types, or velocities of anomalies within a current or adjacent operational sub-region. Based on the time since a last sensor observation of one or more sensors that are observing an operational sub-region, the computer system <NUM> and the one or more processors <NUM> of the edge devices <NUM> can be configured to increase a priority of the operational sub-region when an interval between receptions of data streams from a sensor arrangement of the operational sub-region increases. For example, as the intervals between sensor scans of the operational sub-region <NUM> increase, its weight increases so a dynamic sensor DSn system will be directed to observe the operational sub-region <NUM> in the next available opportunity.

The computer system <NUM> and the one or more processors <NUM> of the edge devices <NUM> can be configured to assign a priority to an operational sub-region <NUM> based on proximity of a sensor arrangement to a detected anomaly. For example, sensor proximity can be weighted to where one or more distant dynamic sensors among DS1-DSn may continuously observe an object, but a dynamic sensor of DS1-DSn in closer proximity may warrant additional observation of an operational sub region <NUM> to ensure artifacts are not overlooked. The weight of an operational sub region <NUM> increases over time if any dynamic sensors among DS1-DSn from different observational perspectives do not observe and transmit data streams regarding the operational sub-regions <NUM>. This process allows the observational perspectives of an operational area or sub-region <NUM> to be diversified with respect to nature/lighting/weather/etc. and any man-made or natural obstructions. Observational perspective is not entirely objective, as details and analysis results can vary substantially based on perspective. According to an exemplary embodiment, compass rose-based quadrants can be assigned to one or more operational sub regions <NUM> and dynamic sensors DS <NUM>-DSn to define and/or establish differentiation in perspective. For example, dynamic sensors DS1-DSn from a first perspective may encounter sunlight, which can interfere with object views in a specified operational sub-region <NUM>. Those of the dynamic sensors DS1-DSn having different observational perspectives of the same operational sub-region <NUM>, for example from different viewpoints or viewpoints that are not impacted by the sunlight, can be instructed by the computing system <NUM> or another of the dynamic sensors DS1-DSn to inspect the specified operational sub-region(s) <NUM>. According to an exemplary embodiment of the present disclosure, sensor types will be assigned to each operational sub region to optimally utilize dynamic sensors DS1 -DSn to obtain diverse data streams from sensors that cover the three-dimensional operational area. For example, a dynamic sensor DSn configured as a software defined radio (SDR) may by instructed and/or controlled to continuously monitor the operational area <NUM> for signals at <NUM> and triangulate any sensed signals. Alternatively, any dynamic sensors DS1-DSn that are configured as visual optical sensors in the operational area <NUM> as well as any dynamic sensors DS1-DSn that are configured as thermal cameras may need to view the operational sub-region (e.g., 3D grid) from many different angles to maintain continuous monitoring. Each sensor deployed to cover the operational area <NUM> is configured by the computing system <NUM> to have its own weighting criteria paired with respective geographic, time, range, and perspective criteria based on sensor type and data stream content.

According to an exemplary embodiment, the computer system <NUM> and the one or more processors <NUM> of the edge devices <NUM> can be configured to actively weight each sub region by comparing the 3D geometries of the respective sub regions to the geometries of the representative geometries of each deployed sensor and associated sensor type. For example, a sphere of a certain size may denote the receive sensitivity of a dynamic sensor DSn having an omni-directional radio frequency antenna. As another example, a pyramid may denote the real-time positioning of a security camera field of view.

According to an exemplary embodiment, the computer system <NUM> and the one or more processors <NUM> of the edge devices <NUM> can be configured to decrease a priority of an operational sub-region over time when data streams received from the operational sub-region <NUM> are acquired by two or more sensors from a common observational perspective. As the geometry of a dynamic sensor DSn overlaps a sub-region geometry, the computing system <NUM> and or the one or more processors of other dynamic sensors among DS1-DSn can confirm overlap to denote coverage is met. As a result of the overlap in coverage, the computing system <NUM><NUM> can relax the real-time weighting criteria of that sub-region. As the real-time weighting dynamically changes, the computing device <NUM> or processor of another dynamic sensor DSn can direct sensors with dynamic properties (e.g., pan-tilt-zoom motors) to change position and obtain additional inputs of the next highest priority sub region.

According to an exemplary embodiment, the computer system <NUM> and the one or more processors <NUM> of the edge devices <NUM> can be configured to: assign on a case-by-case basis: a first priority to an operational sub-region <NUM> determined to have a relevant observational perspective of the operational area of a security operation; a null priority to an operational sub-region <NUM> determined to lack a relevant observational perspective of the observational area; and a second priority lower than the first priority to an operational sub-region <NUM> determined to be on a fringe of the operational area and/or undetectable by a sensor. For example, if the operational sub-region <NUM> is critical to security operations, a high priority is assigned. If an operational sub-region <NUM> is outside observational areas (e.g., inside a building, below ground, or otherwise requested not to be viewed), a null priority is assigned. If the operational sub-region <NUM> is on or at the fringe of the operational area and unreachable by any of the dynamic sensors DS1-DSn, the operational sub-region is assigned a low to null priority relative to other operational sub-regions in the operational area.

According to another exemplary embodiment of the present disclosure, the computing system <NUM> can be configured to assign a priority to one or more operational sub-regions <NUM> or one or more dynamic sensors among DS1-DSn having an observational perspective of an operational sub-region <NUM> based on requirements of a specified operation, wherein at least two or more of the assigned priorities are different. For example, one of the operational sub-regions <NUM> where vehicles and/or troops are being assembled for an exercise or deployment may be given a higher priority than an operational sub-region <NUM> where buildings are being constructed.

According to an exemplary embodiment, the computer system <NUM> and the one or more processors <NUM> of the edge devices <NUM> can be configured to process each data stream by detecting one or more of: the presence of an anomaly in the operational area, a position of the anomaly in an operational sub-region, movement of the anomaly in an operational sub-region <NUM>. The computing device <NUM> is configured to: extract portions of the spatial characteristics from each data stream and combine the extracted portions into a combined data set and identify one or more patterns in the combined data set.

According to an exemplary embodiment, the sensor arrangement can include two or more sensors having observational perspectives in each operational sub-region <NUM> and the system is configured to: prioritize each sensor based on characteristics including a location of the operational sub-region, time between data acquisitions, range from an anomaly in the operational sub-region <NUM>, or an observational perspective within the operational sub-region <NUM>.

According to yet another exemplary embodiment the sensor arrangement can include two or more sensors having observational perspectives in each operational sub-region <NUM> and the system is configured to: prioritize each sensor based on characteristics including a location of the operational sub-region <NUM>, time between data acquisitions, range from an anomaly in the operational sub-region <NUM>, or an observational perspective on the anomaly that is at that time located within the operational sub-region <NUM>.

According to another exemplary embodiment, the sensor arrangement can include a first sensor arrangement and a second sensor arrangement, and the computing system <NUM> can be configured to: dynamically adjust the determined priority of the first operational sub-region, adjust one or more properties of the first sensor arrangement based on the adjusted priority, and acquire data from the second sensor arrangement in a next highest priority operational sub-region <NUM>. The computing system <NUM> can be configured to adjust one or more properties of the first sensor arrangement in the first operational sub-region to eliminate observational gaps in coverage of the first operational sub-region <NUM>, establish granular observation of the first operational sub-region <NUM>, or perform a triangulation of an anomaly under observation in the first operational sub-region <NUM>. The computing system <NUM> can be configured to: identify a first sensor of the first sensor arrangement having observational perspective of the first operational sub-region <NUM> that is currently engaged in a first observation activity, and identify a second sensor in the first sensor arrangement that is available to engage in a second observation activity of the first operational sub-region <NUM>. For example, in parallel operation with sensors collecting sensor data and communicating associated data streams, the computing system <NUM> can be configured to perform resource scheduling of all available, potentially relevant dynamic sensors DS1-DSn (under its control) to reposition them in an effort to diminish temporal and geographic observational gaps, improve granularity of observation, or perform triangulation of a target under observation. This action can involve issuance of Internet Protocol (IP)-based communication to pan-tilt servo motors to "cue and slew" sensors to improve the real-time collection of data. This type of action may also require more extravagant control of robotic (autonomous) vehicles to inspect sub regions. As part of this resource scheduling, the computing system <NUM> will be aware of sensors that were previously engaged in observation activities where a dynamic sensor DSn has detected anomalistic behavior and is deemed busy or occupied in an activity. The computing system <NUM> will then search for other available sensors to support operational needs, where the system uses a methodology for managing sensor resources so that high target accuracy can be attained. The method considers available resources, the prioritization of operational sub-regions, and the prioritization of targets.

According to an exemplary embodiment of the present disclosure, the computer system <NUM> can be configured to: respond to threats by deploying resources and engaging in activities to perform predefined critical mitigation protocols such as, for example, responding to denial of service attacks, attacks that impair or stop functionality or operation of the computer system or associated devices or networks. The system <NUM> can be combined with the one or more subsystems, each of which includes an electronic warfare system, a directed energy system, or a kinetic weapon system. For example, beyond relaxing operational sub region <NUM> prioritization, computer system or edge device software may also control and automate functions of other unique systems such as Electronic Warfare technologies, Directed Energy, and kinetic weapons.

<FIG> illustrate a flow diagram of a method performed by the system of <FIG> in accordance with an exemplary embodiment of the present disclosure.

As shown in <FIG>, the computing system <NUM> receives real-time datastreams from one or more dynamic sensors (DS1-DSn) having an observational perspective of the operational area <NUM> (step <NUM>). The computing system <NUM> is configured to merge the heterogeneous data of the different sensors (and sensor types) via an analysis of the shapes of the sensor fields to triangulate, track, and target an object in the operational area <NUM> (step <NUM>). As a result of the merge, the computing system <NUM> determines whether there are any overlapping shapes of any dynamic sensors among DS1 to DSn to triangulate (step <NUM>). If there are no shapes to triangulate, the processing ends (step <NUM>). On the other hand, if there are shapes to triangulate, then the computing system <NUM> triangulates the shapes (step <NUM>). Once the shapes are processed, the computing system <NUM> can send the processed data to the communication interface <NUM> so that the data can be formatted into a control and/or data signal and communicated to one or more external systems <NUM> for situational awareness or alerting over a network <NUM> (step <NUM>). The data signal can be used to perform various operations including, among others, sensor data fusion analytics where the data signal can be encoded with real-time shapefiles which can be processed to obtain data indicating a distance of an object from sensitive locations, course/speed/heading of an object, movement behavior of an object such as natural or mechanical drive, and assessment of potential threats to an object or an operational area (step <NUM>), which can be used by an external or third party system <NUM> (step <NUM>) to react to or address the threat.

Following or in parallel with the sensor data fusion operation (step <NUM>), the computing system <NUM> determines whether there are any anomalies to interrogate (step <NUM>). If there are, then the computing system <NUM> initiates command/control operations with one or more of the dynamic sensors DS1-DSn of the sensor arrangement (step <NUM>). The command/control operation can involve the computing system <NUM> sending "cue and slew" controls to any of the dynamic sensors with, for example, pan-tilt-zoom features, so that the operational area or operational sub-regions <NUM> can be interrogated to detect distant objects. If distant objects are not detected, then the computing system <NUM> can define a 3D operational area by evenly spacing virtual boxes across a terrain flow to an operational ceiling (step <NUM>). The computing system <NUM> establishes a real-time queue in which all operational sub-regions within the operational area grid are listed to identify underserved operational sub-regions (step <NUM>). If any underserved sub-regions are identified (step <NUM>), then the computing system <NUM> initiates command/control operations as discussed above (step <NUM>). In order to undertake the command/control operations, the computing system <NUM> generates data/control signals to request additional sensor data from one or more sensors DS1-DSn in the sensor arrangement (step <NUM>). If there are no underserved sub-regions, the computing system <NUM> moves to the next box in the queue to determine if the corresponding sub-region is an underserved sub-region (step <NUM>). Steps <NUM> and <NUM> are repeated until all boxes in the list are processed.

As shown in <FIG>, computing system <NUM> can send to a dormant edge sensor DSn of the sensor arrangement a request for additional sensor data, including "cue and slew" commands to orient the sensor in a desired direction for monitoring an operational sub-region to detect an object or activity (step <NUM>). The processor of the dynamic sensor DSn determines whether a pixel morphology (e.g., comparison of each individual pixel values with neighboring pixel values) is greater than a threshold (step <NUM>). If the morphology exceeds the threshold, then the processor performs object detection within areas of detected movement by executing one or more algorithms from a defined list of hierarchical models (step <NUM>). In addition, a 3D bounding box is created around the area of pixel morphology (e.g., around the object) (step <NUM>). If an object matching any model class above a % threshold is detected (step <NUM>), then another (e.g., second) 3D bounding box is created for the object (<NUM>), object recognition is performed to identify unique identifiers of the object (step <NUM>), and pattern tracking is performed on the object for a duration of sensory observation (step <NUM>). Following object recognition at step <NUM>, it is determined whether the detected object matches any class above a percent threshold (<NUM>). If the object does match such a class, then the pattern tracking of step <NUM> is performed and another (e.g., third) 3D bounding box is generated (step <NUM>). Following the pattern tracking (step <NUM>), another (e.g., fourth) bounding box is created (step <NUM>). Once the bounding box(es) is/are created, the dynamic sensor DSn communicates 3D vertices to the computing system <NUM> over the network (step <NUM>). As shown in <FIG>, the computing system <NUM> receives the 3D vertices and performs sensor data fusion (step <NUM>).

<FIG> also shows a process in which a dynamic sensor DSn can be activated upon detection of an anomaly. The process is activated when an anomaly enters the operational area, or operational sub-region <NUM> for which the sensor has observational perspective (Step <NUM>). One or more of the dynamic sensors DS1-DSn that have an observational perspective of the operational area detect the anomaly (step <NUM>). Pixel morphology is performed on the detected anomaly and it is determined whether the morphology result is above a threshold (step <NUM>). If the morphology result is above the threshold, a 3D bounding box is generated (step <NUM>), object identification is performed using to analyze the images within areas of detected movement (step <NUM>), and pattern tracking is performed for the duration of the tracking of the anomaly (step <NUM>). The one or more algorithms used for object identification are defined by one or more hierarchical models selected from a defined list stored in memory or accessible over the network. If the result of the morphology determination is below the threshold, then anomaly detection is repeated (step <NUM>). Following object detection (step <NUM>), the dynamic sensor DSn determines whether any detected object morphology matches any model class by meeting or exceeding a predetermined percent threshold (step <NUM>). If a threshold is met or exceeded, object recognition is performed to identify any unique identifiers of the object (step <NUM>) and another (e.g., second) <NUM>) bounding box is generated. If the detected object does not meet the model class threshold, then the object detection operation is repeated (step <NUM>). Following the object recognition operation (step <NUM>), the processor of the dynamic sensor DSn determines whether the object matches any model class by meeting or exceeding a predetermined percent threshold (step <NUM>). If the predetermined percent threshold is met or exceeded, pattern tracking is performed (step <NUM>) and another (e.g., third) 3D bounding box is generated (step <NUM>). Following pattern tracking, the processor generates another (e.g., fourth) 3D bounding box. After the 3D bounding box(es) is/are created, the dynamic sensor DSn communicates 3D vertices to the computing system <NUM> over the network (step <NUM>).

The computer program code for performing the specialized functions described herein can be stored on a medium and computer usable medium, which may refer to memories, such as the memory devices for both the computing system <NUM> and edge devices <NUM>, which can be memory semiconductors (e.g., DRAMs, etc.). These computer program products can be a tangible non-transitory means for providing software to the computing system <NUM>. The computer programs (e.g., computer control logic) or software can be stored in the memory device. The computer programs can also be received via the communications interface. Such computer programs, when executed, can enable the computing system <NUM> and edge device <NUM> to implement the present methods and exemplary embodiments discussed herein. Accordingly, such computer programs may represent controllers of the computing system <NUM> and edge devices <NUM>. Where the present disclosure is implemented using software, the software can be stored in a non-transitory computer readable medium and loaded into the computing system <NUM> using a removable storage drive, an interface, a hard disk drive, or communications interface, etc., where applicable.

The one or more processors of the computing system <NUM> and the edge devices <NUM> can include one or more modules or engines configured to perform the functions of the exemplary embodiments described herein. Each of the modules or engines can be implemented using hardware and, in some instances, can also utilize software, such as program code and/or programs stored in memory. In such instances, program code may be compiled by the respective processors (e.g., by a compiling module or engine) prior to execution. For example, the program code can be source code written in a programming language that is translated into a lower level language, such as assembly language or machine code, for execution by the one or more processors and/or any additional hardware components. The process of compiling can include the use of lexical analysis, preprocessing, parsing, semantic analysis, syntax-directed translation, code generation, code optimization, and any other techniques that may be suitable for translation of program code into a lower level language suitable for controlling the computer system <NUM> or edge device <NUM> to perform the functions disclosed herein. It will be apparent to persons having skill in the relevant art that such processes result in the computer system <NUM> and/or edge device <NUM> being specially configured computing devices uniquely programmed to perform the functions discussed above.

Claim 1:
A system for monitoring an operational area (<NUM>), the system comprising:
a processor (<NUM>) configured to: receive plural data streams, each data stream including a different spatial characteristic of the operational area (<NUM>); generate a three-dimensional, 3D, virtual visualization of the operational area (<NUM>) based on observational perspectives associated with the data streams and their associated spatial characteristics;
characterized in that the processor (<NUM>) is further configured to
dynamically prioritize operational sub-regions (<NUM>, <NUM>) within the operational area (<NUM>) based on the spatial characteristics by applying a weighting criteria to each data stream according to at least a sensor (<NUM>) that generated the data stream; and generate a signal encoded with data for verifying the 3D virtual visualization of the operational area (<NUM>) including the prioritized operational sub-regions (<NUM>, <NUM>).