Parameter based load balancing in a distributed surveillance system

A distributed video management system for video surveillance that allows for monitoring a camera allocation parameter and dynamic reallocation of video cameras to available camera nodes in response to detecting a change in the allocation parameter. As such, the system may provide for load balancing of the processing of video data from the video cameras with reference to the allocation parameter. The change in allocation parameter may be due to a number of potential contexts, including a change in availability of camera nodes, a change in the nature of the video data captured, a change in computational load, or other change that results in a change in allocation parameter. The allocation parameter may be continually monitored to allocate video cameras to camera nodes in the system for enhanced system performance in view of potentially changing conditions.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is related to U.S. patent application Ser. No. 16/915,824 filed Jun. 29, 2020 entitled “SELECTIVE USE OF CAMERAS IN A SURVEILLANCE SYSTEM,” U.S. patent application Ser. No. 16/915,941 filed Jun. 29, 2020 entitled “LOW LATENCY BROWSER BASED CLIENT INTERFACE FOR A DISTRIBUTED SURVEILLANCE SYSTEM,” U.S. patent application Ser. No. 16/915,905 filed Jun. 29, 2020 entitled “DISTRIBUTED SURVEILLANCE SYSTEM WITH ABSTRACTED FUNCTIONAL LAYERS,” and U.S. patent application Ser. No. 16/915,990 filed Jun. 29, 2020 entitled “DISTRIBUTED SURVEILLANCE SYSTEM WITH DISTRIBUTED VIDEO ANALYSIS,” all of which are filed concurrently herewith and are specifically incorporated by reference for all that they disclose and teach.

BACKGROUND

Video surveillance systems are valuable security resources for many facilities. In particular, advances in camera technology have made it possible to install video cameras in an economically feasible fashion to provide robust video coverage for facilities to assist security personnel in maintaining site security. Such video surveillance systems may also include recording features that allow for video data to be stored. Stored video data may also assist entities in providing more robust security, allowing for valuable analytics, or assisting in investigations. Live video data feeds may also be monitored in real-time at a facility as part of facility security.

While advances in video surveillance technology have increased the capabilities and prevalence of such systems, a number of drawbacks continue to exist that limit the value of these systems. For instance, while camera technology has drastically improved, the amount of data generated by such systems continues to increase. This creates a problem of how to effectively store large amounts of video data in a way that allows for easy retrieval or other processing. In turn, effective management of video surveillance data has become increasingly difficult.

Proposed approaches for the management of video surveillance systems include the use of a network video recorder to capture and store video data or the use of an enterprise server for video data management. As will be explained in greater detail below, such approaches each present unique challenges. Accordingly, the need continues to exist for improved video surveillance systems with robust video data management and access.

SUMMARY

The present disclosure generally relates to a distributed video surveillance system that includes distributed processing resources capable of processing and/or storing video data from a plurality of video cameras at a plurality of camera nodes. One particular aspect of the present disclosure includes the adaptive allocation of video cameras to camera nodes for processing the video data in relation to an allocation parameter. In this regard, the allocation of video cameras to the camera nodes in the system may provide load balancing of the allocation parameter. Furthermore, the dynamic or adaptive allocation of cameras may provide robustness to the system in view of potential changes in the allocation parameter.

Accordingly, a first aspect of the present disclosure includes a method for processing video data in a video surveillance system. The method includes capturing video data at a plurality of video cameras and communicating the video data from the plurality of video cameras to at least one camera node over a communication network according to a first camera allocation configuration. A video processing module is executed at a camera node to process the video data received at the camera node. The method also includes monitoring a camera allocation parameter at the camera node. The camera allocation parameter is at least in part based on the video data received at one or more camera nodes. The method also includes modifying the first camera allocation configuration to a second camera allocation configuration different than the first camera allocation configuration, at least in part based on the camera allocation parameter.

Another aspect of the present disclosure includes a video surveillance system that includes a plurality of video cameras operative to capture video data and at least one camera node operative to receive video data from one or more video cameras of the plurality of video cameras according to a first camera allocation configuration. The system also includes a video processing module executing at the one or more camera nodes to process the video data received at those camera nodes. A master node is in operative communication with the at least one camera node to monitor a camera allocation parameter at least in part based on the video data received at the at least one camera node. The master node is operative to, in response to a change in the camera allocation parameter, modify the first camera allocation configuration to a second camera allocation configuration different than the first camera allocation configuration.

DETAILED DESCRIPTIONS

While the examples in the following disclosure are susceptible to various modifications and alternative forms, specific examples are shown in the drawings and are herein described in detail. It should be understood, however, that it is not intended to limit the scope of the disclosure to the particular form disclosed, but rather, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope defined by the claims.

FIG. 1depicts two prior art approaches for the system architecture and management of a video surveillance system. The two approaches include an appliance-based system1shown in the top portion ofFIG. 1and an enterprise server-based approach 20 in the bottom portion ofFIG. 1. In the appliance-based system1, video cameras10are in operative communication with a network15. An appliance12is also in communication with the network15. The appliance12receives video data from the video cameras10and displays the video data on a monitor14that is connected to the appliance12.

Appliance based systems1generally provide a relatively low-cost solution given the simplicity of the hardware required to implement the system1. However, due to the limited processing capability of most appliances12, the number of cameras that are supported in an appliance-based system may be limited as all video cameras10provide video data exclusively to the appliance12for processing and display on the monitor14. Moreover, the system is not scalable, as once the processing capacity of the appliance12has been reached (e.g., due to the number of cameras in the system1), no further expansion of additional cameras may be provided. Instead, to supplement a system1, an entirely new appliance12must be implemented as a separate, stand-alone system without integration with the existing appliance12. Also, due to the relatively limited processing capacity of the appliance12, appliance-based systems1provide a limited capability for video data analytics or storage capacity. Additionally, such systems1typically facilitate viewing and/or storage of a limited number of live video data feeds from the video cameras10at any given time and usually allow the presentation of such video only on a single monitor14or a limited number of monitors connected to the appliance12. That is, to review real-time or archived video data, a user must be physically present at the location of the appliance12and monitor14.

Enterprise server-based systems20typically include a plurality of video cameras10in operative communication with a network15. A server instance16is also in communication with the network15and receives all video data from all the video cameras10for processing and storage of the data. The server16usually includes a storage array and acts as a digital video recorder (DVR) to store the video data received from the video cameras10. A client18may be connected to the network15. The client18may allow for the viewing of video data from the server16away from the physical location of the server16(e.g., in contrast to the appliance-based system1in which the monitor14is connected directly to the appliance12). However, the server16typically includes platform-dependent proprietary software for digesting video data from the cameras10for storage in the storage array of the server16.

Furthermore, the server16and client18include platform-dependent proprietary software to facilitate communication between the server16and the client18. Accordingly, a user or enterprise must purchase and install the platform-dependent client software package on any client18desired to be used to access the video data and/or control the system20. This limits the ability of a user to access video data from the system20as any user must have access to a preconfigured client18equipped with the appropriate platform-dependent proprietary software, which requires licensing such software at an additional cost.

In contrast to the appliance-based systems1, enterprise server-based systems20are usually relatively expensive implementations that may be targeted to large-scale enterprise installations. For example, such systems20typically require very powerful servers16to facilitate the management of the video data from the cameras10as a single server16handles all processing and storage of all video data from the system. Also, the platform-dependent proprietary software for the server16and clients18require payment of license fees that may be based on the number of cameras10and/or the features (e.g., data analytics features) available to the user. Further still, the proprietary software to allow the functionality of the client18must be installed and configured as a stand-alone software package. In turn, the installation and maintenance of the software at the client18may add complexity to the system1. Further still, in the event a user wishes to use a different client18device, any such device must first be provisioned with the necessary software resources to operate. Thus, the ability to access and manage the system1is limited.

While such an enterprise server-based system20may be scaled, the capital cost of expansion of the system20is high. Specifically, the server16, despite the increased computational complexity relative to an appliance12, does have a limit on the number of cameras10it may support, although this limit is typically higher than the number of cameras10an appliance12can support. In any regard, once the maximum number of cameras10is reached, any additional camera10requires the purchase of, in effect, a new system20with an additional server16or by increasing the capacity of the server16along with increase payment of licensing fees for the additional server16or capacity. Furthermore, the proprietary software that is required to be installed at the client18is typically platform-dependent and needed for any client18wishing to interact with the system20. This adds complexity and cost to any client18and limits the functionality of the system20. Further still, enterprise server-based systems20include static camera-to-server mappings such that in the event of a server unavailability or failure, all cameras10mapped to the server16that fails become unavailable for live video streams or storage of video data, thus rendering the system20ineffective in the event of such a failure.

Accordingly, the present disclosure relates to a distributed video management system (VMS)100that includes a distributed architecture. One example of such a VMS100is depicted inFIG. 2. The distributed architecture of the VMS100facilitates a number of benefits over an appliance-based system1or a server-based system20described above. In general, the VMS100includes three functional layers that may be abstracted relative to one another to provide the ability to dynamically reconfigure mappings between video cameras110, camera nodes120for processing the video data, and storage capacity150/152within the VMS100. While this is discussed in greater detail below, the abstraction of the functional layers of the VMS100facilitates a highly dynamic and configurable system that is readily expandable, robust to component failure, capable of adapting to a given occurrence, and cost-effective to install and operate. Because the functional layers are abstracted, static component-to-component mappings need not be utilized. That is, any one or more video cameras110can be associated with any one of a plurality of camera nodes120that may receive the video data from associated cameras110for processing of the video data from the associated cameras110. In turn, the camera nodes120process the video data (e.g., either for storage in a storage volume150/152or for real-time streaming to a client device130for live viewing of the video data). Camera nodes110may be operative to execute video analysis on the video data of associated cameras110or from stored video data (e.g., of an associated video camera100or a non-associated video camera110). Further still, as the storage resources of the system100are also abstracted from the camera nodes120, video data may be stored in a flexible manner that allows for retrieval by any of the camera nodes120of the system.

In this regard, upon failure of any given node in the system, cameras assigned to the failed camera node may be reassigned (e.g., automatically) to another camera node such that processing of the video data is virtually uninterrupted. Also, camera-to-node associations may be dynamically modified in response to actual processing conditions at a node (e.g., cameras may be associated to another node from a node performing complex video analysis) Similarly, as the camera nodes120may be relatively inexpensive hardware components, additional camera nodes120may be easily added (e.g., in a plug-and-play fashion) to the system100to provide highly granular expansion capability (e.g., versus having to deploy entire new server instances in the case of the server-based system20that only offer low granularity expansion).

The flexibility of the VMS system100extends to clients130in the system. The clients130may refer to a client device or software delivered to a device to execute at the device. In any regard, a client130may be used to view video data of the VMS100(e.g., either in real-time or from storage150/152of the system100). Specifically, the present disclosure contemplates the use of a standard web browser application commonly available and executable on a wide variety of computing devices. As described in greater detail below, the VMS100may utilize processing capability at each camera node120to process video data into an appropriate transport mechanism, which may be at least in part based on a context of a request for video data. As an example, a request from a client130for viewing of live video data in real-time from a camera110may result in a camera node120processing the video data of a camera110into a real-time, low latency format for delivery to the client130. Specifically, such a low latency protocol may include a transport mechanism that allows the data to be received and rendered at the client using a standard web browser using only native capability of the standard web browser or via executable instructions provided by a web page sent to the client130for rendering in the standard web browser (e.g., without requiring the installation of external software at the client in the form of third-party applications, browser plug-ins, browser extensions, or the like). In turn, any computing device executing a standard web browser may be used as a client130to access the VMS100without requiring any proprietary or platform-dependent software and without having any pre-configuration of the client130. This may allow for access by any computing system operating any operating system so long as the computing device is capable of executing a standard web browser. As such, desktop, laptop, tablets, smartphones, or other devices may act as a client130.

The abstracted architecture of the VMS100may also allow for flexibility in processing data. For instance, the camera nodes120of the VMS100may apply analytical models to the video data processed at the camera node120to perform video analysis on the video data. The analytical model may generate analytical metadata regarding the video data. Non-limiting examples of analytical approaches include object detection, object tracking, facial recognition, pattern recognition/detection, or any other appropriate video analysis technique. Given the abstraction between the video cameras110and the camera nodes120of the VMS100, the configuration of the processing of the video data may be flexible and adaptable, which may allow for the application of even relatively complex analytical models to some or all of the video data with dynamic provisioning in response to peak analytical loads.

With continued reference toFIG. 2, a VMS100for management of edge surveillance devices in a surveillance system according to the present disclosure is depicted schematically. The VMS100includes a plurality of cameras110that are each in operative communication with a network115. For example, as shown inFIG. 2, cameras110athrough110gare shown. However, it should be understood that additional or fewer cameras may be provided in a VMS100according to the present disclosure without limitation.

The cameras110may be internet protocol (IP) cameras that are capable of providing packetized video data from the camera110for transport on the network115. The network115may be a local area network (LAN). In other examples, the network115may be any appropriate communication network including a publicly-switched telephone network (PSTN), intranet, wide area network (WAN) such as the internet, digital subscriber line (DSL), fiber network, or other appropriate networks without limitation. The video cameras110may each be independently associable (e.g., assignable) to a given one of a plurality of camera nodes120.

As such, the VMS100also includes a plurality of camera nodes120. For example, inFIG. 2, three camera nodes120are shown, including a first camera node120a, a second camera node120b, and a third camera node120c. However, it should be understood that additional or fewer camera nodes120may be provided without departing from the scope of the present disclosure. Furthermore, camera nodes120may be added to or removed from the system100at any time, in which case, camera-to-node assignments or mappings may be automatically reconfigured. Each of the camera nodes120may also be in operative communication with the network115to facilitate receipt of video data from the one or more of the cameras110associated with each respective node120.

The VMS100also includes at least one master node140. The master node140may be operative to manage the operation and/or configuration of the camera nodes120to receive and/or process video data from the cameras110, coordinate storage resources of the VMS100, generate and maintain a database related to captured video data of the VMS100, and/or facilitate communication with a client130for access to video data of the system100.

While a single master node140is shown and described, the master node140may comprise a camera node120tasked with certain system management functions. Not all management functions of the master node140need to be executed by a single camera node120. In this regard, while a single master node140is described for simplicity, it may be appreciated that the master node functionality described herein in relation to a single master node140may actually be distributed among different ones of the camera nodes120. As such, a given camera node120may act as the master node140for coordination of camera assignments to the camera nodes120, while another camera node120may act as the master node140for maintaining the database regarding the video data of the system. Accordingly, as will be described in greater detail below, various management functions of the master node140may be distributed among various ones of the camera nodes120. Accordingly, while a single given master node140is shown, it may be appreciated that any one of the camera nodes120may act as a master node140for different respective functions of the system100.

Furthermore, the various management functions of the master node140may be subject to leader election to allocate such functions to different ones of the camera nodes120for the execution of the master node functionality. For example, the role of master node140may be allocated to a given camera node120using leader election techniques such that all management functions of the master node140are allocated to a given camera node120. Alternatively, individual ones of the management functions may be individually allocated to one or more camera nodes120using leader election. This provides a robust system in which even the unavailability of a master node140or a camera node120executing some management functions can be readily corrected by applying leader election to elect a new master node140in the system or to reallocate management functionality to a new camera node120.

The hardware of the camera node120and the master node140may be the same. In other examples, a dedicated master node140may be provided that may have different processing capacity (e.g., more or less capable hardware in terms of processor and/or memory capacity) than the other camera nodes120. Furthermore, not all camera nodes120may include the same processing capability. For instance, certain camera nodes120may include increased computational specifications relative to other camera nodes120, including, for example, increased memory capacity, increased processor capacity/speed, and/or increased graphical processing capability.

As may be appreciated, the VMS100may store video data from the video cameras110in storage resources of the VMS100. In one implementation, storage capacity may be provided in one or more different example configurations. Specifically, in one example, each of the camera nodes120and/or the master node140may have attached storage152at each respective node. In this regard, each respective node may store the video data metadata processed by the node and any metadata generated at the node at the corresponding attached storage152at each respective node for video data processed at the node120. In an alternative arrangement, the locally attached storage152at each of the camera nodes120and the master node140may comprise physical drives that are abstracted into a logical storage unit150. In this regard, it may be that video data processed at a first one of the nodes may be, at least in part, communicated to another of the nodes for storage of the data. In this regard, the logical storage unit150may be presented as an abstracted storage device or storage resource that is accessible by any of the nodes120of the system100. The actual physical form of the logical storage unit150may take any appropriate form or combination of forms. For instance, the physical drives associated with each of the nodes may comprise a storage array such as a RAID array, which forms a single virtual volume that is addressable by any of the camera nodes120or the master node140. Additionally or alternatively, the logical storage unit150may be in operative communication with the network115with which the camera nodes120and master node140are also in communication. In this regard, the logical storage unit150may comprise a network-attached storage (NAS) device capable of receiving data from any of the camera nodes120. The logical storage unit150may include storage devices local to the camera nodes120or may comprise remote storage such as a cloud-based storage resource or the like. In this regard, while a logical storage unit150and locally attached storage152are both shown inFIG. 2, the locally attached storage152may comprise at least a portion of the logical storage unit150. Furthermore, the VMS100need not include both types of storage, which are shown inFIG. 2for illustration only.

With further reference toFIG. 3, a schematic drawing illustrating an example of a master node140is shown. The master node140may include a number of modules for management of the functionality of the VMS100. As described above, while a single master node140is shown that comprises the master node modules, it should be appreciated that any of the camera nodes120may act as a master node140for any individual functionality of the master node modules. That is, the role of the master node140for any one or more of the master node functionalities may be distributed among the camera nodes120. In any regard, the modules corresponding to the master node140may include a web server142, a camera allocator144, a storage manager146, and/or a database manager148. In addition, the master node140may include a network interface126that facilitates communication between the master node140and video cameras110, camera nodes120, storage150, a client130, or other components of the VMS100.

The web server142of the master node140may coordinate communication with a client130. For example, the web server142may communicate a user interface (e.g., HTML code that defines how the user interface is to be rendered by the browser) to a client130, which allows a client130to render the user interface in a standard browser application. The user interface may include design elements and/or code for retrieving and displaying video data from the VMS100in a manner that is described in greater detail below.

With respect to the camera allocator144, the master node140may facilitate camera allocation or assignment such that the camera allocator144creates and enforces camera-to-node mappings to determine which camera nodes120are tasked with processing video data from the video cameras110. That is, in contrast to the appliance-based system1or the enterprise server-based system50, subsets of the video cameras110of the VMS100may be assigned to different camera nodes120. For instance, the camera allocator144may be operative to communicate with a video camera110to provide instructions to the video camera110regarding a camera node120the video camera110is to send its video data. Alternatively, the camera allocator144may instruct the camera nodes120to establish communication with and receive video data from specific ones of the video cameras110. The camera allocator144may create such camera-to-node associations and record the same in a database or other data structure. In this regard, the system100may be a distributed system in that any one of the camera nodes120may receive and process video data from any one or more of the video cameras110.

Furthermore, the camera allocator144may be operative to dynamically reconfigure the camera-to-node mappings in a load balancing process. In this regard, the camera allocator144may monitor an allocation parameter at each camera node120to determine whether to modify the camera-to-node mappings. In this regard, changes in the VMS100may be monitored, and the camera allocator144may be responsive to modify a camera allocation from a first camera allocation configuration to a second camera allocation configuration to improve or maintain system performance. The allocation parameter may be any one or more of a plurality of parameters that are monitored and used in determining camera allocations. Thus, the allocation parameter may change in response to a number of events that may occur in the VMS100as described in greater detail below.

For example, in the event of a malfunction, power loss, or another event that results in the unavailability of a camera node120, the camera allocator144may detect or otherwise be notified of the unavailability of the camera node. In turn, the camera allocator144may reassign video cameras previously associated with the unavailable node to another node120. The camera allocator144may communicate with the reassigned cameras110to update the instructions for communication with the new camera node120. Alternatively, the newly assigned camera node may assume the role of establishing contact with and processing video data from the video cameras110that were previously in communication with the unavailable camera node120to update the instructions and establish the new camera-to-node assignment based on the new assignment provided by the camera allocator144. In this regard, the system100provides increased redundancy and flexibility in relation to processing video data from the cameras100. Further still, even in the absence of a camera node120failure, the video data feeds of the cameras110may be load balanced to the camera nodes120to allow for different analytical models or the like to be applied.

A given camera node120may be paired with a subset of the cameras110that includes one or more of the cameras110. As an example, inFIG. 2, cameras110a-110cmay be paired with camera manager120asuch that the camera manager120areceives video data from cameras110a-110c. Cameras110d-110fmay be paired with camera manager120bsuch that the camera manager120breceives video data from cameras110d-110f. Camera110gmay be paired with camera manager120csuch that the camera manager120creceives video data from camera100g. However, this configuration could change in response to a load balancing operation, a failure of a given camera node, network conditions, or any other parameter.

For instance, and with reference toFIG. 8, a first camera allocation configuration is shown. Two camera nodes, camera node120aand camera node120b, may process data from video cameras110a-110evia a network115.FIG. 9is a schematic representation presented for illustration. As such, while the cameras110are shown as being in direct communication with the nodes120, the cameras110may communicate with the nodes120via a network connection. Similarly, while the master node140is shown as being in direct communication with the camera nodes120, this communication may also be via a network115(not shown inFIG. 8). In any regard, in the first camera allocation configuration shown inFIG. 8, video camera110a, video camera110b, and video camera110ccommunicate video data to a first camera node120afor processing and/or storage of the video data by the first camera node120a. Also, video camera110dand video camera110ecommunicate video data to a second camera node120bfor processing and/or storage of the video data by the first camera node120b. The first camera allocation may be established by a camera allocator144of the master node140in a manner that distributes the mapping of the video cameras110among the available camera nodes120to balance the allocation parameter among the camera nodes120.

Upon detection of a change in the allocation parameter, the camera allocator144may modify the first camera allocation in response to the detecting a change in the monitored allocation parameter. Such a change may, for example, be in response to the addition or removal of a camera node120from the VMS100, upon a change in computational load at a camera node120, upon a change in video data from a video camera110, or any other change that results in a change in the allocation parameter. For instance, with further reference toFIG. 9, a scenario is depicted in which camera node120bbecomes unavailable (e.g., due to loss of communication at the camera node120b, loss of power at the camera node120b, or any other malfunction or condition that results in the camera node120blosing the ability to process and/or store video data). In response, the master node140may detect such a change and modify the first camera allocation configuration from that shown inFIG. 8to a second camera allocation configuration, as shown inFIG. 9.

In the second camera allocation configuration shown inFIG. 9, all cameras110a-110eare mapped to communicate with the camera node120a. However, it should be appreciated that other camera nodes120(not shown inFIG. 9) could also have one or more of video camera110dand video camera110eallocated to any available node120in the VMS100. As such, the two camera nodes120aand120bare shown for simplicity of explanation only. In this regard, the modification of the camera allocation configuration may be at least in part based on the allocation parameter. That is, the camera allocation parameter may be used to load balance the system (e.g., based on the allocation parameter) based on the video data of the cameras110across all available camera nodes120. Thus, while all cameras110are reallocated to the first camera node120ainFIG. 9, cameras110dand110ecould be otherwise allocated to alternative camera nodes to balance the computational and storage load or other allocation parameters across all available nodes120.

Also, while a camera node120is shown as becoming unavailable inFIG. 9, another scenario in which load balancing may occur is the addition of one or more camera nodes120to the system such that one or more additional camera nodes may become available. In this scenario, a new camera allocation configuration may be generated to balance the video data processing of all cameras110in the VMS100with respect to an allocation parameter based on the video data generated by the cameras110. In this regard, it may be appreciated that a change in the allocation parameter monitored by the camera allocator144of the master node140may occur in response to any number of conditions, and this change may result in a modification of an existing camera allocation configuration.

As such, the allocation parameter may relate to the video data of the camera nodes110being allocated. The allocation parameter may, for example, relate to a time-based parameter, the spatial coverage of the cameras, a computational load of processing the video data of a camera, an assigned class of camera, an assigned priority of a camera. The allocation parameter may be at least in part affected by the nature of the video data of a given camera. For instance, a given camera may present video data that is more computationally demanding than another camera. For instance, a first camera may be directed at a main entrance of a building. A second camera may be located in an internal hallway that is not heavily trafficked. Video analysis may be applied to both sets of video data from the first camera and the second camera to perform facial recognition. The video data from the first camera may be more computationally demanding on a camera node than the video data from the second camera simply by virtue of the nature/location of the first camera being at the main entrance and including many faces compared to the second camera. In this regard, the camera allocation parameter may be at least in part based on the video data of the particular cameras to be allocated to the camera nodes.

In this regard,FIG. 10depicts another scenario in which a change in a camera allocation parameter is detected, and the camera allocation configuration is modified in response to the change.FIG. 10may modify a first camera allocation configuration fromFIG. 8to a second camera allocation configuration shown inFIG. 10. InFIG. 10, video camera110emay begin to capture video data that results in a computational load on camera module120bincreasing beyond a threshold. In turn, the camera allocator144of the master node140may detect this change and modify the first camera allocation configuration to the second camera allocation configuration such that camera110dis associated with camera node120a. That is, camera node120bmay be exclusively dedicated to processing video data from camera110ein response to a change in the video that increases the computational load for processing this video data. Examples could be the video data including significantly increased detected objects (e.g., additional faces to be processed using facial recognition) or motion that is to be processed. In this example shown inFIG. 10, camera node120amay have sufficient capacity to process the video data from camera110d.

FIG. 11further illustrates an example in which a total computational capacity of the VMS100based on the available camera nodes120is exceeded. In the scenario depicted inFIG. 11, a camera110dmay be disconnected from any camera node120such that the camera110dmay not have its video data processed by the VMS100. That is, cameras may be selectively “dropped” if the overall VMS100capacity is exceeded. The cameras may have a priority value assigned, which may in part be based on an allocation parameter as described above. For instance, if two cameras are provided that have overlapping spatial coverage (e.g., one camera monitors an area from a first direction and another camera monitors the same area but from a different direction), one of the cameras having overlapping spatial coverage may have a relatively low priority. In turn, upon disconnection of one of the cameras, continuity of monitoring of the area covered by the cameras may be maintained, while reducing the computational load of the system. Upon restoration of available computational load (e.g., due to a change in the computational load of other cameras or by adding another node to the system), the disconnected camera may be reallocated to a camera node using a load-balanced approach. In other contexts, other allocation parameters may be used to determine priority, including establishing classes of cameras. For instance, cameras may be allocated to an “internal camera” class or a “periphery camera” class based on a location/field of view of cameras being internal to a facility or external to a facility. In this case, one class of cameras may be given priority over the other class based on a particular scenario occurring which may either relate to the VMS100(e.g., a computational capacity/load of the VMS100) or an external occurrence (e.g., an alarm at the facility, shift change at a facility, etc.).

The master node140may also comprise a storage manager146. Video data captured by the cameras110is processed by the camera nodes120and may be stored in persistent storage once processed. The video data generated by the VMS100may include a relatively large amount of data for storage. Accordingly, the VMS100may generally enforce a storage policy for the video data captured and/or stored by the VMS100. As will be described in greater detail below, abstracted storage resources of the VMS100facilitate persistent storage of video data by the camera nodes120in a manner that any camera node120may be able to access stored video data regardless of the camera node120that processed the video data. As such, any of the camera nodes120may be able to retrieve and reprocess video data according to the storage policy.

For instance, the storage policy may instruct that video data of a predefined currency (e.g., video data captured within the last 24 hours of operation of the VMS100) may be stored in its entirety at an original resolution of the video data. However, long term storage of such video data at full resolution and frame rate may be impractical or infeasible. As such, the storage policy may include an initial period of full data retention in which all video data is stored in full resolution and subsequent treatment of video data after the initial period to reduce the size of the video data on disk.

To this end, the storage policy may dictate other parameters that control how video data is to be stored or whether such data is to be kept. The storage manager146may enforce the storage policy based on the parameters of the storage policy with respect to stored video data. For instance, based on parameters defined in the storage policy, video data may be deleted or stored in a reduced size (e.g., by reducing video resolution, frame rate, or other video parameters to reduce the overall size of the video data on disk). The reduction of the size of the stored video data on disk may be referred to as “pruning.” One such parameter that governs pruning of the video data may relate to the amount of time that has elapsed since the video data was captured. For instance, data older than a given period (e.g., greater than 24 hours) may be deleted or reduced in size. Further still, multiple phases of pruning may be performed such that the data is further reduced in size or deleted as the video becomes less current.

Also, because any camera node120may be operative to retrieve any video data from storage for reprocessing, video data may be reprocessed (e.g., pruned) by a camera node different than the camera node that initially processed and stored the video data from a video camera. As such, reprocessing or pruning may be performed by any camera node120. The reprocessing of video data by a camera node may be performed during idle periods for a camera node120or when a camera node120is determined to have spare computational capacity. This may occur at different times for different camera nodes but may occur during times of low processing load, such as after business hours or during a time in which a facility is closed or has reduced activity.

Still further, a parameter for pruning may relate to analytical metadata of the video data. As described in greater detail elsewhere in the present application, a camera node120may include an analytical model to apply video analysis to video data processed by a camera module. Such video analysis may include the generation of analytical metadata regarding the video. For example, the analytical model may include object detection, object tracking, facial recognition, pattern detection, motion analysis, or other data that is extracted from the video data upon analysis using the analytical model. The analytical metadata may provide a parameter for data pruning. For instance, any video data without motion may be deleted after an initial retention period. In another example, only video data comprising particular analytical metadata may be retained (e.g., only video data in which a given object is detected may be stored). Further still, only data from specific cameras110may be retained beyond an initial retainer period. Thus, a highly valuable video data feed (e.g., video data related to a critical location such as a building entrance or a highly secure area of a facility) may be maintained without a reduction in size. In any regard, the storage manager146may manage the application of such a storage policy to the video data stored by the VMS100.

The master node140may also include a database manager148. As noted above, video cameras110may be associated with any camera node120for processing and storage of video data from the video camera120. Also, video data may be stored in an abstracted manner in a logical storage unit150that may or may not be physically co-located with a camera node120. As such, the VMS100may beneficially maintain a record regarding the video data captured by the VMS100to provide important system metadata regarding the video data. Such system metadata may include, among other potential information, which video camera110captured the video data, a time/date when the video data was captured, what camera node120processed the video data, what video analysis was applied to the video data, resolution information regarding the video data, framerate information regarding the video data, size of the video data, and/or where the video data is stored. Such information may be stored in a database that is generated by the database manager148. The database may include correlations between the video data and the system metadata related to the video data. In this regard, the provenance of the video data may be recorded by the database manager148and captured in the resulting database. The database may be used to manage the video data and/or track the flow of the video data through the VMS100. For example, the storage manager146, as discussed above, may utilize the database for the application of a storage policy to the data. Furthermore, requests for data from a client130may include reference to the database to determine a location for video data to be retrieved for a given parameter such as any one or more metadata portions described above. The database may be generated by the database manager148, but the database may be distributed among all camera nodes120to provide redundancy to the system in the event of a failure or unavailability of the master node140executing the database manager148. Database updates corresponding to at any given camera node120may be driven by specific events or may occur at a pre-determined time interval.

The database may further relate video data to analytical metadata regarding the video data. For instance, as described in greater detail below, analytical metadata may be generated by the application of a video analysis to the video data. Such analytical metadata may be embedded in the video data itself or be provided as a separate metadata file associated with a given video data file. In either regard, the database may relate such analytical metadata to the video data. This may assist in pruning activities or in searching for video data. Concerning the former, as described above, pruning according to a storage policy may include the treatment of video data based on the analysis metadata (e.g., based on the presence or absence of movement or detected objects). Furthermore, a search by a user may request all video data in which a particular object is detected or the like.

With further reference toFIG. 4, a schematic example of a camera node120is shown. As can be appreciated from the foregoing, the camera node120may include an instance of the database132provided by the master node140executing the database manager148. In this regard, the camera node120may reference the database for retrieval and/or serving of video from the logical storage volume of the VMS100and/or for reprocessing video data (e.g., according to a storage policy).

The camera node120may include a video analysis module128. The video analysis module128may be operative to apply an analytic model to the video data processed by the camera node120once received from a camera110. The video analysis module128may apply a machine learning model to the video data processed at the camera node120to generate analytics metadata. For instance, as referenced above, the video analytics module128may apply a machine learning model to detect objects, track objects, perform facial recognition, or other analytics of the video data, which in turn may result in the generation of analytics metadata regarding the video data.

The camera node120may also comprise modules adapted for processing video data into an appropriate transport mechanism based on the nature of the data or the intended use of the data. In this regard, the camera node120includes a codec122(i.e., an encoder/decoder) that may decode received data and re-encode the data into a different encoded video format. The encoded video format may include packetized data such that each packet of data is encoded according to a selected encoded video format. The camera node120may also include a container formatter124that may package the encoded video packets into an appropriate container format. The camera module120further includes a network interface126that is operative to determine a communication protocol for the transfer of the encoded video packets in the digital container format.

The formatting of the video data into an appropriate transport mechanism may allow for optimized delivery and/or storage of video data. For instance, the video data may be delivered from the camera110to the camera node120using a real-time streaming protocol (RTSP). However, RTSP may not be an optimal protocol for storage and/or delivery of video data to a client130(e.g., RTSP is typically not supported by a standard web browser and, thus, usually requires specific software or plug-ins such as a particular video player to render video in a browser display). The camera node120may reformat the video data into an appropriate transfer mechanism based on the context in which the video data is requested.

Upon selection of an appropriate communication protocol, the network interface126may communicate the encoded video packets to a standard web browser at a client device using the communication protocol. In one example, a client130may request to view video data from a given video camera110in real-time. As such, an appropriate encoded video format, container format, and communication protocol may be selected by the codec122, container formatter124, and network interface126, respectively, to facilitate a transport mechanism for serving the video data to the client130in real-time. In contrast, a client130may alternatively request video data from the logical storage unit of the VMS100. As can be appreciated, the currency of such data is not as important as in the context of real-time data. A different one or more of the encoded video format, container format, and communication protocol may be selected. For example, in such a context in which the currency of the data is of less importance, a more resilient or more bandwidth-efficient encoded video format, container format, and communication protocol may be selected that has a higher latency for providing video to the client130.

For purposes of illustration and not limitation, the transport mechanism may comprise any combination of encoded video format, container format, and communication protocol. Example transport mechanisms include JSMpeg, HTTP Live Streaming (HLS), MPEG-1, and WebRTC. JSMpeg utilizes MPEG-1 encoding (e.g., a MPEG-TS Demuxer, WebAssembly MPEG-1 Video & MPEG-2 Audio Decoders). In this regard, the JSMpeg transport mechanism uses Transport Stream (TS) container formatting and WebSocket communication protocol. In turn, the JSMpeg transport mechanism may be decoded at the client130using the JSMpeg program, which may be included in the web page (e.g., the HTML code or the like sent to the browser) and not require the use of a plug-in or other application other than the native web browser. For example, the JSMpeg transport mechanism may use WebGL & Canvas2D Renderers and WebAudio Sound Output. The JSMpeg transport mechanism may provide very low latency to the video data but utilizes somewhat higher bandwidth consumption relative to the other transport mechanisms described herein.

Another transport mechanism may be WebRTC, which may utilize an H.264 encoding, VP8, or another encoding. WebRTC may utilize a container format comprising MPEG-4 or WebM. The communication protocol for WebRTC may include an RTC peer connection to provide signaling. Video may be delivered using Web Socket. In the WebRTC transport mechanism, the standard browser may comprise a native decoder for decoding the encoded video data. WebRTC provides very low latency to the video data but increases the complexity of the system by utilizing the signaling server in the form of the RTC peer connection. However, the bandwidth usage of WebRTC is relatively low.

Yet another transport mechanism that may be utilized comprises HLS or MPEG-DASH. The encoded video format for HLS/MPEG-DASH may be MPEG-2, MPEG-4, or H.264. The container format may be MPEG-4, and the communication protocol may be HTTP. In this regard, the decoder may decode the encoded video data natively. The HLS/MPEG-DASH transport mechanism has higher latency than the other transport mechanisms described but has robust browser support and low network bandwidth usage.

As mentioned above, the VMS100may comprise an abstracted system that allows for the capture of video data, processing of the video data, and the storage of video data to be abstracted among various components of the VMS100. For example, with further reference toFIG. 4, three “layers” of functionality of the VMS100are schematically described. Specifically, an acquisition layer310, a processing layer320, and a storage layer330are shown. The cameras110may comprise the acquisition layer310. The camera nodes120and master node140may comprise the processing layer320. In addition, a logical storage volume may comprise the storage150of the storage layer330. The layers are referred to as abstracted layers because the particular combination of hardware components that acquire, process, and store the video data of the VMS system100may be variable and dynamically associated. That is, network communication among the hardware components of the VMS100may allow each of the acquisition, processing, and storage functions to be abstracted. Thus, for example, any one of the cameras110may provide video data to any one of the camera nodes120, which may store the video data in the logical storage volume of the storage150without limitation.

As described above, the VMS100also includes a client130that may be in operative communication with the network115. The client130may be operative to communicate with the VMS100to request and receive video data from the system100. In this regard, the VMS100may both store video data from the video cameras110as well as provide a real-time stream of video data for observation by one or more users. For example, video surveillance cameras are often monitored in real-time by security personnel. By “real-time” or “near real-time,” it is intended that the data provided have sufficient currency for security operations. In this regard, real-time or near real-time does not require instantaneous delivery of video data but may include delays that do not affect the efficacy of monitoring of the video data such as delays of less than 5 seconds, less than 3 seconds, or less than about 1 second.

One objective of the present disclosure is to facilitate a client130that may present real-time video data to a user in a convenient manner using a standard web browser application. Of particular note, it is particularly beneficial to allow the client130to execute commonly available and low-cost applications for access to the video data (e.g., in contrast to requiring platform-dependent proprietary software be preinstalled and preconfigured to interact with a management system). In this regard, a particular application type contemplated for utilization at a client130is a standard web browser. Examples of such browsers include Google Chrome, Mozilla Firefox, Microsoft Edge, Microsoft Internet Explorer, the Opera browser, and/or Apple Safari. Such standard web browsers are capable of natively processing certain data received via a network for the generation of a user interface at a client device. For instance, such standard web browsers often include native application programming interface (APIs) or other default functionality to allow the web browser to render user interfaces, facilitate user interaction with a web site or the like, and establish communication between the client and a server.

The client130may comprise a standard internet browser that is capable of communication with the web server142and/or one or more of the camera managers120to access the video data of the VMS100. In contrast to previously proposed systems that rely on proprietary client software to be executed to communicate with a server for retrieval of video data, the client130of the VMS100may use any standard web browser application to access the video data. By standard internet browser application, it is meant that the browser application may not require any plug-in, add on, or another program to be installed or executed by the browser application other than the functionalities that are natively provided in the browser. It should be noted that certain functionality regarding a user interface for searching, retrieving, and displaying video may be delivered to the web browser by the web server142as code or the like, but any such functionality may be provided without user interaction or pre-configuration of the web browser. Accordingly, any such functionality is still deemed to be a native functionality of the web browser. In this regard, the client130may receive all necessary data to facilitate access to the video data of the VMS100from a web page served by the VMS100without having to download programs, install plug-ins, or otherwise modify or configure a browser application from a native configuration. That is, all necessary information and/or instruction required to receive and display a user interface and/or video data from the VMS100may either be provided natively with the standard browser or delivered from the VMS system100to allow for the execution of the client130. Any appropriate computing device capable of executing a standard web browser application that is in operative communication with the network115may be used as a client130to access the video data of the VMS100. For instance, any laptop computer, desktop computer, tablet computer, smartphone device, smart television, or another device that is capable of executing a standard internet browser application may act as a client130.

With further reference toFIG. 6, one example of the VMS100providing video data to a client130is depicted. In this context, a reverse proxy200may be utilized to facilitate communication with the client130. Specifically, the reverse proxy200may be facilitated by the web server142of the master node140, as described above. That is, the web server142may act as the reverse proxy200. In this regard, a client130may connect to the reverse proxy200. A user interface400comprising HTML or other web page content may be provided from the reverse proxy200. For instance, the user interface400provided by the reverse proxy400may include a listing404or searchable index of the available video data from the cameras110of the VMS100. This may include a listing of available live video data feeds for delivery in real-time to the client130or may allow for stored video data to be accessed. In the latter regard, a search function that allows for searching to be performed (e.g., using any video metadata including acquisition date/time, camera identify, facility location, and/or analytic metadata including objects identified from the video data or the like). In this regard, the web server142may act as a signaling server to provide information regarding available video data. Upon selection of a given portion of video data, a request may be issued from the client130to the reverse proxy200for specific video data. In turn, the reverse proxy200may communicate with a given one of the camera nodes120to retrieve the video data requested. The user interface400may also include a video display402. The video data may be requested by the web server142from an appropriate camera node120, formatted in an appropriate transport mechanism, and delivered by the web server142acting as the reverse proxy200to the client130for decoding and display of the video data in the video display402. Accordingly, the use of the reverse proxy200allows all data delivered to the client130to be provided from a single server, which may have an appropriate security certificate, which complies with many security requirements of browsers.

In an example, the transport mechanism into which the camera node120processes the data may be at least in part based on a characteristic of the request from the client130. In this regard, the reverse proxy200may determine a characteristic of the request. Examples of such characteristics include the nature of the video data (e.g., real-time or archived video data), an identity of the camera110that captured the video data, the network location of the client130relative to the reverse proxy200or the camera node120from which the video data is to be provided, or another characteristic. Based on the characteristic, an appropriate selection of an encoded video format, a container format, and a communication protocol for the processing of the video data by the camera node120. The camera node120may provide the video data to the reverse proxy200for communication to the client130. As described above, in at least some contexts, the video data provided to the client130may be real-time or near real-time video data that may be presented by the client130in the form of a standard web browser without requiring plug-ins or other applications to be installed at the client130.

A user may wish to change the video data displayed in the user interface400. In turn, a user may select a new video data source. In an implementation, the transport mechanism may be configured such that the new video data may be requested by the web server142from the appropriate camera node120and delivered to the user interface400without requiring a page reload. That is, the data in the video display402may be changed without requiring a reload of the user interface400generally. This may allow for greater utility to a user attempting to monitor multiple video data sources using the standard web browser.

The video data provided to the client130for rendering in the video display402may include metadata such as analytics metadata. As described above, such analytics metadata may relate to any appropriate video analysis applied to the video data and may include, for example, highlighting of detected objects, identification of objects, identification of individuals, object tracks, etc. Thus, the video data may be annotated to include some analytics metadata. The analytics metadata may be embodied in the video data or may be provided via a separate data channel. In the example in which the analytics metadata is provided via a separate channel, the client130may receive the analytics metadata and annotate the video data in the video display402when rendered in the user interface400. Further still, it may be appreciated that different types of data comprising the user interface400may be delivered using different transport mechanisms to the client130. For example, the foregoing examples of transport mechanisms may be used to deliver video data for display in the video display402. However, the user interface itself may be communicated using HTML and secure TLS security protocol over a standard TCP/IP connection. Further still, metadata (e.g., analytical metadata) may be provided as embedded data in the video data or may be provided as a separate data stream for rendering in the user interface130, as described above. In the case where the metadata is delivered using a separate data stream, the delivery of the metadata may be by way of a different transport mechanism than the video data itself.

With returned reference toFIG. 5, the abstraction of the functions of the VMS100into various functional layers may also provide an advantage in relation to the analysis of video data by the camera nodes120. Specifically, the application of an analysis model (e.g., a machine learning module) may be relatively computationally taxing for a camera node120. While the camera nodes120may be equipped with graphics processing units (GPUs) or other specifically adapted hardware that assist in performing the computational load, there may be certain instances in which the processing capacity of a given camera node120may not be capable of applying an analytics model to all of the video data from a given camera110. For example, in certain contexts, video data from a given camera110may advantageously be separated into different portions of data that may be provided to different camera nodes120for separate processing of the different portions of data. By “slicing” the data in this manner, analysis on the different portions of the video data may occur simultaneously at different ones of the camera nodes120, which may increase the speed and/or throughput of the analysis to be performed on the video data.

Thus, as shown inFIG. 7, a camera110of the VMS100may be in operative communication with a network115. At least a first node120aand a second node120bmay also be in communication with the network115to receive video data from the camera110. The first node120amay include a first analytical model210a, and the second node120bmay include a second analytical model210b. The first analytical model210amay be the same or different than the second analytical model210b.

Video data from the camera110may be divided into at least a first video portion212and a second video portion214. While referred to as video data portions, it should be understood that as little as a single frame of video data may comprise the respective portions of video data212and214. The first portion of video data212may be provided to the first camera node120a, and the second portion of video data214may be provided to the second camera node120b.

The second portion of video data214may be provided to the second camera node120bin response to a trigger detected by any of a master node, the camera node120a, the camera node120b, or the camera110. The trigger may be based on any number of conditions or parameters. For example, a periodic trigger may be established such that the second portion of video data214is provided to the second camera node120bin a periodic fashion based on time, an amount of camera data, or other periodic triggers. In this regard, the first analytical model210amay require relatively low computational complexity relative to the second analytical model210b. As such, it may not be computationally efficient to provide all of the video data to the second camera node120bfor processing using the second analytical model210b. However, every Nth portion (e.g., comprising a fixed time duration, size of the video on disk, or given number of frames) may be provided from the camera110to the second camera node210b, where N is a positive integer. In this regard, every hundredth second of video data may comprise the second portion of video data214, every thousandth frame of video data may comprise the second portion of video data214, etc.

In another context, the second portion of video data214may be provided to the second camera node120bbased on system video metadata or analytical video metadata for the first portion of video data212. For instance, upon detection of a given object from the first portion of video data212, subsequent frames of the video data comprising the second portion of video data214may be provided to the second camera node120b. As an example of this operation, a person may be detected by the first camera node120afrom the first video data portion212using the first analytical model210a. In turn, a second portion of video data214may be directed to the second camera node120bfor processing by the second analytical model210b, which may be particularly adapted for facial recognition. In this regard, the video data from the camera110may be directed to a particular node for processing to allow for a different analytical model or the like to be applied.

With reference toFIG. 12, example operations1200are shown according to an aspect of the present disclosure. The operations1200may include a capturing operation1202in which video data is captured at a plurality of video cameras. As described above, the video cameras may be in operative communication with a network. In turn, the operations1200may also include a communicating operation1204to communicate the video data to a plurality of camera nodes. As described above, any one or more of the plurality of cameras may communicate1204their respective video data to any one or more of the camera nodes. Specifically, the cameras and camera nodes may establish communication based on a first camera allocation configuration provided by a camera allocator of a master node of the system (e.g., to load balance the video data processing across the camera nodes based on an allocation parameter).

The operations1200may include a processing operation1206to process the video data received by each respective camera node. Specifically, as described above, in at least one example, the processing operation1206may include encoding the video data into encoded video data packets, packaging the encoded video data packets into a transport container, and selecting a communication protocol for sending the packets of video data. Further still, the processing operation1206may include storing the video data in a storage of a VMS.

A monitoring operation1208may monitor a camera allocation parameter at one or more of the camera nodes. The monitoring operation1208may include, for example, receiving status information from the one or more camera nodes, monitoring system resource usage at the camera nodes, and/or monitoring network traffic. In any regard, the operations1200may also include a detecting operation1210in which the camera allocator may detect a change in the camera allocation parameter across one or more of the camera nodes. As described above, the change in the camera allocation parameter at one or more of the camera nodes may result from any of a number of occurrences including, for example, a camera node becoming unavailable, a new camera node becoming available, an increase in computational load at a camera node, a decrease in computational load at a camera node, a change in the nature of the video data received at a camera node from a video camera, a change in video analytics applied to video data, or the like.

In response to the detecting operation1210, a modifying operation1212may be executed to modify the first camera allocation configuration to a second camera allocation configuration that is different than the first camera allocation configuration. Specifically, the modifying operation1212may be performed to balance the allocation parameter at the camera nodes in response to the detected change in the detecting operation1210. Furthermore, the process may be iterative such that the monitoring operation1208, detecting operation1210, and modifying operation1212may be performed regularly during the operation of a VMS system. For instance, such monitoring may occur at a frequency sufficient such that any change (e.g., unavailability of a camera node) is detected without significant interruption of video data processing of the video data from cameras allocated to the camera nodes. For instance, such monitoring may occur at a rate of at least 1 Hz, although higher frequencies may be employed without limitation.

FIG. 13illustrates an example schematic of a processing device1300suitable for implementing aspects of the disclosed technology. For instance, the processing device1300may generally describe the architecture of a camera node130, a master node140, and/or a client130The processing device1300includes one or more processor unit(s)1302, memory1304, a display1306, and other interfaces1308(e.g., buttons). The memory1304generally includes both volatile memory (e.g., RAM) and nonvolatile memory (e.g., flash memory). An operating system1310, such as the Microsoft Windows® operating system, the Apple macOS operating system, or the Linux operating system, resides in the memory1304and is executed by the processor unit(s)1302, although it should be understood that other operating systems may be employed.

One or more applications1312are loaded in the memory1304and executed on the operating system1310by the processor unit(s)1302. Applications1312may receive input from various input local devices such as a microphone1334, input accessory1335(e.g., keypad, mouse, stylus, touchpad, joystick, an instrument mounted input or the like). Additionally, the applications1312may receive input from one or more remote devices such as remotely-located smart devices by communicating with such devices over a wired or wireless network using more communication transceivers1330and an antenna1338to provide network connectivity (e.g., a mobile phone network, Wi-Fi®, Bluetooth®). The processing device1300may also include various other components, such as a positioning system (e.g., a global positioning satellite transceiver), one or more accelerometers, one or more cameras, an audio interface (e.g., the microphone1334, an audio amplifier and speaker and/or audio jack), and storage devices1328. Other configurations may also be employed.

The processing device1300further includes a power supply1316, which is powered by one or more batteries or other power sources and which provides power to other components of the processing device1300. The power supply1316may also be connected to an external power source (not shown) that overrides or recharges the built-in batteries or other power sources.

An example implementation may include hardware and/or software embodied by instructions stored in the memory1304and/or the storage devices1328and processed by the processor unit(s)1302. The memory1304may be the memory of a host device or of an accessory that couples to the host.

One general aspect of the present disclosure includes a method for processing video data in a video surveillance system. The method includes capturing video data at a plurality of video cameras and communicating the video data from the plurality of video cameras to at least one camera node over a communication network according to a first camera allocation configuration. The method also includes executing a video processing module at a camera node to process the video data received at the camera node and monitoring a camera allocation parameter at the camera node. The camera allocation parameter is at least in part based on the video data received at the at least one camera node. The method further includes modifying the first camera allocation configuration to a second camera allocation configuration different than the first camera allocation configuration at least in part based on the camera allocation parameter.

Implementations may include one or more of the following features. For example, the method may also include operating a plurality of camera nodes such that, in the first camera allocation configuration, each of the plurality of camera nodes is operative to receive video data from a respective subset of the plurality of video cameras. The method may also include detecting a change in a number of the available camera nodes in the video surveillance system. The second camera allocation configuration may include allocation of the plurality of video cameras to the available camera nodes after the detecting the change in the number of the available camera nodes.

In an example, the modifying operation is in response to a computational capacity of the camera node exceeding a threshold. This may be in response to an increased computational load associated with video analysis being performed by the camera node on the video data by the video processing module. In an example, the second camera allocation includes the reallocation of a video camera of the plurality of cameras to another camera node having a video analysis capability of the camera node from which the video camera is reallocated.

In an example, the camera allocation parameter comprises a field of view of each of the plurality of cameras. The second camera allocation may include the camera node disassociating from a video camera that shares an at least partially overlapping field of view with another video camera associated with the camera node.

Another general aspect of the present disclosure includes a video surveillance system. The system includes a plurality of video cameras operative to capture video data and at least one camera node operative to receive video data from one or more video cameras of the plurality of video cameras according to a first camera allocation configuration. The system also includes a video processing module executing at the at least one video camera node to process the video data received at the at least one camera node. The system further includes a master node in operative communication with the at least one camera node to monitor a camera allocation parameter at least in part based on the video data received at the at least one camera node and, in response to a change in the camera allocation parameter, modify the first camera allocation configuration to a second camera allocation configuration different than the first camera allocation configuration.

Implementations may include one or more of the following features. In an example, the system may include a plurality of camera nodes. In the first camera allocation configuration, each of the plurality of camera nodes may be operative to receive video data from a respective subset of the plurality of video cameras. The master node may be operative to detect a change in a number of the available camera nodes in the video surveillance system and the second camera allocation configuration comprises allocation of the plurality of video cameras to the available camera nodes after the detecting the change in the number of the available camera nodes.

In an example, the master node may modify the first camera allocation configuration to the second camera allocation configuration in response to a computational capacity of the camera node exceeding a threshold in response to an increased computational load associated with video analysis being performed by the camera node on the video data by the video processing module. In an example, the second camera allocation may include reallocation of a video camera of the plurality of cameras to another camera node having a video analysis capability of the camera node from which the video camera is reallocated.

In an example, the camera allocation parameter may include a field of view of each of the plurality of cameras. The second camera allocation may include the camera node disassociating from a video camera that shares an at least partially overlapping field of view with another video camera associated with the camera node.

Another general aspect of the present disclosure includes one or more tangible processor-readable storage media embodied with instructions for executing on one or more processors and circuits of a device a process for processing video data in a video surveillance system. The process may include capturing video data at a plurality of video cameras and communicating the video data from the plurality of video cameras to at least one camera node over a communication network according to a first camera allocation configuration. The process also includes executing a video processing module at a camera node to process the video data received at the camera node and monitoring a camera allocation parameter at the camera node. The camera allocation parameter is at least in part based on the video data received at the at least one camera node. The process also includes modifying the first camera allocation configuration to a second camera allocation configuration different than the first camera allocation configuration at least in part based on the camera allocation parameter.

Implementations may include one or more of the following features. In an example, the process may include operating a plurality of camera nodes. In the first camera allocation configuration, each of the plurality of camera nodes may be operative to receive video data from a respective subset of the plurality of video cameras. The process may also include detecting a change in a number of the available camera nodes in the video surveillance system. The second camera allocation configuration may include allocation of the plurality of video cameras to the available camera nodes after the detecting the change in the number of the available camera nodes.

In an example, the modifying may be in response to a computational capacity of the camera node exceeding a threshold in response to an increased computational load associated with video analysis being performed by the camera node on the video data by the video processing module. The second camera allocation may include reallocation of a video camera of the plurality of cameras to another camera node having a video analysis capability of the camera node from which the video camera is reallocated. For example, the camera allocation parameter comprises a field of view of each of the plurality of cameras. The second camera allocation may include the camera node disassociating from a video camera that shares an at least partially overlapping field of view with another video camera associated with the camera node.

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description is to be considered as exemplary and not restrictive in character. For example, certain embodiments described hereinabove may be combinable with other described embodiments and/or arranged in other ways (e.g., process elements may be performed in other sequences). Accordingly, it should be understood that only the preferred embodiment and variants thereof have been shown and described and that all changes and modifications that come within the spirit of the invention are desired to be protected.