Patent Description:
When a video camera breaks down there is no video stream footage, which leads to a lack of recorded video on the recorder. It is often difficult to predict the camera failure and a camera owner can typically react only after a failure occurs.

<CIT> discloses an apparatus capable of recording video from a plurality of video cameras and acquiring and analysing information about the captured video. <CIT> discloses a system of cameras connected to a host, wherein the host can identify a camera that fails to capture an image. <CIT> discloses a surveillance system comprising a coordinated set of cameras that cover a wide area and perform tracking of a subject that moves across the area. <CIT> discloses a system for detecting an object on a traffic way in a parking garage using at least two cameras having overlapping field of views on the traffic way, the captured images are analysed such as to determine a faulty camera among the two cameras.

According to one embodiment, a video camera system is provided. The video camera system including: a plurality of video cameras; a video recorder in communication with each of the plurality of video cameras; a video analytics module, the video analytics module being a computer program product embodied on a computer readable medium, the computer program product including instructions that, when executed by a processor, cause the processor to perform operations including: obtaining video parameters of a plurality of video frames received at the video recorder, the plurality of video frames being transmitted from the plurality of video cameras to the video recorder; determining an abnormality within the video parameters; identifying a malfunctioning video camera of the plurality of video cameras that produced the abnormality within the video parameters; deactivating the malfunctioning video camera; and activating or adjusting another video camera of the plurality of video cameras to capture video frames of an area previously being captured by the malfunctioning video camera, wherein the video parameters include: a video frame lost ratio and the abnormality is the video frame lost ratio being greater than a selected value; a video frame rate and the abnormality is the video frame rate being outside a selected range; a video packet lost ratio and the abnormality is the video packet lost ratio being greater than a selected value; or a video packet rate and the abnormality is the video packet rate being outside a selected range.

Optionally, the computer readable medium is located within the video recorder.

Optionally, embodiments may include an external device in communication with the video recorder, wherein the computer readable medium is located within the external device.

Optionally, embodiments may include: a cloud computing network in communication with the video recorder, wherein the computer readable medium is located within the cloud computing network.

Optionally, the operations include: activating an alarm in response to the malfunctioning video camera.

According to another embodiment, a method of operating a video camera system is provided. The method including: obtaining video parameters of a plurality of video frames received at a video recorder, the plurality of video frames being transmitted from a plurality of video cameras to the video recorder; determining an abnormality within the video parameters; identifying a malfunctioning video camera of the plurality of video cameras that produced the abnormality within the video parameters; deactivating the malfunctioning video camera; and activating or adjusting another video camera of the plurality of video cameras to capture video frames of an area previously being captured by the malfunctioning video camera, wherein the video parameters include: a video frame lost ratio and the abnormality is the video frame lost ratio being greater than a selected value; a video frame rate and the abnormality is the video frame rate being outside a selected range; a video packet lost ratio and the abnormality is the video packet lost ratio being greater than a selected value; or a video packet rate and the abnormality is the video packet rate being outside a selected range.

Optionally, embodiments may include activating an alarm in response to the malfunctioning video camera.

Technical effects of example embodiments include analyzing video metadata captured from a plurality of video cameras and determining abnormalities
using video analytics module installed on a video recorder, an external device, or a cloud computing network.

<FIG> schematically illustrates a video recording system <NUM>. It should be appreciated that, although particular systems are separately defined in the schematic block diagrams, each or any of the systems may be otherwise combined or separated via hardware and/or software. The video recording system <NUM> may include one or more video cameras <NUM> and a video recorder <NUM> in communication with each of the one or more video cameras <NUM>. Each video camera <NUM> may be positioned to capture video frames <NUM> of an area <NUM>. The areas <NUM> of each camera <NUM> may overlap with areas of other cameras <NUM> either partially or in full. Thus, if one camera <NUM> were to fail another camera <NUM> may be activated to capture video frames <NUM> of the area <NUM> where the video camera <NUM> failed. Each video camera <NUM> may be adjustable to be repositioned to capture the same area <NUM> or multiple areas <NUM>.

The video recording system <NUM> may optionally include an external device <NUM> or a cloud computing network <NUM>, although neither is required unless a video analytics module <NUM> is installed thereon. The video recorder <NUM> may be in communication with the external device <NUM> and/or the cloud computing network <NUM>. The external device <NUM> may be a computing device capable of doing real-time analysis, such as, for example, a desktop computer, a laptop computing, a tablet computer, a smart phone, a smart watch, or similar computing device known to one of skill in the art. Although not shown for simplification, the external device <NUM> may include a processor and an associated memory comprising computer-executable instructions that, when executed by the processor, cause the processor to perform various operations. The processor may be, but is not limited to, a single-processor or multi-processor system of any of a wide array of possible architectures, including field programmable gate array (FPGA), central processing unit (CPU), application specific integrated circuits (ASIC), digital signal processor (DSP) or graphics processing unit (GPU) hardware arranged homogenously or heterogeneously. The memory may be but is not limited to a random access memory (RAM), read only memory (ROM), or other electronic, optical, magnetic or any other computer readable medium. The external device <NUM> may be a part of video system in the same local network.

The cloud computing network <NUM> may be a remote device(s) that is outside the local network. The cloud architecture is transparent to the video system and can be scalable, such that the video analytics module <NUM> can be installed on multiple devices in the cloud computing network <NUM>. The remote device(s) of the cloud computing network <NUM> may be a computing device capable of doing real-time analysis, such as, for example, a desktop computer, a laptop computing, a tablet computer, a smart phone, a smart watch, or similar computing device known to one of skill in the art. Although not shown for simplification, the remote device(s) of the cloud computing network <NUM> may include a processor and an associated memory comprising computer-executable instructions that, when executed by the processor, cause the processor to perform various operations. The processor may be, but is not limited to, a single-processor or multi-processor system of any of a wide array of possible architectures, including field programmable gate array (FPGA), central processing unit (CPU), application specific integrated circuits (ASIC), digital signal processor (DSP) or graphics processing unit (GPU) hardware arranged homogenously or heterogeneously. The memory may be but is not limited to a random access memory (RAM), read only memory (ROM), or other electronic, optical, magnetic or any other computer readable medium.

A video analytics module <NUM> is installed on the video recorder <NUM>, the external device <NUM>, or the cloud computing network <NUM>. The video cameras <NUM> capture video frames <NUM> and transmits the video frames <NUM> to the video recorder <NUM>. The video frames <NUM> from one or more video cameras <NUM> arrive at the video recorder <NUM> as an incoming stream <NUM>. The video frames <NUM> may be organized into packets <NUM> by the video camera <NUM>. The video packets <NUM> contain information such as, for example, an IP address from where the video packets <NUM> are coming from, an IP address where to send the video packets <NUM>, a type of the video frame <NUM>, a number of video frames <NUM>, time stamps of each video frame 112and video parameters <NUM> discussed further herein. The video packets <NUM> may be transmitted from the video recorder <NUM> to a video viewing device <NUM>. The video viewing device <NUM> may be a computing device, such as, for example a desktop computer, laptop computer, smart phone, smart watch, or similar device known to one of skill in the art. The video packets <NUM> are received at the video viewing device <NUM> as an incoming stream <NUM> and the video viewing device <NUM> may contains a decoder (not shown) configured to receive the video packets <NUM> and collapse the video packets <NUM> into video frames <NUM> to be sent to a display <NUM>, which then displays the video frames <NUM> received.

A video analytics module <NUM> may be installed within the video recording system <NUM>. The video analytics module <NUM> may be installed directly on the video recorder <NUM>, the external device <NUM>, or the cloud computing network <NUM>. In an embodiment, the video analytics module <NUM> is installed directly on the video recorder <NUM>. In another embodiment, the video analytics module <NUM> is installed on the external device <NUM>, which is in communication with the video recorder <NUM>. In another embodiment, the video analytics module <NUM> is installed on the cloud computing network <NUM>, which is in communication with the video recorder <NUM>.

The video analytics module <NUM> may be a software algorithm configured to collect and analyze video frames <NUM> received at the video recorder <NUM> from the video cameras <NUM>. Additionally, the video analytics module <NUM> may be self-learning and may use neural networks or deep learning. For example, detected abnormalities may be a result of typical camera operation and environment changes therefore further self-learning analysis may be needed. The video analytics module <NUM> is configured to collect and analyze video parameters <NUM> of the video frames <NUM> and/or video packets <NUM>. The video parameters <NUM> are statistical information regarding the video frames <NUM> and/or the video packets <NUM>. The video parameters <NUM> may include but are not limited to time between video frames <NUM>, time between video packets <NUM>, a sequential number of a video frame <NUM>, and a sequential number of a video packet <NUM>, a group of pictures (GOP) structure, a size of the video packet <NUM> and a real frame per second (FPS) value. The size of the video packet <NUM> may be used to calculate frame size and bandwidth.

The video frames <NUM> and/or video packets <NUM> are collected in the incoming stream <NUM> of the video recorder <NUM> and stored in a memory <NUM> of the video recorder <NUM>. The video analytics module <NUM> collects the video parameters <NUM> of the video frames <NUM> and/or video packets <NUM> in the incoming stream <NUM> of the video recorder <NUM> along a path <NUM> from the video camera <NUM>. The path <NUM> between the video cameras <NUM> and the video recorder may be hardwired and/or wireless. The collection of the video parameters <NUM> allows for a quantitative analysis of the video parameters <NUM> to detect abnormalities in the video parameters <NUM> with one of the video cameras <NUM>, such as, for example, a delay between GOPs, a delay video frames <NUM>, lost video packets <NUM>, delays in video packet <NUM> delivery, irregularity in video frame <NUM> bandwidth, and irregularity in video frame <NUM> delivery.

If an abnormality is detected in the video parameters <NUM> then the video analytics module <NUM> may activate an alarm <NUM> on an alert module <NUM>. The alarm <NUM> may be audible, visual, and/or vibratory. The alert module <NUM> may be separate standalone computing devices, such as, for example, a desktop computer, a laptop computing, a tablet computer, a smart phone, a smart watch, or similar computing device known to one of skill in the art. Alternatively, the alert module <NUM> maybe be a computer program product (e.g., software application) installed on at least one of a computing device (not shown), the video recorder <NUM>, the external device <NUM>, and the cloud computing network <NUM>. Additionally, if an abnormality is detected in the video parameters <NUM> of a particular video camera <NUM> then the video camera <NUM> is considered to be malfunctioning and the video analytics module <NUM> automatically directs another video camera <NUM> to capture video frames <NUM> of the area <NUM> that was previously captured by the video camera <NUM> determined to be malfunctioning. This direction by the video analytics module <NUM> involves deactivating the video camera <NUM> determined to be malfunctioning and activating another video camera <NUM> to capture video frames <NUM> of the area <NUM> that was previously captured by the video camera <NUM> determined to be malfunctioning. This direction by the video analytics module <NUM> may involve adjusting (e.g., repositioning, moving, rotating, aiming, angling, focusing, zooming in, zooming out) another video camera <NUM> to capture video frames <NUM> of the area <NUM> that was previously captured by the video camera <NUM> determined to be malfunctioning.

The video parameters <NUM>, the video frames <NUM> and/or video packets <NUM> may be collected and stored on a memory <NUM> within the video recorder <NUM>. The memory <NUM> may be a storage device such as, for example, a random access memory (RAM), read only memory (ROM), or other electronic, optical, magnetic or any other computer readable medium. The video recorder <NUM> may include a processor <NUM> capable of writing to the memory <NUM>. The processor <NUM> may be but is not limited to a single-processor or multi-processor system of any of a wide array of possible architectures, including field programmable gate array (FPGA), central processing unit (CPU), application specific integrated circuits (ASIC), digital signal processor (DSP) or graphics processing unit (GPU) hardware arranged homogenously or heterogeneously.

Although not shown for simplicity the external device <NUM> and the cloud computing network <NUM> may also include a processor and an associated memory comprising computer-executable instructions that, when executed by the processor, cause the processor to perform various operations. The processor may be, but is not limited to, a single-processor or multi-processor system of any of a wide array of possible architectures, including field programmable gate array (FPGA), central processing unit (CPU), application specific integrated circuits (ASIC), digital signal processor (DSP) or graphics processing unit (GPU) hardware arranged homogenously or heterogeneously. The memory may be but is not limited to a random access memory (RAM), read only memory (ROM), or other electronic, optical, magnetic or any other computer readable medium.

The video analytics module <NUM> is configured to analyze the video parameters of the video packets <NUM> and video frames <NUM> contained in the memory <NUM>. The video analytics module <NUM> is configured to determine a video packet lost ratio and a video frame lost ratio in response to the video parameters <NUM> of the video frames <NUM> and video packets <NUM>. The video packet lost ratio may be determined by analyzing the sequential numbers of each video packet <NUM> and determining if a sequential number of a video packet <NUM> is missing in the sequence. The video packet lost ratio may also be calculated in a real-time transport protocol (RTP) stream situation. In an RTP stream situation, the video packets <NUM> are numbered and that information is stored in the RTP header (application layer in the video packet layer). Each next video packet <NUM> has a sequence number about one bigger than a predecessor video packet <NUM>, thus a value of how many video packets <NUM> should be received by the video recorder <NUM> and how many video packets <NUM> were lost may be determined.

The video frame lost ratio may be determined by analyzing the sequential numbers of each video frame <NUM> and determining if a sequential number of a video frame <NUM> is missing in the sequence. The video frame lost ratio may be determined by analyzing the sequential numbers of each video frame <NUM> and determining if a sequential number of a video frame <NUM> is missing in the sequence. The video frame lost ratio may also be calculated in a real-time transport protocol (RTP) stream situation. In an RTP stream situation, the video frames <NUM> are numbered and that information is stored in the RTP header (application layer in the video frame layer). Each next video frame <NUM> has a sequence number about one bigger than a predecessor video frame <NUM>, thus a value of how many video frames <NUM> should be received by the video recorder <NUM> and how many video frames <NUM> were lost may be determined.

Advantageously by determining a video packet lost ratio and a video frame lost ratio, a statistical description of the quality of each of the video cameras <NUM> is provided while considering that one video packet lost for the Intra-frame may cause the whole set of frames (e.g. the whole GOP) to be impossible to decode. A video packet lost ratio or a video frame lost ratio greater than a selected value may indicate that an abnormality in the video parameters <NUM> that is caused by a video camera <NUM> that is malfunctioning.

The video analytics module <NUM> is also configured to determine an abnormality in the video packet rate of the video packets <NUM> and an abnormality in the frame rate of in video frames <NUM> in response to the video parameters <NUM> of the video frames <NUM> and video packets <NUM>. An abnormality in the video packet rate may be that the video packet rate is outside of a selected range (e.g., either faster or slower than the selected range). An abnormality in the video frame rate may be that the video packet frame is outside of a selected range (e.g., either faster or slower than the selected range). For example, an abnormality in the video packet rate of the video packets <NUM> may be a delay amongst the video packets <NUM> and an abnormality in the frame rate of the video frames <NUM> may be a delay amongst the video frames <NUM>.

The delay in video frames <NUM> may be determined by analyzing the sequential time stamps of each video frame <NUM> and determining if a there is a delay between the sequential time stamps of each video frame <NUM> in the sequence. For instance, the video frames <NUM> may be captured at a set frequency, thus the time between the sequential time stamp of each video frame <NUM> should be about equivalent. If the time between each video frames <NUM> begins to vary or become intermittent than it may be indicative of an abnormality in the video parameters <NUM> caused by a video camera <NUM> that is malfunctioning.

The delay in video packets <NUM> may be determined by analyzing the sequential time stamps of each video packets <NUM> and determining if a there is a delay between the sequential time stamps of each video packet <NUM> in the sequence. For instance, the video packets <NUM> may be organized at a set frequency, thus the time between the sequential time stamp of each video packet <NUM> should be about equivalent. If the time between each video packet <NUM> begins to vary or become intermittent than it may be indicative of an abnormality in the video parameters <NUM> caused by a video camera <NUM> that is malfunctioning.

The video analytics module <NUM> is also configured to measure and analyze other real video channel parameters of the video cameras <NUM> including but not limited to a real frame per second (FPS) value and a real GOP value, which is a count of the frames between two I-frames including a first count tacking which I-frames sent by a video recorder <NUM> are real. Sometimes when a video cameras <NUM> are subjected to high stress (e.g., when the camera looks at a static scene but that scene is dramatically changed) the camera can start transmitting a new I-Frame to produce a better video quality instead of the rest of the P-Frames and in this scenario the real GOP is lower than configured on the camera. In a high stress scenario, sometimes I-frames are changed to P-frames because the camera is trying not exceed the bandwidth and in this scenario the real GOP may be higher than configured on the video camera <NUM>.

Additionally, the video analytics module <NUM> is also configured to measure, analyze, and determine a statistical description of a variety of time parameters including: a time taken for sending one particular video frame; a time taken for sending all frames in one GOP; and a time between the last video packet of the predecessor frame and the last video packet of the successor frame.

During the analysis, the video analytics module <NUM> is configured to measure effective bandwidth and instantaneous bandwidth, while considering asynchronous interference of dozens and hundreds of video streams as an average value for a configurable time window such as, for example, <NUM>, <NUM>, <NUM>, <NUM> (~equivalent of 1GOP encoder) <NUM>. The video analytics module <NUM> is also configured to visualize the effective bandwidth and instantaneous bandwidth. The effective bandwidth is calculated for the video packets <NUM>. It is calculated by factor of how much data was received in the specific time window. The instantaneous bandwidth is calculated by a factor of a size of video frame <NUM> divided by time of how long that frame <NUM> was received.

Additionally, during the analysis, the video analytics module <NUM> is configured to measure the "time dispersion between video frames <NUM>" to check if the video is smooth to determine if the video camera <NUM> is malfunctioning. The video analytics module <NUM> is also configured to measure a size of the video frames <NUM> and statistical description of differences between I and P-frames of the video packets <NUM> for the given encoder parameters and the dynamism of the scene in front of the video camera <NUM>. The video analytics module <NUM> is also configured to visualize the size of the video frames <NUM> and statistical description of differences between I and P-frames of the video packets <NUM>.

Referring now to <FIG>, with continued reference to <FIG>, a flow chart of method <NUM> of operating a video recording system <NUM> is illustrated. The method <NUM> may be performed by the video analytics module <NUM>. The video analytics module <NUM> may be a computer program product embodied on a computer readable medium. The video analytics module <NUM> (i.e., computer program product) may include instructions that, when executed by a processor, cause the processor to perform operations comprising method <NUM>. The computer readable medium where the video analytics module <NUM> is stored may be located within the video recorder <NUM>. Alternatively, the computer readable medium where the video analytics module <NUM> is stored may be located within the external device <NUM>. In another alternative, the computer readable medium where the video analytics module <NUM> is stored may be located within the cloud computing network <NUM>.

At block <NUM>, video parameters <NUM> of a plurality of video frames <NUM> received at the video recorder <NUM> are obtained. The plurality of video frames <NUM> are transmitted from the one or more video cameras <NUM> to the video recorder <NUM>.

At block <NUM>, an abnormality is determined within the video parameters <NUM>. In one embodiment, the video parameters <NUM> include at least a video frame lost ratio and an abnormality may be determined if the video frame lost ratio is greater than a selected value. It is understood that one or more of the video parameters <NUM> may be used to determine an abnormality and the video frame lost ratio may be one of many video parameters <NUM> taken into consideration. In another embodiment, the video parameters <NUM> include a video frame rate and an abnormality may be determined if the video frame rate is outside a selected range.

The video frames <NUM> may be organized into video packets <NUM>, as discussed herein. The video parameters <NUM> include a video packet lost ratio and an abnormality may be determined if the video packet lost ratio is greater than a selected value. The video parameters <NUM> include a video packet rate and an abnormality may be determined if the video packet rate is outside a selected range.

At block <NUM>, a malfunctioning video camera <NUM> of the one or more video cameras <NUM> that produced the abnormality within the video parameters <NUM> is identified.

The method <NUM> may include deactivating the malfunctioning video camera <NUM>. The method <NUM> may also include activating a video camera <NUM> of the one or more video cameras <NUM> to capture video frames <NUM> of an area <NUM> previously being captured by the malfunctioning video camera <NUM>. The method <NUM> may further include adjusting a video camera <NUM> of the one or more video cameras <NUM> to capture video frames <NUM> of an area <NUM> previously being captured by the malfunctioning video camera <NUM>. Additionally, an alarm <NUM> may be activated in response to the malfunctioning video camera <NUM>.

While the above description has described the flow processes of <FIG> in a particular order, it should be appreciated that unless otherwise specifically required in the attached claims that the ordering of the steps may be varied.

Claim 1:
A video camera system (<NUM>) comprising:
a plurality of video cameras (<NUM>);
a video recorder (<NUM>) in communication with each of the plurality of video cameras;
a video analytics module (<NUM>), the video analytics module being a computer program product embodied on a computer readable medium, the computer program product including instructions that, when executed by a processor, cause the processor to perform operations comprising:
obtaining video parameters (<NUM>) of a plurality of video frames (<NUM>) received at the video recorder, the plurality of video frames being transmitted from the plurality of video cameras to the video recorder;
determining an abnormality within the video parameters;
identifying a malfunctioning video camera of the plurality of video cameras that produced the abnormality within the video parameters;
deactivating the malfunctioning video camera (<NUM>); and
activating or adjusting another video camera (<NUM>) of the plurality of video cameras to capture video frames (<NUM>) of an area (<NUM>) previously being captured by the malfunctioning video camera,
wherein the video parameters (<NUM>) include:
a video frame lost ratio and the abnormality is the video frame lost ratio being greater than a selected value; or
a video frame rate and the abnormality is the video frame rate being outside a selected range; or
a video packet lost ratio and the abnormality is the video packet lost ratio being greater than a selected value; or
a video packet rate and the abnormality is the video packet rate being outside a selected range.