Patent ID: 12206984

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention now will be described more fully hereinafter with reference to the accompanying drawings, in which illustrative embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. Further, the singular forms including the articles “a” “an” and “the” are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms: includes, comprises, including and/or comprising, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Further, it will be understood that when an element, including component or subsystem, is referred to and/or shown as being connected or coupled to another element, it can be directly connected or coupled to the other element or intervening elements may be present.

FIG.1shows an exemplary surveillance camera system10constructed according to the principles of the present invention.

The system10includes IP surveillance cameras103-1through103-N, a computer workstation224, and possibly a local image data storage system212that communicate over a local network210. The system also possibly includes a cloud image data storage system310that is accessible by the surveillance cameras103via a network cloud50.

A user60-2holds a mobile user computing device400for communicating with the surveillance cameras103or the image data storage systems212,310. Each mobile user device400includes a display screen410and one or more applications412, or “apps.” The apps412display information to the user60-2via the display screen410of the mobile user device400. The apps412execute upon the operating system of the mobile user device400. The apps412communicate with the application server220over a network cloud50via a wireless connection264. Examples of mobile user computing devices400include smartphones, tablet computing devices, and laptop computers running operating systems such as Windows, Android, Linux, or IOS, in examples.

In the illustrated example, surveillance cameras103such as cameral103-1and cameral103-2might be installed within a premises52. Field of view105-1of cameral103-1might be directed to capture individuals60-1as they enter or exit through a doorway66of the premises, for example. Field of view105-2of cameral103-2captures activity near a door62, in the example.

In one embodiment, image data250captured by the camera is stored locally within the respective cameras103. In another embodiment, the surveillance cameras103send their image data250-2over the local network210for storage on the local image data storage system212. In one example, the local image data storage system212is a network video recorder (NVR). In yet another embodiment, the surveillance cameras103send their image data250-3over the network cloud50for remote storage on the cloud image data storage system310.

VoIP communications can occur between peer VoIP endpoint devices, also known as VoIP endpoints263. Each VoIP endpoint263includes a unique IP address262and possibly a phone number260. The phone number260can be private to an organization or local network, or can be a public phone number, in examples. Mobile user device400and computer workstation224are configured as VoIP client endpoint devices, namely VoIP endpointl263-1and VoIP endpoint3263-3, respectfully. Each VoIP client endpoint263first establishes a separate voice communications channel258with a VoIP server endpoint263, and then requests image data250and/or audio data from the VoIP server endpoint263over the associated voice communications channel258.

VoIP server endpoints263can include surveillance cameras103, the local image data storage system212, and the cloud image data storage system310, in examples. In the illustrated example, the VoIP server endpoints263are VoIP server endpoint2263-2(cameral103-1), VoIP server endpoint2263-4(camera4103-4), VoIP endpoint5263-5(local image data storage system212), and VoIP server endpoint6263-6(cloud image data storage system310).

In the illustrated example, mobile user device400is configured as VoIP client endpointl263-1and includes phone number260-1“777-555-2001” and IP address “10.10.10.1.” In a similar vein, computer workstation224is configured as VoIP client endpoint3263-3and includes phone number260-3“777-555-2003” and IP address262-3“10.10.10.3.”

Surveillance cameral103-1is configured as VoIP server endpoint2263-2and includes phone number260-2777-555-2002” and IP address262-2“10.10.10.2.” Surveillance camera2103-2is configured as VoIP endpoint4263-4and includes phone number260-4“777-555-2004” and IP address262-4“10.1.0.10.4.” Local image data storage system212is configured as VoIP server endpoint5263-5and includes phone number260-5“777-555-2005” and [P address262-5“10.10.10.5.” Finally, cloud image data storage system310is configured as a VoIP server endpoint6263-6and includes phone number260-6“777-555-2006” and IP address262-6“10.10.10.6.”

An exemplary voice communications channel258-1between VoIP client endpointl263-1(mobile user device400) and VoIP server endpoint2263-2cameral103-1is shown. The voice communications channel258-1includes a control session240and a data channel252. The control session240enables exchange of control and/or signaling information between the VoIP client endpoint263-1and VoIP server endpoint263-2, while the data channel252enables exchange of audio and/or image data250. To distinguish between control sessions240and therefore between the voice communications channels258, each control session240has a unique session ID256.

The control session240is preferably an IP based control session that is based upon the Session Information Protocol (SIP) but can also be implemented according to other standard and proprietary signaling protocols. Other VoIP signaling protocols include IAX, H.323, and XMPP, in examples. Control functions provided by the control session240include call setup functions between the endpoints263-1/263-2that are analogous to a traditional telephone such as the initial call request, confirmation of call request, termination of call (SIP request messages INVITE, ACK, and BYE, respectively), and ringing of a phone and busy tone (SIP response message codes180“Ringing” and486“Busy Here”) in examples. Other control functions provided by the control session240include commands sent from VoIP client endpointl263-1for requesting available resources on the cameral103-1such as selected frames of image data250and available disk space, in examples.

The data channel252is preferably implemented as a half duplex data channel and is preferably based upon the real-time transfer protocol (RTP). For this reason, the data channel252can also be referred to as an RTP session between two VoIP endpoints263. A half duplex implementation for the data channel252is preferred to maximize the likelihood that image data250is transferred at maximum throughput with minimum loss over the voice communications channel258. This is because when the VoIP server endpoint263-2is transmitting its image data of the current stream over the data channel252to the VoIP client endpoint263-1, the VoIP client endpoint263-2cannot send RTP messages to the VoIP server endpoint263-2until the VoIP server endpoint263-2has completed transmission of the image data250.

When configured as VoIP server endpoints263, the surveillance cameras103function as video phones that automatically answer calls initiated by authenticated VoIP client endpoints263. The authentication (e.g. authorization) is typically provided by the control session240portion of the voice communications channel258. Upon establishing the authenticated control session240, audio and/or image data is transmitted from the surveillance cameras103to the VoIP client endpoints263-1/263-3via the data channel252. The data channel252is preferably operated in half duplex mode to avoid sending unwanted video and/or audio to the cameras103while the cameras103are transmitting their audio and/or image data to the VoIP client endpoints263.

The control session240is preferably based upon an implementation of SIP. In this way, the VoIP client endpoints263can use existing SIP commands to control the streams of image data250-1from the surveillance cameras103in a manner similar to the way in which VoIP client endpoints263control video from VoIP server endpoints263of a traditional VoIP video conferencing call, in one example.

For the SIP based implementation of the data channel252, the VoIP server endpoint263-2assembles the image data250into RTP packets for transmission over the data channel252. In examples, the RTP based data channel252can support best effort delivery over UDP and/or reliable delivery over TCP. Best effort delivery is favored for several reasons. These reasons include the fact that reliable delivery often causes significant delays due to retransmission of lost frames, and that reliable delivery consumes more bandwidth than best effort delivery. Finally, because the content of frames of image data250from surveillance cameras typically do not change much from frame to frame, a small amount of image data loss inherent when using best effort delivery is most often an acceptable tradeoff in favor of the improved transmission speed and bandwidth utilization provided by best effort delivery.

FIG.2shows some of the components of an exemplary surveillance camera103-4. In the example, the surveillance camera103stores its image data250locally and includes an integrated application server220as a process. Surveillance camera103-4is also configured as a VoIP server endpoint263-4.

The camera103includes a processing unit (CPU)138, an imager140, a camera image data storage system174and a network interface142. An operating system136runs on top of the CPU138. The surveillance camera103saves image data250captured by the imager140to the camera image data storage system174. Each camera103can support one or more streams of image data250. The application server220receives and sends messages264via its network interface142.

A number of processes or software applications are executed by the operating system136. The processes include an application server220. The application server220, in turn, includes a SIP based control session240implementation, or SIP layer132, and an RTP based data channel252implementation, or RTP layer134. Because the application server220enables the surveillance camera103-4to function as a Val) endpoint263, the application server220also maintains a phone number260-4and an IP address262-4for the VoIP server endpoint263-4.

The application server220preferably includes a SIP layer132based on an implementation of the Session Initiation Protocol (SIP) for establishing an IP based control session240of the voice communications channel258-1, for controlling the one or more streams of image data250-1sent by the surveillance camera103to the client device400/224. The surveillance camera103operates as a VoIP server endpoint263-1, where the client device400/224operates as a VoIP client endpoint263-1.

The application server220also preferably includes a RTP layer134based on an implementation of the Real Time Protocol (RTP) for establishing a half duplex data channel252of the voice communications channel258-1, for sending the one or more streams of image data250-1to the client device400/224.

FIG.3Ashows a method for a mobile user device400configured as a VoIP client endpoint263-1to access image data250from an IP surveillance camera103configured as a VoIP server endpoint263.

In step502, a mobile user device400establishes a Session Initiation Protocol (SIP) based control session with an IP-enabled surveillance camera103, where the user mobile computer device and the surveillance camera103are VoIP/SIP endpoints263. In step504, the mobile user device400receives image data250over a half duplex data channel252, where the image data250is included within Real time Transport Protocol (RTP) packets sent from the surveillance camera103. Then, in step506, the mobile user device400renders the image data250on the display screen410of the user mobile device400.

According to step508, the method tests whether the control session240was terminated by either endpoint (e.g. via a SIP “BYE” request message of the control session240). If this resolves to true, the method terminates in step510. Otherwise, the method transitions to the beginning of step504to process the next frame of image data250to be sent over the half duplex data channel252to the client device400.

FIG.3Bshows a method for a computer workstation224configured as a VoIP client endpoint263-1to access image data250from an IP surveillance camera103configured as a VoIP server endpoint263.

In step532, a computer workstation224establishes a Session Initiation Protocol (SIP) based control session with an IP-enabled surveillance camera103, where the computer workstation224and the surveillance camera103are VOIP/SIP endpoints263. In step534, the computer workstation224receives image data250over a half duplex data channel252, where the image data250is included within Real time Transport Protocol (RTP) packets sent from the surveillance camera103. Then, in step536, the computer workstation224saves the image data250-2over the local network210to the local image data storage system212. It can also be appreciated that the computer workstation224can alternatively save the image data250-3over the network cloud50to the cloud image data storage system310.

According to step538, the method tests whether the control session240was terminated by either endpoint (e.g. via a SIP “BYE” request message of the control session240). If this resolves to true, the method terminates in step540. Otherwise, the method transitions to the beginning of step534to process the next frame of image data250sent over the half duplex data channel252to the client device224.

FIG.4shows a method for a VoIP client endpoint263device establishing a Session Initiation Protocol (SIP) based control session240with an IP surveillance camera103as a VoIP server endpoint263and sending control messages over the control session240. The control session240is one component of a separate voice communications channel258established between each of the VoIP client endpoints263and each VoIP server endpoint263.

In step552, the client device224/400establishes a Session Initiation Protocol (SIP) based control session240with an IP-enabled surveillance camera103configured as a VoIP server endpoint263. In step554, the client device224/400sends control messages over the SIP connection/control session240to the surveillance camera103. In one example, the control messages query the surveillance camera103to obtain the list of current resources and available functions provided by the surveillance camera103. In another example, the control messages control the transmission of image data250from and configure settable parameters on the surveillance cameras (e.g. zoom setting, resolution of image data250).

In examples, the control messages for controlling the streams of image data250include commands for cancelling requests for image data250, terminating a call to tear down the voice communications channel258, and setting parameters on the surveillance camera103. Control messages based on the SIP request message types CANCEL, BYE, and MESSAGE types, respectively, can be utilized for these purposes, in one implementation.

In other example, the list of current resource information can include the number of streams of image data250supported, the disk space available on the camera image data storage system174, codec information of the imager140, software versions of the SIP layer132and/or RTP layer134within the application server220of the surveillance camera103. A control message based on the SIP request message types OPTIONS and/or INFO can be utilized for these purposes, in one implementation.

According to step556, the client device224/400receives a SIP response message in response to the requested control commands (e.g. Success, Error). In step558, the method tests whether the control session240was terminated by either endpoint (e.g. via a SIP “BYE” request message of the control session240). If this resolves to true, the method terminates in step560. Otherwise, the method transitions to the beginning of step554to process the next control message sent over the SIP based control session252.

While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.