Abstract:
A security system comprises a computer network and a plurality of sensors each connected to the computer network at a respective network address and each generating sensing data. A managing component is connected with the network and communicates with the sensors by access thereof through the associated network address on the network, and processes sensor information received from said sensors. The managing component has a display with an interface screen showing to a user all the sensors in the security system, and an input device through which the user can enter interactive instructions to the managing component. The managing component controls communications to and from the sensors and has a rules engine storing rules therein. Each of the rules being associated with a respective device on the network, and causing the managing computer to take an action in response to output from at least one of the devices.

Description:
RELATED APPLICATIONS  
       [0001]     This application claims priority of U.S. provisional application Ser. No. 60/575,895 filed Jun. 1, 2004 and entitled “METHOD AND SYSTEM FOR PERFORMING VIDEO FLASHLIGHT”, U.S. provisional patent application Ser. No. 60/575,894, filed Jun. 1, 2004, entitled “METHOD AND SYSTEM FOR WIDE AREA SECURITY MONITORING, SENSOR MANAGEMENT AND SITUATIONAL AWARENESS”, and U.S. provisional application Ser. No. 60/576,050 filed Jun. 1, 2004 and entitled “VIDEO FLASHLIGHT/VISION ALERT”. 
     
    
     FIELD OF THE INVENTION  
       [0002]     The present invention generally relates to surveillance systems, and especially to systems and methods for managing sensor devices and viewing data for situational awareness in a surveillance system, such as, e.g., the VIDEO FLASHLIGHT™ system, described in U.S. published patent application 2003/0085992 published on May 8, 2003, herein incorporated by reference, in which videos from a number of cameras in an particular site or environment are managed by overlaying the video from these cameras onto a 2D or 3D model of a scene.  
       BACKGROUND OF THE INVENTION  
       [0003]     Over the years, security has become of major importance. The number of sites or regions in which surveillance is desired has increased. As the number of sites has increased, so has the demand for the number of surveillance security systems. In an environment in which surveillance of a large site or region is desired, a conventional system typically monitors the activity within the site or region using a variety of sensors including video, radar, RFID and access control. The sensors are positioned throughout a site or region, and the sensors provide event (or threat) information relating to the activity within the site or region. For example, an event may be an alarm, video stream or other information sensed by a sensor in an area of the site or region.  
         [0004]     Not only has the demand for the quantity of security systems increased, so has the demand for more sophisticated surveillance techniques and technology to enable an operator to monitor and manipulate sensors located even across the globe. Existing systems, however, normally do not provide for the flexibility of communication or the introduction of distant sensors or components into a security surveillance system.  
         [0005]     Also, certain surveillance systems of the prior art do not provide a full and clear picture of the activity in the area or region. For example, it would be preferable to view all sensors on a single display, but this is not provided in earlier designed systems, and, frequently, only event information from one sensor can be viewed at a time on a display. Furthermore, event information is usually viewable or accessed only within the site or region in which the sensor is located. Event information from one (remote) site cannot be accessed by another site (locally).  
         [0006]     In the current environment, sensors are located across the globe at various sites (typically located great distances from the local site or region) and it would be desirable to have remote access. In addition, these conventional systems do not enable security personnel to configure the sensors as desired locally or remotely. Finally, event information is rarely received instantaneously.  
         [0007]     There is therefore a need for a method and system that will overcome the disadvantages of earlier systems.  
       SUMMARY OF THE INVENTION  
       [0008]     A security system is disclosed in which several separate sites or regions are connected over a network. A variety of sensors are available at each networked site along with its own system and network, devices, computers and sensors. The system includes an assembly of software components that run in a distributed manner on the networked sites. The security system with these software components makes all sensors visible in an integrated display from any site. Further, the security system provides for configuration, control and display of the sensors as well the recording and retrieval of sensor information from any site. Instantaneous sensor information is therefore achieved, and the system can be scaled up readily and without substantial limitations.  
         [0009]     In accordance with an aspect of the invention, a security system comprises a computer network and a plurality of sensors each connected to the computer network at a respective network address and each generating sensing data. A managing component is connected with the network and communicates with the sensors by access thereof through the associated network address on the network, and processes sensor information received from said sensors. The managing component has a display with an interface screen showing to a user all the sensors in the security system, and an input device through which the user can enter interactive instructions to the managing component. The managing component controls communications to and from the sensors and has a rules engine storing rules therein. Each of the rules being associated with a respective device on the network, and causing the managing computer to take an action in response to output from at least one of the devices.  
         [0010]     According to another aspect of the invention, a security system comprises a computer network, a plurality of managing modules each connected with the computer network, and a plurality of sensors, each connected with a respective managing module by a communication link other than the network. Each managing module has a rules engine defining at least one action to be taken in reaction to an output from one of the sensors. One of the managing modules receives a transmission over the network from another of the managing modules acting as a proxy server for a sensor attached thereto and displays on a display device thereof data from the sensor.  
         [0011]     According to still another aspect of the invention, a method of security monitoring and management comprises providing a modular managing terminal connected with a network to which are connected a plurality of sensor devices each having a respective IP address. Communications with the sensor devices are configured for the managing terminal. Transmissions from the sensor devices over the network are received at the managing terminal. A rules engine is maintained at the managing module. The rules engine has stored a respective rule for each of said sensor devices, the rule for each device determining whether the managing module takes no action in response to a transmission from the associated device or takes an associated action in response to said transmission from said device.  
         [0012]     Other objects and advantages of the present invention will be apparent to those of skill in the art with the present disclosure before them. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0013]      FIG. 1  illustrates a security system according to a preferred embodiment of the present invention.  
         [0014]      FIG. 2  is an exemplary screen shot of a computer display operating with the system of the present invention displaying the capabilities and functionalities of the system. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0015]      FIG. 1  shows security system  10  in accordance with a preferred embodiment of the present invention. The security system includes an assembly of software components that run in a distributed manner on a set of networked “sites” or instances, each of which includes potentially its own system, network, devices, computers and sensors. Generally, the security system is run by one or more HAWK terminals, which are supported either on a PC computer with the usual components, i.e., RAM, disk drive and other data storage devices, a mouse, a keyboard and a monitor or a display, or else on a PDA, with the usual connectivity and I/O alternatives thereof, and connected with the network.  
         [0016]     The HAWK terminals are modular devices that act as front-end user access devices with GUI or other interactive interface displays and input devices, and also as servers or connection managers controlling communication among devices across the network based on a set of rules running on an internal rule engine in each HAWK terminal that defines its relationship with each of the devices on the network.  
         [0017]     In the simplest of HAWK systems, a single HAWK terminal is connected with a network, and there are a number of sensor devices, such as detectors, cameras, etc., connected to the network as well. The HAWK terminal communicates with each of the devices through the network and receives data from the devices as well as transmitting commands to the various devices that manages the communications through the network based on a rule engine in the HAWK terminal that takes specific pre-determined actions in response to pre-defined device events. The device events can be alarms when something is detected, or a machine condition, or virtually any hardware or software event that can result in the device issuing an output. On receipt of the output, the HAWK terminal applies the rule relevant to the outputting device and takes whatever action is specified by the rule, which can be no action, or any command possible for the device or devices on the network.  
         [0018]     For example, a simple rule for a HAWK terminal might be “if motion detector  1  senses motion, turn camera  2  to point in a preset direction”. Other rules might be “if camera  2  transmits images, direct camera  2  to transmit its video to digital video recorder  1  on the network for recording”. More complex rules, including some that affect the display shown to the operator or administrator on the screen of the HAWK terminal display device associated with the PC computer supporting the HAWK terminal software and functionalities, may be created, such as “if smoke detector  1  is activated and motion detector  1  has been triggered more than three times in the last hour, and if it is after midnight on a weekend, then adjust camera viewing parameters for a fire condition and display a fire alarm notification to the user”. More regarding rules will be set out below.  
         [0019]     An exemplary more complex and extended security system  10  includes five sites  13 ,  14 ,  16 ,  18 ,  20  connected via network  22 , three of which have a HAWK terminal (site  14  represents a HAWK terminal in a wireless PDA which will be discussed in more detail below). In each of the HAWK terminal sites  13 ,  14  and  16 , the HAWK terminal controls its local group of devices via a local network, or by direct connections to the devices. The HAWK terminal also is the link for its associated group of devices linking them to the network and acts as a server in network  22  Network  22  is may be a limited area network, e.g., an Ethernet network, but may also be the Internet, or another type of communications network.  
         [0020]     Returning to  FIG. 1 , sites  18  and  20  contain devices, a recorder  23  and various sensors  24 ,  25 ,  26 , and  27 , but no HAWK terminals. Each of these devices is connected through a server to the network, and is accessible to each of the other HAWK terminals  15 ,  17  and  19  through the network at its IP address or URL.  
         [0021]     Site  13  includes only a radar sensor  29  and HAWK terminal  17 , while site  14  is simply a HAWK terminal program module running on a personal digital assistant or PDA and that preferably has wireless access to network  22 , whether by cellular, Bluetooth, IEEE 802.11 g, or other technology. HAWK software component  15  at site  14  enables the operator to access to any sensor at any site remotely (wirelessly) using a PDA through the other HAWK terminal as a proxy or directly through the URLs of the recorder  23  or the sensor server  20 .  
         [0022]     Site  16  includes a variety of sensors of different types including a fence sensor  28 , an access control device  30 , RFID sensor  31 , video camera  33 , and video alarm device  35  all connected to HAWK terminal  19 , that links them to the network  22 , and also provides a management functionality as shown in  FIG. 2 .  
         [0023]     As seen in  FIG. 2 , the HAWK terminal displays a scene view or map  37  on which the sensors of the system are all identified by color codes or icons. Below this is an area  39  in which a video view shows video received from a selected sensor camera or a playback of a video from a device such as recorder  23 , which is controlled by the rules of the particular HAWK terminal. The display also includes a situation view  41  that lists a set of events that the rules engine of this terminal has predetermined should be reported to the user, based on the device of the event, the severity, or a more complex determination of a course of action taken in response to events of devices. There is also a device view  43  showing all devices in the system, and a device control/configuration view  45  that allows control of a device therethrough using the interface I/O of the terminal. Controls may be directing a PTZ camera to tilt, pan or zoom in some particular way. Configuration of a device includes setting up the rule for dealing with it in the rules engine of the HAWK terminal.  
         [0024]     Information in each window of the screen of  FIG. 2  may be accessed by mouse click for example or keyboard key stroke. As in any windows environment, these windows can be resized or closed as desired. The set of these Hawk software components and the connections between them are site and operator specific. New software components may be started and plugged in, or stopped and removed based on usage and external and internal events from locally and remotely connected sensors and devices. The contents of the console and attributes of the HAWK software components are described in detail below.  
         [0025]     Security system  10  incorporating Hawk software components allows for control, configuration and visualization of multiple sensors of all types across many sites. For example, HAWK terminal  19  can have a rule that the video from sensor  33  is to be recorded on recorder  23 . When video is available, the HAWK terminal rule will cause the video to be sent to the server  18  and be recorded on recorder  23 . It also allows monitoring and configuration of sensors from a single location. The security system is designed to integrate a variety of sensors including alarm hardware and provide a single platform for complete monitoring, i.e., situational awareness of a site or an arbitrary sized region such as a state, country or global security.  
         [0026]     The organization of the various windows in the display is user-selectable, and other display windows may be set up that are similar to the windows of  FIG. 2 . Some of these other GUI frames are discussed below.  
         [0027]     A. Alarm View  
         [0028]     This is the user interface to all alarms/alert sources that are plugged into the system either locally or remotely. This situation view includes video alarms, fence alarms, access control or breach alarms, radar and other sensor alarms. These may come over the network using some protocol, or be hardwired to the HAWK security system console.  
         [0029]     Alarms are shown in an integrated list view. This view can be sorted by time, sensor type, location, priority, acknowledge state or any other attribute of alarms. Alarm records are saved in a database that is accessible from any authorized security system console on the network. The alarm view also provides the ability to group alarms into situations based on a set of conditions. Situations can be viewed as a whole, or the constituent alarms can be seen. The operator can change the status of an alarm by acknowledging it, ignoring it or turn it off which sends a control signal to the alarming device if applicable.  
         [0030]     This view also lets the operator view additional data related to the alarm such as a video clip, photograph, report or other data about the alarm from the alarming device.  
         [0031]     Alarms are device events, and for each HAWK terminal in the system that has set up a communication rule with the device indicating the alarm condition, there is a rule in its rule engine for what action or actions if any are to be taken responsive to the alarm, as discussed below in greater detail.  
         [0032]     B. Alarm Query  
         [0033]     This display provides a query interface into a database containing the Alarms. Alarms can be searched by any attribute.  
         [0034]     C. Large Area Visualization  
         [0035]     This scene view displays a combination of schematics, aerial/satellite photographs, maps and 3D models of arbitrarily large regions (up to the entire globe) at varying resolutions and in a variety of formats. It is meant to provide spatial context for a security installation. Interactive navigation is possible over the region. Using the mouse, the user can pan to any latitude and longitude at any heading and zoom in or out in a continuous manner. This component provides the following functionality. 
        Display of sensor location at their correct position with respect to the site.     Display of sensor coverage and strength of coverage if applicable     Animated display of sensor coverage on the ground and above the ground for moving sensors     Animated display of alarms at their reported locations with graphical indications of sensor type, priority and response status. Other graphical attributes may be used to indicate additional attributes of the alarm.     Animated display of tracks from radar, video and other devices. The includes the ability to show individual or fused tracks from external fusion processors.     Display of connected devices at their appropriate locations. This display may be animated if the devices are in motion.     Display of connected users at their location.     Display of security zones     Query of location by point and click and report accurate coordinates as latitude and longitude or in site specific coordinates     Query of properties for entities shown as a graphical object. This includes alarms, devices, sensors, users, zones, etc.     Control of sensors or devices by clicking their graphical representations. This includes alarm devices, recording devices, sensors, control devices and remote the security system consoles.     Configuration of devices by clicking their graphical representations.        
 
         [0048]     D. Video Viewer (VV)  
         [0049]     This is a control to view real-time video streams. Each stream is displayed in an on-screen window and provides control for pause and zoom. This will be simple N×M Matrix of the different video feeds, where N possible video sources will be seen in one of M windows on the screen.  
         [0050]     E. Recorder Controls  
         [0051]     This provides playback, play-reverse, seek, pause, single step and other controls of both Digital Video Recorders and Meta-data (Alarm) recorders. Data recorders for radar and other sensors will also be controlled by this recorder. Essentially this as a device control window for a recorder, and the result of clicking or otherwise activating controls is that the HAWK terminal transmits a command signal over the network to the recorder device directing the indicated action, and receives streamed back over the network video being played back.  
         [0052]     F. PTZ Camera Controls  
         [0053]     This would contain controls to all the PTZ units that are connected to the system. It provides the capability to configure presets, control pan tilt and zoom functions and set up tours. When commands are entered, the HAWK terminal sends the camera commands to modify its viewing parameters, e.g., direction or zoom level. These command transmissions are either local to the HAWK terminal, as in e.g., site  16 , and sent by local connection lines or network, or remote as in camera  24 , in which case the signal is sent via IP address from the HAWK terminal to the IP address of the camera  24 .  
         [0054]     G. Video/Matrix Switcher Controls  
         [0055]     This provides a graphical interface to controlling a Video Matrix switcher that would define what video feeds would go into a bank of monitors.  
         [0056]     H. Direct Hardware Controls  
         [0057]     This is an interactive window in the HAWK terminal that allows the terminal to transmit signals to control external devices such as TTL, Dry contact closures or serial communications. The display shows signals that are received from devices over specific hardware interfaces besides the network interface. It also allows the HAWK terminal to generate signals or dry contact to interact with devices that accept such inputs.  
         [0058]     I. Rule Engine  
         [0059]     The rule engine is at the heart of the security system with Hawk software components. Each HAWK terminal has a rules engine defined by stored data that tells the HAWK terminal what action to take in reaction to some event at a device in the system. The rules engine various components to be connected in a dynamic manner, and it manages and it brokers internal component connections and communications in the site and throughout the network.  
         [0060]     Events are dynamically bound to actions that respond to those events, meaning that if an event occurs, the HAWK system will take an action prescribed by the relevant rule. This enables the security system components to be developed independently and then bound together at run time. The rule engine also starts and stops components as required in response to emerging events.  
         [0061]     The security system supports the following functionalities which are given as examples. However, it should be noted that any other functionality that results from a combination of actions that can be individually realized by the security system is also embraced in this invention. The rule engine can dynamically tie components on the network, and route events between local or remote components which allows new functionality to be realized.  
         [0062]     View Manipulation 
        Mouse controls or scrollbars for navigation over the scene.     View change to look at the position that a camera is pointed.     Alarm triggered viewpoint change: when user clicks on the alarm icon, the rule causes the view to center on the alarm location and zooms to it to a predefined level.        
 
         [0066]     PTZ Based Controls 
        Direct control of a PTZ view using GUI buttons     Map based PTZ control: a PTZ camera points in the direction of the location clicked on the visualization view        
 
         [0069]     Matrix Control 
        Camera selection for an output monitor     PTZ selection for an output monitor        
 
         [0072]     Recorder Control 
        Pause     Play     Stop     Reverse     Frame Forward     Frame Reverse     Seek: slider over time and text entry field for cut/paste time        
 
         [0080]     Rules 
        Connection Rules These define the assembly of components that comprise the security system console. This could be different for different sites of the security system. For instance the security system on a PDA may only have an alarm view, but one on a PC may have the alarm, device and visualization components. All of these views are controlled by the rules of the individual HAWK terminal involved.     Configuration Rules This allows a user to configure the system and set device and visualization parameters. Generally this allows for flexibility and also scalability of the system. It is not a complicated matter to add a large number of new sensors, for example, using this type of rule. In fact, the present system affords especially desirable scalability, meaning increase in the size of the system, because the HAWK terminals are modular and adapted to connect to the network and to flexibly control any devices that the configuration rules for the terminal devise. New devices added to the network can be accessed by their IP addresses or URLs, or by any other method when the appropriate rule for communication of the HAWK terminal to the device is set up.     Event Rules: Setting and editing rules for relationships between detected events and actions of the system. Events and actions are selected from menus and associations are established or modified by the user. These rules guide the run-time behavior of the security system and result in functionality.        
 
         [0084]     Rules can be fairly intricate. For example, a rule might be “responsive to a motion detection sensor alarm, rotate a PTZ camera to cover a specific location”. Another rule could be “responsive to a high number of radar detections by a sensor increase sensitivity of sensors in an area”, or “responsive to available video from a camera, direct the recorder to record it” or “responsive to a change in temperature increase the rate of recording of video from a set of video cameras in the system” 
         [0085]     Generally, a rule is triggered by an event of some sort with a device with which the relevant HAWK terminal is associated, and the responsive action can be anything within the range of viewing, control, management or other capabilities of the HAWK terminal acting as either a front-end interactive device or as a controller/proxy/server connected with the network  22  and with the many devices available thereon directly or through another associated HAWK terminal or locally.  
         [0086]     User Roles  
         [0087]     The HAWK security system supports two distinct user roles: administrator and operator. Administrators configure the various devices into a site-specific security solution. Operators use the system to monitor alarms and video and control sensors and other devices in real-time. The user interface and authorization for configuration and control is customized for the user. The security system features single logon to the network for a user who must be authorized only once.  
         [0088]     The security system is the next step in situational awareness for security at medium-to large-scale facilities. As tactical situations become more complex and the number of sensors grows, security forces are increasingly challenged to quickly interpret and respond to emerging threats. The security system simplifies the task by creating an intuitive visual context that permits the rapid assessment of the type, location and output of multiple alarms, as well as integrated monitoring for video, radar, access control and RFID equipped facilities.  
         [0089]     The HAWK based security system provides the following capabilities:  
         [0000]     Visualization: Multiple-perspective geographic view of a site(s), along with visual display of information about sensor locations, coverage and alarm conditions.  
         [0090]     Control: Ability to set or modify the operational characteristics of various sensors, including: 1) alarm parameters, alarm monitoring times and alarm options including alarm on/off; configuration and on-line control for pan/tilt/zoom (PTZ) cameras, radars, access control systems, RFID and RF location systems and matrix-switchers.  
         [0000]     Storage: Recording and retrieval of raw or processed/analyzed sensor information (data) in time and space.  
         [0000]     Rules: Logic for system behavior that enables users to define system function in response to an external event such as an alarm, a screen event such as a mouse click, or an internal system event, such as an operation completion.  
         [0091]     As apparent from the diagram of  FIG. 1 , the security system is scalable and capable of supporting hundreds (and eventually thousands) of sensors. Expansion simply requires the connection of the new devices to the network with a discrete IP address of URL through which each can be communicated with. Larger systems having LAN networks with numerous devices can also be added by providing a HAWK terminal to act as a local proxy server connecting the LAN to the network and the resources thereon, either through HAWK terminals also acting as servers, or thorough servers linking devices directly to the network.  
         [0092]     The system is consequently easily expandable and able to plug and play new components without disruption to system operation.  
         [0093]     While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof.