Security system having an alert signal generated by sensors or analytics on image data

A device is provided, comprising: an image storage unit which is configured to store image data obtained by a monitoring camera; an obtaining unit which is configured to obtain an alert signal which occurs depending on a situation in a monitored region; and a transmitting unit which is configured to transmit alert corresponding information which includes a content of the alert signal and any one of in-the-period image data in a reference duration including an occurrence time of the alert signal among image data stored in the image storage unit or an address of the in-the-period image data in the image storage unit to a file server accessible to another apparatus different from the device.

The contents of the following Japanese patent application(s) are incorporated herein by reference:2019-231424 filed in JP on Dec. 23, 2019

BACKGROUND

1. Technical Field

The present invention relates to a device, a system, a method, and a storage medium.

2. Related Art

For a conventional system which uses a monitoring camera to perform monitoring, various techniques which provide the access to image data shot by the monitoring camera are proposed (for example, see Patent Literature 1).Patent Literature 1: Japanese Unexamined Patent Application Publication No. 2015-162232

However, storing the image data only in a storage which is not appropriate for simultaneous access (a storage which is externally connected via wireless communication, as one example) increases the time and cost for accessing the image data (the cost for wireless communication, for one example). In addition, storing all the image data in the file server which is appropriate for simultaneous access increases the cost for transmitting the image data to the file server.

SUMMARY

To solve the above-described problem, a first aspect of the present invention provides a device. The device may include an image storage unit which stores image data obtained by a monitoring camera. The device may include an obtaining unit which obtains the alert signal occurring depending on the situation in the monitored region. The device may include a transmitting unit which transmits alert corresponding information which includes a content of the alert signal and any one of in-the-period image data in a reference duration including an occurrence time of the alert signal or an address of the in-the-period image data in the image storage unit, among image data stored in the image storage unit, to a file server accessible to another apparatus different from the device.

A second aspect of the present invention provides a system. The system may include a device in the first aspect. The system may include a file server which stores the alert corresponding information transmitted from the device. The system may include a monitoring terminal which displays a content of an alert signal included in the alert corresponding information and in-the-period image data corresponding to the alert signal.

A third aspect of the present invention provides a method performed in a device. The method may include an image storing step to store the image data obtained by the monitoring camera in the storage device. The method may include an obtaining step to obtain an alert signal which occurs depending on the situation in the monitored region. The method may include a transmitting step to transmit, to the file server accessible to another apparatus different from the device, the alert corresponding information including the content of the alert signal and any one of the in-the-period image data in a reference duration including the occurrence time of the alert signal or the address of the in-the-period image data in the storage device, among the image data stored in the storage device.

A fourth aspect of the present invention provides a storage medium which records a program. The program may cause a computer of the device to serve as an image storage unit which stores the image data obtained by the monitoring camera. The program may cause a computer to serve as an obtaining unit which obtains an alert signal which occurs depending on the situation in the monitored region. The program may cause a computer to serve as a transmitting unit which transmits alert corresponding information which includes a content of the alert signal and any one of in-the-period image data in a reference duration including an occurrence time of the alert signal among the image data stored in the image storage unit or an address of the in-the-period image data in the image storage unit to a file server accessible to another apparatus different from the device.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, the present invention will be described through the embodiments of the invention. However, the embodiments described below are not to limit the claimed invention. In addition, all of the combinations of the features described in the embodiments are not necessarily required in the solution of the invention.

FIG. 1shows the system1according to the present embodiment.

The system1includes one or more security sensors2, one or more monitoring cameras3, one or more devices4, a file server5, a delivery server6, a monitoring terminal7, and one or more mobile terminals8, and performs a remote monitoring of the monitored region.

The monitored region may be a region monitored by the monitoring terminal7, and may include a region where abnormalities are to be detected by the security sensor2(also referred to as a security region) and a region shot by the monitoring camera3(also referred to as a shot region). In the present embodiment, the monitored region is a site of a facility as one example, but it may also be a road, a sideway, or the like. The facility may be a plant, or may be a school, a house, a station, a museum, or the like. Examples of the plant include an industrial plant such as a chemical plant or a biological plant, as well as a plant for managing and controlling a wellhead or its surrounding area of a gas field, an oil field or the like, a plant for managing and controlling power generation such as hydraulic power generation, thermal power generation and nuclear power generation, a plant for managing and controlling energy harvesting such as solar photovoltaic generation, wind power generation or the like, and a plant for managing and controlling water and sewerage, a dam or the like.

Herein, among each component of the system1, the device4, the file server5, the delivery server6, the monitoring terminal7, and the mobile terminal8may be connected with each other via a communication network10. In the figure, the device4, the file server5, the delivery server6, the monitoring terminal7, and the mobile terminal8are connected via the single communication network10, but they may be connected via separate communication networks. In addition, each of the security sensors2and the monitoring terminal7may be further connected via a relay device or a communication network which is not shown.

The communication network10may be composed by including various networks such as the Internet, Wide Area Network (WAN), and Local Area Network, or the combination thereof. The communication network10may include a connection point which uses at least one of wired communication or wireless communication. The communication network10may be achieved by a dedicated line separated from a public line such as the Internet.

Each of the security sensors2detects abnormalities in the security region and thereby detects abnormalities in the monitored region. Each of the security sensors2may be any of an intrusion detection sensor which detects an intrusion into the monitored region, a human sensor which detects a human body within the monitored region, an opening and closing sensor which detects the opening and closing of at least one of a door or a window within the monitored region. Each sensor may be an ultrasound sensor, a microwave sensor, an infrared sensor, a wire sensor, a trap sensor, an electric field sensor, a magnet sensor, or the like.

Each of the security sensors2may output an alert signal in response to a detection of an abnormality. The alert signal may include the occurrence time of the alert signal and the identification information (also referred to as a sensor ID) of the security sensor2which is the source of the alert signal. The occurrence time of the alert signal may be a time when the abnormality is detected by the security sensor2.

Each of the security sensors2may supply the alert signal to the device4. Each of the security sensors2may also supply the alert signal to another device (as one example, the monitoring terminal7).

Each of the monitoring cameras3performs a shooting of the shot region. Each of the monitoring cameras3may supply to the device4the image data obtained through the shooting. The monitoring camera3may be a camera for a visible light or may be a camera for an infrared light or an ultraviolet light (as one example, X-ray).

Note that the number of the monitoring cameras3included in the system1may be equal to or may be different from the number of the security sensors2. For example, the system1may include the same number of monitoring cameras3as that of the security sensors2so that the monitoring camera3is installed for each of the security sensors2. In addition, the number of the monitoring cameras3included in the system1may be smaller than that of the security sensor2so that the monitoring camera3is installed for each group of the plurality of security sensors2. The shot region of the monitoring camera3may be the same as or may be at least partially different from the security region of the security sensor2.

Each of the devices4is connected to at least one of the security sensors2and at least one of the monitoring cameras3. Each of the devices4has an image storage unit40, an analyzing unit41, an obtaining unit42, a position storage unit43, an identifying unit44, and a communication unit45.

The image storage unit40stores the image data obtained by the monitoring camera3. The image storage unit40may store each image data obtained by the plurality of monitoring cameras3.

The image storage unit40may store the image data shot by each of the monitoring cameras3in association with the date and time of the shooting and the identification information (also referred to as camera ID) of the monitoring camera3. The image storage unit40may save the image data for at least a reference period (as one example, five days) and may successively overwrite with the new image data. The capacity of the image storage unit40may be 1 TB as one example.

The analyzing unit41performs an image analysis on image data and outputs an alert signal in response to a detection of a target object to be detected among objects which is possibly included in the image data. The target object may be set for the device4by an operator in advance or may be optionally changed. The analyzing unit41may perform the image analysis on the image data stored in the image storage unit40or may supply the alert signal to the obtaining unit42.

The alert signal output from the analyzing unit41includes at least an occurrence time of the alert signal and a camera ID of the monitoring camera3which shot the image data. The alert signal may further include information (as one example, the type, feature, position in the image data of the target object, or the like) of the target object and the indication that the source of the alert signal is the analyzing unit41. The occurrence time of the alert signal may be the shooting time of the image data in which the target object is detected.

Note that the target object may be an object to be noticed for a security reason among the objects which can be detected through the image analysis. For example, the target object may be at least one of a human face, a whole body of a human (as one example, an adult or a child), clothes, a vehicle (a vehicle in motion, a vehicle which is stopped), a car model, a car color, a license plate, a particular object (a bag, a weapon), or the like. In addition, the target object may be a face of a particular person or a particular vehicle. The target object may also be a person or a vehicle which moves across images (across frames) or may be an action performed across a plurality of images.

The analyzing unit41is one example of the first analyzing unit and may perform the image analysis on the image data through one or more image analysis engines (not shown). As one example, in the present embodiment, the analyzing unit41may perform the image analysis by operating the plurality of image analysis engines in parallel or by operating it at different timings.

Operating the plurality of image analysis engines at different timings may mean, for example, operating one image analysis engine in a period when the image analysis is performed on the image data shot during the daytime and operating another image analysis engine in a period when the image analysis is performed on the image data shot in the nighttime. Operating the plurality of image analysis engines at different timings may also mean operating one image analysis engine in a period when the image analysis is performed on the image data shot with a follow light and operating another image analysis engine in a period when the image analysis is performed on the image data shot with a back light. Operating the plurality of image analysis engines at different timings may also mean operating one image analysis engine in a period when the image analysis is performed on the image data shot in the environment of the sunny weather and operating another image analysis engine in a period when the image analysis is performed on the image data shot in the environment of the cloudy or rainy weather. Operating the plurality of image analysis engines at different timings may also mean operating a different image analysis engine for each reference interval (as one example, one minute). Operating the plurality of image analysis engines at different timings may also mean operating a different image analysis engine such that the target objects are switched during a period when the same (or similar) objects are included in the image data. Switching the target object during a period when the same (or similar) objects are included in the image data may mean, as one example, switching the target object between a face and an action during a period when a person is included in the image data as the object, or may mean switching the target object between a car model and a license plate during a period when a car is included in the image data as the object.

At least one of the target object or the shooting condition of the image data appropriate for the image analysis may be different among the plurality of image analysis engines. The shooting condition of the image data may be the condition determined by the monitoring camera3which performs the shooting or may be the condition determined by the installation situation of the monitoring camera3or the timing of shooting. For example, the shooting condition of the image data may be at least one of the performance of the monitoring camera3which performs the shooting (such as the number of pixels, a frame rate, or the like), the application (indoor, outdoor, or the like), the shooting direction (the bearing, the orientation relative to the light source, the orientation relative to the target object which is possibly included in the image data, or the like), the installed location, (the address, the GPS data, the position relative to the target object which is possibly included in the image data, indoor, outdoor, or the like), the number of target objects possibly included in the image data (single or multiple), the situation of the target object possibly included in the image data (moving, stopped, or the like), the hours of the shooting, the situation of the shot region at a time of the shooting (the color and brightness of the illumination, the weather, or the like), or the like.

Note that the image analysis engine may be a model obtained through the machine learning such as the deep learning as one example, but it is not limited thereto.

The obtaining unit42obtains the alert signal occurring depending on the situation within a monitored region. The obtaining unit42may obtain the alert signal from at least one of the security sensors2, or may obtain the alert signal from the analyzing unit41. The obtaining unit42may supply the obtained alert signal to the identifying unit44.

The position storage unit43stores the position of the shot region of each of the plurality of monitoring cameras3. The position storage unit43may further store the position of the security region of each of the plurality of security sensors2.

The identifying unit44identifies each of the monitoring cameras3having the shot region corresponding to the location where the cause of the alert signal occurs, among the plurality of monitoring cameras3. The identifying unit44may refer to the position storage unit43to identify the monitoring camera3. The identifying unit44may supply the camera ID of the identified monitoring camera3and the alert signal to the communication unit45.

Herein, if the alert signal is output from the security sensor2, the cause of the alert signal may be an abnormality of a target to be detected by the security sensor2and the location where the cause of the alert signal occurs may be in the security region of the security sensor2which outputs the alert signal. In this case, the identifying unit44may refer to the position storage unit43to identify each of the monitoring cameras3having the shot region corresponding to the security region of the security sensor2which outputs the alert signal. The shot region corresponding to the location of the security region of the security sensor2may be the shot region within a reference distance from the security region, may be the shot region in the same floor or building as the security region, or may be the shot region on the route including the security region.

If the alert signal is output from the analyzing unit41, the cause of the alert signal may be the target object and the location where the cause of the alert signal occurs may be the shot region of the monitoring camera3which shoots the image data where the target object is detected. In this case, the identifying unit44may identify the monitoring camera3corresponding to the camera ID included in the alert signal.

The communication unit45communicates with the file server5or the like via the communication network10.

The communication unit45is one example of the transmitting unit and transmits to the file server5the alert corresponding information including in-the-period image data during a reference duration including an occurrence time of the alert signal, among the image data stored in the image storage unit40, and the content of the alert signal. In response to one alert signal being obtained by the obtaining unit42, the communication unit45may transmit the alert corresponding information including the in-the-period image data obtained by each of the monitoring cameras3identified by the identifying unit44among a plurality of monitoring cameras3during a reference duration including the occurrence time of the one alert signal and the content of the one alert signal. The reference duration is two minutes as one example in the present embodiment, but it may be a different duration such as 10 minutes, one day, or the like. Transmitting the in-the-period image data may be transmitting a copy of the in-the-period image data.

Note that the communication unit45may transmit the alert corresponding information further including identification information of the device4(also referred to as a device ID) or the camera ID of the monitoring camera3identified by the identifying unit44, that is, the monitoring camera3which shoots the in-the-period image data.

Furthermore, if the communication unit45receives no alert signal supplied from the identifying unit44, it may transmit the thumbnail image of the image data obtained by each of the monitoring cameras3to the file server5or the monitoring terminal7at periodic timings. Furthermore, the communication unit45may transmit the image data stored in the image storage unit40to the file server5before the image data is overwritten. In this case, the communication unit45may compress and transmit the image data. However, if the image data to be transmitted before overwritten includes image data already transmitted as the in-the-period image data, the communication unit45may transmit the image data in the duration (as one example, 30 minutes) including the in-the-period image data without compression. Thereby, the image data shot at a time close to the in-the-period image data can be saved in the file server5.

The file server5is accessible to another apparatus (in the present embodiment, the monitoring terminal7or the mobile terminal8, as one example) different from the device4and shares the stored file with other apparatuses. The file server5may be simultaneously accessible to a plurality of apparatuses without increasing the communication cost. The file server5has an analyzing unit51and a storage unit52.

The analyzing unit51is one example of the second analyzing unit and uses the image analysis engine to perform the image analysis on the in-the-period image data corresponding to the alert corresponding information (in the present embodiment, the in-the-period image data included in the alert corresponding information). If the file server5receives the alert corresponding information including the alert signal from the analyzing unit41of the device4, the analyzing unit51may perform the image analysis. The analyzing unit51may attempt to detect the same target object as that of the analyzing unit41of the device4of the transmission source of the alert corresponding information by performing the image analysis. For example, the analyzing unit51may attempt to detect the same type of target object based on the type of the target object included in the alert signal.

The image engine of the analyzing unit51may have a higher processing capability required for the image analysis and may have a higher precision of the image analysis than the image analysis engine of the analyzing unit41. As one example, if each image analysis engine of the analyzing unit41and the analyzing unit51is a neural network, the image analysis engine of the analyzing unit51may be a neural net which has a larger number of hierarchies than that of the image analysis engine of the analyzing unit41. The analyzing unit51may supply the analysis result to the storage unit52.

The storage unit52stores the alert corresponding information transmitted from the device4. The storage unit52may store only the alert corresponding information corresponding to the in-the-period image data in which the target object is detected by the analyzing unit51, among the alert corresponding information transmitted by the device4. For the alert corresponding information including the alert signal from the security sensor2, the storage unit52may store each of the alert corresponding information regardless of the analysis result by the analyzing unit51.

The storage unit52may further store the image data of the thumbnail image transmitted from the device4when no alert signal occurs or the image data transmitted by the device4before deleted due to overwriting in association with the device ID of the transmission source. The storage unit52may have a larger capacity than the image storage unit40.

The delivery server6delivers the image analysis engine to each of the devices4. The delivery server6has an analysis engine storage unit61, a shooting condition storage unit62, a detection target storage unit63, a setting unit64, and a supplying unit65.

The analysis engine storage unit61stores a plurality of image analysis engines. At least one of the target object or the shooting condition of the image data appropriate for the image analysis may be different among the plurality of image analysis engines. If the image analysis engine is generated by machine learning, each image analysis engine stored in the analysis engine storage unit61may be updated through the learning of whether the analysis result is correct.

For each of the plurality of devices4, the shooting condition storage unit62stores the shooting condition of the image data obtained by the device4. If the shooting condition changes, the shooting condition storage unit62may store a plurality of shooting conditions. The stored shooting condition may be set by the operator.

Alternatively, the shooting condition to be stored may be set based on the result of the image analysis performed on the image data output from the device4using each image analysis engine within the analysis engine storage unit61. For example, the image analysis may be performed on the image data from one device4using two or more image analysis engines each having a different shooting condition of the image data appropriate for the image analysis and the same target object, among the plurality of image analysis engines within the analysis engine storage unit61. Then, the shooting condition corresponding to the image analysis engine which has detected the largest number of the target objects as a result of the image analysis may be set as the shooting condition of the image data obtained by the one device4. The image data which is the target for the image analysis may be the image data such as the thumbnail image stored in, for example, the file server5.

For each of the plurality of devices4, the detection target storage unit63stores the target object which is set for the device4. The detection target storage unit63may store two or more target objects for one device4. The target object to be stored may be set by the operator or may be optionally changed.

The setting unit64sets the operating condition of each image analysis engine in each of the devices4. The operating condition of the image analysis engine may be the condition of determining the timing at which the image analysis engine performs the image analysis or may be the shooting condition of the image data which is the target of the image analysis. As one example, the operating condition may be that the image data is shot in the daytime or the nighttime. In this case, the image analysis engine performs the image analysis on the image data shot in the daytime or the nighttime.

For the analyzing unit41of each of the devices4, the setting unit64may set the operating condition of the image analysis engine used by the analyzing unit41via the supplying unit65. Thereby, the analyzing unit41of each of the devices4causes each of the image analysis engines to operate with the set operating condition.

The supplying unit65supplies, to at least one of the devices4, any of the plurality of image analysis engines stored in the analysis engine storage unit61. The supplying unit65may supply to the device4the image analysis engine corresponding to at least one of the shooting condition of the image data obtained by the device4or the target objects which are set in the device4.

The supplying unit65may refer to the shooting condition storage unit62to identify the shooting condition of the image data obtained by the device4. The supplying unit65may refer to the detection target storage unit63to identify the target object which is set by the device4.

The supplying unit65may supply the same image analysis engines to each of two or more of the devices4which obtain the image data having at least partially the same shooting condition, among the plurality of devices4included in the system1. Thereby, for example, if each of the two or more of the devices4obtains the image data shot in the same hours, the image analysis engine appropriate for the image data shot in the hours is supplied to each of the two or more of the devices4.

The supplying unit65may supply the same image analysis engine to each of the two or more of the devices4having the same target object, among the plurality of devices4included in the system1. The supplied image analysis engine may be received by the communication unit45of the device4and used by the analyzing unit41.

The monitoring terminal7performs the monitoring of the monitored region. The monitoring terminal7has a display control unit71and a reporting unit72.

If the device4outputs alert corresponding information, the display control unit71displays the in-the-period image data included in the alert corresponding information and the content of the alert signal. If the source of the alert signal is the security sensor2, displaying the content of the alert signal may mean indicating the occurrence time of the alert signal, the sensor ID of the security sensor2, and the content of the abnormality detected by the security sensor2of the sensor ID. If the source of the alert signal is the analyzing unit41, displaying the content of the alert signal may mean indicating the occurrence time of the alert signal, the information on the target object, and the camera ID of the monitoring camera3.

If the device4outputs no alert corresponding information, the display control unit71may list the thumbnail images of the image data obtained from each of the monitoring cameras3via each of the devices4. If one or more pieces of alert corresponding information are output in such a state, the thumbnail images of the image data obtained by the monitoring camera3with each camera ID included in the alert corresponding information may be identified and displayed. Then, in response to the selection by the operator of the thumbnail image of any of the monitoring cameras3, the display control unit71may display the in-the-period image data obtained by the monitoring camera3and the alert signal corresponding to the in-the-period image data.

Alternatively, if one or more pieces of alert corresponding information are output with the thumbnail image listed, the display control unit71may list and display the contents of the alert signal included in each alert corresponding information and display the content of the alert signal and the in-the-period image data corresponding to the alert signal in response to the selection of any of the alert signals by the operator.

Furthermore, if the device4outputs no alert corresponding information with the thumbnail images listed, the display control unit71may, in response to the selection of the thumbnail image of any of the monitoring cameras3by the operator, display the image data obtained from the monitoring camera3by reading it from the image storage unit40of the device4.

In response to the transmission of alert corresponding information from one device4among the plurality of devices4, the reporting unit72performs a reporting to a report destination corresponding to the one device4. The reporting unit72may have a table (not shown) which associates each of the devices4to a report destination and may refer to this table to determine the report destination. The reporting may be performed through a data communication, and the communicated data may include alert corresponding information.

In the present embodiment, the report destination may be the security company in the area including the monitored region of the device4as one example, but it may also be a police station, a fire station, or an emergency hospital.

Each of the mobile terminals8is carried by a security guard of a security company in each area. The mobile terminal8may be able to access the file server5.

[1-8. Advantage Obtained Through the System1]

According to the delivery server6of the above-described system1, the device4is supplied with the image analysis engine corresponding to at least one of the shooting condition of the image data obtained by the device4or the target object which is set for the device4, among the plurality of image analysis engines stored in the analysis engine storage unit61. Thereby, the device4can perform the image analysis using an appropriate analysis engine. For example, if the shooting conditions of the image data are different among the monitoring cameras3, the device4can perform the image analysis by using the image analysis engine corresponding to the shooting condition of the monitoring camera3. Furthermore, if the target object is changed, the device4can perform the image analysis by using the image analysis engine corresponding to the target object after the change.

Furthermore, since the same image analysis engines are supplied to each of two or more of the devices4which obtain the image data having at least partially the same shooting condition, the image analysis can be performed with the same image analysis engine on the image data which is obtained by each of the devices4and has at least partially the same shooting condition.

Furthermore, since the same image analysis engines are supplied to each of two or more of the devices4having the same target objects, each of the devices4which should detect the same target object can perform the image analysis by using the same image analysis engine.

Furthermore, since the operating condition of each image analysis engine within the device4is set by the setting unit64, the device4for which any operating condition is set in advance can perform the image analysis with any operating condition.

Furthermore, according to the device4of the above-described system1, since the alert corresponding information including the in-the-period image data within the reference duration including the occurrence time when the alert signal occurs depending on the situation within the monitored region among the image data stored in the image storage unit40and the content of the alert signal is transmitted to the file server5, the cost of communication can be reduced in comparison with the case in which the image data during the entire period of the shooting is transmitted to the file server5. Furthermore, unlike the case in which the image data is not transmitted to the file server5but is saved only in the image storage unit40of the device4, the time and cost required for accessing the image data can be reduced.

Furthermore, each image data obtained by the plurality of monitoring cameras3is stored in the image storage unit40of the device4and the alert corresponding information is transmitted to the file server5. Therefore, the cost for communication can be significantly reduced in comparison with the case in which the image data of the entire period of the shooting by each of the monitoring cameras3is transmitted to the file server5. Note that, since the file server5is appropriate for the simultaneous access from a plurality of apparatuses and can allow for the constant cost of communication regardless of the amount of the communicated data except the communication with the mobile terminal8, the communication cost can remain constant even with the repeated access to image data.

Furthermore, since each of the monitoring cameras3having the shot region corresponding to the location where the cause of an alert signal occurs among the plurality of monitoring cameras3is identified in response to the alert signal being obtained and the alert corresponding information including the in-the-period image data obtained by each of the monitoring cameras3which is identified is transmitted, each of the situations of the regions related to the alert can be checked.

Furthermore, since the alert signal from the security sensor2is obtained by the obtaining unit42, the alert signal is obtained in response to a danger approaching in the monitored region and the alert corresponding information is transmitted to the file server5. Therefore, monitoring via the file server5can avoid the danger in the monitored region.

Furthermore, since the alert signal is output in response to the detection of the target object which is set for the device4as a result of the image analysis performed on the image data, presetting any target object (as one example, an object which possibly causes a danger) allows the alert corresponding information to be transmitted to the file server5in the presence of the target object. Therefore, through the monitoring via the file server5, a target object present in the monitored region can be found early.

Furthermore, since a plurality of image analysis engines operates in parallel or at different timings in the analyzing unit41, a target object which is present in the monitored region can be assured to be detected.

Furthermore, according to the monitoring terminal7, since the content of the alert signal included in the alert corresponding information and the in-the-period image data corresponding to the alert signal are displayed by the monitoring terminal7, the monitored region can be monitored based on the alert signal and the in-the-period image data in an integrated manner.

Furthermore, since in response to the transmission of the alert corresponding information from one device4among the plurality of devices4the reporting to the report destination corresponding to the one device4is performed, a personnel can be dispatched early to the monitored region.

Furthermore, according to the device4and the file server5, the device4uses one image analysis engine to perform the image analysis on image data and outputs the alert signal in response to the detection of the target object. Furthermore, the file server5uses another image analysis engine to perform the image analysis on the in-the-period image data corresponding to the alert corresponding information and stores only the alert corresponding information corresponding to the in-the-period image data where the same target object as that of the device4is detected. Therefore, unnecessary in-the-period image data where the image analysis in the file server5detects no target object can be prevented from being uselessly stored in the file server5.

Furthermore, since the image analysis engine in the analyzing unit41of the device4requires less processing capability needed for the image analysis than the image analysis engine in the analyzing unit41of the file server5, the load of the image analysis on the device4can be reduced.

FIG. 2shows the stored content of the position storage unit43. In the present embodiment, as one example, the position storage unit43stores the camera ID (for example, such as “c-01”) of the monitoring camera3and the position of the shot region (for example, such as “inside the building entrance”) in association with each other. Furthermore, the position storage unit43stores the sensor ID (for example, such as “s-01”) of the security sensor2and the position of the security region (for example, such as “the entrance automatic door”) in association with each other.

FIG. 3shows the stored content of the shooting condition storage unit62. In the present embodiment, as one example, the shooting condition storage unit62stores, for each of the plurality of devices4, the device ID (for example, such as “aaa1”) of the device4, the camera ID (for example, such as “c-01”) of the monitoring camera3connected to the device4, and the shooting condition of the image data (for example, the monitoring camera3is for the indoor application and the installed location is indoor, or the like) in association with each other.

FIG. 4shows the stored content of the detection target storage unit63. In the present embodiment, as one example, the detection target storage unit63stores, for each of the plurality of devices4, the device ID (for example, such as “aaa1”) of the device4and the target object (for example, such as “a human face”) which is set for the device4in association with each other.

5. The Operation of the System1

[5. 1. The Delivery of the Image Analysis Engine]

FIG. 5shows the operation related to the delivery of the image analysis engine. The system1delivers the image analysis engine to each of the devices4by performing the processes of steps S11to S13. Note that at the start of this operation the analysis engine storage unit61of the delivery server6may have already stored the plurality of image analysis engines for each of which at least one of a shooting condition of the image data appropriate for the image analysis or a target object to be detected is different. Each image analysis engine may include the information indicating the detection object to be detected by the image analysis and the shooting condition of the image data appropriate for the image analysis.

At step S11, the supplying unit65supplies to the device4any of the plurality of image analysis engines stored in the analysis engine storage unit61. The supplying unit65may refer to at least one of the shooting condition storage unit62or the detection target storage unit63to supply to the device4the image analysis engine corresponding to at least one of the shooting condition of the image data obtained by the device4or

the target objects which is set in the device4. Note that in this step S11the supplying unit65may receive from the setting unit64the operating condition of the image analysis engine to be supplied and further supply it to the device4.

The supplying unit65may update the image analysis engine in the device4with the image analysis engine to be supplied or may add into the device4the image analysis engine to be supplied. For example, the supplying unit65may instruct the analyzing unit41of the device4to update or add the image analysis engine. If the supplying unit65instructs the update of the image analysis engine, it may designate the image analysis engine to be updated among the one or more image analysis engines used by the analyzing unit41.

The supplying unit65may supply the image analysis engine in response to the operation by the operator. For example, if at least one of the update or the addition of the image analysis engine stored in the analysis engine storage unit61is performed in response to the operation by the operator, the supplying unit65may supply the image analysis engine to the device4. In this case, the supplying unit65may detect the detection object of each image analysis engine which is updated or added and the shooting condition of the image data appropriate for the image analysis and supply the image analysis engine to the device4corresponding to at least one of these detection object or shooting condition.

Furthermore, if the information of at least one of the detection target storage unit63or the shooting condition storage unit62in response to the operation by the operator is changed, the supplying unit65may supply the image analysis engine. In this case, the supplying unit65may supply the image analysis engine corresponding to the detection object after the change, to each of the devices4corresponding to the detection object changed in the detection target storage unit63. Furthermore, the supplying unit65may supply the image analysis engine corresponding to the shooting condition after the change, to each of the devices4corresponding to the shooting condition changed in the shooting condition storage unit62.

At step S13, the analyzing unit41of the device4obtains the image analysis engine via the communication unit45. If instructed to update the image analysis engine by the supplying unit65, the analyzing unit41updates any of the image analysis engines with the supplied image analysis engine. If instructed to add the image analysis engine by the supplying unit65, the analyzing unit41adds the supplied image analysis engine to the target for use. If receiving the operating condition of the image analysis engine from the supplying unit65, the analyzing unit41performs setting such that the supplied image analysis engine is operated with the operating condition.

According to the above-described operation, since the image analysis engine in the device4is updated with the supplied image analysis engine, the device4can use a new image analysis engine to perform the image analysis even in the presence of the limitation on the storage area in the device4.

Furthermore, since the supplied image analysis engine is added into the device4, the device4can use a plurality of image analysis engines to perform the image analysis.

FIG. 6shows the operation related to the monitoring. The system1monitors the monitored region by performing the processes of steps S21to S37. Note that the device4may cause the image data obtained by the monitoring camera3to be stored in the image storage unit40during this operation. Furthermore, the analyzing unit41of the device4may use one or more image analysis engines to perform the image analysis on the obtained image data.

At step S21, the obtaining unit42of the device4obtains the alert signal in response to the output of the alert signal from at least one of the security sensor2or the analyzing unit41.

At step S23, the identifying unit44identifies each of the monitoring cameras3having the shot region corresponding to the location where the cause of the alert signal occurs, among the plurality of monitoring cameras3.

At step S25, the communication unit45transmits to the file server5the alert corresponding information including in-the-period image data during a reference duration including the occurrence time of the alert signal, among the image data obtained by each of the identified monitoring cameras3and stored in the image storage unit40, and the content of the alert signal. The communication unit45may also transmit to the monitoring terminal7or the mobile terminal8the indication that the alert corresponding information has been output. In the present embodiment, as one example, the communication unit45transmits only the content of the alert signal to the monitoring terminal7or the mobile terminal8, but it may also transmit the alert corresponding information itself along with the in-the-period image data. Note that inFIG. 6the device4ends the process after the transmission of the alert corresponding information, but the process may continue to step S21.

At step S27, the storage unit52of the file server5stores the alert corresponding information. If the alert corresponding information including the alert signal output from the analyzing unit41of the device4is received by the file server5, the analyzing unit51performs the image analysis and, only in the case where the same target object as that of the analyzing unit41in the device4which is the transmission source of the alert corresponding information is detected in the in-the-period image data, the storage unit52may store the alert corresponding information.

At step S31, in response to the reception of the alert signal, the display control unit71of the monitoring terminal7reads from the file server5the alert corresponding information including the alert signal. In response to this, at step S33, the file server5may provide the monitoring terminal7with the access to the storage unit52. Note that, if the device4transmits no alert signal, the display control unit71may receive the alert signal from the security sensor2without using the device4.

At step S35, the display control unit71displays, on one or more displays (not shown), the in-the-period image data and the content of the alert signal included in the alert corresponding information. Note that, if the alert corresponding information including the in-the-period image data is transmitted from the communication unit45of the device4to the monitoring terminal7, the display control unit71may display the in-the-period image data included in the received alert corresponding information and the alert signal without performing the process of step S31.

If the alert corresponding information includes a plurality of in-the-period image data shot by the plurality of monitoring cameras3, the display control unit71may preferentially display the in-the-period image data where an object moving across images is detected, among these in-the-period image data. The detection of the object moving across images may be performed by the analyzing unit51of the file server5or may be performed by the analyzing unit in the monitoring terminal7which is not shown in the figure. Preferentially displaying particular in-the-period image data may mean displaying only the in-the-period image data or may mean displaying the in-the-period image data larger than another in-the-period image data.

Herein, the processes of steps S31and S35may be performed by the mobile terminal8. In this case, the file server5provides the mobile terminal8with the access to the storage unit52and the in-the-period image data and the content of the alert signal are displayed in the mobile terminal8. Thereby, the situation of the monitored region can be checked by a security guard.

At step S37, in response to the transmission of the alert corresponding information from one device4among the plurality of devices4, the reporting unit72of the monitoring terminal7performs the reporting to the report destination corresponding to the one device4.

6. The Modified Example

Note that in the above-described embodiment it has been described that the communication unit45transmits the alert corresponding information including the in-the-period image data shot by each of the monitoring cameras3having the shot region corresponding to the location where the cause of the alert signal occurs, but it may also transmit the alert corresponding information including the shot in-the-period image data regardless of the shot region. For example, in response to one alert signal being obtained by the obtaining unit42, the communication unit45may transmit the alert corresponding information including the plurality of the in-the-period image data obtained by the plurality of the monitoring cameras3during a reference duration including the occurrence time of the one alert signal and the content of the one alert signal. In this case, the situation in a wide region can be checked by the monitoring terminal7.

Furthermore, it has been described that the alert corresponding information includes the alert signal and the in-the-period image data, but the alert corresponding information may include, instead of the in-the-period image data, the address of the in-the-period image data in the image storage unit40. In this case, the communication cost between the device4and the file server5can be reduced in comparison with the case where the alert corresponding information including the in-the-period image data is transmitted. Furthermore, since the in-the-period image data in the image storage unit40is accessed based on the transmitted address, the time or cost required for accessing the image data in the device4can be reduced in comparison with the case where the image data of the entire period in the image storage unit40is accessed and the image data around the occurrence time of the alert signal is searched. Herein, if the alert corresponding information includes the alert signal and the address of the in-the-period image data, the monitoring terminal7may read the in-the-period image data indicated by the address from the image storage unit40of the device4and display the content of the alert signal included in the alert corresponding information and the in-the-period image data corresponding to the alert signal. Furthermore, the monitoring terminal7may cause the in-the-period image data which is read based on the address included in the alert corresponding information to be stored in the file server5in association with the alert corresponding information.

Furthermore, it has been described that the delivery server6has the setting unit64which sets the operating condition of each image analysis engine in each of the devices4, but it may have an instructing unit (not shown) which instructs each image analysis engine in the device4to perform the image analysis. In this case, the image analysis engine can perform the image analysis each time depending on the situation.

Furthermore, it has been described that the device4has the analyzing unit41, the position storage unit43, and the identifying unit44, but it may not have at least one of them. For example, if the device4does not have the analyzing unit41, it may output the alert corresponding information in response to the alert signal from the security sensor2.

Furthermore, it has been described that the file server5has the analyzing unit51, but it may not have the analyzing unit51. In this case, the file server5may store all the received alert corresponding information.

Furthermore, it has been described that the monitoring terminal7and the delivery server6are distinct devices, but they may also be the same device.

Furthermore, it has been described that the delivery server6has the shooting condition storage unit62, the detection target storage unit63, and the setting unit64(or the instructing unit), but it may not have at least one of these units. For example, if the delivery server6does not have the setting unit64(or the instructing unit), the operating condition of the image analysis engine may be set at the side of the device4.

Furthermore, it has been described that the device4stores the image data obtained by the monitoring camera3in the image storage unit40, but the image data may be stored in response to the detection of the target object by the analyzing unit41. In this case, since the amount of the image data saved in the device4can be reduced, the device4may not transmit the alert corresponding data or the metadata to the file server5.

Furthermore, it has been described that the device4has the single analyzing unit41, but they may also have a plurality of analyzing units41of the same type. In this case, the image analysis can be performed in parallel at a fast speed.

Also, various embodiments of the present invention may be described with reference to flowcharts and block diagrams. Blocks may represent (1) steps of processes in which operations are performed or (2) sections of apparatuses responsible for performing operations. Certain steps and sections may be implemented by dedicated circuitry, programmable circuitry supplied with computer-readable instructions stored on computer-readable media, and/or processors supplied with computer-readable instructions stored on computer-readable media. Dedicated circuitry may include digital and/or analog hardware circuits and may include integrated circuits (IC) and/or discrete circuits. Programmable circuitry may include reconfigurable hardware circuits including logical AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, memory elements, etc., such as field-programmable gate arrays (FPGA), programmable logic arrays (PLA), etc.

Computer-readable media may include any tangible device that can store instructions for execution by a suitable device, such that the computer-readable medium having instructions stored therein comprises an article of manufacture including instructions which can be executed to create means for performing operations specified in the flowcharts or block diagrams. Examples of computer-readable media may include an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, etc. More specific examples of computer-readable media may include a floppy (registered trademark) disk, a diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an electrically erasable programmable read-only memory (EEPROM), a static random access memory (SRAM), a compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a BLU-RAY (registered trademark) disc, a memory stick, an integrated circuit card, etc.

Computer-readable instructions may include assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, JAVA (registered trademark), C++, etc., and conventional procedural programming languages, such as the “C” programming language or similar programming languages.

Computer-readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device, or to programmable circuitry, locally or via a local area network (LAN), wide area network (WAN) such as the Internet, etc., to execute the computer-readable instructions to create means for performing operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

FIG. 7shows an example of a computer2200in which a plurality of aspects of the present invention may be implemented entirely or partially. A program that is installed in the computer2200can cause the computer2200to function as or perform operations associated with apparatuses of the embodiments of the present invention or one or more sections thereof, or perform the operations or the one or more sections, and/or cause the computer2200to perform processes of the embodiments of the present invention or steps of the processes. Such a program may be executed by the CPU2212to cause the computer2200to perform certain operations associated with some or all of the blocks of flowcharts and block diagrams described herein.

The computer2200according to the present embodiment includes a CPU2212, a RAM2214, a graphic controller2216, and a display device2218, which are mutually connected by a host controller2210. The computer2200also includes input/output units such as a communication interface2222, a hard disk drive2224, a DVD-ROM drive2226and an IC card drive, which are connected to the host controller2210via an input/output controller2220. The computer also includes legacy input/output units such as a ROM2230and a keyboard2242, which are connected to the input/output controller2220through an input/output chip2240.

The CPU2212operates according to programs stored in the ROM2230and the RAM2214, thereby controlling each unit. The graphic controller2216obtains image data generated by the CPU2212on a frame buffer or the like provided in the RAM2214or in itself, and causes the image data to be displayed on the display device2218.

The communication interface2222communicates with other electronic devices via a network. The hard disk drive2224stores programs and data used by the CPU2212within the computer2200. The DVD-ROM drive2226reads the programs or the data from the DVD-ROM2201, and provides the hard disk drive2224with the programs or the data via the RAM2214. The IC card drive reads programs and data from an IC card, and/or writes programs and data into the IC card.

The ROM2230stores therein a boot program or the like executed by the computer2200at the time of activation, and/or a program depending on the hardware of the computer2200. The input/output chip2240may also connect various input/output units via a parallel port, a serial port, a keyboard port, a mouse port, or the like to the input/output controller2220.

A program is provided by computer readable media such as the DVD-ROM2201or the IC card. The program is read from the computer readable media, installed into the hard disk drive2224, RAM2214, or ROM2230, which are also examples of computer readable media, and executed by the CPU2212. The information processing described in these programs is read into the computer2200, resulting in cooperation between a program and the above-mentioned various types of hardware resources. An apparatus or method may be constituted by realizing the operation or processing of information in accordance with the usage of the computer2200.

For example, when communication is performed between the computer2200and an external device, the CPU2212may execute a communication program loaded onto the RAM2214to instruct communication processing to the communication interface2222, based on the processing described in the communication program. The communication interface2222, under control of the CPU2212, reads transmission data stored on a transmission buffering region provided in a storage medium such as the RAM2214, the hard disk drive2224, the DVD-ROM2201, or the IC card, and transmits the read transmission data to a network or writes reception data received from a network to a reception buffering region or the like provided on the storage medium.

In addition, the CPU2212may cause all or a necessary portion of a file or a database to be read into the RAM2214, the file or the database having been stored in an external storage medium such as the hard disk drive2224, the DVD-ROM drive2226(DVD-ROM2201), the IC card, etc., and perform various types of processing on the data on the RAM2214. The CPU2212may then write back the processed data to the external storage medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored in the storage medium to undergo information processing. The CPU2212may perform various types of processing on the data read from the RAM2214, which includes various types of operations, processing of information, condition judging, conditional branch, unconditional branch, search/replace of information, etc., as described throughout this disclosure and designated by an instruction sequence of programs, and writes the result back to the RAM2214. In addition, the CPU2212may search for information in a file, a database, etc., in the storage medium. For example, when a plurality of entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored in the storage medium, the CPU2212may search for an entry matching the condition whose attribute value of the first attribute is designated, from among the plurality of entries, and read the attribute value of the second attribute stored in the entry, thereby obtaining the attribute value of the second attribute associated with the first attribute satisfying the predetermined condition.

The above-explained program or software modules may be stored in the computer readable media on or near the computer2200. In addition, a storage medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as the computer readable media, thereby providing the program to the computer2200via the network.

EXPLANATION OF REFERENCES