Patent ID: 12212901

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all combinations of features described in the embodiments are essential to the solution of the invention.

1. Security System1

FIG.1shows a security system1according to the present embodiment. The security system1comprises one or more monitoring cameras2, a transcoder3, and a monitoring terminal4.

The transcoder3and the monitoring terminal4may be connected to each other via a communication network10. The communication network10may be configured to include various networks such as the Internet, a wide area network (WAN), and a local area network, or a combination thereof. The communication network10may include a connection point by at least one of wired or wireless manners. The communication network10may also be implemented by a dedicated line separated from a public line such as the Internet.

[1.1. Monitoring Camera2]

Each monitoring camera2is an example of the camera, and is configured to perform image capturing of an image capturing region to generate a captured image. Each monitoring camera2may be configured to supply an image that has been captured (also referred to as a ‘captured image’) to the transcoder3. The image may be a moving image, and the monitoring camera2may be configured to always perform the image capturing to generate moving image (also referred to as video) data. For example, each monitoring camera2may be configured to perform image capturing at 30 fps. A time stamp may be added to each frame of the captured image. Note that, the image may also be a still image. In this case, the monitoring camera2may be configured to perform image capturing at a regular timing and to generate a still image. The monitoring camera2may be a visible light camera, or may be an infrared or ultraviolet (X-ray as an example) camera. In a case where the security system1is provided with a plurality of monitoring cameras2, each monitoring camera2may be different from each other in terms of a type (as an example, a manufacturer or a model number).

Each monitoring camera2may be arranged on a road, a parking lot, a lamp or a utility pole, and may also be arranged at an entrance/exit, a passage or the like of a facility. In a case where the security system1is provided with the plurality of monitoring cameras2, each monitoring camera2may be different from each other in terms of an installation position and an image capturing region.

Note that, the facility may be a plant, or may be a school, a house, a station, an airport, a museum, a hospital or a store (a restaurant, as an example), or may also be a theme park, an amusement park or establishments for attraction thereof. Examples of the plant may include a plant for managing and controlling wells such as a gas field and an oilfield and surroundings thereof, a plant for managing and controlling hydroelectric, thermo-electric and nuclear power generations and the like, a plant for managing and controlling environmental power generation such as solar power and wind power, a plant for managing and controlling water and sewerage, a dam and the like, and the like, in addition to chemical and bio industrial plants and the like.

[1.2. Transcoder3]

The transcoder3is an example of the apparatus, and is arranged between the monitoring camera2and the monitoring terminal4. The transcoder3may be configured to compress the captured image from each monitoring camera2and to supply the same to the monitoring terminal4. The transcoder3comprises an image acquisition unit30, a compression unit31, a reproduction unit32, an evaluation acquisition unit33, a storage unit35, a learning processing unit36, a supply unit37, and a transmission unit38.

[1.2(1). Image Acquisition Unit30]

The image acquisition unit30is configured to acquire a captured image. The image acquisition unit30may be configured to acquire captured images from the plurality of monitoring cameras2. Note that, in the present embodiment, as an example, the image acquisition unit30is directly connected to each monitoring camera2, but may also be connected via the communication network10.

The image acquisition unit30may be configured to supply the acquired captured image to the compression unit31, the learning processing unit36, the storage unit35, and the supply unit37. When the image acquisition unit30acquires the captured images from the plurality of monitoring cameras2, the image acquisition unit may be configured to add identification information (also referred to as ‘camera ID’) of the monitoring cameras2to the captured images to be supplied.

[1.2(2). Compression Unit31]

The compression unit31is configured to compress the captured image to generate compressed data. In the present embodiment, as an example, the compression unit31may be configured to generate a compressed image as the compressed data. In addition, the compression unit31may be configured to perform compression irreversibly.

The compression unit31may be configured to change a compression parameter value to sequentially generate compressed images. Further, when the compression parameter value is supplied from the supply unit37, which will be described later, the compression unit31may be configured to apply the compression parameter value to generate a compressed image. The compression unit31may be configured to perform compression within each frame included in the captured image as a moving image, or may be configured to perform compression within continuous frames. For example, the compression unit31may be configured to generate, as the compressed data, a moving image obtained by thinning out frames from the captured image. The compression parameter value may be a value indicating at least one of a compression rate, a resolution, the number of gradations, or a frame rate. The same time stamp as that of the frame of the corresponding captured image may be added to each frame of the compressed image.

The compression unit31may be configured to compress each of the captured images from the plurality of monitoring cameras2. In this case, the compression unit31may be configured to add the camera ID to the compressed image.

The compression unit31may be configured to supply the compressed image to the reproduction unit32, the storage unit35and the transmission unit38. Further, the compression unit31may be configured to supply the compression parameter value used for generation of the compressed image and the captured image before compression corresponding to the compressed image to the learning processing unit36.

[1.2(3). Reproduction Unit32]

The reproduction unit32is configured to generate a reproduced image that reproduces the captured image, from the compressed data (compressed image as an example, in the present embodiment). When the moving image obtained by thinning out frames by the compression unit31is generated as a compressed image, the reproduction unit32may be configured to generate a moving image that reproduces the thinned-out frames, as a reproduced image. Note that, in the present embodiment, as an example, since the compressed image is irreversibly compressed, the reproduced image may not match the captured image. The same time stamp as that of the frame of the corresponding compressed image may be added to each frame of the reproduced image. Further, the same camera ID as that of the corresponding compressed image may be added to the reproduced image. The reproduction unit32may be configured to supply the generated reproduced image to the evaluation acquisition unit33.

[1.2(4). Evaluation Acquisition Unit33]

The evaluation acquisition unit33is configured to acquire an evaluation corresponding to a degree of approximation between the reproduced image and the captured image. The evaluation acquisition unit33may be configured to calculate a degree of approximation between the captured image supplied from the image acquisition unit30and the reproduced image supplied from the reproduction unit32, and to calculate the evaluation from the calculated degree of approximation. The evaluation acquisition unit33may be configured to calculate a degree of approximation between the reproduced image generated from the compressed data and the captured image that is a source of the compressed data. In the present embodiment, as an example, the evaluation acquisition unit33may be configured to calculate a degree of approximation between the captured image and the reproduced image corresponding to the same time stamp and camera ID.

A degree of approximation between images may be a value calculated by a conventionally known method, and may be, for example, a degree of coincidence of analysis results by image analysis. The evaluation corresponding to the degree of approximation may be a binary value of a positive evaluation indicating that the degree of approximation between the captured image and the reproduced image is equal to or higher than a lower limit degree of approximation and a negative evaluation indicating that the degree of approximation is lower than the lower limit degree of approximation degree. The lower limit degree of approximation may be the lowest degree of approximation of degrees of approximation with which a user can check whether an abnormality has occurred in the image capturing region by seeing the reproduced image, or may be arbitrarily set. The user may be an operator or a security guard who operates the monitoring terminal4. The evaluation acquisition unit33may be configured to supply the acquired evaluation to the learning processing unit36together with the time stamp and the camera ID added to the reproduced image.

[1.2(5). Storage Unit35]

The storage unit35is configured to store a variety of information. The storage unit35may be configured to store a model350and one or more image files351.

The model350is configured to output, in response to an input of a new captured image, a compression parameter value to be applied in compression of the captured image. The model350may be subjected to learning processing by the learning processing unit36. Note that, the model350may be any conventionally known machine learning algorithm, such as a neural network, random forest, gradient boosting, logistic regression, and a support vector machine (SVM).

The image file351is configured to store the compressed image supplied from the compression unit31for each monitoring camera2. The image file351may also be configured to further store the captured image before compression. The storage unit35may be configured to erase the captured image after a shorter period, as compared to that for the compressed image. For example, the storage unit35may be configured to erase the captured image after a first period (as an example, one hour or one day) has elapsed since storing the captured image. The storage unit35may be configured to erase the compressed image after a second period (as an example, one week or one month), which is longer than that for the captured image, has elapsed since storing the compressed image.

[1.2(6). Learning Processing Unit36]

The learning processing unit36is configured to perform learning processing of the model350by using learning data including an evaluation acquired by the evaluation acquisition unit33, a captured image corresponding to the evaluation, and a compression parameter value applied in compression of the captured image. The learning processing unit36may be configured to perform learning processing of the model350stored in the storage unit35.

Here, the captured image corresponding to the evaluation may be a captured image, which corresponds to the time stamp and the camera ID supplied together with the evaluation from the evaluation acquisition unit33, among the captured images supplied from the compression unit31to the learning processing unit36. The compression parameter value applied in compression of the captured image may be the compression parameter value supplied together with the captured image from the compression unit31.

[1.2(7). Supply Unit37]

The supply unit37is configured to supply a captured image newly acquired by the image acquisition unit30to the model350subjected to the learning processing by the learning processing unit36. The supply unit37may be configured to supply, to the compression unit31, the compression parameter value output from the model350. Thereby, the compression parameter value, which is output from the model350, in response to the supply of the new captured image by the supply unit37, is applied by the compression unit31, and a compressed image is generated from the new captured image. When the captured images of the plurality of monitoring cameras2are supplied from the image acquisition unit30, the supply unit37may be configured to supply the camera IDs added to the captured images to the compression unit31in association with the compression parameter value, thereby causing the same to be applied to compression of the captured images to which the camera IDs are added.

[1.2(8). Transmission Unit38]

The transmission unit38is configured to transmit the compressed data (compressed image as an example, in the present embodiment) to the monitoring terminal4. The transmission unit38may be configured to sequentially supply the compressed images supplied from the compression unit31to the monitoring terminal4via the communication network10.

[1.3. Monitoring Terminal4]

The monitoring terminal4is configured to sense an abnormality in the image capturing region by using the compressed data (compressed image as an example, in the present embodiment) transmitted from the transcoder3. The monitoring terminal4includes a reception unit40, a reproduction unit42, a display unit44, and a determination unit45.

[1.3(1). Reception Unit40]

The reception unit40is configured to receive the compressed data (compressed image as an example, in the present embodiment) from the transcoder3. The reception unit40may be configured to supply the received compressed image to the reproduction unit42and the determination unit45.

[1.3(2). Reproduction Unit42]

The reproduction unit42is configured to generate a reproduced image from the compressed data (compressed image as an example, in the present embodiment), in response to an operation from the user. When the compressed images of the plurality of monitoring cameras2are supplied from the reception unit40, the reproduction unit42may be configured to generate a reproduced image from each of the compressed images. The reproduction unit42may be configured to generate a reproduced image in the same manner as the reproduction unit32of the transcoder3, and may be configured to cause the display unit44to display the generated reproduced image. When an operation to instruct generation of a reproduced image is not performed by the user, the reproduction unit42may be configured not to generate a reproduced image.

[1.3(3). Display Unit44]

The display unit44is configured to display the reproduced image. When the reproduced images of the plurality of monitoring cameras2are supplied from the reproduction unit42, the display unit44may be configured to display the reproduced images together.

[1.3(4). Determination Unit45]

The determination unit45is configured to analyze the compressed data (compressed image as an example, in the present embodiment) and to determine whether an abnormality has occurred in the image capturing region. A determination result may be a binary value of a value indicating that an abnormality has occurred and a value indicating that an abnormality has not occurred, or may be a value of a ratio between 0 and 1 indicating a degree of certainty that an abnormality has occurred.

When a preset detection target is detected in the image capturing region, the determination unit45may be configured to determine that the image capturing region is abnormal. In this case, the determination unit45may be configured to generate feature data indicating a feature of the detection target. The determination unit45may be configured to supply the determination result to the display unit44and to cause the display unit to display the same. When the detection target is detected, the determination unit45may be configured to further supply the feature data thereof to the display unit44and to cause the display unit to display the same.

Here, the detection target may be a photographic subject (also referred to as a target photographic subject) to be detected, among photographic subjects that may exist in the image capturing region. The target photographic subject may be a photographic subject to be noted for security reasons, among photographic subjects that can be detected by image analysis. For example, the target photographic subject may be at least one of a person's face, the whole body of a person (as an example, an adult or a child), clothes, a vehicle (as an example, a running vehicle or a stopped vehicle), a vehicle type, a vehicle color, a license plate, or a specific object (as an example, a bag, a weapon). Further, the target photographic subject may also be a face of a specific person or a specific vehicle.

In a case where the target photographic subject is a person's face, the feature data may indicate a feature of the face in an image. The feature of the face in the image may also be a positional relationship of feature points of the face, a color of eyes, skin, and hair, a color and a shape of an accessory (as an example, glasses and piercings), and the like. In addition, in a case where the target photographic subject is a vehicle, the feature data may indicate a vehicle type of the vehicle in an image. Further, in a case where the target photographic subject is a license plate, the feature data may indicate a license number of a vehicle in an image.

An image of an analysis target for which the determination unit45performs image analysis may be a frame extracted from a moving image as a reproduced image. Note that, the determination unit45may be a model obtained by machine learning such as deep learning, but is not limited thereto. In addition, the detection target by the determination unit45may also be an operation performed over a plurality of frames (as an example, an operation of stealing a commercial product). In this case, the determination unit45may be configured to supply the display unit44with a determination result, which indicates that the image capturing region is abnormal, and feature data, which indicates a feature of the photographic subject that has performed an operation, in response to detection of a specific operation, and to cause the display unit to display the same.

According to the above-described transcoder3, an evaluation corresponding to a degree of approximation between a reproduced image reproduced from compressed data (compressed image as an example, in the present embodiment) and a captured image that is a source of the compressed data is acquired, and the learning processing of the model350configured to output, in response to an input of a new captured image, a compression parameter value to be applied in compression of the captured image, is performed by using learning data including the evaluation, a captured image corresponding to the evaluation, and a compression parameter value applied in compression of the captured image. Therefore, the compression parameter value output from the model350to which the captured image is input is used to compress the captured image, so that compressed data with high reproducibility can be generated.

In addition, the compression parameter value is changed, so that the compressed data is sequentially generated. Therefore, the learning processing of the model350can be performed by using learning data including a plurality of compression parameter values having different values and an evaluation corresponding to a degree of approximation between a captured image and a reproduced image when the compression parameter values are applied. Therefore, the compressed data with high reproducibility can be reliably generated.

Further, the captured image is irreversibly compressed to generate the compressed data. Therefore, it is possible to generate compressed data with a small amount of data. Therefore, the captured image is irreversibly compressed by using the compression parameter value output from the model350to which the captured image is input, so that compressed data with high reproducibility and a small amount of data can be generated.

Further, a moving image obtained by thinning out frames from the captured image is generated as compressed data, and a moving image that reproduces the thinned-out frames is generated as a reproduced image. Therefore, by using, in the learning processing, the evaluation corresponding to the degree of approximation between the reproduced image and the captured image and the compression parameter value applied to the compression, it is possible to generate compressed data with high reproducibility with respect to the captured image before the frame thinning.

Further, the compression parameter value that is output from the model350, in response to supply of a newly acquired captured image, is applied to the model350subjected to the learning processing by the learning processing unit36, so that compressed data is generated from the new captured image. Therefore, compressed data with high reproducibility can be appropriately generated from the newly acquired captured image.

Further, since the compressed data is transmitted to the monitoring terminal4, it is possible to monitor the image capturing region with the monitoring terminal4.

Further, since the monitoring terminal4analyzes the compressed data and determines whether an abnormality has occurred in the image capturing region, it is possible to sense an abnormality from the compressed image. Further, since the reproduced image is generated from the compressed data, in response to the operation from the user, it is possible to prevent the reproduced image from being generated unnecessarily.

2. Operations

[2.1. Learning Step]

FIG.2shows an operation in a learning step of the security system1according to the present embodiment. The transcoder3of the security system1learns the model350by performing processing of steps S11to S25.

First, in step S11, the image acquisition unit30acquires a captured image from each monitoring camera2.

In step S13, the compression unit31compresses the captured image to generate compressed data (compressed image as an example, in the present embodiment). The compression unit31may compress the captured image within a period for which it is not compressed, among the acquired captured images. For example, in a case where the processing of step S13is first performed, the compression unit31may compress the entire captured images so far. In a case where the processing of step S11and thereafter is repeated and the processing of step S13is iteratively performed, the compression unit31may compress the captured image in a period after the captured image compressed in the previous processing of step S13.

In addition, in a case where the processing of step S13is first performed, the compression unit31may apply an initial value of a preset compression parameter value to generate compressed data. In a case where the processing of step S11and thereafter are repeated and the processing of step S13is performed multiple times, the compression unit31may apply a compression parameter value set in step S21, which will be described later, to generate the compressed data. The compression unit31may apply a same compression parameter value or separate compression parameter values to the captured images from the plurality of monitoring cameras2.

In step S15, the reproduction unit32generates a reproduced image that reproduces the captured image, from the compressed data (compressed image as an example, in the present embodiment). The reproduction unit32may generate a reproduced image by restoring the captured image from the compressed image by a conventionally known method.

In step S17, the evaluation acquisition unit33acquires an evaluation corresponding to a degree of approximation between the reproduced image and the captured image. In the present embodiment, as an example, the evaluation acquisition unit33calculates a degree of approximation between the reproduced image and the captured image, and acquires an evaluation by comparing the calculated degree of approximation and the lower limit degree of approximation, but may also acquire the degree of approximation and the evaluation from an external computation apparatus or the like.

In step S19, the learning processing unit36determines whether the processing of step S17has been performed by a reference number of steps. When it is determined that the processing has not been performed by the reference number of steps (step S19: No), the processing proceeds to step S21. When it is determined in step S19that the processing has been performed by the reference number of steps (step S19: Yes), the processing proceeds to step S23.

In step S21, the compression unit31changes the compression parameter value that is applied to the compression. Thereby, the changed compression parameter value is applied in step S13described above, so that compressed data (compressed image as an example, in the present embodiment) is generated. Then, the processing of steps S11to S17is repeated, so that learning data, which includes the evaluation corresponding to the degree of approximation between the reproduced image and the captured image, a captured image corresponding to the evaluation, and the compression parameter value applied in compression of the captured image, is generated by the reference number of steps.

Note that, the compression unit31may increase or decrease the compression parameter value until a negative evaluation is acquired by the evaluation acquisition unit33. For example, when a negative evaluation is not acquired by the evaluation acquisition unit33in the processing of step S17, the compression unit31may change any one (also referred to as ‘first type parameter’) of a compression rate, a resolution, a number of gradations, and a frame rate indicated by the compression parameter values in one direction of increase or decrease each time the processing of step S21is performed. When a negative evaluation is acquired by the evaluation acquisition unit33in the processing of step S17, the compression unit31may change the first type parameter, which has been changed in one direction until then, in an opposite direction, or may increase or decrease any other one, which is different from the first type parameter, of the compression rate, the resolution, the number of gradations, and the frame rate indicated by the compression parameter values.

The compression unit31may change continuously or discontinuously, i.e., stepwise the compression parameter value each time the processing of step S21is performed. When changing discontinuously the compression parameter value, the compression unit31may set an amount of change constant each time the processing of step S21is performed, or may not set the same constant.

In step S23, the learning processing unit36performs learning processing of the model350by using learning data including an acquired evaluation, a captured image corresponding to the evaluation, and a compression parameter value applied in compression of the captured image.

The learning processing unit36may perform the learning processing so that a compression parameter value to be applied so as to generate compressed data of an evaluation that is not negative is output from the model350, in response to an input of a new captured image to the model350. For example, the learning processing unit36may perform the learning processing of the model350so that a compression parameter value to be applied in compression is between a compression parameter value of compressed data corresponding to a positive evaluation and a compression parameter value of compressed data corresponding to a negative evaluation.

The compressed data corresponding to a positive evaluation may be compressed data corresponding to a captured image and a reproduced image for which an evaluation corresponding to a degree of approximation is a positive evaluation. The compressed data corresponding to a negative evaluation may be compressed data corresponding to a captured image and a reproduced image for which an evaluation corresponding to a degree of approximation is a negative evaluation.

The learning processing unit36may detect a boundary between a group of the compression parameter values of the plurality of compressed data corresponding to the positive evaluation and a group of the compression parameter values of the plurality of compressed data corresponding to the negative evaluation. Then, the learning processing unit36may perform the learning processing of the model350so that a compression parameter value to be applied in compression is between a compression parameter value, which is closest to the boundary, of the compression parameter values of the plurality of compressed data corresponding to the positive evaluation, and a compression parameter value, which is closest to the boundary, of the compression parameter values of the plurality of compressed data corresponding to the negative evaluation.

In step S25, the learning processing unit36determines whether the processing of steps S11to S23has been performed by a reference number of iterations. When it is determined that the processing has not been performed by the reference number of iterations (step S23: No), the processing proceeds to step S11. When it is determined that the processing has been performed by the reference number of iterations (step S23: Yes), the processing ends.

According to the above-described operations, since the compression parameter value is increased or decreased until a negative evaluation is acquired, it is possible to generate extreme compressed data (compressed image as an example, in the present embodiment) whose reproducibility is not deteriorated. Therefore, by changing the compression parameter value so as to reduce an amount of data of compressed data, it is possible to generate compressed data with good reproducibility and a small amount of data and to provide the same for learning processing.

In addition, the learning processing of the model350is performed by the learning processing unit33so that the compression parameter value to be applied in compression is between the compression parameter value of the compressed data corresponding to the positive evaluation and the compression parameter value of the compressed data corresponding to the negative evaluation. Therefore, by applying, to the compression, the compression parameter value output from the learned model350, it is possible to generate extreme compressed data whose reproducibility is not deteriorated.

[2.2. Operating Step]

FIG.3shows an operation in an operating step of the security system1according to the present embodiment. The security system1performs monitoring by using the learned model350by performing processing of steps S41to S57.

First, in step S41, the image acquisition unit30acquires a captured image from each monitoring camera2.

In step S43, the supply unit37supplies the captured images to the model350. The supply unit37may supply the captured image of each monitoring camera2to the model350. The supply unit37may supply, to the model350, the captured image within a period for which it is not compressed, among the acquired captured images. For example, in a case where the processing of step S43is first performed, the supply unit37may supply the entire captured images so far to the model350. In a case where the processing of step S41and thereafter are repeated and the processing of step S43is iteratively performed, the supply unit37may supply, to the model350, the captured image in a period after the captured image supplied in the previous processing of step S43.

In step S45, the supply unit37acquires the compression parameter value output from the model350. The supply unit37may acquire the compression parameter value from the model350for each captured image, i.e., for each monitoring camera2that has captured the captured image.

In step S47, the compression unit31applies the compression parameter value from the supply unit37to compress the captured image. The compression unit31may perform compression by applying the compression parameter value, which is output from the model350, in response to the input of the captured image, to the captured image.

In step S49, the transmission unit38transmits compressed data (compressed image as an example, in the present embodiment) to the monitoring terminal4. Thereby, in step S51, the reception unit40of the monitoring terminal4receives the compressed data. When the processing of step S49ends, the transcoder3may shift the processing to step S41.

In step S53, the determination unit45of the monitoring terminal4analyzes the compressed data (compressed image as an example, in the present embodiment) and determines whether an abnormality has occurred in the image capturing region. The determination unit45may cause the display unit44to display a determination result. In addition, when a detection target is detected in the image capturing region, the determination unit45may cause the display unit44to further display feature data thereof.

In step S55, the reproduction unit42of the monitoring terminal4determines whether an operation for displaying a reproduced image is performed. When it is determined in step S53that the operation is not performed (step S55: No), the processing may proceed to step S51. When it is determined in step S53that the operation has been performed (step S55: Yes), the processing may proceed to step S57.

In step S57, the reproduction unit42generates a reproduced image from the compressed data (compressed image as an example, in the present embodiment), and causes the display unit44to display the same. This allows the user to visually recognize the reproduced image and to check presence or absence of an abnormality. Note that, when the compressed data of the plurality of monitoring cameras2is supplied from the reception unit40, the reproduction unit42may generate reproduced images from the compressed data, respectively, and cause the display unit44to display the same. When the processing of step S57ends, the monitoring terminal4may shift the processing to the above-described processing of step S51.

3. Modified Embodiments

Note that, in the above-described embodiment, the transcoder3has been described as including the storage unit35, the supply unit37and the transmission unit38, but may not include one of them. For example, in a case where the transcoder3does not include the storage unit35, the learning processing unit36may be configured to perform learning processing on the model350in an externally connected storage apparatus.

In addition, the compression unit31has been described as generating one compressed data from one captured image, but may generate a plurality of compressed data different from each other. For example, the compression unit31may be configured to generate a plurality of compressed data by applying separate compression parameter values to the captured images added with the same time stamp. In this case, the reproduction unit32may be configured to generate a reproduced image from each compressed data. Further, the evaluation acquisition unit33may be configured to acquire evaluations corresponding to degrees of approximation between the captured images and the plurality of reproduced images generated from the captured images. Thereby, the evaluation acquisition efficiency, and further, the learning efficiency of the model350can be improved.

In addition, the learning processing unit36has been described as performing the learning processing of the model350so that a compression parameter value to be applied in compression is between a compression parameter value of compressed data corresponding to a positive evaluation and a compression parameter value of compressed data corresponding to a negative evaluation, but may also be configured to perform the learning processing so that the compression parameter value becomes other value. For example, the learning processing unit36may be configured to perform the learning processing of the model350so that a compression parameter value to be applied in compression becomes a compression parameter value, which is closest to a compression parameter value corresponding to a negative evaluation, of compression parameter values of compressed data corresponding to a positive evaluation. As an example, the learning processing unit36may be configured to detect a boundary between a group of compression parameter values of a plurality of compressed data corresponding to a positive evaluation and a group of compression parameter values of a plurality of compressed data corresponding to a negative evaluation. Then, the learning processing unit36may be configured to perform the learning processing of the model350so that a compression parameter value to be applied in compression becomes a compression parameter value, which is closest to the boundary, of the compression parameter values of the plurality of compressed data corresponding to the positive evaluation.

Further, the compression unit31has been described as performing compression by applying the compression parameter value indicating at least one of the compression rate, the resolution, the number of gradations, or the frame rate. However, in addition to or instead of this, the compression unit may be configured to perform compression by applying a compression parameter value indicating an image effect. In other words, the compression unit31may be configured to perform compression by applying an image effect indicated by a compression parameter value to a captured image. For example, the image acquisition unit30may be configured to acquire a captured image captured under other image capturing condition different from the reference image capturing condition, and the compression unit31may be configured to generate, as compressed data, an image obtained by performing compression by applying an image effect corresponding to the reference image condition to the captured image. The image capturing condition may be a condition corresponding to a brightness of the image capturing region or the weather, the reference image capturing condition may be an image capturing condition under which a photographic subject is clearly image-captured (for example, an image capturing condition in fine weather during a day), and the other image capturing condition may be an image capturing condition under which a photographic subject may be unclearly image-captured (for example, an image capturing condition at night, snowfall, or rainfall). Applying an image effect corresponding to the reference image capturing condition to a captured image captured under the other image capturing condition different from the reference image capturing condition may mean removing an image effect corresponding to the other image capturing condition from the captured image or setting a captured image captured under the other image capturing condition to an image captured under the reference image capturing condition. As an example, the compression unit31may be configured to apply an image effect corresponding to an image capturing condition in fine weather to a captured image captured under an image capturing condition of snowfall and to generate a compressed image in which snow particles have been removed. When an image effect is applied according to the image capturing condition, the transcoder3may be provided with a sensing unit (not shown) configured to sense an image capturing condition of an image capturing region of each monitoring camera2and to provide the same to the compression unit31, and the sensing unit may be provided with a brightness sensor, a thermometer, a rain gauge, an anemometer, and the like. Further, when an image effect is applied according to the image capturing condition, the transmission unit38may be configured to transmit, to the monitoring terminal4, a compressed image and identification information indicating an image capturing condition of an image capturing region. Thereby, it is possible to notify the user of the monitoring terminal4that a feature of an image corresponding to the image capturing condition has disappeared due to compression.

Further, when an image effect is applied by the compression unit31, according to an image capturing condition of an image capturing region, the reproduction unit32may be configured to generate, as a reproduced image, an image for which reproduction has been performed by applying, to an image of compressed data, an image effect corresponding to an image capturing condition (i.e., the other image capturing condition described above) of the image capturing region. An image, for which compression has been performed by applying an image effect corresponding to the reference image capturing condition to a captured image captured under the other image capturing condition different from the reference image capturing condition, is generated as compressed data, and an image for which reproduction has been performed by applying an image effect of the other image capturing condition to the image of the compressed data is generated as a reproduced image. Therefore, by using a degree of approximation between the reproduced image and the captured image and the compression parameter value applied to the compression in the learning processing, it is possible to generate compressed data with high reproducibility with respect to the captured image captured under the other image capturing condition different from the reference image capturing condition. Note that, the description ‘reproduction is performed by applying an image effect’ may mean performing reproduction by applying an image effect, or performing reproduction (as an example, reproduction performed by the reproduction unit32in the above embodiment) in addition to the applying of the image effect.

Further, the evaluation acquisition unit33has been described as acquiring the evaluation corresponding to the degree of approximation between the captured image and the reproduced image, but may also be configured to acquire an evaluation (also referred to as integrated evaluation) corresponding to the degree of approximation and an amount of data of compressed data. In this case, the compression unit31may be configured to acquire an amount of data of the compressed data and to supply the same to the evaluation acquisition unit33. For example, the compression unit31may be configured to supply the amount of data of the compressed data to the evaluation acquisition unit33via the reproduction unit32, and as an example, may be configured to supply the amount of data of the compressed data to the reproduction unit32together with the compressed data, and to cause the reproduction unit32to supply the amount of data of the compressed data to the evaluation acquisition unit33together with the reproduced image. In addition, in a case where the integrated evaluation corresponding to the degree of approximation and the amount of data of the compressed data is acquired by the evaluation acquisition unit33, the learning processing unit36may be configured to perform learning processing by using the learning data including the integrated evaluation. Thereby, it is possible to generate compressed data with high reproducibility with respect to the original captured image and with a small amount of data. The integrated evaluation may be a negative evaluation, regardless of an evaluation corresponding to visibility, when the amount of data of the compressed data is larger than a preset allowable amount of data. In addition, when the amount of data of the compressed image is equal to or smaller than the allowable amount of data, the integrated evaluation may be the evaluation itself corresponding to the visibility. The integrated evaluation may be a binary value of a positive evaluation and a negative evaluation. The allowable amount of data may be set based on a communication band of the communication network10, and as an example, may be an amount of data with which compressed images can be sequentially displayed on the monitoring terminal4without missing of information. Instead of this, the allowable amount of data may be set based on a communication fee in the communication network10, and may be an amount of data in which a communication fee corresponding to a data communication amount is equal to or less than an allowable fee.

Further, the compression unit31has been described as generating a compressed image as compressed data, but may also be configured to generate a feature map of the captured image. The feature map may be data indicating features that are extracted in image recognition processing of the captured image, or may be data that is extracted in the middle of the image recognition processing. For example, the feature map may be a set (also referred to as a vector) of values that are output at an intermediate layer of a determination model (as an example, the model used by the determination unit45in the above-described embodiment) learned by deep learning so as to perform image recognition and to determine presence or absence of an abnormality in an image. Thereby, the amount of data of the compressed data can be reduced, as compared to a case where the compressed data is image data. In a case where the compression unit31generates a feature map, the compression unit31may be a part from an input layer to the intermediate layer of the above-described determination model, and the determination unit45of the monitoring terminal4may be a part from the intermediate layer to an output layer of the determination model. Thereby, the determination unit45can determine presence or absence of an abnormality from the feature map.

Here, when the feature map is generated as compressed data, the reproduction unit32may be configured to decompress (also referred to as expand) the compressed data to generate a reproduced image. In a case where the feature data is a vector output at the intermediate layer, the reproduction unit32may have a generative model configured to generate a reproduced image from the feature map. The generative model may be learned in advance by an algorithm using a machine learning or deep learning method represented by GAN (Generative Adversarial Network) or the like. The GAN is a generative adversarial network using a determination model configured to determine whether an input image is a captured image. In the GAN, learning of the determination model and learning of the generative model may be performed alternately. In the learning of the determination model, learning may be performed so that a determination result to the effect that the input image is not a captured image is output, in response to an input of a reproduced image to the determination model, and a determination result to the effect that the input image is a captured image is output, in response to an input of a captured image to the determination model. In the learning of the generative model, learning may be performed so that when a generated reproduced image is input to the determination model, a determination result to the effect that the input image is a captured image is output from the determination model.

Further, the learning processing unit36has been described as performing learning processing of the model350common to the plurality of monitoring cameras2, but may also be configured to perform learning processing of the model350different for each monitoring camera2.

Further, the apparatus has been described as a transcoder, but may also be other apparatus. For example, the apparatus may be a learning apparatus and may be configured to generate a learned model350and to supply the same to the transcoder.

In addition, various embodiments of the present invention may be described with reference to flowcharts and block diagrams whose blocks may represent (1) steps of processes in which operations are performed or (2) sections of apparatuses responsible for performing operations. Certain steps and sections may be implemented by dedicated circuitry, programmable circuitry supplied together with computer-readable instructions stored on computer-readable media, and/or processors supplied together with computer-readable instructions stored on computer-readable media. The dedicated circuitry may include a digital and/or analog hardware circuit, or may include an integrated circuit (IC) and/or a discrete circuit. The programmable circuitry may include a reconfigurable hardware circuit including logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, a memory element such as a flip-flop, a register, a field programmable gate array (FPGA) and a programmable logic array (PLA), and the like.

Computer-readable media may include any tangible device that can store instructions to be executed by a suitable device, and as a result, the computer-readable media having instructions to be stored in the device comprise an article of manufacture including instructions that can be executed to provide means for performing operations specified in the flowcharts or block diagrams. Examples of the computer-readable media may include an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, and the like. More specific examples of the computer-readable media may include a floppy (registered trademark) disk, a diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an electrically erasable programmable read-only memory (EEPROM), a static random access memory (SRAM), a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a BLU-RAY (registered trademark) disk, a memory stick, an integrated circuit card, and the like.

Computer-readable instructions may include assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code described in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk (registered trademark), JAVA (registered trademark) and C++, and a conventional procedural programming language such as a ‘C’ programming language or similar programming languages.

Computer-readable instructions may be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, or to programmable circuitry, locally or via a local area network (LAN), wide area network (WAN) such as the Internet, and the like, and the computer-readable instructions may be executed so as to provide means for executing operations specified in the flowcharts or block diagrams. Examples of a processor include a computer processor, a processing unit, a microprocessor, a digital signal processor, a controller, a microcontrollers, and the like.

FIG.4shows an example of a computer2200in which a plurality of aspects of the present invention may be entirely or partially implemented. A program that is installed in the computer2200can cause the computer2200to function as or execute operations associated with the apparatus of the embodiment of the present invention or one or more sections thereof, and/or cause the computer2200to execute the processes of the embodiment of the present invention or steps thereof. Such a program may be executed by a CPU2212so as to cause the computer2200to execute certain operations associated with some or all of the flowcharts and the blocks in the block diagrams described herein.

The computer2200according to the present embodiment includes the CPU2212, a RAM2214, a graphic controller2216and a display device2218, which are mutually connected by a host controller2210. The computer2200also includes input and output units such as a communication interface2222, a hard disk drive2224, a DVD-ROM drive2226and an IC card drive, which are connected to the host controller2210via an input and output controller2220. The computer also includes legacy input and output units such as a ROM2230and a keyboard2242, which are connected to the input and output controller2220via an input and output chip2240.

The CPU2212is configured to operate according to programs stored in the ROM2230and the RAM2214, thereby controlling each unit. The graphic controller2216is configured to acquire image data generated by the CPU2212on a frame buffer or the like provided in the RAM2214or in itself, and to cause the image data to be displayed on the display device2218.

The communication interface2222is configured to communicate with other electronic devices via a network. The hard disk drive2224is configured to store programs and data that are used by the CPU2212within the computer2200. The DVD-ROM drive2226is configured to read programs or data from a DVD-ROM2201, and to provide the programs or data to the hard disk drive2224via the RAM2214. The IC card drive is configured to read programs and data from an IC card, and/or to write programs and data into the IC card.

The ROM2230is configured to store therein a boot program or the like that is executed by the computer2200at the time of activation, and/or a program depending on the hardware of the computer2200. The input and output chip2240may also be configured to connect various input and output units to the input and output controller2220via a parallel port, a serial port, a keyboard port, a mouse port and the like.

A program is provided by a computer-readable medium such as the DVD-ROM2201or the IC card. The program is read from the computer-readable medium, is installed into the hard disk drive2224, the RAM2214or the ROM2230, which are also examples of the computer-readable medium, and is executed by the CPU2212. Information processing described in these programs is read into the computer2200, resulting in cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constituted by realizing the operation or processing of information according to a use of the computer2200.

For example, when communication is performed between the computer2200and an external device, the CPU2212may be configured to execute a communication program loaded onto the RAM2214to instruct the communication interface2222for communication processing, based on processing described in the communication program. The communication interface2222is configured, under control of the CPU2212, to read transmission data stored on a transmission buffer processing region provided in a recording medium such as the RAM2214, the hard disk drive2224, the DVD-ROM2201, or the IC card, and to transmit the read transmission data to a network or to write reception data received from the network to a reception buffer processing region or the like provided on the recording medium.

In addition, the CPU2212may be configured to cause all or a necessary portion of a file or a database, which has been stored in an external recording medium such as the hard disk drive2224, the DVD-ROM drive2226(DVD-ROM2201), the IC card and the like, to be read into the RAM2214, thereby executing various types of processing on the data on the RAM2214. Next, the CPU2212is configured to write the processed data back to the external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and may be subjected to information processing. The CPU2212may be configured to execute, on the data read from the RAM2214, various types of processing including various types of operations, processing of information, conditional judgment, conditional branching, unconditional branching, search and replacement of information and the like described in the present disclosure and specified by instruction sequences of the programs, and to write a result back to the RAM2214. In addition, the CPU2212may be configured to search for information in a file, a database, and the like in the recording medium. For example, when a plurality of entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, is stored in the recording medium, the CPU2212may be configured to search for an entry having a specified attribute value of the first attribute that matches a condition from the plurality of entries, and to read the attribute value of the second attribute stored in the entry, thereby acquiring the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

The program or software modules described above may be stored in the computer-readable medium on the computer2200or near the computer2200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable medium, thereby providing the programs to the computer2200via the network.

While the embodiments of the present invention have been described, the technical scope of the invention is not limited to the above described embodiments. It is apparent to persons skilled in the art that various alterations and improvements can be added to the above-described embodiments. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the present invention.

The operations, procedures, steps, stages and the like of each process performed by an apparatus, system, program, and method shown in the claims, embodiments, or diagrams can be performed in any order as long as the order is not indicated by ‘prior to,’ ‘before,’ or the like and as long as the output from a previous process is not used in a later process. Even if the process flow is described using phrases such as ‘first’ or ‘next’ in the claims, embodiments, or diagrams, it does not necessarily mean that the process must be performed in this order.

EXPLANATION OF REFERENCES

1: security system2: monitoring camera3: transcoder4: monitoring terminal10: communication network30: image acquisition unit31: compression unit32: reproduction unit33: evaluation acquisition unit35: storage unit36: learning processing unit37: supply unit38: transmission unit40: reception unit42: reproduction unit44: display unit45: determination unit350: model351: image file2200: computer2201: DVD-ROM2210: host controller2212: CPU2214: RAM2216: graphic controller2218: display device2220: input and output controller2222: communication interface2224: hard disk drive2226: DVD-ROM drive2230: ROM2240: input and output chip2242: keyboard