Image processing device, image processing method, and image processing system

The present disclosure provides an image processing device, image processing method, and image processing system that are capable of freely deciding a position of a cutout region when cutting out an image from an original image. The image processing device includes an object detection unit configured to detect an object in a first image, and a cutout region deciding unit configured to decide, as a cutout region, a region positioned in a relative direction based on a position at which the object is detected in the first image, the relative direction varying depending on a detection condition.

TECHNICAL FIELD

The present disclosure relates to image processing devices, image processing methods, and image processing systems.

BACKGROUND ART

Conventionally, various kinds of technologies for cutting out a region of a detection target object such as a person from a captured image have been developed.

For example, Patent Literature 1 describes a technology for detecting moving objects in an image captured by a camera with a fisheye lens and cutting out a circumscribed quadrangle region of each of the detected moving objects. In addition, Patent Literature 2 describes a technology for calculating a central position of a cutout region in a captured image on the basis of a position at which a remarkable point of a target object is detected in the captured image and a previously-stored distance and direction from the remarkable point to a cutout position.

CITATION LIST

Patent Literature

DISCLOSURE OF INVENTION

Technical Problem

However, when using the technology described in Patent Literature 1 or Patent Literature 2, the position of the cutout region cut out from the original image is limited. For example, in the case where positions at which remarkable points are detected are the same, positions of cutout regions are the same according to the technology described in Patent Literature 2 even if cutout target objects are different from each other.

Accordingly, the present disclosure proposes a novel and improved image processing device, image processing method, and image processing system that are capable of freely deciding a position of a cutout region when cutting out an image from an original image.

Solution to Problem

According to the present disclosure, there is provided an image processing device including: an object detection unit configured to detect an object in a first image; and a cutout region deciding unit configured to decide, as a cutout region, a region positioned in a relative direction based on a position at which the object is detected in the first image, the relative direction varying depending on a detection condition.

In addition, according to the present disclosure, there is provided an image processing method including: detecting an object in a first image; and deciding, as a cutout region, a region positioned in a relative direction based on a position at which the object is detected in the first image, the relative direction varying depending on a detection condition.

In addition, according to the present disclosure, there is provided an image processing system including: an object detection unit configured to detect an object in a first image; a cutout region deciding unit configured to decide, as a cutout region, a region positioned in a relative direction based on a position at which the object is detected in the first image, the relative direction varying depending on a detection condition; a cutout image generation unit configured to generate a cutout image by cutting out the cutout region decided by the cutout region deciding unit from the first image; and a storage unit configured to store the generated cutout image.

Advantageous Effects of Invention

As described above, according to the present disclosure, it is possible to freely decide a position of a cutout region when cutting out an image from an original image. Note that the effects described here are not necessarily limited, and any effect that is desired to be described in the present disclosure may be exhibited.

MODE(S) FOR CARRYING OUT THE INVENTION

Note that, in this specification and the drawings, structural elements that have substantially the same function and structure are sometimes distinguished from each other using different alphabets after the same reference numeral. For example, structural elements that have substantially the same function and structure are distinguished into a video cropping unit106aand a video cropping unit106bas necessary. However, when there is no need in particular to distinguish structural elements that have substantially the same function and structure, the same reference numeral alone is attached. For example, in the case where it is not necessary to distinguish the video cropping unit106aand the video cropping unit106bfrom each other, they are simply referred to as the video cropping units106.

In addition, description proceeds in this section “Mode(s) for Carrying Out the Invention” in the following order.1. Basic configuration of image processing system2. Detailed description of embodiment3. Application example4. Modified example
<211. Basic Configuration of Image Processing System>>
<1-1. Basic Configuration>

As specifically described in “2. Detailed description of embodiment” as an example, the present disclosure may be executed in a variety of forms. First, with reference toFIG. 1, a basic configuration of the image processing system according to the embodiment will be described.

As illustrated inFIG. 1, the image processing system according to the embodiment includes a camera10, a storage20, a monitoring terminal22, and a communication network24.

The camera10is an example of the image processing device according to the present disclosure. The camera10is a device for capturing images of an external environment. The camera10may be installed in a place crowded with people or automobiles, a monitoring target place, or the like. For example, the camera10may be installed in a road, a station, an airport, a commercial building, an amusement park, a park, a parking lot, a restricted area, or the like.

In addition, the camera10is capable of generating another image by using a captured image (hereinafter, referred to as an original image), and transmitting the generated image to another device via the communication network24(to be described later). Here, the original image is an example of the first image according to the present disclosure. For example, the original image is an image with the maximum resolution captured by the camera10. For example, the original image may be a 4K image.

For example, the camera10generates another image with smaller data volume on the basis of the original image. This is because it is not preferable to transmit the original image itself to the another device since transmission of the original image with large data volume takes a long time.

Here, examples of the another image generated by the camera10include a shrunken image obtained by simply reducing the resolution of the original image, and a cropped image obtained by cropping (cutting out) a gaze target region. For example, the shrunken image may be a full HD image.

FIG. 2is an explanatory diagram illustrating an example of the shrunken image (shrunken image32). The shrunken image32includes all regions included in the original image. However, as illustrated inFIG. 2, the gaze target region such as a face of a person may be so small in the shrunken image32, and therefore it may be difficult to see the gaze target region. Note that, the regions40illustrated inFIG. 2are regions corresponding to cropping regions (to be described later). In general, the cropping region is set within a frame image. However, inFIG. 2, regions corresponding to the cropping regions in the shrunken image32are referred to as the regions40for convenience of description.

In addition,FIG. 3is an explanatory diagram illustrating an example of a plurality of cropped images (a set52of the cropped images) generated from one original image. Although the cropped images50have the same resolution as the original image, each of the cropped images50includes only a partial region of the original image, as illustrated inFIG. 3. The camera10according to the embodiment basically generates one shrunken image and one or more cropped images from one original image. In such a generation example, a user can check the entire scene captured by the camera10and can check the gaze target region at high resolution. In addition, it is possible to reduce total data volume in comparison with the original image.

Next, with reference toFIG. 4, an internal configuration of the camera10will be described. As illustrated inFIG. 4, the camera10includes an image capturing unit100, a video shrinking unit102, a region setting unit104, a plurality of video cropping units106, and a communication unit108. Note that, althoughFIG. 4shows an example in which there are four video cropping units106, the number of the video cropping units is not limited thereto. For example, there are any number of the video cropping units as long as the minimum number is one.

The image capturing unit100has a function of acquiring the original image by causing an image sensor such as a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) to form an image of video of an outside through a lens.

The video shrinking unit102generates the shrunken image by shrinking the original image acquired by the image capturing unit100down to a predetermined size.

The region setting unit104sets a cropping region in the original image acquired by the image capturing unit100. The cropping region is a source region from which a cropped image is to be generated. For example, the region setting unit104sets the same number of cropping regions as the number of the video cropping units106in the camera10, in the original image acquired by the image capturing unit100.

FIG. 5is an explanatory diagram illustrating an example in which the region setting unit104sets a cropping region. Note that, inFIG. 5, “crop width” represents a width of the cropping region, and “crop height” represents a height of the cropping region.

As illustrated inFIG. 5, the region setting unit104detects a detection target object such as a person300in the original image30, and sets the cropping region40on the basis of a detection position302of the object.

The video cropping unit106is an example of the cutout image generation unit according to the present disclosure. The video cropping unit106generates a cropped image by cutting out the cropping region set by the region setting unit104from the original image acquired by the image capturing unit100.

For example,FIG. 3illustrates four cropped images50respectively generated by the four video cropping units106. As illustrated inFIG. 3, for example, the video cropping unit106agenerates a cropped image50afrom a cropping region corresponding to a region40aillustrated inFIG. 2that is set by the region setting unit104. In addition, the video cropping unit106bgenerates a cropped image50bfrom a cropping region corresponding to a region40billustrated inFIG. 2that is set by the region setting unit104.

Via the communication network24to be described later, the communication unit108exchanges various kinds of information with devices connected with the communication network24. For example, the communication unit108transmits, to the storage20, the shrunken image acquired by the video shrinking unit102and the plurality of cropped images generated by the plurality of video cropping units106.

The storage20is a storage device configured to store the shrunken image and the cropped images received from the camera10. For example, the storage20stores the received shrunken image and the plurality of received cropped images in association with identification information of the camera10and image capturing date and time. Note that, the storage20may be installed in a datacenter, a monitoring center where observers are working, or the like.

The monitoring terminal22is an information processing terminal configured to display the shrunken image and the cropped images generated by the camera10. For example, the monitoring terminal22may be installed in the monitoring center.

Next, details of the configuration of the monitoring terminal22will be described.FIG. 6is a functional block diagram illustrating the configuration of the monitoring terminal22according to the embodiment. As illustrated inFIG. 6, the monitoring terminal22includes a control unit220, a communication unit222, a display unit224, and an input unit226.

The control unit220controls entire operation of the monitoring terminal22by using hardware such as a central processing unit (CPU), random access memory (RAM), and read only memory (ROM) embedded in the monitoring terminal22.

Via the communication network24to be described later, the communication unit222exchanges various kinds of information with devices connected with the communication network24. For example, the communication unit222receives, from the storage20, the shrunken image and the cropped images stored in the storage20.

Note that, it is also possible for the communication unit222to directly receive the shrunken image and the plurality of cropped images generated by the camera10from the camera10.

For example, the display unit224is implemented by a display such as a liquid crystal display (LCD), or an organic light emitting diode (OLED). For example, the display unit224displays a monitoring screen including the shrunken image and the cropped images received from the storage20.

The input unit226includes an input device such as a mouse, a keyboard, or a microphone. The input unit226receives various kinds of input performed by the user on the monitoring terminal22.

The communication network24is a wired or wireless communication channel through which information is transmitted from devices connected with the communication network24. For example, the communication network24may include a public network, various kinds of local area networks (LANs), a wide area network (WAN), and the like. The public network includes the Internet, a satellite communication network, a telephone network, and the like, and the LANs include Ethernet (registered trademark). In addition, the communication network24may include a dedicated line network such as an Internet Protocol Virtual Private Network (IP-VPN).

Note that, the image processing system according to the embodiment is not limited to the above described configurations. For example, the storage20may be integrated with the monitoring terminal22. Alternatively, the image processing system does not have to include the storage20or the monitoring terminal22.

As described above, the region setting unit104sets a cropping region on the basis of the detection position of the object detected in the original image.

Examples of a method for setting the cropping region include a method for setting a cropping region such that a detection position of a detection target object is at the center of the cropping region. According to this setting method, it is possible to generate the cropped image such that the user can easily see the detection target object in the cropped image.

On the other hand, in the case where a region size of an object is larger than the size of a cropping region, sometimes the cropping region does not include a part of the object that the user wants to detect by using such a setting method. Next, with reference toFIG. 7, details of this matter will be described.FIG. 7is an explanatory diagram illustrating an example of setting the cropping region40such that the detection position302of the person300is at the center of the cropping region40.

In general, in many cases, a human face is set as a detection target part when detecting a human. However, in the example illustrated inFIG. 7, the facial region of the person300is out of the cropping region40since the size of the person300is extremely larger than the size of the cropping region40.

Therefore, the camera10according to the embodiment has been developed in view of the above described circumstance. The camera10according to the embodiment is capable of setting a cropping region such that a detection target part is included in the cropping region in accordance with the type of a detection target object.

<<2. Detailed Description of Embodiment>>

The region setting unit104is one of the features of the camera10according to the embodiment. Next, with reference toFIG. 8, details of the configuration of the region setting unit104according to the embodiment will be described.

As illustrated inFIG. 8, the region setting unit104includes an object detection unit120, a detection region calculation unit122, and a cropping region deciding unit124.

The object detection unit120detects an object in the original image on the basis of a set detection condition. For example, in the case where a detection mode is set in advance (as the detection condition) in accordance with the type of a detection target object, the object detection unit120detects the same number of objects as the number of the video cropping units106in the camera10or the like, from the original image. The types of the objects correspond to the set detection mode. In this case, the types of the detection target objects may include a human and an automobile. In addition, the types of the detection target objects may further include a ship, an airplane, a motorcycle, and a bicycle.

For example, in the case where the set detection mode is a “human detection mode”, the object detection unit120detects regions in which people are captured in the original image.

Note that, the detection mode may be set as a mode for detecting only one type of object, or may be set as a mode for detecting a plurality of types of objects such as “human and automobile detection mode”. In addition, the detection mode may be set or changed by an administrator at any timing, for example. Alternatively, a specific detection mode (such as “human detection mode”) is set in the camera10at the time of development of the camera10.

In addition, the types of the objects may be classified into a plurality of stages such as a broad category and a narrow category, and different detection modes may be set for classes in different stages. For example, in the case where the broad category is “automobile”, the narrow categories include “truck”, “standard-sized car”, and the like. In addition, a “truck detection mode”, a “standard-sized-car detection mode”, or the like may be set as the detection mode.

The detection region calculation unit122calculates a region size of the object detected by the object detection unit120. For example, the detection region calculation unit122calculates, as the region size of the object, the region size of the object detected in the original image acquired by the image capturing unit100.

Alternatively, the detection region calculation unit122calculates, as the region size of the object, an average value of region sizes of the object detected in a predetermined number of original images (hereinafter, sometimes referred to as frame images) such as several original images or a few dozen original images that have been captured at the last minute. According to such a calculation example, it is possible to suppress large change in content of cropped images generated from a series of frame images even in the case where the size of the object detected in the series of frame images varies significantly.

Note that, in the modified example, it is also possible for the detection region calculation unit122to calculate the region size of the object by applying an infinite impulse response (IIR) filter, least-squares fitting, or a Kalman filter on a predetermined number of past frame images instead of calculating the average value.

In general, in the case where the region size of the object is increasing, the average value of the region sizes of the object in the past frame images is smaller than the region size of the object in a current frame image. In the modified example, it is possible to control the region size of the object to be calculated such that the region size is kept from becoming too small (in comparison with the region size of the object in the current frame image) even in the case where the region size of the object is increasing.

Alternatively, according to another modified example, it is possible for the detection region calculation unit122to calculate a final region size of the object by adding a predetermined margin to the region size calculated on the basis of the above described calculation method. For example, the detection region calculation unit122may calculate a final region size of the object by enlarging the region size calculated on the basis of the above described calculation method to a predetermined percentage such as 110%. Alternatively, the detection region calculation unit122may calculate a final region size of the object by adding predetermined lengths to the width and the height of the calculated region size of the object.

The cropping region deciding unit124is an example of the cutout region deciding unit according to the present disclosure. On the basis of the size of the object calculated by the detection region calculation unit122, the cropping region deciding unit124decides, as the cropping region, a region positioned in a relative direction based on the detection position of the object in the original image, the relative direction corresponding to the type of the object. For example, in the case where the size of the object calculated by the detection region calculation unit122is larger than the size of the cropping region, the cropping region deciding unit124calculates a position (correction position) moved from the detection position of the object in the original image by a predetermined distance in the direction corresponding to the type of the object, and decides the cropping region such that the calculated correction position is at the center of the cropping region. Note that, for example, the camera10may store a database in which correspondence relations between types of objects, movement directions (correction directions) of cropping regions, and movement distance calculation formulas are stored. In addition, the cropping region deciding unit124is capable of calculating the correction position by using this database.

Alternatively, in the case where the size of the object calculated by the detection region calculation unit122is smaller than the size of the cropping region, the cropping region deciding unit124decides the cropping region such that the detection position of the object in the original image is at the center of the cropping region.

Next, with reference toFIG. 9toFIG. 11, details of the above described functions will be described.FIG. 9is an explanatory diagram illustrating an example of deciding a cropping region in the case where a type of a detection target object is a human. Note that, inFIG. 9, a dashed rectangle represents a cropping region44decided such that a detection position302of a person is at the center of the cropping region44(hereinafter, referred to as a standard cropping region44). In addition, a solid rectangle represents a cropping region40finally decided by the cropping region deciding unit124.

For example, as illustrated inFIG. 9, in the case where the type of the detection target object is a “human” and the region size of the detected person300is larger than the size of the cropping region, the cropping region deciding unit124decides the cropping region40by moving the standard cropping region44in the relative upper direction in the original image30. For example, in such a case, the cropping region deciding unit124first calculates a difference between the height of the upper side of the standard cropping region44and the height of the top of the head of the detect person300. Next, the cropping region deciding unit124calculates a position obtained by moving the detection position302of the person in the upper direction by the calculated difference or more. Subsequently, the cropping region deciding unit124decides the cropping region40such that the calculated position is at the center of the cropping region40.

In general, in the case where the type of the detection target object is a “human”, a human face is designated as the detection target part in many cases. In the decision example 1, it is possible to decide the cropping region such that a face of a detection target person is included in the cropping region.

Note that,FIG. 10is an explanatory diagram illustrating an example of a cropped image (cropped image50) generated by the video cropping unit106on the basis of the cropping region40illustrated inFIG. 9. As illustrated inFIG. 10, the cropped image50is generated by the video cropping unit106cutting out the cropping region40from the original image30.

Alternatively, as illustrated inFIG. 11, in the case where the type of the detection target object is a “large-size automobile” such as a truck, the cropping region deciding unit124decides the cropping region40by moving the standard cropping region44in the relative lower direction in the original image30. For example, in such a case, the cropping region deciding unit124first calculates a difference between the height of the base of the standard cropping region44and the height of the bottom of a tire of the detected truck300. Next, the cropping region deciding unit124calculates a position obtained by moving the detection position302of the truck in the lower direction by the calculated difference. Subsequently, the cropping region deciding unit124decides the cropping region40such that the calculated position is at the center of the cropping region40.

In general, in the case where the type of the detection target object is an “automobile”, a vehicle registration plate is designated as the detection target part in many cases. In the decision example 2, it is possible to decide the cropping region such that a vehicle registration plate of a detection target truck is included in the cropping region.

Note that, in the above description, the example in which the cropping region deciding unit124decides the cropping region by moving the position of the standard cropping region in the direction corresponding to the type of the detection target object has been described. However, the present disclosure is not limited thereto.

In the modified example, it is also possible for the cropping region deciding unit124to decide the cropping region by enlarging the size of the standard cropping region in a direction corresponding to the type of a detection target object on the basis of a detection position of the detection target object. For example, in the example illustrated inFIG. 9, the cropping region deciding unit124may enlarge the standard cropping region44in the upper direction by the difference between the height of the upper side of the standard cropping region44and the height of the top of the head of the detected person300or more, and decide the obtained (rectangular) region as the cropping region.

The configurations according to the embodiment have been described above. Next, with reference toFIG. 12toFIG. 14, operation according to the embodiment will be described. Note that, an operation example will be described in which the camera10includes four video cropping units106and one shrunken image and four cropped images are generated from one original image.

FIG. 12is a flowchart illustrating an operation example according to the embodiment. As illustrated inFIG. 12, first, the image capturing unit100of the camera10acquires an original image by capturing video of an outside when a predetermined timing comes (S101).

Next, the video shrinking unit102generates a shrunken image by shrinking the original image acquired in S101down to a predetermined size (S103).

Subsequently, the camera10performs a “cropped image generation process” (to be described later) the same number of times as the number of the video cropping units106(in other words, four times) (S105to S111).

Next, the communication unit108transmits the shrunken image generated in S103and four cropped images generated in S107to the storage20(S113).

Next, with reference toFIG. 13toFIG. 14, details of operation in the “cropped image generation process” in S107will be described. As illustrated inFIG. 13, first the object detection unit120of the camera10detects a detection target object in the original image acquired in S101, on the basis of a preset detection mode (S151).

Next, the detection region calculation unit122calculates a region size of the object detected in S151. Subsequently, the cropping region deciding unit124determines whether the calculated region size is larger than the size of the cropping region (S153).

In the case where the calculated region size is less than or equal to the size of the cropping region (No in S153), the cropping region deciding unit124decides that the standard cropping region is used as the cropping region. In other words, the cropping region deciding unit124decides the cropping region such that the detection position of the object detected in S151is at the center of the cropping region (S163). Next, the camera10performs operation in S171to be described later.

On the other hand, in the case where the calculated region size is larger than the size of the cropping region (Yes in S153), the cropping region deciding unit124determines whether the type of the object corresponding to the set detection mode is a human (S155). In the case where the type of the detection target object is a “human” (Yes in S155), the cropping region deciding unit124decides the cropping region by moving the standard cropping region in the upper direction in the original image acquired in S101(S157) Next, the camera10performs operation in S171to be described later.

On the other hand, in the case where the type of the object corresponding to the set detection mode is an “automobile” (No in S155and Yes in S159), the cropping region deciding unit124decides the cropping region by moving the standard cropping region in the lower direction in the original image acquired in S101(S161). Next, the camera10performs operation in S171to be described later.

Alternatively, in the case where the type of the object corresponding to the set detection mode is not a human or an automobile (No in S155and No in S159), the cropping region deciding unit124performs operation in S163described above.

Next, with reference toFIG. 14, operation after S163will be described. As illustrated inFIG. 14, the video cropping unit106generates a cropped image by cutting out the cropping region decided in S157, S161, or S163from the original image acquired in S101(S171).

As described with reference toFIG. 4,FIG. 8,FIG. 12toFIG. 14, and the like, the camera10according to the embodiment detects an object from a captured original image, and decides, as a cropping region, a region positioned in a relative direction based on a detection position of the object in the original image, the relative direction varying depending on a detection condition. Therefore, it is possible to change the position of the cutout region depending of the detection condition even if detection positions of objects are the same.

For example, in the case where the size of the detected object is larger than the size of the cropping region, the camera10decides the cropping region such that the correction position obtained by moving the detection position of the object in the original image by a moving distance corresponding to the type of the object in the direction corresponding to the type of the object is at the center of the cropping region. Therefore, it is possible to set the cropping region such that a detection target part of the detection target object is included in the cropping region even in the case where the region size of the detection target object is larger than the size of the cropping region. As a result, visibility of the detection target part (such as human face) in the cropped image can be improved.

In addition, since the method for deciding a cropping region by the cropping region deciding unit124is simple, the camera10can generate cropped images in real time.

In addition, according to the embodiment, it is possible to generate a shrunken image and cropped images simply by the camera10. Accordingly, the camera10does not have to transmit the original image to another device such as a server to generate the shrunken image and the cropped images, and this enables reducing communication traffic.

The embodiment has been described above. Next, an application example of the embodiment will be described. Note that, the configuration of the image processing system according to the application example is the same as the embodiment illustrated inFIG. 1. First, a background where the application example has been developed will be described.

Another problem of the method for setting a cropping region such that a detection position of a detection target object is at the center of the cropping region is that, sometimes a part of the cropping region is out of the original image when the target object is positioned near an edge of the original image.

A known technology for solving this problem proposes a method for setting a position of a cropping target object at a position different from the center of the cropping region. For example, as illustrated inFIG. 15, the known technology decides the cropping region40such that a part of an edge of the cropping region40is identical to a part of an edge of the original image30.

However, in the object detection process, the detection target object is not always surely detected, and sometimes the detection fails. In addition, in the case where the detection fails, the position of the object (person) in the original image30is generally different from the central position410of the cropping region40as illustrated inFIG. 16, for example.

As described above, the known technology includes a problem that a user cannot determine whether detection of a detection target object has failed or the detection target object is near an edge of the original image simply by seeing the cropped image.

As described later, according to the application example, it is possible to clearly show a region out of the original image in a cropped image to a user in the case where a part of the cropped image is out of the original image.

Next, a configuration of the camera10according to the application example will be described. Note that, the structural elements included in the camera10according to the application example are similar to the embodiment described with reference to inFIG. 4andFIG. 6. Hereinafter, only a structural element having a different function from the above-described embodiment will be described.

In the case where a part of the cropping region decided by the cropping region deciding unit124is out of the original image, the video cropping unit106according to the application example generates a cropped image such that the cropped image includes display showing the region out of the original image. In such a case, the video cropping unit106fills the region out of the original image with a predetermined color or a predetermined pattern in the cropped image, for example. In addition, in such a case, the video cropping unit106places a predetermined character string in the region out of the original image. In such a generation example, it is possible to clearly show the region out of the original image in the cropped image to a user.

Next, with reference toFIG. 17toFIG. 18, details of the above description will be described.FIG. 17is an explanatory diagram illustrating an example of the cropping region40decided by the cropping region deciding unit124in the case where the person300is near an edge of the original image30. In the example illustrated inFIG. 17, the region size of the person300is smaller than the size of the cropping region40. Therefore, the cropping region deciding unit124decides the cropping region40such that the detection position302of the person is at the center of the cropping region40. As a result, as illustrated inFIG. 17, a partial region402in the cropping region40becomes out of the original image30.

In addition,FIG. 18is an explanatory diagram illustrating an example of a cropped image (cropped image50) generated by the video cropping unit106in the case where the cropping region40is decided as illustrated inFIG. 17. As illustrated inFIG. 18, the video cropping unit106generates the cropped image50such that the cropped image50includes a region500cut out from the original image30, and an extruded region502that is a region out of the original image30. Here, the cutout region500is identical to the region400included in the original image30in the cropping region40illustrated inFIG. 17.

In addition, as illustrated inFIG. 18, the video cropping unit106fills the extruded region502with a predetermined color, and places a character string such as “Out of sight” in the extruded region502. The character string clearly shows that this region is an extruded region.

This cropped image50can clearly show that the detection of the detection target object has succeeded and the object is near the edge in the original image. In addition, it is also possible to clearly show which of the edges the object is close to in the original image (for example, in the example illustrated inFIG. 18, the object is close to the right edge of the original image).

Note that, the other structural elements in the camera10have functions similar to the embodiment described above.

The configuration according to the application example has been described above. Next, with reference toFIG. 19, operation according to the application example will be described.FIG. 19is a flowchart illustrating a part of operation of the “cropped image generation process” according to the application example. Note that, operation similar to the operation in S101to S163illustrated inFIG. 12andFIG. 13according to the embodiment is also used in the application example. In addition, the operation in S201ofFIG. 19is similar to S171inFIG. 14according to the above described embodiment. Hereinafter, operation after S201will be described.

As illustrated inFIG. 19, the video cropping unit106determines whether a part of the cropping region decided in S157, S161, or S163is out of the original image acquired in S101(S203) after S201. In the case where the part of the cropping region is not out of the original image (No in S203), the camera10finishes the “cropped image generation process”.

On the other hand, in the case where the part of the cropping region is out of the original image (Yes in S203), the region out of the original image in the cropping region in the cropped image generated in S201is filled with a predetermined color or a predetermined pattern (S205).

As described with reference toFIG. 19and the like, the camera10according to the application example generates a cropped image such that the cropped image includes display showing the region out of the original image in the case where the part of the decided cropping region is out of the original image. Therefore, the user can determine whether the detection of the object has failed or the detection of the detection target object has succeeded but the object is near an edge in the original image, simply by seeing the cropped image.

In addition, in the case where the detection target object is near the edge of the original image, the user can recognize which of the edges the object is close to, simply by seeing the cropped image.

In the above described embodiment, the example in which the camera10serves as the image processing device according to the present disclosure has been described. However, the present disclosure is not limited thereto. For example, the monitoring terminal22may serve as the image processing device according to the present disclosure in the case where (the control unit220of) the monitoring terminal22includes all the video shrinking unit102, the region setting unit104, and the plurality of video cropping units106instead of the camera10.

Alternatively, a separately-provided server (not illustrated) may serve as the image processing device according to the present disclosure in the case where the server is capable of communicating with the camera10via the communication network24and the server includes all the video shrinking unit102, the region setting unit104, and the plurality of video cropping units106instead of the camera10. In addition, the server may be integrated with the storage20.

In addition, the example in which there is a preset detection mode has mainly been described in the above described embodiment. However, the present disclosure is not limited thereto. For example, the camera10is capable of identifying the type of the object included in the captured original image, and dynamically setting a detection mode corresponding to the identified type. For example, in the case where the type of the object included in the captured original image is a human only, it is possible for the camera10to identify that the object is a human by calculating a width-height ratio of a detected object region and dynamically set a detection mode (such as a “human detection mode”) corresponding to the identification result.

In addition, according to the above described embodiment, it is also possible to provide a computer program for causing a hardware such as CPU, ROM, and RAM to execute functions equivalent to the video shrinking unit102, the region setting unit104, and the video cropping units106described above. Moreover, it may be possible to provide a recording medium having the computer program stored therein.

An image processing device including:

an object detection unit configured to detect an object in a first image; and

a cutout region deciding unit configured to decide, as a cutout region, a region positioned in a relative direction based on a position at which the object is detected in the first image, the relative direction varying depending on a detection condition.

The image processing device according to (1),

in which the detection condition is associated with a type of a detection target object.

The image processing device according to (2),

in which the type of the detection target object includes a human and an automobile.

The image processing device according to (3),

in which, in the case where the detection target object is a human, the cutout region deciding unit decides, as the cutout region, a region positioned in an upper direction relative to a position at which the human is detected in the first image.

The image processing device according to (3) or (4),

in which, in the case where the detection target object is an automobile, the cutout region deciding unit decides, as the cutout region, a region positioned in a lower direction relative to a position at which the automobile is detected in the first image.

The image processing device according to any one of (3) to (5),

in which the cutout region deciding unit decides the cutout region such that a position obtained by moving the position at which the object is detected in the first image in a relative direction that varies depending on the type of the object is put on the center of the cutout region.

The image processing device according to any one of (3) to (6), in which

a size of the cutout region is decided in advance,

the image processing device further includes a detection region calculation unit configured to calculate a region size of a detected object, and

in the case where the region size calculated by the detection region calculation unit is larger than the size of the cutout region, the cutout region deciding unit decides, as the cutout region, a region positioned in a relative direction based on a position at which the object is detected in the first image, the relative direction varying depending on the type of the detection target object.

The image processing device according to (7), in which

the first image is a frame image in a moving image, and

the detection region calculation unit calculates the region size of the object in the first image on the basis of region sizes of the object calculated in a plurality of frame images before the first image.

The image processing device according to any one of (1) to (8), further including

a cutout image generation unit configured to generate a cutout image by cutting out the cutout region decided by the cutout region deciding unit from the first image.

The image processing device according to (9),

in which, in the case where a part of the cutout region decided by the cutout region deciding unit is out of the first image, the cutout image generation unit generates the cutout image such that the cutout image includes display showing the region out of the first image.

The image processing device according to (9),

in which, in the display showing the region out of the first image, the region out of the first image is filled with a predetermined color or a predetermined pattern.

The image processing device according to (10) or (11),

in which, in the display showing the region out of the first image, a predetermined character string is placed in the region out of the first image.

An image processing method including:

detecting an object in a first image; and

deciding, as a cutout region, a region positioned in a relative direction based on a position at which the object is detected in the first image, the relative direction varying depending on a detection condition.

An image processing system including:

an object detection unit configured to detect an object in a first image;

a cutout region deciding unit configured to decide, as a cutout region, a region positioned in a relative direction based on a position at which the object is detected in the first image, the relative direction varying depending on a detection condition;

a cutout image generation unit configured to generate a cutout image by cutting out the cutout region decided by the cutout region deciding unit from the first image; and

a storage unit configured to store the generated cutout image.

REFERENCE SIGNS LIST