Source: https://patents.google.com/patent/JP5338498B2/en
Timestamp: 2019-12-16 04:40:22
Document Index: 364634826

Matched Legal Cases: ['art 305', 'art 310', 'art 500', 'art 704', 'art 705', 'art 710']

JP5338498B2 - Control device, camera system and program used in surveillance camera system - Google Patents
Control device, camera system and program used in surveillance camera system Download PDF
JP5338498B2
JP5338498B2 JP2009138594A JP2009138594A JP5338498B2 JP 5338498 B2 JP5338498 B2 JP 5338498B2 JP 2009138594 A JP2009138594 A JP 2009138594A JP 2009138594 A JP2009138594 A JP 2009138594A JP 5338498 B2 JP5338498 B2 JP 5338498B2
JP2009138594A
JP2010287965A (en
和則 坂木
2009-06-09 Application filed by ソニー株式会社 filed Critical ソニー株式会社
2009-06-09 Priority to JP2009138594A priority Critical patent/JP5338498B2/en
2010-12-24 Publication of JP2010287965A publication Critical patent/JP2010287965A/en
2013-11-13 Publication of JP5338498B2 publication Critical patent/JP5338498B2/en
The present invention relates to a control device, a camera system, and a program.
2. Description of the Related Art Conventionally, as described in Patent Document 1 below, for example, an imaging system is known that aims to quickly move the imaging direction of an imaging device in a desired direction.
JP 2007-43505 A
The technique described in Patent Literature 1 controls to move the imaging direction of the imaging device based on a rectangular panoramic image. However, there arises a problem that the control based on the panoramic image is not intuitive only when the imaging target corresponding to the panoramic image is a rectangle.
Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to control the imaging direction of the imaging apparatus based on the panoramic image even when the imaging target is other than a rectangle. To provide an intuitive control device, camera system, and program.
In order to solve the above problem, according to an aspect of the present invention, a panorama image display area for displaying a panorama image captured by a camera and an area corresponding to a position specified by the panorama image are enlarged and enlarged. An enlarged image display area for displaying an image; a display control unit that controls to display an image; a camera control unit that controls to capture an area corresponding to a specified position on the panoramic image with the camera; And the display control unit displays an image captured by the camera under the control of the camera control unit in the enlarged image display area, and the panorama image is a circular panorama image. Is done.
The circular panoramic image may be generated based on an image captured a plurality of times by the camera.
The display control unit may switch a panorama image displayed in the panorama image display area from a circular panorama image to a rectangular panorama image in response to a panorama image switching operation.
The display control unit may display a circular panorama image and a rectangular panorama image in the panorama image display area.
Further, the display control unit may be capable of moving the panoramic image display area on the screen.
In order to solve the above problem, according to another aspect of the present invention, a panorama image display area for displaying a panorama image captured by a camera and an area corresponding to a position specified by the panorama image are enlarged. A display control unit for controlling to display an enlarged image display area for displaying an enlarged image, and camera control for controlling the camera to capture an area corresponding to a specified position on the panoramic image. The display control unit displays an image captured by the camera under the control of the camera control unit in the enlarged image display area,
The display control unit is provided with a control device that switches the shape of the panoramic image in response to a panoramic image switching operation.
The display control unit may switch the shape of the panorama image from a circle to a rectangle in response to a panorama image switching operation.
In order to solve the above problem, according to another aspect of the present invention, a camera system including a camera that captures an image and a control device that controls the camera, wherein the control device includes the camera A display that controls to display a panoramic image display area that displays a panoramic image captured in step, and an enlarged image display area that displays an enlarged image by enlarging the area corresponding to the position specified in the panoramic image. A control unit, and a camera control unit that controls the camera to capture an area corresponding to a position specified on the panoramic image, and the display control unit is controlled by the camera control unit. An image captured by a camera is displayed in the enlarged image display area, and the panorama image is a circular panorama image.
In order to solve the above problem, according to another aspect of the present invention, there is provided a camera system including a camera that captures an image and a control device that controls the camera, wherein the control device is a camera. Display control for controlling to display a panoramic image display area for displaying a captured panoramic image and an enlarged image display area for displaying an enlarged image by enlarging an area corresponding to a position specified by the panoramic image. And a camera control unit that controls the camera to capture an area corresponding to a position designated on the panoramic image, and the display control unit is controlled by the camera control unit. A camera system that displays the image captured in step S in the enlarged image display area, and the display control unit switches a shape of the panoramic image according to a panoramic image switching operation. It is provided.
In order to solve the above problem, according to another aspect of the present invention, a panoramic image display area for displaying a circular panoramic image captured by a camera, and the panoramic image among images captured by the camera. An enlarged image display area for displaying an enlarged image by enlarging an area corresponding to the position designated in (1), and an area corresponding to the position designated on the panoramic image. A program for causing a computer to function as a second means for controlling the camera to take an image is provided.
The first means may switch a panorama image displayed in the panorama image display area from a circular panorama image to a rectangular panorama image in response to a panorama image switching operation.
The first means may display a circular panorama image and a rectangular panorama image in the panorama image display area.
Further, the first means may be capable of moving the panoramic image display area on the screen.
According to the present invention, it is possible to provide a control device, a camera system, and a program that can more intuitively control the imaging direction of the imaging device on a panoramic image.
1 is a schematic diagram illustrating an overall configuration of an imaging system according to an embodiment of the present invention. FIG. 2 is a schematic diagram illustrating in detail the configuration of a camera terminal device and a center server in the system of FIG. 1. 6 is a schematic diagram for explaining a pan operation and a tilt operation of the camera terminal device 200. FIG. It is a schematic diagram which shows the display state of a panoramic image and a bird's-eye view image. It is a schematic diagram for demonstrating the process which produces | generates the bird's-eye view image signal DF. It is a schematic diagram for demonstrating the process which produces | generates the bird's-eye view image signal DF. It is a schematic diagram for demonstrating the process which produces | generates the bird's-eye view image signal DF. It is a schematic diagram for demonstrating the process which produces | generates the bird's-eye view image signal DF. It is a schematic diagram which shows the display screen of a monitor. It is a schematic diagram for demonstrating operation by an operation panel. FIG. 10 is a schematic diagram showing a state in which menus such as “Screen Mode”, “View Size”, “Image Codec”, and “Frame Rate” are displayed when the panel “View” is clicked. It is a schematic diagram showing a menu displayed when the panel “Camera” is clicked. It is a schematic diagram which shows each menu displayed when "Presetposition", "Trigger", "Other", and "information" are clicked. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram which shows the example which displayed the panoramic image on the cylindrical curved surface instead of the normal panoramic image, and made it a reference image. It is a schematic diagram for demonstrating the effect by the display of a 1st panoramic image. It is a schematic diagram for demonstrating the effect by the display of a 1st panoramic image. It is a schematic diagram for demonstrating the effect by the display of a 1st panoramic image.
1. First embodiment (an example of an imaging system including a camera terminal device and a server)
(1) Configuration example of imaging system (2) Configuration example of camera terminal device and center server (3) Display operation of first panorama image and second panorama image (4) Processing to generate first panorama image (5) About the GUI (6) Specifying the shooting direction on the first panoramic image
(1) Configuration Example of Imaging System FIG. 1 is a schematic diagram illustrating an overall configuration of an imaging system according to an embodiment of the present invention. The imaging system 100 includes a camera terminal device (IP camera) 200, a center server 300, and a client 400. The plurality of camera terminal devices 200, the center server 300, and the client 400 are connected via a network 500. . Further, a proxy server 600 is connected to the network 500, and a plurality of camera terminal devices 200 are connected to the proxy server 600. A monitor 310 is connected to the center server 300. In the present embodiment, the camera terminal device 200 is a surveillance camera installed indoors or outdoors, for example, and displays an image acquired by the camera terminal device 200 on the monitor 310 of the center server 300, whereby the camera terminal device 200. You can monitor the area where is installed. The client 400 is connected to the camera terminal device 200, the center server 300, or the proxy server 600 via the network 500. Further, the client 400 may have a function of the center server 300 and a function of the monitor 310 described later. Video captured by the camera terminal device 200 is sent to the client 400 via the network 500, and a first panoramic image and a second panoramic image are displayed on the display screen of the client 400 together with a live video described later. In addition, although the IP camera is mentioned as an example of the camera terminal apparatus 200, it is not limited to this, An analog camera may be sufficient.
The panoramic image indicates a range that can be captured by the camera terminal device 200. The panoramic image has various shapes, for example, a circular panoramic image or a rectangular panoramic image as shown in FIG. 4, a ring-shaped panoramic image projected onto a cylindrical surface as shown in FIG. is there. Hereinafter, in the embodiment, a circular panorama image and a rectangular panorama image will be described as examples of the first panorama image and the second panorama image, respectively.
(2) Configuration Example of Camera Terminal Device and Center Server FIG. 2 is a schematic diagram showing in detail the configuration of the camera terminal device 200 and the center server 300 in the system of FIG. Each component of the camera terminal device 200 or the center server 300 shown in FIG. 2 can be configured by hardware (circuit or the like) or an arithmetic processing device such as a CPU and software (program) for causing it to function. The camera unit 201 of the camera terminal device 200 includes an imaging optical system, performs an imaging operation based on an imaging control signal CTa supplied from a terminal control unit 210 described later, and generates a video signal Dcam. In addition, the camera unit 201 supplies the generated video signal Dcam to the signal processing unit 202 via the bus 220. Note that a memory unit 203, an imaging direction control unit 204, a network interface unit 206, a storage unit 207, and a terminal control unit 210 are also connected to the bus 220.
The signal processing unit 202 performs compression processing of the video signal Dcam, and stores the obtained video encoded signal DV in the memory unit 203. Also, the video signal of the first panorama image (hereinafter referred to as “first panorama image signal” DF and the image of the second panorama image is used by using the video signal Dcam obtained by sequentially moving the imaging direction of the camera unit 201. A signal DP (hereinafter referred to as “second panorama image signal”) DP is generated and stored in the storage unit 207. Note that the compression processing of the video signal Dcam, the first panorama image signal DF, or the second panorama image signal is performed. The DP is generated based on a signal processing control signal CTb supplied from the terminal control unit 210 described later.
The imaging direction adjustment unit 205 includes a pan operation motor that moves the camera unit 201 in the left-right direction and a tilt operation motor that moves the camera unit 201 in the up-down direction. The imaging direction adjustment unit 205 adjusts the imaging direction of the camera unit 201 to the direction instructed by the direction control signal CTc by driving the pan operation motor by the drive signal MDp and the tilt operation motor by the drive signal MDt, respectively. In addition, the imaging direction adjustment unit 205 performs an endless pan operation without limiting the operation range.
FIG. 3 is a schematic diagram for explaining the pan operation and tilt operation of the camera terminal apparatus 200, and shows a case where the camera terminal apparatus 200 is mounted on a ceiling or the like. Here, FIG. 3B shows a pan / tilt operation of the camera terminal device 200 according to the present embodiment. On the other hand, FIG. 3A shows a pan / tilt operation of a general camera terminal device for comparison. As shown in FIG. 3A, in a general camera terminal device, pan operation is performed endlessly at 360 °, and tilt operation is performed in a range of 90 ° from the vertical direction to the horizontal direction. On the other hand, the camera terminal device 200 according to the present embodiment can perform the pan operation endlessly at 360 °, and can perform the tilt operation within a range of 220 ° centering on the vertical direction. Therefore, the video signal DF of the first panoramic image can be generated and stored in the storage unit 207 by a method described later. A first panoramic image to be described later is a circular panoramic image, and when operating the imaging direction across the center of the circle, the camera terminal device 200 according to the present embodiment is the shortest to the target imaging direction. Can be moved mechanically. On the other hand, in a general camera terminal device, the camera imaging direction cannot be mechanically moved so as to cross the center of a circle.
The network interface unit 206 is an interface for performing communication between the camera terminal device 200 and the center server 300 via the network 500.
The terminal control unit 210 controls the camera unit 201 by the imaging control signal CTa, controls the signal processing unit 202 by the signal processing control signal CTb, controls the imaging direction control unit 204 by the direction control signal CTc, and sets the imaging direction. Imaging is performed while sequentially moving. Then, the terminal control unit 210 generates the first panorama image signal DF and the second panorama image signal DP based on the obtained video signal Dcam and stores them in the storage unit 207.
The imaging system 100 according to the present embodiment includes a camera terminal device that includes both a live video based on the video signal Dcam and the first panorama image or the second panorama image based on the first panorama image signal DF or the second panorama image signal DP. 200 can be transmitted to the center server 300 and displayed on the monitor 310. In the present specification, the first panorama image and the second panorama image may be referred to as a reference image. FIG. 4 is a schematic diagram illustrating a display state of the second panorama image and the first panorama image. The center server 300 can display either the first panorama image or the second panorama image together with the live video on the monitor 310 according to the user's selection. The center server 300 may be configured to display both the first panorama image and the second panorama image together with the live video. In the right diagram in FIG. 4, a circular panoramic image lacking an image near the center is used as the first panorama image. However, the present invention is not limited to this. One panoramic image may be used.
The camera terminal device 200 is installed on a ceiling or the like, and displays a downward image of the camera terminal device 200. Therefore, as shown in FIG. 4, the first panoramic image is acquired in a range of pan 360 ° and tilt 220 ° with a vertical line extending downward from the camera terminal device 200 as a center, and its outline is circular. It becomes.
Further, the imaging direction adjustment unit 205 is configured to be able to perform an endless pan operation as described above. Accordingly, the second panoramic image stored in the storage unit 207 is, for example, as illustrated in FIG. 4, so that one end portion has an angle difference of “+ 180 °” from the reference direction and the other end portion so that the images do not overlap. Is assumed to have an angle difference of “−180 °” from the reference direction. Further, since the operation range of the pan operation is not limited, the center position of the operation range cannot be set as the reference direction as in the case where the operation range is limited, for example. Accordingly, a reference direction is set in advance, and the second panorama image signal DP is generated so that the reference direction becomes the center position of the second panorama image. Further, the imaging direction control unit 204 generates, as the camera position information signal PM, a signal indicating, for example, an angle difference between the imaging directions with respect to a preset reference direction.
In this way, if the reference direction is set in advance, it is not necessary to determine which direction the second panoramic image is generated by the imaging operation based on which direction. If the camera position information signal PM indicates an angle difference between the image capturing direction and the reference direction, the position in the image capturing direction based on the camera position information signal PM on the second panoramic image can be easily determined.
Also in the tilt operation, a reference direction is set in advance, and the angle difference between the imaging direction and the reference direction is indicated by the camera position information signal PM. Thereby, the position in the imaging direction based on the camera position information signal PM can be easily determined in both the first panorama image and the second panorama image. Regarding the tilting operation, when the camera terminal device 200 is installed on the ceiling, the vertical direction can be set as the reference direction.
The terminal control unit 210 analyzes the command signal CM supplied from the center server 300 via the network interface unit 206, and generates an imaging control signal CTa, a signal processing control signal CTb, and a direction control signal CTc. In addition, the video encoded signal DV stored in the memory unit 203, the first panorama image signal DF, the second panorama image signal DP, and the imaging direction control unit 204 stored in the storage unit 207 are supplied. The camera position information signal PM is sent to the center server 300.
The network interface unit 301 of the center server 300 is an interface for performing communication between the camera terminal device 200 and the center server 300 via the network 500. The network interface unit 301 supplies the video encoded signal DV supplied from the camera terminal device 200 to the decompression processing unit 302. The network interface unit 301 supplies the camera position information signal PM to the position determination processing unit 303 and supplies the first panorama image signal DF and the second panorama image signal DP to the image processing unit 304.
The position determination processing unit 303 determines whether the imaging direction indicated by the camera position information signal PM corresponds to the position of the first panorama image or the second panorama image, and uses the position determination result KP as the image processing unit. 304 is supplied. Here, the camera position information signal PM generates the first panorama image signal DF or the second panorama image signal DP so that the preset reference direction is the center position of the pan operation and tilt operation as described above. Assuming that the camera position information signal PM indicates an angle difference between the reference direction and the current imaging direction, the current imaging direction is at either the overhead image or the second panoramic image based on the camera position information signal PM. It can be easily determined whether it corresponds.
The image processing unit 304 performs image processing based on the position determination result KP, and generates a first panorama image signal DFC of the first panorama image and a second panorama image signal DPC of the second panorama image. As for the second panorama image signal DPC, the second panorama image signal DPC in which the position determined by the position determination result KP is the center position can be generated. In this case, the image processing unit 304 determines a difference amount between the determined position and the center position of the second panoramic image, and based on the determined difference amount, the determined position becomes the center position of the image. The second panoramic image is processed so that That is, the second panorama image display image frame of “± 180 °” is set with the determined position as the center position, and the second panorama image in the region corresponding to the difference amount deviating from the second panorama image display image frame is set. Is pasted into a region without an image to generate a second panoramic image signal DPC of the second panoramic image with the determined position as the center position. Even in the case of the first panorama image signal DFC, in the case of a “first panorama image rotation mode” to be described later, as in the case of the second panorama image signal DPC, the determined position and the first panorama image are detected. A difference amount from the reference position in the pan direction is determined, and the first panoramic image is processed based on the determined difference amount so that the determined position is positioned on the image. In addition, the image processing unit 304 determines the position based on the position determination result KP (the position where the optical axis of the imaging optical system of the camera unit 201 faces) in each of the first panorama image signal DFC and the second panorama image signal DPC. ) Is displayed. Further, the image processing unit 304 supplies the first panorama image signal DFC and the second panorama image signal DPC generated by performing image processing to the display processing unit 305.
The display processing unit 305 uses the video signal Dcam supplied from the decompression processing unit 302 and the first panorama image signal DFC and the second panorama image signal DPC supplied from the image processing unit 304 to generate the display drive signal HD. Generated and supplied to the monitor 310. Further, the display processing unit 305 generates a display drive signal HD by using a GUI video signal generated based on a GUI display control signal CTg supplied from the control unit 310 described later, and supplies the display drive signal HD to the monitor 310.
The monitor 310 drives a display element such as a liquid crystal display element, a plasma display element, or a cathode ray tube on the basis of the display drive signal HD, and the first panorama image, the second panorama image with the imaging direction as the center position, A captured image (live video) of the camera unit 201, a GUI image, and the like are displayed on the screen.
The user interface unit 315 uses a GUI, presents information to the user using the monitor 310, and when a user operation is performed with a pointing device or a keyboard as an operation input unit based on the presented information Then, an operation signal US corresponding to a user operation is supplied to the control unit 310 to perform a desired operation or the like.
Based on the operation signal US and the display state of the monitor 310, the control unit 310 determines what kind of processing the user has requested to select and what kind of processing is executed, and generates a control signal CTm based on the determination result. To control the operation of the center server 300. Further, a command signal CM is generated based on the determination result, supplied to the camera terminal device 200 via the network interface unit 301, and the operation of the camera terminal device 20 is controlled. Further, the control unit 310 generates a GUI display control signal CTg and supplies it to the display processing unit 305.
In the above-described example, the camera terminal device 200 generates the first panorama image signal DF and the second panorama image signal DP from the image signal Dcam. However, the center server 300 or the client 400 performs the first panorama image signal DP. The panoramic image video signal DF and the second panoramic image video signal DP may be generated. In this case, the network interface unit 206 of the camera terminal device 200 sends the video signal Dcam output from the camera block 201 to the center server 300 or the client 400. The center server 300 or the client 400 includes a signal processing unit similar to that of the camera terminal device 200, and can generate the video signal DF of the first panoramic image and the video signal DP of the second panoramic image. The video signal DF of the first panorama image and the video signal DP of the second panorama image are stored in a memory such as a hard disk provided in the center server 300 or the client 400.
(3) Display Operation of First Panorama Image and Second Panorama Image Next, the first panorama image and second panorama image display operation will be described. At the start of operation, the control unit 310 requests the camera terminal apparatus 200 for the first panorama image signal DF, the second panorama image signal DP, the camera position information signal PM, and the video encoding signal DV. Send. When the first panorama image signal DF and the second panorama image signal DP are stored in the storage unit 207, the camera terminal device 200 reads out the first panorama image signal DF and the second panorama image signal DP. To the center server 300. In addition, when the first panorama image signal DF or the second panorama image signal DP is not stored in the image storage unit 207, the camera terminal device 200 controls the camera unit 201, the signal processing unit 202, and the imaging direction control unit 204. The image pickup operation is performed while moving the image pickup direction, and the first panorama image signal DF and the second panorama image signal DP are generated and transmitted to the center server 300. The camera terminal device 200 stores the generated first panorama image signal DF and second panorama image signal DP in the storage unit 207.
The camera terminal apparatus 200 supplies the camera position information signal PM generated by the imaging direction control unit 204 to the center server 300. In addition, the camera terminal device 200 starts an imaging operation of the camera unit 201, compresses the obtained video signal Dcam by the signal processing unit 202, and stores the video encoded signal DV in the memory unit 203. Further, the encoded video signal DV stored in the memory unit 203 is supplied to the center server 300. As shown in FIG. 1, when the camera terminal device 200 and the network 500 are connected via a proxy server 600, information exchange between the camera terminal device 200 and the center server 300 is performed by the proxy server 600. Is done via.
The position determination processing unit 303 of the center server 300 determines whether the imaging direction indicated by the camera position information signal PM corresponds to the position of the first panorama image or the second panorama image, and the position determination result KP. Is supplied to the image processing unit 304. Taking the second panoramic image as an example, for example, when the angle difference indicated by the camera position information signal PM is “0 °”, the imaging direction is set to the reference direction, so the position in the imaging direction is the second. This is the center position of the panoramic image.
The image processing unit 304 performs image processing based on the position determination result KP. Here, when the position determination result KP indicates that the position in the imaging direction is the center position of the second panoramic image, the image in the imaging direction is the center of the second panoramic image. The second panorama image signal DP is supplied to the display processing unit 305 as the second panorama image signal DPC without performing the second panorama image processing. On the other hand, when the position in the imaging direction is not the center position of the second panoramic image, the difference amount between the determined position in the imaging direction and the center position of the second panoramic image is determined, and the determined difference amount is Based on this, the second panorama image is processed so that the determined position becomes the center position of the image, and the second panorama image signal DPC is generated.
For this reason, in the second panoramic image, for example, when the angle difference indicated by the camera position information signal PM is “0 °”, it is displayed on the screen of the monitor 310 driven by the display drive signal HD from the display processing unit 305. Displays a captured image captured by the camera unit 201, a second panoramic image whose reference direction is the center position, and an image related to the GUI.
In the case of the first panorama image, in the case of a “first panorama image rotation mode” to be described later, as in the case of the second panorama image signal DPC, the determined position and the pan direction of the first panorama image are changed. A difference amount from the reference position is determined, and based on the determined difference amount, the first panoramic image is rotated so that the determined position is positioned below the image (on a straight line L described later), 1 panoramic image signal DFC is generated. The image processing unit 304 also determines the position in the imaging direction based on the position determination result KP (the optical axis of the imaging optical system of the camera unit 201 is oriented) in each of the first panorama image signal DFC and the second panorama image signal DPC. Display position).
(4) Processing for Generating First Panorama Image Next, processing for generating the first panorama image signal DF will be described with reference to FIGS. When generating the first panorama image signal DF, an image is acquired for each region of the first panorama image. First, as shown in FIG. 5, the zoom of the camera terminal device 200 is fixed, panning and tilting are performed at certain arbitrary intervals, and a still image is acquired. The numbers shown in FIG. 5 indicate the order of image acquisition.
Next, coordinate conversion of the still image coordinate system is performed by the following procedures [1] to [3]. Here, as shown in FIG. 6, each point of the still image is converted into a relative position (x, y, z) viewed from the rotation center of the camera terminal device 200.
[1] A camera with a horizontal angle of view thh [degree] and an aspect ratio of a is set at the origin, and is directed in the -z direction. Considering the object plane (x, y, -L) at the position of distance L, the visible range is a rectangle of 2Lx × 2Ly. Here, the relationship of Lx = L · tan (thh / 2) and Ly = Lx / a is established.
[2] The camera's posture is originally (x, y,
z, p, q, r), but assuming that the viewpoint position is fixed at the origin, x = y = z = 0, where (p, q, r) (p: Pan ( (y-axis rotation), q: Tilt (x-axis rotation), r: roll (z-axis rotation)) only.
[3] When a point (j, i) in a rectangle of (2Lx) × (2Ly) is used, the following equation is established.
x = Lx × (j-Lx) / Lx
y = Ly × (i-Ly) / Ly
z = -L
[4] The following roll conversion, tilt conversion, and pan conversion formula are applied using (x, y, z).
The obtained (x3, y3, z3) is a value represented by a coordinate system when a point (x, y, z) in the still image is viewed from the rotation center of the camera.
Next, the relative position is converted into latitude and longitude. Here, as shown in FIG. 7, (x3, y3, z3) obtained above is converted into latitude and longitude. The angle formed by the (x3, y3, z3) direction with respect to the -z direction is the longitude, and the angle formed with the xz plane is the latitude.
・ Latitude
La = tan -1
(y3 / :: sqrt (x3 * x3 + z3 * z3))
・ Longitude (Lo)
When 0.0 <z3
If 0.0 <x3 Lo = 90.0 + La
When 0.0 ≧ x3 Lo = -90.0-La
When 0.0 ≥ z3
tan -1 (x3 / z3) / RAD
RAD = π / 180 (π: Pi ratio)
Next, conversion into a first panoramic image is performed. Here, as shown in FIG. 8, the obtained latitude and longitude (La, Lo) are represented by a latitude axis and a longitude coordinate system in the circumferential direction.
As described above, the first panorama image signal DF can be generated.
(5) GUI Next, the GUI will be described. FIG. 9 is a schematic diagram showing a display screen of the monitor 310. FIG. 9 shows both a normal screen and a full screen display. In either case, an operation panel (control panel) is displayed on the left side of the screen and a live image is displayed on the right side of the screen. When “Screen Mode” on the operation panel is set to “Full”, the full screen is displayed.
FIG. 10 is a schematic diagram for explaining an operation by the operation panel. The operation panel is a panel for each function, and each panel can be folded and stored. The panel classification is as follows.
Display settingsCamera control
Settings related to camera operationPanorama
Panorama display / Preset position
Settings related to presets ・ Trigger
Trigger settings-Other
Clicking on each panel opens the panel. FIG. 10A shows a state where “View” is clicked, and a menu is displayed under the “View” column. On the other hand, FIG. 10B shows a state where “Camera control” is clicked, and a menu is displayed under the “Camera control” field. FIG. 10C shows a state where the panel “Camera control” is dragged, and the panel “Camera control” can be made a floating window by dragging.
FIG. 11 shows a state where menus such as “Screen Mode”, “View Size”, “Image Codec”, and “Frame Rate” are displayed when the panel “View” is clicked. The “Screen Mode” menu allows you to change the display mode between normal and full screen. In “View Size”, you can specify 1/4, 1/2, same size, Fit mode of the image. Fit mode displays images according to the size of the current display area. In “Image Codec” menu, Codec can be changed. In the “Frame Rate” menu, the JPEG frame rate can be changed. The movie save button is used for saving a movie, and the still image save button is used for saving a still image. In addition, a microphone volume slider, a microphone mute on / off button, an audio volume slider, an audio mute on / off button, and the like are provided.
FIG. 12 shows a menu displayed when the panel “Camera” is clicked. In the “Operation Mode” menu, the operation mode on the image can be switched. With the pan / tilt control buttons, the pan / tilt direction can be specified by operating the buttons in the arrow direction. The button located in the middle of the pan and tilt control buttons is a home position button. The zoom button is a button for designating the zoom of the camera terminal device 200, and is configured so that the boundary between the optical zoom and the digital zoom can be seen. If you continue to hold down the “W” and “T” zoom buttons, the zoom operation will continue. The focus button is used to set the focus to “far” or “near”. In addition, various buttons such as a ONE PUSH AUTO FOCUS button and an exclusive control right acquisition button are provided.
FIG. 13 shows menus displayed when “Preset position”, “Trigger”, “Other”, and “information” are clicked. “Preset
“position” is a thumbnail display of images in a plurality of predetermined shooting directions. By clicking and selecting one of the thumbnail images, the camera terminal device 200 can be directed in the direction of the selected thumbnail image. it can.
(6) Specification of shooting direction on first panorama image Next, display of the first panorama image will be described with reference to FIGS. FIG. 14 is a schematic diagram showing a state where the first panorama image is displayed by clicking “Preset position” on the operation panel on the display screen of the monitor 310. A live image is displayed on the right side of the operation panel. As described above, by dragging and moving the “Preset position” column, the sub-panel on which the first panoramic image is displayed can be arranged outside the operation panel as shown in FIG. Further, the sub-panel on which the first panoramic image is displayed can be moved within the display screen.
FIG. 16 is a schematic diagram showing in detail the sub-panel on which the first panoramic image is displayed. In the first panoramic image, the direction of the image displayed in the live video on the right side (the direction of the optical axis of the imaging optical system of the camera terminal device 200) is indicated by +. Also, as shown in FIG. 16, the first panorama image can be switched to the second panorama image by clicking the panorama switching button at the upper right of the first panorama image.
FIG. 17 shows an operation of changing the direction of the live video using the first panoramic image. The pan / tilt direction of the camera terminal device 200 can be changed using the pan / tilt control buttons described above, but can also be performed by designating an arbitrary point in the first panoramic image. In the state of FIG. 17, the optical axis of the imaging optical system of the camera terminal device 200 is directed in the direction of the + mark, and a live image in that direction is displayed. In this state, when the portion marked with ● shown in the first panoramic image of FIG. 17 is clicked, an operation signal US corresponding to the user operation is sent to the control unit 310, and based on this, the imaging direction control of the camera terminal device 200 is performed. By controlling the unit 204, the optical axis of the imaging optical system of the camera terminal device 200 is directed in the direction of the mark ●. Accordingly, the live image displayed on the right side of the operation panel is switched to the image in the direction of the mark ●.
Thereby, the user can designate the pan / tilt direction of the camera terminal device 200 by designating an arbitrary point in the first panoramic image. Since the first panoramic image includes all videos in the pan / tilt direction that the camera terminal device 200 can shoot, the user designates the direction of the camera terminal device 200 on the first panoramic image. Thus, a live image in a desired direction can be displayed. In addition, instead of the + mark indicating the clicked position, the direction of the optical axis may be controlled by specifying an area with a frame surrounding a predetermined range.
When the direction of the camera terminal device 200 is designated on the first panorama image, there are a mode in which only the live video is switched without changing the display state of the first panorama image, and a mode in which the first panorama image is rotated. is there. In the mode in which the first panoramic image is rotated, as shown in FIG. 18, the position (+ mark) where the live video is currently displayed is positioned on the straight line L upward from the center of the first panoramic image. Is set to the angular position of the first panoramic image. Next, when you click the ● mark on the first panorama image, the live image displayed on the right side of the operation panel switches to the image in the direction of the ● mark. At the same time, the position of the ● mark in the first panorama image is a straight line. The first panoramic image is rotated so as to be positioned on L. Therefore, the user can immediately recognize that the direction of the live video is on the straight line L of the first panoramic image by viewing the first panoramic image.
Similarly, in the second panorama image, the user can designate the pan direction of the camera terminal device 200 by designating an arbitrary point in the second panorama image, and based on this, the live video can be designated. The direction can be switched.
FIG. 19 shows an example in which a panoramic image is displayed on a cylindrical curved surface as a reference image instead of the panoramic image shown in FIG. In this way, a rectangular panoramic image displayed in the range of + 180 ° to −180 ° may be displayed on the cylindrical surface. Thereby, the visibility by a user can be improved and the image | video of a desired direction can be displayed. Also in the case of FIG. 19, the user can designate the pan direction of the camera terminal device 200 by designating the “display designated point” on the curved surface, and the direction of the live video can be switched based on this. it can. Note that the above-described processing including designation of the imaging direction is performed by the display processing unit 305 based on the GUI display control signal CTg.
(7) Effect of Displaying First Panorama Image Next, the effect of displaying the first panorama image will be described with reference to FIGS. FIG. 20 shows a state in which “safe” and “table” are arranged on the floor, and the camera terminal device 200 is installed directly above the intermediate position (point O) between “safe” and “table”. It is assumed that a suspicious person approaching the “safe” can be monitored by the live video of the camera terminal device 200.
FIG. 21 shows a state in which the first panoramic image is displayed on the monitor 310 together with the live video (FIG. 21A) and a state in which the second panoramic image is displayed on the monitor 310 together with the live video in the state of FIG. FIG. 21 (B)) is schematically shown. As described above, when an arbitrary position on the first panorama image or the second panorama image is clicked, the direction of the optical axis of the camera terminal device 200 can be changed to the clicked position, and a live video can be captured. The direction can be changed.
As shown in FIG. 21B, in the case of the second panoramic image, since the image is distorted immediately below the camera terminal device 200, that is, between the “safe” and the “table”, the clicked position. Even if the direction of the optical axis is changed, the direction of the optical axis does not become the actual intermediate position (point O shown in FIG. 20) between the “safe” and the “table”. In other words, if you want to see the area around the center of the “safe” and “table”, if you operate on the second panorama image (rectangular panorama image), the image will be out of the location that the user originally wanted to see, and intuitively. Control is not possible. For this reason, as shown in FIG. 21B, a situation occurs in which the videos of “safe” and “table” protrude from the lower side of the live video.
On the other hand, as shown in FIG. 21A, since the image is displayed in an overhead view, it is possible to intuitively click an intermediate position (point O shown in FIG. 20) between “safe” and “table”. it can. Further, in the case of the first panoramic image, since the distortion of the image near the point O is relatively small, the direction of the optical axis is accurately directed to the position of the point O. Therefore, even in the live video, the “safe” and “table” videos will not protrude from the display screen.
Further, FIG. 22 shows a state in which the suspicious person is moving in the vicinity of the “safe” in the live video and the first panorama image and the second panorama image in the case of FIGS. 20 and 21. . When a suspicious person is moving in the vicinity of the “safe” in the live video, in order to track the suspicious person, it is necessary to change the direction of the optical axis of the camera terminal device 200 according to the movement of the suspicious person. is there.
In such a case, in the case of the second panoramic image, the image is distorted at an intermediate position between the “safe” and the “table”, so that the click position is intentionally operated so as to change the moving direction immediately below the image. Need to do.
On the other hand, in the case of the first panorama image, there is little distortion of the image between the “safe” and the “table”, so you can intuitively click according to the movement of the suspicious person while watching the live video. By going, the direction of the optical axis can be moved in accordance with the movement of the suspicious person.
Note that when the user wants to see the vicinity of the periphery rather than directly under the camera terminal device 200, the second panoramic image has less distortion and can be controlled more intuitively than the first panoramic image. In this embodiment, since it is possible to switch between the first panorama image and the second panorama image according to the situation, it is possible to improve the ease of use of the user according to the scene. Further, if there is a space in the display screen, both the first panorama image and the second panorama image are displayed, so that the switching operation becomes unnecessary and the usability can be improved.
In the above embodiment, the rectangular panoramic image is controlled so that the imaging direction becomes the center position. However, the present invention is not limited to this, and even if the imaging direction is changed, the panoramic image is not rotated and fixed. Good.
DESCRIPTION OF SYMBOLS 100 Imaging system 200 Camera terminal device 300 Center server 301 Network interface part 305 Display processing part 310 Control part 500 Network 700 Imaging device 701 Storage part 704 Image processing part 705 Display processing part 710 Control part
Control is performed to display a panoramic image display area for displaying a panoramic image captured by the camera, and an enlarged image display area for displaying an enlarged image by enlarging an area corresponding to a position specified by the panoramic image. A display control unit;
A camera control unit that controls the camera in a pan direction or a tilt direction so that an area corresponding to a position designated on the panoramic image is imaged by the camera;
The display control unit displays an image captured by the camera under the control of the camera control unit in the enlarged image display area,
The panoramic image, Ri circular panoramic image der,
The display control unit is a control device used in a surveillance camera system that switches a panoramic image displayed in the panoramic image display area from a circular panoramic image to a rectangular panoramic image in response to a panoramic image switching operation .
The control device used in the surveillance camera system according to claim 1, wherein the circular panoramic image is generated based on an image captured a plurality of times by the camera .
The control device used in the surveillance camera system according to claim 1, wherein the display control unit is capable of moving the panoramic image display area on a screen.
A camera system having a camera that captures an image and a control device that controls the camera,
Control to display a panoramic image display area for displaying a panoramic image captured by the camera and an enlarged image display area for displaying an enlarged image by enlarging an area corresponding to a position specified by the panoramic image. A display control unit,
The panoramic image is a circular panoramic image,
The display control unit switches a panoramic image to be displayed in the panoramic image display area from a circular panoramic image to a rectangular panoramic image in response to a panoramic image switching operation.
The camera system according to claim 4, wherein the circular panoramic image is generated based on an image captured a plurality of times by the camera.
The camera system according to claim 4, wherein the display control unit is capable of moving the panoramic image display area on a screen.
A panoramic image display area for displaying a circular panoramic image captured by a camera, and an enlarged image for displaying an enlarged image by enlarging an area corresponding to a position specified by the panoramic image among images captured by the camera And a display area, and a panorama image displayed in the panorama image display area is switched from a circular panorama image to a rectangular panorama image according to a panorama image switching operation. ,
A second means for controlling the camera in the pan direction or the tilt direction so that an area corresponding to a position designated on the panoramic image is imaged by the camera;
The program according to claim 7, wherein the circular panoramic image is generated based on an image captured a plurality of times by the camera.
The program according to claim 7, wherein the first means is capable of moving the panoramic image display area on a screen.
JP2009138594A 2009-06-09 2009-06-09 Control device, camera system and program used in surveillance camera system Active JP5338498B2 (en)
JP2009138594A JP5338498B2 (en) 2009-06-09 2009-06-09 Control device, camera system and program used in surveillance camera system
RU2011149201/07A RU2528566C2 (en) 2009-06-09 2010-06-01 Control device, camera system and programme
BRPI1012976A BRPI1012976A2 (en) 2009-06-09 2010-06-01 control device, camera system and program
PCT/JP2010/059248 WO2010143558A1 (en) 2009-06-09 2010-06-01 Control device, camera system, and program
KR1020167026684A KR101782282B1 (en) 2009-06-09 2010-06-01 Control device, camera system, and control method of performing camera control
CN201080024248.1A CN102461153B (en) 2009-06-09 2010-06-01 The control device and a camera system
EP10786089.2A EP2442548B1 (en) 2009-06-09 2010-06-01 Control device, camera system, and program
KR1020117028827A KR101662074B1 (en) 2009-06-09 2010-06-01 Control device, camera system, and recording medium
US13/320,550 US20120062695A1 (en) 2009-06-09 2010-06-01 Control device, camera system, and program
EP18166839.3A EP3367662A1 (en) 2009-06-09 2010-06-01 Control device, camera system, and program
TW99117802A TWI422218B (en) 2009-06-09 2010-06-02 Control devices, camera systems and programs for monitoring camera systems
JP2010287965A JP2010287965A (en) 2010-12-24
JP5338498B2 true JP5338498B2 (en) 2013-11-13
ID=43308813
JP2009138594A Active JP5338498B2 (en) 2009-06-09 2009-06-09 Control device, camera system and program used in surveillance camera system
US (1) US20120062695A1 (en)
EP (2) EP2442548B1 (en)
JP (1) JP5338498B2 (en)
KR (2) KR101662074B1 (en)
CN (1) CN102461153B (en)
BR (1) BRPI1012976A2 (en)
RU (1) RU2528566C2 (en)
TW (1) TWI422218B (en)
WO (1) WO2010143558A1 (en)
JP5820210B2 (en) * 2011-09-15 2015-11-24 キヤノン株式会社 Imaging device and imaging device control method
CN103731630B (en) 2012-10-16 2019-01-11 华为技术有限公司 A kind of video monitoring method, equipment and system
NO334902B1 (en) * 2012-12-07 2014-07-07 Kongsberg Defence & Aerospace As A system and method for monitoring at least one observation area
JP2016033620A (en) * 2014-07-31 2016-03-10 キヤノン株式会社 Image acquisition device
JP6543108B2 (en) * 2015-06-29 2019-07-10 キヤノン株式会社 Information processing apparatus, control method thereof, and program
JP2019009520A (en) * 2017-06-21 2019-01-17 キヤノン株式会社 Information processing apparatus, information processing method, and program
CN107682621A (en) * 2017-09-06 2018-02-09 深圳天珑无线科技有限公司 Rotate photographic method, device, mobile terminal and computer-readable recording medium
JP2001148850A (en) * 1999-11-18 2001-05-29 Canon Inc Video recessing unit, video processing method, video distribution system and storage medium
DE102006032239A1 (en) * 2006-07-12 2008-01-17 Patrick Fischer Cable car with changeable carrier rope layer
AT521054T (en) * 2006-12-20 2011-09-15 Axis Ab Method and device for detecting sabotage on a monitoring camera
JP2008301191A (en) * 2007-05-31 2008-12-11 Victor Co Of Japan Ltd Video monitoring system, video monitoring control device, video monitoring control method, and video monitor controlling program
RU2008139232A (en) * 2008-10-03 2010-04-10 Закрытое акционерное общество "ГОЛЛАРД" (RU) System and method of video monitoring
2009-06-09 JP JP2009138594A patent/JP5338498B2/en active Active
2010-06-01 US US13/320,550 patent/US20120062695A1/en active Pending
2010-06-01 KR KR1020117028827A patent/KR101662074B1/en active IP Right Grant
2010-06-01 BR BRPI1012976A patent/BRPI1012976A2/en active Search and Examination
2010-06-01 EP EP10786089.2A patent/EP2442548B1/en active Active
2010-06-01 WO PCT/JP2010/059248 patent/WO2010143558A1/en active Application Filing
2010-06-01 CN CN201080024248.1A patent/CN102461153B/en active IP Right Grant
2010-06-01 KR KR1020167026684A patent/KR101782282B1/en active IP Right Grant
2010-06-01 EP EP18166839.3A patent/EP3367662A1/en active Pending
2010-06-01 RU RU2011149201/07A patent/RU2528566C2/en active
2010-06-02 TW TW99117802A patent/TWI422218B/en active
TWI422218B (en) 2014-01-01
US20120062695A1 (en) 2012-03-15
KR101782282B1 (en) 2017-09-26
EP2442548B1 (en) 2018-05-02
RU2528566C2 (en) 2014-09-20
BRPI1012976A2 (en) 2018-01-16
JP2010287965A (en) 2010-12-24
EP3367662A1 (en) 2018-08-29
EP2442548A4 (en) 2012-11-28
EP2442548A1 (en) 2012-04-18
WO2010143558A1 (en) 2010-12-16
CN102461153B (en) 2015-02-25
KR101662074B1 (en) 2016-10-05
KR20120030403A (en) 2012-03-28
CN102461153A (en) 2012-05-16
KR20160116044A (en) 2016-10-06
RU2011149201A (en) 2013-06-10
TW201123856A (en) 2011-07-01
JP3382276B2 (en) 2003-03-04 Electronic device and control method thereof
JP2012090157A (en) 2012-05-10 Display control device and display control method
CN101198030B (en) 2010-06-23 Camera locating method and locating device of video monitoring system
JP3934521B2 (en) 2007-06-20 Video remote control device, video remote control method, video remote control program, and recording medium recording video remote control program
2013-05-07 A911 Transfer of reconsideration by examiner before appeal (zenchi)