Remote control camera system and image transmission method

A live camera whose image pickup direction is remote-controlled is utilized, and a plurality of frames picked up in different pickup directions are composed so as to generate a composite image, and this is stored in a memory. Upon request from the user for altering the pickup direction, in the composite image within the memory, an extraction area is shifted in the horizontal direction and vertical direction in response to the request for alteration in a manner so as to correspond to a pickup image in the live camera, and the image within the extracted area is transmitted to the user. This arrangement eliminates the necessity of altering the pickup direction of the live camera mechanically, and independent of the mechanical alteration, only the electrical image processing of the composite image is carried out, with the result that the user is allowed to feel as if he or she were actually operating the live camera. In other words, even if a plurality of users request to alter the pickup direction, one live camera can deal with these requests; therefore, it is possible to effectively utilize the live camera, and consequently to construct a remote control camera system at low costs.

This application is based on application No. 2000-041026 filed in Japan, the contents of which are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a remote control camera system which displays images picked up by a camera through the Internet, and also concerns an image transmission method for such a system.

2. Description of the Background Art

There have been many cases in which various companies place Internet live cameras in their own buildings, showrooms, sightseeing spots, etc. and provide images on their home page so as to introduce their companies and enhance advertising campaigns.

Moreover, there have been many monitoring systems in which monitoring cameras, etc. are connected to the Internet and these images are monitored from a remote place.

In these cases, with respect to functions of the remote control cameras such as the Internet live cameras and monitor cameras, it is necessary to provide a remote controlling operation for directing the camera in a direction desired by the user; however, one camera is not capable of dealing with a plurality of accesses.

In particular, with respect to sites which transmit video images of the Internet live cameras that can be remote-controlled by a predetermined user, those sites having many accesses have a plurality of live cameras; however, one live camera only deals with one access, with the result that the live cameras are not effectively utilized.

Moreover, the installation of a number of Internet live cameras results in installations of computers (servers) and necessitates the constant connections of the respective cameras to the Internet, causing high installation costs.

SUMMARY OF THE INVENTION

The present invention relates to a remote control camera system.

In accordance with the present invention, this system is provided with: (a) an image pickup device; (b) a changing element for changing a pickup direction of the image pickup device; (c) a composing element for forming composite image information by composing a plurality of pieces of image information picked up by the image pickup device in different image pickup directions; (d) a memory for storing the composite image information; (e) an extraction element for forming extracted image information derived from the composite image information in response to a request from outside; and (f) an output device for outputting the extracted image information. Therefore, it is possible to effectively utilize the remote control cameras and also to construct a remote control camera system at low costs.

In one preferred embodiment of the present invention, the composite image information is information formed by continuously composing the plurality of pieces of image information. Consequently, the extraction of image from the composite image information is easily made.

Moreover, the present invention also relates to an image transmission method.

Therefore, the objective of the present invention is to effectively utilize the remove control cameras, and also to provide a remote control camera technique at low costs.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

<Essential Structure of a Remote Control Camera System>

FIG. 1is a perspective view that shows an essential structure of a remote control camera system1in accordance with the first preferred embodiment of the present invention.

A remote control camera system1is provided with a live camera2, a direction-altering section3for altering the image pickup direction of the live camera2, a server4for controlling these, and a cable5for electrically connecting the live camera2and the direction-altering section3to the server4.

The live camera2has a casing20having a box shape, and a disc-shaped transparent pickup window21.

The direction-altering section3is provided with a rotation driving mechanisms31,32for altering the pickup direction of the live camera2and a holding member30that are connected to these and supports the live camera2.

The rotation driving mechanism31carries out rotation of the live camera2in a direction for angles of elevation around a shaft31c.

The rotation driving mechanism32also carries out rotation of the live camera2in the horizontal direction around a shaft32c.

The holding member30is provided with a member30ahaving its two ends bent vertically and a member30bthat is connected to the center of the member30a. Here, the rotation driving mechanism31is attached to the two ends of the member30a, and the rotation driving mechanism32is attached to the end of the member30b. With this arrangement, with respect to the alternation of the pickup direction, the live camera2is allowed to have two degrees of freedom.

The server4has a casing40having an approximate box shape, and a lamp41for indicating the operational state, etc. is attached to the front face of the casing40. Moreover, a communication line9for transferring data to the user, etc. is connected to the server4.

FIG. 2is a block diagram that shows a functional construction of the remote control camera system1.

The live camera2has a construction using CCD23that are solid state imaging devices. The CCD23converts a subject image that is formed by a lens section22placed in the recessed place of the image pickup window21into an analog signal, and outputs this to a signal processing section24. The lens section22is provided with a zoom lens22aand a focus lens22b. After eliminating noise, etc. from the analog signal output from the CCD23, the signal processing section24subjects this to an A/D conversion, and then outputs the resulting digital image signal to the CPU25.

A CPU25carries out various processes such as a white balancing process on the digital image signal while utilizing a memory26. Moreover, the CPU25is electrically connected to a lens driving section27having a motor, and the zoom lens22aand the focus lens22bare allowed to shift respectively by instructions from the CPU25to the lens driving section27so that an altering process of the zoom ratio and a focusing process are carried out on the subject. Moreover, an interface28is electrically connected to the CPU25so that the digital image signal is transferred to the server4through this interface28.

The rotation driving mechanism31is provided with a motor31aserving as an actuator for rotating the live camera2around the rotation shaft31cand a sensor31bfor detecting the rotation angle. With this construction, the motor31ais allowed to have the fed-back angular information from the sensor31bso that the rotation angle is set to a target value.

Moreover, the rotation driving mechanism32is also provided with a motor32aserving as an actuator for rotating the live camera2around the rotation shaft32cand a sensor32bfor detecting the rotation angle. With this construction, it is possible to set the rotation angle to a target value in the same manner as the rotation driving mechanism31.

The server4is provided with a CPU41for carrying out a composing process (which will be described later), etc. on pickup images and an image memory42for storing the resulting composite image, etc. Interfaces43,44and45are electrically connected to the CPU41so that signal transmission and receipt are carried out among the CPU41, the live camera2and the direction-altering section3through the interfaces43,44and45. Moreover, the server4is provided with a communication-use interface46for receiving an access from the user and for transmitting the digital information, and a display section47such as a lamp48.

The cable5serves as a signal line for transferring signals to the live camera2and the direction-altering section3, and a power supply line for supplying driving power.

<Operation of the Remote Control Camera System1>

FIG. 3is a flow chart that explains the outline of the basic operation of the remote control camera system1. Here, with respect to the operation of the remote control camera system1, the CPU41of the server4automatically executes it, and detailed explanations of the following steps S1, S2and S3will be given later.

First, as step S1, a composite image is formed by composing a plurality of images picked up by the live camera2.

At step S2, a composite image within the image memory42in the server4is updated by the composite image that has been formed by composing the images picked up by the live camera2.

At step S3, a judgment is made as to whether or not an access is made to the server4from the user, that is, as to whether or not there is any input to the remote control camera system, through the communication line9. Here, if there is any access, the sequence proceeds to step S4.

At step S4, in response to a request from the user, an image extracted from the composite image is transmitted, that is output to the communication line9.

<Operation of Composite Image Generation>

FIG. 4is a flow chart that explains the operation of a composite image generation. Moreover,FIG. 5is a conceptual drawing that explains the composite image generation.

At step S11, one frame6as shown inFIG. 5is picked up by the live camera2.

At step S12, the frame6thus picked up is stored in the image memory42of the server4.

At step S13, with respect to an image-pickup area60schematically shown by a cylindrical projection area inFIG. 5, a judgment is made as to whether or not an image-pickup operation has been completed by the live camera2. The image-pickup area60is preliminarily set by taking into consideration the movable area of the direction-altering section3. In the case when the area image-pickup operation has been completed, the sequence proceeds to step S15, and if it has not been completed, then the sequence proceeds to the step S14.

At step S14, the direction-altering section3is driven so that the pickup direction of the live camera2is altered.

At step S15, a plurality of images that have been picked up in different directions are formed into a composite image that corresponds to the continuous pickup area60shown inFIG. 5(which will be described later).

At step S16, the composite image that has been composed in step S15is stored in the image memory42of the server4.

<Detailed Description of an Image Composing>

An explanation will be given of an example of a method by which, at step S15, based upon a plurality of frames6picked up by the live camera2, a composite image corresponding to the image-pickup area60is generated.

FIGS. 6A and 6Bare drawings that explain a composing process of a frame6.

The frame6forms an image corresponding to the width Wx with a pickup angle of 33° in the horizontal direction of the live camera2and the width Wy with a pickup angle 22° in the vertical direction of the live camera2. Moreover, each of right and left edges Rx with a pickup angle of 3° of the frame6and each of upper and lower edges Ry with a pickup angle of 4° of the frame6are allowed to serve as overlapping portions that are used when the respective frames6are joined to each other.

As illustrated inFIG. 6A, each frame6is provided with overlapped areas61(parallel portions indicated by slanting lines) on its four corners at which the four frames6are joined to one after another. The frames6are superposed so as to make the center pixels (A0,0, A0,1, A1,0, A1,1, etc.) in these areas61coincident with each other; thus, positioning processes are virtually carried out so as to compose the respective frames6(seeFIG. 6B).

Then, after the respective frames6have been virtually positioned, in order to improve the precision in the corresponding areas61of the respective frames6are pattern-matched so as to carry out the positioning correction for the respective frames6.

For example, it is supposed that, as a result of the pattern matching process of upper areas61aand61bin the adjacent frames (see frames F0,0, F1,0ofFIG. 6A) in the horizontal direction, the positions of the center pixel62aand the center pixel62bhave an offset corresponding to vector Va as shown inFIG. 7A, and that, as a result of the pattern matching process of lower areas61cand61din the adjacent frames6in the same manner as described above, the positions of the center pixel62cand the center pixel62dhave an offset corresponding to vector Vb. In this case, as illustrated inFIG. 7C, vector Vc, which is the average between vector Va and vector Vb, is found. Then, vector Vc is used so as to correct positions in the joining process between the adjacent frames6so that it becomes possible to create a composite image with high precision in the joining process that is free from a feeling of incongruousness.

FIG. 8Ais a drawing that explains a continuous composite image formed by composing the frames6using the above-mentioned method.

A composite image7is equivalent to an image obtained by developing the image-pickup area60shown inFIG. 5onto a plane, and as illustrated inFIG. 8A,65frames (F0,0to F11,4) have their edges continuously joined to one after another. The composite image7has virtually60jointed frames6having picked up images. In other words, the same frames as the five frames6of the left edge70are joined to the right edge71in a duplicated manner. Thus, it is possible to simplify the image extracting processes in the vicinity of the two edges of the composite image7.

With respect to the composite image7, for example, supposing that one frame has 640×400 pixels, it has an image size of 7680×2000 pixels. Therefore, in the case of a request for altering the pickup direction from the user (which will be described later), for example, the shift corresponding to 1° in the pickup direction toward the horizontal direction Mx is equivalent to a shift of pixels corresponding to 21.33 (=7680/360) pixels/deg.

<Operation for Updating the Composite Image>

FIG. 9is a flow chart that explains an operation for updating the composite image. This operation is similar to the operation for generating a composite image as shown in the flow chart ofFIG. 4; therefore, the following description will discuss only the portions different from the operation for generating a composite image.

At step ST25, the composite image formed at step ST24is overwritten on the image memory42of the server4and stored therein; that is, the storage is carried out in an updating manner. The generation of such a new composite image and the updating process are repeatedly carried out with predetermined time intervals so that the composite image is updated to the latest one regularly. This arrangement is effective in the case when a moving object is used as the subject.

FIG. 10is a flow chart that explains the operation for transmitting images.

At step S41, the initial setting for an extraction area extracted from the composite image7within the image memory42of the server4is carried out. In other words, a area72is extracted from the composite image7shown inFIG. 8B.

At step S42, the image of the extraction area72extracted at step S41is transmitted to the user through the communication interface46.

At step S43, a judgment is made as to whether or not there is a request from the user for altering the pickup direction, that is, whether or not there is a request for the shift of the area72shown inFIG. 8B. Here, if there is a request for the alteration, then the sequence proceeds to step S44, and if there is not, then the sequence proceeds to step S46.

At step S44, in response to the request for the alteration for the pickup direction, the extraction area72is shifted in the horizontal direction Mx and in the vertical direction My, as illustrated inFIG. 8B.

At step S45, the image within the extraction area72corresponding to the request for the alteration for the pickup direction is transmitted to the user through the communication interface46.

At step S46, a judgment is made as to whether or not the access from the user to the server4through the communication line9has been completed. Here, upon completion of the access, the sequence proceeds to the step S3, and if it has not been completed, the sequence proceeds to step S43.

With the above-mentioned operation, even upon request from the user to alter the pickup direction, it is not necessary to mechanically alter the pickup direction of the live camera2, and independent of the mechanical alteration, only the electrical image processing of the composite image7is carried out, with the result that the user is allowed to feel as if he or she were actually operating the live camera. In other words, even if a plurality of user's requests to alter the pickup direction, one live camera can deal with these requests; therefore, it is possible to effectively utilize the live camera, and consequently to construct a remote control camera system at low costs.

The essential construction of the remote control camera system in accordance with the second preferred embodiment of the present invention is the same as that of the remote control camera system1of the first preferred embodiment, except for portions related to the image memory42.

FIG. 11is a drawing that explains the concept of an image processing operation in a remote control camera system1A in accordance with the second preferred embodiment.

In an image memory42A of a server4in a remote control camera system1A, a composite image80picked up at 1× zoom magnification and composed and a composite image81picked up at 6× zoom magnification and composed are stored. Here, the live camera2of the second preferred embodiment is capable of picking up images at one-power to six-power zoom.

<Operation of the Remote Control Camera System1A>

The basic operation of the remote camera system1A is the same as the basic operation of the remote control camera system1shown in the flow chart ofFIG. 3.

<Operation for Forming Composite Images>

FIG. 12is a flow chart that explains the operation of a composite image generation in the remote control camera system1A.

At step ST11, n, which represents the ratio of zoom, is substituted by the initial value 1.

At step ST12, the frame6is picked up with the live camera2being set at zoom magnification of n times.

At step ST13, the frame6thus picked up is stored in the image memory42of the server4.

At step ST14, with respect to the cylindrical pickup area60shown inFIG. 5, a judgment is made as to whether or not the image pickup has been completed by the live camera. If the image pickup has been completed, the sequence proceeds to step ST16, and if it has not been completed, the sequence proceeds to step ST15.

At step ST15, the direction-altering section3drives the live camera2so as to alter its pickup direction.

At step ST16, a plurality of images picked up at zoom magnification of n times in different directions are subjected to the same composing process as the remote control camera system1of the first preferred embodiment, with the result that a composite image80(81) of zoom magnification of n times is formed as shown inFIG. 11.

At step ST17, the composite image80(81) of zoom magnification of n times composed at the step ST16is stored in the image memory42of the server4.

At step ST18, a judgment is made as to whether or not n is 1. If n is equal to 1, the sequence proceeds to step ST19, and if n is not 1, that is, if n is equal to 6, the sequence proceeds to step ST20.

At step ST19, n is substituted by 6. Thus, after the composite image of 1× zoom magnification has been formed and stored, the operation for generating a composite image of 6× zoom magnification is initiated.

At step ST20, n is substituted by 1. Here, the value 1 is set as the initial value of n for an operation for updating the composite image, which forms a post-process.

<Operation for Updating the Composite Image>

FIG. 13is a flow chart that explains an operation for updating the composite image. This operation is similar to the operation for generating a composite image as shown in the flow chart ofFIG. 12; therefore, the following description will discuss only the portions different from the operation for generating a composite image.

At step ST25, the composite image of zoom magnification of n times formed at step ST24is overwritten on the image memory42of the server4and stored therein; that is, the storage is carried out in an updating manner. So that the composite image of zoom magnification of n times is updated to the latest one regularly. This arrangement is effective in the case when a moving object is used as the subject.

Moreover, at steps ST26to28, a judgment is made as to whether or not n=1, and if n=1, n is substituted by 6, while if n is not equal to 1, n is substituted by 1; thus, composite images at different zoom ratios, that is, 1× zoom magnification and 6× zoom magnification, are alternately generated.

FIG. 14is a flow chart that explains the operation for transmitting images.

At step ST41, the initial setting for an extraction area of the composite image80of 1× magnification within the image memory42of the server4is carried out. In other words, a area80ais extracted from the composite image80shown inFIG. 11.

At step ST42, the image of the extraction area80aextracted at step ST41is transmitted to the user through the communication interface46.

At step ST43, a judgment is made as to whether or not there is a request from the user for altering the pickup direction, that is, whether or not there is a request for the shift of the area80a. Here, if there is a request for the alteration, then the sequence proceeds to step ST44, and if there is not, then the sequence proceeds to step ST46.

At step ST44, in response to the request for the alteration for the pickup direction, the extraction area80ais shifted in the same manner as in the first preferred embodiment.

At step ST45, the image within the extraction area80acorresponding to the request for the alteration for the pickup direction is transmitted to the user through the communication interface46.

At step ST46, a judgment is made as to whether or not there is a request for the alteration for the zoom ratio from the user. Here, upon receipt of the request, the sequence proceeds to the step S44, and if there is no request, the sequence proceeds to step S46.

At step ST47, in response to the request for the alteration of the zoom ratio, the image within the area81acut out from the composite image81of 6× zoom magnification is subjected to the image processing so as to generate an extracted image82(83). In other words, as illustrated inFIG. 11, in the case of 1× to 6× magnification, the image within the cut-out area81ain the composite image81of 6× zoom magnification is reduced through an image processing to generate a desired extraction image82. Moreover, in the case of not less than 6× zoom magnification, the image within the cut-out area81ain the composite image81of 6× zoom magnification is enlarged through an image processing to generate a desired extracted image83. In this case, the composite image of 6× zoom magnification having the maximum magnification is selected and used for the image processing; therefore, since the image is reduced in the case of an intermediate magnification (1× to 6×), it is possible to avoid degradation in the image quality.

At step ST48, the extracted image82(83) formed at step S47is transmitted to the user through the communication interface46.

At step ST49, a judgment is made as to whether or not the access to the server4, made by the user through the communication line9, has been completed. Here, if the access has been completed, the sequence proceeds to step S3, and if it has not been completed, the sequence proceeds to step ST43.

With the above-mentioned operation, the same effects as those of the first preferred embodiment are also obtained in the remote control camera system1A. Moreover, since the zoom ratio can be altered, it is possible to deal with a high-level request from the user.

⊚ With respect to the composite image7in the first preferred embodiment, the frames which have been picked up at the maximum magnification may be joined to each other so as to generate the image. In this case, if there is a request for alteration of the zoom magnification from the user, the image processing may be applied to the composite image having the maximum magnification so as to generate an extracted image.

⊚ With respect to the zoom ratio in the second preferred embodiment, not limited to the two kinds of combinations of 1× and 6× magnification, for example, three kinds of combinations of, for example, 1×, 3× and 6×, or combinations of 4 kinds or more, may be used.

⊚ With respect to the composite images of the second preferred embodiment, it is not necessary to use frames that have been picked up by the live camera by actually altering the magnification, and based upon a composite image of the frame picked up at the maximum magnification, the composite image may be reduced to a plurality of images having different magnifications such as ½ and ¼ with respect to the maximum magnification.

⊚ With respect to the pickup area related to the above-mentioned preferred embodiments, it is not necessary to use the cylindrical shape as shown inFIG. 5, and a dome-shaped pickup area60A as shown inFIG. 15Amay be used.

In this case, with respect to the resulting composite image7A corresponding to the composite image7shown inFIG. 8A, frames6are composed in a manner as illustrated inFIG. 15B.

⊚ With respect to the composite images in the above-mentioned preferred embodiments, among all the area to be picked up, those frames that have no changes in the motion (for example, the ceiling portion, etc. of a room) may be omitted after they have once been picked up, and the frames other than those frames may be repeatedly picked up so as to update the composite image.

⊚ With respect to the composite images in the above-mentioned preferred embodiments, it is not necessary to carry out the composing process thereof in the server, and the composing process may be executed in the live camera, and the data may be transferred to the server.

⊚ With respect to the number of the live cameras in the above-mentioned preferred embodiments, it is not necessary to limit the number to one; and two or more cameras may be installed. The application of a plurality of live cameras makes it possible to share the load in collecting picked-up images for generating a composite image, and consequently to readily form the composite image.

⊚ With respect to the live cameras in the above-mentioned preferred embodiments, as illustrated inFIG. 16, in addition to the camera2A for composing the image, another camera2B for transmitting live (actual) image just picked up directly to the user may be installed separately. This camera2B makes it possible to transmit an image without a time lag, and consequently to easily follow a moving subject, etc. by the camera.

In this case, the live image and the composite image stored in the image memory may be combined and transmitted. More specifically, as illustrated inFIG. 17A, a composite image85and a live image86provided in a manner so as to enlarge a partial image85aof the composite image85may be combined, or as illustrated inFIG. 17B, a composite image87and a live image88provided in a manner so as to enlarge a partial image87aof the composite image85may be combined.

Moreover, in the case when there is not any request for altering the pickup direction for a predetermined period of time, it is judged that the user wishes to view in the constant direction, and after having directed the camera in the direction corresponding to the image last extracted and transmitted, live images in the corresponding direction picked up by the camera may be transmitted to the user. In this case, the camera used for forming the composite image may be sharedly used for this purpose, or as illustrated inFIG. 16, in the case of the application of a plurality of cameras, one camera may be assigned for the transmission of live images.FIG. 19shows a flow chart for explaining this operation.

⊚ With respect to the lens driving mechanism in the live camera in the above-mentioned preferred embodiments, it is not inevitable to use a motor; and for example, a piezoelectric element may be utilized as described below.

FIG. 18is a cross-sectional view that shows an essential structure of a camera in which piezoelectric elements are used.

A camera29is provided with a zoom lens section291, a focus lens section292, a driving shaft293connected to the zoom lens section291, and a driving shaft294connected to the focus lens section292. Moreover, the camera29is provided with piezoelectric (conversion) elements295and296connected to the respective ends of the driving shafts293and294, a CCD297and an optical low-pass filter298placed on the CCD297. These piezoelectric elements295and296have such a characteristic that upon application of a driving voltage, they expand and shrink in the driving axis direction.

Light that has been made incident on the pickup window299is transmitted through the zoom lens section291, the focus lens section292and the optical low-pass filter298, and made incident on the CCD297. At this time, the piezoelectric elements295and296are driven so that the zoom lens section291and the focus lens section292are shifted; thus, an image of a subject is properly formed on the CCD297.

With this arrangement, it becomes possible to provide a small-size, light-weight camera.