IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND VEHICLE MONITORING SYSTEM

Abstract: Images of vehicles traveling on a road are sensed by a plurality of monitor cameras arranged along the road. Video signals obtained by these monitor cameras are A/D-converted into digital images. These digital images are so corrected that the road shown in each digital image runs in the longitudinal or lateral direction of the image. The corrected digital images are composited such that the roads shown in these images continue. This allows the whole of the road as an object of traffic monitoring to be displayed on a single screen. Since the entire road can be monitored without switching images to be displayed, the speed of traffic abnormality detection increases.

Description: Technical Field The present invention relates to an image processing apparatus, image processing method, and vehicle monitoring system used for, e.g., monitoring of vehicles. Background Art Generally, CCTV (Closed-Circuit TeleVision) cameras are installed on express highways and general highways to monitor traffic. Especially in long tunnels and the like, a large number of cameras are installed to avoid monitoring dead angles. Output images from these cameras are transmitted to a highway control center, displayed on monitors, and monitored by observers. It is usually difficult to install monitors equal in number to cameras owing to, e.g., the installation space. Therefore, the number of installed monitors is made smaller than that of installed cameras, and camera images from a plurality of cameras are displayed on one monitor. That is, a plurality of camera images are sequentially switched and displayed at intervals of a few seconds. Disclosure of Invention Unfortunately, the above traffic monitoring method has the problem that the detection of a traffic abnormality is delayed. That is, since camera images are displayed on each monitor as they are switched, camera images which are not displayed exist. This may delay the detection of a traffic abnormality. Also, frequently switching camera images increases the fatigue of observers and makes them overlook traffic abnormalities. To solve these problems, the introduction of image processing technologies is being examined. However, it is technically difficult to detect traffic abnormalities with perfect accuracy. The present invention has been made to solve the above problems, and has as its object to provide an image processing apparatus, image processing method, and vehicle monitoring system capable of improving the accuracy of traffic abnormality detection. US-A-5862508 discloses a traffic monitoring system in which moving objects are detected from a movie/video. A spatial-temporal display is produced by reproducing successive slit images from the movie/video along a horizontal time axis, so that stationary features in the image appear as horizontal lines, whereas moving objects appear as images along the time axis. If the slit image is taken at an oblique angle to the traffic direction, the moving object appears in the spatial-temporal image to be inclined forward or backward providing an indication of direction of travel and velocity. EP-A-0403193 discloses a traffic monitoring system in which first and second traffic scene images are video processed to form an edge image and to determine the location of traffic in the scene. Movement is detected from a difference image and the edge and difference images are divided into an array of image cells, in each of which the presence or absence of traffic and the presence or absence of movement is determined. Image cells are.further analysed to detect traffic objects such as blocks, platoons or queues. The detected traffic objects are displayed and objects from several cameras are combined to represent the state of the whole road. According to the present invention, an image processing apparatus and method are provided as set out in Claims 1 and 3 respectively. The image processing apparatus may further comprise storage means for storing the composite image as image data. The image processing method may further comprise the storage step of storing the composite image as image data. A vehicle monitoring system is also contemplated as set out in Claim 5. The image sensing means may each have a wide angle lens in an image sensing optical system thereof, and said correcting means may be operative to correct distortion, caused by the wide angle lens, of each digital image. The display means may be a projector having a display area which is relatively long from side to side. In these arrangements described above, the whole of the road as an object of traffic monitoring can be displayed on a single screen. Accordingly, the entire road can be monitored without switching images to be displayed. Since there is no area which cannot be monitored owing to image switching, therefore, it is possible to prevent a delay of traffic abnormality detection and increase the speed of traffic abnormality detection. Also, since no image switching is performed, traffic abnormalities can be detected on the basis of changes in an image. This facilitates visual traffic abnormality detection and reduces the monitoring load. Additionally, since an image of a road to be monitored is sensed sideways by using a wide angle lens, image correction can be readily performed. Furthermore, since corrected composited image data is stored, after a traffic abnormality occurs it is possible to analyze the abnormality by checking this composite image. Brief Description of Drawings Fig. 1 is a view for explaining a vehicle monitoring system according to the first embodiment of the present invention; Fig. 2 is a flow chart showing the operation of an image processing apparatus in the vehicle monitoring system shown in Fig. 1; Fig. 3 is a view for explaining an A/D-converted digital image; Fig. 4 is a view for explaining a corrected image subjected to a correcting process; Fig. 5 is a view for explaining a composite image subjected to a compositing process; Fig. 6 is a view for explaining a vehicle monitoring system and the like according to the second embodiment; Fig. 7 is a view for explaining an A/D-converted digital image; Fig. 8 is a view for explaining a corrected image subjected to a correcting process; Fig. 9 is a view for explaining a composite image subjected to a combining process; and Fig. 10 is a view for explaining a vehicle monitoring system and the like according to the third embodiment. Embodiments of the present invention will be described below with reference to the accompanying drawings. Note that the same reference numerals denote the same parts in these drawings, and a duplicate description thereof will be omitted. Note also that the dimensional ratio of each drawing is not necessarily consistent with that of the explanation. (First Embodiment) Fig. 1 is a view for explaining a vehicle monitoring system. As shown in Fig. 1, this vehicle monitoring system monitors the state of traffic on a highway, e.g., detects traffic abnormalities in a tunnel 1. The vehicle monitoring system includes a plurality of monitor cameras 2. These monitor cameras 2 are image sensing means for sensing an image of a vehicle 11 traveling on a road in the tunnel 1, and are installed at predetermined intervals in the tunnel 1. The installation interval of the monitor cameras 2 is, e.g., about 200 m. The image sensing direction of the monitor camera 2 is a direction in which an image of the front or rear of the vehicle 11 traveling on a road 10 is sensed. Each monitor camera 2 is connected to a distributor 3 which is installed in an institution 4 built near the tunnel 1. This distributor 3 receives sensed images from the monitor cameras 2 and outputs these images to an image processing apparatus 5. The image processing apparatus 5 is an image processing means for receiving images from the monitor cameras 2, processing these images, and transmitting the processed images to a control center 6. This image processing apparatus 5 comprises, e.g., an A/D converter, video frame memory, image processor, and signal output interface. The A/D converter A/D-converts an input analog video signal into a digital signal. For example, a video A-D converter is used as this A/D converter. The video frame memory stores converted digital video signals as image data. This image data is an aggregate of pixels having tone values at individual coordinate runs on the x and y axes. The pixel tone has, e.g., 8-bit 256 gray levels. The image processor processes the image data stored in the video frame memory. The signal output interface outputs the processed image to the outside in accordance with a command from the image processor. In the control center 6, the image transmitted from the image processing apparatus is input to a monitor 61, and this monitor 61 displays the image sensed in the tunnel 1. Through this image, a supervisor can visually monitor the traveling states of vehicles in the tunnel 1. The monitor 61 is an image display means, for example, a projector is used as this monitor 61. The operation of the vehicle monitoring system, the operation of the image processing apparatus, and an image processing method will be described below. Referring to Fig. 1, the monitor cameras 2 sense images of the vehicle 11 traveling on the road 10 and input video signals to the distributor 3. The distributor 3 suitably transfers these video signals from the monitor cameras 2 to the image processing apparatus 5. The image processing apparatus 5 processes the video signals. Fig. 2 is a flow chart of this image processing performed by the image processing apparatus 5. As shown in S100 of Fig. 2, an input video signal to the image processing apparatus 5 is A/D-converted. This A/D conversion is the process of converting an analog video signal into a digital video signal. The converted digital image is stored as image data into the video frame memory. The flow advances to S200 to perform image correction. This image correction is the process of correcting the image data such that the road 10 shown in the A/D-converted digital image runs in the longitudinal or lateral direction of the image. By this image correction, a digital image shown in Fig. 3 is corrected into a corrected image shown in Fig. 4. Referring to Fig. 3, the road 10 is so displayed in a digital image 20 as to go away from the lower right portion to the upper left portion. As shown in Fig. 4, this digital image 20 is corrected such that the road 10 runs in the lateral direction of the image. This correcting process forms a corrected image 21. During the correction, the vehicle 11 traveling on the road 10 is also processed in accordance with the correction. The flow then advances to S300 to perform image composition. This image composition is the process of combining corrected images subjected to the image correction. For example, corrected images 21 pertaining to video signals obtained by the monitor cameras 2 at the same time are combined into one image. This combining process is performed such that the roads 11 shown in the individual corrected images 21 appear to continue. Also, the combining is so performed that more than one image of the same vehicle 11 will not be displayed. Fig. 5 shows a composite image 22 subjected to the combining process. After this image combining process is completed, the composite image 22 is transmitted to the control center 6. The whole of this composite image 22 is displayed on the monitor 61 in the control center 6. As described above, the image processing apparatus, image processing method, and vehicle monitoring system according to this embodiment can display, on a single screen, the whole of the road 10 as an object of traffic monitoring. Accordingly, the entire road can be monitored without switching images to be displayed. Since there is no area which cannot be monitored owing to image switching, therefore, it is possible to prevent a delay of traffic abnormality detection and increase the speed of traffic abnormality detection. Also, since no image switching is performed, traffic abnormalities can be detected on the basis of changes in an image. This facilitates visual traffic abnormality detection and reduces the monitoring load. In the image processing apparatus, image processing method, and vehicle monitoring system according to this embodiment, the composite image 22 is transmitted to the control center 6. However, the image processing apparatus, image processing method, and vehicle monitoring system according to the present invention are not limited to this embodiment. For example, it is also possible to transmit a plurality of corrected images 21 to the control center 6 and combine these images in the control center 6. The control center 6 can also perform all the image processing operations (A/D conversion, image correction, and image combining). Furthermore, the image processing apparatus, image processing method, and vehicle monitoring system according to this embodiment relate to vehicle monitoring on a road in the tunnel 1. However, the image processing apparatus, image processing method, and vehicle monitoring system are not restricted to this embodiment. For example, the image processing apparatus, image processing method, and vehicle monitoring system can also be used for vehicle monitoring on roads in other places. (Second Embodiment) An image processing apparatus, image processing method, and vehicle monitoring system according to the second embodiment will be described below. The image processing apparatus, image processing method, and vehicle monitoring system according to this embodiment have the same configurations as the image processing apparatus, image processing method, and vehicle monitoring system according to the first embodiment, except that a monitor camera 2 as an image sensing means includes an image sensing optical system having a wide angle lens, and that an image processing apparatus 5 as an image processing means corrects image distortion caused by this wide angle lens. Fig. 6 is a view for explaining the monitor cameras in the image processing apparatus, image processing method, and vehicle monitoring system according to this embodiment. As shown in Fig. 6, the monitor cameras 2 are so arranged as to sense images of vehicles 11 traveling on a road 10 from the side of the road 10. Each monitor camera 2 has a wide angle lens in its image sensing optical system and can sense images over a wide view angle. As this wide angle lens, a fish-eye lens is preferably used. A "wide angle lens" herein mentioned means a lens having a view angle wider than that of a standard lens, e.g., a view angle of 60° or more. The operation of the vehicle monitoring system, the operation of the image processing apparatus, and the image processing method will be described below. Similar to the vehicle monitoring system according to the first embodiment, the monitor cameras 2 sense images of the vehicles 11 traveling on the road 10 and input video signals to a distributor 3. This distributor 3 properly inputs these video signals from the monitor cameras 2 to the image processing apparatus 5. The image processing apparatus 5 processes these video signals. As this image processing in the image processing apparatus 5, A/D conversion is performed as in the first embodiment. In subsequent image correction, correction for removing distortion of the image sensed through the wide angle lens is performed. As shown in Fig. 7, an A/D-converted digital image 30 has barrel-shaped form distortion because the image is sensed through the wide angle lens. By correcting this barrel form distortion of the digital image 30, as shown in Fig. 8, a corrected image 31 from which the distortion is removed is obtained. Subsequently, an image combining process combines the corrected images 31 pertaining to the video signals obtained at the same time by the monitor cameras 2 into a single composite image 32. As in the first embodiment, this image combining process is performed such that the roads 10 shown in the individual corrected images 21 appear to continue. Also, the combining is so performed that more than one image of the same vehicle 11 will not be displayed. Fig. 9 shows a composite image 32 subjected to the combining process. After the image combining process is completed, the composite image 32 is transmitted to a control center 6. The whole of this composite image 32 is displayed on a monitor 61 in the control center 6. As described above, the image processing apparatus, image processing method, and vehicle monitoring system according to this embodiment can display, on a single screen, the whole of the road 10 as an object of traffic monitoring, similar to the image processing apparatus, image processing method, and vehicle monitoring system according to the first embodiment. Accordingly, the entire road can be monitored without switching images to be displayed. Since there is no area which cannot be monitored owing to image switching, therefore, it is possible to prevent a delay of traffic abnormality detection and increase the speed of traffic abnormality detection. Also, since no image switching is performed, traffic abnormalities can be detected on the basis of changes in an image. This facilitates visual traffic abnormality detection and reduces the monitoring load. Additionally, since an image of a road to be monitored is sensed sideways by using a wide angle lens, image correction can be readily performed. Furthermore, a wide angle lens is used in the image sensing optical system of the monitor camera 2. Since this allows each monitor camera 2 to monitor a wide range, the number of the monitor cameras 2 to be installed can be reduced. (Third Embodiment) An image processing apparatus, image processing method, and vehicle monitoring system according to the third embodiment will be described below. The image processing apparatus, image processing method, and vehicle monitoring system according to this embodiment have the same configurations as the image processing apparatuses, image processing methods, and vehicle monitoring systems according to the first and second embodiments, except that image storage is performed after image combining in an image processing means and in an image processing step. Fig. 10 is a flow chart of image processing performed in the image processing apparatus, image processing method, and vehicle monitoring system according to this embodiment. A/D conversion in S100, image correction in S200, and image composition in S300 of Fig. 10 are the same as explained in the first embodiment, so a detailed description thereof will be omitted. In S400, image storage is performed. This image storage is done by storing a composite image 22 or 32 subjected to image composition into a storage device such as a memory. Since the composite image 22 or 32 is thus stored, after a traffic abnormality occurs it is possible to analyze the abnormality by checking this composite image. In the present invention as has been described above, the whole of a road as an object of traffic monitoring can be displayed on a single screen. Accordingly, the entire road can be monitored without switching images to be displayed. Since there is no area which cannot be monitored owing to image switching, therefore, it is possible to prevent a delay of traffic abnormality detection and increase the speed of traffic abnormality detection. Also, since no image switching is performed, traffic abnormalities can be detected on the basis of changes in an image. This facilitates visual traffic abnormality detection and reduces the monitoring load. Additionally, since an image of a road to be monitored is sensed sideways by using a wide angle lens, image correction can be readily performed. Furthermore, since corrected composited image data is stored, after a traffic abnormality occurs it is possible to analyze the abnormality by checking this composite image. Industrial Applicability The image processing apparatus, image processing method, and vehicle monitoring system of the present invention are useful in monitoring traffic of vehicles traveling on a road.

Claims: An image processing apparatus comprising: converting means (2,3,5,S100) for receiving a plurality of video signals obtained by sensing images taken at different positions along a road (10), and A/D-converting the video signals into digital images; correcting means (5,S200) for correcting the digital images such that the road in each image runs parallel to the longitudinal or lateral axis of the image; and image combining means (5,S300) for combining a plurality of corrected images obtained by said correcting means into a composite image, such that the roads shown in the corrected images appear as a continuing road. An apparatus according to claim 1, characterized by further comprising storage means (S400) for storing the composite image as image data. An image processing method comprising: the conversion step of receiving a plurality of video signals obtained by sensing images taken at different positions along a road, and A/D-converting the video signals into digital images; the correction step of correcting the digital images such that the road in each image runs parallel to the longitudinal or lateral axis of each digital image; and the image combining step of combining a plurality of corrected images obtained in the correction step, such that the roads shown in the corrected images appear as a continuing road. A method according to claim 3, characterized by further comprising the storage step of storing the combined images as image data. A vehicle monitoring system comprising: a plurality of image sensing means arranged along a road to be monitored to sense images of vehicles traveling on the road; image processing apparatus as claimed in Claim 1 for receiving video signals obtained by said image sensing means, and display means for displaying the combined images processed by said image processing apparatus. A vehicle monitoring system as claimed in claim 5, wherein said image sensing means each have a wide angle lens in an image sensing optical system thereof; and said correcting means are operative to correct distortion, caused by said wide angle lens, of each digital image. A system according to any one of claims 5 to 6, characterized in that said image processing means stores the combined image as image data.

Inventors: IWAMOTO TAKESHI, SAKAI KUNIO, SUZUKI KUNIHIKO, IWAMOTO, TAKESHI, SAKAI, KUNIO, SUZUKI, KUNIHIKO, IWAMOTO, Takeshi, Sumitomo Electric Ind., Ltd.
Applicants: SUMITOMO ELECTRIC INDUSTRIES, SUMITOMO ELECTRIC INDUSTRIES, LTD.
Application year: 2000
Main classification: G08G1
IPC codes: G09G5, H04N1, G08G1, H04N7