A METHOD IN AN IMAGE RECORDING SYSTEM

Abstract: An image recording system consists of two fixed sensors (1a, 1b) which record image data, which are A/D converted and transformed to three-dimensional information. The three-dimensional information is used for controlling a movable camera (2) which is capable of panning, tilting, zooming and focussing. The sensors (1a, 1b) extract image data in the close IR spectrum, the IR light source emitting light to the scene from which image data are to be recorded. Further, a high-pass or band-pass filter is provided in the fixed optoelectronic sensors.

Description: A method in an image recording system The invention concerns a method in an image recording system for extracting detailed images of an arbitrary section of a scene. Usually, both moving objects and stationary objects are present in a scene. In image reproduction of such a scene it is important to be able to track the moving objects, and in case of several moving objects then to be able to track a specific object. In the classic image reproduction technique a moving ob- ject is tracked in a scene simply by manually moving a camera toward the moving object and then tracking the ob¬ ject in the scene. More recent techniques employ automatic systems for tracking specific moving objects in a scene. Thus, the published PCT Application No. WO 94/17636 discloses a video system consisting of a movable camera with auto¬ matic focusing and a static camera. The movable camera is controlled by means of two-dimensional information from the static camera. Thus, the system has no three-dimen¬ sional information, which means that the physical size of the object and the distance to the camera cannot be de¬ termined. This, in turn, has the consequence that the im¬ age variation caused by the object during motion must be significantly slower than the autofocusing time for the autofocusing system to be stable. Furthermore, the system cannot handle more than one person in the scene. Further, US Patent Specification No. 4928175 discloses a camera system for three-dimensional determination of ob¬ jects. Images of an object are recorded by means of the two cameras, and any point in an x,y,z system of coordi¬ nates may be determined on the basis of this. Thus, no image of a scene is recorded. The patent discloses a method of optimizing all parameters in a specific cali- bration model implemented in one algorithm. The invention finds particular use for the calibration of measuring systems. Accordingly, the object of the invention is to provide a system of the type mentioned in the opening paragraph which is capable of automatically tracking moving objects in a scene and locking to an object without suddenly loosing the object in the field of vision of the movable camera, in a fast and simple manner. The object of the invention is achieved by a method of the type defined in the introductory portion of claim 1 and characterized in that the dynamic objects in the scene are determined by means of three-dimensional meas- urements performed by comparing registered images of fixed sensors with partly overlapping field of vision, and that the dynamic objects are tracked by at least one movable unit which is controlled on the basis of the im¬ age data of the fixed sensors. It is hereby ensured that the image recording system pro¬ vided is capable of computing an accurate three-dimen¬ sional estimate of the positions of fixed as well as dy¬ namic objects in the scene with respect to the sensor system as a function of time. When, as stated in claim 2, several pairs of sensors are used to provide several pairs of video vision for the de¬ termination of the three-dimensional information, the system can cover larger areas, and the accuracy of the three-dimensional position estimate can be improved. When the sensors have parallel optical axes, the calibra¬ tion of the system is less complicated, while obtaining a good precision with a simpler algorithm comprising fewer computation parameters. Expediently, CCD arrays may be used as sensors, said ar¬ rays being extremely suitable for the generation of image information for data processing purposes. When the movable unit can pan, tilt, zoom and focus, the moving object may be tracked in all directions, while enabling zooming or focusing of the object or a particu¬ lar portion thereof. When, as stated in claim 7, a user-controlled control unit is provided, capable of selecting one of the dynamic objects to be tracked by a movable unit, great flexibil¬ ity is imparted to the system, since even complicated scenes with several local moving objects may easily be controlled. To obtain improved imaging in the fixed sensors and thus improve the data basis for the subsequent image process- ing procedures, it is an advantage that the scene is il¬ luminated by a light source from the same approximate point of vision as the fixed sensors. It is hereby ensured that shadows caused by other sources of light are suppressed. Shadows caused by the light source positioned at the sensor, will not be visible to the sensors, since they are concealed behind the object. As stated in claim 10, it is possible to use an IR light source, which has the advantage that it is not visible. Finally, as stated in claim 11, it is an advantage that a high-pass filter or a band-pass filter is arranged in the optoelectronic system. It is hereby ensured that just a spectral section of light is included in the sensor expo- sure. For the light source placed at the sensor to be used optimally, an optical filter having a spectral char¬ acteristic as coincident with that of the light source as possible, is selected. To obtain the greatest possible suppression of the undesired light sources, the light source and the filter are positioned in a spectral range where the undesired light sources emit as little light as possible. The invention will now be explained more fully with ref- erence to an example shown in the drawing, in which fig. 1 shows a basic setup of an image recording system for use in the method of the invention, fig. 2 is a view of the basic components incorporated in the image recording system, fig. 3 shows a block diagram of the image recording sys¬ tem, fig. 4 shows how a light source is positioned in connec¬ tion with the image recording system of the invention, Fig. 5 shows a typical spectral response distribution for a CCD camera without an IR filter, fig. 6 shows the spectral distribution for an IR light source, and an example of the spectral distribution of the sunlight at the surface of the sea, as well as the characteristic of a high-pass filter, while fig. 7 is a similar view, using a band-pass filter. As will be seen in fig. 1, the image recording system of the invention consists of a movable camera 2 capable of panning, tilting, zooming and focusing (i.e. rotating vertically and horizontally and taking close-ups) . The camera 2 receives control signals from a camera control unit 5, whose signals are computed on the basis of data from two fixed sensors, such as registration by cameras la, lb of image information from a scene 3. A three-di¬ mensional interpreter 4 computes the three-dimensional position of the object on the basis of the image informa¬ tion from the cameras la, lb. Further, an external system 6, such as a recorder, may be connected to the camera 2. As will be seen in fig. 2, it is possible to control the entire system by means of a user unit, such as a PC, which may be used for introducing special commands, as will be described more fully below. Additionally, a cen- tral main system 8 may be provided, which, in addition to being connectible to e.g. a recorder, may communicate with e.g. local and public networks. As will be seen in fig. 3, which shows a more detailed system diagram of the image recording system of the in¬ vention, image data arriving from the cameras la, lb are A/D converted, and data are passed further on to a unit 10 which segments the dynamic objects detected by the cameras la, lb and controls the objects as a function of time. The dynamic objects may be determined by comparing image date for each of the cameras la, lb. Further, three-dimensional position data may be extracted from these image date in a unit 11, and these position data may be supplied to calibration data which, together with the three-dimensional position data, generate a complete three-dimensional model 13 of a scene. Three-dimensional positions from several static detection units are transformed in an object correspondence process 14 by an ordinary geometrical transformation to a common system of coordinates. Then, a complete three-dimensional scene model 15 comprising all the detected objects is provided. Furthermore, the unit 16 may accommodate an object prior- ity facility which is expedient in scenes including sev¬ eral moving objects. The unit 16 may e.g. be used for prioritizing moving objects. The prioritization may be based on predefined priority criteria. These may e.g. comprise the position of the object in the scene, the speed of the object, physical dimensions, colour composi¬ tion or combinations of these. Where several cameras are used, a unit 17 may be employed for controlling priori¬ tized activation of the various cameras. Control data are generated in the unit 19 for the camera 2 on the basis of partly three-dimensional transformation data from the unit 20 and partly three-dimensional position data from the unit 17. The camera 2 may hereby be caused to pan, tilt, zoom and focus. The right side of fig. 3 shows a block diagram of a pos¬ sible user face for the recording system of the inven¬ tion. As will be seen, it may consist of a unit 7 which may be a PC controlling a technical setup 22, which may e.g. communicate with an alarm trigger 25 or with a com- munication interface for transmission of images via a unit 23. The communication connections from the unit 23 may communicate with a remote control 24 from which the system may be controlled. Fig. 4 shows a system sketch where a movable object is designated 26, and 27 is the shadow of the object which occurs because of the light source 29. 30 is the station¬ ary unit, e.g. as shown in fig. 3, la, lb. Further, the system sketch shows a lamp 28 and sunlight which illumi¬ nate the scene. A filter 31 is arranged in front of the fixed unit 30. As will be explained more fully below in connection with figs. 5 to 7, the setup shown in fig. 4 ensures that very accurate data of the scene are pro¬ vided, as regards both movable objects and fixed objects, a light source 29, which may e.g. be an IR light source, being arranged adjacent the fixed unit 30, whereby the shadow from the object 26 is "concealed" behind the ob¬ ject 26. As will be seen, also a filter 31 is arranged in front of the camera, thereby additionally ensuring that undesirable light is not captured by the fixed unit 30. Fig. 5 shows an example of a spectral distribution for a CCD camera which may be used in connection with the image recording system of the invention, i.e. a CCD camera that may be incorporated as the cameras la, lb in fig. 2. The numeral 33 in fig. 6 and the numeral 36 in fig. 7 represent an example of a spectral distribution of sun¬ light at the surface of the sea. Since part of the influx of sunlight is absorbed in the atmosphere, the spectral distribution depends on the molecular composition of the atmosphere and the angle of incidence of the sunlight. It will be seen from the spectral distribution that the cam¬ era has a good response at 930 nanometres. A relatively great part of the sunlight is absorbed in the atmosphere at precisely this wavelength. Thus, the use of a light source 29 at 930 nanometres, i.e. in the IR range, pro¬ vides a good signal to noise ratio between the energies of the light source and the sunlight, which means that the image data recording will be accurate. When arranging the illumination as shown in figs. 4-6, the following advantages are obtained: a) all shadows are behind the objects, thereby ruling out any need for shadow elimination, b) since the IR light source cannot be seen, a rather strong light source may be used, and when suitably posi¬ tioned and pulsed, it will involve no discomfort. Although, generally, the invention has been explained in connection with a scene including fixed and moving ob¬ jects, the invented system may be used in many specific connections. By way of example, it may be used for the surveillance of business premises, such as banks, department stores, etc., where objects are to be protected, since the system can rapidly capture e.g. a person who is present in the business premises. The system may also be used for con¬ trol and surveillance in connection with weapon systems. Other examples of use may be: Perimeter protection (surveillance of fences, doors, boundaries, etc. ) Teleconference system Traffic control Automatization of TV transmission recording systems Training systems for e.g. teachers and athletes Automatization of stage lighting Surveillance of waters, harbours and the like. Finally, it should be mentioned that although the inven¬ tion has been explained especially in connection with special camera types, it is possible to use other detect¬ ing and control systems within the scope of the patent claims.

Claims: P a t e n t C l a i m s : 1. A method in an image recording system for extracting detailed images of an arbitrary section of a scene, c h a r a c t e r i z e d in that a) dynamic objects in the scene are determined by mans of three-dimensional measurements performed by comparing registered image pairs of at least to fixed sensors, and b) that the dynamic objects are tracked by one or more movable units which are controlled on the basis of motion information of the fixed sensors. 2. A method according to claim 1, c h a r a c t e r ¬ i z e in that several pairs of sensors are used to provide several image pairs for the determination of the three-dimensional information. 3. A method according to claims 1-2, c h a r a c t e r ¬ i z e d in that the sensors have partly overlapping fields of vision, and that the sensors have approximately parallel optical axes. 4. A method according to claim 1 or 2, c h a r a c ¬ t e r i z e d in that CCD arrays are used as sensors. 5. A method according to claims 1-3, c h a r a c t e r - i z e d in that the movable unit is capable of panning, tilting, zooming and focusing. 6. A method according to claims 1-5, c h a r a c t e r ¬ i z e d in that the movable unit is a camera, a display or a projector, a microphone, a loudspeaker, a weapon or a combination of these. 11 7. A method according to any of the preceding claims, c h a r a c t e r i z e d in that, where several dynamic objects are present in the scene, a user-controlled con¬ trol unit is provided, said control unit being adapted to select one of the dynamic objects to be tracked by the movable unit or units. 8. A method according to any of the preceding claims, c h a r a c t e r i z e d in that the scene is illumi- nated by a light source from a position adjacent the fixed sensors. 9. A method according to claim 8, c h a r a c t e r ¬ i z e d in that the light source is pulsed synchronously with the image recording/closing function of the fixed sensors. 10. A method according to claim 8 or 9, c h a r a c ¬ t e r i z e d in that an IR light source is used. 11. A method according to claims 8-9, c h a r a c ¬ t e r i z e d in that a high-pass filter or a band-pass filter is provided in the fixed optoelectronic sensor systems.

Inventors: DUE JESPER, HEDER THOMAS, JUERGENSEN PETER, NIELSEN NIELS P HEESER, DUE, JESPER, HEDER, THOMAS, JUERGENSEN, PETER, NIELSEN, NIELS, P., HEESER
Applicants: DEVITECH IS, DUE JESPER, HEDER THOMAS, JUERGENSEN PETER, NIELSEN NIELS P HEESER, DUE, JESPER, HEDER, THOMAS, JUERGENSEN, PETER, NIELSEN, NIELS, P., HEESER
Application year: 1995
Main classification: G01S3
IPC codes: G01B11, G01S3, G01S11, G06T1, H04N5, H04N7