Patent Publication Number: US-7596240-B2

Title: Object tracking method and object tracking apparatus

Description:
BACKGROUND OF THE INVENTION 
   The present invention relates to an object tracking method and an object tracking apparatus, and more particularly, to an object tracking method and an object tracking apparatus for controlling an imaging field (visual field range) of an imaging device in accordance with motions of an object. 
   While remote monitoring systems relying on an imaging device such as a TV (television) camera have been conventionally used in wide applications, many of them are so-called manned monitoring systems which employ a watchman who is monitoring while viewing images displayed on a monitor. In this type of manned monitoring system, a watchman is required to watch images displayed on a monitor at all times to identify in real time an intruding object such as a human, a car or the like which can come into a field under monitoring, so that a large burden is imposed on the watchman. Since a human has limitations in concentration, the manned monitoring system experiences overlooked intruding objects, which cannot be ignored, and therefore has a problem on the reliability. Also, as monitoring cameras become increasingly widespread, a single watchman often monitors images from a number of TV cameras on a plurality of monitors. The watchman can also overlook an intruding object when a plurality of TV cameras simultaneously capture intruding objects. Therefore, in recent years, an automatic tracking monitoring system has been highly demanded for automatically detecting an intruding object through image processing of images captured by TV cameras, instead of the human-based monitoring, automatically adjusting the visual field, viewing direction, and viewing angle of the TV cameras in accordance with motions of the intruding object, and generating a predetermined report and/or alarm. 
   For realizing such a system, a function is required for detecting an intruding object from an image signal, and detecting motions of the intruding object in accordance with a predetermined monitoring scheme. An example of such a monitoring scheme for detecting an intruding object is a method called a “subtraction method” (see, for example, U.S. Pat. No. 6,088,468) which has been conventionally used in wide applications. The subtraction method involves comparing an input image captured by a TV camera with a previously created reference background image, i.e., an image without intruding object to calculate, for example, a difference in luminance at each pixel, and detecting a region having a large difference value as an object. 
   An example of monitoring scheme for detecting a moving amount of an intruding object is a method called a “template matching method” (see, for example, “Introduction of Computer Image Processing,” pp149-153, 1985, published by Soken Shuppan Ltd.) which has been conventionally used in wide applications just like the subtraction method. The template matching method involves registering an image of an intruding object detected by a subtraction method or the like as a template, and detecting the position most similar to the template image in sequentially inputted images. Generally, when an object of interest is tracked using the template matching, the template is sequentially updated with an image of the object of interest at a position detected by the matching processing in order to follow a change in posture of the object of interest. 
   There is also a camera control system which sets a region which should not be captured by a monitor camera, and controls the direction of the monitor camera to avoid capturing the region which should not be captured by the monitor camera (see, for example, JP-A-9-93573). 
   Further, JP-A-2001-69494 discloses a monitor camera apparatus which comprises a monitor camera capable of panning over 360 degrees and tilting beyond 90 degrees, and a control function therefor, where the monitor camera holds data for masking a private zone included in an image, and masks part of the image in accordance with the data. This prior art monitor camera apparatus, however, does not have a function of tracking an intruding object. 
   SUMMARY OF THE INVENTION 
   The aforementioned monitoring system is disadvantageous in that the TV camera can capture private houses which are not to be monitored, and that essential facilities within a field under monitoring can be hidden in a blind zone for a long time, thus making the system more likely to overlook an intruding object. 
   It is an object of the present invention to provide an object tracking method and an object tracking apparatus for tracking an object in a field under monitoring. 
   It is another object of the present invention to provide an object tracking method and an object tracking apparatus which are capable of tracking an object, even when a field under monitoring partially includes a particular region such as a region to be excluded from monitoring, or an essential region which must be monitored without fail, in consideration of such a particular region. 
   According to one aspect of the present invention, a tracking method is provided for tracking an object using an imaging device configured for controlling at least one of an imaging direction and zooming. The method comprises the steps of detecting the object from the image based on an image from the imaging device, calculating a current or a predicted visual field range of the imaging device, determining whether or not the current or predicted visual field range includes an imaging prohibited region, controlling at least one of the imaging direction and zooming of the imaging device based on the detected information on the object when the current or predicted visual field range does not include an imaging prohibited region, and tracking the object. 
   In one embodiment, the operation of the imaging device is stopped when the calculated or current visual field range includes an imaging prohibited region. 
   In one embodiment, the tracking method further comprises the step of operating the imaging device in an opposite direction for a predetermined time and stopping the operation of the imaging device when the calculated or current visual field range includes an imaging prohibited region. 
   In one embodiment, the tracking method further comprises the step of masking an imaging prohibited region included in a captured image when the captured image of the calculated current or predicted visual field range includes the imaging prohibited region. 
   In one embodiment, the tracking method further comprises the step of displaying a message which indicates that operation of the imaging device is stopped because the imaging prohibited region is detected. 
   In one embodiment, the calculation of the predicted visual field range of the imaging device includes the steps of calculating a moving amount of the imaging device based on a positional relationship between a current position of the object and a predetermined position within the image, and calculating the predicted visual field range based on a control amount of the imaging device including the moving amount. 
   According to another aspect of the present invention, an object tracking apparatus comprises an imaging device, a driving mechanism coupled to the imaging device for changing the position of a visual field range of the imaging device, an image signal processor for processing an image signal from the imaging device, and a controller for controlling the driving mechanism in response to a control signal from the image signal processor, wherein the image signal processor includes means for detecting an object based on an image from the imaging device, means for calculating a current or a predicted visual field range of the imaging device, means for determining whether or not the calculated current or predicted visual field range includes an imaging prohibited region, and means responsive to the determining means for generating the control signal for controlling at least one of the imaging direction and zooming of the imaging device based on the detected information on the object when the determining means determines that the calculated current or predicted visual field range does not include an imaging prohibited region. 
   Other objects, features and advantages of the invention will become apparent from the following description of the embodiments of the invention taken in conjunction with the accompanying drawings. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1  is a flow chart illustrating a sequence of operations according to one embodiment of the present invention; 
       FIG. 2  is a block diagram generally illustrating the configuration of a system according to one embodiment of the present invention; 
       FIG. 3  is a diagram for explaining the basic principles of a subtraction method employed in the present invention; 
       FIG. 4  is a diagram for explaining the basic principles of a template matching method employed in the present invention; 
       FIG. 5  is a diagram showing how a TV camera is controlled in accordance with the present invention; 
       FIG. 6  is a diagram showing how an imaging range is determined in a panning direction for the TV camera in accordance with the present invention; 
       FIG. 7  is a diagram showing how an imaging range is determined in a tilting direction for the TV camera in accordance with the present invention; 
       FIG. 8  is an explanatory diagram showing the relationship between an imaging range of the TV camera and a monitor map in the present invention; 
       FIG. 9  is a diagram showing how the TV camera is controlled to avoid capturing an imaging prohibited region within the imaging range in accordance with the present invention; 
       FIG. 10  is a flow chart illustrating the flow of an object tracking method according to another embodiment of the present invention; 
       FIG. 11  is a flow chart illustrating the flow of an object tracking method according to a further embodiment of the present invention; 
       FIG. 12  a flow chart illustrating the flow of an object tracking method according to a further embodiment of the present invention; and 
       FIG. 13  is a diagram for explaining an alternative way of controlling the TV camera to avoid capturing an imaging prohibited region in accordance with the present invention. 
   

   DETAILED DESCRIPTION OF THE EMBODIMENTS 
   Several embodiments of the present invention will be described with reference to the accompanying drawings, wherein like components are designated the same reference numerals. 
   One embodiment of the present invention will be first described with reference to  FIGS. 1 and 2 .  FIG. 2  is a block diagram illustrating the configuration of a system for embodying the present invention. The system illustrated in  FIG. 2  has an imaging device  201  which is composed of a TV camera  201   a , a zoom lens (imaging lens)  201   b , and a camera pan and tilt head (camera driving mechanism, hereinafter referred to as camera pan head)  201   c . A signal processor  202  comprises an image input unit  202   a , a pan and tilt head controller (hereinafter referred to as pan head controller)  202   b , a lens controller  202   c , a manipulation input unit  202   d , an image memory  202   e , an MPU (Micro Processing Unit)  202   f , a work memory  202   g , an external input/output unit  202   h , an image output unit  202   i , an alarm output unit  202   j , and a data bus  202   k . A manipulation unit  203  has a joy stick  203   a , a first button  203   b , and a second button  203   c . The system further has an external storage device  204 , an image monitor  205 , and an alarming device  206 , for example, an alarming lamp. The output of the TV camera  201   a  is connected to the data bus  202   k  through the image input unit  202   a , and a control unit of the zoom lens  201   b  is connected to the data bus  202   k  through the lens controller  202   c.    
   The camera pan head  201   c , which carries the TV camera  201   a  thereon, is connected to the data bus  202   k  through the pan head controller  202   b . The output of the manipulation unit  203  is connected to the data bus  202   k  through the manipulation input unit  202   d . The external storage device  204  in turn is connected to the data bus  202   k  through the external input/output unit  202   h ; the image monitor  205  for monitoring is connected to the data bus  202   k  through the image output unit  202   i ; and the alarming device  206  is connected to the data bus  202   k  through the alarm output unit  202   j . The MPU  202   f  and work memory  202   g  are connected to the data bus  202   k  without any intervention. 
   The TV camera  201   a  has functions of capturing a field under monitoring within its visual field, imaging the field under monitoring to output a video signal, and imaging an intruding object. For this purpose, the TV camera  201   a  is mounted on the camera pan head  201   c  which can freely swing the TV camera  201   a  in a panning and a tilting direction. The TV camera  201   a  also has the zoom lens  201   b , and has a function of enlarging an image. A captured video signal is stored in the image memory  202   e  through the data bus  202   k  from the image input unit  202   a . The external storage device  204  works to store programs, data and the like, such that a program, data and the like are read into the work memory  202   g  through the external input/output unit  202   h  as required, and conversely, data in the work memory  202   g  is preserved in the external storage device  204 . 
   Upon activation of the signal processor  202 , the MPU  202   f  reads an appropriate program preserved in the external storage device  204  into the work memory  202   g  for executing necessary signal processing. Specifically, the MPU  202   f  analyzes images stored in the image memory  202   e  within the work memory  202   g . Then, in accordance with the result of the processing, the MPU  202   f  controls the camera pan head  201   c  and zoom lens  201   b  to change the imaging field of the TV camera  201   a  and track an intruding object. Also, the MPU  202   f  displays the result of detecting an intruding object on the image monitor  205  as required, and drives the alarming device  206  to generate an alarm. 
   Next, the operation of the present invention, implemented in the foregoing system configuration, will be described with reference to  FIG. 1 . First, at an initialization step  101 , the manipulation unit  203  is used to initialize external devices, variables associated with respective components, image memory, and the like in order to execute an object tracking method according to the present invention. The flow next proceeds to step  102 . Step  102  involves a process for executing an object recognition based on the subtraction method. 
   Now, the object recognition based on the subtraction method will be described with reference to  FIG. 3 . At the first image input step  102   a , an input image  301  is captured from the TV camera  201   a . The input image  301  may have, for example, 320 pixels in the horizontal direction and 240 pixels in the vertical direction. At a subtraction processing step  102   b , a subtractor  306  (shown in  FIG. 3 ) of the MPU  202   f  calculates a difference value of the luminance at each pixel between the input image  301  captured at the first image input step  102   a  and a reference background image  302 , which does not include any intruding object, previously created and recorded in the image memory  202   e . At a binarizing step  102   c , the pixel value (differential value) of each pixel on the differential image  303  generated at the subtraction processing step  102   b  is calculated, for example, in eight bits per pixel, and represented in 256 gradation levels. A pixel value lower than a predetermined threshold Th (for example, Th=20) is set to “0,” and a pixel value equal to or higher than the threshold Th is set to “255” to generate a binary image  303 . While the threshold Th is set to 20 in this embodiment, the threshold Th is not limited to 20 but is determined empirically in accordance with conditions under which an intruding object is detected, the state of an input image, and the like. 
   At a labeling step  102   d , blocks of pixels having the pixel value “255” are detected in the binary image  304  generated at the binarizing step  102   c , and the respective blocks are numbered for distinction from one to another. At an intruding object existence determination step  102   e , it is determined that an intruding object exists within the field under monitoring if any block of pixels having the pixel value “255,” numbered at the labeling step  102   d , meets predetermined conditions (for example, a block has dimensions of 20 pixels or more in the horizontal direction and 50 pixels or more in the vertical direction). 
   In this way, a human-shaped object  309  captured in the input image  301  is calculated as an area  210  which causes a difference by the subtractor  306 , so that the human-shaped object  310  remains, while the background is not displayed. Therefore, an image  311  representative of the human-shaped object  310  is detected by a binarizer  307  (shown in  FIG. 3 ). The foregoing is the outline of the processing involved in the intruding object detection process  102  based on the subtraction method. 
   If it is determined at the intruding object existence determination step  102   e  that an intruding object exists, the flow proceeds to an alarm/detection information display step  103 . At the alarm/detection information display step  103 , information on the intruding object (position, the number of persons, and the like) is displayed on the image monitor  205  through the image output unit  202   i  for informing a watchman, for example, with a warning indicative of the detection of the intruding object, an alarm is generated from the alarming device  206  through the alarm output unit  202   j , and the like. If it is determined that no intruding object exists, the flow again returns to the image input step  102   a , where the subtraction method is repeated for detecting an intruding object. 
   Next, if it is determined at an intruding object existence determination step  102   e  that an intruding object exists, the flow proceeds to a template matching based intruding object tracking step  104 . Here, the template matching based intruding object tracking step  104  will be described with reference to  FIGS. 3 and 4 . At a template registration step  104   a , the image of the intruding object in the input image  301  is marked off by an extractor  308  (shown in  FIG. 3 ) based on a circumscribed rectangle  312  (shown in  FIG. 3 ) of the block of pixels having the pixel value “255,” numbered at the labeling step  102   d  to generate a template image  305 . The template image  305  includes a template  313  representative of the intruding object  309 , and this is registered as an initial template for the template matching based intruding object tracking step  104 . 
   Next, at a second image input step  104   b , in input image having, for example, 320 pixels in the horizontal direction and 240 pixels in the vertical direction is captured from the TV camera  201   a  in a manner similar to the first image input step  102   a . At a template matching step  104   c , the image presenting the highest matching degree to the template  313  is detected from the input image captured at the second image input step  104   b . Generally, a comparison of a template with an overall image takes a long calculation processing time, so that a predetermined range is defined as a search zone which may be extended from the template  313 , for example, by 20 pixels for each of the upper end and lower ends of the template and 50 pixels for each of the right and left ends of the template, and the image presenting the highest matching degree to the template  313  is detected in the search zone. A normalized correlation value r(Δx, Δy) can be applied to the matching degree, as expressed by Equation 1: 
                   r   ⁡     (       Δ   ⁢           ⁢   x     ,     Δ   ⁢           ⁢   y       )       =               ∑     i   ,     j   ∈   D         ⁢     (       f   ⁡     (         x   0     +     Δ   ⁢           ⁢   x     +   i     ,       y   0     +     Δ   ⁢           ⁢   y     +   j       )       -                         f   ⁡     (         x   0     +     Δ   ⁢           ⁢   x       ,       y   0     +     Δ   ⁢           ⁢   y         )       _     )     ·               (       g   ⁡     (         x   0     +   i     ,       y   0     +   j       )       -       g   ⁡     (       x   0     ,     y   0       )       _       )                               ∑     i   ,     j   ∈   D         ⁢     (       f   ⁢     (         x   0     +     Δ   ⁢           ⁢   x     +   i     ,       y   0     +     Δ   ⁢           ⁢   y     +   j       )       -                         f   ⁡     (         x   0     +     Δ   ⁢           ⁢   x       ,       y   0     +     Δ   ⁢           ⁢   y         )       _     )     2             ·                   ∑     i   ,     j   ∈   D         ⁢       (       g   ⁡     (         x   0     +   i     ,       y   0     +   j       )       -       g   ⁡     (       x   0     ,     y   0       )       _       )     2                         Eq   .           ⁢   1               
where
 
   
     
       
         
           
             
               f 
               ⁡ 
               
                 ( 
                 
                   
                     
                       x 
                       0 
                     
                     + 
                     
                       Δ 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       x 
                     
                   
                   , 
                   
                     
                       y 
                       0 
                     
                     + 
                     
                       Δ 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       y 
                     
                   
                 
                 ) 
               
             
             _ 
           
           = 
           
             
               1 
               
                  
                 D 
                  
               
             
             ⁢ 
             
               
                 ∑ 
                 
                   i 
                   , 
                   
                     j 
                     ∈ 
                     D 
                   
                 
               
               ⁢ 
               
                 f 
                 ⁡ 
                 
                   ( 
                   
                     
                       
                         x 
                         0 
                       
                       + 
                       
                         Δ 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         x 
                       
                       + 
                       i 
                     
                     , 
                     
                       
                         y 
                         0 
                       
                       + 
                       
                         Δ 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         y 
                       
                       + 
                       j 
                     
                   
                   ) 
                 
               
             
           
         
       
     
     
       
         
           
             
               g 
               ⁡ 
               
                 ( 
                 
                   
                     x 
                     0 
                   
                   , 
                   
                     y 
                     0 
                   
                 
                 ) 
               
             
             _ 
           
           = 
           
             
               1 
               
                  
                 D 
                  
               
             
             ⁢ 
             
               
                 ∑ 
                 
                   i 
                   , 
                   
                     j 
                     ∈ 
                     D 
                   
                 
               
               ⁢ 
               
                 f 
                 ⁡ 
                 
                   ( 
                   
                     
                       
                         x 
                         0 
                       
                       + 
                       i 
                     
                     , 
                     
                       
                         y 
                         0 
                       
                       + 
                       j 
                     
                   
                   ) 
                 
               
             
           
         
       
     
   
   f(x, y) represents the input image, and g(x, y) represents the template image. (x 0 , y 0 ) represents the coordinates at the upper left corner of the template  313  ( 401   a  in  FIG. 4 ). The image  401  has a coordinate system which defines the origin (0, 0) at the upper left corner of the image  401 . D represents the size of the template  313  ( 401   a ), for example, the size of the circumscribed rectangle  312  associated with the intruding object detected on the digitized image  304 , and represents, for example, an area of 20 pixels in the horizontal direction and 50 pixels in the vertical direction. The normalized correlation value r(Δx, Δy) takes a value which satisfies −1≦r(Δx, Δy)≦1, and takes “1” when an input image completely matches a template. The template matching refers to processing for detecting the position (Δx, Δy) at which the normalized correlation value r(Δx, Δy) is maximized when Δx, Δy are scanned within the search zone, i.e., varied in the ranges of −50≦Δx≦50, and −20≦Δy≦20, respectively, in the foregoing case. 
   Next, at a matching degree determination step  104   d , the matching degree r(Δx, Δy) is determined. When the normalized correlation value expressed by Equation 1 is used, a matching degree equal to or higher than 0.7, for example, is determined to be high, causing the flow to proceed to a template update step  104   e . If the matching degree is lower than 0.7, the flow returns to the first image input step  102   a  to repeat the foregoing processing. A high matching degree means that an image similar to the template  313  is detected in the second input image, i.e., an intruding object exists within the field under monitoring at a position offset by (Δx, Δy) relative to the position of the template  313 . In this event, the moving amount of the intruding object is detected subsequent to the detection. A low matching degree means that there is no image similar to the template  313  in the second input image, i.e., no intruding object exists within the field under monitoring, in which case the flow returns to the first image input step  102   a  to repeat the foregoing processing. It should be understood that though the normalized correlation value has been described to be 0.7, the normalized correlation value is not limited to this value but is set on an empirical basis. 
   Next, at the template update step  104   e , the image of the intruding object is marked off from the input image captured at the second image input step  104   b  based on the position of the newly found intruding object, and is defined to be a new template image. By thus updating the template in sequence, the image of the latest intruding object is recorded in the template. Also, even if the intruding object moves or changes in posture, the moving amount of the intruding object can be detected in a stable manner, thereby readily tracking the intruding object. 
   Now, an intruding object moving amount detection process will be described in greater detail with reference to  FIGS. 3 and 4 . 
   In  FIG. 3 , an intruding object  309  existing in an input image  301  is detected as a cluster of pixels having the pixel value “255” within a binary image at the labeling step  102   d . The intruding object  311  is marked off by the extractor  308  based on a circumscribed rectangular thereof to create a template image  305 . The template image  305  includes the template  313  of the intruding object  309  which is used as an initial template in the template matching based intruding object tracking process. 
   Next, the template matching is performed based on the initial template  313 .  FIG. 4  illustrates the flow of the template matching based intruding object tracking process for tracking the intruding object. A template image  401  includes a template  401   a . Here, parenthesized reference numerals  305 ,  313  are the same as the number given to the template image  305  and template  313 , respectively, shown in  FIG. 3 . In  FIG. 4 , the time at which the template image  401  is captured is represented by t 0 . Subsequently, in a similar manner, input images  402 ,  404 ,  406  sequentially inputted at predetermined time intervals (for example 100 ms) are captured at t 0 +1, t 0 +2, t 0 +3, . . . , respectively, in the order in which they are inputted. The image  401  represents a template image at time t 0 , while the image  401   a  represents a template at time t 0 . The image  402  in turn represents an input image at time t 0 +1, a rectangular area  402   b  in the image  402  indicates the position of the intruding object at time t 0  (position of the template  401   a ), and a rectangular area  402   c  indicates an area (search zone) subjected to the template matching. 
   Here, as the template matching process  409  (template matching step  104   c ) is executed, the matching degree is maximized in the image  402   a  which most matches the template  401   a  in the search zone  402   c  of the template matching, thus revealing that the intruding object is present at the position of the image  402   a  at time t 0 +1. In other words, it is understood that the intruding object has moved by a distance represented by an arrow  402   d  (by a moving amount (Δx, Δy)). 
   Next, in the template update process  410  (template update step  104   e ), the image  402   a  which most matches the template  401   a  is used to update the previous template  401   a  is updated to a new template at time t 0 +1. Specifically, the area  402   a  of the intruding object is marked off from the input image  402 , and is defined as a template image  403 , and the image  402   a  of the intruding object is used to update the template  401   a  to a new template  403   a  at time t 0 +1. As this series of processing is executed sequentially for input images sequentially captured by the TV camera, a search zone  404   c  is set within an input image  404  at time t 0 +2 based on the area  404   b  of the template  403   a  in a manner similar to the foregoing, and the area  404   a  of the intruding object is detected by the template matching process  409  using the template  403   a  within the template image  403  at time t 0 +1. As a result, it is understood that the intruding object has moved as indicated by an arrow  404   d . Further, through the template update process  410 , the template image  403  and template  403   a  at time t 0 +1 are updated respectively to a template image  405  and a template  405   a  of the intruding object at time t 0 +2. 
   Further, a search zone  406   c  is set within an input image  406  at time t 0 +3 based on the area  406   b  of the template  405   a , and the area  406   a  of the intruding object is detected using the template  405   a  within the template image  405  at time t 0 +2 through the template matching process  409 . As a result, it is understood that the intruding object has moved as indicated by an arrow  406   d . Further, through the template updating process  410 , the template image  405  and template  405   a  at time t 0 +2 are updated respectively to a template image  407  and a template  407   a  of the intruding object at time t 0 +3. A search zone  408   c  is set within an input image  408  at time t 0 +4 based on the area  408   b  of the template  407   a , and the area  408   a  of the intruding object is detected using the template  407   a  within the template image  407  at time t 0 +3 through the template matching process  409 . As a result, it is understood that the intruding object has moved as indicated by an arrow  408   d . In other words, the intruding object can be tracked by sequentially executing the template matching. 
   Prior to description on the next visual field range determination step  105 , problems to be solved by this embodiment will be described below in greater detail. 
   In recent years, the incease of monitoring cameras cause a problem of infringing the privacy of general residents. Since the template matching based object tracking method described above operates to automatically control the camera pan head  201   c  in response to a movement of an intruding object to capture the intruding object within the visual field of the TV camera  201   a , private houses not subjected to monitoring, a privacy sensitive place, and the like, if any, would be also captured within the visual field of the TV camera  201   a  unless the object tracking method is brought into practice without any limitations. When private houses not subjected to monitoring, and a privacy sensitive place are excluded from the visual field of the TV camera  201   a , the camera pan head  201   c  must be controlled in response to motions of an intruding object to track the intruding object without fail. For this purpose, in the present invention, the visual field of the TV camera  201   a  is controlled to avoid capturing private houses not subjected to monitoring and a privacy sensitive place, but to track an intruding object without fail in response to motions of the intruding object. In the following, this feature will be described in greater detail. 
   As an intruding object is detected at the intruding object tracking step  104  based on the template matching method, the camera pan head  201   c  is controlled in response to motions of the aforementioned template at a camera pan head control step  106  to track the object. In this event, a visual field range determination step  105  is executed for preventing the TV camera  201   a  from capturing private houses not subjected to monitoring and a privacy sensitive place into the visual field range. Before describing the visual field range determination step  105 , a camera pan head/zoom control amount calculation step  106   a  and a camera pan head/zoom control step  106   b  will be described with reference to  FIG. 5  as to how the camera pan head  201   c  can track an intruding object without fail in response to motions of the intruding object. 
   In  FIG. 5 , assume that an intruding object  505  is detected at a position as indicated by a template  502  in a template image  501 . In this event, assuming that the center of the intruding object  505  is defined as the center  503  of the template, displacements dx, dy of the center  503  from the center  504  of the template image  501  are calculated. 
   Here, if the center  503  of the template  502  is found to the left of the center  504  of the template image  501  (corresponding to an input image) by more than a predetermined amount S (when an intruding object exists near the center of the image, it is not necessary to control the camera pan head  201   c , so that a position at which the camera pan head control is started can be specified by the predetermined amount S. For example, S=50 pixels) (dx&lt;-S), the camera pan head  201   c  is panned to the left. Conversely, if the center  503  of the template  502  is to the right of the center  504  of the template image  501  by more than the amount S (dx&gt;S), the camera pan head  201   c  is panned to the right. If the center  503  of the template  502  is above the center  504  of the input image (dy&lt;-S), the camera pan head  201   c  is tilted upward, and if below the center  504  (dy&gt;S), the camera pan head  201   c  is tilted downward. Further, an amount of change in the focal length of the zoom lens (imaging lens)  201   b  may be acquired for changing the zooming of the TV camera  201   a  depending upon the size of the updated template, namely, the size of the detected intruding object on the screen. For example, when the height of the template is equal to or less than a predetermined value, the imaging lens  201   b  is zoomed in and, when larger than the predetermined value, it is zoomed out. The predetermined value may be for example 400 pixels assuming that the input image has the size of 640 pixels (width)×480 pixels (height). In this case, when for example the height of the current template is 300 pixels and the focal length of the current imaging lens  201   b  stored on the work memory  202   g  is f=30 mm, the focal length f of the imaging lens  201   b  may be set to 40 mm (=30×400/300) in order to make the height of the template 400 pixels. Accordingly, the MPU 202   f  controls the focal length of the imaging lens  201   b  to be 40 mm through the lens controller  202   c . In this way, the intruding object can be captured in an appropriate size within the visual field of the TV camera  201   a . It is a matter of course that the change of the zooming is not limited to this, but may be made based on arbitrary criterion and arbitrary manner. For example, the zooming may be changed only when the height of the template is outside a preset range. Alternatively, the imaging lens  201   b  may be controlled by such a simplified process that the focal length is lengthened in an increment of 1.0 mm as a zooming-in operation and it is shortened in a decrement of 1.0 mm as a zooming-out operation. The tracking processing for the intruding object is sequentially and repeatedly executed. With such simplified processing, therefore, even if a sufficient control of the focal length has not been attained, similar processing will be executed again in the next processing frame. Thus, by repeatedly executing the intruding tracking processing, the focal length of the imaging lens  201   b  can be controlled to become an appropriate value so that the height of the template may attain a predetermined value. Note here that the increment/decrement amount of 1.0 mm is an empirically determined value. When the value of the amount is large, the predetermined height of the template can be attained quickly with the possibility of generating an oscillation phenomenon in the focal length in the vicinity of the appropriate focal length value (over damping). Conversely, when the amount of increment/decrement in the focal length is small, it may take a long time for the template height to attain the predetermined value (under dumping). The amount of the increment/decrement in the focal length may be stored on the work memory  202   g  in advance. Note that the zooming may be realized by other methods than changing the focal length of the imaging lens  201   b , such as executing an electronic image-enlargement on the input image obtained from the TV camera  201   a.    
   Specifically, at the camera pan head/zoom control amount calculation step  106   a , the displacements dx, dy are calculated, and control amounts are also calculated for panning or tilting the camera pan head  201   c  (moving amount of the camera pan head  201   c ) and for zooming (the forcal distance of the imaging lens  201   b ). Then, at the camera pan head/zoom control step  106   b , the camera pan head and the imaging lens are controlled based on the calculated control amounts. Alternatively, the control velocity for a panning motor and a tilting motor may be varied by absolute values of dx, dy (the control velocity is increased as the absolute value of dx or dy is larger). 
   In this embodiment, “tracking” refers to following an intruding object with a controlled camera pan head. By doing so, the camera pan head  201   c  can be automatically controlled, while capturing an intruding object within the visual field of the TV camera  201   a  to track the intruding object. 
   Next, description will be made on the visual field range determination step  105 . First, at a visual field information acquisition step  105   a , the current direction of the camera pan head  201   c  (indicated by a panning angle θ P  and a tilting angle θ T , where the origin is defined on the front of the camera pan head) is acquired through the pan head controller  202   b , and a current focal length f of the imaging lens  201   b  is acquired through the lens controller  202   c.    
   At a visual field range calculation step  105   b , the position of an imaging range is calculated based on the current direction θ P , θ T  of the camera pan head  201   c , and the current focal length f of the imaging lens  201   b  acquired at the visual field information acquisition step  105   a . Here, a method of calculating the position of the imaging range will be described with reference to  FIGS. 6 and 7 . For simplicity, assume that a field under monitoring is plane and does not include any ragged ground.  FIG. 6  is a plane view taken from directly above a TV camera  601  at which the origin of coordinates is defined. Using this figure, the angle of view θ H  (the viewing angle of the camera in the lateral direction) can be calculated by Equation 2 
                   θ   H     =     2   ×   tan   ⁢     w     2   ×   f                 Eq   .           ⁢   2               
where w represents the width of an imaging device of the TV camera  601 , for example, a CCD (Charge Coupled Device), and w=4.8 mm when a ⅓-inch imaging device (device size: 4.8 mm×3.6 mm) is used. When a ⅓-inch imaging device is used with the focal length of an imaging lens set at f=8.0 mm, the TV camera  601  has the angle of view θ H  equal to 33.4°. In other words, the visual field of the TV camera  601  has a range of 33.4° in the horizontal direction. The TV camera  601  is installed in most cases at a position higher than a field under monitoring, so that the TV camera  601  cannot capture a region beneath itself in accordance with the current direction θ T  of the camera pan head  102   c . This region appears in a range of the line of sight L N  from the TV camera  601  (indicated by an area  602 ). The remaining area  603  (area indicated by L F -L N ) does appear within the visual field of the TV camera  601  without fail.
 
   The distances L N , L F  will be described with reference to  FIG. 7 .  FIG. 7  is a diagram showing a TV camera  701  (identical to the TV camera  601 ) viewed from one side, where the line of sight direction of the TV camera is defined to be the horizontal axis. As is the case with Equation 2, the angle of view θ V  (viewing angle of the camera in the vertical direction) of the TV camera  701  is calculated by Equation 3: 
                   θ   V     =     2   ×   tan   ⁢     h     2   ×   f                 Eq   .           ⁢   3               
where h represents the height of an imaging device of the TV camera  701 , for example, a CCD device, and h=3.6 mm when a ⅓ inch imaging device, for example, is used. When an ⅓ inch imaging device is used with the focal length of an imaging lens set at f=8.0 mm, similar to the aforementioned conditions, the TV camera  701  has the angle of view θ V  equal to 25.4°. In other words, the visual field of the TV camera  701  has a range of 25.4° in the vertical direction. Here, assuming that the TV camera  701  is installed at a height H, L N , L F  can be calculated by Equations 4:
 
   
     
       
         
           
             
               
                 
                   
                     L 
                     N 
                   
                   = 
                   
                     H 
                     × 
                     
                       tan 
                       ⁡ 
                       
                         ( 
                         
                           
                             90 
                             ⁢ 
                             ° 
                           
                           - 
                           
                             θ 
                             T 
                           
                           - 
                           
                             
                               θ 
                               V 
                             
                             2 
                           
                         
                         ) 
                       
                     
                   
                 
                 ⁢ 
                 
                   
 
                 
                 ⁢ 
                 
                   
                     L 
                     F 
                   
                   = 
                   
                     H 
                     × 
                     
                       tan 
                       ⁡ 
                       
                         ( 
                         
                           
                             90 
                             ⁢ 
                             ° 
                           
                           - 
                           
                             θ 
                             T 
                           
                           + 
                           
                             
                               θ 
                               V 
                             
                             2 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 Eqs 
                 . 
                 
                     
                 
                 ⁢ 
                 4 
               
             
           
         
       
     
   
   As a result of the foregoing calculations, a range which can be imaged by the TV camera  601  ( 701 ) extends over an area  603  shown in  FIG. 6  and an area  702  shown in  FIG. 7 . With a ⅓ inch imaging device, when the focal length f of the imaging lens is set at 8.0 mm (f=8.00); the camera pan head  201   c  is currently directed at θ T  of 30°; and the TV camera  701  is installed at a height of 5.0 m (H=5.0), under similar conditions to the foregoing, L N =5.42 m and L F =16.1 m are found from Equations 4. 
   Next, at an imaging prohibited region exclusion determination step  105   c , when the visual field range of the TV camera  201   a  (shaded areas  603 ,  702  in  FIGS. 6 ,  7 , respectively) calculated at the visual field range calculation step  105   b  does not include any region other than the field under monitoring (imaging prohibited region), the flow proceeds to the camera pan head control amount calculation step  106   a . On the other hand, the flow proceeds to a camera pan head stopping step  108  when the visual field of the TV camera  201   a  includes an imaging prohibited region. 
   Now, the determination made at the imaging prohibited region exclusion determination step  105   c  will be described with reference to  FIGS. 8 and 9 .  FIG. 8  is, for example, called a monitor map  801  which represents information on structures within a field under monitoring by such means as map information (or vector information). A TV camera  802 , visual field range  803 , essential monitor point  804 , and imaging prohibited region  805  are superimposed on the monitor map  801 . The map information on the monitor map  801  may be converted, for example, into (x, y) coordinates on x-, y-coordinate axes, and stored in the image memory  202   e  of the monitoring system in the form of a memory table (or may be stored in the external storage device  204 ). The imaging prohibited region  805  includes, for example, private houses adjacent to the field under monitoring, a place subjected to privacy protection, so that the region  805  should not be imaged by the TV camera  802 . However, the field under monitoring includes immediately in front of a gate  806 . 
   On the other hand, since the position of the visual field range  803  of the TV camera  802  is calculated by Equations 2-4 as described in connection with visual field range calculation step  105   b , the position of the visual field range  803  is converted into (x, y) coordinates on the monitor map  801  based on the result of the calculation. Specifically, each position within the visual field range  803  (indicated by shading) shown in  FIG. 8  is converted into x- and y-coordinates. For example, respective corners P 1 , P 2 , P 3 , P 4  of the visual field range  803 , represented by P 1 =(x 1 , y 1 ), P 2 =(x 2 , y 2 ), P 3 =(x 3 , y 3 ), and P 4 =(x 4 , y 4 ), can be calculated by Equations 5: 
                     x   1     =         L   N     ⁢   cos   ⁢           ⁢     θ   P       +       L   N     ⁢   tan   ⁢       θ   H     2     ⁢   sin   ⁢           ⁢     θ   P           ⁢     
     ⁢       y   1     =         -     L   N       ⁢   sin   ⁢           ⁢     θ   P       +       L   N     ⁢   tan   ⁢       θ   H     2     ⁢   cos   ⁢           ⁢     θ   P           ⁢     
     ⁢       x   2     =         L   N     ⁢   cos   ⁢           ⁢     θ   P       -       L   N     ⁢   tan   ⁢       θ   H     2     ⁢   sin   ⁢           ⁢     θ   P           ⁢     
     ⁢       y   2     =         -     L   N       ⁢   sin   ⁢           ⁢     θ   P       -       L   N     ⁢   tan   ⁢       θ   H     2     ⁢   cos   ⁢           ⁢     θ   P           ⁢     
     ⁢       x   3     =         L   F     ⁢   cos   ⁢           ⁢     θ   P       +       L   F     ⁢   tan   ⁢       θ   H     2     ⁢   sin   ⁢           ⁢     θ   P           ⁢     
     ⁢       y   3     =         -     L   F       ⁢   sin   ⁢           ⁢     θ   P       +       L   F     ⁢   tan   ⁢       θ   H     2     ⁢   cos   ⁢           ⁢     θ   P           ⁢     
     ⁢       x   4     =         L   F     ⁢   cos   ⁢           ⁢     θ   P       -       L   F     ⁢   tan   ⁢       θ   H     2     ⁢   sin   ⁢           ⁢     θ   P           ⁢     
     ⁢       y   4     =         -     L   F       ⁢   sin   ⁢           ⁢     θ   P       -       L   F     ⁢   tan   ⁢       θ   H     2     ⁢   cos   ⁢           ⁢     θ   P                   Eqs   .           ⁢   5               
where θ P , which represents a panning angle of the TV camera  802 , indicates here the angle formed by the x-axis and the optical axis of the TV camera  802 . As the respective x-, y-coordinates are calculated within the visual field range  803  in a similar manner, the position of the visual field range  803  of the TV camera  802  can be converted into the x-, y-coordinate system. It should be understood that the coordinates representative of the position of the visual field range  803  changes in response to panning and tilting of the TV camera  802  and zooming of the associated imaging lens.
 
   Next, a control method for avoiding the capture of the imaging prohibited region  805  within the visual field range  803  of the TV camera  802 , as mentioned above, will be described with reference to  FIG. 9 . In  FIG. 9 , a monitor map  901 , similar to the monitor map  801  is stored in the image memory  202   e , for example, in the form of x-, y-coordinate positions. An imaging prohibited region is indicated by shading  902 . An image  903  on a memory (hereinafter called the “visual field range image  903 ”) represents a TV camera  904  (identical to the TV camera  802 ) and a visual field range  905  of the TV camera  904 , the positions of which are found through calculations. Similar to the monitor map  901 , the image  903  schematically represents that the TV camera  904  and its visual field range  905  stored in the form of x-, y-coordinate positions. The origin (0, 0) of the x-y coordinate system is defined at the front end of the lens mounted to the TV camera  904 . Next, the monitor map  901  and visual field range image  903  are read from the image memory  202   e , and processed (ANDed)  903  by the MPU  292   f . Specifically, a logical AND operation is performed between the imaging prohibited region  902  and visual field range  905  on a pixel-by-pixel basis over the whole screen of the monitor map  901  to generate a processed image indicated by  906 . As a result, if the imaging prohibited region  902  is included in the visual field range  905  of the TV camera  904 , an imaging prohibited region fragment  907  is displayed. In other words, the detection of the imaging prohibited region fragment  907  means that part of the imaging prohibited region  902  is captured in the visual field range  905 . In this event, the MPU  202   f  stops the operation of the panning motor and tilting motor associated with the camera pan head  201   c  through the pan head controller  202   b  (camera pan head stopping step  108 ). Similarly, in step  108  the control of the imaging lens  201   b  is stopped. By doing so, it is possible to suppress part of the imaging prohibited region  902  included in the visual field range  905 . Thus, the TV camera  904  can be controlled in its panning, tilting and zooming operations to exclude the imaging prohibited region  902  from the visual field range  905 . 
   In this way, the camera pan head stopping step  108  involves stopping the operation of the panning motor and tilting motor associated with the camera pan head  201   c  when the visual field range  905  of the TV camera  201   a  includes a region other than the field under monitoring (imaging prohibited region  907 ), and may additionally include a masking process for masking any imaging prohibited region fragment  907  unintentionally captured in the visual field range  905  for exclusion from a resulting image. 
   In the foregoing description, the operation of the panning motor and tilting motor associated with the camera pan head  201   c  is stopped in response to detection of the imaging prohibited region fragment  907 . Alternatively, when the imaging prohibited region fragment  907  is detected, the panning motor and tilting motor associated with the camera pan head  201   c  may be controlled to rotate in the reverse directions, and then stop after the lapse of a predetermined time (for example, one control cycle, for example, 100 mS). Here, in controlling the panning motor and tilting motor in the reverse directions, when the camera pan head  201   c  is controlled to direct the TV camera in an upper right direction at the camera pan head control step  106   b  in the previous frame, the camera pan head  201   c  is controlled to direct the TV camera in the reverse direction, namely, in the lower left direction. By doing so, the imaging prohibited region  902  can be excluded from the visual field range  905  without fail. Further, when the imaging prohibited region fragment  907  is detected, the camera pan head stopping step  108  may include displaying a message on the monitor  205  for notifying that the camera pan head control is stopped because the imaging prohibited region fragment  907  has been detected, in addition to stopping the camera pan head. 
   As a further alternative, when the imaging prohibited region fragment  907  is detected, it becomes possible to prevent the imaging prohibited region fragment from being detected any longer by controlling the camera pan head  201   c  and imaging lens (zooming lens)  201   b . This method will now be described with reference to  FIG. 13 . 
   In this method, denoting the pan angle, tilt angle and focal length of the imaging lens as θ P0 , θ T0  and f 0 , respectively, Equations 2-5 are applied to the following θ P , θ T  and f 0  to obtain respective corners P 1 , P 2 , P 3  and P 4  of the visual field range  905  in order to determine whether or not the visual field range  905  overlaps the imaging prohibited region  902 .
 
θ P =θ P0   +lΔθ   P  
 
θ T =θ T0   +mΔθ   T  
 
 f=f 0+ nΔf  
 
Here, Δθ P , Δθ T  and Δθf are minimum increment/decrement of pan angle, tilt angle and focal length of the zoom lens, respectively, which are for example, 1 degree, 0.5 degree and 1 mm, respectively. In order to prevent the imaging prohibited area fragment from being included in the visual field range  905 , l, m and n are changed in the range of −10≦l≦10, −10≦m≦10, −10≦n≦10, respectively, to find a combination of l, m and n that provides no overlapping of the view field range  905  and imaging prohibited area  902 . By controlling the camera pan head  201   c  and zoom lens  201   b  in accordance with the then θ P , θ T  and f, it becomes becomes possible to prevent the imaging prohibited region fragment  907  from being included in the view field range  905 . These processing is executed by the MPU  202   f . As a simplified alternative method thereof is to simply make the focal length of the zoom lens  201   b  longer by a predetermined distance to thereby narrow the view field range  905  when the imaging prohibited region fragment  907  is detected.
 
   The camera pan head control step  106   b  is followed by execution of an alarm/tracking information display step  107 . Since details on step  107  is similar to step  103  previously described above, description thereon is omitted. 
   The alarm/tracking information display step  107  is also executed subsequent to the camera pan head stopping step  108  when NO is determined at imaging prohibited region exclusion determination step  105   c.    
   According to the monitoring system of the foregoing embodiment, as described in detail with reference to  FIG. 5 , when an intruding object is detected, the intruding object is tracked at the intruding object tracking step  104  based on the template matching method, where the intruding object can be tracked by controlling the camera pan head  201   c  in response to motions of the template. Therefore, with the use of this object tracking method, when an object, for example, intruding from the gate  804 , moves to the gate  806 , the position of the visual field range  803  of the TV camera  802  shown in  FIG. 8  can be moved in response to the moving object immediately before the moving object enters the gate  806 . However, a movement of the position of the visual field range  803  can be limited by controlling the panning, tilting and zooming operations of the TV camera  802  such that the imaging prohibited region  805  is not included in the visual field range  803 , thereby providing an extremely exact object tracking method. It should be noted that the template image  501  shown in  FIG. 5  is rectangular whereas the visual field range  803  is in the shape of trapezoid. This is because the visual field of the TV camera  802  is projected onto the monitor map  801  vertically from above in  FIG. 8 . Since the template image  501  is created by marking off part of the visual field range  803 , the difference in shape will not particularly cause problems. 
   Further, in the embodiment described above, when the imaging prohibited region  805  is partially included in the visual field range  803 , the TV camera  904  is controlled in its panning, tilting and zooming operations to exclude the imaging prohibited region  902  from the visual field range  905 . However, an overlapping area of the visual field range  803  of the TV camera  904  with the imaging prohibited region  902  also depends on the nature of the imaging prohibited region  902 , so that the imaging prohibited region  902  is empirically determined as appropriate upon installation of the monitoring system. When the monitoring of an intruding object causes the visual field range  803  to overlap with part of the imaging prohibited region  902  as described above, the overlapping portion of the visual field range  803  with the imaging prohibited region  902  may be masked, for example, by a known masking technique, such that information on the imaging prohibited region can be readily prevented from appearing in an image captured by the TV camera  904 . 
   Next, an object tracking method according to another embodiment of the present invention will be described with reference to  FIG. 10 . In the embodiment of  FIG. 1 , the camera pan head control amount calculation step  106   a  is placed behind the visual field range determination step  105 , whereas in a flow chart of  FIG. 10 , the camera pan head control amount calculation step  106   a  is placed between the intruding object tracking step  104  and visual field range determination step  105 . 
   In  FIG. 10 , at the camera pan head/zoom control amount calculation step  106   a , a panning amount and a tilting amount of the camera pan head  201   c  and a zooming amount of the imaging lens  201   b  are calculated based on the position of an intruding object calculated at the template update step  104   e . The calculations are made as previously described in connection with  FIG. 5 . At the visual field information acquisition step  105   a , a current panning angle θ P  and a current tilting angle θ T  are acquired and a current focal length f of the imaging lens  201   b  is also acquired through the lens controller  202   c . At the visual field range calculation step  105   b   1 , the orientation (direction) of the camera pan head  201   c  after controlling of the camera pan head is predicted taking the current direction θ P  and θ T  acquired in the visual field information acquisition step  105   a  and pan and tilt amounts calculated in the step  106   a  into account and the pan angle θ P1  and tilt angle θ t1  are calculated which are the results of the prediction. Further, taking account of the current focal length of the imaging lens  201   b  acquired at the visual field information acquisition step  105   a  and the zooming amount of the imaging lens  201   b  calculated in step  106   a , the focal length of the imaging lens  201   b  after controlling of the imaging lens  201   b  is predicted and the focal length f 1  which is the result of the prediction, is calculated. And, the position of the imaging range is calculated based on the calculated direction θ P1  and θ t1  and the focal length f 1  of the zooming lens  201   b . The calculated position of the imaging range is a predicted position. Therefore, at the imaging prohibited region exclusion determination step  105   c , it is determined whether or not the imaging prohibited region is included within the predicted visual field range after the camera pan head and zooming are controlled. At the camera pan head/zoom control step  106   b , if the imaging prohibited region is not included in the predicted visual field range after controlling of the camera pan head and zooming, the camera pan head  201   c  and zooming lens  201   b  are controlled based on the control amounts calculated at the camera pan head/zoom control amount calculation step  106   a.    
   According to this embodiment, whether or not a predicted visual field range includes an imaging prohibited region before the visual field range is actually updated, thereby making it possible to more accurately avoid imaging the imaging prohibited region and expect rapid tracking of an intruding object. Since this embodiment is similar to the flow chart of  FIG. 1  except that it differs from  FIG. 1  in that the camera pan head/zoom control amount calculation step  106   a  is placed between the template update step  104   e  and visual field acquisition step  105   a  and the visual field range calculation step  105   b  is replaced by the visual field range calculation step  105   b   1 , further description is omitted. 
   As described above, in the present embodiment, the TV camera is controlled to change the panning direction, tilting direction, and focal point of the lens in order to track an intruding object, but once an imaging prohibited place is included in the visual field range of the TV camera, the position of the visual field range of the TV camera is controlled to exclude the imaging prohibited place, thereby making it possible to realize an extremely excellent TV camera visual field control which enables the TV camera to track an intruding object without fail, and prevents the TV camera from capturing the imaging prohibited place. 
   An intruding object tracking method according to a further embodiment of the present invention will be described with reference to  FIG. 11 . In the foregoing embodiments, the TV camera is prevented from capturing an imaging prohibited region within its visual field range, whereas a monitoring essential region may be defined within a monitor map such that an essential (important) region is captured without fail within the visual field range. In doing so, a detected intruding object can be tracked in a range in which the essential region is captured in the visual field range, so that the essential region can be imaged in addition to the intruding object.  FIG. 11  is a flow chart illustrating such an intruding object tracking method according to one embodiment which can monitor the monitoring essential region at all times. The flow chart of  FIG. 11  is similar to that of  FIG. 1  except that the imaging prohibited region exclusion determination step  105   c  in  FIG. 1  is replaced by an essential region inclusion determination step  105   c   1 . Specifically, in the flow chart of  FIG. 1 , when an imaging prohibited region is included in the visual field range of the TV camera  201   a , the flow takes the NO branch from the imaging prohibited region exclusion determination step  105   c  to stop the camera pan head  201   c . In this embodiment, on the other hand, when the essential region is not included in the visual field range of the TV camera  201   a , the flow takes a NO branch from the essential region inclusion determination step  105   c   1  to a camera pan head stopping step  108 . At the camera pan head stopping step  108 , the camera pan head  201   c  undergoes camera pan head stopping processing. Note that the camera pan head stopping step  108  is configured to stop the camera pan head  201   c . Alternatively, the step  108  may be configured to control the pan and tilt motors of the camera pan head  201   c  to move in the reverse direction and stop after lapse of a predetermined time so that the orientation of the TV camera  201   a  is directed back to the inside of the essential region. Further, when the visual field of the TV camera  201   a  includes no part of the essential region, the camera pan head stopping step  108  may be adapted to cause the monitor  205  to display a message that the control of the camera pan head is stopped due to missing of the essential region, in addition to providing the camera pan head stopping processing. 
   It is also possible to combine the processing explained with reference to  FIG. 1  and the processing explained with reference to  FIG. 11  so as to control the zoom lens  201   b  and camera pan head  201   c  in such a manner that the imaging prohibited region is not included in the visual filed range of the TV camera and the essential region is always included in the view field range. 
   An intruding object tracking method according a further embodiment of the present invention will be described with reference to  FIG. 12 . 
   While all the foregoing embodiments are directed to automatic tracking of an intruding object, the present invention is not limited to such automatic tracking but can be applied to manual tracking of an intruding object as well. 
   A flow chart of  FIG. 12  illustrates an intruding object tracking method which can avoid imaging an imaging prohibited region even during automatic tracking conducted by a watchman, in addition to the automatic tracking of an intruding object. 
   In  FIG. 12 , subsequent to the initialization step  101 , the watchman (operator) selects automatic tracking or manual tracking (step  110 ). Upon selection of automatic tracking, the automatic tracking described with reference to  FIG. 1  is executed. Upon selection of manual tracking, an input image, for example, having 320 pixels in the horizontal direction and 240 pixels in the vertical direction is captured from the TV camera  201   a  (step  111 ). The MPU  202   f  checks whether or not an operator manipulates the TV camera  201   a  for monitoring (step  112 ). The TV camera  201   a  is manipulated (for controlling an imaging direction) on the manipulation unit  203 . The operator can also manually control the zooming of the TV camera  201   a  by means of the manipulation unit  203 . Upon detection of a monitoring manipulation for the TV camera  201   a , information on the manipulation input is acquired through the manipulation input unit  202   d  (step  113 ) and the processing proceeds to a visual field range determination step  105 ′ which is a similar step to the visual field range determination step  105  in the automatic tracking. At the visual field information acquisition step  105   a ′, the current direction of the camera pan head (indicated by a panning angle θ P  and a tilting angle θ T , where the origin is defined on the front of the camera pan head) is acquired through the pan head controller  202   b , and a current focal length f of the imaging lens  201   b  is acquired through the lens controller  202   c.    
   At a visual field range calculation step  105   b ′, the position of the imaging range is calculated based on the current direction θ P , θ T  of the camera pan head  201   c , and the current focal length f of the imaging lens  201   b  acquired at the visual field information acquisition step  105   a . Since a method of calculating the position of the imaging range is as previously described, description thereon is omitted. 
   Next, at the imaging prohibited region exclusion determination step  105   c ′, if the visual field range of the TV camera  201   a  calculated at the visual field range calculation step  105   b ′ does not include any region (imaging prohibited region) other than a range under monitoring, the flow proceeds to a camera pan head/zoom control step  106   b ′. Conversely, if the visual field range of the TV camera  201   a  includes the imaging prohibited region, the flow proceeds to a camera pan head stopping step  108 ′. 
   At the camera pan head/zoom control step  106   b ′, the TV camera  201   a  is moved in a direction specified by the operator based on the manipulation input information acquired in the manipulation input acquisition step  113 , and at a subsequent input image display step  114 , an image captured by the TV camera  201   a  is displayed on the monitor  205 . When the manipulation input information acquired at the manipulation input acquisition step  113  includes information on an amount of zoom change, the zooming of the imaging lens  201   c  is also controlled at the camera pan head/zoom control step  106   b′.    
   On the other hand, if the visual field range of the TV camera  201   a  calculated at the visual field range calculation step  105   b ′ includes the region (imaging prohibited region) other than the range under monitoring, the camera pan head is stopped at the camera pan head stopping step  108 ′. In addition, control may be made to move the camera pan head in a reverse direction and a message may be displayed on the monitor  205  for announcing that the TV camera  201   a  is stopped because the imaging prohibited region is included in the visual field range of the TV camera  201   a.    
   Since details on steps  105 ′,  105   a ′,  105   b ′,  105   c ′,  106 ′,  108 ′ are similar to previously described step  105 ,  105   a ,  105   b ,  105   c ,  106 ,  108 , respectively, detailed description thereon is deemed to be unnecessary. It is a matter of course that even in conducting manual tracking by the watchman, it is possible to control such that the essential region is always included in the visual field range without fail, as in the case of the method explained in connection with  FIG. 11 . Further, in the description above the watchman manipulated the TV camera  201   a  through the manipulation unit  203  connected to the signal processor  202 . However, it is also possible to manipulate the TV camera  201   a  by means of a remotely placed PC (personal computer) which is connected to the signal processor  202  through an external network such as LAN or the Internet. 
   While the foregoing embodiments have been described in connection with the subtraction method given as an example of the object detecting method and the template matching method given as an example of the object moving amount detecting method, it should be understood that methods other than those can be used as long as they can implement the object detection and object moving amount detection. 
   While the present invention has been described above in detail, it should be understood that the present invention is not limited to the object tracking method and object tracking apparatus described above, but can be widely applied to object tracking methods and object tracking apparatuses other than the foregoing. 
   The conventional object tracking methods and object tracking apparatuses have suffered from problems of private houses and a privacy sensitive place unintentionally captured by a TV camera, though they are not subjected to monitoring, and essential facilities within a field under monitoring hidden in a blind zone. According to the present invention, the camera pan head can be limited in its moving range such that private houses, privacy sensitive places, and the like are not captured by the TV camera, while an object is being tracked. In addition, an object can be tracked while essential facilities within the field under monitoring are captured without fail in the visual field of the TV camera. 
   According to the foregoing embodiments, an object can be tracked while essential facilities within a field under monitoring are kept captured without jeopardizing the privacy of general residents or capturing privacy sensitive places and the like. Thus, the resulting object tracking method and object tracking apparatus provided by the present invention can largely extend an applicable range of a video monitoring apparatus. 
   It should be further understood by those skilled in the art that although the foregoing description has been made on embodiments of the invention, the invention is not limited thereto and various changes and modifications may be made without departing from the spirit of the invention and the scope of the appended claims.