Patent Publication Number: US-2007116328-A1

Title: Nudity mask for use in displaying video camera images

Description:
BACKGROUND OF THE INVENTION  
      1. Field of the Invention.  
      The present invention relates to a method of removing unwanted content from a displayed image acquired by a video camera and, more particularly, to a method of masking particular unwanted sections from a displayed image acquired by a video camera.  
      2. Description of the Related Art  
      Video surveillance camera systems are found in many locations and may include either fixed cameras that have a fixed field of view and/or adjustable cameras that can pan, tilt and/or zoom to adjust the field of view of the camera. The video output of such cameras is typically communicated to a central location where it is displayed on one of several display screens and where security personnel may monitor the display screens for suspicious activity.  
      Closed circuit television cameras mounted high up on buildings or on street lamp poles for monitoring traffic or for other security purposes are often fully functional. With the latest low light technology and with powerful zoom lenses, these cameras are capable of capturing scenes in private locations to a much greater extent than most people think possible. Even though people are aware that they may be under video surveillance, the majority of the public is unaware of the sophistication of these cameras and of the wide range of images that these cameras are capable of acquiring. This is especially true of people who live in a downtown area and believe that they are safely out of view in their homes when in fact they are not.  
      In addition to the violation of unsuspecting peoples&#39; privacy, another problem presented by these cameras is that the scenes of nudity which the cameras enable video screens to display may distract guards from their primary purpose of watching for breaches of security. It is even possible that a guard with prurient interests may redirect such a camera from the premises to be monitored toward scenes of potential nudity, thereby further increasing the chances of a security breach going undetected.  
      What is needed in the art is a method of inhibiting the display of scenes of nudity that are captured by video surveillance cameras.  
     SUMMARY OF THE INVENTION  
      The present invention provides a surveillance camera system that recognizes human skin and obscures the display of the skin, thereby inhibiting the display of any potential scenes of nudity. The vision system may identify images of nudity by detecting skin-colored regions, extracting very simple features from these regions and making a classification decision. A two-stage skin filtering algorithm using likelihood matrices in hue, saturation, value (HSV) space followed by some local clustering may be used.  
      The invention comprises, in one form thereof, a surveillance camera system including a camera that acquires images. A display screen is operably coupled with the camera wherein images captured by the camera are displayable on the display screen. A processing device is operably coupled to the camera and/or the display screen. The processing device outputs a nudity mask for display on the display screen such that the nudity mask obscures at least a portion of a person&#39;s skin that is included in the images captured by the camera.  
      The invention comprises, in another form thereof, a method of operating a surveillance camera system, including acquiring images with a camera. Human skin within the acquired images is recognized. The acquired images are displayed on a display screen such that at least a portion of the recognized human skin is obscured in the displayed images.  
      The invention comprises, in yet another form thereof, a method of operating a surveillance camera system, including acquiring images with a camera. Sections including pixels having color values approximately equal to color values of human skin are identified in the acquired images. Information is removed from the identified sections in the acquired images. The acquired images are displayed after said removing step.  
      An advantage of the present invention is that it protects the privacy of people within the camera&#39;s field of view and lessens the chance of a guard becoming distracted by displayed scenes of nudity.  
      Another advantage is that the invention may operate automatically and may be used with any security camera.  
      Yet another advantage is that the invention enables very precise nudity masking, such as pixel-by-pixel.  
      A further advantage is that the nudity mask may be applied to either non-stationary or stationary images.  
      Still another advantage is that the invention may be used in conjunction with dynamic zooming.  
      Still yet another advantage is that the invention does not require any camera calibration.  
      Another advantage is that the invention may be used to mask any color of skin.  
      Yet another advantage is that the invention may employ different forms of nudity masks, such as solid, translucent, low-resolution and opaque masks. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
      The above mentioned and other features and objects of this invention, and the manner of attaining them, will become more apparent and the invention itself will be better understood by reference to the following description of an embodiment of the invention taken in conjunction with the accompanying drawings, wherein:  
       FIG. 1  is a schematic view of a video surveillance system in accordance with the present invention.  
       FIG. 2  is a schematic view of the processing device of  FIG. 1 .  
       FIG. 3  is a schematic view of a portion of the processing device which may be used with an analog video signal.  
       FIG. 4  is an illustrative histogram of hue and saturation values for human skin.  
       FIG. 5  illustrates an intersection between a slice of the histogram of  FIG. 4  with a fixed saturation value and a histogram of the hue values of a section of an acquired image with the same fixed saturation value.  
       FIG. 6  is a flow chart illustrating one embodiment of a method of the present invention for applying a nudity mask. 
    
    
      Corresponding reference characters indicate corresponding parts throughout the several views. Although the exemplification set out herein illustrates an embodiment of the invention, the embodiment disclosed below is not intended to be exhaustive or to be construed as limiting the scope of the invention to the precise form disclosed.  
     DESCRIPTION OF THE PRESENT INVENTION  
      In accordance with the present invention, a video surveillance system  20  is shown in  FIG. 1 . System  20  includes a camera  22  which is located within a partially spherical enclosure  24 . Enclosure  24  may be tinted to allow the camera to acquire images of the environment outside of enclosure  24  and simultaneously prevent individuals in the environment who are being observed by camera  22  from determining the orientation of camera  22 . Camera  22  includes motors which provide for the panning, tilting and adjustment of the focal length of camera  22 . Panning movement of camera  22  is represented by arrow  26 , tilting movement of camera  22  is represented by arrow  28  and the changing of the focal length of the lens  23  of camera  22 , i.e., zooming, is represented by arrow  30 . As shown with reference to coordinate system  21 , panning motion corresponds to movement along the x-axis, tilting motion corresponds to movement along the y-axis and focal length adjustment corresponds to movement along the z-axis. In the illustrated embodiment, camera  22  and enclosure  24  are a Philips AutoDome® Camera Systems brand camera system, such as the G3 Basic AutoDome® camera and enclosure, which are available from Bosch Security Systems, Inc. formerly Philips Communication, Security &amp; Imaging, Inc. having a place of business in Lancaster, Pa. A camera suited for use with the present invention is described by Sergeant et al. in U.S. Pat. No. 5,627,616, entitled Surveillance Camera System, which is hereby incorporated herein by reference.  
      System  20  also includes a head end unit  32 . Head end unit  32  may include a video switcher or a video multiplexer  33 . For example, the head end unit may include an Allegiant brand video switcher available from Bosch Security Systems, Inc. formerly Philips Communication, Security &amp; Imaging, Inc. of Lancaster, Pa. such as a LTC 8500 Series Allegiant Video Switcher which provides inputs for up to sixty-four cameras and may also be provided with eight independent keyboards and eight monitors. Head end unit  32  includes a keyboard  34  and joystick  36  for operator or user input. Head end unit  32  also includes a display device in the form of a monitor  38  for viewing by the operator. A 24 volt AC power source  40  is provided to power both camera  22  and a processing device  50 . Processing device  50  is operably coupled to both camera  22  and head end unit  32 .  
      Illustrated system  20  is a single camera application, however, the present invention may be used within a larger surveillance system having additional cameras which may be either stationary or moveable cameras or some combination thereof to provide coverage of a larger or more complex surveillance area. One or more VCRs or other form of analog or digital recording device may also be connected to head end unit  32  to provide for the recording of the video images captured by camera  22  and other cameras in the system.  
      The hardware architecture of processing device  50  is schematically represented in  FIG. 2 . In the illustrated embodiment, processing device  50  includes a system controller board  64 . A power supply/IO section  66  of processing device  50  is illustrated as a separate board in  FIG. 2 , however, this is done for purposes of clarity and the components of power supply/IO section  66  may be directly mounted to system controller board  64 . A power line  42  connects power source  40  to converter  52  in order to provide power to processing device  50 . Processing device  50  receives a raw analog video feed from camera  22  via video line  44 , and video line  45  is used to communicate video images to head end unit  32 . In the illustrated embodiment, video lines  44 ,  45  are coaxial, 75 ohm, 1 Vp-p and include BNC connectors for engagement with processing device  50 . The video images provided by camera  22  can be analog and may conform to either NTSC or PAL standards. Board  72  can be a standard communications board capable of handling biphase signals and including a coaxial message integrated circuit (COMIC) for allowing two-way communication over video links.  
      Via another analog video line  56 , an analog-to-digital converter  58  receives video images from camera  22  and converts the analog video signal to a digital video signal. After the digital video signal is stored in a buffer in the form of SDRAM  60 , the digitized video images are passed to video content analysis digital signal processor (VCA DSP)  62 . A video stabilization algorithm is performed in VCA DSP  62 . Examples of image stabilization systems that may be employed by system  20  are described by Sablak et al. in a U.S. patent application entitled “IMAGE STABILIZATION SYSTEM AND METHOD FOR A VIDEO CAMERA”, filed on the same date as the present application and having a common assignee with the present application, the disclosure of which is hereby incorporated herein by reference. The adjusted display image is sent to digital-to-analog converter  74  where the video signal is converted to an analog signal. The resulting annotated analog video signal is sent via analog video lines  76 ,  54 , analog circuitry  68  and analog video line  70  to communications plug-in board  72 , which then sends the signal to head end unit  32  via video line  45 .  
      Processor  62  may be a TIDM  642  multimedia digital signal processor available from Texas Instruments Incorporated of Dallas, Tex. At start up, the programmable media processor  62  loads a bootloader program. The boot program then copies the VCA application code from a memory device such as flash memory  78  to SDRAM  60  for execution. In the illustrated embodiment, flash memory  78  provides four megabytes of memory and SDRAM  60  provides thirty-two megabytes of memory. Because the application code from flash memory  78  is loaded on SDRAM  60  upon start up, SDRAM  60  is left with approximately twenty-eight megabytes of memory for video frame storage and other software applications.  
      In the embodiment shown in  FIG. 2 , components located on system controller board  64  are connected to communications plug-in board  72  via a high speed serial communications bus  63 , biphase digital data bus  80 , an  12 C data bus  82 , and RS-232 data buses  84 ,  88 . An RS-232/RS-485 compatible transceiver  86  may also be provided for communication purposes. Coaxial line  45  provides communication between processing device  50  and head end unit  32  via communications plug in board  72 . Various additional lines, such as line  49 , which can be in the form of an RS-232 debug data bus, may also be used to communicate signals from head end unit  32  to processing device  50 . The signals communicated by these lines, e.g., lines  45  and  49 , can include signals that can be modified by processing device  50  before being sent to camera  22 . Such signals may be sent to camera  22  via line  48  in communication with a microcontroller  90 . In the illustrated embodiment, microcontroller  90  is a H8S/2378 controller commercially available from Renesas Technology America, Inc. having a place of business in San Jose, Calif.  
      Microcontroller  90  operates system controller software and is also in communication with VCA components  92 . Although not shown, conductive traces and through-hole vias lined with conductive material are used provide electrical communication between the various components mounted on the printed circuit boards depicted in  FIG. 2 . Thus, VCA components such as VCA DSP  62  can send signals to camera  22  via microcontroller  90  and line  48 . It is also possible for line  46  to be used to communicate signals directly to camera  22  from head end unit  32  without communicating the signals through processing device  50 . Various alternative communication links between processing device  50  and camera  22  and head unit  32  could also be employed with the present invention.  
      System controller board  64  also includes a field programmable gate array (FPGA)  94  including three memory devices, i.e., a mask memory  96 , a character memory  98 , and an on-screen display (OSD) memory  100 . In the illustrated embodiment, FPGA  94  may be a FPGA commercially available from Xilinx, Inc. having a place of business in San Jose, Calif. and sold under the name Spartan  3 . In the illustrated embodiment, mask memory  96  is a 4096×16 dual port random access memory module, character memory  98  is a 4096×16 dual port random access memory module, and OSD memory  100  is a 1024×16 dual port random access memory module. Similarly, VCA components  92  includes a mask memory  102 , a character memory  104 , and an on-screen display (OSD) memory  106  which may also be dual port random access memory modules. These components may be used to mask various portions of the image displayed on-screen  38  or to generate textual displays for screen  38 . More specifically, this configuration of processing device  50  enables the processor to apply nudity masks, privacy masks, virtual masks, and on-screen displays to either an analog video signal or a digital video signal.  
      If it is desired to apply the nudity masks and on-screen displays to a digital image signal, memories  102 ,  104  and  106  may be used, and the processing necessary to calculate the position of the nudity masks and on-screen displays would take place in processor  62 . If the nudity masks and on-screen displays are to be applied to an analog video signal, memories  96 ,  98 , and  100  would be used and the processing necessary calculate the position of the nudity masks and on-screen displays would take place in microprocessor  90 . The inclusion of VCA components  92 , including memories  102 ,  104 ,  106  and processor  62 , in processing device  50  facilitates video content analysis, such as for recognizing human skin in the image. Alternative embodiments of processing device  50  which do not provide the same video content analysis capability, however, may be provided without VCA components  92  to thereby reduce costs. In such an embodiment, processing device  50  would still be capable of applying nudity masks, privacy masks, virtual masks, and on-screen displays to an analog video signal through the use of microprocessor  90  and field programmable array (FPGA)  94  with its memories  96 ,  98 , and  100 .  
      Processing device  50  also includes rewritable flash memory devices  95 ,  101 . Flash memory  95  is used to store data including character maps that are written to memories  98  and  100  upon startup of the system. Similarly, flash memory  101  is used to store data including character maps that are written to memories  104  and  106  upon startup of the system. By storing the character map on a rewritable memory device, e.g., either flash memory  95 ,  101 , instead of a read-only memory, the character map may be relatively easily upgraded at a later date if desired by simply overwriting or supplementing the character map stored on the flash memory. System controller board  64  also includes a parallel data flash memory  108  for storage of user settings including user-defined privacy masks wherein data corresponding to the user-defined privacy masks may be written to memories  96  and/or  102  upon startup of the system.  
       FIG. 3  provides a more detailed schematic illustration of FPGA  94  and analog circuitry  68  than that shown in  FIG. 2 . As seen in  FIG. 3 , in addition to mask memory  96 , character memory  98  and OSD memory  100 , FPGA  94  also includes an OSD/Masking control block  94   a , an address decoder  94   b , and an optional host-port interface HPI16 94c for communicating frame accurate position data. The HPI16 interface is used when the privacy mask and informational displays, e.g., individual text characters, are to be merged with a digital video image using VCA components  92 .  
      As also seen in  FIG. 3 , analog circuitry (shown in a more simplified manner and labeled  68  in  FIG. 2 ) includes a first analog switch  68   a , a second analog switch  68   b , a filter  68   c , an analog multiplexer  68   d , and a video sync separator  68   e . A “clean” analog video signal, i.e., although the image may be stabilized, the video signal includes substantially all of the image captured by camera  22  without any substantive modification to the content of the image, is conveyed by line  54  to the second analog switch  68   b , mixer  68   c  and sync separator  68   e . An analog video signal is conveyed from mixer  68   c  to first analog switch  68   a . Mixer  68   c  also includes a half tone black adjustment whereby portions of the video signal may be modified with a grey tone. Sync separator  68   e  extracts timing information from the video signal which is then communicated to FPGA  94 . A clean analog video signal, such as from FPGA  94  or line  54 , is also received by filter  68   c . Passing the analog video signal through filter  68   c  blurs the image and the blurred image is communicated to analog switch  68   a . Analog switch  68   a  also has input lines which correspond to black and white inputs. Two enable lines provide communication between analog switch  68   a  and FPGA  94 . The two enable lines allow FPGA  94  to control which input signal received by analog switch  68   a  is output to analog switch  68   b . As can also be seen in  FIG. 3 , second analog switch  68   b  includes two input lines, one corresponding to a “clean” analog video signal from line  54  and the output of analog switch  68   a . Two enable lines provide communication between analog switch  68   b  and FPGA  94  whereby FPGA  94  controls which signal input into analog switch  68   b  is output to line  70  and subsequently displayed on display screen  38 .  
      Each individual image, or frame, of the video sequence captured by camera  22  is comprised of pixels arranged in a series of rows and the individual pixels of each image are serially communicated through analog circuitry  68  to display screen  38 . When analog switch  68   b  communicates clean video signals to line  70  from line  54 , the pixels generated from such a signal will generate on display screen  38  a clear and accurate depiction of a corresponding portion of the image captured by camera  22 . To blur a portion of the image displayed on-screen  38  (and thereby generate a nudity mask or privacy mask or indicate the location of a virtual mask), analog switch  68   a  communicates a blurred image signal, corresponding to the signal received from filter  68   c , to analog switch  68   b . Switch  68   b  then communicates this blurred image to line  70  for the pixels used to generate the selected portion of the image that corresponds to the nudity mask, privacy mask or virtual mask. If a grey tone nudity mask, privacy mask or virtual mask is desired, the input signal from mixer  68   d  (instead of the blurred image signal from filter  68   c ) can be communicated through switches  68   a  and  68   b  and line  70  to display screen  38  for the selected portion of the image. To generate on-screen displays, e.g., black text on a white background, analog switch  68   a  communicates the appropriate signal, either black or white, for individual pixels to generate the desired text and background to analog switch  68   b  which then communicates the signal to display screen  38  through line  70  for the appropriate pixels. Thus, by controlling switches  68   a  and  68   b , FPGA  94  generates nudity masks, privacy masks, virtual masks, and informational displays on display screen  38  in a manner that can be used with an analog video signal. In other words, pixels corresponding to nudity masks, privacy masks, virtual masks, or informational displays are merged with the image captured by camera  22  by the action of switches  68   a  and  68   b.    
      In the illustrated embodiment, commands may be input by a human operator at head end unit  32  and conveyed to processing device  50  via one of the various lines, e.g., lines  45 ,  49 , providing communication between head end unit  32  and processing device  50  which also convey other serial communications between head end unit  32  and processing device  50 . In the illustrated embodiment, processing device  50  is provided with a sheet metal housing and mounted proximate camera  22 . Processing device  50  may also be mounted employing alternative methods and at alternative locations. Alternative hardware architecture may also be employed with processing device  50 . It is also noted that by providing processing device  50  with a sheet metal housing, its mounting on or near a PTZ (pan, tilt, zoom) camera is facilitated and system  20  may thereby provide a stand alone embedded platform which does not require a personal computer-based system.  
      The provision of a stand-alone platform as exemplified by processing device  50  also allows the present invention to be utilized with a video camera that outputs unaltered video images, i.e., a “clean” video signal that has not been modified. After being output from the camera assembly, i.e., those components of the system within camera housing  22   a , the “clean” video may then have a nudity mask and on-screen displays applied to it by the stand-alone platform. It is also possible, however, for processing device  50  to be mounted within housing  22   a  of the camera assembly.  
      The present invention may generally include acquiring images with camera  22 , and identifying, in the acquired images, sections including pixels having color values approximately equal to color values of human skin. Information may be removed from the identified sections such that segments of human skin in the acquired images are less recognizable to a human observer. In general, the content or color values of the pixels in the identified image sections may be altered to make the human skin in the images more difficult for the viewer to discern. In one embodiment, removing information from the identified sections includes outputting a nudity mask that obscures at least a portion of a person&#39;s skin that is included in the acquired images. The removal of the information may be dependent upon a size, shape, and/or orientation of the identified sections. After the undesired information is removed from the acquired images, the images may be displayed.  
      In one embodiment, the present invention identifies human skin based upon clusters or sections of commonly skin-colored pixels in the acquired images. In one particular embodiment, the image is analyzed in the hue, saturation, and value (HSV) color space, which may be derived from the red, green blue (RGB) color space. The present invention may employ a direct pixel-based segmentation technique in which the HSV color space is partitioned into a skin color region and a non-skin color region. The pixels in which the hue, saturation and value (brightness) values are all within the skin color region of the HSV color space may be recognized as skin.  
      The segmentation of a color image may include classifying the pixels within the image into a set of clusters each having a uniform color characteristic. The color clusters that correspond to the colors of human skin may be detected and isolated.  
      Color values that correspond to the colors of human skin may be empirically measured, and the normalized frequency of each of these color values may be stored in a lookup table. In order to achieve intensity invariance, and to reduce the amount of computation, only the chromacity (i.e., hue and saturation) color values may be considered, to the exclusion of brightness. Thus, a two-dimensional histogram of the combinations of hue and saturation color values that correspond to human skin may be created, as shown in  FIG. 4 . The values of the histogram of  FIG. 4  have been chosen for purposes of ease of illustration, and are not indicative of any actual skin color values. The values of the histogram of  FIG. 4  may have a normal distribution in both the hue direction and in the saturation direction, with a peak normalized frequency at approximately hue=5 and saturation=4.  
      Generally, pixels in the image that color values approximately equal to color values of human skin are recognized. More specifically, in order to identify a section of an acquired image that may include human skin, the processor may first look for pixels in the image that have color values corresponding to relatively high normalized frequencies in the histogram of  FIG. 4 . For example, the processor may initially look for pixels having color values of hue=5 and saturation=4. A single pixel having color values in the skin color range is not a good indication that the pixel is part of an image of skin, however, because an image may contain many isolated pixels that have the same color as skin but are associated with the background. Another problem is that for any particular image of human skin, the distribution of color values is not guaranteed to closely correspond to the  FIG. 4  histogram of the range of skin colors. However, a legitimate assumption may be that skin regions are of reasonable area compared to the total image area and contain a locally maximum likelihood value, such as hue=5 and saturation=4. The present invention may employ a region-growing algorithm that uses as its seed points likelihood local maxima above a certain threshold. The regions may be grown out to a lower likelihood threshold. The putative skin regions may correspond to the largest area granules with an underlying likelihood above a lower likelihood threshold.  
      Thus, after identifying an initial seed pixel having color values of hue=5 and saturation=4, the processor may then look at the color values of the adjacent or surrounding pixels and determine whether those color values correspond to relatively high normalized frequencies in the histogram of  FIG. 4 . As a simple example, the processor may determine whether a majority of the adjacent pixels have hue values between threshold values of 4 and 6 and saturation values between threshold values of 3 and 5. If a majority of the adjacent pixels do in fact have color values within that range, it may be more probable that the pixels are within a section of the image that includes human skin. The processor may then examine the color values of the next ring of pixels that surround the pixels that are adjacent the initial pixel. Because of the increased confidence that the pixels are part of a skin section of the image, the range of the threshold color values may be expanded. For example, the processor may determine whether the color values of the next ring of pixels have hue values between threshold values of 3 and 7 and saturation values between threshold values of 2 and 6. If a majority, or some predetermined percentage, of the ring of pixels do in fact have color values within that range, it may be still more probable that the pixels are within a section of the image that includes human skin.  
      The above-described process may continue so long as some percentage of the examined color values of the pixels are within the threshold range of skin color values. After a boundary of the potentially skin colored image section has been found, that is, after a group of pixels having color values that are outside the threshold range of skin color values has been found, examination of the color values of additional pixels on the opposite side of the of the potentially skin colored image section may continue until all of the boundaries of the potentially skin colored image section have been located in the image.  
      Within the potentially skin colored image section, any small pockets of pixels having color values that are outside the threshold range of skin color values may have their color values changed to color values that are inside the threshold range of skin color values. This process may be referred to as “flood filling.” 
      After the potentially skin colored image section, or a portion of the potentially skin colored image section, has been identified, the color values of the associated pixels may be examined as a group in order to make a decision as to whether the image section, or portion of image section, is sufficiently skin colored for a nudity mask to be applied thereto. To this end, a histogram of the normalized frequency of the color values of the identified image pixels may be compared to the known skin color histogram of  FIG. 4 . The greater the similarity between the two histograms, the greater the probability that the image section, or portion of the image section, includes human skin. That is, human skin, or segments thereof, may be recognized in the acquired images dependent upon the comparison of the color values. In one embodiment, a measure of the similarity s between the two histograms H skin  and H image  may be defined as the intersection of the two histograms:  
         s   ⁡     (       H   skin     ,     H   image       )       =       ∑     i   ,   j     n     ⁢     min   ⁢              H   skin     ⁡     (     i   ,   j     )       ,       H   image     ⁡     (     i   ,   j     )                      
 
      The two histograms H skin  and H image  may be normalized such that 0≦s≦1. The term min|H skin (i, j), H image (i, j)| in the above equation may be thought of as the lesser of the two histogram values at the particular values of hue (i) and saturation (j). Thus, because the histograms are normalized, the greater the summation of the lesser histogram values (i.e., the greater the value of s), the greater the similarity between the two histograms and the more likely that the image pixels are part of an image of human skin.  
      A “slice” through illustrative overlapping histograms is shown in  FIG. 5 , with saturation set to a value of 4. The values of the histogram of  FIG. 4  as a function of hue with saturation set to a value of 4 are indicated in  FIG. 5  in solid lines. Illustrative histogram values of an exemplary image section being analyzed are indicated in  FIG. 5  in dashed lines. For example, at hue=7, the histogram of known skin color values has a normalized frequency value of 4, and the histogram of the color values of the exemplary image section has a normalized frequency value of 8. The intersection between the two histograms, i.e., the lesser of the two histograms at each value of hue, is indicated in cross hatching. It is the summation of such intersections for each value of saturation that may be used as a measure of similarity of the two histograms, and thus to decide whether the image section of interest is skin colored.  
      Having determined that a section of the image is skin colored, the processor may then further analyze other features of the section of the image to determine whether they are consistent with the existence of nudity in the image. More particularly, the processor may determine whether the size (area), shape or orientation of the skin colored image section is such that there is more than a threshold level of probability that the skin colored image section does indeed include some nudity. The nudity mask may be output dependent upon this determination. The recognition of human skin within the acquired images may be dependent upon the size, shape, and/or orientation of an image section that has color values approximately equal to color values of human skin.  
      Before a nudity mask is applied, it may be determined whether the skin colored image section is of sufficient size such that it would present a privacy concern or a source of distraction for the viewer. This threshold size of the skin colored image section for applying a nudity mask may be expressed in terms of number of pixels, displayed image size in length and/or width, or as a percentage of the total displayed image.  
      A partially clothed person may have several discrete or separate segments of exposed skin. For example, a person wearing only shorts may have two exposed legs and a third segment including the torso, arms and head. Several such segments in close proximity to one another may be considered to be one continuous “blob” of skin in one embodiment of the invention. If there is more than one person in the image, there may also be more than one corresponding “blob” in the image. One of the features of the section of the image that may be considered by the processor is the size of the blob, i.e., its area. Other features of the image section may be derived by first finding an ellipse that best approximates or fits the size and shape of the blob. The use of an ellipse may be advantageous because the shape of the human body approximates an ellipse. Features of the image section that may be used by the processor in determining the presence of nudity may include, for example, an x-centroid and/or y-centroid of the blob ellipse; the length of the major axis and/or minor axis of the blob ellipse; the eccentricity of the ellipse; the orientation of the ellipse; the area of a convex hull fitted to the blob; and the diameter of a circle that has the same area as the blob.  
      Some of the above-described image features may be more important than others in determining the existence of nudity. Thus, the various image features may be ranked, and the decision whether to apply a nudity mask may be dependent upon the rankings of the image features. The features may be ranked using the mutual information of the class given the single feature. This process provides a subset of features that are used in one embodiment to make the nudity masking determination: the area of the largest blob in the image; the blob&#39;s centroid coordinates; the major and minor lengths of the fitted ellipse; and the orientation of the ellipse. These features may be evaluated using a k-nearest neighbor classifier algorithm, for example.  
       FIG. 6  provides a flowchart indicating one embodiment of a method  600  by which nudity masks are displayed on display screen  38  during normal operation of the surveillance camera system  20 . In a first step  602 , a lookup table including normalized frequency values for each combination of hue and saturation values in the skintone color histogram may be downloaded into processing device  50 . In step  604 , a new quarter common intermediate format (QCIF) color image in RGB color space is acquired by camera  22 . In step  606 , the color image acquired by camera  22  is converted from RGB to HSV color space. It is also possible that the skintone color histogram is downloaded into the processing device in RGB color space, in which case it too may be converted to HSV color space in step  606 . In step  608 , a histogram of a section of the acquired image, or of the entire acquired image, is computed and compared to the known histogram of skin colors in order to determine the intersection of the histograms. If the histogram is computed for only a section of the image, then additional histograms for each of the other image sections may be computed and compared to the known histogram of skin colors in order to find additional intersections. Thus, it may determined, either for the image as a whole or for each image section individually, whether the image likely includes skin. In a final step  610  of the skintone detection process, the region of skin color pixels in the acquired image is “grown” by examining small groups of pixels surrounding a core of skin colored pixels, as described above, until a complete set of boundaries of the skin colored segments in the image are located. Any small pockets of pixels within the boundaries that are not skin colored may be converted into skin colored pixels in the flood filling process. In the embodiment of method  600 , the computation of the histogram intersection occurs before the computation of the region growing algorithm. However, it is also possible for the order of these two steps to be reversed.  
      In a first step  612  of the nudity classification process, segments of skin that may belong to the same person in the image are grouped together into a blob. For example, three separate segments of skin may include two legs and a torso, respectively, and may be grouped together into a blob. Various features of separate image sections, each corresponding to one of the skin segments, may be analyzed to determine whether a combination of one or more of the image sections is consistent with at least a portion of a human body. Such features may include the x-centroid, y-centroid, length of elliptical axis, and orientation of the individual image sections, for example.  
      Alternatively, or additionally, a number of blobs may be formed by various combinations of the image sections, and the features of these blobs may be analyzed individually to thereby determine whether the features of that particular combination of image sections are indicative of, or consistent with, a human body. Such features may include the x-centroid, y-centroid, length of elliptical axis, and orientation of the individual blobs, for example.  
      In a next step  614 , a k-nearest neighbor classifier algorithm may be applied to decide whether the image includes an objectionable level of nudity based upon the above-described image features, which may include the skin area and orientation of one or more blobs. If it is decided in step  616  that there is a sufficient amount of exposed skin in the image, then the program proceeds to step  618  where the nudity mask is applied to the detected section of the image that includes exposed skin to thereby obscure at least a portion of the skin.  
      In one embodiment, substantially all of the person&#39;s skin that is included and recognized in the images captured by the camera is obscured. In another embodiment, the nudity mask may obscure a continuous section of the image that includes a plurality of separate segments of the person&#39;s skin. The continuous section may be in the form of a blob that is created by joining together the separate segments of the person&#39;s exposed skin.  
      Different types of obscuring infill may be used with the nudity mask. For example, the nudity mask may employ a solid infill, a translucent infill, a blurred infill, or an opaque infill. A solid mask infill may take the form of a solid color infill, such as a homogenous gray or white infill, that obscures the video image within the mask by completely blocking that section of the video image that corresponds to the nudity mask. A translucent infill may be formed by reducing the resolution of the video image contained within the nudity mask area to thereby obscure the video image within the nudity mask without blocking the entirety of the video image within the mask. For example, for a digital video signal, the area within the nudity mask may be broken down into blocks containing a number of individual pixels. The values of the individual pixels comprising each block are then averaged and that average value is used to color the entire block. For an analog video signal, the signal corresponding to the area within the mask may be filtered to provide a reduced resolution. These methods of reducing the resolution of a selected portion of a video image are well known to those having ordinary skill in the art.  
      These methods of obscuring the image may be desirable in some situations where it is preferable to reduce the resolution of the video image within the nudity mask without entirely blocking that portion of the image. For example, if the human subject of the nudity mask is also suspected of committing a breach of security, by using a translucent nudity mask, the details of the image corresponding to the person&#39;s exposed skin may be sufficiently obscured by the reduction in resolution to provide the desired privacy while still allowing security personnel to perceive the general movements of the individual to whom the nudity mask is applied.  
      After the nudity mask is applied in step  618 , the image may be displayed on screen  38  in step  620 , and operation then returns to step  606  to begin processing of the next acquired image. If it is determined in step  616  that there is not a sufficient level of nudity to apply a nudity mask, then the image is displayed in step  620 , and operation returns to step  606  to begin processing of the next acquired image.  
      Processing device  50  may perform several functions in addition to the provision of nudity masking, privacy masking, virtual masking, and on-screen displays. One such function may be an automated tracking function. For example, processing device  50  may identify moving target objects in the field of view (FOV) of the camera and then generate control signals which adjust the pan, tilt and zoom settings of the camera to track the target object and maintain the target object within the FOV of the camera. An example of an automated tracking system that may be employed by system  20  is described by Sablak et al. in U.S. patent application Ser. No. 10/306,509 filed on Nov. 27, 2002 entitled “VIDEO TRACKING SYSTEM AND METHOD” the disclosure of which is hereby incorporated herein by reference. It is possible for automatic tracking to be applied to the same human subject to which the nudity masking of the present invention is applied.  
      While this invention has been described as having an exemplary design, the present invention may be further modified within the spirit and scope of this disclosure. This application is therefore intended to cover any variations, uses, or adaptations of the invention using its general principles.