Abstract:
A system and method for detecting the precise location of a reflective element in a video field wherein the determined location is subsequently utilized to generate a graphic to be added to a foreground video scene utilizing compositing techniques. A signal is generated specifying a precise point in the foreground video scene identifying a position of a reflective element. The signal is derived by comparing objects within the video field with a user specified size, shape and color. Graphics can be repositioned or lines or other graphic elements can be generated which track the movement of the reflective element.

Description:
This is a continuation of application Ser. No. 08/303,063 filed Sep. 8, 1994 now abandoned. 
    
    
     SUMMARY OF THE INVENTION 
     A system is disclosed for detecting the precise location of a reflective element in a video field wherein the determined location is subsequently utilized to generate a graphic to be added to a foreground video scene utilizing compositing techniques well known in the art. The system has application in many situations wherein it is desired to generate an image which is a composite of a foreground and background video. For example, weather broadcasts frequently utilize a composite video image wherein the weather person appears in front of a backing (typically blue) and the background video source is a weather map with clouds, temperatures, cold fronts and other graphics which may be added to the weather map in a predetermined sequence. Frequently, the weather person points to a position on the blue backing which corresponds to a position on the weather map at which a desired graphic is to appear, and, so long as the weather person&#39;s hand position and timing are correct, the graphic, such as a temperature for a particular city, appears in the composite video. However, with such systems, all graphics to be displayed must be positioned in advance and a sequence must be created which cannot be dynamically changed by the weather person other than, perhaps, to skip through or to the end of a particular preprogrammed sequence. 
     With the invented system, since a signal is generated specifying a precise point in the foreground video scene identifying a position the weather person is pointing at, as represented by a reflective element, graphics can be repositioned as desired by the weather person or lines or other graphic elements can be generated which track the movement of the reflective element. 
     Another possible use of the invented system is by a football analyst who can diagram a play showing the movement of various players by moving the reflective element across a frame of a background video scene of the playing field with the players lined up at the beginning of the play. The invention may also be used by military analysts to illustrate movement by ground, sea or air forces, either for public consumption or field officers, or wherever it is desired to illustrate a background video scene by a pointer device. 
     The invention is an improvement of the system described in co-pending application Ser. No. 07/867,048 filed Apr. 10, 1992 owned by the present assignee which is directed to an analog system for detecting the position of colored reflective objects in a video field having a spherical or generally spherical shape. The present invention is a digital system which is able to detect colored reflective objects which are much smaller in size than can be detected by the prior system and which may be of any shape and color. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a block overview diagram showing a system in which the present invention may be utilized. 
     FIG. 2 is a block diagram of the invented detector system. 
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     Referring to FIG. 1, the present invention is shown as detector system  11  which, along with a video source such as video camera  13 , are all that are needed to practice the invention. However, for completeness, FIG. 1 shows other elements which would be utilized in conjunction with the invented system to provide composite images as described above. These additional elements include a graphics computer  15  which by itself or from a background source  17  provides a background video to compositing system  21 . The background video includes the video from the background source or a video image preprogrammed into the graphics computer, plus graphics created by the graphics computer based upon inputs from the detector system. The compositing system  21  makes a composite image of the background image including graphics superimposed on the background image by the graphics computer along with the foreground video from video source  13 . The superimposed graphics are generated as a result of inputs from the detector system. The composite image created by compositing system  21  is displayed on composite image monitor  23 . Also shown is prompt monitor  25  which typically would display text for prompting the talent in the foreground scene as to the next element which will be appearing in the background. 
     The invented detector system  11  is shown in block diagram form in FIG.  2 . Referring first to FIG. 2, video camera  13  generates a conventional red, green and blue (RGB) signal containing the foreground video which is converted to three digital signals representing the video RGB signal by a B+254 triple channel 8 bit digitizer  33  with associated analog circuitry. The digitized signals are stored in a VRAM buffer  35  which is divided into two sections buffer 1  and buffer 2 . The buffers are filled using timing signals such that while one buffer is being filled by digitized RGB signals from A/D converter  33 , the other buffer is being read by digital signal processor  37  in a cyclical operation as commonly employed in the art. 
     Suitable modifications could be made that would allow the RGB signals from the camera to be replaced by some combination of luminance, chroma, and hue from the camera. Alternatively, ultraviolet light combined with camera filtering could be used to produce the desired peak signal. However, in the preferred embodiment RGB signal processing is utilized. 
     Digital signal processor  37  reads and processes the RGB signals in buffer  35  to detect the position in a field of video of an object having a user specified size, shape and color. 
     The following is a pseudo code description of the processing performed by a suitable digital signal processor  37 . Assumptions made are that the user has specified ranges of values for each of red, blue and green which the object to be detected must contain; the shape is a sphere whose dimensions are at least 3 pixels by 3 pixels; and the color to be detected is specified by capturing a reference color through a camera or other video source. 
     The buffer format in this case is xRGB where each of x, R, G and B is assigned a byte used to store corresponding values representing a single pixel. In the present embodiment of the invention, R represents the red value, G represents the green value, B represents the blue value of the pixel and x is unused, but is available to represent transparency or where the color space uses four variables to represent color instead of three. 
     
       
         
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
             
               
               
             
               
               
             
               
             
               
               
             
               
               
             
               
             
               
               
             
               
               
             
               
             
               
               
             
               
               
             
               
             
               
               
             
               
             
               
               
             
               
             
               
               
             
               
               
             
               
               
             
               
             
           
               
                   
               
             
             
               
                 XSize = XSize in pixels of Buffers 
               
               
                 XSize = YSize in pixels of Buffers 
               
               
                 RedMin = User Defined 
               
               
                 RedMax = User Defined 
               
               
                 GreenMin = User Defined 
               
               
                 GreenMax = User Defined 
               
               
                 BlueMin = User Defined 
               
               
                 BlueMax = User Defined 
               
               
                 PixelSize = 4 
               
               
                 /* 4 bytes per xRGB pixel; Undefined/Red/Green/Blue */ 
               
               
                 XPos = Unknown 
               
               
                 YPos = Unknown 
               
               
                 /* The last X/Y position of the object */ 
               
               
                 SearchObject( ) { 
               
             
          
           
               
                   
                 YPos = 0 
               
               
                   
                 while (YPos &lt; YSize) { 
               
             
          
           
               
                   
                 XPos = 0; 
               
               
                   
                 While (XPos &lt; XSize) { 
               
             
          
           
               
                   
                 If (CheckColor(XPos, YPos)) { 
               
             
          
           
               
                   
                 If (CheckSize( )) { 
               
             
          
           
               
                   
                 return (TRUE) 
               
               
                   
                 } 
               
             
          
           
               
                   
                 } 
               
             
          
           
               
                   
                 XPos = XPos+1 
               
               
                   
                 } 
               
             
          
           
               
                   
                 YPos = YPos+1 
               
               
                   
                 } 
               
             
          
           
               
                   
                 return (FALSE) 
               
             
          
           
               
                 } 
               
               
                 CheckColor (X, Y) { 
               
             
          
           
               
                   
                 Red = Buffer [1+X*PixelSize+Y*xSize*PixelSize] 
               
               
                   
                 Green = Buffer [2+X*PixelSize+Y*XSize*PixelSize] 
               
               
                   
                 Blue = Buffer [3+X*PixelSize+Y*Size*PixelSize] 
               
               
                   
                 if((Red &gt;= RedMin) &amp;&amp; (Red &lt;= RedMax)) { 
               
             
          
           
               
                   
                 if((Green &gt;= GreenMin) &amp;&amp; (Green &lt;= GreenMax)) { 
               
             
          
           
               
                   
                 If(Blue &gt;= BlueMin) &amp;&amp; (Blue &lt;= BlueMax)) { 
               
             
          
           
               
                   
                 return (TRUE) 
               
               
                   
                 } 
               
             
          
           
               
                   
                 } 
               
             
          
           
               
                   
                 } 
               
             
          
           
               
                   
                 return (FALSE) 
               
             
          
           
               
                 } 
               
               
                 CheckColor (X, Y) { 
               
             
          
           
               
                   
                 Red = Buffer[1+X*PixelSize+Y*XSize*PixelSize] 
               
               
                   
                 Green = Buffer [2+X*PixelSize+Y*XSize*PixelSize] 
               
               
                   
                 Blue = Buffer{3+X*PixelSize+Y*XSize*PixelSize] 
               
               
                   
                 if ((Red &gt;= RedMin) &amp;&amp; (Red &lt;= RedMax)) { 
               
             
          
           
               
                   
                 if ((Green &gt;= GreenMin) &amp;&amp; (Green &lt;= GreenMax)) { 
               
             
          
           
               
                   
                 If (Blue &gt;= BlueMin) &amp;&amp; (Blue &lt;= BlueMax)) { 
               
             
          
           
               
                   
                 return (TRUE) 
               
               
                   
                 } 
               
             
          
           
               
                   
                 } 
               
             
          
           
               
                   
                 } 
               
             
          
           
               
                   
                 return (FALSE) 
               
             
          
           
               
                 } 
               
               
                 CheckSize( ) { 
               
             
          
           
               
                   
                 X1 = XPos 
               
               
                   
                 X2 = XPos 
               
               
                   
                 Y1 = YPos 
               
               
                   
                 Y2 = YPos 
               
               
                   
                 while ((CheckColor (X1, YPos) == TRUE) &amp;&amp; (X1 &gt;0)) { 
               
             
          
           
               
                   
                 X1 = X1−1; 
               
               
                   
                 } 
               
             
          
           
               
                 /* Now X1 is the Left edge position of the object */ 
               
             
          
           
               
                   
                 while ((CheckColor (X2,YPos) == TRUE) &amp;&amp; (X2 &lt;= XSize)) { 
               
             
          
           
               
                   
                 X2 = X2+1; 
               
               
                   
                 } 
               
             
          
           
               
                 /* Now X2 is the Right edge position of the object */ 
               
             
          
           
               
                   
                 while((CheckColor(XPos, Y1) == TRUE) &amp;&amp; (Y1 &gt;0)) { 
               
             
          
           
               
                   
                 Y1 = Y1−1; 
               
             
          
           
               
                 /* Now Y1 is the Top edge position of the object */ 
               
             
          
           
               
                   
                 while ((CheckColor(XPos,Y2) == TRUE) &amp;&amp; (Y2 &lt;= XSize)) { 
               
             
          
           
               
                   
                 Y2 = Y2+1; 
               
             
          
           
               
                 /* Now Y2 is the bottom edge position of the object */ 
               
             
          
           
               
                   
                 Width = X2 − X1 
               
             
          
           
               
                 /* Now Width is the width of the object with XPos and YPos as the 
               
               
                 center */ 
               
             
          
           
               
                   
                 Height = Y2 − Y1 
               
             
          
           
               
                 /*Now Height is the height of the object with XPos and YPos as the 
               
               
                 center */ 
               
             
          
           
               
                   
                 if((Width &lt; 3) ∥ (Height &lt; 3) ∥ (abs(Width − Height) &gt; 3)) { 
               
             
          
           
               
                   
                 return(FALSE) 
               
               
                   
                 } 
               
             
          
           
               
                   
                 return(TRUE) 
               
             
          
           
               
                 /*Return false if the Width or Height is smaller than 3 pixels, or if the 
               
               
                 difference between the Width and Height is greater than 3 */ 
               
               
                 } 
               
               
                   
               
             
          
         
       
     
     The invention operates by capturing an analog RGB video signal  31  which has been converted by A/D converter  33  to a 24 bit true color form stored in a VRAM buffer  35 . VRAM  35  stores two video fields so that a current video field is stored in one portion of the buffer (designated buffer 1 ) and the next video field is stored in another portion of the buffer (designated buffer 2 ). While one field is being captured in one of buffer 1  or buffer 2 , the data in the other one of buffer 1  or buffer 2  is processed by DSP  37  according to the pseudo code which generates an output defining a contour based on chroma/luminance differences, i.e., the size and color and shape of an object within the current buffer area, i.e., buffer 1  or buffer 2  being processed. The color, minimum size, and spherical shape of the video are compared with the color specified by the user and the minimum size/spherical shape specified by the user to determine if the object to be tracked is within the portion of the image currently being examined. If a match is detected, the routine SearchObject( ) returns TRUE, otherwise it returns FALSE and the next buffer area is processed. Once the processing of the data in the current one of buffer 1  or buffer 2  has been completed, the process is repeated on the data in the other one of the buffer 1 /buffer 2  pair. The process of loading one buffer while processing the other continues so long as it is desired to track the object. Since the data is being loaded into VRAM buffer  35  at the rate of  30  frames per second, i.e., 60 fields per second, each one of buffer 1  and buffer 2  must be processed in {fraction (1/60)} of a second. This can be accomplished by a DSP using one of numerous commercially available microprocessors such as Motorola 680x0. 
     As each video field is processed, SRAM  39  is used to store the various variables required for and produced by the processing. These variables are as follows: 
     XSize: the size of the x dimension of VRAM buffer  35  in pixels. 
     YSize: the size of the y dimension of VRAM buffer  35  in pixels. 
     RedMin: the minimum value of the red component of the object to be tracked. 
     RedMax: the maximum value of the red component of the object to be tracked. 
     BlueMin: the minimum value of the blue component of the object to be tracked. 
     BlueMax: the maximum value of the blue component of the object to be tracked. 
     GreenMin: the minimum value of the green component of the object to be tracked. 
     GreenMax: the maximum value of the green component of the object to be tracked. 
     PixelSize: a variable indicating the number of bytes used to store the values for Red, Green, Blue and x. 
     XPos: the X or horizontal position of the center of the detected object within the video field in pixels from the origin (typically, the origin is the upper leftmost pixel in the video field). 
     YPos: the Y or vertical position of the center of the detected object within the video field in pixels from the origin (typically, the origin is the upper leftmost pixel in the video field). 
     X: a local variable corresponding to XPos 
     Y: a local variable corresponding to YPos 
     Red: the red value of the pixel currently being processed. 
     Green: the green value of the pixel currently being processed. 
     Blue: the blue value of the pixel currently being processed. 
     X 1 : variable used to find the left edge of the reflective object. 
     Yl: variable used to find the top, edge of the reflective object. 
     X 2 : variable used to find the right edge of the reflective object. 
     Y 2 : variable used to find the bottom edge of the reflective object. 
     Width: the width of the object with XPos and YPos as the center Height: the height of the object with XPos and YPos as the center. SRAM  39  may be implemented using commercially available static random access memory, the specifics of which would depend on the DSP being used. 
     Communications with main computer supplying user defined information allowing the user to set the user defined variables defining the color and size of the object to be tracked by writing to SRAM  39 . Also, the computer reads the XY position from the SRAM  39 . 
     Once an object has been detected, the position of the detected object (XPos and YPos) is passed to graphics computer  15  through bus interface  41  which uses this information as in the prior art. Bus interface  41  is also used to pass information from the graphics computer  15  or other host which is used by DSP  37 . The information which is passed from host is the RedMin, RedMax, BlueMin, BlueMax, GreenMin, GreenMax values set by the user. Bus interface may be implemented using for example, ISA bus or other suitable bus interface.