Patent Publication Number: US-2003225513-A1

Title: Method and apparatus for providing multi-level blended display of arbitrary shaped textures in a geo-spatial context

Description:
[0001] This non-provisional application claims the benefit of U.S. provisional application serial No. 60/372,301 filed Apr. 12, 2002, which is hereby incorporated herein by reference. 
    
    
     [0002] This invention was made with U.S. government support under contract number NMA202-97-D-1033 D0#33 of NIMA. The U.S. government has certain rights in this invention. 
    
    
     
       [0003] The invention is generally related to image processing systems and, more specifically, to a method and apparatus for performing geo-spatial registration and visualization within an image processing system.  
       BACKGROUND OF THE INVENTION  
       [0004] The ability to locate scenes and/or objects visible in a video/image frame with respect to their corresponding locations and coordinates in a reference coordinate system is important in visually-guided navigation, surveillance and monitoring systems.  
       [0005] Various digital geo-spatial products are currently available. Generally, these are produced as two dimensional maps or imagery at various resolutions. Current systems (e.g., MAPQUEST™) display these products as two-dimensional images which can be panned and zoomed at discrete levels of resolution (in several steps), but not continuously in a smooth manner. Additionally, the user is often limited to a rectangular viewing region.  
       [0006] Therefore, there is a need in the art for a method and apparatus that allows overlaying of multiple geo-spatial maps/images of arbitrary shapes within a region.  
       SUMMARY OF THE INVENTION  
       [0007] The present invention is a method and apparatus for displaying geo-spatial images. The invention advantageously provides a method for displaying an arbitrary defined region and its respective geographical image. Specifically, the method displays a geo-spatial image by providing a textured region of interest; selecting an arbitrary shaped area within the textured region of interest; and overlaying an image over the selected arbitrary shaped area.  
       [0008] Furthermore, the invention does not limit the arbitrary defined region to be a rectangular shape. Arbitrary shaped regions can be visualized simultaneously and at resolutions different from each other. 
     
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
     [0009] So that the manner in which the above recited features of the present invention are attained and can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments thereof which are illustrated in the appended drawings.  
     [0010] It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments.  
     [0011]FIG. 1 depicts a block diagram of an embodiment of a system incorporating the present invention;  
     [0012]FIG. 2 depicts a functional block diagram of an embodiment of a geo-registration system for use with the invention;  
     [0013]FIG. 3 depicts a flowchart of a method for displaying arbitrary shaped regions in accordance with the present invention;  
     [0014]FIG. 4 depicts a flowchart of a method for displaying arbitrary shaped regions in accordance with the present invention;  
     [0015] FIGS.  5 - 6  depict respective images used to create an embodiment of a textured geographical reference image;  
     [0016]FIG. 7 depicts an embodiment of a textured geographical reference image created from respective images depicted in FIGS.  5 - 6 ;  
     [0017]FIG. 8 depicts an embodiment of a textured geographical reference image smaller in geographical size than the images depicted in FIGS.  5 - 7 ;  
     [0018]FIG. 9 depicts an embodiment of a textured geographical reference image smaller in geographical size than the reference image depicted in FIG. 8;  
     [0019]FIG. 10 depicts an outline of an arbitrary defined region; and  
     [0020]FIG. 11 depicts an image within the arbitrary defined region. 
    
    
     [0021] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures.  
     DETAILED DESCRIPTION  
     [0022]FIG. 1 depicts a block diagram of a comprehensive system  100  containing a geo-registration system  106  of the present invention. The figure shows a satellite  102  capturing images of a scene at a specific locale  104  within a large area  108 . The system  106  identifies information in a reference database  110  that pertains to the current video images being transmitted along path  112  to the system  106 . The system  106  “geo-registers” the satellite images to the reference information (e.g., maps) or imagery stored within the reference database  110 , i.e., the satellite images are aligned with the map images and other information if necessary. After “geo-registration”, the footprints of the satellite images are shown on a display  114  to a user overlaid upon the reference imagery or other reference annotations. As such, reference information such as latitude/longitude/height of points of interest are retrieved from the database and are overlaid on the relevant points on the current video. Consequently, the user is provided with a comprehensive understanding of the scene that is being imaged.  
     [0023] The system  106  is generally implemented by executing one or more programs on a general purpose computer  126 . The computer  126  contains a central processing unit (CPU)  116 , a memory device  118 , a variety of support circuits  122  and input/output devices  124 . The CPU  116  can be any type of high speed processor. The support circuits  122  for the CPU  116  include conventional cache, power supplies, clock circuits, data registers, I/O interfaces and the like. The I/O devices  124  generally include a conventional keyboard, mouse, and printer. The memory device  118  can be random access memory (RAM), read-only memory (ROM), hard disk storage, floppy disk storage, compact disk storage, or any combination of these devices. The memory device  118  stores the program or programs (e.g., geo-registration program  120 ) that are executed to implement the geo-registration technique of the present invention. When the general purpose computer executes such a program, it becomes a special purpose computer, i.e., the computer becomes an integral portion of the geo-registration system  106 . Although the invention has been disclosed as being implemented as an executable software program, those skilled in the art will understand that the invention may be implemented in hardware, software or a combination of both. Such implementations may include a number of processors independently executing various programs and dedicated hardware such as application specific integrated circuits (ASICs).  
     [0024]FIG. 2 depicts a functional block diagram of the geo-registration system  106  of the present invention. Illustratively, the system  106  is depicted as processing a video signal as an input image; however, from the following description those skilled in the art will realize that the input image (referred to herein as input imagery) can be any form or image including a sequence of video frames, a sequence of still images, a still image, a mosaic of images, a portion of an image mosaic, and the like. In short, any form of imagery can be used as an input signal to the system of the present invention.  
     [0025] The system  106  comprises a video mosaic generation module  200  (optional), a geo-spatial aligning module  202 , a reference database module  204 , and a display generation module  206 . Although the video mosaic generation module  200  provides certain processing benefits that shall be described below, it is an optional module such that the input imagery may be applied directly to the geo-spatial aligning module  202 . When used, the video mosaic generation module  200  processes the input imagery by aligning the respective images of the video sequence with one another to form a video mosaic. The aligned images are merged into a mosaic. A system for automatically producing a mosaic from a video sequence is disclosed in U.S. Pat. No. 5,649,032, issued Jul. 15, 1997, and incorporated herein by reference.  
     [0026] The reference database module  204  provides geographically calibrated reference imagery and information that is relevant to the input imagery. The satellite  102  provides certain attitude information that is processed by the engineering sense data (ESD) module  208  to provide indexing information that is used to recall reference images (or portions of reference images) from the reference database module  204 . A portion of the reference image that is nearest the video view (i.e., has a similar point-of-view of a scene) is recalled from the database and is coupled to the geo-spatial aligning module  202 . The module  202  first warps the reference image to form a synthetic image having a point-of-view that is similar to the current video view, then the module  202  accurately aligns the reference information with the respective satellite image. The alignment process is accomplished in a coarse-to-fine manner as described in detail below. The transformation parameters that align the video and reference images are provided to the display module  206 . Using these transformation parameters, the original video can be accurately overlaid on a map.  
     [0027] In one embodiment, image information from a sensor platform (not shown) provides engineering sense data (ESD), e.g., global positioning system (GPS) information, INS, image scale, attitude, rotation, and the like, that is extracted from the signal received from the platform and provided to the geo-spatial aligning module  202  as well as the database module  204 . Specifically, the ESD information is generated by the ESD generation module  208 . The ESD is used as an initial scene identifier and sensor point-of-view indicator. As such, the ESD is coupled to the reference database module  204  and used to recall database information that is relevant to the current sensor video imagery. Moreover, the ESD can be used to maintain coarse alignment between subsequent video frames over regions of the scene where there is little or no image texture that can be used to accurately align the mosaic with the reference image.  
     [0028] More specifically, the ESD that is supplied from the sensor platform along with the video is generally encoded and requires decoding to produce useful information for the geo-spatial aligning module  202  and the reference database module  204 . Using the ESD generation module  208 , the ESD is extracted or otherwise decoded from the signal produced by the camera platform to define a camera model (position and attitude) with respect to the reference database. Of course, this does not mean that the camera platform and system can not be collocated, i.e., as in a hand held system with a built in sensor, but means merely that the position and attitude information of the current view of the camera is necessary.  
     [0029] Given that ESD, on its own, can not be reliably utilized to associate objects seen in videos (i.e., sensor imagery) to their corresponding geo-locations, the present invention utilizes the precision in localization afforded by the alignment of the rich visual attributes typically available in video imagery to achieve exceptional alignment rather than use ESD alone. For example, in aerial surveillance scenarios, often a reference image database in geo-coordinates along with the associated DEM maps and annotations is readily available. Using the camera model, reference imagery is recalled from the reference image database. Specifically, given the camera&#39;s general position and attitude, the database interface recalls imagery (one or more reference images or portions of reference images) from the reference database that pertains to that particular view of the scene. Since the reference images generally are not taken from the exact same perspective as the current camera perspective, the camera model is used to apply a perspective transformation (i.e., the reference images are warped) to create a set of synthetic reference images from the perspective of the camera.  
     [0030] The reference database module  204  contains a geo-spatial feature database  210 , a reference image database  212 , and a database search engine  214 . The geo-spatial feature database  210  generally contains feature and annotation information regarding various features of the images within the image database  212 . The image database  212  contains images (which may include mosaics) of a scene. The two databases are coupled to one another through the database search engine  214  such that features contained in the images of the image database  212  have corresponding annotations in the feature database  210 . Since the relationship between the annotation/feature information and the reference images is known, the annotation/feature information can be aligned with the video images using the same parametric transformation that is derived to align the reference images to the video mosaic.  
     [0031] The database search engine  214  uses the ESD to select a reference image or a portion of a reference image in the reference image database  204  that most closely approximates the scene contained in the video. If multiple reference images of that scene are contained in the reference image database  212 , the engine  214  will select the reference image having a viewpoint that most closely approximates the viewpoint of the camera producing the current video. The selected reference image is coupled to the geo-spatial aligning module  202 .  
     [0032] The geo-spatial aligning module  202  contains a coarse alignment block  216 , a synthetic view generation block  218 , a tracking block  220  and a fine alignment block  222 . The synthetic view generation block  218  uses the ESD to warp a reference image to approximate the viewpoint of the camera generating the current video that forms the video mosaic. These synthetic images form an initial hypothesis for the geo-location of interest that is depicted in the current video data. The initial hypothesis is typically a section of the reference imagery warped and transformed so that it approximates the visual appearance of the relevant locale from the viewpoint specified by the ESD.  
     [0033] The alignment process for aligning the synthetic view of the reference image with the input imagery (e.g., the video mosaic produced by the video mosaic generation module  200 , the video frames themselves that are alternatively coupled from the input to the geo-spatial aligning module  202  or some other source of input imagery) is accomplished using two steps. A first step, performed in the coarse alignment block  216 , coarsely indexes the video mosaic and the synthetic reference image to an accuracy of a few pixels. A second step, performed by the fine alignment block  222 , accomplishes fine alignment to accurately register the synthetic reference image and video mosaic with a sub-pixel alignment accuracy without performing any camera calibration. The fine alignment block  222  achieves a sub-pixel alignment between the images. The output of the geo-spatial alignment module  202  is a parametric transformation that defines the relative positions of the reference information and the video mosaic. This parametric transformation is then used to align the reference information with the video such that the annotation/features information from the feature database  210  are overlaid upon the video or the video can be overlaid upon the reference images or both. In essence, accurate localization of the camera position with respect to the geo-spatial coordinate system is accomplished using the video content.  
     [0034] Finally, the tracking block  220  updates the current estimate of sensor attitude and position based upon results of matching the sensor image to the reference information. As such, the sensor model is updated to accurately position the sensor in the coordinate system of the reference information. This updated information is used to generate new reference images to support matching based upon new estimates of sensor position and attitude and the whole process is iterated to achieve exceptional alignment accuracy. Consequently, once initial alignment is achieved and tracking commenced, the geo-spatial alignment module may not be used to compute the parametric transform for every new frame of video information. For example, fully computing the parametric transform may only be required every thirty frames (i.e., once per second). Once tracking is achieved, the indexing block  216  and/or the fine alignment block  222  could be bypassed for a number of video frames. The alignment parameters can generally be estimated using frame-to-frame motion such that the alignment parameters need only be computed infrequently. A method and apparatus for performing geo-spatial registration is disclosed in commonly assigned U.S. Pat. No. 6,512,857 B1, issued Jan. 28, 2003, and is incorporated herein by reference.  
     [0035] Once the images are stored and correlated with geodetic position coordinates, the coordinated images can now be used in accordance with the methods as disclosed below. Specifically, these images are used for overlaying of geo-spatial maps/images of arbitrary shapes within a region of interest.  
     [0036] Specifically, FIG. 3 depicts a method  300  for overlaying geo-spatial maps/images of arbitrary shapes within a geographical region. To better understand the invention, the reader is encouraged to collectively refer to FIGS.  3 , and  5 - 11  as method  300  is described below.  
     [0037] The method  300  begins at step  302  and proceeds to step  304 . At step  304 , the method  300  renders a geometric model of a geographical region. For example, FIG. 5 illustratively depicts this geometric model as a model of the earth  500  and is also referred to hereinafter as “G1”. The geographic rendition  500  comprises latitudinal lines  502  and longitudinal lines  504 . Lines  502  and  504  form a grid over the entire geographic rendition  500 .  
     [0038] In addition, a texture corresponding to the image of the area rendered by the geometric model (e.g., an image of the earth as viewed from space) is obtained. For example, FIG. 6 depicts a texture that is an image of the earth  600  (also referred to hereinafter as “T1”). A texture in computer graphics consists of texels (texture elements) which represent the smallest graphical elements in two-dimensional (2-D) texture mapping to “wallpaper” a three-dimensional (3-D) object to create the impression of a textured surface.  
     [0039] At step  304 , the texture  600  is mapped to the geometric model  500 . The end result is a textured rendition of the earth  700  which shows the topology of the earth as depicted in FIG. 7. The textured rendition  700  serves as a starting point and is an initial background layer of the present invention. The initial background layer is also referred to as “Layer 1” herein. Layer 1 is the first layer generated by performing step  304  using the Equ. 1 (described with further detail below).  
     [0040] Layer 1 is computed in accordance with:  
     Layer 1 =OP 1( G 1)+ OP 2( T 1),  (Equ. 1) 
     [0041] where the function OP1(arg) renders an uncolored, untextured geometry specified in the arg, (where G1 is a model of the earth); and OP2(arg) textures the last defined geometry using a texture specified in the arg, (where T1 is an image of the earth viewed from space). OP2(T1) applies texels from image T1 to the uncolored geometry OP1(G1).  
     [0042] Although the exemplary combination of rendered image  500  and textured image  600  serve to produce textured rendition  700  which serves as Layer 1 of the invention, this is for illustrative purposes only. A person skilled in the art appreciates that Layer 1 may be any geographical area and that the geographical area is not limited to the size of the earth. For example, Layer 1 may be a country, a state, a county, a city, a township, and so on.  
     [0043] In order to provide a more detailed image than that provided by rendered textured image  700 , the geographical region or region of interest can be made smaller than that encompassed by the rendered textured image  700 . The method  300  provides optional steps  306  and  308  for the purpose of providing a more detailed view when desired. As such, neither of these respective steps is necessary to practice the invention and is explained for illustrative purposes only.  
     [0044] At optional step  306 , the method  300  renders a geo-polygon of a geographical region smaller than the previously rendered geographical region G1  500 . A geo-polygon is a three-dimensional patch of the earth&#39;s surface, defined as a set of vertices which have latitude, longitude, and a constant altitude. The geo-polygon consists of an arbitrary shaped triangulated surface conforming to the curvature of the earth at some altitude with one or more textures applied over its extents. The opacity, altitude, applied textures, and shape of geo-polygons can be dynamically altered. Any standard that provides latitude, longitude, and altitude may be used in accordance with the invention, e.g., the WGS-84 or KKJ standard model of the earth.  
     [0045] For example, optional step  306  renders a geo-polygon of a country G2  800  as shown in FIG. 8 (referred to with greater detail below). The rendering process occurs similarly to the rendering described above with respect to G1 and for brevity will not be repeated. In addition, method  300  obtains a texture T2 that can be applied to the geometric model of the country G2.  
     [0046] At step  306 , a texture T2 is mapped to the rendered image G2 and forms what is referred to hereafter as a “Layer 2” image. Layer 2 is the second layer generated by performing step  306  using Equ. 2 (described with further detail below).  
     [0047]FIG. 8 depicts the Layer 2 image  800  and a portion of the Layer 1 image  700 . FIG. 8 depicts the Layer 2 image  800  as already rendered and textured in accordance with step  306 . Layer 1  700  serves as a background with respect to Layer 2  800 . For simplicity, Layer 1  700  is depicted as the darkened area outside of Layer 2  800 . At step  306  the method renders and textures a map of the sub-region in accordance with:  
     Layer 2 =OP 1( G 2)+ OP 2( T 2)  (Equ. 2) 
     [0048] where the function OP1(arg) renders an uncolored, untextured geometry specified in the arg, (where G2 is a geo-polygon of a country corresponding to T2); and OP2(arg) textures the last defined geometry using texture specified in the arg, (where T2 is an image of the country, e.g., a medium resolution map of the country). OP2(T2) applies texels to the uncolored geo-polygon OP1 (G2). The map T2 depicts a greater degree of detail than the image depicted in step  304 . For example, the map T2 depicts items such as major cities, highways, and state roads.  
     [0049] At optional step  308 , the method  300  renders a geo-polygon of a geographical region smaller than the previously rendered geographical region G2  800 . For example, optional step  308  renders a geo-polygon of a city G3  900  as shown in FIG. 9 (referred to with greater detail below). The rendering process occurs similarly to the rendering described above with respect to G1 and G2 and for brevity will not be repeated. In addition, step  308  applies a texture T3 (as similarly described above with respect to T1 and T2) of the area rendered by the geo-polygon of the city G3.  
     [0050] The textured image T3  900  is an image having a higher resolution than the images T1 and T2. For example, T3 can be a high resolution local map depicting buildings, roads, and other points of interest.  
     [0051] At step  308 , the texture T3 is mapped to the rendered image G3 and forms what is referred to hereafter as a “Layer 3” image. Layer 3 is optional and is a third layer generated by performing step  308  using the Equ. 3 (described with further detail below).  
     [0052]FIG. 9 depicts the Layer 3 image  900  and a background layer  902 . The background layer is a combination of Layer 1 and Layer 2, and is the background with respect to Layer 3.  
     [0053] The Layer 3 image is acquired by rendering and texturing in accordance with the following:  
     Layer 3 =OP 1( G 3)+ OP 2( T 3)  (Equ. 3) 
     [0054] where the function OP1(arg) renders an uncolored, untextured geometry specified in the arg. (where G3 is a geo-polygon of a city corresponding to T3), and OP2(arg) textures the last defined geometry using texture specified in the arg, (where T3 is a very high resolution image of the city, e.g., an aerial, satellite, or other sensor image). OP2(T3) applies texels to the uncolored geo-polygon OP1(G3).  
     [0055] Steps  304 ,  306 , and  308  are preprocessing steps used to generate one or more geo-polygon of respective textured regions (textured regions of interest). As indicated above, steps  306  and  308  are optional steps that can be applied depending upon the level of resolution desired by a user and/or the availability of these texture images. Although several layers of textured regions of interest are disclosed above, the present invention is not so limited. Specifically, any number of preprocessing steps  304 ,  306 , and  308  of the present invention can be implemented.  
     [0056] At step  310 , a user begins the actual selection of an arbitrary defined region for conversion into a 3D geo-polygon from the 2D user selected area. The user may use any type of device (e.g., a mouse, joystick, keypad, touchscreen, or wand) for selecting (a.k.a. “painting”) the desired viewing area. Generally, a 3D geo-polygon is created calculating every point on the 2D outline into the ellipsoidal representation of the earth. This is accomplished by extending a ray from every point on the 2D outline into the ellipsoidal earth, and finding the latitude and longitude of the point of intersection of the ray with the surface of the ellipsoidal earth. Thus, a set of latitudes and longitudes is computed from the 2D outline. This defines the vertices of a 3D geo-polygon which is saved in arg. Alternately, a brush footprint, which may be of arbitrary shape, may be intersected with the ellipsoidal earth. This generates a set of latitudes and longitudes per brush intersection, which are again used as vertices of a 3D geo-polygon. The selection of the arbitrary defined region is defined in accordance with:  
     OP5(G4(i))  (Equ. 4) 
     [0057] where OP5 computes a 3D geo-polygon from a 2D outline drawn on the screen; and G4  represents a set of geo-polygons and the combination of these geo-polygons defines an arbitrary shaped textures. G4(i) also represents the immediate selected position (illustratively by the user input device, e.g., a mouse or joystick) for association with a set of geo-polygons that are used to determine the arbitrary defined region. As such, G4(i) is indicative of an arbitrary shaped region or geo-polygon for association with the arbitrary defined region.  
     [0058] At step  312 , other pixels/regions are selected for association with the already existing arbitrary shaped region(s)/geo-polygon(s) within the arbitrarily shaped region. The addition of other arbitrary shaped pixel(s)/region(s) is performed in accordance with:  
     Add G4(i) to G4  (Equ. 5) 
     [0059] where G4(i) represents a currently selected pixel or region for addition to the set of geo-polygons G4 which define the arbitrary shaped region.  
     [0060] At step  314 , the method  300  highlights the selected area by defining the arbitrary shape of the region and storing the entire image (both the arbitrary shape and the background) as a binary mask. Ones are indicative of the presence of the arbitrary shaped region and zeroes are indicative of the background (i.e., the image outside of the arbitrary shaped region).  
     [0061] In accordance with steps  312  and  314 , FIG. 10 depicts an outline of an arbitrary defined region  1020  selected within a desired geographical area. Illustratively, FIG. 10 depicts the desired geographical area as a very high resolution image T4  1010 . Method  300  performs step  314  in accordance with:  
     OP3(G4(i))  (Equ. 6) 
     [0062] where the function OP3(arg) draws the geometry in offscreen-mode and saves the resulting image as a binary mask with ones and zeros. Ones indicate where geometry was present and zeroes indicate background where no geometry was drawn. G4(i) is each respective geo-polygon for association with the arbitrary defined region.  
     [0063] At step  316 , the method  300  applies texels up to and including where the last OP3(arg) function is performed, i.e., where there is the masked arbitrary shaped region defined by Equ. 6. Specifically, step  316  fills in texels within the masked region resulting in a higher resolution image (e.g., a satellite image or aerial photo) within the masked region (the arbitrary defined region) than the resolution of the image outside of the arbitrary defined region. Step  316  is performed in accordance with:  
     OP4(T4, G4(i))  (Equ. 7) 
     [0064] where the OP4(Targ, Garg) function blends the masked drawing of textured geometry. Fills in texels only where the mask resulting from the last OP3(Garg) is one. Subsequently, blending the resulting image with the image generated from the last OP1 or OP2 operation. The final product of this is the texels of Targ blended into pre-rendered geometry only where Garg geometry would have been rendered.  
     [0065] Illustratively, FIG. 11 depicts a “blended textured image” resulting from Equ. 7 having a textured background image with an arbitrary shape image. Specifically, FIG. 11 shows the acquisition of an image  1110  within the arbitrary defined region where the image  1110  has a higher resolution image than the background layer  1100 . The background layer  1110  comprises a number of layers dependent upon the desired viewing area. Illustratively, the background layer  1110  comprises Layer 1, Layer 2, and Layer 3, as explained above.  
     [0066] At step  318 , the method queries whether there are other points for selection into the arbitrary shaped region. If answered affirmatively, the method  300  proceeds, along path  320 , to step  310  for the insertion of more geo-polygons into the arbitrary shaped region. If answered negatively, the method  300  ends at step  322 .  
     [0067] The above method  300  describes an illustrative embodiment of a method of selecting an arbitrary defined region in accordance with the invention. This method may also be referred to as a “painting” method.  
     [0068]FIG. 4 depicts another illustrative method of the invention. Specifically, FIG. 4 depicts an interactive viewing method  400 . In one embodiment, the interactive viewing method  400  can utilize the information from method  300  (i.e., the 3D arbitrary defined region acquired from method  300 ). For example, after the method  300  has obtained a 3D arbitrary defined region, the interactive method  400  can change the perspective viewing angle of the arbitrary defined region.  
     [0069] Method  400  contains steps similar to steps described with respect to method  300 . As such, reference will be made to a described step of method  300  when explaining a corresponding step in method  400 .  
     [0070] Again, referring to FIG. 4, method  400  allows a user to alter the perspective view of the previously painted arbitrary shaped area. As already explained, the interactive viewing method  400  is preceded by the interactive painting method  300 . In other words, method  400  occurs after the “ending” step  322  of method  300 .  
     [0071] The method  400  begins at step  402  and proceeds to step  304 . The operation of the functions performed in steps  304 ,  306 , and  308  have already been explained with respect to method  300  of FIG. 3 and for brevity will not be repeated. Steps  304 ,  306 ,  308  serve to define a background layer with respect to the arbitrary defined region. As explained with respect to method  300 , steps  306  and  308  are optional steps which are implemented when there is a desire to view a smaller geographical region than originally obtained. As such, interactive viewing method  400  may contain more or less steps “layer creating steps” than steps  304 ,  306 , and  308 .  
     [0072] After proceeding through step  304  and optional steps  306 , and  308 , the method  400  proceeds to step  404 . At step  404 , the method  400  defines a 3D area as the entire set of pixel(s)/region(s) within an arbitrary defined region (e.g., the arbitrary region ascertained from method  300 ). Method  400  defines the 3D area within an iterative loop:  
     where  i= 1 to length( G 4 )  (Equ. 8) 
     [0073] where i represents an initial pixel/region within the arbitrary defined region and the function length(G4 ) represents the textured set of geo-polygons within the arbitrary defined region.  
     [0074] At step  314 , method  400  draws the arbitrary defined region and stores the entire image (both the arbitrary shape and the background) as a binary mask, as similarly described with respect to Equ. 6.  
     [0075] The method  400  proceeds to step  316  where method  400  fills the arbitrary defined region with texels and blends the result with a previously rendered image (i.e., a background image, e.g., Layer 1, Layer 2, and Layer 3) as explained above with respect to Equ. 7. However, step  316  as applied in method  400  allows viewing of the arbitrary defined region from the perspective of the pixel/region selected in step  314  of method  400 . FIG. 11 depicts a perspective view of an arbitrary defined image  1110  blended with the previously rendered background image  1100  (i.e., Layer 1, Layer 2, and Layer 3).  
     [0076] Thereafter, the method  400  proceeds along path  408  and forms an iterative loop including steps  404 ,  314 , and  316  whereby each of the geo-polygons within the arbitrary defined region is available for selection of pixel/region within the arbitrary defined region.  
     [0077] The method proceeds to step  406 , where a user can optionally select (e.g., using a pointing device) another perspective view within the arbitrary shape, e.g., “bird&#39;s eye view” or eye level. However, to achieve that it requires a shift in the viewing angle of the geo-polygons. As such, method  400  proceeds along path optional path  410  towards step  304 , where method  400  renders the background layer (i.e., Layer 1) for re-computation of the geo-polygons within the arbitrary defined region. Thereafter the method  400  proceed as discussed above.  
     [0078] Although the invention has been described with respect to the association of maps, satellite images, and photos with a geographical location, the above description is not intended in any way to limit the scope of the invention. Namely, an arbitrarily created marker or indicator  1120  can be selectively placed on the blended textured image.  
     [0079] For example, an indicator  1120  (e.g., an arrow) may be associated with a geographical location. As a user changes the perspective of the image, the perspective of the arrow changes accordingly. For example, an arrow may be associated with an image to point towards a building. If the desired perspective is behind the arrow then the user will view the tail end of the arrow. If a different perspective is desired then (e.g., a “bird eye view”) a user has a perspective looking down upon the arrow.  
     [0080] While the foregoing is directed to illustrative embodiments of the invention, other and further embodiments of the invention may be discussed without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.