## 2 Color Analysis
### Explicit Incorporation of Position Information  
Inherent to our system, as well as to the work of others such as the system of [PERSON], [PERSON], and [PERSON] [7] or [PERSON] [8], is a behavior that often functions as a bit of a two-edged sword. It is typical after a query to be surprised by images that strongly deviate from expectations. This is because, in some sense, quantization techniques are a generalization of color histogram measurement, which is subject to coincidences where two images that look obviously very different to a human just happen to have similar color distributions. For example, the four sets of images in Figure 3 show the top 12 images from four queries made to our current prototype system on the 1500 image COREL set (the upper left image of each set is the query image). As can be seen in Figure 3, the beach query has a number of responses from the mountains, one of the responses to the horse query comes from the elephant set, and one response from the elephant set comes from the beach set. In some cases, the top 12 responses are all from the correct class, as is the case for the architecture query. However, a basic limitation is that codebook entries are relatively small in dimension (e.g., feature vectors derived from 2 \(\times\) 2 or 4 \(\times\) 4 sub arrays of pixels) as compared to the image size, and do not capture global image structure.  
To incorporate positional information, we have performed experiments with a straightforward approach where the mean XY coordinates of all blocks associated with a particular codebook entry are added to that entry (so the dimension of codebook entries is increased from 6 to 8). Figure 4 shows some database images with the code vector positions marked on the image. The color within each  
Figure 2: Our VQ (LUV Color) _vs._ GMVQ (HSV Color).  
black-bordered box represents the color of the code vector. In order to appropriately weigh each feature (color and position), the two features (the 6-dimensional color feature and the 2-dimensional positional feature) are normalized. The distance between two feature vectors is computed using a weighted norm \(d(x,y)=(x-y)^{\mathrm{t}}W(x-y)\) where \(W\) is a diagonal weight matrix. The weight matrix was chosen empirically after performing tests on various weighing matrices that assigned different weights to the positional features.  
Figure 4: Locations of code vectors within an image.  
Figure 5: VQ with color only _vs._ VQ with color and position.  
Figure 3: Sample results.  
Using the same database and the same set of 210 query images as the previous section, Figure 5 compares our basic method using color only (6 element feature vectors) and color + XY (8 element feature vectors), again using codebooks of size 8. From this graph it can be seen that the use of extended feature vectors gives an overall modest improvement in performance. However, the real benefit of adding XY to the feature vector appears in classes with highly contrasting regions. For example, Figure 6 shows on the left the first 10 images retrieved for one of the dinosaur images with color only and on the right shows the first 10 for color + XY. For this query, the difference is highly significant, as can be seen from the graph on the left in Figure 6; the right shows the average of querying all 100 images in the dinosaur class, where again the difference can be seen as highly significant. In general, this straightforward addition of XY to the feature vectors provides \"insurance\", where it makes little difference for many classes but can yield significant improvements for some. A drawback, however, is the need to choose the relative weighting empirically; we address this issue in the next section.