Content-Based Exclusion Queriesin Keyword-Based Image Retrieval
Eisaku Yoshikawa, Keishi Tajima (Kyoto University)

Exclusion Queries "A -B"
Exclude those including "B" from the results of the query "A".
Use exclusion queries for page retrieval
Use exclusion queries for image retrieval• A popular strategy for implementing keyword-based image retrieval is to
use the keywords proximate to the image, which achieves high precision(and low recall).
• By contrast, exclusion queries for image search based on the proximate
keywords result in low precision (and high recall).
We propose exclusion queries based on image content
rather than textual information.

japan spring scenery 🔎

The results of the query “A -B” are far from the results of the query “A B”.
⇓If wejapan spring scenery -"cherry blossoms" 🔎Appropriate images depend on the query.
⇓exclude some of the results of the query “A” which are close to “A B”

classify queries

We need toto determine correctness of output.

we get the results of the query “A -B”

Queries to classify polysemous words* Mathematical Definition● Exclude images of polysemic words Acorresponding to the meaning of B
● Example: “Jaguar -car”
Queries without B● Exclude images that contain B in the image● Example: “Egypt -pyramid”
Queries where B is not the subject● Exclude images where the subject ofthe image is B
● Example: “Japan spring scenery -cherryblossom”

Queries

Distance

Imagess

Search results

Queries that remove by meta information● Exclude images with B information (e.g. artist)

“A”

“A B”

A=
[a1.png, a2.png, …]
B=[b1.png, b2.png, …]||a1 – B|| = 7||a2 – B|| = 24||a3 – B|| = 13||a4 – B|| = 2||a5 – B|| = 49…

||a1 – B|| = 7||a2 – B|| = 24||a3 – B|| = 13||a4 – B|| = 2||a5 – B|| = 49…

that does not appear directly in the image● Example: “David -Michelangelo”

For the exclusion query “A-B•(3) Calculate distances(4) Determine thresholds(5) Filtering

Schematic of the proposed method

relevant• images that should not be excluded from negative searches•(1) Collect images “A”(2) Collect images “A B”

Examples of each quereisirrelevant• images that should be excluded from negative search results• assumed to be similar to the results of the query “A B”•irrelevant to A
• image that is not desirable as a search result for “A”

Proposed method achieves• the search system will include it in the “A” solution

higher performance
than existing methods (Bing, Google)* Comparison of 61 queries for Bing and 64 queries for Google𝜃
Images in image set A and thresholds
We choose θ where
the separation metrics takes its beak.

CONCLUSION
Considerations

Future works● Negative image search using image content is effective● Since the proposed method has many elements that can be selected or• Negative image search with image content is effective and also
improves the exclusion performance.
● The proposed method cannot achieve high performance if the dataset
has few matching solutions• The query with low performance in this study could be considered adataset problem.

adjusted, consider whether there are more appropriate elements.
• Feature vector extraction for images other than CNN• Mechanism for determining threshold values● Consider what is appropriate as a performance measure• Both precision@10 and MRR are indicators of whether or not there isa top matching solution, and evaluation of reproducibility and diversity
is also desirable for “exclusion queries”.