Package: coin-clip
Coin-CLIP 🪙 : Enhancing Coin Image Retrieval with CLIP[中文]| [English]Coin-CLIPbreezedeus/coin-clip-vit-base-patch32is built upon
OpenAI'sCLIP(ViT-B/32)model and fine-tuned on
a dataset of more than340,000coin images using contrastive learning techniques. This specialized model is designed to significantly improve feature extraction for coin images, leading to more accurate image-based search capabilities. Coin-CLIP combines the power of Visual Transformer (ViT) with CLIP's multimodal learning capabilities, specifically tailored for the numismatic domain.Key Features:State-of-the-art coin image retrieval;Enhanced feature extraction for numismatic images;Seamless integration with CLIP's multimodal learning.To further simplify the use of theCoin-CLIPmodel, this project provides tools for quickly building a coin image retrieval engine.Comparison: Coin-CLIP vs. CLIPExample 1 (Left: Coin-CLIP; Right: CLIP)Example 2 (Left: Coin-CLIP; Right: CLIP)More ExamplesmoreExample 3 (Left: Coin-CLIP; Right: CLIP)Example 4 (Left: Coin-CLIP; Right: CLIP)Example 5 (Left: Coin-CLIP; Right: CLIP)Example 6 (Left: Coin-CLIP; Right: CLIP)Installpipinstallcoin_clipUsageCode ExamplesExtract Feature Vectors from Coin Imagesfromcoin_clipimportCoinClip# Automatically download the model from Huggingfacemodel=CoinClip(model_name='breezedeus/coin-clip-vit-base-patch32')images=['examples/10_back.jpg','examples/16_back.jpg']img_feats,success_ids=model.get_image_features(images)print(img_feats.shape)# --> (2, 512)⚠️Note:The above code automatically downloads thebreezedeus/coin-clip-vit-base-patch32model from Huggingface.
If you cannot download automatically, please manually download the model locally, and then initializeCoinClipby specifying the local directory of the model through themodel_nameparameter, likemodel_name='path/to/coin-clip-vit-base-patch32'.Command line toolsBuilding a Vector Retrieval Enginecoin-clip build-dbcan be used to build a vector search engine. It extracts features from all coin images 🪙 in a specified directory and builds a ChromaDB vector search engine.$coin-clipbuild-db-h
Usage:coin-clipbuild-db[OPTIONS]Extractvectorsfromacandidateimagesetandbuildasearchenginebasedonit.

Options:-m,--model-nameTEXTModelName;eitherlocalpathorhuggingfacemodelname[default:breezedeus/coin-clip-vit-base-patch32]-d,--deviceTEXT['cpu','cuda'];Either'cpu'or'gpu',orspecifyaspecificGPUlike'cuda:0'.Defaultis'cpu'.[default:cpu]-i,--input-image-dirTEXTFolderwithCoinImagestobeindexed.[required]-o,--output-db-dirTEXTFolderwherethebuiltsearchengineisstored.[default:./coin_clip_chroma.db]-h,--helpShowthismessageandexit.For instance,$coin-clipbuild-db-iexamples-ocoin_clip_chroma.dbQueryingAfter building the vector search engine with the above command, you can use thecoin-clip retrievecommand to retrieve the coin images 🪙 most similar to a specified coin image.$coin-clipretrieve-h
Usage:coin-clipretrieve[OPTIONS]Retrieveimagesfromthesearchengine,basedonthequeryimage.

Options:-m,--model-nameTEXTModelName;eitherlocalpathorhuggingfacemodelname[default:breezedeus/coin-clip-vit-base-patch32]-d,--deviceTEXT['cpu','cuda'];Either'cpu'or'gpu',orspecifyaspecificGPUlike'cuda:0'.Defaultis'cpu'.[default:cpu]--db-dirTEXTFolderwherethebuiltsearchengineisstored.[default:./coin_clip_chroma.db]-i,--image-fpTEXTImagePathtoretrieve[required]-h,--helpShowthismessageandexit.For instance,$coin-clipretrieve--db-dircoin_clip_chroma.db-iexamples/10_back.jpg