Package: image-searcher
ImageSearcherLeveraging CLIP to perform image search on personal picturesThis repository implements an Image Search engine on local photos powered by the CLIP model.
It is surprisingly precise and is able to find images given complex queries. For more information, refer to
the Medium blogposthere.The added functionality of classifying pictures depending on the persons portrayed is implemented
with theface_recognitionlibrary. Several filters are also available, enabling you to find your
group pictures, screenshots, etc...SetupIn a Python 3.8+ virtual environment, install either from PIP or from source:Installation from the PIP package:pipinstallimage-searcher

pipinstallface_recognition# Optional to enable face featurespipinstallflaskflask_cors# Optional to enable a flask apiInstallation from sourcepipinstall-rdev_requirements.txt

pipinstallface_recognition# Optional to enable face featurespipinstallflaskflask_cors# Optional to enable a flask apiTroubleshooting: If problems are encountered building wheels for dlib during the face_recognition installation, make sure to install thepython3.8-devpackage (respectivelypython3.x-dev) and recreate the virtual environment from scratch with the aforementionned command
once it is installed.UsageCurrently, the usage is as follows. The library first computes the embeddings of all images one by one, and stores them in
a picked dictionary for further reference. To compute and store information about the persons in
the picture, enable theinclude_facesflag (note that it makes the indexing process up to 10x slower).fromimage_searcherimportSearchsearcher=Search(image_dir_path="/home/manu/perso/ImageSearcher/data/",traverse=True,include_faces=False)Once this process has been done once, through Python, the library is used as such:fromimage_searcherimportSearchsearcher=Search(image_dir_path="/home/manu/perso/ImageSearcher/data/",traverse=True,include_faces=False)# Option 1: Pythonic APIfromPILimportImageranked_images=searcher.rank_images("A photo of a bird.",n=5)forimageinranked_images:Image.open(image.image_path).convert('RGB').show()# Option 2: Launch Flask api from codefromimage_searcher.apiimportrunrun(searcher=searcher)Using tags in the queryAdding tags at the end of the query (example:A bird singing #photo) will filter the search based on the tag list.
Supported tags for the moment are:#{category}: Amongst "screenshot", "drawing", "photo", "schema", "selfie"#groups: Group pictures (more than 5 people)To come is support for:#dates: Filtering based on the time periodRunning through the API for efficient useAfter having indexed the images of interest, a Flask API can be used to load models once and then search efficiently.Specify a Config YAML file:image_dir_path:/home/manu/Downloads/facebook_logs/messages/inbox/save_path:/home/manu/traverse:trueinclude_faces:truereindex:falsen:42port:host:debug:threaded:Start a server:fromimage_searcher.apiimportrun# Option 1: Through a config filerun(config_path="path_to_config_file.yml")# Option 2: Through an instanciated Search objectfromimage_searcherimportSearchrun(searcher=Search(image_dir_path="/home/manu/perso/ImageSearcher/data/",traverse=True,include_faces=False))A gunicorn process can also be launched locally with:gunicorn"api.run_flask_gunicorn:create_app('path_to_config_file.yml')"\--nameimage_searcher\--bind0.0.0.0:${GUNICORN_PORT:-5000}\--worker-tmp-dir/dev/shm\--workers=${GUNICORN_WORKERS:-2}\--threads=${GUNICORN_THREADS:-4}\--worker-class=gthread\--log-level=info\--log-file'-'\--timeout30Note: Adapt the timeout parameter (in seconds) if a lot of new images are being indexed/Query it:By opening in a browser the webpage with the demo search enginesearch.html.Through the API endpoint online:http://127.0.0.1:5000/get_best_images?q=a+photo+of+a+birdIn Python:importrequestsimportjsonimporturllib.parsequery="a photo of a bird"r=requests.get(f"http://127.0.0.1:5000/get_best_images?q={urllib.parse.quote(query)}")print(json.loads(r.content)["results"])TipsUsing this tool with vacation photos, or Messenger and Whatsapp photo archives leads to rediscovering
old photos and is amazing at locating long lost ones.TestsRun the tests withpython-munittestand lint with:pylintimage_searcherContributingThis repo is a work in progress that has recently been started. As is, it computes about 10 images per second during the initial indexing phase, then is almost instantaneous during the querying phase.Feature requests and contributions are welcomed. Improvements to the Search Web interface would also
be greatly appreciated !Todo listSimplify and robustify the Search class instanciation:Check indexation arguments are compatible with pre-loaded fileStore indexation arguments in pre-loaded file and give option to index new pictures with these optionsAdd the option to index for faces on previously CLIP indexed imagesSpeed:Parallel indexation / dynamic batching based on image sizeData loader before indexationOptimized vector computation with optimized engine (FAISS)Features:Image auto-tagging (screenshot, drawing, photo, nature, group picture, selfie, etc)Image deduplication (perceptual hashing)Embedding files:Integrate with local version control (git-lfs ?)Frontend:Overall UX and design changesEnable Image uploadDeployment:Dockerize and orchestrate containers (image uploader, storage, indexation pipeline, inference)