Package: python-pisces
Why use pisces?This project origins from the needs of image dataset for algorithm training.Image search engines, such as Google, are quite powerful now. They are enough if we could use them.
However, they didn't provide convenient API for us.Pisces usesselenuim, which can work with chromedriver to download the images we need.
Pisces supports these search engines: google, yahoo, bing, baidu(china) and more mainstream engines in the future.Why use pisces?(Chinese)本项目起源于算法训练需要图像数据集的需求.现有的图片搜索引擎,比如Google,相当强大了,能够利用起来,已经可以满足我们的需求. 然而,这些网站并没有提供方便的API让我们得到图片链接.Pisces使用了selenuim, 可调用主流的浏览器下载搜索到的图片. Pisces还整合了其他的搜索引擎的搜索结果:google/yahoo/bing/百度.InstallationFirst, make sure you havechromebrowser installed on your computer.
If not, selenium gives "selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary".If you're working on a server without interface (for example, CentOS), you can also follow the steps below to install it:wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm
yum -y localinstall google-chrome-stable_current_x86_64.rpmUse pip:pip install python-piscesFrom source code:git clone https://github.com/wolfhong/python-pisces.git
cd python-pisces

pip install -r requirements.txt
export PYTHONPATH=./
python scripts/update_chromedriver.py  # download newest chromedriver according to platform
python setup.py installIn addition to chrome, you can also usefirefox. However, you need to do extra settings for firefox. You have to downloadgeckodriverand put it into your$PATH, which helps selenium interacting with firefox browser.Pisces uses chrome as the default browser. You needn't downloadchromedriveryouself because Pisces has built-in ones.Pisces first finds chromedriver (for Chrome) or geckodriver (for Firefox) inpisces/tools/in its package. If not, then finds it in$PATH.Console CommandOnce you have installed Pisces, you can easily use it to search for and download images by keywords:$ pisces -e google -w 8 -n 500 fire "kitchen fire" -o ./outputThe above command will start up chromedriver and then google "fire" and "kitchen fire" with its image search engine,
download images with 8 threads parallelly and then restore these images in "./output" directory.If you're in China, you're recommended to use-e baiduinstand of-e google(default), because of some network problems. (在中国由于一些网络原因，推荐使用参数-e baidu，而不是默认的-e google)Usepisces -hto show the usage:usage: command.py [-h] [-q] [--display] [-e ENGINE] [-w WORKERS] [-n NUMBER][-o OUTPUT_DIR] [-v][keywords [keywords ...]]Use keywords to search for and download images.positional arguments:keywords              keywords to search for imagesoptional arguments:-h, --help            show this help message and exit-q, --quiet           quiet (no output)--display             start up browser with a graphical display, default to no-e ENGINE, --engine ENGINEthe image search engine you want to use, default togoogle. select within [google, bing, yahoo, baidu]-b BROWSER, --browser BROWSERthe browser you have installed, default to chrome.select within [chrome, firefox]-w WORKERS, --workers WORKERSthe number of threads when downloading images, defaultto cpu core count-n NUMBER, --number NUMBERthe max number of images you want to download, defaultto 100-o OUTPUT_DIR, --output_dir OUTPUT_DIRdestination to store downloaded images, default to./output-v, --version         print the version and exitPisces can work on these platforms: windows, linux, mac, and it had beed tested.
If not, you're welcome tofile an issue.More examples:$pisces"kitchen fire""forest fire"$pisces-n500"厨房火灾""森林火灾"$pisces-ebaidu-n500-opath-to-output火灾水灾Code ExamplePisces can also be included in your projects:frompiscesimportPisces# recommended to use `with`:withPisces(quiet=False,headless=False)asclient:output_dir='./output_water/'client.download_by_word('water',output_dir,engine='google',image_count=20)# or call `close()` handly:client=Pisces(quiet=False,headless=True,workers=4,browser='chrome')output_dir='./output_fire/'client.download_by_word('fire',output_dir,engine='baidu',image_count=20)client.close()TipsThe code uses selenuim. More info to seeReadTheDocsIf you're in China, you're recommended to use-e baiduinstand of-e google(default), because of some network problems.AboutPisces is just a tools to search for and download images by keywords, using image search engine such as google, bing, baidu, etc.
I hope it could help you in somewhere.Issue trackerSource CodePyPI