Package: wechaty-meme-bot
Wechaty-Meme-BotPrefaceThis project is supported byWechaty CommunityandInstitute of Software of Chinese Academy of Sciences.PowerPoint Demonstration:https://www.bilibili.com/video/BV1kZ4y1M7F6/Demo Live Video on bilibili:https://www.bilibili.com/video/BV17f4y197ut/My community mentor isHuang, contributor of python-wechaty. I won't make such progress without his support.IntroductionI was required to build a meme bot based onpython-wechaty, which should possess at least following functions:receive & save meme image from specific contactanalyse received meme imageresponse meme image accordingly based on analysis given aboveTo achieve such requirements, I came out with a cross-functional diagram below to assist my development(written in Chinese):Some ConceptsFrontend: Run on user end, be in charge of communicating with python-wechaty-puppet and backend, acting as a middleware.Backend: Run on server end equipping a NVIDIA GPU, be in charge of analyzing meme image and choose response meme based on certain strategy.Directory Layout$tree-L3-I'__pycache__'.
├──LICENSE
├──Makefile
├──README.md
├──backend# backend files│├──chineseocr_lite# modified OCR module││├──Dockerfile
││├──LICENSE
││├──__init__.py
││├──angle_class
││├──config.py
││├──crnn
││├──model.py
││├──models
││├──psenet
││└──utils.py
│├──config.yaml# config file in yaml format│├──conversation# conversation GPT2 model path (~600MB), download from GDrive mentioned before│├──cosine_metric_net.py# definition of CosineMetricNet│├──cosine_train# CosineMetricNet Training scripts││├──dataset.py
││├──metric.py
││└──train_and_eval.py
│├──dataset.py# Common training dataset module│├──feature_extract.py# feature extract module│├──hanlp_wrapper.py# NLP wrapper│├──logs# log dir│├──meme# default dir for meme import││├──classified
││├──others
││└──unclassified
│├──meme_importer.py
│├──ocr_wrapper.py
│├──requirements.txt
│├──response
││├──__init__.py
││├──conversation.py
││├──dispatcher.py
││└──feature.py
│├──spider# custom spider dir, any spiders should derive from BaseSpider││├──BaseSpider.py
││└──FaBiaoQingSpider.py# example spider to crawl FaBiaoQing│├──stopwords.txt# stop words list for NLP tokenizer│├──utils.py# backend public utils│└──web_handler.py# backend Flask module├──frontend
│├──config.py# frontend configuration│├──image# image cache dir│├──logs# log dir│├──main.py
│└──meme_bot.py
├──gdrive.sh# bash to download from GDrive├──image# static image files│├──logo.png
│├──summer2020.svg
│└──wechaty-logo.svg
├──orm.py# orm module├──test.db# SQLite database└──tests# unittests using pytest├──conftest.py├──test_conversation.py├──test_dataset.py└──test_orm.pyDeploy Tutorialgit clone https://github.com/MrZilinXiao/python-wechaty-meme-bot.gitFrontendVia PyPi (Pending...)pip3installwechaty-meme-botManually1.Correctly configure backend settings infrontend/config.yamlgeneral:image_temp_dir:'./image'allow_img_extensions:('.jpg', '.png', '.jpeg', '.gif')backend:# change to your backend serverbackend_upload_url:'http://192.168.10.102:5000/meme/upload'backend_static_url:'http://192.168.10.102:5000/static'2.Run lines below in your shell:exportWECHATY_PUPPET=wechaty-puppet-hostieexportWECHATY_PUPPET_HOSTIE_TOKEN=your-donut-token# replace `your-donut-token` with your wechaty donut tokenmakerun-frontend# if no `make` in your system, try run `pip3 install -r frontend/requirements.txt`, `python3 frontend/main.py`BackendCurrently we only get backend tested on Ubuntu, while frontend possesses cross-platform feature.You may refer toGithub Action Configurationto learn how we deploy backend when you encounter issues.Nvidia-dockerPending...Manuallypip3install-rbackend/requirements.txt
pythonbackend/web_handler.py# this will trigger chineseocrlite compiling processOpen-Source Referencechineseocr_lite: Powerful Chinese OCR module with accurate results and fast inference.HaNLP: Multilingual NLP library for researchers and companies, built on TensorFlow 2.0.Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.GPT2-Chinese: Chinese version of GPT2 training code, using BERT tokenizer.Academic Citation# in backend/cosine_metric_net.py
[1]N. Wojke and A. Bewley, “Deep Cosine Metric Learning for Person Re-identification,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, Mar. 2018, pp. 748–756, doi: 10.1109/WACV.2018.00087.
# GPT2 Original Paper
[2]Radford, Alec, et al. "Language models are unsupervised multitask learners." OpenAI Blog 1.8 (2019): 9.