Weekend Hack: Building an Image Recognition bot for Telegram using Python | by Alain Perkaz | Towards Data Science

The goal of this post is to build a Telegram bot with Python that performs image recognition using neural networks. The post is divided into three main sections: introduction, technology choices, and step-by-step setup.
This post is the fourth instance of the Weekend Hack series, a series of short development stories about the exploration of new concepts. Focused on learning by doing, step-by-step instructions are provided.
Automated image recognition (ex. object recognition and image classification) can provide immense value for businesses. If you run a forum, dating app, or any platform where users upload content, automating the recognition of images can be critical. The benefits are twofold:
On the one hand, it provides value to the end-users, allowing them to perform searches over the images by classified topic while avoiding the tedious task of manual classification. A practical example of this is the Unsplash search functionality, which searches high-quality images by topic.
On the other hand, automated image recognition also provides value to the system owners, as it allows to filter out images that contain nudity, for example. In short, it allows improving the quality of the provided services in an automated manner. Good right?
As we see, applications are endless, and benefits clear. The main question remains, which solution to choose for this task? Plenty of third party solutions exists, along with the possibility of building your own image recognition pipeline from open-source or proprietary software. On the next section, we will explore the different alternatives.
As hinted above, there is a myriad of providers that provide image and video recognition using APIs: Google Vision, Amazon Rekognition, Clarify... One of the main questions is, will my data be secure? As Google says in its Data Usage FAQs:
When you send an image to Cloud Vision API, we must store that image for a short period of time in order to perform the analysis and return the results to you. The stored image is typically deleted in a few hours. Google also temporarily logs some metadata about your Vision API requests (such as the time the request was received and the size of the request) to improve our service and combat abuse.
“tipically deleted”... what does tipically means?
If data ownership and privacy are concerns for you, there is also the option to build your own image recognition pipeline leveraging existing technologies. There are also a bunch of alternatives in open-source: TensorFlow, Darknet, MLpack, Keras... Which allow you to better control where the image data ownership lays.
In this tutorial, we will focus on building our own image classifier and object recognition bot, using Darnet. We will set up a Telegram bot written in Python hosted in Glitch to interface with the image classifier.
The user flow will be the following: select a picture and send it to our Telegram bot. It will classify and perform object recognition over it, and send us the results. Easy right?
Alright, let's get into it!
In order to not repeat myself, I will recommend you to follow the step-by-step instructions given in the post below. Follow the first two steps, and modify the app name to your liking. Don't focus too much on the bot handles, as we will add a specific handle, later on, to act on received pictures.
medium.com
In order to analyze the images send to the bot, first we need to install and build Darknet. All the installation commands are available on the install.sh file in the Glitch project.
To input the commands, in our Glitch project select Tools > Full Page Console.
To install Darknet, navigate to .data directory within your Glitch app. This is crucial, as this directory will be persisted when your Glitch app goes to sleep!
cd .datagit clone https://github.com/pjreddie/darknet.gitcd darknetmake
It is important to mention that while you can train your own models (and should, depending on the use case), it is usually a quite computationally expensive operation. Considering that we will be running our bot from a Glitch instance with very limited power and space (1 CPU, 512 RAM, 200 MB storage), training the models is quite unfeasible.
For our case, using pre-trained model weights is the best solution. We will use existing weights generated from previous trainings, allowing us to quickly get up to speed. We will download two weights files, darknet19.weights for the image classification, and yolov3-tiny.weights for the object recognition.
# execute in the ./darkent directorywget https://pjreddie.com/media/files/darknet19.weightswget https://pjreddie.com/media/files/yolov3-tiny.weights
Note that we have selected rather small weight files, due to the limitations of space and CPU power available in Glitch at the moment. Feel free to try out other weights (image classification, object detection) if you are running this in another more powerful environment. You will also get better results!
Perfect, now that we have our bot skeleton ready, and Darknet installed, we can wire it up. I will not explain each line of code, for the full commented code you can head to the Glitch project.
Below there is the code extract of the main handler (note that it uses helper functions). It is triggered every time a new picture is sent to the bot.
@bot.message_handler(content_types=['photo'])def handle(message):  # extract the image name for further operations  image_name = save_image_from_message(message)    # execute object recognition  object_recognition_image(image_name)  # send object recognition results  bot.send_photo(message.chat.id, open('.data/darknet/predictions.jpg','rb'), 'Identified objects')    # execute image classification  classification_list_result = classify_image(image_name)    # send classification results  output = 'The image classifies as:\n'  for result in classification_list_result:    output += result  output += '\n🚀 Gimme more pics! 🚀'      bot.reply_to(message, output)    # remove picture from server  cleanup_remove_image(image_name);
The full source code is available in:
glitch.com
Perfect, now that we are ready, let's do some testing!
Our bot’s Telegram ID is: @wh_image_classificator_bot, feel free to test it out. The classification and recognition capabilities are limited by the technical restrictions in place, but the results are encouraging.
On-demand image recognition APIs provide unparallel capabilities, but when privacy or offline processing is critical, custom image recognition pipelines are a great alternative.
Note that our example only scratched the surface, and more accurate image recognition can be achieved with other training sets and approaches.
Glitch project → https://glitch.com/~telegram-image-classfication-bot
Darknet docs → https://pjreddie.com/darknet/
Other Weekend Hack posts: