Qweetgis — QGIS 3 plugin for Twitter
By: Alessandro Cristofori
Tags: Qgis, GIS, Maps, Twitter, Python

Plugin logo designed by Alessia Nawell credits: QGIS — Twitter

The idea of developing a QGIS plugin came from my willingness to refresh my knowledge after having collaborated to the development of a plugin a few years ago. At that time, rather inexperienced, I developed features for a plugin called Roadnet having a very detailed scope, the spatial data management of the Local Street Gazetter according to the British Standard. Back then in 2015, the latest QGIS version was 2.8, plugins were in Python 2 and Qt/PyQt latest version was 4.x. Within that project I deepened my Python knowledge, I learnt how to create Qt UI objects and its processing threading classes QThreadPool and QThread.

Fast Forward 2019, Python 3 is now the standard, QGIS supporting this Python version is now at 3.4 and Qt/PyQt last version is now 5.x, I am a less inexperienced GIS developer and I felt ready to contribute to the QGIS community with a plugin built entirely by myself. My first intention was to write a story on how to implement a general multi-purpose threaded process in QGIS, I eventually ended up building a QGIS plugin which is using a threaded process but is doing something a bit more interesting than geometry validation/reporting. Qweetgis uses the Twitter streaming API with concurrent processing in Python to download and display on a map real time tweets without locking the resources in the main QGIS UI thread. You can watch tweets pop up as points in your map while they are created without having to wait for the download process to finish. If this seems interesting enough for you, please download the plugin, use it in QGIS and keep reading this story. Here is the link for the the GitHub repository. Qweetgis is based on the work of Riccardo Klinger - Geolicious- who developed Geotweet, a twitter plugin for QGIS 2.x.

Prerequisites

Qweetgis uses a Python package called Tweepy that makes the requests to the Twitter streaming API. Standard QGIS Python does not come with Tweepy and it has to be installed separately. There are a lot of guides online that teach you how to do it and there is not a single method that works for every user since operating systems and Python configurations may differ. I installed it on both a Windows 10 and Linux (Ubuntu 18.04) machine. On Ubuntu (where I also have Anaconda) I simply did a $ pip3 install tweepy from the terminal and Tweepy installed into the global environment Python 3 which is the same used in QGIS. On Windows, the solution that worked for me is here.🔴 I did it on a standalone QGIS install, if you have QGIS installed within OSGEO4W this method may brake you Python2 environment 🔴. For QGIS in OSGEO4W I also managed to install Tweepy following this example to set up the OSGEO4W Python to point to Python 3, the command is py3_env . After that, I downloaded the tar.gz Tweepy module from PyPi, extracted it and from the OSGEO4W shell, cd-ing into the extracted Tweepy folder tweepy-3.7.0, I ran the python3 setup.py install .

The other prerequisite is having the four authentication tokens to access the Twitter API, the two Twitter API developer keys and the two user tokens used by OAuth to authorise Qweetgis to use your account. To get the first two keys you will have to sign-up for a developer Twitter API account and create an application, instructions are here. Once you get these two keys you will be able to generate your OAuth access tokens, instructions are here. I know it’s awkward, you install an application and you’d like to use it straight away, without having to subscribe to a Twitter Dev account that you probably will never use if it wasn’t for this plugin. I tried to make the authentication process as seamless as possible, creating an authentication web service providing the OAuth tokens based on my personal DEV keys, stored in a secure server. The problem is that the Twitter API requires DEV keys in every request, these keys must be stored secretly and I couldn’t distribute mine for you to use. The risk is that someone with malicious intents could use these keys to read the tokens of the users subscribed to the plugin and potentially hack plugin users’ Twitter accounts.

Login

Once obtained your four items for authentication, the first time you’ll start Qweetgis after it is installed you will be presented with the following login dialog.

Note: If you are behind a proxy server, Qweetgis could be set to point to it, it has to be configured before logging in, see instructions here.

Qweetgis Login Dialog — testing user credentials

Every time a new session is started the plugin looks into the config/config.json file in the plugin root directory. If credentials are not found you will be prompted to input them. When you click on OK, the inserted tokens are written and stored in the config file and you won’t need to supply them again. You can also test the credentials once inserted before proceeding. Qweetgis will ping Twitter and search for 10 random tweets to check if the connection is up and the API is responsive.

Main Dialog