A Node-RED Twitter-follower-flow
Tags: nodered, javascript, twitterapi, ibmcloud
Published: 2020-03-26 09:18:46

---
title: A Node-RED Twitter-follower-flow
published: true
description: Node-RED is very good for prototyping, that is the reason why it is often used in hackathons. If you are new to Node-RED and you start to develop a Node-RED flow, you normally have following challenges: define own **REST endpoints**, automate the **authentication**, **extract data**.
tags: node-red, javascript, twitterapi, ibmcloud
---

In my last blog post I did an introduction "how use the starter kit for Node-RED on IBM Cloud" and in that new blog post I want to highlight the Node-RED Twitter-follower-flow I developed as an introduction to different topics I frequently face at hackathons. Here is the link to the ["Twitter-follower-flow" GitHub project](https://github.com/thomassuedbroecker/node-red-twitter-follower). 

Node-RED is very good for prototyping, that is the reason why it is often used in hackathons. If you are new to Node-RED and you start to develop a Node-RED flow, you normally have following challenges: 

* **How to ...**

  1. ... define own **REST endpoints** to encapsulate an external API?
  2. ... automate the **authentication** to that external API?
  3. ... **extract data** from the external API?
  4. ... **customize data** and **CRUD** with databases? 

The Node-RED flow of that project has the objective to provide an (little advanced) introduction to the first three topics above.
The CRUD (Create, Read, Update, Delete) topic is not covered here. Visit that [CRUD example](https://github.com/gitjps/Node-RED-Cloudant-CRUD) for more information.

* **The "Twitter-follower-flow" example ...**

  * ... uses open technologies (Node-RED is a Project of the [OpenJS foundation](https://openjsf.org/)

  * ... creates no additional costs 
    * Uses a free runtime on [IBM Cloud](https://ibm.biz/BdqZTm)
    * Uses the free [Twitter developer API](https://developer.twitter.com/en)

  * ... has following technical level:
    * Beginner to intermediate 
    * Needs a basic knowledge of JavaScript and REST

  * ... takes 30 - 45 min to setup the example from scratch
    1. Register on IBM Cloud 
    2. Create a Node-RED instance on IBM Cloud
    3. Register at Twitter for a developer API Account 
    4. Copy the existing Node-RED flow
    5. Configure the flow
    6. Run the flow

The YouTube video gives a 13 min introduction to the _Twitter-follower-flow_.

[![Introduction to the Node-RED flow](https://img.youtube.com/vi/4HvBya4Zhn8/0.jpg)](https://www.youtube.com/watch?v=4HvBya4Zhn8 "Click play on youtube")

## 1. The UseCase

The UseCase **_"Extract the twitter follower list"_** addresses three topics listed above in the Node-RED _Twitter-follower-flow_.

  1. Define own **REST endpoints** to encapsulate Twitter developer API calls.
  2. Automate the **authentication** the two step authentication of the Twitter developer API.
  3. **Extract data** from the Twitter follower list and build an own list.

## 2. Topics related to the Node-RED usage

The _Twitter-follower-flow_ is an concrete example of following topics ...

 * ... the **implementation** of Node-RED REST endpoints.

   1. Create a very basic authentication to protect each Node-RED endpoint
   2. Realize the two step authentication _(also known as two-factor authentication)_ to access the Twitter developer API:
        1. Basic authentication with a key and secret to request a bearer token from Twitter
        2. Use the bearer token for authentication to use the Twitter API 
   3. Get the follower list from Twitter using the bearer token

 * ... the **usage** of the implemented REST endpoints to automate following sequence to get the Twitter follower.

   1. Get the **authorization** bearer token from Twitter to access the Twitter API
   2. Get all followers from Twitter using a **bearer token** ( _the challenge is: how implement the paging thought the followers list from Twitter_ )
   3. Extract the names of the follower from the follower list

 * ... the **work** with following Nodes in the Node-RED _Twitter-follower-flow_.

   1. Function _(Do small programming in javascript.)_
   2. Inject _(Start a flow in Node-RED.)_
   3. HTTP _in_ _(Creates an HTTP end-point for creating web services.)_
   4. HTTP _request_ _(Sends HTTP requests and returns the response.)_
   5. HTTP _response_ _(Sends responses back to requests received from an HTTP Input node.)_
   6. Base64 (A function that converts the chosen property (default msg.payload) to and from base64 format.)
   7. JSON _(Converts between a JSON string and its JavaScript object representation, in either direction.)_
   8. Switch _(Define a decision how to route the payload. Route messages based on their property values or sequence position.)_

 * ... the **usage** of flow variables to exchange values between functions in the _Twitter-follower-flow_.

---

## 3. Setup the Twitter-follower-flow example 

That section contains six major steps to setup the _Twitter-follower-flow_ example on IBM Cloud. 

1. Setup Twitter developer application
2. Create a Node-RED instance on IBM Cloud
3. Install a additional Node to the Node-RED instance
4. Import the Node-RED flow
5. Configure the Node-RED flow
6. Introduction to the Node-RED flow (13 min video)

In the image you see the full flow.

![](https://suedbroecker.files.wordpress.com/2020/03/twitter-follower-flow.png)

---

### Step 1: Setup Twitter developer application

You need a Twitter account and a Twitter developer application.
With your Twitter account can register for the **Twitter developer API** and create a Twitter developer application. That application provides the needed credentials to access the Twitter API, which is used in that example.

#### a. [Link to add a Twitter developer application](https://developer.twitter.com/en/apps). 

Here is a blog post with an example how to setup a Twitter application: [How to Register a Twitter App in 8 Easy Steps](https://iag.me/socialmedia/how-to-create-a-twitter-app-in-8-easy-steps/)

Here is a preview, how to get the credentials from your Twitter developer application:

![](https://suedbroecker.files.wordpress.com/2020/03/configure-twitter-auth.png?w=1024)

#### [b. Twitter API documentation `get-followers-list`](https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-followers-list)

---

### Step 2: Create a Node-RED instance on IBM Cloud

We use a Node-RED instance on IBM Cloud with an IBM Lite Account.

1. Create an IBM Cloud Lite Account just by register [here](https://ibm.biz/Bdzr7G).
2. Follow the steps in my blog post to [setup a Node-RED instance on IBM Cloud](https://suedbroecker.net/2020/03/09/a-short-introduction-of-the-node-red-starter-kit-on-ibm-cloud-for-hackathons/)

---

### Step 3: Install a additional Node to the Node-RED instance

Install the `node-red-node-base64` **Node** to the Node-RED instance. The gif shows the installation.

![](https://suedbroecker.files.wordpress.com/2020/03/install-node.gif?w=617&zoom=2)

---

### Step 4: Import the Node-RED flow

Import the flow from the [flow.json](flows/flows.json) file in `flows` folder of that project. The gif below shows how to import the flow in the Node-RED.

![](https://suedbroecker.files.wordpress.com/2020/03/import-flow-1.gif?w=617&zoom=2)

---

### Step 5: Configure the Node-RED flow

We need to configure ...

* ... the Twitter authentication 
* ... the Twitter username
* ... the Node-RED URL 

---

#### a. Set Twitter API key and secret

Insert the values for the `user` and `secret` of the Twitter API credentials in the _function_ `set user and secret`. The image shows an example, where you get the credentials information.

![](https://suedbroecker.files.wordpress.com/2020/03/configure-twitter-auth.png?w=1024)

The mapping for the Node-RED flow:
  1. `user` = _Twitter API_ key
  2. `secret` = _Twitter API_ secret key
  3. `nodereduser` = Your own definition to secure the Node-RED REST Endpoints
  4. `noderedpassword` = Your own definition to secure the Node-RED REST Endpoints

Here is the source code for the `set user and secret` function.

```javascript
var user = "USER";
var secret = "SECRET"
var nodereduser = "admin";
var noderedpassword = "notreallysecure"

flow.set("nodereduser", nodereduser);
flow.set("noderedpassword", noderedpassword);

msg.payload = user + ":" + secret;

return msg;
```

#### b. Set Twitter username

Define _"username display name"_ you want get the follower list from, by inserting 
in function `set_basic_auth` the `"YOUR_TWITTER_DISPLAY_NAME"`.

```javascript
// Set basic auth
flow.set("auth", msg.payload);
msg.payload = flow.get("auth");

// Init parameters:
// For more information visit get-followers-list:
// https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-followers-list
flow.set("twitterCursor", "-1");
flow.set("twitterResultCount", "50");
flow.set("twitterSN", "YOUR_TWITTER_DISPLAY_NAME");

msg.nodereduser = flow.get("nodereduser");
msg.noderedpassword = flow.get("noderedpassword");

return msg;
```
---

#### c. Set Node-RED URL

Configure the HTTP request nodes. Replace the `https://node-red-my-hackathon.mybluemix.net` URL with your URL in each _HTTP request node_:

* `getTwitterFollower-Indicrect`
* `getTwitterToken-Indicrect`

The image shows how you enter the URL in the _HTTP requests node_ `getTwitterFollower-Indicrect`.

![](https://suedbroecker.files.wordpress.com/2020/03/configure-http-node.png)

That image shows _HTTP requests nodes_ to be change.

![](https://suedbroecker.files.wordpress.com/2020/03/request-automation-followers-setup.png?w=2196)

---

### Step 6: Introduction to the Node-RED flow

The YouTube video gives a 13 min introduction to the _Twitter-follower-flow_.

[![Introduction to the Node-RED flow](https://img.youtube.com/vi/4HvBya4Zhn8/0.jpg)](https://www.youtube.com/watch?v=4HvBya4Zhn8 "Click play on youtube")


---

I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

[Blog post also at www.suedbroecker.net](https://suedbroecker.net/2020/03/26/a-node-red-twitter-follower-flow/)