Joyful Tweets
Tags: gftwhackathon, node, machinelearning, ibmwatson
Published: 2020-06-03 16:34:44

## What I built

There are a lot of pretty bad things going on in the world right now. Whilst not wanting to ignore or minimise them, sometimes it would be nice to just take a break for our own mental health. Wouldn't it be great if we could look at just the "joyful" tweets in our Twitter timeline?

I have build a small web app that authenticates to Twitter and fetches your Twitter timeline and filters it to analyse each of the tweets to just show the ones that might be "joyful" (and remove those that are classified as "angry").

### Submission Category: 

Creative Catalyst

## Demo

![Screenshot of Joyful Tweets](https://dev-to-uploads.s3.amazonaws.com/i/jwkqhh4mfyh5q96vjbzt.png)

A working demo is running at:

https://joyfultweets-daring-wildebeest-eq.eu-gb.mybluemix.net/ 

This service may not be up all the time as is using a paid tone analyser service. So may be stopped to minimize costs. The idea is that WM will help fund services in the future.

## Link to Code

{% github https://github.com/hammertoe/JoyfulTweets %}

Edit: When the hackathon was extended I went for a stretch goal and built my own tone analysis service, which the site now uses:

{% github https://github.com/hammertoe/JoyDetector %}

## How I built it 

A complete video of the build session (2.5 hours) is available on Youtube or Cinnamon:

{% youtube AYNu9L5JnSw %}

https://www.cinnamon.video/watch?v=333183211095983753

The app is built using the following technologies:

- Node.js, and Express, hosted on IBM Cloud Foundry
- Passport.js for the OAuth dance with Twitter
- Axios for making async parallel HTTP requests
- IBM Watson Tone Analyzer to detect the tone of the tweets
- Twitter API to fetch and render the tweet widgets
- Masonry.js to layout the screen
- And of course, Web Monetization via Coil

The browser makes a request to the backend which then directs it to do an OAuth dance with Twitter in order to obtain an API key to access the user's tweets. 

```javascript
app.get('/tweets',
        require('connect-ensure-login').ensureLoggedIn(),
        function(req, res) {

            var T = new Twit({
                consumer_key: process.env['TWITTER_CONSUMER_KEY'],
                consumer_secret: process.env['TWITTER_CONSUMER_SECRET'],
                access_token: req.user.token,
                access_token_secret: req.user.tokenSecret,
                timeout_ms:           60*1000,  // optional HTTP request timeout to apply to all requests.
                strictSSL:            true,     // optional - requires SSL certificates to be valid.
            })

            T.get('statuses/home_timeline', { count: 20,
                                              tweet_mode: 'extended' },
```

The backend then downloads the tweets from Twitter and then makes requests to the Watson Tone Analyzer:

```javascript
                      const agent = axios.create({
                          timeout: 1000,
                          auth: {username: 'apikey',
                                 password: tone_key},
                          adapter: throttleAdapterEnhancer(axios.defaults.adapter, { threshold: 1000 })
                      });

                      let tweets = await Promise.all(data.map(async tweet => {
                          let status = tweet.retweeted_status || tweet;
                          let text = status.full_text;

                          // connect to tone analyser
                          try {
                              let tones = await agent.post(tone_url, {text: text});
                              tweet.tones = tones.data.document_tone.tones;
                          } catch (error) {
                              console.error(error);
                          }
                          return tweet;
                      }))
```

The tweets are then filtered to remove any that are protected, or have the `anger` tone, and just returns the ones that have the `joy` tone:

```javascript
	                      let joy_tweets = tweets.filter(tweet => {
                          if (tweet.tones) {
                              for (let i=0; i<tweet.tones.length; i++) {
                                  if(tweet.tones[i].tone_id == 'anger') {
                                      return false;
                                  }
                                  if(tweet.user.protected) {
                                      return false;
                                  }
                                  
                              }
                              for (let i=0; i<tweet.tones.length; i++) {
                                  if(tweet.tones[i].tone_id == 'joy') {
                                      return true;
                                  }
                              }
                          }
                      })
```

The tweets are then returned to the browser and the client-side JS then generates the `<blockquote>` tags needed for the Twitter widget to work, puts a colourful border around them and lays them out with Masonry.js:

```javascript
   <script>
      $(document).ready(function() {
          $.getJSON("https://joyfultweets-daring-wildebeest-eq.eu-gb.mybluemix.net/tweets", data => {
              let i = 0;
              let colours = ['one', 'two', 'three', 'four', 'five'];
              data.tweets.forEach(tweet => {
                  let colour = colours[i % colours.length];
                  let url = `https://twitter.com/${tweet.user.screen_name}/status/${tweet.id_str}`;
                  i++;
                  console.log("tweet: ", url)
                  $("#container").append(`<div class="grid-item border ${colour}"><blockquote class="twitter-tweet"><a href="${url}"></blockquote></div>`)
              })
          }).then(() => {
              return twttr.widgets.load(
                  $("#container")
              );
          }).then(() => {
              return $("#fetching").hide();
          }).then(() => {
              return $('.grid').masonry({
                  // options
                  itemSelector: '.grid-item',
                  columnWidth: 560
              });
          })
      })
      </script>
```

Generally it all went well, even though I'm more comfortable in Python than Node.js. I decided to use Node as would be easier to do multiple parallel async HTTP requests out to the APIs. However this turned out be a blessing and a curse as I quickly overwhelmed the APIs and had to implement rate limiting to prevent overloading the API.

## Additional Resources/Info 

~~My plan is to continue developing this further after the submission and train my own neural network to detect tweet emotions. This will be written in Python and likely also hosted on Cloud Foundry and present a similar API to Watson Tone Analyzer than I can then redirect this app to.~~ 

Edit: due to the hackathon extension I went ahead and did this, see link above in code section.

I will not be submitting this to the main Grant for the Web process, as I am already submitting a larger, different project to that.