Building the frontend - Part III (Live tweet sentiment analysis)
Tags: python, docker
Published: 2020-08-15 08:39:42

This is the final article of the series __Live tweet sentiment analysis__. In the previous article, we saw how to create an API to serve our predictions. This article will cover how to build a frontend.

Normally, frontends are written in Reactjs or Vuejs(these are the only ones I know). But I'm a big fan of Python and I would like to build the frontend using python. [Streamlit](https://www.streamlit.io/) helps you build quickly an interface to serve your machine learning apps. 

> Even though the backend isn't necessary here(streamlit could've handled the prediction part also), it is important to shift the heavy-duty task to a backend that can spawn multiple workers(`gunicorn` instead of `uvicorn`) and implement better job-queuing and async methods.

### Building the frontend

`make sure you are in frontend directory`

#### Streaming the hashtags
```python
# tweets.py

import tweepy as tw

auth = tw.OAuthHandler(config.consumer_key, config.consumer_secret)
auth.set_access_token(config.access_token, config.access_token_secret)
api = tw.API(auth, wait_on_rate_limit=True)

tweets = tw.Cursor(api.search, q=hashtag, lang="en", since=date).items()
```
`tweets` object will hold the stream of tweets retrieved from twitter since the date given.

#### Building the UI
```python
# tweets.py

import streamlit as st

st.title("Twitter Live Sentiment Visualizer (beta)")

# design the sibar elements
st.sidebar.title("Enter a hashtag")
hashtag = st.sidebar.text_input("hashtag", "trump")
date = st.sidebar.date_input("Analyse tweets from", datetime.date(2020, 8, 1))

# if "Live analysis" is clicked
if st.sidebar.button("Live analysis", key="analyse"):
    pos_count = 0
    neg_count = 0
    tweets_count = 0

    st.subheader(f"Analysing #{hashtag} from {date}")
    d = {"Positive": [pos_count], "Negative": [neg_count]}
    df = pd.DataFrame(data=d)

    hashtag = f"#{hashtag}"

    # get the tweets
    with st.spinner("Getting tweets..."):
        tweets = tw.Cursor(api.search, q=hashtag, lang="en", since=date).items()

    total_tweets = st.empty()
    pos_tweets = st.empty()
    neg_tweets = st.empty()

    sentiments = ["Positive", "Negative"]
    chart = st.line_chart(df)

    # iterate through all the tweets
    for idx, tweet in enumerate(tweets):
        tweets_count += 1
        output = requests.post("http://backend:8000/api", json={"tweet": tweet.text})
        output = output.content.decode("utf8")
        output = json.loads(output).get("sentiment")

        # count the number of each sentiment
        if output == "Negative":
            neg_count += 1
        elif output == "Positive":
            pos_count += 1

        total_tweets.text("Tweets Analysed: %d" % tweets_count)
        pos_tweets.text("Positive tweets: %d" % pos_count)
        neg_tweets.text("Negative tweets: %d" % neg_count)

        # update the DataFrame
        df2 = pd.DataFrame({"Positive": [pos_count], "Negative": [neg_count]})
        df.update(df2)
        # draw the cumulative chart
        chart.add_rows(df)

    if neg_count == 0 and pos_count == 0:
        st.warning(f"No Tweets Found on {hashtag}")
    else:
        st.success("Tweets classified")
```

Basically what this does is get the tweets from twitter API, send it to the backend for inference and plot the cumulative of each sentiment. You could even change this to a `seven tweets moving average`😉. One important thing to notice is that 
```python
output = requests.post("http://backend:8000/api", json={"tweet": tweet.text})
```
we used `backend:8000` instead of `localhost:8000`. This is because the service that we build for the backend will be called `backend` and the host will be `backend` instead of `localhost`

#### Dockerize the frontend
```Dockerfile
FROM python:3.8-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install -r requirements.txt

EXPOSE 8501

COPY . .

CMD ["streamlit", "run", "tweets.py"]

# nothing fancy here
```

#### Now let's bind everything to build our app
`cd ..` to move back to the root directory. Your project structure should look like this.
```bash
❯ tree .
.
├── LICENSE
├── README.md
├── backend
│   ├── Dockerfile
│   ├── classify.py
│   ├── models
│   │   └── sentiment_model.pickle
│   ├── requirements.txt
│   ├── server.py
│   └── utils.py
├── docker-compose.yml
└── frontend
    ├── Dockerfile
    ├── config.py
    ├── requirements.txt
    └── tweets.py
```

#### Docker-compose
Create a `docker-compose.yml` in the root directory
```yml
version: "3"
services:
  frontend:
    build: frontend
    ports:
      - "8501:8501"
    depends_on:
      - backend

  backend:
    build: backend
    ports:
      - "8000:8000"
```

- version: [Compose file version 3](https://docs.docker.com/compose/compose-file/)
- services: here we define the services(frontend and backend)
- depends_on: since frontend depends on backend

```bash
❯ docker-compose build
❯ docker-compose up -d
Starting sentwitter_backend_1 ... done
Starting sentwitter_frontend_1 ... done
# -d for detach mode (it'll run in background)
```

Navigate to `localhost:8501` and you should see your app running. Let's do a quick run on `#2Articles1Week`.

![2articles1week](https://j.gifs.com/XLOgq8.gif)

> That's a lot of red flags right there for the model. Nobody will write negatively about `2articles1week`. It's just the model is so bad.

### Reference
- Article cover by [Patrick Tomasso](https://unsplash.com/@impatrickt)