Real-time Twitter Sentiment Analysis for Brand Improvement and Topic Tracking (Chapter 3/3)
By: Chulong Li
Tags: Twitter, Web App Development, Dash, Data Science, Heroku

Migrate Data Analytics & Visualizations from Plotly to Dash

To put our data visualization from the previous chapter on the Heroku app, we need to wrap our Plotly-based dashboard with Dash framework. All data analysis and visualizations will be processed in the file app.py , and you may check my entire code for this file here.

Start with a new app server with the Dash default CSS sheet. Note: I recommend to improve the appearance of the web app using Bootstrap.

external_stylesheets=['https://codepen.io/chriddyp/pen/bWLwgP.css'] app = dash.Dash(__name__, external_stylesheets=external_stylesheets)

app.title = 'Real-Time Twitter Monitor' server = app.server

The core framework of Dash web app is using app.layout as a background layout. Note: Click on two links below to understand how they work and check some typical samples as they are VERY IMPORTANT to utilize the Dash.

Dash Layout allows us to display integrated data visualization along with other text descriptions.

allows us to display integrated data visualization along with other text descriptions. Dash Callbacks enables the application to update everything consistently with real-time data.

Since Dash is built on top of HTML, it uses HTML in its own way (Dash HTML Components). But some of HTML features are not allowed in the Dash.

html.Div(id='live-update-graph') describes the top part of the dashboard in the application, including descriptions for tweet number and potential impressions. html.Div(id='live-update-graph-bottom') describes the bottom part of dashboard in the web app.

dcc.Interval is the key to allow the application to update information regularly. Although the data collector, which will be explained later, on another dyno works in real-time, the analytical dashboard only analyzes and visualizes data per 10 seconds because of the utilization of aggregated data for visualization and the consideration of cost-efficiency.

app.layout = html.Div(children=[



# Parts of Title hided

html.H2('This-is-your-tittle'), html.Div(id='live-update-graph'),

html.Div(id='live-update-graph-bottom'),



# Parts of Summary hided

html.Div(

dcc.Markdown("Author's Words: ...... ")

) # Timer for updating data per 10 seconds

dcc.Interval(

id='interval-component-slow',

interval=1*10000, # in milliseconds

n_intervals=0

)



], style={'padding': '20px'}) }

For each Div in the HTML of Dash, it has className , children , and style . children is a list of dcc.Graph (Graphs in Dash Core Components), dcc.Markdown , html.Div , html.H1 , and other interactive buttons (e.g. dcc.Dropdown and dcc.Slider ).

html.Div(

className='row',

children=[

dcc.Markdown("..."),

dcc.Graph(...),

html.Div(...)

],

style={'width': '35%', 'marginLeft': 70}

)

style is important for building a good layout and ensuring proper distances among several visualization graphs, but can be time-consuming to tune the details.

Give an example of dcc.Graph below (Graphs in Dash Core Components). dcc.Graph has attributes id and figure . Under the figure , there are 'data' , containing different kinds of graphs in Plotly (e.g. go.Scatter and go.Pie ), and 'layout' , which is only the layout of this graph rather than app.layout .

# import dash_core_components as dcc

# import plotly.graph_objs as go dcc.Graph(

id='pie-chart',

figure={

'data': [

go.Pie(

labels=['Positives', 'Negatives', 'Neutrals'],

values=[pos_num, neg_num, neu_num],

)

],

'layout':{

'showlegend':False,

'title':'Tweets In Last 10 Mins',

'annotations':[

# ...

]

}

}

)

'annotations' is another important part to ensure the current labels. Note: Reference for Annotation in Dash is quite ambiguous, since parameters may need to be expressed as go.layout.Annotation , a dictionary dict(...) , and sometimes a list.

In the Dash app layout, reactive and functional Python callbacks provide connections between inputs and outputs, allowing customizable declarative UIs.

Note: We skip all data analysis parts that were explained in the previous chapter, although there might be a little bit difference (or improvement). Thus, we dive directly into Dash-based data visualization parts, which are implemented through complicated Plotly graphs plotly.graph_objs (a.k.a. go).

# Multiple components can update everytime interval gets fired.

@app.callback(

Output('live-update-graph', 'children'),

[Input('interval-component-slow', 'n_intervals')]

)

def update_graph_live(n): # Lots of nested Div to ensure the proper layout

# Graphs will be explained later

# All codes are hided return children

All three line charts use go.Scatter along with stack groups to ensure the overlapped areas under lines in the same graph.

# import plotly.graph_objs as go

go.Scatter(

x=time_series,

y=result["Num of A-Brand-Name mentions",

name="Neutrals",

opacity=0.8,

mode='lines',

line=dict(width=0.5, color='rgb(131, 90, 241)'),

stackgroup='one'

)

Pie chart was explained in the example of dcc.Graph above.

For text descriptions (e.g. Tweet Number Change per 10 Mins), we use two HTML paragraphs to embed data info. By comparing the previous 10-min interval and current 10-min interval, we could generate such a result.

html.P(

'Tweets/10 Mins Changed By',

style={'fontSize': 17}

), html.P(

'{0:.2f}%'.format(percent) if percent <= 0 \

else '+{0:.2f}%'.format(percent),

style={'fontSize': 40}

)

For Potential Impressions Today, we want to figure out how many people at most have a chance to see these tweets. By counting the sum of follower numbers of people who posted these tweets, we can get the potential impressions. Add a dynamic numeric unit to better display this value in a very large range.

html.P(

'{0:.1f}K'.format(daily_impressions/1000) \

if daily_impressions < 1000000 else \

('{0:.1f}M'.format(daily_impressions/1000000)

if daily_impressions < 1000000000 \

else '{0:.1f}B'.format(daily_impressions/1000000000)), style={'fontSize': 40}

)

Counting Daily Tweet Number is an easy approach by storing and adding up tweet numbers in each app data update. And this value will be reset to zero at midnight.

html.P(

'{0:.1f}K'.format(daily_tweets_num/1000),

style={'fontSize': 40}

)

The callback function for the bottom dashboard is similar to the first one.

@app.callback(Output('live-update-graph-bottom', 'children'),

[Input('interval-component-slow', 'n_intervals')])

def update_graph_bottom_live(n): # Lots of nested Div to ensure the proper layout

# Graphs will be explained later

# All codes are hided return children

Bar chart for hottest topic tracking. Set orientation as horizontal to better display bars and related word names

go.Bar(

x=fd["Frequency"].loc[::-1],

y=fd["Word"].loc[::-1],

name="Neutrals",

orientation='h',

marker_color=fd['Marker_Color'].loc[::-1].to_list(),

marker=dict(

line=dict(

color=fd['Line_Color'].loc[::-1].to_list(),

width=1

),

)

)

Focus the scope on State-level Map by setting 'layout': {'geo':{'scope':'usa'}}

go.Choropleth(

locations=geo_dist['State'], # Spatial coordinates

z = geo_dist['Log Num'].astype(float), # Color-coded data

locationmode = 'USA-states',

text=geo_dist['text'], # hover text

geo = 'geo',

colorbar_title = "Num in Log2",

marker_line_color='white',

colorscale = ["#fdf7ff", "#835af1"]

)

In order to parallel multiple graphs in a single row, you may consider using the following. width should be 1/N percentage (N = number of graph), and we may subtract 1% to give enough gaps between graphs.