Using SAS Viya's Forecasting Platform to Forecast Future Vaccination Numbers
Tags: machinelearning, datascience
Published: 2021-05-27 19:28:20

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6gjkiybe6p3iai9hhjq0.PNG)

The past year has been an interesting one, as people around the world worked from home and completed classes and degrees online. We can finally see the light at the end of the tunnel, mostly due to the incredible work that pharmaceutical companies around the world completed over the last year. Creating a vaccine for COVID-19 was a massive undertaking, but so was quickly developing a plan to manufacture and distribute the vaccine. Using [SAS Viya's forecasting platform](https://www.sas.com/en_us/software/visual-forecasting.html), we can use machine learning to accurately predict a state's future total vaccinations, allowing a state to make better decisions regarding reopening plans.

We will create our application in three steps:

1. Create a basic webservice using FastAPI and Python to generate and return a model.
2. Use the [SWAT Python](https://sassoftware.github.io/python-swat/getting-started.html) package to leverage the power of SAS Viya and create a forecasting model given state vaccination data. [repo](https://github.com/benwoodman/viya-forecasting-covid-demo)
3. Create a frontend website so users can interact with the website. [repo](https://github.com/benwoodman/frontend-viya-forecasting-covid-demo)

For this project, we will use the [USA COVID-19 Vaccinations](https://www.kaggle.com/paultimothymooney/usa-covid19-vaccinations) dataset from Kaggle. 

Here is a sample of the data:

|date      |location                      |total_vaccinations|total_distributed|people_vaccinated|people_fully_vaccinated_per_hundred|total_vaccinations_per_hundred|people_fully_vaccinated|people_vaccinated_per_hundred|distributed_per_hundred|daily_vaccinations_raw|daily_vaccinations|daily_vaccinations_per_million|share_doses_used|
|----------|------------------------------|------------------|-----------------|-----------------|-----------------------------------|------------------------------|-----------------------|-----------------------------|-----------------------|----------------------|------------------|------------------------------|----------------|
|2021-01-12|Alabama                       |78134.0           |377025.0         |70861.0          |0.15                               |1.59                          |7270.0                 |1.45                         |7.69                   |                      |                  |                              |0.207           |
|2021-01-13|Alabama                       |84040.0           |378975.0         |74792.0          |0.19                               |1.71                          |9245.0                 |1.53                         |7.73                   |5906.0                |5906.0            |1205.0                        |0.222           |
|2021-01-14|Alabama                       |92300.0           |435350.0         |80480.0          |                                   |1.88                          |                       |1.64                         |8.88                   |8260.0                |7083.0            |1445.0                        |0.212           |
|2021-01-15|Alabama                       |100567.0          |444650.0         |86956.0          |0.28                               |2.05                          |13488.0                |1.77                         |9.07                   |8267.0                |7478.0            |1525.0                        |0.226           |

## 1: Create a FastAPI Webservice

Let's go ahead and create a new boilerplate FastAPI project to start our webservice.

```python
# All the packages we will eventually need for our project
from datetime import datetime
from fastapi.datastructures import UploadFile
from fastapi.param_functions import File
from numpy import str_
from swat import *
import pandas as pd
from matplotlib import pyplot
from fastapi import FastAPI, Form, File, UploadFile, HTTPException
from fastapi.responses import FileResponse
import base64

origins = [
    "http://localhost:3000",
]

app.add_middleware(
    CORSMiddleware,
    allow_origins=origins,
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

@app.post('/')
async def hello_world():
    return 'Hello World'
```

We can now run our new API by running `uvicorn main:app --reload` in our terminal. To learn more about FastAPI, [click here](https://fastapi.tiangolo.com/tutorial/).

Great, now we should see a page in our browser that displays the text "Hello World".

Let's add another endpoint that we will use to accept the user's username, password, and uploaded dataset. The API will then send this data to the CAS server for processing (step 2).

```python
@app.post('/forecast')
async def forecast(user: str = Form(...), password: str = Form(...), file: UploadFile = File(...)):
    return True
```

Here, we have created a new HTTP POST endpoint, where the user will submit a username, password, and file as form data. 


## 2: Use SAS Viya to Create a Forecasting Model

Now, let's connect to SAS Viya using the SWAT package. This is where we get to the cool stuff. 

We'll start by defining a function that will connect to CAS, Viya's Cloud Analytics Service, which will do all of the heavy lifting on the data for us.

```python
async def connect_to_cas(user, password):
    cas_server = 'YOUR_CAS_SERVER_URL'
    try: 
        s = CAS(cas_server, '8777', user, password)
    except:
        raise HTTPException(401)
    return s
```

Now we're connected to SAS's cloud analytics service. Let's first load our data into a pandas dataframe. If you recall, we passed a CSV file into our forecast function named 'file'. We will open this file and save it to our local drive.

```python 
async def forecast(user: str = Form(...), password: str = Form(...), file: UploadFile = File(...)):
    str_file = file.file.read()
    f = open('in.csv', 'wb')
    f.write(str_file)
    ...
``` 

From here, we can load our new data into a pandas dataframe.

```python
...
df = pd.read_csv('in.csv')
...
```

It is important to recognize that there are many different date formats when it comes to computers. That is why it will be important for us to use a standard format; in this case we will use the CAS date format. We can convert the dates in our dataset to CAS dates using the following code.

```python
...
# Convert string to python date obj
python_dates = pd.to_datetime(df['date'], format='%Y-%m-%d')
# Create list of new CAS dates from
# Python dates
new_dates = [cas.utils.datetime.python2cas_date(python_dates.iloc[i]) for i in range(len(python_dates))]
# Add CAS dates to our pandas dataframe
df.insert(1, 'dates_cas', new_dates)
...
```

The dates from our dataset are now in a format that CAS can understand. It's now time to upload our dataframe to CAS and apply the forecast action on it.

Documentation for the forecast action can be found [here](https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.3/casactecon/casactecon_forecast_syntax.htm). Information on using the forecasting action with SWAT can be found [here](https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.2/casactforecast/casactforecast_forecast_examples01.htm).

| parameter  | explanation                                                                                      |
|------------|--------------------------------------------------------------------------------------------------|
| table      | uploaded table to apply forecast to                                                              |
| timeid     | the name of the column with the dates                                                            |
| interval   | interval of the dates (month, day, year, etc.)                                                   |
| tStart     | start date for the data in your uploaded table                                                   |
| tEnd       | end date for the data in your uploaded table                                                     |
| dependents | the dependent variable for your forecast in this case, it will be the 'people_vaccinated' column |
| forout     | output table where the predictions for 'people_vaccinated' will be stored                        |
| lead       | amount of time we are forecasting                                                                |

```python
...
# Connect to the server
s = await connect_to_cas(user, password)
# Uploading our dataframe to the CAS server
resp = s.upload(
    df, 
    casOut={'name': 'covid_vaccines_upload', 'caslib': 'casuser', 'replace': 'true'}
)

# Add the timedata action set so that we can perform a forecast
s.loadActionSet('timedata')

# Perform that forecast using our uploaded table
# and spit out the forecast for the 
# next 30 days to a table named covid_vaccines.
s.timedata.forecast(
    table={'name':'covid_vaccines_upload', 'caslib': 'casuser'},
    timeid={'name':'dates_cas'},
    interval='day',
    tStart='2021-01-12',
    tEnd='2021-05-23',
    dependents=[{'accumulate':'SUM','name':'people_vaccinated'}],
    forout={'name': 'covid_vaccines', 'caslib': 'casuser', 'replace': 'true'}
    lead=30
)

# Retreive the forecasted data from the
# CAS server and create a dataframe from it
forecasted_data = s.CASTable('covid_vaccines',caslib='casuser').to_frame().drop(columns='_NAME_')
...
```



From here we can generate a plot and return the forecasted data to the user. 

```python
...
predictions_after = datetime.strptime(df.sort_values(by='date').iloc[-1]['date'], '%Y-%m-%d')
pyplot.plot(forecasted_data['dates_cas'], forecasted_data['PREDICT'])
pyplot.plot(forecasted_data['dates_cas'], forecasted_data['ACTUAL'])
pyplot.axvline(predictions_after)
pyplot.xlabel('Date')
pyplot.ylabel('Total Vaccinations (millions)')
pyplot.savefig('out.jpg', type='jpg')

return {'plot': base64.b64encode(open('out.jpg', 'rb').read()), 'tbl': forecasted_data.to_html()}
...
```

Our completed function will look something like this:

```python
@app.post('/forecast')
async def forecast(user: str = Form(...), password: str = Form(...), file: UploadFile = File(...)):
    str_file = file.file.read()
    f = open('in.csv', 'wb')
    f.write(str_file)
    # df = pd.DataFrame([x.split(',') for x in str_file.split('\n')])
    df = pd.read_csv('in.csv')
    python_dates = pd.to_datetime(df['date'], format='%Y-%m-%d')
    new_dates= [cas.utils.datetime.python2cas_date(python_dates.iloc[i]) for i in range(len(python_dates))]
    df.insert(1, 'dates_cas', new_dates)

    s = await connect_to_cas(user, password)

    resp = s.upload(df, casOut={'name': 'covid_vaccines_upload', 'caslib': 'casuser', 'replace': 'true'})

    s.loadActionSet('timedata')

    s.timedata.forecast(
        table={'name':'covid_vaccines_upload', 'caslib': 'casuser'},
        timeid={'name':'dates_cas'},
        interval='day',
        tStart='2021-01-12',
        tEnd='2021-05-23',
        dependents=[{'accumulate':'SUM','name':'people_vaccinated'}],
        forout={'name': 'covid_vaccines', 'caslib': 'casuser', 'replace': 'true'},
        selectOut={"name":'covid_vaccines_select', "caslib" :'casuser', "replace" : 'true'},
        infoOut={"name":'covid_vaccines_info', "caslib" :'casuser', "replace" : 'true'},
        lead=30
    )

    forecasted_data = s.CASTable('covid_vaccines', caslib='casuser').to_frame().drop(columns='_NAME_')

    # Last date in our original dataframe
    # used to generate the vertical line
    # to split the graph into actual and
    # forecasted data
    predictions_after = datetime.strptime(df.sort_values(by='date').iloc[-1]['date'], '%Y-%m-%d')
    pyplot.plot(forecasted_data['dates_cas'], forecasted_data['PREDICT'])
    pyplot.plot(forecasted_data['dates_cas'], forecasted_data['ACTUAL'])
    pyplot.axvline(predictions_after)
    pyplot.xlabel('Date')
    pyplot.ylabel('Total Vaccinations (millions)')
    pyplot.savefig('out.jpg', type='jpg')

    # return FileResponse('out.png')
    return {'plot': base64.b64encode(open('out.jpg', 'rb').read()), 'tbl': forecasted_data.to_html()}
```

Again, run the server using `uvicorn main:app`, and your new webservice will start!

## 3. Create a frontend web application for users to interact with

You can complete and design this part however you'd like. I built a basic page using [Bulma](https://bulma.io) for styling. I then wrote up a quick script using jQuery Ajax to send a request to our new server. You can view my frontend [here](https://github.com/benwoodman/frontend-viya-forecasting-covid-demo).

Once we run our code, our web application will show both the plot and the forecasted data for our project!

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sk0nefweiqrbz3iwm1w9.PNG)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t6ox9xb2iw0vfj3qnn8c.PNG)

## Testing

You can run this forecasting model on any dataset similar to the one above. We can use Excel or pandas to create subsets of the data, filtered by state. You can then submit that state's specific data at any time to get a new forecast.