Visualizing Cryptocurrency Data With AWS Serverless Technology
Tags: aws, quicksight, serverless, visualization
Published: 2022-11-19 09:43:36

We have about 20,000 cryptocurrencies, with over 9,000 of them actively trading. Similarly, we have over 4,000 exchanges, with over 500 of them being active. The [CoinMarketCap website](https://coinmarketcap.com/) has all of this information. However, I wanted to be able to analyse cryptocurrency data beyond just how many cryptocurrencies are active. To accomplish it, I ended up leveraging AWS Serverless services.   


This blog post describes how I used AWS serverless services to visualize cryptocurrency data using AWS Quicksight. So, if someone wants to do something similar, they know where to start. To achieve my goal, I used the following AWS services: 

- Lambda Functions

- S3

- AWS EventBridge

- Step Function

- QuickSight


## Lambda Functions
In this use scenario, I have three lambda functions. Each lambda function runs a given function and saves the result to an S3 bucket. One of the lambda functions is shown below, which uses the Coinmarketcap map API and saves the response to an S3 bucket.

```python
from os import path
import json, boto3, sys, uuid

# from botocore.vendored import requests

from requests import Request, Session
from requests.exceptions import ConnectionError, Timeout, TooManyRedirects

s3_client = boto3.client('s3')

def lambda_handler(event, context):
    url = "https://pro-api.coinmarketcap.com/v1/cryptocurrency/map"
    headers = {
        'Accepts': 'application/json',
        'X-CMC_PRO_API_KEY': 'xxxxxxxx',
    }
    
    session = Session()
    session.headers.update(headers)

    try:
        response = session.get(url)
        data=json.loads(response.text)
        json_body=json.dumps(data['data'])
        AWS_BUCKET_NAME='coinmarketcap-data'
        S3 = boto3.resource("s3")
        bucket_name = S3.Bucket(AWS_BUCKET_NAME)
        bucket_name.put_object(
            ContentType='application/json',
            Key="map-file.json",
            Body=json_body,
            )

        return {'statusCode': 200,'body': json.dumps('file is created')}
    except (ConnectionError, Timeout, TooManyRedirects) as e:
        print(e)


```

## S3
Lambda functions write their data to an S3 bucket, data in S3 is then later consumed by Quicksight for visualization.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/58iu2catyyceebg6pzka.png)

Here is an extract from the map data file stored in S3 bucket.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9dude81hjeeahp2eo29q.png) 

## AWS Step Function
The step function is used to connect Lambda functions into a serverless workflow. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yws5ra0mvbstrlh3wsgp.png)

## AWS EventBridge
For this specific use case, the EventBridge is used to schedule to run the Step function every 24 hours.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5rfw0gsecdhldxqwhk7u.png)

## AWS QuickSight
Finally AWS QuickSight is used to load the data from S3 bucket and visualize it as shown below. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r3oerx616nmt5kyokh9d.png)

The whole AWS serverless architecture for this use case looks like this:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5j9ov7wn93ywmllvbvkg.png)

------------------------------------------------------------------

Thanks for reading!

If you enjoyed this article feel free to share it on social media 🙂

Say Hello on: [Linkedin](https://www.linkedin.com/in/harinderseera/) | [Twitter](https://twitter.com/harinderseera) | [Polywork](https://www.polywork.com/harinderseera) 

Blogging: [Dev](https://dev.to/harinderseera) | [Hashnode](https://harinderseera.hashnode.dev/)

Github: [hseera](https://github.com/hseera)