Explore how countries produce energy for MongoDB Atlas Hackathon
Tags: atlashackathon, python, flask, mongodb
Published: 2022-01-01 17:03:51

### Overview of My Submission
The **climate change** is an important topic we should all care to protect our planet. So, I wanted to create a simple web application to explore how countries produce energy, if they rely on renewable energy or fossil fuels.

To visit my web application click [here](https://secure-castle-44065.herokuapp.com/)! Similarly to the TV show *Our Planet*, I wanted to call my app *Our Energy* because it is a resource we all should be care of!

Below you can see the homepage:

![homepage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1ryb6lxtgg540n7vo0lu.png)

You can look the amount of MegaWatts of energy produced by a country. To find the countries in the Mongo Database I used [**Atlas Search** service](https://docs.atlas.mongodb.com/atlas-search/). Below you can see the values for my country, [Italy](https://secure-castle-44065.herokuapp.com/country/Italy):

![italy energy production](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zjtp3p8lldojwkezyzgr.png)

Moreover, from the homepage you can explore the **top 5 countries** producing energy using a particular renewable source. You can choose among solar, wind, waves and tidal, and hydro. Below there is a screenshot for the [wind energy](https://secure-castle-44065.herokuapp.com/fuel/Wind): 

![top 5 wind](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qu9dwtpbmp532ujhv5gq.png)

### Data preparation 
The original dataset was in `.csv` format. Firstly I removed many columns that were not necessary to my scope to reduce the size. Then I converted it to a JSON file which is easy to be uploaded to MongoDB Atlas.

Connecting to Atlas Cloud is as easy as connecting to any local database thank you to this [official guide](https://docs.mongodb.com/guides/cloud/connectionstring/). To upload the dataset onto Mongo Cloud I used literally one function: `insert_many()`. You can explore the full code [here](https://github.com/andregri/OurEnergy/blob/main/db_preparation/upload.py).


### Atlas Search
To search data by country I had to match the input string with the `country_long` field of the dataset. I used the **Atlas UI** to create an index with *static mapping* for this field. You can refer to this [guide](https://docs.atlas.mongodb.com/atlas-search/) to create one. 


### Aggregation pipeline
The dataset consists of many power plants located in many countries. To retrieve a summary information for a certain country I exploited the power **aggregation pipeline** feature of MongoDB. Look at this [code snippet](https://github.com/andregri/OurEnergy/blob/main/db.py) that I use to find the 5 countries producing the biggest quantity of energy from a particular `fuel` (oil, gas, wind, solar, etc..., renewable energy sources are considered fuels as well):
```python
result = collection.aggregate([
      {
         '$match': {
            'primary_fuel': fuel
         }
      },
      {
         # Accumulate capacity by fuel type
         '$group':
         {
            '_id': '$country_long',
            'totCapacity': {
               '$sum': {'$toDecimal': '$capacity_mw'}
            }
         }
      },
      {
         '$sort': {'totCapacity': -1}
      },
      {
         '$limit': 5
      }
   ])
```

A **pipeline** is made of many stages that performs different operations to extract the data we want from the database. The output of a stage is the input of the next stage.

1. The first step find all the entries whose field `'primary_fuel'` matches the string `fuel`:
```python
{
    '$match': {
        'primary_fuel': fuel
    }
},
```
2. This step groups the output of the previous step by country (using the field `country_long`, the country name) and accumulate the energy production of a power plant. Up to now, we know the amount of wind energy produced by each country.
```python
{
     '$group':
     {
        '_id': '$country_long',
        'totCapacity': {
           '$sum': {'$toDecimal': '$capacity_mw'}
        }
     }
},
```

3. We sort the output in descending order
```python
{
    '$sort': {'totCapacity': -1}
},    
```

4. Eventually, we limit the output to only 5 results
```python
{
     '$limit': 5
}
```

And in the web app we can see the Top 5 countries producing that kind of energy. For instance, if you want to discover who are the 5 biggest producer of energy from [waves and tydal](https://secure-castle-44065.herokuapp.com/fuel/Wave%20and%20Tidal) are:

![top 5 wave](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/me054y8xm3s4zb0ttjyd.png)

### Conclusion

It was exciting to produce a simple web application from scratch to production using Python and MongoDB.

If you want to contribute to my project or propose some improvements, visit the [GitHub repository](https://github.com/andregri/OurEnergy).


### Submission Category:
Choose Your Own Adventure


### GitHub repository
{% github https://github.com/andregri/OurEnergy %}


### Resources
- [Global Power Plant Database](https://datasets.wri.org/dataset/globalpowerplantdatabase)