Programming the lambda – Tweeting from a lambda
Tags: python, github, aws, lambda
Published: 2022-02-14 11:35:24

---
title: Programming the lambda – Tweeting from a lambda
published: true
description: Let's implement the Python code to query an API, generate a GeoPandas map and tweet it with Twython
tags: python, github, aws, lambda
cover_image: https://ik.imagekit.io/thatcsharpguy/posts/python-lambdas/cycles-part-2_AXPvmxNGH.jpg
series: Tweeting from an AWS Lambda
---

 > ⚠️ I like to use Pipenv to manage dependencies and virtual environments for Python applications – you can use any other dependency manager that suits you best.

The libraries I am going to use are:

- geopandas
- matplotlib
- mind-the-gap
- pandas
- seaborn
- twython

## Getting the information

The first thing I need to do is download the information of the London bike stations, for this I will use the library that I create myself (I can tell you about it in the future) to query the TFL API, I will create a separate file to be able to modularize information. The way we can do it with `mind-the-gap` is:

```python
from tfl.api import bike_point

all_bike_points = bike_point.all()

# Now we can take a single element and verify its content
place = all_bike_points[0]
print(f"{place.commonName} (LAT: {place.lat}, LON: {place.lon})")
# out: Vicarage Gate, Kensington (LAT: 51.504723, LON: -0.192538)
```

Additionally, each of those elements like place contain a set of additional properties, or `AdditionalProperties` from which we can extract information such as the number of docks available, how many of these docks are in use, and how many bikes you have available. To extract this additional information, I created this this helper function:

```python
def get_number(additional_properties: List[AdditionalProperties], key: str) -> int:
    [nb] = [prop.value for prop in additional_properties if prop.key == key]
    return int(nb)

# Then we can use it as:
bikes = get_number(place.additionalProperties, "NbBikes")
empty_docks = get_number(place.additionalProperties, "NbEmptyDocks")
docks = get_number(place.additionalProperties, "NbDocks")

print(f"{place.commonName} tiene {bikes} bicicletas disponibles y {docks} docks en total")
# out: Vicarage Gate, Kensington tiene 3 bicicletas disponibles y 18 docks en total
```

Then we can create a data frame using a *for* cycle:

```python
def download_cycles_info() -> pd.DataFrame:
    all_bike_points = bike_point.all()
    query_time = datetime.now()
    data = []

    for place in all_bike_points:
        bikes = get_number(place.additionalProperties,"NbBikes")
        empty_docks = get_number(place.additionalProperties,"NbEmptyDocks")
        docks = get_number(place.additionalProperties,"NbDocks")
        data.append(
            (
								place.id, place.commonName,
                place.lat, place.lon,
                bikes, empty_docks, docks,
            )
        )

    data_df = pd.DataFrame(
        data, columns=["id","name","lat","lon","bikes","empty_docks","docks"]
    ).set_index("id")
    data_df["query_time"] = pd.to_datetime(query_time).floor("Min")
		data_df["proportion"] = (data_df["docks"] - data_df["empty_docks"]) / data_df["docks"]

    return data_df

bike_info_data_frame = download_cycles_info()
bike_info_data_frame.head()
```

```python
| id             | name                      |     lat |       lon |   bikes |   empty_docks |   docks | query_time          |   proportion |
|:---------------|:--------------------------|--------:|----------:|--------:|--------------:|--------:|:--------------------|-------------:|
| BikePoints_103 | Vicarage Gate, Kensingt   | 51.5047 | -0.192538 |       1 |            17 |      18 | 2022-01-28 16:18:00 |    0.0555556 |
| BikePoints_105 | Westbourne Grove, Baysw   | 51.5155 | -0.19024  |      14 |            11 |      26 | 2022-01-28 16:18:00 |    0.576923  |
| BikePoints_106 | Woodstock Street, Mayfa   | 51.5141 | -0.147301 |      13 |             8 |      21 | 2022-01-28 16:18:00 |    0.619048  |
| BikePoints_107 | Finsbury Leisure Centre's | 51.526  | -0.096317 |       8 |            12 |      20 | 2022-01-28 16:18:00 |    0.4       |
| BikePoints_108 | Abbey Orchard Street, W   | 51.4981 | -0.132102 |      21 |             8 |      29 | 2022-01-28 16:18:00 |    0.724138  |
```

I have placed these two functions in a file named
Yo he puesto estas dos funciones en un archivo llamado *[download.py](http://download.py)* en la raíz de mi repositorio; más adelante lo usaré.

## Plotting the information

There are about 750 bike stations in London, as I want to make this information as accessible as possible, it occurred to me that the best way to do it was through an image showing the occupation of each of these stations.

### Getting a map

Before I start, I need a map of London in a format that the computer can interpret, and I just found one that I can even download programmatically from the city government's website. To make my life easier, I created a *Makefile* with a task called `shapefiles` that downloads and moves the necessary files:

```makefile
shapefiles:
	wget https://data.london.gov.uk/download/statistical-gis-boundary-files-london/9ba8c833-6370-4b11-abdc-314aa020d5e0/statistical-gis-boundaries-london.zip
	unzip statistical-gis-boundaries-london.zip
	mv statistical-gis-boundaries-london/ESRI shapefiles/
	rm -rf statistical-gis-boundaries-london statistical-gis-boundaries-london.zip
```

Which should leave us with a folder called *shapefiles* whose content is as follows:

```makefile
shapefiles
├── London_Borough_Excluding_MHW.GSS_CODE.atx
├── London_Borough_Excluding_MHW.NAME.atx
├── London_Borough_Excluding_MHW.dbf
├── London_Borough_Excluding_MHW.prj
├── London_Borough_Excluding_MHW.sbn
├── London_Borough_Excluding_MHW.sbx
├── London_Borough_Excluding_MHW.shp
├── London_Borough_Excluding_MHW.shp.xml
└── London_Borough_Excluding_MHW.shx
```

### Plotting a map

This function is more or less simple, there is a follow up post where I go into further detail about how I created the map, for the time being, I will just post the code and talk generally about what is going on:

```python
def plot_map(cycles_info: pd.DataFrame) -> str:
    london_map = gpd.read_file("shapefiles/London_Borough_Excluding_MHW.shp").to_crs(epsg=4326)

    fig = plt.figure(figsize=(6, 4), dpi=170)
    ax = fig.gca()

    london_map.plot(ax=ax)
    sns.scatter(y="lat", x="lon", hue="proportion", palette="Blues", data=cycles_info, s=25, ax=ax)

    prepare_axes(ax, cycles_info)

    map_file = save_fig(fig)

    return map_file
```

The first thing is to read the *.shp* file of the map that we are going to use. Then we create a shape and take the axes to draw on it. We draw the map using the `plot` method of `GeoDataFrame`. We use *seaborn* to put the stations on the map, keep in mind that we are specifying the location (`lat`, `lon`) for each point, the coloration of each point will be defined by the column `proportion` and finally the size of each of them will be 25. To finish we make some adjustments to the axis and save the figure in a temporary address only to return the path where the generated image is saved.

I have saved this function in a separate file named *plot.py*.

## Tweeting the information

We already have the image, it's time to tweet it using *Twython*, we're going to need a few secrets that we got from Twitter in the previous post, for now. Let's use those secrets to create a client of *Twython*:

```python
app_key = os.environ["API_KEY"]
app_secret = os.environ["API_SECRET"]
oauth_token = os.environ["ACCESS_TOKEN"]
oauth_token_secret = os.environ["ACCESS_TOKEN_SECRET"]

twitter = Twython(app_key, app_secret, oauth_token, oauth_token_secret)
```

The way the Twitter API works requires that we first upload the image to their service and then tweet it, for both we are going to use the newly created `twitter` variable, the trick is to use the `media_id` we retrieved from uploading the image:

```python
with open(image_path, "rb") as cycles_png:
    image = twitter.upload_media(media=cycles_png)

now = datetime.now().strftime("%m/%d/%Y, %H:%M")
twitter.update_status(
    status=f'London Cycles update at {now}',
    media_ids=[image['media_id']]
)
```

Just to modularize the code I put this code inside a function and this function in its own *tweeter.py* file. 

## Conclusion

We already have everything in place, now we can combine all our functions to achieve that with a single script we download information, generate a map and tweet it:

```python
from download import download_cycles_info
from plot import plot_map
from tweeter import tweet

def execute():
    information = download_cycles_info()
    map_image = plot_map(information)
    tweet(map_image)
```

I saved this code in a file called *app.py*. And this is [how it looks like](https://github.com/fferegrino/tweeting-cycles-lambda/tree/part-1-python) by the end of this post.

Remember that you can find me on Twitter [at @feregri_no](https://twitter.com/feregri_no) to ask me about this post – if something is not so clear or you found a typo. The final code for this series is on [GitHub](https://github.com/fferegrino/tweeting-cycles-lambda) and the account tweeting the status of the bike network is [@CyclesLondon](https://twitter.com/CyclesLondon).