How I Solved My NYC Parking Problem With Python, the Search Tweets API and Twilio
Tags: twitter, twilio, python, tutorial
Published: 2019-03-26 20:19:46

---
title: How I Solved My NYC Parking Problem With Python, the Search Tweets API and Twilio
published: true
description: Using the Twitter API with Python
tags: twitter, twilio, python, tutorial 
---
![car](https://images.unsplash.com/photo-1489824904134-891ab64532f1?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=1489&q=80)

_This tutorial was [originally posted to developer.twitter.com](https://developer.twitter.com/en/docs/tutorials/nyc-parking)._

Since I have a car in New York City, my car is subject to the city’s alternate side of the street parking regulations. This means most nights I need to move my car before the early morning street cleaning that happens in my neighborhood. I had developed a nightly routine around moving my car before I go to bed. I am sometimes a bit too good at this and I often move my car on days I don’t need to. Since alternate side of the street parking is often canceled on days where there are holidays, or bad weather, there is a Twitter handle [@NYCASP](https://twitter.com/NYCASP), which posts daily and whenever there is an emergency situation. 

To solve my problem I used Twitter data and Twilio. I now get a text message whenever I don’t need to move my car. I used [the Search Tweets wrapper for Python](https://github.com/twitterdev/search-tweets-python) to grab data from the aforementioned Twitter handle and Twilio so that I get a text message whenever I don’t need to move my car. I’ve created a script that will look to see if the words suspended and tomorrow appear in a Tweet and will send me a text message if those conditions are met. This tutorial will walk you through how I was able to create this solution.

## Getting our requirements set
The code I wrote was written with Python 3.6. I recommend using this version as well. In order to use pip, [you will need to install pip first](https://pip.pypa.io/en/stable/installing/). For this tutorial we’ll use [Atom](https://atom.io/). You will want to make sure you have the [command line tools installed](https://flight-manual.atom.io/getting-started/sections/installing-atom/).

We will also need to run the following in our command line to make sure we have the right dependencies set:

```bash
pip install pandas
pip install twilio
pip install searchtweets
```

Let’s now create on the command line a new directory for this project and change into it.

```bash
mkdir parking
cd parking
```

## Setting up a Jupyter notebook
I wrote my code using a Jupyter notebook, so I could work with the data in an interactive way. This was important for me in this project so I could see the Tweets I was searching for. You can use the Anaconda distribution of Python, which has Jupyter preinstalled or use the following syntax:

```bash
pip install jupyter 

```

Once you have Jupyter installed, you can run your notebook by typing this into your command line:

```bash
jupyter notebook

```

If you need to stop your Jupyter notebook at any point you can press `ctrl+c` to do so.

If the setup was done correctly, a list of what’s in your directory should launch in your browser. In the right hand corner click where it says `new`. From there you will want to click where it says `Python 3` in the dropdown option to launch a notebook. Once this launches, click where it says `untitled` to change the name of the notebook to parking.

## Setting up to connect with Twilio

You will need to create a script so that we can connect to [Twilio](https://www.twilio.com/). To do this you will need a Twilio account and check out the [getting started documentation on the subject](https://www.twilio.com/docs/sms/tutorials/how-to-send-sms-messages-python). To create a script to connect with Twilio. Let’s open up the directory we’re working in.

In our command line let’s type:

```bash
atom .
```

We’ll also need to set the environment variables for our Twilio account. These are found in your console once you have created a Twilio account: 

```bash
export 'TWILIO_ACCOUNT_SID'='xxxxxxxxxxxxxxxxxxx'
export 'TWILIO_AUTH_TOKEN'='xxxxxxxxxxxxxxxxxxxxxxx'
export 'TWILIO_PHONE_NUMBER'='xxxxxxxxxxx'
export 'CELL_PHONE_NUMBER'='xxxxxxxxxxx'
```

When we add in our phone numbers as environment variables we’ll need to put our phone number in one string with the country code, area code, and number. For example if I was in the US with the phone number (555) 555-5555 my string for the phone number variable would be ‘15555555555’.

From here we can add a file called `twilio_connect_demo.py`:

We will first add the two import statements we’ll need:

```python
import os
from twilio.rest import Client
```
Now we can write a function to help us connect to the Twilio API.

```python
def twilio_connect():
    account_sid = os.environ.get('TWILIO_ACCOUNT_SID')
    auth_token = os.environ.get('TWILIO_AUTH_TOKEN')
    client = Client(account_sid, auth_token)
    return client
```
Now we can write another function to send the text message: 

```python
def send_message(client):
    return client.messages.create(from_=os.environ.get('TWILIO_PHONE_NUMBER'),
                       to=os.environ.get('CELL_PHONE_NUMBER'),
                       body="You don't have to move your car tonight. Enjoy your night!")
```

Now we have our script that sends a text message, this will be helpful later on in this tutorial. 


## Importing what we need
Inside of our notebook, in the first cell we’ll need to import the libraries we’ll be working with. We’ll be using `datetime`, which is part of the Python standard library that lets us work with the dates to run our search. We’ll also need to use pandas which we will use to take our data and turn it into a dataframe. To work with the Search Tweets API, we’ll import [the search tweets Python wrapper](https://github.com/twitterdev/search-tweets-python). We’ll need the `ResultStream`, `gen_rule_playload`, and the `load_credentials` functions from this library. We’ll also import  the two functions from the script we just wrote to send text messages later on in the code.

```python
import datetime
import pandas as pd
from searchtweets import ResultStream, gen_rule_payload, load_credentials

from twilio_connect_demo import twilio_connect, send_message
```

To run this code we can press the play button or use the keyboard shortcut of `shift-enter`.

## Connecting to the Twitter API
 
You will need to create a Twitter app which allows you to connect to API. Before you can do that, you are going to need an approved developer account. After you have completed the initial set up, you will need to create a [developer account](https://developer.twitter.com/en/docs/basics/developer-portal/overview). You can apply for a developer account [here](https://developer.twitter.com/en/apply-for-access).
 
Once your account is approved, you will need to [create an app](https://developer.twitter.com/en/apps). Then, set up that app for the [Search Tweets: 30-Days environment](https://developer.twitter.com/en/account/environments) and set a dev environment label name.

After you have completed the initial set up, you will need to find your consumer API key and API secret key from your Twitter app. [For more information about getting set up to locate your keys, please review our documentation on the subject](https://developer.twitter.com/en/docs/basics/apps/overview). 

In Atom, we will need to add a new file that has our app key and secret in it. Let’s create a file called `secret.yaml` that contains the following:

```yaml
search_tweets_api:
  account_type: premium
  endpoint: https://api.twitter.com/1.1/tweets/search/30day/env_name.json
  consumer_key: xxxxxxxxxxxxxxxxxxx
  consumer_secret: xxxxxxxxxxxxxxxxxxx
```

The `env_name` in the endpoint is the name of the dev environment you created on developer.twitter.com. You will need to change this to the name of your own dev environment. 

You are going to want to make sure you add this file to your `.gitignore` before you push it to GitHub. For more information about working with an gitignore file [check out this page](https://help.github.com/en/articles/ignoring-files). You also might want to [review our guide on keeping tokens secure](https://developer.twitter.com/en/docs/basics/authentication/guides/securing-keys-and-tokens).

## Working with Twitter data
Back in the browser where we are running our Jupyter notebook, we can create a variable called search_args where we can load our credentials in to connect to the Twitter API.

```python
search_args = load_credentials(filename="secret.yaml",
                              yaml_key="search_tweets_api",
                              env_overwrite=False)
```
Since we want the dates we are searching to be dynamic, meaning that we don’t have to enter the date we’ll need to create some variables. Let’s start by creating one called `today` that grabs today’s date from the `datetime` library that we’ve imported in earlier. 

```python
today = datetime.date.today()
print(today)
```

When we run this line of code we should get back today’s date. To get the start date which is 30 days from today we’ll create a variable called start date. This will take today’s date and subtract 30 days using the built in method of timedelta from the datetime library. 

```python
start_date = today + datetime.timedelta(-30)
print(start_date)
```
After we run this we should get the start date which is 30 days from today. Now we have the dates we can pass into our `gen_rule_payload` we imported from the search tweets python wrapper that allows us to create a rule which will allow us to pull data from the past 30 days from the [@NYCASP twitter handle](https://twitter.com/NYCASP). 

```python
rule = gen_rule_payload("from:NYCASP",
                        from_date=str(start_date),
                        to_date=str(today),
                        results_per_call=500,
                       )
print(rule)
```
After you run this line of code, you should get back the rule that you can also paste into the body of a REST client such as [Postman](https://www.getpostman.com/) or [Insomnia]() to the view data ahead of time. 
 
```
{"query": "from:NYCASP", "maxResults": 500, "toDate": "201811060000", "fromDate": "201810070000"}
```

Let’s now create a variable called `rs`, which is short for ResultStream since we are going to create a result stream of Tweets. To do this we can pass the rule we just created into along with our credentials and a few other parameters into this variable as follows: 

```python
rs = ResultStream(rule_payload=rule,
                 max_results=500,
                 max_pages=1,
                 **search_args)

print(rs)
```
Once we print the variable called rs we can see the information we passed into it. This should look something like this: 

```python
ResultStream: 
	{
    "username": null,
    "endpoint": "https://api.twitter.com/1.1/tweets/search/30day/env_name.json",
    "rule_payload": {
        "query": "from:NYCASP",
        "maxResults": 500,
        "toDate": "201811060000",
        "fromDate": "201810070000"
    },
    "tweetify": true,
    "max_results": 500
}
```
We can pass the stream of the results using the attribute `.stream()` into a variable called `tweets`. 

```python
tweets = rs.stream()
```
From here we can convert the result stream into a list so we can work with this data in a more robust way. This is the point when we will start to see the Tweets we are working with. Let’s use a list comprehension to print out the first 5 rows, just to make sure we are on the right track here. 

```python
list_tweets = list(tweets)
[print(tweet.all_text, end='\n\n') for tweet in list_tweets[0:5]];
```

When we run this code we’ll get something that looks like this:  

```
#NYCASP rules will be suspended tomorrow, Wednesday, November 7 for Diwali. Parking meters will be in effect.

#NYCASP rules are suspended today, November 6 for Election Day. Parking meters are in effect.

#NYCASP rules will be suspended tomorrow, Tuesday, November 6 for Election Day. Parking meters will be in effect.

#NYCASP rules are in effect today, November 5.

#NYCASP rules will be in effect tomorrow, Monday, November 5.
```
Now that we know we are receiving the data properly we can create two lists one for the date, and one for the Tweet text. We’ll start two empty lists and use a for loop to iterate through our data. 

```python
tweet_text = []
tweet_date = []

for tweet in list_tweets:
    tweet_text.append(tweet['text'])
    tweet_date.append(tweet['created_at'])
```

Now we can use pandas to create a dataframe consisting of these two columns. 

```python
df = pd.DataFrame({'tweet':tweet_text, 'date':tweet_date})
```
We can use the head attribute to see the first 5 rows of the dataframe.

```python
df.head()
```
We’ll see something that looks like this:
![df image](https://thepracticaldev.s3.amazonaws.com/i/z85yxao9ry19jtui0rs9.png)

## Sending the message

Now that we have the Twitter data in the right shape. We can set up to send the message if the right conditions are met. We’ll first need to create a variable called `client` that allows us to connect to Twilio.

```python
client = twilio_connect()
```

From here we can set the logic to send us a text message if the words suspended and tomorrow appear in the last tweet. 

```python
if 'suspended' in df['tweet'].values[0]:
    if 'tomorrow' in df['tweet'].values[0]:
        send_message(client=client)
        print('text sent')
    else:
        print('suspended but not tomorrow, no text sent')
else: 
    print('not suspended, no text sent')
```

As a result, we should get the corresponding print messages depending on if alternate side of the street parking suspended tomorrow. This allows us to debug from our command line needed.

If the words suspended and tomorrow appear in the last tweet we’ll get a text message that looks like this:
![text](https://thepracticaldev.s3.amazonaws.com/i/14bk7cf1g2xxuc9xitpp.png)

To download this script as a `.py` file click in the right hand corner where it says file and select the option for download as.  From there you will be prompted to select the format, be sure to select `Python`.

## Deployment
As this currently stands, this is the only time you will get this text message unless you deploy it to a server. If you’d like to get a text message whenever you don’t need to move your car you can set this script that it runs daily on a server with a cron job. I have it set so this script runs daily at 7:30pm. 

## Next steps 
The full code is found [here](https://github.com/twitterdev/parking). My coworker in the London office mentioned this code could be easily adapted to run with [this handle](https://twitter.com/Oxford_Tube) to let her see when there are delays on the Oxford Tube. While talking with others about this project they mentioned that this project might make more sense if it tells when you need to move your car, instead of when you don’t need to move it. You could easily make a few changes to the code to make this happen. As it currently stands, this code could serve as a template to build off for other similar ideas.

Let us know on the forums or Tweet us at [@TwitterDev](https://twitter.com/TwitterDev) if this inspires you to build anything.