Sending out SMS messages via Twilio
Tags: tutorial, python, sms, twilio
Published: 2024-03-10 22:12:07

In the [previous post](https://dev.to/idadelveloper/web-scrapping-in-python-part-2-j0m), we scrapped a job posting site and were able to get a couple of jobs. Rememeber, I mentioned in [Part 1](https://dev.to/idadelveloper/web-scraping-with-python-part-1-43jb) we will scrap a webpage and sent jobs to a user via SMS. Talking about SMS you should definitely checkout this [article](https://dev.to/idadelveloper/how-sms-messages-are-handled-between-2g-3g-4g-5g-2kk9).

Before we jump right into creating the SMS functionality, let's first of all store our scrapped data into a very simple SQLite database. I will be creating a new file called `data.py`. This is where the code handling our data storage will reside. 
We will create a class which will handle all operations in relation to the data base from creating to adding to deleting data. For now we will implement a few methods to be sure the basic functionality at least works. When the class will be instantiated, a SQLite connection will be established and both the database and the table will be created if they don't exist. We will have 2 methods to save the data to the database and also retrieve data (for now). When storing our data, we will make sure the URL is the primary key to make sure duplicate data won't be saved. With time we will implement error handling and refine the class.

```python
import sqlite3

class JobsDataBase:
    def __init__(self):
        self.conn = sqlite3.connect("jobs_data.sqlite")
        self.cur = self.conn.cursor()
        self.cur.execute('CREATE TABLE IF NOT EXISTS jobs (id VARCHAR, description VARCHAR, PRIMARY KEY (id))')
        self.conn.commit()

    def save(self, id, body):
        self.cur.execute(f'INSERT OR IGNORE INTO jobs (id, description) values ("{id}", "{body}")')
        self.conn.commit()
        print("done")

    def getjobs(self):
        self.conn = sqlite3.connect("jobs_data.sqlite")
        self.cur.execute('SELECT * FROM jobs')
        data = self.cur.fetchall()
        return data
```
Now let's create a function called `save_data` within `scrapper.py`. Don't forget to import `data.py`.

```
def save_data(jobs):
    jobs_data = data.JobsDataBase()
    for j in jobs:
        jobs_data.save(j[0], j[1])
```

The `get_job_details` function in `scrapper.py` has been modified to return a list of lists containing job URLs and descriptions.

```python
def get_job_details(urls):
    job_details = []
    for url in urls:
        page = requests.get(url)
        soup = BeautifulSoup(page.content, "html.parser")

        job = soup.find("div", class_="job-description")
        job_details.append([url, job.find("p").text])
    return job_details
```

Now let's dive into implementing our SMS functionality. After savind data into our database, we want to send out an SMS to the user. Considering the fact that there won't be duplicte data stored in the database, there are zero chances of sending out an SMS more than once (unless there's a bug in the code)

We will be using Twilio to implement the SMS functionality. wilio is a cloud-based communications company that offers tools for making and receiving calls, sending messages, and more. Twilio's products include voice, video, messaging, authentication, and more.

Before we dive into the code, we will start by creating a [Twilio](https://www.twilio.com/try-twilio) account and download its python [helper library].(https://github.com/twilio/twilio-python). Run the following command.
```
pip3 install twilio
```
Make sure you setup your Twilio console and get your `account SID` and `Auth token`. Create the variables `account_sid` and `auth_token` in your code and assign them their values. Be sure to also generate your Twilio phonenumber which you will use to send SMS messages.

Now lets start by importing our Twilio client.

```python
from twilio.rest import Client
```
Now we will create a `send_sms` function. In the fuction you can see we when conposing the message to be sent we limited the characters. That's because there's a character limit and for now we won't dwell on hacking that limitation as our main focus is sending out an SMS via Twilio.

```python
def send_sms(url, job):
    client = Client(account_sid, auth_token)
    text = f"{job[:500]}\n\nVisit site: {url}"
    message = client.messages.create(
    from_='twilio-phone-number',
    to='receiver-phone-number',
    body=text
    )
```
We will be sending out an SMS after saving any job into the database. So we will add an extra line in our `save_data` function.

```python
def save_data(jobs):
    jobs_data = data.JobsDataBase()
    for j in jobs:
        jobs_data.save(j[0], j[1])
        send_sms(j[0], j[1])
```

Running the `save_data` function will send out SMS messages depending on the number of jobs. In our next articles, we will refine the data base, send out SMS messages with all the details, etc