Message bot to find a PS5 on sale
Tags: python, twilio, ps5, scraping
Published: 2020-11-30 03:32:02

---
title: Message bot to find a PS5 on sale
published: true
description: Like many consumers, I found myself frustrated trying to find a PS5 on sale. Rather than continually searching sites hoping to find a unit on sale, I decided to build a bot to send me a text message when one of the main retailers has one on sale.
tags: python, twilio, ps5, scraping
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/5q680k35rvjkgf5efzi0.jpg
---

As an avid gamer, I was committed to acquire a next-generation console, the PS5. But, like many consumers, I found myself frustrated trying to find a PS5 on sale.

Walmart seemed like a viable option, but its site was infuriating with posted times and problems adding items to a cart and purchasing. Other retailers such as Amazon, GameStop, and Target did not have advertised times for which the units would go on sale.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/fq9ubhd9wn6ilz9xa9ht.jpeg)

Instead of aimlessly trying to search these major retailers sporadically throughout the day, let's create a bot to do it for us. My intention is to use Python, Selenium with Chromedriver, Heroku Scheduler, and Twilio to send my personal phone a message whenever a unit is available.

# Prerequisites

- Python
- Git
- Selenium with Chrome Driver
- Twilio
- Heroku

# Python

I created a stub repository for this project which anyone could use, given they supply valid keys to an environment file: [python-flask-heroku-ps5-notifier](https://github.com/DEV3L/python-flask-heroku-ps5-notifier). In order to deploy to Heroku, I recommend forking or Rambo cloning this repository.

To download and run the project, run the following commands:
```
git clone https://github.com/DEV3L/python-flask-heroku-ps5-notifier.git
cd python-flask-heroku-ps5-notifier
cp .env.local .env
python3 -m venv python-flask-heroku-ps5-notifier
source python-flask-heroku-ps5-notifier/bin/activate
python setup.py develop
pytest
python app.py runserver
```

The result of the `pytest` command should result in passing tests. With `python app.py runserver` you should be able to visit http://localhost:5000 in a browser to see a "Hello, world!" message.

If you got a successful result on both of these commands, congratulations, you are ready to move forward! The repository has .env in the .gitignore file, please do not check in any secrets into your own public repository. These secrets will be hidden and managed inside of Heroku.

# Selenium with Chrome Driver

This can be a tricky step. In order to run headless on Heroku, we need to make sure our initialization of the Selenium Chrome Web Driver has appropriate path variables. Here is a short YouTube video that explains what we are trying to do, [Running ChromeDriver with Python Selenium on Heroku ](https://www.youtube.com/watch?v=Ven-pqwk3ec&ab_channel=AndresSevilla).

On the server, the variables will be as the video describes. But, in order to test locally, we need to make sure the `CHROMEDRIVER_PATH` and `GOOGLE_CHROME_BIN` are set appropriately for your unique machine paths. If you are using a mac, BIN file location will not have to change. In the project, an instance of chrome driver for mac is available. Set the path to whatever this value is for you.

```
# Selenium
CHROMEDRIVER_PATH=/Users/dev3l/workspace/python-flask-heroku-ps5-notifier/drivers/chromedriver
GOOGLE_CHROME_BIN=/Applications/Google Chrome.app/Contents/MacOS/Google Chrome
HEADLESS=True
USER_AGENT=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36
```

The `USER_AGENT` is configurable incase that needs to change in the future to accommodate any bot detection. In addition, if you want to see the magic happen (not headless) just change the value of `HEADLESS` to False and the browser will display. This helps out with debugging when scraping code has to change.

With your virtual environment active, run the following command:
```
python scrape_stub.py
```

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/527h6bz50eju73fks45a.png)

If you see the message "Selenium with Chrome configured correctly", then Selenium is appropriately configured for your local machine. Let's move on to setting up Twilio.

# Twilio

Setting up a free Twilio account was a bit tricky at first. Visit [Twilio](https://www.twilio.com/) and sign up for a trial account. As long as you are sending messages only to verified numbers and ok with the message starting with "Sent from your Twilio trial account - ", you can get away with the free trial.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/1q7mvfwmud0i7u663tk6.png)

Create a new project, giving it any name you desire. Then from the _All Products and Services_ menu (... on the left menu while logged in), select _Programmable Messaging_. Select _Try it out_ and _Try SMS..._, at this point you should be able to acquire a free SMS number. Using the UI, verify that you can send a message to your validated phone number.

Once completed, update the variables in the test project to your appropriate values:

```
# Twilio
TWILIO_ACCOUNT_SID=TWILIO_ACCOUNT_SID
TWILIO_AUTH_TOKEN=TWILIO_AUTH_TOKEN
TWILIO_TO=+12316851234
TWILIO_FROM=+15555555555
```

Given you have updated these variables appropriately, run the following command to verify that Twilio is setup appropriately:

```
python scripts/twillio-message.py
```

If you received a "Hello, world!" text message, you are ready to move forward!

# Heroku

If you do not have one already, sign up for a free [Heroku](https://dashboard.heroku.com/) account. Create a new app and give it a name, mine is `ps5-notifications`. Once created, go to the "deploy" tab and select GitHub as a Deployment method.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/91br0free0b5e5h3vad4.png)

Search for and connect the repository. Be sure to press the button to enable automatic deployments. The project should automatically deploy and be available at your `<project-name>.herokuapp.com`, for example, mine is available at https://ps5-notifications.herokuapp.com/

Without any modification, you should be able to visit your site URL (Open app button at the top of the Heroku project dashboard). If this works for you, once again you are ready to move forward!

# Update Heroku Environment Variables

Without any environment variables set, we are not going to be able to send messages to our target phone. Within the Heroku dashboard for your app, go to _Settings_ and then click on the _Reveal Config Vars_ button. Enter each of your specific four key-value pairs for `TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, 
TWILIO_TO, TWILIO_FROM`. Once completed, redeploy your app by going to the _Deploy_ tab and select _Deploy Branch_ from the _Manual deploy_ section.

With this done, you should now be able to manually send yourself a text message through a URL in your browser. `<project-name>.herokuapp.com/twilio` will send a default "Hello, World!" text message, or you can add a **message** query string parameter to customize this message. I may regret putting this here, but you can hit my example at [Send Message Example Link](https://ps5-notifications.herokuapp.com/twilio?message=Hello from Dev.to!)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/fisqosymrmqujz68cvph.png)

If you received a text message on your phone, move forward!

# Setting up Selenium with Chrome Driver in Heroku

Following the steps in the video linked above, or [here](https://www.andressevilla.com/running-chromedriver-with-python-selenium-on-heroku/), we need to add two build packs and two environment variables.

After this is done you should be able to run the `scrape_stub.py` file directly on your Heroku instance. If you don't have the CLI tools installed, this can be easily verified through the Heroku dashboard by clicking the _More_ button then selecting _Run console_. Then, in the dialog box send `python scrape_stub.py`. If you received the same success message as from running the command locally, then congratulations! We have one more step to set up before our bot is ready to let us know when a PS5 is on sale.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/7ce9i5udabzvw27whts7.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/7bikvb00khpea7zng9zx.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/48bfvbzhdm5qbwkk2pfs.png)

# Setting Up Scheduler Task in Heroku

The final step is to hook up the screen scraping code to be called on a regular interval. I assumed that the sites would be easy to bypass using various scraping techniques, but I found that this was not an easy task. Particularly Walmart, it seems that using Selenium and Chromium I could fake it out sometimes, with the assist of solving a captcha manually a few times, but it kept resetting itself so it would have to be checked. Fortunately, Amazon, BestBuy, and GameStop are not this sophisticated.

To set up the Heroku Scheduler, go to the _Overview_ tab and select the _Configure Add-ons_ link. Search for **Heroku Scheduler** and select _Heroku Scheduler_ and accept the dialog box that comes up. Click on the now available _Heroku Scheduler_ add-on and click the _Create job_ button 

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/vi3nwjb6r4ngu4kyl8xh.png)

# Conclusion

With our bot deployed and complete, all there is left to do is wait.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/p2912sgdar5drab0yg6g.jpg)

I received a message from Amazon that a PS5 was for sale, but it was from a secondary seller for almost $1800. Still excited to see it working!

Given our totally free solution, we will be notified if Amazon, Best Buy, Game Stop, or Target has an available unit once every ten minutes. In the future, it would not be hard to add more sites or even Xbox Series X.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/a7m2nx6d0z24007vfpdx.jpg)