AWS Lambda + Selenium in 2023
Published: 2023-07-24 10:38:11

There are a ton of resources online explaining how to do this, but they leave out a key detail: It only works with Python 3.7 and Selenium 3.8.0. You will need to use Lambda Layers because Selenium+Chromium are too big on their own. I'll try to keep this short.

NOTE: If you prefer video, here is where I got the info from: https://www.youtube.com/watch?v=b49Y3NGJX68

**Download Selenium, Chromium, Headless Chromium**
You need 3 packages: Selenium, Chromium, and Headless Chromium. You can do this with commandline, but I'm just going to link to the zip files. Grab the files below first and I'll show you how to rezip everything.

chromedriver: https://chromedriver.storage.googleapis.com/2.37/chromedriver_linux64.zip

headless-chromium: https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-41/stable-headless-chromium-amazonlinux-2017-03.zip

Selenium 3.8.0: https://files.pythonhosted.org/packages/1a/7d/1e4aadfffc9c0de596839347e29a799c4124987ec6cdb3bcb46fa1970ffa/selenium-3.8.0.tar.gz

You will take Chromium and Headless Chromium and put them into a single zip file. Make sure you use max compression settings to get it under the Lambda file size limit. The zip file should be structured like this:

```
chrome.zip
/headless-chromium
/chromedriver
```

Then you'll take Selenium out of the tar.gz. Create a folder called /python/lib/python3.7/site-packages/selenium

Your zip should be like this
```
selenium.zip
/python/lib/python3.7/site-packages/selenium/__init__.py
/python/lib/python3.7/site-packages/selenium/common/

...etc...
```

**Create Selenium Lambda Layer**
Name: selenium
Compatible runtimes: Python 3.7
Compatible Architecture: x86_64
Upload: selenium.zip (the one we created above)

**Create Chrome Lambda Layer**
Name: chrome
Compatible runtimes: Python 3.7
Compatible Architecture: x86_64
Upload: chrome.zip (the one we created above)

**Create Lambda Function**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9ow3ku0r39uc5br4ulcx.png)

Function name: [anything you want]
Runtime: Python 3.7
Architecture: x86_64

```python
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

def lambda_handler(event, context):
    options = Options()
    options.binary_location = '/opt/headless-chromium'
    options.add_argument('--headless')
    options.add_argument('--no-sandbox')
    options.add_argument('--single-process')
    options.add_argument('--disable-dev-shm-usage')

    driver = webdriver.Chrome(executable_path='/opt/chromedriver', chrome_options=options)
    driver.get('https://www.example.com/')
    
    title = driver.title;

    driver.close()
    driver.quit()

    response = {
        "statusCode": 200,
        "title": title
    }

    return response
```
Go to the Layers section of this function and add both the Selenium layer and Chrome layer we created above by referencing the ARN number for each layer.

Now go to the Configuration Tab, then General Configuration, then Edit. Set memory to 1000mb. Set timeout to 1min. You can fine tune this later.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kbq2dasbjhma3fkyljw5.png)

Run test and it should be running.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bc1bf9o0cipt5kpu7dh0.png)