AWS Lambda Retrieving Data From RDS (MySQL)
By: Michael Whittle
Tags: Nodejs, MySQL, Serverless, Lambda, Rds

We are going to create your Lambda function next and we’re going to need to install the “mysql” node module. There are several ways this can be done but I’m going to use the AWS Serverless Application Model (SAM) as covered in “Developing Lambda applications using the AWS Serverless Application Model (SAM) framework”.

If you don’t already have AWS SAM installed, the documented for your operating system here. This is an optional step as you could just upload a zip file with your project into the AWS Management Console. I just thought it would be useful to show you the less well-known way.

If you are using a Mac and don’t already have Homebrew installed you will need to install it. This is comparable to “yum” in Redhat/Centos, and “aptitude” in Ubuntu. As a Mac user this probably is “not your first rodeo” so you will have it installed. If so just make sure it is updated.

In a nutshell this is how you install Homebrew (brew)…

And if you already have it installed just update it as follows…

$ brew upgrade

You can confirm your Homebrew version like this.

% brew --version

Homebrew 2.5.2

Homebrew/homebrew-core (git revision 3be15; last commit 2020-09-26)

Homebrew/homebrew-cask (git revision 37b5d2; last commit 2020-09-27)

The next step is to install the AWS SAM CLI.

% brew tap aws/tap

% brew install aws-sam-cli

And confirm it is installed and working.

% sam --version

SAM CLI, version 1.3.2

If you already have the AWS SAM CLI installed, make sure it is up to date.

% brew upgrade aws-sam-cli

The next step is to initialise a SAM CLI project.

% sam init --runtime nodejs12.x

Which template source would you like to use?

1 - AWS Quick Start Templates

2 - Custom Template Location

Choice: 1 Project name [sam-app]: lambda-rds-mysql Cloning app templates from https://github.com/awslabs/aws-sam-cli-app-templates.git AWS quick start application templates:

1 - Hello World Example

2 - Step Functions Sample App (Stock Trader)

3 - Quick Start: From Scratch

4 - Quick Start: Scheduled Events

5 - Quick Start: S3

6 - Quick Start: SNS

7 - Quick Start: SQS

8 - Quick Start: Web Backend

Template selection: 3 -----------------------

Generating application:

-----------------------

Name: lambda-rds-mysql

Runtime: nodejs12.x

Dependency Manager: npm

Application Template: quick-start-from-scratch

Output Directory: . Next steps can be found in the README file at ./lambda-rds-mysql/README.md

Now we’ll enter the “lamda-rds-mysql” project directory and install our required node module for “mysql”.

% cd lambda-rds-mysql

lambda-rds-mysql % yarn add mysql *** INSTALLATION PROCESS ***

You will want to edit the “template.yml“ file.

Remove this:

# This is a Lambda function config associated with the source code: hello-from-lambda.js

helloFromLambdaFunction:

Type: AWS::Serverless::Function

Properties:

Handler: src/handlers/hello-from-lambda.helloFromLambdaHandler

Runtime: nodejs12.x

MemorySize: 128

Timeout: 100

Description: A Lambda function that returns a static string.

Policies:

# Give Lambda basic execution Permission to the helloFromLambda

- AWSLambdaBasicExecutionRole

And replace it with this:

# This is a Lambda function config associated with the source code: lambda-rds-mysql.js lambdaRDSMysqlFunction:

Type: AWS::Serverless::Function

Properties:

Handler: src/handlers/lambda-rds-mysql.lambdaRDSMysqlHandler

Runtime: nodejs12.x

MemorySize: 128

Timeout: 100

Description: A Lambda function that queries RDS MySQL.

Policies:

# Give Lambda basic execution Permission to the lambdaRDSMysql

- AWSLambdaBasicExecutionRole

Rename the “./src/handlers/hello-from-lambda.js“ to “./src/handlers/lambda-rds-mysql.js“ and replace the content with this.

var mysql = require('mysql'); var connection = mysql.createConnection({

host: "mediumtutorial.cy65mjdphub6.eu-west-1.rds.amazonaws.com",

user: "mediumtutorial",

password: "mediumtutorial",

database: "mediumtutorial",

}); exports.lambdaRDSMysqlHandler = (event, context, callback) => {

connection.query('show tables', function (error, results, fields) {

if (error) {

connection.destroy();

throw error;

} else {

console.log(results);

callback(error, results);

connection.end(function (err) { callback(err, results);});

}

});

}

It goes without saying that you need to replace the connection parameters with the relevant values for you. The “host” can be found by opening your MySQL instance in RDS.

This Lambda function is going to execute the “show tables” query. The problem is we don’t have any tables created so let’s do that now.

If you don’t already have a MySQL client installed you need to install it now. On my Mac I’ll install it with Homebrew but it is a very similar process on Linux with “yum” or “apt-get”. Or you can install a GUI for your operating system called MySQL Workbench which is very good and easy to use.

% brew install mysql-client *** INSTALLATION PROCESS *** ==> Caveats

mysql-client is keg-only, which means it was not symlinked into /usr/local, because it conflicts with mysql (which contains client libraries). If you need to have mysql-client first in your PATH run:

echo 'export PATH="/usr/local/opt/mysql-client/bin:$PATH"' >> ~/.zshrc For compilers to find mysql-client you may need to set:

export LDFLAGS="-L/usr/local/opt/mysql-client/lib"

export CPPFLAGS="-I/usr/local/opt/mysql-client/include" For pkg-config to find mysql-client you may need to set:

export PKG_CONFIG_PATH="/usr/local/opt/mysql-client/lib/pkgconfig"

Please note this doesn’t add it to your path automatically. Run the command in bold above and then either re-open your terminal window or run:

% source ~/.zshrc

It looks like it installed properly…

% which mysql

/usr/local/opt/mysql-client/bin/mysql % mysql --version

mysql Ver 8.0.21 for osx10.15 on x86_64 (Homebrew)

Let’s try and connect to our database now. If this doesn’t work make sure you did not include the “ --no-publicly-accessible “ argument when creating the database and make sure that the security groups are allowing your system to connect to the database host on TCP port 3306.

% mysql -u mediumtutorial -h mediumtutorial.cy65mjdphub6.eu-west-1.rds.amazonaws.com -P 3306 -D mediumtutorial -p

Enter password: <YourPassword!>

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 37

Server version: 8.0.20 Source distribution Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show tables;

Empty set (0.02 sec) mysql>

We don’t have any tables yet and this is what our Lambda function will query. So let’s create a basic table now.

mysql> CREATE TABLE names ( id INT(6) UNSIGNED AUTO_INCREMENT PRIMARY KEY, name VARCHAR(30) NOT NULL );

Query OK, 0 rows affected, 1 warning (0.05 sec) mysql> show tables;

+--------------------------+

| Tables_in_mediumtutorial |

+--------------------------+

| names |

+--------------------------+

1 row in set (0.02 sec) mysql>

Time to build our project using the SAM CLI.

% sam build

Building function 'lambdaRDSMysqlFunction'

Running NodejsNpmBuilder:NpmPack

Running NodejsNpmBuilder:CopyNpmrc

Running NodejsNpmBuilder:CopySource

Running NodejsNpmBuilder:NpmInstall

Running NodejsNpmBuilder:CleanUpNpmrc Build Succeeded Built Artifacts : .aws-sam/build

Built Template : .aws-sam/build/template.yaml Commands you can use next

=========================

[*] Invoke Function: sam local invoke

[*] Deploy: sam deploy --guided

What is quite strange and I haven’t quite figured out yet is why “sam local invoke” can’t query for the tables. The Lambda function runs with no errors but no results are logged. This function actually works in AWS. Maybe a bug with “sam local invoke”. If I figure out what is wrong I’ll include the solution here.

% sam local invoke

Invoking src/handlers/lambda-rds-mysql.lambdaRDSMysqlHandler (nodejs12.x)

Skip pulling image and use local one: amazon/aws-sam-cli-emulation-image-nodejs12.x:rapid-1.3.2. Mounting /Repos/Lambda/lambda-rds-mysql/.aws-sam/build/lambdaRDSMysqlFunction as /var/task:ro,delegated inside runtime container START RequestId: c2919edc-1054-144e-9308-f9f097e53753 Version: $LATEST END RequestId: c2919edc-1054-144e-9308-f9f097e53753 REPORT RequestId: c2919edc-1054-144e-9308-f9f097e53753 Init Duration: 275.11 ms Duration: 7.97 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 50 MB null

All we want to do here is deploy the Lambda function and node module dependencies.