Automate emails using Google API
Tags: automation, node, email, google
Published: 2021-06-05 01:42:07

The Gmail API provides a neat way to automate email tasks such as reading and sending among others. This piece illustrates how you could read emails and create and send them, all automatically. 

Automated emails could be used as a part of a larger process of automating routine maintenance tasks for example. In my case, the intent is to download a CSV file which is sent in an email, process it and send the result as an email. 

###Steps:
1. Set-up your system to work with the Gmail API
2. Read an email and download CSV from it 
3. Create email body and send email to concerned recipient


####First, Set-up the Gmail API

You can read about all of it from [here](https://developers.google.com/gmail/api/guides). There are language-specific guides to enable the Gmail API on your Google ID; I chose Node. 

Open the terminal in your machine or use a code editor (I am using VSCode). _Familiarity with node is assumed._

* Install the Google API library and required libraries:

```
npm install googleapis@39 --save
npm install fs readline
```

* Create an initial file with the code below. Call this file `index.js`

```
// index.js

const fs = require('fs');
const readline = require('readline');
const {google} = require('googleapis');
const SCOPES = [
    'https://www.googleapis.com/auth/gmail.readonly',
    'https://www.googleapis.com/auth/gmail.modify',
    'https://www.googleapis.com/auth/gmail.compose',
    'https://www.googleapis.com/auth/gmail.send'
  ];
const TOKEN_PATH = 'token.json';

// Load client secrets from a local file.
fs.readFile('credentials.json', (err, content) => {
  if (err) return console.log('Error loading client secret file:', err);
  // Authorize a client with credentials, then call the Gmail API.
  authorize(JSON.parse(content), getAuth);
});

/**
 * Create an OAuth2 client with the given credentials, and then execute the
 * given callback function.
 * @param {Object} credentials The authorization client credentials.
 * @param {function} callback The callback to call with the authorized client.
 */
function authorize(credentials, callback) {
  const {client_secret, client_id, redirect_uris} = credentials.installed;
  // console.log(redirect_uris);
  const oAuth2Client = new google.auth.OAuth2(
      client_id, client_secret, redirect_uris[0]);

  // Check if we have previously stored a token.
  fs.readFile(TOKEN_PATH, (err, token) => {
    if (err) return getNewToken(oAuth2Client, callback);
    oAuth2Client.setCredentials(JSON.parse(token));
    callback(oAuth2Client);
  });
}

/**
 * Get and store new token after prompting for user authorization, and then
 * execute the given callback with the authorized OAuth2 client.
 * @param {google.auth.OAuth2} oAuth2Client The OAuth2 client to get token for.
 * @param {getEventsCallback} callback The callback for the authorized client.
 */
function getNewToken(oAuth2Client, callback) {
  const authUrl = oAuth2Client.generateAuthUrl({
    access_type: 'offline',
    scope: SCOPES,
  });
  console.log('Authorize this app by visiting this url:', authUrl);
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });
  rl.question('Enter the code from that page here: ', (code) => {
    rl.close();
    oAuth2Client.getToken(code, (err, token) => {
      if (err) return console.error('Error retrieving access token', err);
      oAuth2Client.setCredentials(token);
      // Store the token to disk for later program executions
      fs.writeFile(TOKEN_PATH, JSON.stringify(token), (err) => {
        if (err) return console.error(err);
        console.log('Token stored to', TOKEN_PATH);
      });
      callback(oAuth2Client);
    });
  });
}

function getAuth(auth) {
    //var check = new Check(auth);
  
    console.log("Auth'ed")
  }

```

#####Let's see what's happening here.
1. 'Require' the libraries 
2. Declare the scopes: these are the activities we are going to use the Gmail API for. There is a list of them in the documentation linked before.
3. Read the credentials for the Gmail account that is going to be used to send and read the emails. The `credentials.json` file is derived from Google Cloud Platform. 
4. The following lines use the credentials from step 3 and then create an OAUTH2 client and authorize the client to perform the desired activity.

#####To create the `credentials.json` file:
1. Login to [GCP](https://console.cloud.google.com/) using your account
2. Create a project (Click the little downward arrow near the Google Cloud Platform logo on the left) 
3. Once the project is created, select it and then click Credentials on the sidebar.
4. From the top menu, select Create Credentials and OAuth Client ID from the resultant drop-down. You will have to configure a few things depending on your situation:
* An app name
* the scopes that you saw from the `index.js` file will have to be selected here also. For our needs, be sure to manually add the scopes if you don't see them in the table.  

![Screen Shot 2021-04-12 at 1.26.31 PM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/buj63qzrtkqooqqchjh9.png)

5. That didn't create a set of credentials if you have been following with the process. It just creates an application under the project. To create a set of credentials, select Create Credentials again. 
Then select the application type; I selected the Web application as I intended to use the email reader utility as a web app.

![Screen Shot 2021-04-12 at 1.29.47 PM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dzxe34nlfp6xj7ybjgmz.png)

Add information pertaining to this new set of credentials. Make sure all the fields are filled as they are required for the utility to work. The fields have helpful tool-tips to guide you.
 
![Screen Shot 2021-04-12 at 1.32.31 PM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ukhbagz65w5mfctwkusr.png)

Click Create. A pop-up with the Client ID and Client Secret shows up. You can close the pop-up and instead select the Download button on the row created on your Credentials page to download the credentials JSON file. 

![Screen Shot 2021-04-12 at 1.36.16 PM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/afk82o0tfkco8dv6ku3e.png)

Rename the file to `credentials.json` and move it to your project folder.  

My file looks like this:
![Screen Shot 2021-04-12 at 1.41.21 PM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wmxnpdhnpbmenpgb7ezi.png)
 
With that out of the way, you can now test the Gmail API setup. 
[UPDATE: You will have to rename "web" in the credentials file to "installed"]

* Run the file 
```
node index.js
```
Follow the instructions on the terminal. You will have to grant access to the email ID you want to use and then get the code from the resultant page (or address bar) and paste that on the terminal. (Side note: This took me a while to figure out but look out for a code in the aforementioned places and you can proceed).

Once authentication is complete, you should see something like this: 

![Screen Shot 2021-04-12 at 1.51.23 PM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b4oe2agxwo2vg1uu4pqi.png)

A file `token.json` is now created in your folder which will be used hereafter by your application to read emails and send them. 

#### Second, Read emails

* Install the necessary libraries
```
npm install js-base64 cheerio open dotenv https fs mailparser
```

* Create another file `readMail.js` 

```
// readEmail.js

const {google} = require('googleapis');
var base64 = require('js-base64').Base64;
const cheerio = require('cheerio');
var open = require('open');
const dotenv = require('dotenv');
const https = require('https');
const fs = require('fs');
var Mailparser = require('mailparser').MailParser;

dotenv.config();

class Check{

    //auth is the constructor parameter.
    constructor(auth){
        this.me = process.env.GMAIL_USER;
        this.gmail = google.gmail({version: 'v1', auth});
        this.auth = auth;
    }

    //Check for emails
    checkForEmails(){
        var query = "from:support@example.com is:unread";
        // console.log(this.me);
        this.gmail.users.messages.list({
            userId: this.me,
            q: query 
        }, (err, res) => {
            if(!err){
                //mail array stores the mails.
                var mails = res.data.messages;
                // console.log(mails);
                this.getMail(mails[0].id);
                // console.log(mails[0].id)
            }
            else{
                console.log(err);
            }
        });        
    }

    // read mail 
    getMail(msgId){
        //This api call will fetch the mailbody
        this.gmail.users.messages.get({
            userId: this.me,
            id: msgId
        }, (err, res) => {
            if(!err){
                // console.log(res.data.payload);
                var body = res.data.payload.body.data;
                var htmlBody = base64.decode(body.replace(/-/g, '+').replace(/_/g, '/'));
                // console.log(htmlBody);
                var mailparser = new Mailparser();

                mailparser.on("end", (err,res) => {
                    if(err) {
                        console.log(err);
                    }
                })

                mailparser.on('data', (dat) => {
                    if(dat.type === 'text'){
                        const $ = cheerio.load(dat.textAsHtml);
                        var links = [];
                        // Get all links in the HTML
                        $('a').each(function(i) {
                            links[i] = $(this).attr('href');
                        });
                        
                        console.log("Email read!");
                        // You can further process the email and parse for necessary information
                    }
                })

                mailparser.write(htmlBody);
                mailparser.end();
            }
        });
    }

module.exports = Check;
```

What is happening here? 
1. `Require` the libraries
2. Initialize a `.env` file which stores your Gmail Username and password. This is then used in the Check class constructor. 
3. Unread emails from the address `support@example.com` are checked for. Replace this with the sender whose emails you want to read. Here, the first email (the latest) will be read.
4. The mail body of the first email is read. You can parse and process this mail body. 

But how would you run this file? Make a few changes to the `index.js` file. The updated code is this:

```
// index.js

const fs = require('fs');
const readline = require('readline');
const {google} = require('googleapis');
const Check = require('./readEmail');
const SCOPES = [
    'https://www.googleapis.com/auth/gmail.readonly',
    'https://www.googleapis.com/auth/gmail.modify',
    'https://www.googleapis.com/auth/gmail.compose',
    'https://www.googleapis.com/auth/gmail.send'
  ];
const TOKEN_PATH = 'token.json';

// Load client secrets from a local file.
fs.readFile('credentials.json', (err, content) => {
  if (err) return console.log('Error loading client secret file:', err);
  // Authorize a client with credentials, then call the Gmail API.
  authorize(JSON.parse(content), getAuth);
});

/**
 * Create an OAuth2 client with the given credentials, and then execute the
 * given callback function.
 * @param {Object} credentials The authorization client credentials.
 * @param {function} callback The callback to call with the authorized client.
 */
function authorize(credentials, callback) {
  const {client_secret, client_id, redirect_uris} = credentials.installed;
  // console.log(redirect_uris);
  const oAuth2Client = new google.auth.OAuth2(
      client_id, client_secret, redirect_uris[0]);

  // Check if we have previously stored a token.
  fs.readFile(TOKEN_PATH, (err, token) => {
    if (err) return getNewToken(oAuth2Client, callback);
    oAuth2Client.setCredentials(JSON.parse(token));
    callback(oAuth2Client);
  });
}

/**
 * Get and store new token after prompting for user authorization, and then
 * execute the given callback with the authorized OAuth2 client.
 * @param {google.auth.OAuth2} oAuth2Client The OAuth2 client to get token for.
 * @param {getEventsCallback} callback The callback for the authorized client.
 */
function getNewToken(oAuth2Client, callback) {
  const authUrl = oAuth2Client.generateAuthUrl({
    access_type: 'offline',
    scope: SCOPES,
  });
  console.log('Authorize this app by visiting this url:', authUrl);
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });
  rl.question('Enter the code from that page here: ', (code) => {
    rl.close();
    oAuth2Client.getToken(code, (err, token) => {
      if (err) return console.error('Error retrieving access token', err);
      oAuth2Client.setCredentials(token);
      // Store the token to disk for later program executions
      fs.writeFile(TOKEN_PATH, JSON.stringify(token), (err) => {
        if (err) return console.error(err);
        console.log('Token stored to', TOKEN_PATH);
      });
      callback(oAuth2Client);
    });
  });
}

function getAuth(auth) {
    var check = new Check(auth);
  
    console.log("Auth'ed");
    check.checkForEmails();
  }

```

* Run the file again 
```
node index.js
```

Output on the console:
![Screen Shot 2021-04-12 at 2.59.58 PM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8v81t2he6cqdtzqqkoaj.png)

You can do a number of things with the resultant mail body like parse it, get a download link etc. 

Kudos on getting here! Now for the last part: create an email and send it! 

#####Third, Compose email and send 

* Install the library
```
npm install nodemailer
```

* Create a file `composeEmail.js` and copy this code : 
```
// composeEmail.js

const {google} = require('googleapis');
const mailComposer = require('nodemailer/lib/mail-composer');
var base64 = require('js-base64').Base64;
const dotenv = require('dotenv');
dotenv.config();

class CreateMail{
    
    constructor(auth, to, sub, body){
        this.me = process.env.GMAIL_USER;    
        this.auth = auth;
        this.to = to;
        this.sub = sub;
        this.body = body;
        this.gmail = google.gmail({version: 'v1', auth});
    }

    // Construct the mail
    makeBody(){
		
        let mail = new mailComposer({
            to: this.to,
            text: this.body,
            subject: this.sub,
            textEncoding: "base64"
        });
		
	//Compiles and encodes the mail.
	mail.compile().build((err, msg) => {
	if (err){
		return console.log('Error compiling email ' + error);
		} 
		
	const encodedMessage = Buffer.from(msg)
			  .toString('base64')
			  .replace(/\+/g, '-')
			  .replace(/\//g, '_')
			  .replace(/=+$/, '');
			
			
        this.sendMail(encodedMessage);
      });
	   }

    //Send the message to specified receiver
    sendMail(encodedMessage){
        this.gmail.users.messages.send({
            userId: process.env.GMAIL_USER,
            resource: {
                raw: encodedMessage,
            }
         }, (err, result) => {
            if(err){
                return console.log('NODEMAILER - Returned an error: ' + err);
            }
                    
            console.log("NODEMAILER - Sending email reply:", result.data);
        });
    }
}

module.exports = CreateMail;
```

What's happening here ? 
1. 'Require' the libraries
2. Construct the email body using base64 
3. Send the email using nodemailer to the recipient selected

But again, how would you run this? Let's update the file `readEmail.js` to call `composeEmail.js`. The final code for `readEmail.js` is below:

```
const {google} = require('googleapis');
var base64 = require('js-base64').Base64;
const cheerio = require('cheerio');
var open = require('open');
const dotenv = require('dotenv');
const https = require('https');
const fs = require('fs');
var Mailparser = require('mailparser').MailParser;
const Email = require('./composeEmail');

dotenv.config();

class Check{

    //auth is the constructor parameter.
    constructor(auth){
        this.me = process.env.GMAIL_USER;
        this.gmail = google.gmail({version: 'v1', auth});
        this.auth = auth;
    }

    //Check for emails
    checkForEmails(){
        var query = "from:support@figma.com is:unread";
        // console.log(this.me);
        this.gmail.users.messages.list({
            userId: this.me,
            q: query 
        }, (err, res) => {
            if(!err){
                //mail array stores the mails.
                var mails = res.data.messages;
                // console.log(mails);
                this.getMail(mails[0].id);
                // console.log(mails[0].id)
            }
            else{
                console.log(err);
            }
        });        
    }

    // read mail 
    getMail(msgId){
        //This api call will fetch the mailbody
        this.gmail.users.messages.get({
            userId: this.me,
            id: msgId
        }, (err, res) => {
            if(!err){
                // console.log(res.data.payload);
                var body = res.data.payload.body.data;
                var htmlBody = base64.decode(body.replace(/-/g, '+').replace(/_/g, '/'));
                // console.log(htmlBody);
                var mailparser = new Mailparser();

                mailparser.on("end", (err,res) => {
                    if(err) {
                        console.log(err);
                    }
                })

                mailparser.on('data', (dat) => {
                    if(dat.type === 'text'){
                        const $ = cheerio.load(dat.textAsHtml);
                        var links = [];
                        var modLinks = [];
                        // Get all links in the HTML
                        $('a').each(function(i) {
                            links[i] = $(this).attr('href');
                        });
                        
                        console.log("Email read!");
                    }
                })

                mailparser.write(htmlBody);
                mailparser.end();

                // Finally send the email 
                this.sendEmail("This is where the email's body goes.");
            }
        });
    }

    sendEmail(mail_body) {    
        var makeEmail = new Email(this.auth, <recipient_email_address>, 'Test subject', mail_body);

        // send the email
        makeEmail.makeBody();
    }
}

module.exports= Check;
```
_Remember to replace the recipient's address in the sendEmail function above_ 

* Now run `index.js`
```
node index.js
```

This in turn runs `readEmail.js` which lastly runs `composeEmail.js`. Phew!

My console output looked like this:
![Screen Shot 2021-04-12 at 3.16.56 PM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bimvrfxhqca5o9kh5mh1.png)

And finally, the email!

![Screen Shot 2021-04-12 at 3.18.47 PM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kuv943lqi3sf1tenc6yp.png)

Automating emails has many use cases and I hope this helped. Thanks and feedback is welcome!