Read data from Gmail using Node JS
Tags: node, webdev, devops, javascript
Published: 2021-04-11 10:41:35

I've struggled a lot with this topic. Finally, after a lot of searches and referring documentation, I'm finally writing this article. I hope it helps everyone. In this article, the code will be written in NodeJS. This app will include 2 parts, the first is creating and setting up a project on google console and the next is the coding part.
## 1) Creating a project on Google Console
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7fpkslwr3fn40ibyyxyh.png)
Open [https://console.developers.google.com/](https://console.developers.google.com/) website and you need to create a project. On the top left corner, there will be a list of your previous projects or option **select a project**. 
A modal will pop up. Select on the option **NEW PROJECT**.
* **select -> NEW PROJECT**
Enter a project name and create the project
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ya8ehrvc4yuosfschls0.png)

## 2) Enable Gmail API
Ensure that you've selected that project, then in the search box search Gmail API. Enable Gmail API.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tpgvh8lqfeu0eldgcqve.png)

## 3) Configure Project
You need credentials but before creating credentials you need to configure credentials. On the left-hand side, you'll find OAuth consent Screen. Click on it. 
* **User Type -> External**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p75cbdhv49hexby5pldd.png)
* Enter app information ie, app name, supporting email, developer contact information.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oyqauv7o3enn7mggn9b2.png)
* Scope page **save and continue**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zud98qzfjizmbpqykvkm.png)
* Test User Tab: Click on add user and you can add up to 100 emails. Add your email for testing. Save and continue.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ifh4658cph2rrsybtxhm.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n7x3nnhkvaw9wy93n6gy.png)

* Finally, after setting up, click on Credentials.



## 4) Create Credentials
After landing on Credentials, on the top click on **CREATE CREDENTIALS**. Click on OAuth client ID. Select your application type. As we're using NodeJS, it is a web application. Add URI's as
http://localhost:3000. Create and you'll get your credentials.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nd0ph24wfrp8wqoecitd.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1lt42zg4enr91vyki5n3.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6ih3284g7mbo692o84nl.png)

Now comes part two, the coding part

## 5) Code set up:
Go to this page, [Node JS quickstart]
(https://developers.google.com/gmail/api/quickstart/nodejs).
* Copy Paste the sample code and then copy-paste in a file named index.js. 
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ti4ty3pvm2vww9yjqdmb.png)

* In the folder, where you created this file, the terminal add the command ``` npm init ``` and initializes ``` package.json ``` 

* You need to install some dependencies with the command  ``` npm i googleapis cheerio mailparser js-base64 open ``` 

* Go to google developers console in your project. Navigate to the credentials part. In OAuth 2.0 Client IDs, you'll find a small download icon, download your credentials file from there and add to your folder where you've created this project. Name this file as ``` credentials.json ```

* Run your code in your terminal. When you run for the first time, you'll get something like this

```
Authorize this app by visiting this url: https://accounts.google.com/o/oauth2/v2/auth?access_type=offline&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fgmail.readonly&response_type=code&client_id=479559853488-050lms0ffusprdhh938s954q0s36kg4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob
‍Enter the code from that page here:
```

Click on that URL and enter the code. 
Now, in order to be able to manage the labels of the messages, you need to change the initial scope from *gmail.readonly* to *gmail.modify*.

``` 
const SCOPES = ['https://www.googleapis.com/auth/gmail.modify'];
```

Delete **token.json** from your working directory.

One error that some of you might get. The reference code has *credentials.installed* but it should be *credentials.web*. When you check the file, *credentials.json* you'll find everything is inside *web* object. So, if you get that error just check your *creddentials.json* file once and replace *installed* accordingly.

```
function authorize(credentials, callback) {
  const {client_secret, client_id, redirect_uris} = credentials.installed;
...
}
```
* If you're facing any other issue regarding app verification then navigate to [verification process](verification process) and follow them.

## 6)Final Output:
Then, go to the *listLabels* function and add the label id in the console log.
```
functionlistLabels(auth) {  
  ...      
    labels.forEach((label) => {        
      console.log(`- ${label.name} : ${label.id}`);      
    });  
  ...}

```
Finally, run the application by using the command 
``` 
npm run server
 ``` to authorize the app again and update the access permissions. You should be able to see the label list but now each label with its corresponding id.

Now, to read mail box, 
```
function listMessages(auth, query){
  query = 'unnatibamania8@gmail.com';
  return new Promise((resolve, reject) => {    
    const gmail = google.gmail({version: 'v1', auth});
    gmail.users.messages.list(      
      {        
        userId: 'me',  
        q:query,      
        maxResults:5     
      },            (err, res) => {        
        if (err) {                    reject(err);          
          return;        
        }        
        if (!res.data.messages) {                    resolve([]);          
          return;        
        }                resolve(res.data);  

                         getMail(res.data.messages[0].id, auth);
      }    
    );  
  })
}


```

Here, the function *listMessages* takes 2 parameters auth and query, you can call this function 
```
fs.readFile('credentials.json', (err, content) => {
  if (err) return console.log('Error loading client secret file:', err);
  // Authorize a client with credentials, then call the Gmail API.
  authorize(JSON.parse(content), listMessages);
  
});

```
after console logging *res.data.messages*, you'll get threadId and id. Pass the id into *getMail* function. 

```
function getMail(msgId, auth){
  console.log(msgId)
  const gmail = google.gmail({version: 'v1', auth});
  //This api call will fetch the mailbody.
  gmail.users.messages.get({
      userId:'me',
      id: msgId ,
  }, (err, res) => {
    console.log(res.data.labelIds.INBOX)
      if(!err){
        console.log("no error")
          var body = res.data.payload.parts[0].body.data;
        
          var htmlBody = base64.decode(body.replace(/-/g, '+').replace(/_/g, '/'));
          console.log(htmlBody)
          var mailparser = new Mailparser();

          mailparser.on("end", (err,res) => {
              console.log("res",res);
          })

          mailparser.on('data', (dat) => {
              if(dat.type === 'text'){
                  const $ = cheerio.load(dat.textAsHtml);
                  var links = [];
                  var modLinks = [];
                  $('a').each(function(i) {
                      links[i] = $(this).attr('href');
                  });

                  //Regular Expression to filter out an array of urls.
                  var pat = /------[0-9]-[0-9][0-9]/;
                  
                  //A new array modLinks is created which stores the urls.
                  modLinks = links.filter(li => {
                      if(li.match(pat) !== null){
                          return true;
                      }
                      else{
                          return false;
                      }
                  });
                  console.log(modLinks);

                  //This function is called to open all links in the array.
                  
              }
          })

          mailparser.write(htmlBody);
          mailparser.end();
          
      }
  });
}


```

### Final Output
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uq72d5x0sdnuhndzblav.png)
Don't read all the messages all together it might crash the application. Hence, I've read only 1 mail at a time. You can just play around with the code and find a lot of other stuff!
I hope this article helps.