Using OTP sent via email in E2E UI tests.
Tags: testing, java, tutorial, python
Published: 2023-01-24 06:09:41

## Background
When implementing end-to-end automated testing of an application with one-time password  (OTP) authentication, we need to establish a way to test the entire flow of OTP delivery to the user.
Here I provide two classes that are ready to be used in this kind of test to receive OTP via email sent by the backend and a step-by-step guide on integrating into your framework.

## Constraints
To implement this approach, the following conditions must be met:
* The QA team manages a dedicated email account.
* Emails delivering OTP have a known constant Subject text
* A known constant phrase precedes OTP in the body of the email
* The OTP length is fixed

## Algorithm
Along execution, an autotest: 
1. Creates and keeps an instance of the Email Provider service based on the configuration settings (Email Provider API credentials)
2. Gets and keeps a pointer to the last received email with a subject that is set for OTP emails (or retains null if no one was received before)
3. Triggers the OTP generation and delivery
4. Waits for email delivery to the Email Provider
5. Gets the email message
6. Parses OTP from the message
7. Uses OTP to verify login to the application under test

Steps 1-2 and 4-6 are implemented in the code provided.

## EmailProviderHandler interface
To make the approach flexible, the `EmailProviderHandler` interface has been declared. The implementation must provide the following:
- a service initiation to expect emails with specified subject
- checking if a new email with the subject has already been received
- getting the message from the received email

{% details *Interface* %} 

```java
interface EmailProviderHandler {
    void init(String emailSubject);
    boolean isEmailReceived();
    String getMessage();
}
```

{% enddetails %}

I have implemented the interface for Gmail (see below), but you can implement it for another provider you use.

## EmailedOTPHandler class
To process emails with OTP, use the `EmailedOTPHandler` class.
An EmailedOTPHandler instance should be created for a specific combination of an email Subject, a passphrase followed by an OTP in the email body, and an OTP length. To give the handler a tool to access emails on a particular Email Provider, we should inject an instance of the implemented `EmailProviderHandler`.

{% details *Class variables list* %} 

```java
    private final String emailSubject; // Subject pattern of the emails containing the OTP
    private final String otpKeyPhrase; // Phrase that precedes the OTP in the email body
    private final int otpLength; // Length of the OTP
    private final EmailProviderHandler emailProvider; // Gmail service
```

{% enddetails %}
 
Before using an `EmailedOTPHandler` instance for getting OTP, you need to initiate it using the `init()` method. 
Then you can trigger OTP generation and delivery by either mimic user login behavior through your application's UI or by querying the BE endpoint.
To get the OTP from an email, use the `getOTPEmailSent()` method. The method waits for a new email with Subject set and then tries to parse the OTP from it.
If there is no new message within the time period, NULL is returned.

The complete class code is shown [below](#emailedotphandler).

## GmailHandler class
`GmailHandler` implements `EmailProviderHandler` to handle a Gmail service through API. 
The methods described in the [Google Gmail Java Quick Start](https://developers.google.com/gmail/api/quickstart) guide are used to start the Gmail service and get credentials.

On the first Gmail API call, `GmailHandler` creates a credential file in the project to authenticate all future calls to the Gmail service (see detailed description [below](#add-json-client-id-file)).

The complete class code is shown [below](#gmailhandler).

## Repository

[Java](https://github.com/vishin-andrey/OTP-handler) (used in this article)
[Python](https://github.com/vishin-andrey/otp-handler-python)

## Notice

* To enable Java `assert` validation,  use the JVM parameter `-ea`

{% details IntelliJ IDEA %}

![IntelliJ IDEA interface for setting the JVM parameters](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zh7gvcftxk9z95k33neh.png)

{% enddetails %}

* Before using the class, you must enable and configure the API for your Gmail account, as shown [below](#how-to-set-up-the-gmail-account-api).

* `GmailHandler` extracts OTP from the email snippet. If your OTP in the email body is too far from the beginning and therefore not included in the snippet, use `getPayload()` instead of `getSnippet()`.

## How to set up the Gmail account API
Before moving on, you must activate and configure the API for the Gmail account you will be using to receive OTP emails. Using the [Google Cloud Console](https://console.cloud.google.com/) follow the steps below.

#### Register a new project

{% details Step by step %} 

* Click on CREATE PROJECT.

![Adding a new project in the Google Cloud Console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cuic3ksh7bfqcxyoucbf.png)

* Then give your project a name.

![Setting the project name](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7rjx5d9e8ky7sny9feed.png)

{% enddetails %}

#### Enable API

{% details Step by step %} 

* Click on ENABLE APIS AND SERVICES.

![Enabling APIs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rceb36estm0gjegi78g1.png)

* Search for Gmail in the API Library.

![Searching for Gmail API](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5g84okq1lrivovwb9q1y.png)

* Enable Gmail API

![Enabling Gmail API](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xlt9lfph4dx8efjkzpqr.png)

{% enddetails %}

#### Create credentials for autotests to access your Gmail account

{% details Step by step %} 

* Click on CREATE CREDENTIALS.

![Start the credentials creating](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0to4156boduxjdmm46oc.png)

* Choose for Gmail API a `User data` type.

![Setting the data type](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nqml60ezt25ie819k0jc.png)

* Customize the OAuth Consent Screen - enter any name for the app and add your contact email address

![Customizing the OAuth Consent scr](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6237zf0vd8rwjppz6j4d.png)

* Set the scope
It make sense to choose the `read only` scope

![Scope setting](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wdy1pbviwdlir0kj3d67.png)

![Scope setting 2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7izys9bhdu750rsdr63o.png)

* Choose the `Desktop app` application type and give it a name

![Choosing the app type](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dv1xpfn3xi5vqw0uwnu0.png)

* Your credentials have been created; you need to download the Client ID file in JSON format.

![Credentials created scr](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ikwzjgktw2j0orso8bvn.png)

* You can also customize your credentials at any time in the Credentials tab and then download the updated JSON file.

![Customize credentials](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ngy8ibr01ls83e3dton7.png)

![Customize credentials 2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s6na8ivc3rarcr0x9vex.png)

{% enddetails %}

#### Register a trusted test user

{% details Step by step %} 

* Navigate to the OAuth consent screen tab and click on `ADD USERS`

![Add test user step 1 scr](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lccybxoxman269bidfmo.png)

* Add your any real Gmail account email address. You will need to act under this account later to verify access for the Client ID

![Add test user step 2 scr](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4onacy77d16gea850vns.png)

{% enddetails %}

## How to add the Gmail account credentials to the project
After receiving the Client ID file in JSON format (as shown above), you must exchange it for the `StoredCredential` file the first time you call the Gmail API.

#### Add JSON Client ID file 

{% details Step by step %} 

* Put the file into `src/main/resources/credentials`.

![JSON Client ID file in the project structure](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t47rol8ailyqaqbpozg9.png)

{% enddetails %}

#### Verify the autotest access to the account on Gmail 

{% details Step by step %} 
Run your project first time. At the first call to the Gmail API, a browser will be opened by Google. Your should follow the Google dialog.

* Choose the account you add on the [Register a trusted test user](#register-a-trusted-test-user) step.

![Access verification - choose account alert](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uj8qaoq1etfkmopooi09.png)

* Click on `Continue` to verify the app

![Access verification - continue scr](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q17846jqktuxzjpwbxrp.png)

* Click on `Continue` to grant the access

![Access verification - granting access scr](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/deayf0bgpytwxb6sve98.png)

* Check for the confirmation

![Gmail confirmation scr](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6lq5g14lfiyjwkb97b26.png)

* Stop the first test project execution.

{% enddetails %}

#### Check for the StoredCredential file 

{% details Step by step %} 

* The `StoredCredential` file should already be created automatically in `src/main/resources/credentials` during your first Gmail API call; if it is not, repeat [this section](#how-to-add-the-gmail-account-credentials-to-the-project) again.

![StoredCredential file in the project structure](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ae2bbrwrzbvvqcf80mdk.png)

If you change the Gmail API configuration in the console in the future, you should delete the `StoredCredential` file and repeat these steps to add a new one.

{% enddetails %}

## Full code

### EmailedOTPHandler

{% details Code %}

```java
package gmail;

import static java.lang.Thread.sleep;

interface EmailProviderHandler {
    void init(String emailSubject);
    boolean isEmailReceived();
    String getMessage();
}

public class EmailedOTPHandler {
    private final String emailSubject; // Subject pattern of the emails containing the OTP
    private final String otpKeyPhrase; // Phrase that precedes the OTP in the email body
    private final int otpLength; // Length of the OTP
    private final EmailProviderHandler emailProvider; // Gmail service

    public EmailedOTPHandler(String emailSubject, String otpKeyPhrase, int otpLength, EmailProviderHandler emailProvider) {
        this.emailSubject = emailSubject;
        this.otpKeyPhrase = otpKeyPhrase;
        this.otpLength = otpLength;
        this.emailProvider = emailProvider;
    }

    public void init() {
        emailProvider.init(emailSubject);
    }

    /**
     * Parse and return OTP from the email with id = this.emailID
     */
    private String parseOTP() {
        String mailText = emailProvider.getMessage();
        // Parse OTP
        int pos = mailText.indexOf(otpKeyPhrase);   // Find the OTP key phrase
        assert pos != -1 : "OTP key phrase not found in the email";
        pos = pos + otpKeyPhrase.length();         // Move to the OTP start position
        return mailText.substring(pos, pos + otpLength);
    }

    /**
     * Trying to get a new email, checking for a new message every 5 sec for 6 times.
     * If gotten a new message, return the OTP from it.
     * If there is no new message during the time period, return NULL
     */
    public String getOTP() {
        String otp = null;
        for (int attempt = 0; attempt < 6; attempt++) {
            if (emailProvider.isEmailReceived()) {
                otp = parseOTP();
                break;
            }
            try {
                sleep(5000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        return otp;
    }
}

```

{% enddetails %}

### GmailHandler

{% details Code %}

```java
package gmail;

import com.google.api.client.auth.oauth2.Credential;
import com.google.api.client.extensions.java6.auth.oauth2.AuthorizationCodeInstalledApp;
import com.google.api.client.extensions.jetty.auth.oauth2.LocalServerReceiver;
import com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeFlow;
import com.google.api.client.googleapis.auth.oauth2.GoogleClientSecrets;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.util.store.FileDataStoreFactory;
import com.google.api.services.gmail.Gmail;
import com.google.api.services.gmail.GmailScopes;
import com.google.api.services.gmail.model.ListMessagesResponse;
import com.google.api.services.gmail.model.Message;

import java.io.*;
import java.nio.file.Files;
import java.security.GeneralSecurityException;
import java.util.*;

public class GmailHandler implements EmailProviderHandler{
    private static final String GMAIL_AUTHENTICATED_USER = "me";
    private final JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
    private static final String APPLICATION_NAME = "Gmail handler";
    private static final String PATH_TOKEN_DIRECTORY = System.getProperty("user.dir") + "/src/main/resources/credentials";
    private static final String PATH_CREDENTIALS_FILE = PATH_TOKEN_DIRECTORY + "/gmail_credentials.json";
    private Gmail gmailService;
    private final List<String> SCOPES = Arrays.asList(GmailScopes.MAIL_GOOGLE_COM);
    private String emailSubject;
    private String emailID; // ID of the last email with the provided Subject


    public void init(String emailSubject) {
        this.emailSubject = emailSubject;
        startService();
        emailID = getEmailID();
    }

    /**
     * Start a new Gmail service
     */
    public void startService () {
        final NetHttpTransport HTTP_TRANSPORT;
        try {
            HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
            gmailService =  new Gmail.Builder(HTTP_TRANSPORT, JSON_FACTORY, getCredentials(HTTP_TRANSPORT))
                    .setApplicationName(APPLICATION_NAME)
                    .build();
        } catch (IOException | GeneralSecurityException e) {
            throw new RuntimeException(e);
        }
    }

    /**
     * Creates an authorized Credential object.
     * This is the code from the Google Developer Console documentation.
     *
     * @param HTTP_TRANSPORT The network HTTP Transport.
     * @return An authorized Credential object.
     * @throws IOException If the credentials.json file cannot be found.
     */
    private Credential getCredentials(final NetHttpTransport HTTP_TRANSPORT) throws IOException {

        InputStream in = Files.newInputStream(new File(PATH_CREDENTIALS_FILE).toPath());
        if (in == null) {
            throw new FileNotFoundException("Resource not found: " + PATH_CREDENTIALS_FILE);
        }
        GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(JSON_FACTORY, new InputStreamReader(in));

        // Build flow and trigger user authorization request.
        GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(
                HTTP_TRANSPORT, JSON_FACTORY, clientSecrets, SCOPES)
                .setDataStoreFactory(new FileDataStoreFactory(new File(PATH_TOKEN_DIRECTORY)))
                .setAccessType("offline")
                .build();
        LocalServerReceiver receiver = new LocalServerReceiver.Builder().setPort(8888).build();
        return new AuthorizationCodeInstalledApp(flow, receiver).authorize("user");
    }

    /**
     * Return ID of the last email with subject = emailSubject
     * @return - last email id
     */
    public String getEmailID() {
        List<Message> listOfMessages;
        try {
            ListMessagesResponse response = gmailService.users().messages()
                    .list(GMAIL_AUTHENTICATED_USER)
                    .setQ("subject:" + emailSubject)
                    .execute();
            listOfMessages = response.getMessages();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        return (listOfMessages == null || listOfMessages.isEmpty()) ? null :  listOfMessages.get(0).getId();
    }

    /**
     * Return the email snippet text
     * @return - email snippet text as a String
     */
    public String getMessage() {
        Message message;
        try {
            message = gmailService.users().messages()
                    .get(GMAIL_AUTHENTICATED_USER, emailID)
                    .execute();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        return message.getSnippet(); // Use getPayload() instead if you want to get the full email body
    }

    /**
     * Check if there is a new email with subject = emailSubject
     * @return
     */
    public boolean isEmailReceived() {
        String newEmailID = getEmailID();
        if (newEmailID == null) {
            return false;
        }
        if (newEmailID.equals(emailID)) {
            return false;
        }
        emailID = newEmailID;
        return true;
    }
}
```

{% enddetails %}

### Test example

{% details Code %}

```java
import gmail.EmailedOTPHandler;
import gmail.GmailHandler;

public class EmailedOTPTest {

    static String subject = "OTP test";
    static String keyPhrase = "Your OTP is: ";
    static int otpLength = 6;

    public static void main(String[] args) {

        GmailHandler gmail = new GmailHandler();
        EmailedOTPHandler otpHandler = new EmailedOTPHandler(subject, keyPhrase, otpLength, gmail);

        otpHandler.init();
        // -> Here trigger the OTP email sending
        String otp = otpHandler.getOTP();

        assert otp != null : "No new email with subject = '" + subject + "' was received during the time period";
        System.out.println("OTP: " + otp);
    }

}

```

{% enddetails %}