Processing Email - Single Script / Multiple Emails
Tags: google, python, api, email
Published: 2021-02-08 04:45:13

Part 6 in a series of articles on implementing a notification system using Gmail and [Line Bot](https://notify-bot.line.me/doc/en/)


Greetings. In this article I will be going over the thought process of how I handled multiple emails in a single script.  
In most cases there will only be a single email that needs to be processed at any given time. But to allow for more flexibility and to have a single place of control, and unfortunately complexity, I have decided to use a single script.

## Some background
The services I receive email from have some well defined static attributes.
- Sending Address
- Subject Line

About once a month they will also send billing information using the same *Sending Address* which can be ignored by the script.  
Because of this; the logic will be something along the lines of.

1. Check the subject line. If it exist in the list of known subject lines process the email.
2. If 1 true. Double check that the sending address is an approved/known sending address.
3. If 1 is false. Log an issue (This should not occur due to the filtering which is being done within Gmail.)
4. If 1 is true, but the sending address is false. Log the message. 
5. If both 1 and 2 are true. This is a valid notification email. 
     1. Determine which notification is being handled using the *Sender's Address*.
     1. Apply the correct regex expression(s) to get the information
     1. Send the notification and log its successful handling.
     1. Attach the *notified* label to the email.

## Organization

### constants.py
```python
SUB_1 = "something"
SUB_2 = "something"

SUBJECTS = [SUB_1, SUB_2]

FROM_BUS = "<email address of sender>"
FROM_KIDZDUO = "<email address of sender>"
```

### gmail.py

I have intentionally used a somewhat defensive coding style here.

### The code look something like this
```python
def handle_each_email(service, message_id, logger) -> tuple:
    data = notifier = None
    single_email = get_message(service, message_id, logger)
    # Check the subject is an expected notification subject line
    subject = single_email.get("subject")
    if subject in constants.SUBJECTS:
        # workout which notification we are dealing with and use the correct
        # regular expression string
        sender = single_email.get("from")
        email_body = single_email.get_content()
        if sender == constants.FROM_BUS:
            data = patterns.findMatches(email_body,
                                        patterns.BUS_DATA)
            datetime = patterns.findMatches(email_body,
                                            patterns.BUS_DATE_TIME)
            # Merge data and datetime into a single dictionary
            data.update(datetime)
            notifier = "BUS"
        elif sender == constants.FROM_KIDZDUO:
            data = patterns.findMatches(email_body,
                                        patterns.KIDZDUO_ENTEREXIT)
            notifier = "KIDZDUO"
        else:
            # This needs to be logged. Means failed to match sender.
            # if notifier is None:
            logger.warning(f"Failed to process message_id: {message_id} "
                           f"Matched Subject: {subject} "
                           f"Sender not matched: {sender}")
            pass
        return notifier, data
    else:
        # Not an expected subject line. Ignore this email
        logger.info("This is not a notifcation.")
        logger.info(f"sender: {sender}\n\t")
        logger.info(f"subject: {subject}")
        return notifier, data
```

### The break down

1. First we set *data* and *notifier* to *None*
2. Get the full email
3. Get the subject line
4. Check that the subject line is one of the expect subject strings
5. Get the sending address and email body
6. Check who the sender is:
     1. If it's the bus company. set *data* using the bus regex and set *notifier* to "BUS"
     2. If it's Kidsduo. Set the *data* using the kidzduo regex and set *nofifer* to "KIDZDUO"
     3. Otherwise log the warning and return *notifier* and *data*
7. If item 4 above is false, log an info message and return *notifier* and *data* both of which should be *None*

### Main loop
Let's see how we will process this in a batch form if needed.

```python
for message_id in list_of_message_ids:
    processed = False
    notifier, data = handle_each_email(service, message_id, logger)
    # Notifier tells us how the data dict is structured
    if notifier == "BUS":
        logger.debug("Bus")
        # Your notification / bot code here
        processed = True
    elif notifier == "KIDZDUO":
        logger.debug("KidsDuo")
        # Your notification / bot code here
        processed = True
    elif notifier is None and data is not None:
        logger.warning(f"Subject matched but From was not matched")
    elif notifier is None and data is None:
        logger.info(f"Non-Notification email from expected sender")
    else:
        # We should not get here. But log it.
        logger.warning(
                       f"Something went wrong. Unexpected match."
                       f"Don't know how to handle data."
                      )
    if processed:
        # Mail was processed. Add label so it's not processed again
        # Gmail only allows for the shortest time interval of one day.
        logger.debug("adding label")
        add_label_to_message(service, message_id, secrets.LABEL_ID)
# End of the program
logger.info("Ending cleanly")

```

Note that there are several checks that should probably be made. The return of `add_label_to_message()` for example.

I will be making the code available in GitHub later. Along with the *systemd* configuration I used  to have the script run at select times of the day and week. Watch for the future post on this.