AWS CodeCommit への Git Push 時の変更内容を、Lambda (Python スクリプト) で作成し、SNS 経由でメール通知してみる

# ０．はじめに

AWS CodeCommit を利用していたんですが、

* [Mac から AWS CodeCommit を使ってみる \- Qiita](https://qiita.com/kusokamayarou/items/531e4340046fa42110c8)
* [AWS CodeCommit の特定リポジトリへの専用ユーザーを作成してみる \- Qiita](https://qiita.com/kusokamayarou/items/a43faf4b7e264da4f5e0)
* [AWS CodeCommit から Bitrise に接続する \- Qiita](https://qiita.com/kusokamayarou/items/3b4f20bd47295ef56ca2)

Git Push した時の変更情報の通知が欲しかったので、試してみました。

大枠の流れは、以下。

![Qiita_画像_02.001.jpeg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/26e64505-7a82-33a6-9729-082c961a495c.jpeg)

# １．SNS のトピックを作成し、所定のメールアドレスを登録する。

基本的に、以下のページの手順に従って設定します。

* [\[基本操作\]Amazon SNSでメールを送信する ｜ DevelopersIO](https://dev.classmethod.jp/cloud/aws/amazon-sns-2017/)

1. SNS のトピックを作成します。設定は、以下。
    * トピック名  : ※任意
    * アクセスポリシー : ※以下参考。

    ```json
{
  "Version": "2008-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      "Sid": "__default_statement_ID",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": [
        "SNS:GetTopicAttributes",
        "SNS:SetTopicAttributes",
        "SNS:AddPermission",
        "SNS:RemovePermission",
        "SNS:DeleteTopic",
        "SNS:Subscribe",
        "SNS:ListSubscriptionsByTopic",
        "SNS:Publish",
        "SNS:Receive"
      ],
      "Resource": "arn:aws:sns:ap-northeast-1:XXXXXXXXXXXX:SysOps-Lambda-Mnt",
      "Condition": {
        "StringEquals": {
          "AWS:SourceOwner": "XXXXXXXXXXXX"
        }
      }
    }
  ]
}
    ```
    * ![FireShot Capture 110 - Simple Notification Service - ap-northeast-1.console.aws.amazon.com.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/0cedc12e-385a-d238-4eb7-49bdd6812e02.png)
<br><br>
1. 作成した SNS のトピックへ、所定のメールアドレスをサブスクリプションとして登録する。
    * ![FireShot Capture 111 - Simple Notification Service - ap-northeast-1.console.aws.amazon.com.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/30e17772-447b-dba7-c6b0-160aeb85aac5.png)
　
1. 登録後、作成した SNS のトピックへ、パブリッシュして SNS  トピックが正常に設定されているか、確認する。

# ２．Lambda ファンクションを作成する。

1. IAM ロールを作成します。設定は、以下。
    * ロール名  : ※任意
    * ポリシー名  : ※任意
    * ポリシー : ※以下参考。

    ```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "sns:Publish",
            "Resource": [
                "arn:aws:sns:*:XXXXXXXXXXXX:SysOps-Lambda-DLQ",
                "arn:aws:sns:ap-northeast-1:XXXXXXXXXXXX:DevOps-Team-XXXX",
                "arn:aws:sns:ap-northeast-1:XXXXXXXXXXXX:SysOps-Lambda-Mnt"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "codecommit:GetCommit",
                "logs:CreateLogStream",
                "codecommit:GetDifferences",
                "codecommit:GetBlob",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}
    ```
    * ![FireShot Capture 107 - IAM Management Console - console.aws.amazon.com.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/13e4f2fe-1cec-1b30-a964-aa069e062d05.png)
    * ![_Key.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/2daa9551-cb73-7fb4-ebe9-c7ec0dbb0204.png)
<br><br>
1. Lambda ファンクションを作成します。設定は、以下。
    * ファンクション名  : ※任意
    * 実行ロール  : ※作成した IAM ロール
    * コード : ※以下参考。

```python
#!/usr/bin/env python
# -*- coding: utf-8-unix; -*-
"""AWS Lambda Function - Maintenance DevOps CodeCommit Push to SNS
"""
from __future__ import print_function

import logging
import boto3
import os
import difflib

# 「python - aws lambda Unable to import module 'lambda_function': No module named 'requests' - Stack Overflow」
# https://stackoverflow.com/questions/48912253/aws-lambda-unable-to-import-module-lambda-function-no-module-named-requests
from botocore.vendored import requests

# 「Python 3 で少しだけ便利になった datetime の新機能 - Qiita」
# <https://qiita.com/methane/items/d7ac3c9af5a2c659bc51>
from datetime import datetime, timezone, timedelta
TimeZone = timezone(timedelta(hours=+9), 'JST')

# ISO 8601の日付フォーマットをPythonでparseするには
# http://blog.kzfmix.com/entry/1311336119
import dateutil.parser

# 「Lambdaの本番業務利用を考える① - ログ出力とエラーハンドリング | ナレコムAWSレシピ」
# <https://recipe.kc-cloud.jp/archives/9968>
logger = logging.getLogger()
logLevelTable={'DEBUG':logging.DEBUG,'INFO':logging.INFO,'WARNING':logging.WARNING,'ERROR':logging.ERROR,'CRITICAL':logging.CRITICAL}
if os.environ.get('logging_level') in logLevelTable :
    logger.setLevel(logLevelTable[os.environ['logging_level']])
else:
    logger.setLevel(logging.WARNING)

#
StartDateTime = datetime.now(TimeZone)

# Aws

# Mail
MailSubjectTemplate = "AWS CodeCommit Push ({0}/{1}) By {2} At {3:%Y/%m/%dT%H:%M:%S.%f%z}"

MailMessageTemplate00 = "\
\n\
{0}\n\
\n\
　■処理時間 : {1:%Y/%m/%d %H:%M:%S} ～ {2:%Y/%m/%d %H:%M:%S}\n\
\n\
　■コミット\n\
　　・ID　　　　　:　{3}\n\
　　・Last ID 　　:　{4}\n\
　　・Comment 　　:\n\
----\n\
{5}\
----\n\
　　・Author　　　:　{6}\n\
　　・Committer 　:　{7}\n\
　　・AddData 　　:\n\
----\n\
{8}\
----\n\
\n\
　■差分\n\
"

MailMessageTemplate01 = "{0}\n\
[{1}] {2}\n\
{3}\
"

MailMessageTemplate02 = "{0}\n\
\n\
----\n\
"

# ------------------------------------------------------------------------------
# CodeCommit Info Get
# ------------------------------------------------------------------------------
def CodeCommitInfoGet(prmRepositoryName, prmBranchName, prmCommitID, prmInfo):
    logging.info("prmRepositoryName:[%s]", prmRepositoryName)
    logging.info("prmBranchName:[%s]", prmBranchName)
    logging.info("prmCommitID:[%s]", prmCommitID)
    result = 0
    try:
        # CodeCommit — Boto 3 Docs 1.9.95 documentation
        # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit.html
        # Amazon CodeCommit にPushしたファイルを S3 に自動保存してみた - 挫折から何かしら学んでいきたい
        # https://rioner2525.hatenablog.com/entry/2018/11/10/161030
        # Automatically publish to S3 using AWS CodeCommit and Lambda – nlytn
        # https://nlytn.me/archives/2018/09/06/204
        codecommit = boto3.client('codecommit')
        commit = codecommit.get_commit(repositoryName=prmRepositoryName,commitId=prmCommitID)
        logging.info("commit:[%s]", commit)
        prmInfo['commit'] = commit['commit']
        prmInfo['differences'] = {}
        if 'commit' in commit:
            if 'parents' in commit['commit'] and len(commit['commit']['parents']) > 0:
                differences = []
                response = codecommit.get_differences(
                    repositoryName=prmRepositoryName,
                    beforeCommitSpecifier=commit['commit']['parents'][0],
                    afterCommitSpecifier=prmCommitID)
                while 'nextToken' in response:
                    differences += response['differences']
                    response = codecommit.get_differences(
                        repositoryName=prmRepositoryName,
                        beforeCommitSpecifier=commit['commit']['parents'][0],
                        afterCommitSpecifier=prmCommitID,
                        nextToken=response['nextToken'])
                else:
                    differences += response['differences']
                logging.info("differences:[%s]", differences)
                for difference in differences:
                    logging.info("difference:[%s]", difference)
                    changeType = difference.get('changeType', '')
                    beforeBlobPath = ''
                    beforeBlobContent = ''
                    if 'beforeBlob' in difference:
                        beforeBlobPath = difference['beforeBlob'].get('path', '')
                        blobID = difference['beforeBlob'].get('blobId', '')
                        if blobID:
                            try:
                                beforeBlobContent = codecommit.get_blob(repositoryName=prmRepositoryName, blobId=blobID)['content'].decode('utf-8')
                            except UnicodeDecodeError:
                                beforeBlobContent = "??? beforeBlobContent UnicodeDecodeError ???\n"
                    logging.info("beforeBlobPath:[%s]", beforeBlobPath)
                    logging.info("beforeBlobContent:[%s]", beforeBlobContent)
                    afterBlobPath = ''
                    afterBlobContent = ''
                    if 'afterBlob' in difference:
                        afterBlobPath = difference['afterBlob'].get('path', '')
                        blobID = difference['afterBlob'].get('blobId', '')
                        if blobID:
                            try:
                                afterBlobContent = codecommit.get_blob(repositoryName=prmRepositoryName, blobId=blobID)['content'].decode('utf-8')
                            except UnicodeDecodeError:
                                afterBlobContent = "??? afterBlobContent UnicodeDecodeError ???\n"
                    logging.info("afterBlobPath:[%s]", afterBlobPath)
                    logging.info("afterBlobContent:[%s]", afterBlobContent)
                    path = ''
                    blobid = ''
                    if changeType:
                        if beforeBlobPath:
                            path = beforeBlobPath
                        elif afterBlobPath:
                            path = afterBlobPath
                    if path:
                        key = changeType + '_' + path
                        prmInfo['differences'][key] = {
                            'changeType': changeType,
                            'path': path,
                            'difference': difference,
                            'beforeBlobContent': beforeBlobContent,
                            'afterBlobContent': afterBlobContent,
                        }
    except Exception as e:
        logger.exception("Error dosomething: %s", e)
        result = 1
        raise
    else:
        pass
    finally:
        pass
    return result
    
# 「【Python】Lambdaからメールを送信 | ハックノート」
# https://hacknote.jp/archives/35679/
def MailSend(prmTopic, prmSubject, prmMessage):
    result = -1
    try:
        sns = boto3.client('sns')
        response = sns.publish(
            TopicArn = 'arn:aws:sns:ap-northeast-1:XXXXXXXXXXXX:' + prmTopic,
            Message = prmMessage,
            Subject = prmSubject
        )
    except Exception as e:
        logger.exception("Error dosomething: %s", e)
        result = -1
        raise
    else:
        pass
    finally:
        pass
    return result

def lambda_handler(event, context):
    try:
        global StartDateTime
        StartDateTime = datetime.now(TimeZone)
        logger.info("StartDateTime:[%s]", StartDateTime)
        #
        logger.info("event:[%s]", event)
        commitID = ""
        commitRef = ""
        eventTime = ""
        eventSourceARN = ""
        userIdentityARN = ""
        customData = ""
        if 'Records' in event and len(event['Records']) > 0:
            eventTime = event['Records'][0].get('eventTime', '')
            eventTime = dateutil.parser.parse(eventTime).astimezone(TimeZone)
            eventSourceARN = event['Records'][0].get('eventSourceARN', '')
            userIdentityARN = event['Records'][0].get('userIdentityARN', '')
            customData = event['Records'][0].get('customData', '')
            if 'codecommit' in event['Records'][0] and 'references' in event['Records'][0]['codecommit'] and len(event['Records'][0]['codecommit']['references']) > 0:
                commitID = event['Records'][0]['codecommit']['references'][0]['commit']
                commitRef = event['Records'][0]['codecommit']['references'][0]['ref']
        logger.info("commitID:[%s]", commitID)
        logger.info("commitRef:[%s]", commitRef)
        logger.info("eventTime:[%s]", eventTime.isoformat())
        logger.info("eventSourceARN:[%s]", eventSourceARN)
        logger.info("userIdentityARN:[%s]", userIdentityARN)
        logger.info("customData:[%s]", customData)
        logger.info("context:[%s]", context)
        #
        branchName = os.path.basename(commitRef)
        userName = os.path.basename(userIdentityARN)
        repositoryName = eventSourceARN.split(":")[-1]
        logger.info("branchName:[%s]", branchName)
        logger.info("userName:[%s]", userName)
        logger.info("repositoryName:[%s]", repositoryName)
        # ----------------------------------
        # CodeCommit Info Get
        # ----------------------------------
        dicInfo = {}
        CodeCommitInfoGet(repositoryName, branchName, commitID, dicInfo)
        logging.info("dicInfo:[%s]", dicInfo)
        # ----------------------------------
        # Mail Create Message
        # ----------------------------------
        # Subject
        tmpSubject = MailSubjectTemplate.format(repositoryName, branchName, userName, eventTime)
        # Message
        tmpMessage = MailMessageTemplate00.format(
            tmpSubject,
            StartDateTime,
            datetime.now(TimeZone),
            dicInfo['commit']['commitId'],
            dicInfo['commit']['parents'],
            dicInfo['commit']['message'],
            dicInfo['commit']['author'],
            dicInfo['commit']['committer'],
            dicInfo['commit']['additionalData'])
        #
        for key in sorted(dicInfo['differences']):
            logging.info("key:[%s]", key)
            logging.info("difference:[%s]", dicInfo['differences'][key])
            # 6.3. difflib — 差分の計算を助ける — Python 3.6.5 ドキュメント
            # https://docs.python.jp/3/library/difflib.html
            # text 文字列 差分 - ファイルを比較するpython difflib - CODE Q&A 問題解決
            # https://code.i-harness.com/ja-jp/q/f21341
            # Pythonのdifflibモジュールを用いて複数行テキストどうしの差分を取得する - 試験運用中なLinux備忘録
            # http://d.hatena.ne.jp/kakurasan/20100308/p1
            diff = difflib.unified_diff(
                dicInfo['differences'][key].get('beforeBlobContent', '').splitlines (True),
                dicInfo['differences'][key].get('afterBlobContent', '').splitlines (True),
                fromfile=dicInfo['differences'][key].get('path', ''),
                tofile=dicInfo['differences'][key].get('path', ''),
                lineterm='\n')
            changes = ""
            for l in diff:
                changes += l
            #changes = [l for l in diff if l.startswith('+ ') or l.startswith('- ')]
            logging.info("changes:[%s]", changes)
            tmpMessage = MailMessageTemplate01.format(
                tmpMessage,
                dicInfo['differences'][key].get('changeType'),
                dicInfo['differences'][key].get('path'),
                changes)
        #
        tmpMessage = MailMessageTemplate02.format(tmpMessage)
        
        # Mail Send
        tmpTopic = 'SysOps-Lambda-Mnt'
        if customData:
            tmpTopic = customData
        logging.info("tmpTopic:[%s]", tmpTopic)
        logging.info("tmpSubject:[%s]", tmpSubject)
        logging.info("tmpMessage:[%s]", tmpMessage)
        MailSend(tmpTopic, tmpSubject, tmpMessage)
        
    except Exception as e:
        logger.exception("Error dosomething: %s", e)
    else:
        pass
    finally:
        pass
    #
    return "normal end"
```

* ![DevOps_CodeCommit_Push_to_SNS.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/2b752209-5bea-b454-be45-42db03f19279.png)
* ![_AWSCodeCommit.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/6814e02a-2db5-ab67-c86e-538ce48a39de.png)
* ![_AmazonCloudWatchLogs.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/6374f005-e872-b2a4-591f-2ca3e99b1822.png)
* ![_AmazonSNS.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/253dc874-386e-c2b3-347e-0e0d7b49e150.png)

# ３．AWS CodeCommit のトリガーを追加する。

1. Lambda のコンソールから、「トリガーを追加」ボタンを押下し、「CodeCommit」を選択する。
<br><br>
1. Lambda のコンソールから、「トリガーを追加」ボタンを押下し、「CodeCommit」を選択する。
    * リポジトリ名 : ※所定のリポジトリ
    * トリガー名 : ※任意
    * イベント : ※任意
    * ブランチ名 : ※任意
<br><br>
    * ![_CodeCommit.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/161939/c7e05d92-6be1-fa4f-11f8-4de3f5e408a9.png)

# ９９．ハマりポイント

* Python スクリプトで、Diff 情報を作成するのが結構面倒でしたね…。

# ＸＸ．まとめ

ご参考になれば♪