AWS Fraud Detector for classifying fraudulent online registered accounts - Part 2
Tags: api, aws, machinelearning, serverless
Published: 2022-09-14 20:21:06


In the [first part](https://dev.to/aws-builders/aws-fraud-detector-for-classifying-fraudulent-online-registered-accounts-part-1-1j4p) of this blog, we trained and evaluated the fraud detector model performance. Now we will need to make it active by deploying it and then make predictions. We will also setup a **REST API** with **API gateway (Lambda Integration)** for making realtime predictions. The architecture diagram below shows the workflow. All the references to code snippets are in [Github](https://github.com/ryankarlos/AWS-ML-services/tree/master/projects/fraud).


![Fraud Predict Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bmrinc4jlz4ixzk4zibd.png)

In the AWS Fraud Detector console, choose the model we have trained from the **Models** page. Scroll to the top of the **Version details** page and choose **Actions** and **Deploy model version**. On the **Deploy model version** prompt that appears, choose **Deploy version**.

![variables](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uwovbhpq6jp7545m5mt8.png)

The **Version details** shows a **Status** of 'Deploying'. When the model is ready, the Status changes to **Active**. Once the model has finished deploying and status changed to active, we will need to associate the model with Fraud Detector for predictions. However, we will also need to update the rule expressions as the default Fraud Detector version 1 created from Cloudformation uses the variable **amt** in the rule expression as seen in the screenshot below. 

![detector-version-1-default-rules](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eifk9pzhoqsagpylcx8t.png)

We need to change this to model insight score which is a new variable created after model training has completed. This variable is not available when the Cloudformation stack is created as the model has not been trained yet so we needed to have a placeholder existing variable so the rule expression is valid to avoid the stack for throwing an error. We can run the following custom [script](https://github.com/ryankarlos/AWS-ML-services/blob/master/projects/fraud/deploy.py) from the command line to update the detector rules and associate the new model with it. 

```bash
python projects/fraud/deploy.py --update_rule 1 --model_version 1.0 --rules_version 2 
```
This will carry out two steps: 

* Update the existing rule version with the correct expression based on the number passed to the `--update_rule` argument. It will create a new rule version (incremented from the original version number). 
* Then it will create a new detector version and associate the model version (`--model_version` argument) and the rules_version (`--rules_version` argument) which should be set to be the increment of the existing rule version. This will automatically increment the detector version to **2.0** as the existing version is **1.0**

If the script runs successfully, we should see the following output streamed to stdout.

```               
26-06-2022 04:50:34 : INFO : deploy : main : 121 : Updating rule version 1
26-06-2022 04:50:34 : INFO : deploy : update_detector_rules : 71 : Updating Investigate rule ....
{'detectorId': 'fraud_detector_demo', 'ruleId': 'investigate', 'ruleVersion': '2'}

26-06-2022 04:50:35 : INFO : deploy : update_detector_rules : 80 : Updating review rule ....
{'detectorId': 'fraud_detector_demo', 'ruleId': 'review', 'ruleVersion': '2'}

26-06-2022 04:50:35 : INFO : deploy : update_detector_rules : 89 : Updating approve rule ....
{'detectorId': 'fraud_detector_demo', 'ruleId': 'approve', 'ruleVersion': '2'}

26-06-2022 04:50:35 : INFO : deploy : main : 123 : Deploying trained model version 1.0 to new detector version 
{'detectorId': 'fraud_detector_demo', 'detectorVersionId': '2', 'status': 'DRAFT', 'ResponseMetadata': {'RequestId': 'da37d973-2c43-4c56-93e5-f9b9bd132bb3', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Sun, 26 Jun 2022 03:50:36 GMT', 'content-type': 'application/x-amz-json-1.1', 'content-length': '77', 'connection': 'keep-alive', 'x-amzn-requestid': 'da37d973-2c43-4c56-93e5-f9b9bd132bb3'}, 'RetryAttempts': 0}}

```

The image below show the model associated with the new version and the correct rules expressions which use the **fraud_model_insightscore** variable.

![fraud-detector-update-rules-version2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3x4yu1mvf73kwre9x7ox.png)

![fraud-detector-associate-model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8toxp0ntn9qmj6qydhcn.png)

In the next section, we will set up API gateway to create a rest api endpoint to serve http requests, with lambda in the backend.

### Setting up API gateway 

In this section, we will walk through the steps to create the REST API 

* Open the **API Gateway console**, and select **CreateAPI** and the type as **RestAPI**. 
* To create an empty API, select **Create New API** and then **New API**. In **Settings**, choose a **API name** such as `FraudLambdaProxy` and optional **Description**. Then choose `Create API`.
* Choose the root resource (**/**) just created and select **Create Method** from the **Actions** menu. Then Select `Get` from the dropdown menu.
* For **Integration Type** select `Lambda Function` and choose `Use Lambda Proxy integration`. The Lambda function should already have been created via Cloudformation in [part 1](https://dev.to/aws-builders/aws-fraud-detector-for-classifying-fraudulent-online-registered-accounts-part-1-1j4p)). Select the **Lambda Region** where the function was created (`us-east-1`) and for the **Lambda Function** field, select `PredictFraudModel` from the dropdown, and then click **Save**.
* In the **Method Execution** pane, choose **Method Request**.
* In settings, set **Request Validator** to `Validate query string parameters and headers`. Leave **Authorization** as `None`.
* Expand the **URL Query String Parameters** dropdown, then choose **Add query string**.
* Enter the following variables one by one as a separate name field. Mark all as required except for `flow_definition` variable

```
 amt, category, cc_num, city, city_pop, event_timestamp, first, flow_definition, gender, job, last, merchant, state, street, trans_num, zip
```

![API-gateway-method-request-console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ybztcjttoatrr681k3dd.png)

* Go back to the **Method Execution** pane.It should look like below.

![API-gateway-method-execution-console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l8h4xo4dnpxyvv4vv9ww.png)

* Choose **Integration Request**.
* Choose the **Mapping Templates** dropdown and then choose **Add mapping template**.
* For the **Content-Type** field, enter `application/json` and then choose the check mark icon.
* In the pop-up that appears, choose **Yes** to secure this integration.
* For **Request body passthrough**, choose `When there are no templates defined (recommended)`.
* In the **mapping template** editor, copy and replace the existing script with the following code:

```json
#if("$input.params('flow_definition')" != "")
#set( $my_default_value = "$input.params('flow_definition')")
#else
#set ($my_default_value = "ignore")
#end


{
  "variables": {
        "trans_num":"$input.params('trans_num')",
        "amt":"$input.params('amt')",
        "zip":"$input.params('zip')",
        "city":"$input.params('city')",
        "first":"$input.params('first')",
        "job":"$input.params('job')",
        "street":"$input.params('street')",
        "category":"$input.params('category')",
        "city_pop":"$input.params('city_pop')",
        "gender":"$input.params('gender')",
        "cc_num":"$input.params('cc_num')",
        "last":"$input.params('last')",
        "state":"$input.params('state')",
        "merchant":"$input.params('merchant')"
  },
  "EVENT_TIMESTAMP":"$input.params('event_timestamp')",
  "flow_definition":"$my_default_value"
}
```

![API-gateway-integration-request-console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/whbz6iwa4vd57i7qzyqw.png)

* Choose **Save**, and go back to **MethodExecution** pane. Click on **Test** button on the left.
* In the **Query Strings box** paste the following

```
trans_num=6cee353a9d618adfbb12ecad9d427244&amt=245.97&zip=97383&city=Stayton&first=Erica&job=Engineer, biomedical&street=213 Girll Expressway&category=shopping_pos&city_pop=116001&gender=F&cc_num=180046165512893&last=Walker&state=OR&merchant=fraud_Macejkovic-Lesch&event_timestamp=2020-10-13T09:21:53.000Z&flow_definition=arn:aws:sagemaker:us-east-1:376337229415:flow-definition/fraud-detector-a2i-1656277295743
```

If successful you should see the response and logs as in screenshot below. You can also navigate to CloudWatch log stream group for Lambda invocation and check it has run successfully.

![API-gateway-get-method-test1-console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uli6lsn7v562o8rs5fxe.png)

![API-gateway-get-method-test2logs-console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bpuyzmyh67uy2swxh2fr.png)

We can then proceed to deploying the API. Go back to **Resources**,  **Actions** and **Deploy API**. Select **Deployment Stage** as  `New Stage` and choose **name** as `dev`. You should see the API endpoint to invoke on the console. Finally make sure logging is setup to allow debugging errors in  the REST API, by following the instructions [here] (https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-cloudwatch-logs/). The setup should look like below. Note that when you add the IAM role to gateway console, it should automatically add the log group in the format `API-Gateway-Execution-Logs_apiId/stageName`. The Arn for the log group end with `dev:*`. You need to only include the Arn upto the stagename `dev` as shown in the image below. Otherwise it will throw issues with the validation checks.

![api-rest-stage-editor-logs-config](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hea2ri13ze15gkmsx2ih.png)

 To test the API's new endpoint, we can use [postman](https://learning.postman.com/docs/getting-started/sending-the-first-request/) for sending an API request. Create a postman account and select **GET** from the list of request types. Since the **GET** method is configured in `/` root resource, we can invoke the api endpoint `https://d9d16i7hbc.execute-api.us-east-1.amazonaws.com/dev` with the query string parameters appended at the end  (`key=value` format and separated by `&`). Paste the following command in the box as in screenshot below. You should see the parameters and values automatically parsed and populated in the `KEY/VALUE` rows below.
Click send and you should see the response body at the bottom.

```
https://d9d16i7hbc.execute-api.us-east-1.amazonaws.com/dev?trans_num=6cee353a9d618adfbb12ecad9d427244&amt=245.97&zip=97383&city=Stayton&first=Erica&job='Engineer, biomedical'&street='213 Girll Expressway'&category=shopping_pos&city_pop=116001&gender=F&cc_num=180046165512893&last=Walker&state=OR&merchant=fraud_Macejkovic-Lesch&event_timestamp=2020-10-13T09:21:53.000Z
```

![postman-api-gateway](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ujtijy4e82raznlp4e87.png)

You can check the log streams associated with the latest invocation in the Cloudwatch log group for API gateway. This will show  messages with the execution or access details of your request.

![API-gateway-cloudwatch-log-group](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3bbpmggbv06vsfn51q0l.png)

**Note**: If any changes are made to the api configuration or parameters - it would need to be redeployed for the changes to take effect.

### Generate Predictions 

You can use a batch predictions job in Amazon Fraud Detector to get predictions for a set of events that do not require real-time scoring. You may want to generate fraud predictions for a batch of events. These might be payment fraud, account take over or compromise, and free tier misuse while performing an offline proof-of-concept. 

You can also use batch predictions to evaluate the risk of events on an hourly, daily, or weekly basis depending upon your business need. If you want to analyze fraud transactions after the fact, you can perform batch fraud predictions using Amazon Fraud Detector.  Then you can store fraud prediction results in an Amazon S3 bucket. Although beyond the scope of this example, we could have also used additional services like Amazon Athena  to help analyze the fraud prediction results (once delivered in S3) and Amazon QuickSight for visualising the results on a dashboard.Copy the batch sample file delivered in the  glue_transformed folder (following successful glue job run) to batch_predict folder. This will trigger notification to SQS queue which has Lambda function as target, which starts the batch prediction job in Fraud Detector

```bash
$ aws s3 cp s3://fraud-sample-data/glue_transformed/test/fraudTest.csv s3://fraud-sample-data/batch_predict/fraudTest.csv
```

We can monitor the batch prediction jobs in Fraud Detector. Once complete, we should see the output in S3. An example of 
a batch output is available [here](https://github.com/ryankarlos/AWS-ML-services/blob/master/datasets/fraud-sample-data/dataset1/results/DetectorBatchResults.csv)

![batch_prediction_jobs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fpe0pe8kyk1b73kb6bg8.png)

In realtime mode, we will make use of the API gateway created and integrated with the lambda function which makes the 
`get_event_prediction` api call to FraudDetector. In this example we are using the same lambda for batch and realtime predictions. The code in lamdba checks the checks the event payload to see if certain keys are present which are expected from a request from API gateway (.i.e after the request 
is transformed via the mapping template in api gateway). We have configured the mapping template to create a variables key, so we can check if the payload has 'variables' key,  to run realtime prediction. If the event payload has 'Records' key, it indicates the event is coming from  SQS and will run a batch prediction job. 

Ideally, separate lambdas for realtime and batch prediction could have been used, to make it easier to manage. 
To run realtime prediction, API gateway REST API has been configured to accept query string parameters and send the request to lambda as explained in the previous section.

### Teardown resources 

The custom bash script below can be executed to teardown all the fraud detector resources, including the trained fraud model, detector (including rules), event type (including outcomes, variables, labels).

```bash
#!/bin/bash

variables=( "trans_num" "amt" "city_pop" "street" "job" "cc_num" "gender" "merchant" "last" "category" "zip" "city" "state" "first")
labels=("legit" "fraud")
rules=("investigate" "review" "approve")
outcomes=("high_risk" "low_risk" "medium_risk")
event_type="credit_card_transaction"
entity_type="customer"
detector_name="fraud_detector_demo"
model_name=fraud_model

echo "Delete model versions"    
aws frauddetector  delete-model-version --model-id $model_name --model-type ONLINE_FRAUD_INSIGHTS --model-version-number 1.0
aws frauddetector  delete-model-version --model-id $model_name --model-type ONLINE_FRAUD_INSIGHTS --model-version-number 2.0

echo ""
echo "Delete model"
aws frauddetector  delete-model --model-id $model_name --model-type ONLINE_FRAUD_INSIGHTS

  
echo ""
echo "Deleting detector version id 1"
aws frauddetector delete-detector-version --detector-id $detector_name --detector-version-id 1

echo ""
for var in "${rules[@]}";
    do
        echo "Deleting rule $var"
        aws frauddetector  delete-rule --rule detectorId=$detector_name,ruleId=$var,ruleVersion=1
    done;

echo ""
echo "Deleting detector id $detector_name"
aws frauddetector delete-detector --detector-id $detector_name

echo ""
echo "Deleting event-type $event_type"
aws frauddetector delete-event-type --name $event_type

echo ""
echo "Deleting entity-type $entity_type"
aws frauddetector delete-entity-type --name $entity_type

echo ""
for var in "${variables[@]}";
    do
        echo "Deleting variable $var"
        aws frauddetector  delete-variable --name $var
    done;


echo ""
for var in "${labels[@]}";
    do
        echo "Deleting label $var"
        aws frauddetector  delete-label --name $var
    done;

echo ""
for var in "${outcomes[@]}";
    do
        echo "Deleting outcome $var"
        aws frauddetector  delete-outcome --name $var
    done;

echo ""
echo "Deleting cloud formation stack"
aws cloudformation delete-stack --stack-name FraudDetectorGlue
```

Finally, from the S3 console, empty the bucket contents and then delete the bucket.

![bucket-final-folder-structure](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dih03km0z0jual0nlte3.png)