Detecting Sensitive Data in Salesforce With Amazon Macie
By: Dave Norris
Tags: Amazon Macie, AWS, Sensitive Data, AWS Lambda, Salesforce

Since I haven’t based the AppFlow on a Salesforce event I can only run the flow on demand or on a schedule. Since I want to include data that may have been modified I’ll select an incremental transfer based on LastModifiedDate to be scheduled every night at midnight.

AppFlows can be run on schedule, on demand or triggered by a Salesforce event.

Now I can map my Salesforce fields to destination fields. Since I don’t need Amazon Macie to check every field in my source object I can map only those fields I need. This helps to keep the file size to a minimum. In this case I need the unstructured text to analyse (New_Value__c in my use case) and the unique identifier for the Chatter Post in Salesforce so I can directly link to it for investigation. AppFlow also lets me include validations. In the example below I have included a clause to skip the record if there is no value to analyse.

Mapping Salesforce data to an S3 object

That is it. I repeated the steps to create 3 flows. One for each Salesforce source.

One AppFlow per Salesforce Object

They all populated their own unique object in the same S3 bucket as seen below after the first run. So the chatter-data AppFlow job ended with the results being put into the chatter-data object in the salesforce-data-analysis bucket etc. This meant other objects could be added at a later date.

One S3 object per Salesforce Object.

Step 2: Decode Base64 encoded strings

Most standard and custom objects have their field values arrive as text that can be analysed directly. FileVersion data from Salesforce arrives as as a Base64 encoded string. This required a Lambda function to be written on the PUT event of the file-data S3 object that read the originating string and decoded it.

The resulting output was put into another S3 object to prevent the Lambda function from firing recursively.

Step 3: Configure Amazon Macie

Configure S3 bucket for data discovery results

To ensure long-term retention of discovery results you must configure an S3 bucket so Macie can write its findings. I created a bucket called data-discovery-findings.

Configuring where to store the data discovery results

Configure data analysis job

Next you will need to create an analysis job on the S3 bucket. A wizard guides you through the process. For costing purposes Macie will estimate the cost based on the number of objects in the S3 bucket and the size.

Specify the S3 buckets to analyse

The Macie job then needs to know how often to run which can be Daily, Weekly or Monthly.

Macie analyses your data set for pre-defined managed data identifiers. Things like personally identifiable information (birth dates, driver’s licence numbers, passport numbers, national identifiers etc), personal health information (health insurance or medical identification number etc) and financial information (bank account numbers, credit card expiration dates, credit card numbers etc) and even credentials (AWS secrets, Open SSH private keys etc).

For a full list of managed data identifiers check out the following link

However, during the Macie job setup you get the option to include custom data identifiers. These are regular expressions that define patterns to match and keyword combinations to look for within a specified match distance.

That’s it. Once you give the job a name and description you’re good to start analysing your Salesforce data.

Step 4: Review Findings

Once the Macie job runs you can start to see the results of the findings. Amazon Macie will group findings into types.