Position: AWS Developer
Company: UDC Software
Experience required: 3y
Category: DevOps
Published: 2022-03-01T00:00:00+02:00

The purpose of this work is to use a combination of AWS serverless services to orchestrate a reliable, scalable, and cost-effective email invoice pipeline.

Use AWS SAM with Step Functions, SQS, Lambda, SNS, S3, and Textract (and any other services) to build the following flow:

1. The process is triggered by Eventbridge event with the following data:
 provider: "outlook", access_token: "xyz"
 provider: "gmail", access_token: "xyz"

2. Use the access_token to search the email API for pdf attachments. See the  and Microsoft Graph  for API specifics

3. Download the attachment and store in an S3 bucket in the /uploads prefix

4. Pass the S3 file key to Textract and store the results in the same bucket under the /processed path

  a) Use the synchronous  endpoint if you can use the  directly from the Step Function. Please donʼt call the synchronous endpoint from lambda as it will incur charges while the lambda waits for the results. 
  b) Otherwise use the asynchronous  API. Use the Step Function Task Token to resume the Step Function execution once the StartExpenseAnalysis call is complete

4.Submit the file to the Butler API

5.The Step Function should poll the Butler  endpoint once every 10 minutes until the extraction completes

6. Store the json results alongside the step 4 Textract results 

Question: Weʼve outlined one way to accomplish a reliable, scalable, and cost efficient. What suggestions do you have to improve?