Package: kicksaw-integration-utils
API ReferenceAWSSQSOverviewHigh-level ExampleInheriting the OrchestratorUsing the OrchestratorLow-level ExampleAPI ReferenceAWSHelper classes and functions to interact with and manipulate AWS services.SQSMake sure to provide type hintSQSQueue[Patient]to enable type hints for the queue
methods.Send messages:fromkicksaw_integration_utils.awsimportSQSQueuefrompydanticimportBaseModelclassPatient(BaseModel):first_name:strlast_name:strage:intmessages=[Patient(first_name="John",last_name="Doe",age=40),Patient(first_name="Jane",last_name="Doe",age=30),]queue:SQSQueue[Patient]=SQSQueue(name="my-queue-name",message_model=Patient)queue.send_messages(messages)Receive and delete messages:handles,messages=queue.receive_messages()queue.delete_messages(handles)OverviewA set of helper functions for CSV to Salesforce procedures, with reporting in AWS S3.
The use case is extremely specific, but the helpers should be modular so they can be cherry-picked.Typical use case:Receive an S3 eventDownload the S3 objectSerialize the file into JSONBulk upsert the JSON data to SalesforceParse the results of the upsert for errorsConstruct a CSV error reportMove the triggering S3 object to an archive folderPush the error report to an error folder in the same bucketPush an object to Salesforce that details information about the above execution2nd typical use case:Start an AWS Step FunctionPass the payload to the KicksawSalesforce client and create an execution object, recording this payloadupsert a bunch of data, parsing the responsesif any errors, push error objects into salesforce, chidling them to the execution object aboveHigh-level ExampleUsing theOrchestratorclass, you can skip manually setting up a lot of the above
steps. This class is intended to be subclassed, and should provide plenty of options
for overriding methods to better suit your use-case.Inheriting the Orchestrator# orchestrator.pyfromkicksaw_integration_utils.classesimportOrchestratorasBaseOrchestratorclassOrchestrator(BaseOrchestrator):def__init__(self,*args,**kwargs)->None:super().__init__(*args,**kwargs)# This must be defined in the child class because your Salesforce object could be named anythingself.execution_object_name="Integration_Execution__c"@propertydefexecution_sfdc_hash(self):# And it could have any number of fieldsreturn{"Number_of_Errors__c":self.error_count,"Error_Report__c":self.error_report_link,"Data_File__c":self.s3_object_key,}@propertydeferror_report_link(self):returnf"https://{self.bucket_name}.{config.AWS_REGION}.amazonaws.com/{self.error_file_s3_key}"Using the Orchestrator# biz_logic.pyfromkicksaw_integration_utils.classesimportSfClient# import the custom Orchestrator defined abovefrom.orchestratorimportOrchestratorsalesforce=SfClient()orchestrator=Orchestrator("some/s3/key/file.csv",config.S3_BUCKET,sf_client=salesforce)upsert_key="My_External_ID__c"accounts_data=[{"Name":"A name",upsert_key:"123"}]results=salesforce.bulk.Account.upsert(results,upsert_key)# You'll call log_batch for each batch you upload. This method# will parse the results in search of errorsorchestrator.log_batch(results,accounts_data,"Account",upsert_key)# This will create the error report, archive the source s3 file, and push# the integration object to Salesforce. You'll definitely want to customize# this by overriding this method or the methods it invokesorchestrator.automagically_finish_up()Low-level Examplefromkicksaw_integration_utils.csv_helpersimportcreate_error_reportfromkicksaw_integration_utils.s3_helpersimportdownload_file,respond_to_s3_event,upload_filefromkicksaw_integration_utils.sfdc_helpersimportextract_errors_from_results# handler for listening to s3 eventsdefhandler(event,context):respond_to_s3_event(event,download_and_process)defdownload_and_process(s3_object_key,bucket_name):download_path=download_file(s3_object_key,bucket_name)# This function contains your own biz logic; does not come from this libraryresults=serialize_and_push_to_sfdc(download_path)sucesses,errors=parse_bulk_upsert_results(results)report_path,errors_count=create_error_report([errors])upload_file(report_path,bucket_name)Just take what'cha need!