Package: salesforce-bulk
Salesforce BulkPython client library for accessing the asynchronous Salesforce.com Bulk
API.Installationpipinstallsalesforce-bulkAuthenticationTo access the Bulk API you need to authenticate a user into Salesforce.
The easiest way to do this is just to supplyusername,passwordandsecurity_token. This library will use thesimple-salesforcepackage to handle password based authentication.fromsalesforce_bulkimportSalesforceBulkbulk=SalesforceBulk(username=username,password=password,security_token=security_token)...Alternatively if you run have access to a session ID and instance_url
you can use those directly:fromurlparseimporturlparsefromsalesforce_bulkimportSalesforceBulkbulk=SalesforceBulk(sessionId=sessionId,host=urlparse(instance_url).hostname)...OperationsThe basic sequence for driving the Bulk API is:Create a new jobAdd one or more batches to the jobClose the jobWait for each batch to finishBulk Querybulk.create_query_job(object_name,contentType='JSON')Using API v39.0 or higher, you can also use the queryAll operation:bulk.create_queryall_job(object_name,contentType='JSON')Exampleimportjsonfromsalesforce_bulk.utilimportIteratorBytesIOjob=bulk.create_query_job("Contact",contentType='JSON')batch=bulk.query(job,"select Id,LastName from Contact")bulk.close_job(job)whilenotbulk.is_batch_done(batch):sleep(10)forresultinbulk.get_all_results_for_query_batch(batch):result=json.load(IteratorBytesIO(result))forrowinresult:printrow# dictionary rowsSame example but for CSV:importunicodecsvjob=bulk.create_query_job("Contact",contentType='CSV')batch=bulk.query(job,"select Id,LastName from Contact")bulk.close_job(job)whilenotbulk.is_batch_done(batch):sleep(10)forresultinbulk.get_all_results_for_query_batch(batch):reader=unicodecsv.DictReader(result,encoding='utf-8')forrowinreader:print(row)# dictionary rowsNote that while CSV is the default for historical reasons, JSON should
be prefered since CSV has some drawbacks including its handling of NULL
vs empty string.PK Chunk HeaderIf you are querying a large number of records you probably want to turn onPK Chunking:bulk.create_query_job(object_name,contentType='CSV',pk_chunking=True)That will use the default setting for chunk size. You can use a different chunk size by providing a
number of records per chunk:bulk.create_query_job(object_name,contentType='CSV',pk_chunking=100000)Additionally if you want to do something more sophisticated you can provide a header value:bulk.create_query_job(object_name,contentType='CSV',pk_chunking='chunkSize=50000;startRow=00130000000xEftMGH')Bulk Insert, Update, DeleteAll Bulk upload operations work the same. You set the operation when you
create the job. Then you submit one or more documents that specify
records with columns to insert/update/delete. When deleting you should
only submit the Id for each record.For efficiency you should use thepost_batchmethod to post each
batch of data. (Note that a batch can have a maximum 10,000 records and
be 1GB in size.) You pass a generator or iterator into this function and
it will stream data via POST to Salesforce. For help sending CSV
formatted data you can use the salesforce_bulk.CsvDictsAdapter class.
It takes an iterator returning dictionaries and returns an iterator
which produces CSV data.Full example:fromsalesforce_bulkimportCsvDictsAdapterjob=bulk.create_insert_job("Account",contentType='CSV')accounts=[dict(Name="Account%d"%idx)foridxinxrange(5)]csv_iter=CsvDictsAdapter(iter(accounts))batch=bulk.post_batch(job,csv_iter)bulk.wait_for_batch(job,batch)bulk.close_job(job)print("Done. Accounts uploaded.")Concurrency modeWhen creating the job, passconcurrency='Serial'orconcurrency='Parallel'to set the concurrency mode for the job.