Package: salesforce-bulkipy
A Python library for the Salesforce Bulk API (that actually works)Changes oversalesforce-bulkThesalesforce-bulklibrary was used to export 18k records toWingify’s Salesforce system. Even
though the library was super useful, it’s broken, not maintained anymore
and was a pain to work with while figuring out the bugs.
[@bholagabbar](https://github.com/bholagabbar) decided to fix all the
issues faced and release a new, usable library**salesforce-bulkipy**.
This library has been tested successfully on our Salesforce Sandbox.Added support forTwo-Factor
Authenticationby routing authentication viasimple-salesforceAdded support forSalesforce
SandboxAdded support for parsing unicode characters in CSVExplicitUpsertSupportFixed various other bugssalesforce-bulkipywill be actively maintained, unlike
salesforce-bulkInstallation``sudo pip install salesforce-bulkipy``Incase your setup fails, you may have a few essential tools missing. Trysudoapt-getinstallbuild-essentiallibssl-devlibffi-devpython-devAuthenticationTo access the Bulk API, you need to authenticate a user into Salesforce.
There are 2 possible ways to achieve this. These methods work
irrespective of whether your organisation hasTwo-Factor
Authenticationenabled or not, so that’s a massive overhead taken care of.The code samples shown read credentials from aconfig.propertiesfile. Feel free to adapt the input method to your setting1. username, password,security_tokenfrom salesforce_bulkipy import SalesforceBulkipy
import ConfigParser

config = ConfigParser.RawConfigParser()
config.read('config.properties')

username = config.get('Section', 'username')
password = config.get('Section', 'password')
security_token = config.get('Section', 'security_token')

bulk = SalesforceBulkipy(username=username, password=password, security_token=security_token) #optional parameter: sandbox=True

# Authentication Successful!2. session_id, hostfrom salesforce_bulkipy import SalesforceBulkipy
import ConfigParser

config = ConfigParser.RawConfigParser()
config.read('config.properties')

session_id = config.get('Section', 'session_id')
session_id = config.get('Section', 'session_id')

bulk = SalesforceBulkipy(session_id=session_id, host=host) #optional parameter: sandbox=True

# Authentication Successful!OperationsThe basic sequence for driving the Bulk API is:Create a new jobAdd one or more batches to the jobWait for each batch to finishClose the jobBulk Insert, Update, Upsert, DeleteAll Bulk upload operations work the same. You set the operation when you
create the job. Then you submit one or more documents that specify
records with columns toinsert/update/delete.For theupsertoperation, we also need to specify some thing called
theexternal_keywhich can be any attribute(preferably unique) of
your custom Salesforce object. Every record to upsert is checked against
this key in Salesforce. Say your external key isId. Now for every
record you are pushing, it is checked it you have a record with the sameIdalready. If yes, then it is updated else that record is created.For thedeleteoperation, you should only submit the Id for each
record.For efficiency you should use thepost_bulk_batchmethod to post
each batch of data. (Note that a batch can have a maximum 10,000 records
and be 1GB in size.) You pass a generator or iterator into this function
and it will stream data via POST to Salesforce. For help sending CSV
formatted data you can use the salesforce_bulk.CsvDictsAdapter class.
It takes an iterator returning dictionaries and returns an iterator
which produces CSV data.Concurrency mode: When creating the job, you can passconcurrency=Serialorconcurrency=Parallelto set the
concurrency mode for the job.Bulk Insert Examplefrom salesforce_bulkipy import SalesforceBulkipy
from salesforce_bulkipy import CsvDictsAdapter

bulk = SalesforceBulkipy(username=username, password=password, security_token=security_token)

records_to_insert = [{}, {}]  # A list of A Custom Object dict

# Bulk Insert
job = bulk.create_insert_job("CustomObjectName", contentType='CSV')
csv_iter = CsvDictsAdapter(iter(records_to_insert))
batch = bulk.post_bulk_batch(job, csv_iter)
bulk.wait_for_batch(job, batch)
bulk.close_job(job)Bulk Query Examplefrom salesforce_bulkipy import SalesforceBulkipy

bulk = SalesforceBulkipy(username=username, password=password, security_token=security_token)

records_to_insert = [{}, {}]  # A list of A Custom Object dict

# Bulk Query
query = '' # SOQL Query
job = bulk.create_query_job("Object_Name", contentType='CSV')
batch = bulk.query(job, query)
bulk.wait_for_batch(job, batch)
bulk.close_job(job)
# Result
results = bulk.get_batch_result_iter(job, batch, parse_csv=True)Bulk Upsert Examplefrom salesforce_bulkipy import SalesforceBulkipy

bulk = SalesforceBulkipy(username=username, password=password, security_token=security_token)

records_to_upsert = [{}, {}]  # A list of A Custom Object dict

# Bulk Query
query = '' # SOQL Query
job = bulk.create_upsert_job("Object_Name", external_id_name="Unique_id", contentType='CSV')
csv_iter = CsvDictsAdapter(iter(records_to_insert))
batch = bulk.post_bulk_batch(job, csv_iter)
bulk.wait_for_batch(job, batch)
bulk.close_job(job)Credits and ContributionsThis repository is a maintained fork ofheroku/salesforce-bulk.
The changes incorporated here are a result of a joint effort by
[@lambacck](https://github.com/lambacck),
[@Jeremydavisvt](https://github.com/Jeremydavisvt),
[@alexhughson](https://github.com/alexhughson) and
[@bholagabbar](https://github.com/bholagabbar). Thanks to
[@heroku](https://github.com/heroku) for creating the original useful
library.Feel free to contribute by creating Issues and Pull Requests. We’ll test
and merge them.