Data Replication: Oracle SaaS Fusion to Data Warehouse
By: Ary Sharifian
Tags: SaaS, Object Storage, Platform As A Service, Oracle Analytics Cloud, Oracle Database

Pre-Requisites

OAC’s Data Replication tool only works with certain Oracle SaaS Fusion applications. Check the supported data sources here. As of the time of writing this article 11.1.1.9+ or Fusion Applications Release 8 and later are supported. For additional detail, check here.

Object Storage is needed as a staging area before transferring the SaaS application data to ADW.

Oracle Analytics Cloud Enterprise Edition is required to access the Data Replication Tool

An ADW instance (or Oracle Cloud Database) is required as the target for the data replication. Please check out my earlier article (Step 4) to provision ADW instance. Make sure to save the “Wallet Credential” file since it will be used in a few different parts of this tutorial.

Install SqlDeveloper or Toad to establish a connection to ADW. We will use this to validate that our SCM SaaS data properly replicated to ADW.

The right role and privileges are required to access the SaaS External Storage. First check if you can visit your Fusion SaaS application host url followed by “/biacm”. If you do not have access to this web page, then check the roles and responsibilities of your user by accessing tools>security console. It’s best to have ORA_ASM_APPLICATION_IMPLEMENTATION_ADMIN_ABSTRACT, ASM_APPLICATION_IMPLEMENTATION_CONSULTANT_JOB, or ORA_ASM_APPLICATION_IMPLEMENTATION_CONSULTANT_JOB, role provided to your user or some higher privileged role. Example URL for accessing the Oracle Business Intelligence Cloud Connector Console (BICC):

Ensure that the objects needed for extraction are available through BICC. There are some objects and corresponding tables not currently available for extraction. Oracle is actively working to make more objects are available for extraction.

SaaS BICC Page

Object Storage Configuration

Next, we want to create an Oracle Object Storage bucket to serve as the staging area for our SaaS data before replicating the data to ADW.

Please follow Step 1 and Step 2 of this article by Jay Pearson of the Oracle A-Team to create an OCI Object Storage Bucket and gathering the OCI tenancy OCID information to configure the SaaS External Storage in the next step.

Note: if there is an authorization error to access the bucket with a 404 error, please check your OCI User policies. There may need to a new policy that allows the user to manage the object within a compartment: Allow group <GROUP NAME> to manage object-family in compartment <COMPARTMENT NAME>

SaaS External Storage Configuration

Next, Please follow Jay Pearson’s Step 3–5 (beginning with part D. Log onto Oracle Business Intelligence Cloud Connector Console) in the above article to configure the SaaS External Storage.

Note: when testing the connection for the first time, it might fail. Please wait a few moments and try testing the connection again.

Connect to ADW Instance in OAC

Since ADW is the target for the SaaS application data, we will need to connect OAC to ADW. Once this connection is established, OAC Data Replicator can select this ADW instance as the target. Here is a link that shows how to connect to ADW in your OAC instance. Here is what my ADW Connection looks like in OAC:

Note: You can simply drag your zipped wallet credential file in the “Client Credentials” field where it should then auto-populate the “Service Name” field.

OAC Data Replicator Connection

The OAC Data Replicator will orchestrate the SaaS application data push into ADW using Object Storage as the stagging area and BICC for establishing the connection. When at the landing page for OAC, click on “Create”:

Then select “Replication Connection”:

We are then given the following source options:

In my case, since I am using Oracle Supply Chain Management, I will select Oracle Fusion Application Storage. Here are the fields for establishing a connection. We will be using the same Oracle Object Storage we configured earlier to pull the SaaS Application data. Here is some documentation regarding the Data Replicator fields and the expected inputs.

N ote: The “Storage Connection” field is expecting the “Name” field of the BICC Connection entered earlier. In Jay Pearson’s article, the name is “OAC”. N ote: the Storage Tenancy OCID and Storage user OCID is the same as the BICC SaaS External Storage configuration completed earlier. Note: the url is the host name followed by “/bi/ess/esswebservice”. Example is : https://gaf1-zkbg-fb-ext.oracledemos.com/bi/ess/esswebservice

Before using the OAC Data Replicator, we must copy the “Storage API Key” and add this key to our user. To do this, you can follow Jay Pearson’s article, Step 4: Add Public Key to OCI once more.

After we entered all the fields, you can hit “Save”. To validate the connection is established, you can select the hamburger menu, select “Data”, and then select “Data Replications” and see the replication connection.

SCM SaaS Replication to ADW

Now that we have created a SaaS Data Replication connection in OAC, we can replicate data from SaaS to ADW. First, select the Connection tab under “Data” in the OAC hamburger menu:

Next, simply click the Data Replication Connection you just created. Here, you can view the SaaS Object(s) you would like to replicate:

Next, we can select a table within the SaaS Object:

Then, let’s select the target, our ADW instance and default Scheme (Admin) that we connected to earlier:

Finally, we can save and run the Replication Flow:

Note: there are many factors that influence the speed of the replication, network connection, OAC OCPUs, ADW OCPU, concurrency, etc. Thus, the replication process could take some time.

Verify SaaS data in ADW

We can now use SQL Developer or Toad to connect to our ADW instance and verify that the data replication worked. Since I selected the “LookupValuesPVO” in the Customer Data Management Object, I see the corresponding table in my ADW instance. Please check out this link that demonstrates how to connect SQL Developer to your ADW instance using the Wallet Credential.

Conclusion

OAC Data Replicator is a powerful tool that allows organizations to leverage ADW as a single-source-of-truth and data warehouse. This single-source-of-truth can be leverage to allow Data Scientist to leverage SaaS application data to build predictive models about nearly every facet of your business. Your ETL and Integration engineers can leverage the ADW data to update other SaaS applications which might require data from your SCM application or other application. Next steps would be to utilize OAC out-of-the-box machine learning to build models with SaaS application data. You can also REST enable your ADW instance and simply use GET requests to access pertinent data in Jupyter Notebook or other open source tools.