Integrating the authentication systems of IBM Cloud Pak for Data and Palantir Foundry
Tags: Authentication, IBM, Palantirtech, Palantir, Integration

Integrating the authentication systems of IBM Cloud Pak for Data and Palantir Foundry Palantir Follow Feb 8 · 5 min read

This week, Palantir and IBM announced a partnership that will help businesses supercharge their data impact with easily deployable AI applications. The new solution combines key modules from Palantir Foundry with IBM’s AI for business capabilities and hybrid cloud data platform — Cloud Pak for Data.

A particularly interesting technical challenge — and the topic for this blog post — is the integration of the respective authentication and authorization subsystems in IBM Cloud Pak for Data and Palantir Foundry.

For brevity, throughout this post, we will refer to IBM Cloud Pak for Data as either “Cloud Pak for Data” or “Cloud Pak,” and the key modules from Palantir Foundry as “Palantir.”

Auth-n/z in Cloud Pak and Palantir

Let’s start with a brief description of the respective IBM Cloud Pak for Data and Palantir auth-n/z architectures. Each platform encapsulates auth-n/z concerns in a central component, Multipass in Palantir and the User Management service in IBM Cloud Pak for Data. In a common enterprise Single-Sign-on (SSO) configuration, this component negotiates user logins with existing identity providers such as Azure AD or Okta through SAML or Kerberos+LDAP, X509+LDAP, etc. Upon successful login, Multipass and Cloud Pak’s User Management service issue JSON Web Tokens (JWT) that are typically stored in the user’s browser session as a cookie. Users then pass the JWT to Cloud Pak / Palantir backend APIs to prove their identity.

In Cloud Pak for Data, JWTs convey both authentication claims and authorization grants. Cloud Pak’s auth component maps user groups to grants and encodes grants in the JWTs issued to Cloud Pak users. Each Cloud Pak application then makes local decisions to grant or deny users access to a particular resource or operation, depending on the grants encoded in the JWT. In contrast, Palantir relies on the central Multipass service to handle the authorization decision: when a user tries to perform a particular operation (e.g., “read-data”) on a given resource (e.g., a dataset identifier) in some application, the application calls Multipass’s hasOperation(token, operation, resource) API to determine whether to grant or deny access. The additional complexity of encapsulating authorization rules in a central service allows us to implement complex access control workflows such as provenance-aware access control.

Seamless integration

In order to provide a seamless user experience between Cloud Pak for Data and Palantir, we have worked with IBM engineers to devise an integrated auth-n/z solution. Users typically log in through Cloud Pak’s SSO integration to obtain a Cloud Pak JWT that is valid for all Cloud Pak applications. When switching to a Palantir application, we transparently exchange the Cloud Pak JWT for a Palantir JWT so that users won’t have to log in a second time before accessing the Palantir application. Let’s look at the token exchange mechanics in more detail.

The core authentication system in Multipass is split into so-called collectors, credentials, and providers. The job of a collector is to obtain user credentials from an external system; for example, the form collector allows users to enter their credentials as username and password through a UI, or the more complex SAML collector receives credentials from an upstream identity provider in the form of signed SAML assertions. Multipass then internally forwards the obtained credentials to a configurable provider; the job of the provider is to return group information (e.g., the user’s AD groups in the case of an Azure AD configuration) and user details (e.g., the user’s email address).

Notably, collectors and providers are configurable and polymorphic; this means that Multipass can plug together different collectors and providers in order to combine their capabilities. We used exactly this flexibility to integrate with Cloud Pak for Data: we combine a CookieCollector that extracts and verifies the Cloud Pak JWT with a Cloud Pak specific provider that reaches out to the Cloud Pak User Management API to fetch additional user details. When a user hits a Palantir application without presenting a valid Palantir JWT, the user gets redirected (via OAuth2) to Multipass. Multipass checks for the presence of a Cloud Pak JWT, redirects through the Cloud Pak login flow if necessary, and invokes the collector and provider to verify the Cloud Pak JWT and construct a corresponding Palantir JWT. After the redirect, the browser holds both the Cloud Pak JWT and the corresponding Palantir JWT for the user.

API gateway

A missing piece in the puzzle is how Palantir’s backend services forward user identify to Cloud Pak backend APIs. For example, a user may load a Palantir dashboard, thereby presenting a Palantir JWT to the dashboard backend service. This backend service now needs to reach out to Cloud Pak in order to fetch the data underlying the dashboard. Since the Palantir user only passes their Palantir JWT to the Palantir backend service, this service needs to exchange the user’s Palantir JWT for a valid Cloud Pak JWT that can be handed to the Cloud Pak backend.

Fortunately, our Multipass Cloud Pak provider is aware of this mapping from Palantir user to Cloud Pak user and can offer a suitable token exchange API endpoint that Palantir backend services can call. We take the user’s unique Palantir user ID and match that with their unique Cloud Pak user ID, then use Cloud Pak APIs to generate a signed, short-lived JWT for that user.

In order to make life a bit easier for our services and developers, we decided to also implement a passthrough Palantir-to-Cloud-Pak API gateway that performs this token exchange behind the scenes. As extra convenience for our developers, this gateway exposes the relevant Cloud Pak APIs in our standard RPC framework, Conjure.

Conclusions

Integrating two transformational software platforms poses a ton of interesting technical challenges — like this Cloud Pak auth-n/z integration. But, critically, this integration instantly unlocks the full potential of key Palantir modules for existing and future IBM Cloud Pak customers — supercharging their operational decision making, and accelerating their digital transformation.

Authors

Trey D., Robert F., Tom P., Gaurav T., Will M.