Apache Pulsar OAuth2 dev setup with Docker and KeyCloak
Tags: pulsar, docker, keycloak, oauth2
Published: 2020-12-28 00:35:41

So I've been eager to use something easier to set up than generating those TLS certificates and such for development environment, yet still somewhat more secure than just hardcoded JWT inside a file in Pulsar VM (container in this case). If you're not familiar with JWT setup for Pulsar, it's not necessary, but you can think of it as lesser OAuth2 since they both rely mainly on verifying JWT but process for logging in with Brokers and Clients can be different.

#KeyCloak setup
KeyCloak requires only client app to be set up for this. App needs to Service Accounts Enabled to be on which is basically _client_credentials_ grant type in OAuth2. In order to see this option _Access Type_ has to be set to _"confidential"_.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/e3ci70vwpbpxom1dvsnv.png)

Next thing to set it the "specialised" claim. This claim would be easy to set as _realm role_ or _client role_ but Apache Pulsar has issues reading complex types in JWT by default, and so to skip writing custom authentication resolver we must enforce claim inside JWT to be simple string. Problem is that role claims by default go into array even if specified as non-multivalued. So one could get something like ```json 
"role": "[admin]" or
"role": ["admin"]
```
but we need ```json 
"role": "admin"
```
for Pulsar to play nice with defaults.
which is unexpected given that it should be only 1 string. To skip this hassle go and set-up _Hardcoded claim_.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/me3uafx57s6dhbamfmok.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/jb1kguoinpvjdrhc4nsx.png)

__Token claim name__ can be __*role*__ and value can be for this purpose __superuser__. These values are important for Pulsar setup so keep track of what was put in here. Pulsar will require you to set up role name for the _superuser_ which is dedicated role for managing all Pulsar stuff. On the other hand other roles might be used but then you need to remember to _setup tenant namespaces to be accessible by that role_. This means one would have to manage that pulsar instance manually through REST or CLI or something else to setup tenant namespaces manageable by user role you wish to use by certain app.

Next, get the public key of the realm. Easy you might thing? Well no, again because of Pulsar. Go to _Realm Settings_ and into _Keys_ tab. 
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/stf966kovcqst01r0458.png)
There you should find _RS256_ under active tab if all defaults were left as-is. __If you wish to use different algorithm please also mark that as important and keep track of it as well as of the key value__. If not then please continue and click on _Public Key_ of that RS256 row.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/wqrhsn4mt4ct5zmbwtbk.png)
You should get some Base64 text. It will look like gibberish of alphanumeric and some special characters. 

Now copy that value and somehow __convert it to bytes and store as file__. Pulsar lib for java at 2.7 will try to read it as x509 which will fail as it needs to be decoded into bytes prior to this. Steps to do it are:
1. Copy base64 value
2. Decode it back to bytes using either something online or simply program your own mini script
3. Store bytes into file
A Groovy example:
```groovy
byte[] key = "MACakje21/adkjwp9qmk4231/ea\d;qwdq=="..decodeBase64()
File f = new File("yourfilename")
f.bytes = key
```
Why Groovy? Well it was fast to write and has all those `.decode` and `.bytes` and... you can execute it as script through IntelliJ - Tools -> Groovy Console. I was mainly using JVM things with pulsar and Python but Python fails to use KeyCloak settings because of some previous bug which was fixed for Java and C++ but apparently not for Python which should rely on C++ lib.

#Pulsar image
After setting up KeyCloak and storing that key to some file we can build a customised image that uses OAuth2 in __standalone__ mode.

Now create a file that you will later copy into image. Please visit [Pulsar docs](https://pulsar.apache.org/docs/en/security-oauth2/) if you want more info. If not then just treat this as credentials file for OAuth2 client app so client ID and secret from KeyCloak (go to App then first tab contains ID and Credentials tab has the secret). Below is simple example
```json
{
    "type": "client_credentials",
    "client_id": "pulsar",
    "client_secret": "dadada7a7a7-a7ad77ad7da-da7a7ad7ad77",
    "issuer_url": "https://host.docker.internal/auth/realms/test"
}
```
I named it __oauth2.json__. _Download files conf/standalone.conf and conf/client.conf from [Pulsar repository](https://github.com/apache/pulsar/tree/master/conf). These files will be changed to configure Pulsar image to use Authentication.

Configure __standalone.conf__. Set properties in that file to something like in below example@
```properties
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken
# here goes the role you've picked in case you want KeyCloak client app to behave in superuser or else just leave default
superUserRoles=superuser

#brokers need to login to other things so they must also be set up
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2
#oauth2.json path should be set to wherever Dockerfile is copying it
brokerClientAuthenticationParameters={"issuerUrl": "https://host.docker.internal/auth/realms/test","privateKey": "/pulsar/oauth2.json","audience": "pulsar"}
#bytes from base64, file path must be as defined by Dockerfile
tokenPublicKey=file:///pulsar/oauth_public.key
```
This will tell brokers to use JSON passed as a parameter, extract file under __privateKey__ and post it's content as payload to __isssuerUrl__. Thus generating the JWT and refreshing when necessary. __*brokerClientAuthenticationPlugin*__ is different from the __authenticationProviders__ because broker generates JWT every now and then by authenticating to KeyCloak while main provider will only verify that token is valid (non-expired and signature is good). __*tokenPublicKey*__ will instruct which file to use to verify JWT signature. It's the same as for simple JWT setup without the OAuth2. What's different here is that we need public key from the server while JWT can have key pair generated for Pulsar only and used by it or symmetric key for the same purpose. OAuth2 should not share it's private keys thus symmetric key is not a good option. KeyCloak by default uses asymmetric so it helps quite a bit.

_Alternatively, if KeyCloak is set to never expire tokens, one can be generated in advance for Pulsar brokers and then copied into Pulsar container. This could be used instead of AuthenticationOAuth2 and parameters for broker would be file path to JWT token file._ I would not advice alternative approach as it forces you to use never-expiring tokens on all of your realm not just that particular app.

Don't forget to configure your CLI tools. These are needed to run, add tenants/namespaces..., test topics, and do any kind of administration through CLI.
```properties
authPlugin=org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2

authParams={"issuerUrl": "https://host.docker.internal/auth/realms/test","privateKey": "/pulsar/oauth2.json","audience": "pulsar"}
```
So same values are used as for broker configuration plugin and params.

And finally Dockerfile. I made something like this:
```Dockerfile
FROM apachepulsar/pulsar-standalone:2.7.0
WORKDIR /pulsar
#using workdir /pulsar we copy public key and it's path will be /pulsar/oauth_public.key and this value must be same as in properties for standalone
COPY oauth_public.key oauth_public.key
#note oauth2.json is being copied from into workdir /pulsar which is linked in brokerClientAuthenticationParameters
COPY oauth2.json oauth2.json
#client for CLI tools
COPY client.conf conf/client.conf
#
COPY standalone.conf conf/standalone.conf
```
If you used same naming then you should be good to go with this image. One note is that I'm relying on __-standalone__ instead of just _pulsar_ or _pulsar-all_. If you choose something else you might want to add start-up command at the end to run pulsar when you trigger container run from docker. Standalone version runs automatically but others I used didn't so optionally use at the end
```Dockerfile
CMD [ "/pulsar/bin/pulsar", "standalone"]
```
if you used other image as a base.

#Build image and run
This should be sufficient to run it. I made something similar and tested it with CLI, Java, and Python clients where I've noticed that Python complains about `.well-known/...` which was exception I got when Java library was not fixed to respect path of Issuer URL. What was wrong with Java lib is that URL was stripped of path and only root part was used so if you have http://keycloak/realms/test it would only take http://keycloak/ so keep that in mind while testing.

Hope it goes well.