Containerising .Net/Java Gateways (or Kafka Integration Demo)
Tags: beginners, docker, java, net
Published: 2022-01-25 18:00:34

In this article, I will show how you can easily containerize  .Net/Java Gateways.

For our example, we will develop an Integration with [Apache Kafka](https://kafka.apache.org/).

And to interoperate with Java/.Net code we will use [PEX](https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls?KEY=EPEX) .

 

## Architecture

Our solution will run completely in docker and look like this:

 

![](https://raw.githubusercontent.com/intersystems-community/pex-demo/master/architecture.PNG) 

 

## Java Gateway

First of all, let's develop Java Operation to send messages into Kafka. The code can be written in your IDE of choice and it can [look like this](https://github.com/intersystems-community/pex-demo/blob/master/java/src/dc/rmq/KafkaOperation.java).

In short:

  * To develop new PEX Business Operation we need to implement  abstract <tt>com.intersystems.enslib.pex.BusinessOperation </tt>class
  * Public properties are Business Host Settings
  * <tt>OnInit</tt> method is used to init connection to Kafka and get a pointer to InterSystems IRIS
  * <tt>OnTearDown</tt> is used to disconnect from Kafka (at process shutdown)
  * <tt>OnMessage</tt> receives [dc.KafkaRequest message](https://github.com/intersystems-community/pex-demo/blob/master/iris/src/dc/KafkaRequest.cls) and sends it to Kafka

Now let's pack it into Docker!

Here's our [dockerfile](https://github.com/intersystems-community/pex-demo/blob/master/java/Dockerfile):

    FROM openjdk:8 AS builder
    
    ARG APP_HOME=/tmp/app
    
    COPY src $APP_HOME/src
    
    COPY --from=intersystemscommunity/jgw:latest /jgw/*.jar $APP_HOME/jgw/
    
    WORKDIR $APP_HOME/jar/
    ADD https://repo1.maven.org/maven2/org/apache/kafka/kafka-clients/2.5.0/kafka-clients-2.5.0.jar .
    ADD https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar .
    ADD https://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar .
    ADD https://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.jar .
    
    WORKDIR $APP_HOME/src
    
    RUN javac -classpath $APP_HOME/jar/*:$APP_HOME/jgw/* dc/rmq/KafkaOperation.java && \
        jar -cvf $APP_HOME/jar/KafkaOperation.jar dc/rmq/KafkaOperation.class
    
    FROM intersystemscommunity/jgw:latest
    
    COPY --from=builder /tmp/app/jar/*.jar $GWDIR/

Let's go line by line and see what's going on here (I assume familiarity with [multi-stage docker builds](https://docs.docker.com/develop/develop-images/multistage-build/)):

    FROM openjdk:8 AS builder

Our starting image is JDK 8.

    ARG APP_HOME=/tmp/app
    COPY src $APP_HOME/src

We're copying our sources from <tt>/src</tt> folder into <tt>/tmp/app</tt> folder.

COPY --from=intersystemscommunity/jgw:latest /jgw/*.jar $APP_HOME/jgw/

We're copying Java gateway sources into <tt>/tmp/app/jgw </tt>folder.

    WORKDIR $APP_HOME/jar/
    ADD https://repo1.maven.org/maven2/org/apache/kafka/kafka-clients/2.5.0/kafka-clients-2.5.0.jar .
    ADD https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar .
    ADD https://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar .
    ADD https://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.jar .
    
    WORKDIR $APP_HOME/src
    
    RUN javac -classpath $APP_HOME/jar/*:$APP_HOME/jgw/* dc/rmq/KafkaOperation.java && \
        jar -cvf $APP_HOME/jar/KafkaOperation.jar dc/rmq/KafkaOperation.class

Now all dependencies are added and <tt>javac</tt>/<tt>jar</tt> is called to compile the jar file. For a real-life projects it's better to use maven or gradle.

    FROM intersystemscommunity/jgw:latest
    
    COPY --from=builder /tmp/app/jar/*.jar $GWDIR/

And finally, the jars are copied into base jgw image (base image also takes care of starting the gateway and related tasks).

## .Net Gateway

Next is .Net Service which will receive messages from Kafka. The code can be written in your IDE of choice and it can [look like this](https://github.com/intersystems-community/pex-demo/blob/master/dotnet/KafkaConsumer.cs).

In short:

  * To develop new PEX Business Service we need to implement  abstract <tt>InterSystems.EnsLib.PEX.BusinessService </tt>class
  * Public properties are Business Host Settings
  * <tt>OnInit</tt> method is used to init connection to Kafka and subscribe to topics and get a pointer to InterSystems IRIS
  * <tt>OnTearDown</tt> is used to disconnect from Kafka (at process shutdown)
  * <tt>OnMessage</tt> consumes messages from Kafka and sends <tt>Ens.StringContainer</tt> messages to other Interoperability hosts

Now let's pack it into Docker!

Here's our [dockerfile](https://github.com/intersystems-community/pex-demo/blob/master/dotnet/Dockerfile):

    FROM mcr.microsoft.com/dotnet/core/sdk:2.1 AS build
    
    ENV ISC_PACKAGE_INSTALLDIR /usr/irissys
    ENV GWLIBDIR lib
    ENV ISC_LIBDIR ${ISC_PACKAGE_INSTALLDIR}/dev/dotnet/bin/Core21
    
    WORKDIR /source
    COPY --from=store/intersystems/iris-community:2020.2.0.211.0 $ISC_LIBDIR/*.nupkg $GWLIBDIR/
    
    # copy csproj and restore as distinct layers
    COPY *.csproj ./
    RUN dotnet restore
    
    # copy and publish app and libraries
    COPY . .
    RUN dotnet publish -c release -o /app
    
    # final stage/image
    FROM mcr.microsoft.com/dotnet/core/runtime:2.1
    WORKDIR /app
    COPY --from=build /app ./
    
    # Configs to start the Gateway Server
    RUN cp KafkaConsumer.runtimeconfig.json IRISGatewayCore21.runtimeconfig.json && \
        cp KafkaConsumer.deps.json IRISGatewayCore21.deps.json
    
    ENV PORT 55556
    
    CMD dotnet IRISGatewayCore21.dll $PORT 0.0.0.0

Let's go line by line:

    FROM mcr.microsoft.com/dotnet/core/sdk:2.1 AS build

We use full .Net Core 2.1 SDK to build our app.

    ENV ISC_PACKAGE_INSTALLDIR /usr/irissys
    ENV GWLIBDIR lib
    ENV ISC_LIBDIR ${ISC_PACKAGE_INSTALLDIR}/dev/dotnet/bin/Core21
    
    WORKDIR /source
    COPY --from=store/intersystems/iris-community:2020.2.0.211.0 $ISC_LIBDIR/*.nupkg $GWLIBDIR/

Copy .Net Gateway NuGets from official InterSystems Docker image into our builder image

    # copy csproj and restore as distinct layers
    COPY *.csproj ./
    RUN dotnet restore
    
    # copy and publish app and libraries
    COPY . .
    RUN dotnet publish -c release -o /app

Build our library.

    # final stage/image
    FROM mcr.microsoft.com/dotnet/core/runtime:2.1
    WORKDIR /app
    COPY --from=build /app ./

Copy library dlls into the final container we will actually run.

    # Configs to start the Gateway Server
    RUN cp KafkaConsumer.runtimeconfig.json IRISGatewayCore21.runtimeconfig.json && \
        cp KafkaConsumer.deps.json IRISGatewayCore21.deps.json

Currently, .Net Gateway must load all dependencies on startup, so we make it aware of all possible dependencies.

    ENV PORT 55556
    
    CMD dotnet IRISGatewayCore21.dll $PORT 0.0.0.0

Start gateway on port 55556 listening on all interfaces.

And we're done!

Here's a complete [docker-compose](https://github.com/intersystems-community/pex-demo/blob/master/docker-compose.yml) to get it all running together (including Kafka and Kafka UI to see the messages).

To run the demo you need:

  1. Install: 
      * <a href="https://docs.docker.com/get-docker/" rel="nofollow">docker</a>
      * <a href="https://docs.docker.com/compose/install/" rel="nofollow">docker-compose</a>
      * <a href="https://git-scm.com/book/en/v2/Getting-Started-Installing-Git" rel="nofollow">git</a>
  2. Execute:

    git clone https://github.com/intersystems-community/pex-demo.git
    cd pex-demo
    docker-compose pull
    docker-compose up -d
    

 

**Important notice**: Java Gateway and .Net Gateway libraries MUST come from the same version as InterSystems IRIS client.