Splunk logging in lambda using low code approach
Tags: serverless, aws, microservices, kubernetes
Published: 2020-06-30 12:25:14

![Splunk Kumo header](https://miro.medium.com/max/1400/1*2-nOfUKSGDqeh2FHCRKHkg.png)

A centralised logging management solution like: Splunk, Datadog, Sumologic, etc. enables organisations to collect, analyse, and display logs through a single pane of glass.

In this article we will see how application logs can be sent to Splunk from lambda using **Kumologica Splunk** node.

[Kumologica](https://www.kumologica.com/) is a free low-code development tool to build serverless integrations. You can learn more in this [medium article](https://medium.com/@kumologica/kumologica-first-low-code-development-tool-for-aws-lambda-8831b5237428)

**Use case**

In this use case we have a mobile app application that consumes a Mobile Channel API. Behind the mobile API we have an Order FulFilment API. The Mobile API invokes the Order FulFilment API for completing the process. When the mobile app sends the very initial request it has a Tracking id. The Tracking id traverses from the Mobile Channel API to Order FulFilment API and back.

The message passed across both applications needs to be traced based on the Tracking id in Splunk.

![Use case design](https://cdn-images-1.medium.com/max/2000/1*7NYm88g4W9ERXqXQ0PQyDQ.png)

**Prerequisite**

1. Kumologica designer installed in your machine. [https://kumologica.com/download.html](https://kumologica.com/download.html)

2. Splunk cloud [Trial](https://www.splunk.com/en_us/download/get-started-with-your-free-trial.html) account

**Implementation**

For the above mentioned use case we will be creating two API’s , a Mobile Channel API and an Order FulFilment API.

*Order fulfilment API*

1 . Open the Kumologica designer and *Create a New project*. And provide the name of the project as **OrderFulFilmentService**

2 . Remove the default hello world flow.

3 . Drag and drop **EventListener** Node from the pallet to the canvas and open the EvenListener node settings and configure the following.

```
    Display Name : POST /order
    Event Source : Amazon API gateway
    Verb : POST
    URL : /order
```

4 . Add **Splunk Node** from Logging category.

By default Splunk Node will not be available as part of the default **Logging** category.

In order to install the Splunk node , Go to **Add More Nodes** Option on the bottom of the pallet and click **Install** Splunk. Once Installation is completed a restart of the Designer will be required. Open the Splunk Node settings and configure the following.

```
    Display Name : Log_Entry
    Host : hostname of your splunk trial account
    Port : 8088
    Message : {“TrackingID” : msg.payload.TxnID,”ServiceName”:”Order_FullFillment”, “Data”: msg.payload }
    Splunk HEC Token : Splunk HTTP Event collector Token
```

5 . Wire the EventListener Node to the Splunk Node.

6 . Drag and drop **Datamapper** node from the palette to the canvas.

We are using the Datamapper node to mimic the response after an Order processing.

Open the Datamapper settings and configure the following.

```
    Display Name : Order_ProcessResponse

    Mapping : {
                “TrackingID” : msg.payload.TxnID,
                “OrderID” : msg.payload.OrderID,
               “Status” : “Processed”
              }
```

If you are not familiar using the DataMapper node then would recommend to go through the [following article](https://medium.com/@arunkumar.satyam/data-mapping-in-kumologica-5913dc4f4c09).

7 . Wire the Splunk Node to the DataMapper Node.

8 . Add the **EventListener End** node from the palette to the canvas. Open the settings and configure the following.

```
    Display Name : Success
    Payload : msg.payload
```

With this we have completed the development of our Order FulFilment API.

![Order Fulfilment API Flow](https://cdn-images-1.medium.com/max/2444/0*YcIA3BGEAoKXt75F)

Now let’s deploy the API as a lambda to your AWS account. If you are not familiar with the deployment of a Kumologica flow I would recommend to have a walkthrough on the following [medium article](https://medium.com/@kumologica/kumologica-first-low-code-development-tool-for-aws-lambda-8831b5237428).

**Note** : Copy the service url that you receive on the terminal after a successful deployment.

*Mobile Channel API*

1 . Open the Kumologica designer and *Create a New project*. And provide the name of the project as **MobileChannelAPIService**

2 . Remove the default hello world flow.

3 . Drag and drop **EventListener** Node from the pallet to the canvas and open the EvenListener node settings and configure the following.

```
    Display Name : POST /mobile
    Event Source : Amazon API gateway
    Verb : POST
    URL : /mobile
```

4 . Add **Splunk Node** from Logging category.

By default Splunk Node will not be available as part of the default **Logging** category.

In order to install the Splunk node , Go to **Add More Nodes** Option on the bottom of the Pallet and click Install Splunk. Once Installation is completed a restart of the Designer will be required. Open the Splunk Node settings and configure the following.

```
    Display Name : Log_Entry
    Host : hostname of your splunk trial account
    Port : 8088
    Message : {“TrackingID” : msg.payload.TxnID,”ServiceName”:”Mobile_Channel”, “Data”: msg.payload }
    Splunk HEC Token : Splunk HTTP Event collector Token
```

5 . Wire the EventListener node to the Splunk Node.

6 . Add the **HTTP Req** node from the web category in the palette to the canvas. Open the settings and configure the following.

    Display Name : Invoke_OrderFulFilmentService
    Method : POST
    Url : your order service url. Configure this once Order FulFilment Service is deployed and you receive the url .
    Return : a UTF-8 string
    Response Timeout : 120000ms (default)
    Authentication : None

7 . Wire the Splunk Node to the HTTP Req node.

8 . Add the **EventListener End** node from the palette to the canvas. Open the settings and configure the following.

```
    Display Name : Success
    Payload : msg.payload
```

With this we have completed the development of our Mobile Channel API.

![Mobile Channel API Flow](https://cdn-images-1.medium.com/max/2220/0*u3nTS3ELRMOVbMdC)

Now let’s deploy both the services separately as two different lambda in your AWS account.

### Testing

For testing purposes I am sending the following request as input to the Mobile API.

```json
    {
    “OrderID”: “APQ-00001”,
    “TxnID” : “0948939-AWHDH”,
    “Item” : {
    “Name” : “ABC Laptop”,
    “Type” : “Mini-5”,
    “Qty” : 1
    },
    “Address” :{
    “Unit” : “9”,
    “Street” : “TTK Street”,
    “Road” : “Tower Road”,
    “State” : “NSW”,
    “Country” : “Australia”,
    “PIN” : 94839
    }
    }
```

After you invoke the mobile channel API and when you search in Splunk with the **TxnID** value as the search parameter

![Splunk search](https://cdn-images-1.medium.com/max/3200/0*oXZ8yvL_PzD1n8GY)

you would get the following log entries.

![Event Received by Mobile Channel API](https://cdn-images-1.medium.com/max/3200/0*8eZ7c2ikA89t_N_J)

![Event Received by Order FulFilment API](https://cdn-images-1.medium.com/max/3200/0*6_hQkOaIFAHaHEoF)

If you want to try out this application quickly you can import the loan broker flow from this [project’s repo](https://github.com/KumologicaHQ/kumologica-demos/tree/master/SplunkLogging).

**Summary**

Thanks for reading and I hope you enjoyed this use case and appreciate how simple it is to implement an orchestration service with [Kumologica](https://kumologica.com/). We would love to hear your feedback, and don’t hesitate to contact us if you have any questions.

To learn more about Kumologica checkout our official documentation at [kumologica.com](http://kumologica.com/) and subscribe to our youtube [channel](https://www.youtube.com/channel/UCsTDAlhijUFpsfThJu3hF9g)