npm package: @itentialopensource/adapter-kafka
Description: [Deprecated] Itential adapter to connect to kafka
Author: Itential
Latest version: 0.5.6

# _Deprecation Notice_
This Adapter has been deprecated as of 09-01-2024 and will be end of life on 09-01-2025. The capabilities of this Adapter have been replaced by the [Adapter Kafkav2](https://gitlab.com/itentialopensource/adapters/adapter-kafkav2)

# Kafka Adapter

## Notice
New kafka adapter utilizing kafkajs library available [here](https://gitlab.com/itentialopensource/adapters/notification-messaging/adapter-kafkav2)

- [Overview](./SUMMARY.md)
- [Versioning](#versioning)
- [Supported IAP Versions](supported-iap-versions)
- [Release History](#release-history)
- [Getting Started](#getting-started)
- [Helpful Background Information](#helpful-background-information)
  - [Enviornment Prerequisites](#enviornment-prerequisites)
  - [Adapter Prerequisites](#adapter-prerequisites)
  - [Additional Prerequisites for Development and Testing](#additional-prerequisites-for-development-and-testing)
  - [Specific Prerequisites](#specific-prerequisites)
  - [Creating a Workspace](#creating-a-workspace)
  - [How to Install](#how-to-install)
- [Testing](#testing)
  - [Unit Testing](#unit-testing)
  - [Integration - Standalont Testing](#integration-testing---standalone)
  - [Integration Testing](#integration-testing)
- [Configuration](./PROPERTIES.md)
- [Using this Adapter](./CALLS.md)
- [Troubleshoot](./TROUBLESHOOT.md)
- [Contributing](./CONTRIBUTING.md)
- [License & Maintainers](#license--maintainers)
- [Product License](#product-license)



## [Overview](./SUMMARY.md)
This adapter is used to integrate the Itential Automation Platform (IAP) with the Kafka System. The adapter utilizes the Kafka-Node library to provide the integrations that are deemed pertinent to IAP. Documentation is available here: [https://www.npmjs.com/package/kafka-node?activeTab=readme]. This ReadMe file is intended to provide information on this adapter.

>**Note**: It is possible that some integrations will be supported through the Kafka adapter while other integrations will not.

Itential provides information on all of its product adapters in the Customer Knowledge Base. Information in the [Customer Knowledge Base](https://itential.atlassian.net/servicedesk/customer/portals) is consistently maintained and goes through documentation reviews. As a result, it should be the first place to go for information.

## Versioning

Itential Product adapters utilize SemVer for versioning. The current version of the adapter can be found in the `package.json` file or viewed in the IAP GUI on the System page. For Open Source Adapters, the versions available can be found in the [Itential OpenSource Repository](https://www.npmjs.com/search?q=itentialopensource%2Fadapter).

## Supported IAP Versions 
Itential Product adapters are built for particular versions of IAP and packaged with the versions they work with.
Itential opensource adapter as well as custom adapters built with the Itential Adapter Builder work acoss many releases of IAP. As a result, it is not often necessary to modify an adapter when upgrading IAP. If IAP has changes that impact the pronghorn.json, like adding a new required section, this will most likely require changes to all adapters when upgrading IAP.
Many of the scripts that come with all adapters built using the Itential Adapter Builder do have some dependencies on IAP or the IAP database schema and so it is possible these scripts could stop working in different versions of IAP. If you notify Itential of any issues, the Adapter Team will attempt to fix the scripts for newer releases of IAP.

## [Release History](./CHANGELOG.md)

Any release prior to 1.0.0 is a pre-release. Initial builds of adapters are generally set up as pre-releases as there is often work that needs to be done to configure the adapter and make sure the authentication process to Kafka works appropriately.

Release notes can be viewed in CHANGELOG.md or in the [Customer Knowledge Base](https://itential.atlassian.net/servicedesk/customer/portals) for Itential adapters.

## Getting Started

These instructions will help you get a copy of the project on your local machine for development and testing. Reading this section is also helpful for deployments as it provides you with pertinent information on prerequisites and properties.

## Helpful Background Information 
There is adapter documentation available on the Itential Developer Site <a href="https://www.itential.com/automation-platform/integrations/adapters-resources/" target="_blank">HERE</a>. This documentation includes information and examples that are helpful for:

```text
Authentication
Properties
Code Files
Action Files
Schema Files
Mock Data Files
Linting and Testing
Troubleshooting
```

Others will be added over time.
Want to build a new adapter? Use the <a href="https://adapters.itential.io" target="_blank">Itential Adapter Builder</a>


### Environment Prerequisites

The following is a list of required packages for an adapter.

```json
Node.js
Git
```

### Adapter Prerequisites

The following list of packages are required for Itential product adapters or custom adapters that have been built utilizing the Itential Adapter Builder.

| Package | Description |
| ------- | ------- |
| @itentialopensource/adapter-utils | Runtime library classes for all adapters;  includes request handling, connection, throttling, and translation. |
| ajv | Required for validation of adapter properties to integrate with Kafka. |
| fs-extra | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
| readline-sync | Utilized by the testRunner script that comes with the adapter;  helps to test unit and integration functionality. |

### Additional Prerequisites for Development and Testing

If you are developing and testing a custom adapter, or have testing capabilities on an Itential product adapter, you will need to install these packages as well.

```json
chai
eslint
eslint-config-airbnb-base
eslint-plugin-import
eslint-plugin-json
mocha
nyc
testdouble
winston
```

### Specific Prerequisites

At the current time the Kafka adapter does not utilize the adapter utilities as it makes use of the following library instead.

| Package | Description |
| ------- | ------- |
| kafka-node | Library that provides kafka connectivity through nodejs. |

### Creating a Workspace

The following provides a local copy of the repository along with adapter dependencies.

```json
git clone git@gitlab.com:\@itentialopensource/adapters/adapter-Kafka
npm install
```

## How to Install 
1. Set up the name space location in your IAP node_modules.

```json
cd /opt/pronghorn/current/node_modules
if the @itentialopensource directory does not exist, create it:
   mkdir @itentialopensource
```

2. Clone the adapter into your IAP environment.

```json
cd \@itentialopensource
git clone git@gitlab.com:\@itentialopensource/adapters/adapter-Kafka
```

3. Install the dependencies for the adapter.

```json
cd adapter-Kafka
npm install
```

4. Add the adapter properties for Kafka (created from Adapter Builder) to the `properties.json` file for your Itential build. You will need to change the credentials and possibly the host information below.
[Kafka sample properties](sampleProperties.json)

5. Restart IAP

```json
systemctl restart pronghorn
```

## Testing an Itential Product Adapter

Mocha is generally used to test all Itential Product Adapters. There are unit tests as well as integration tests performed. Integration tests can generally be run as standalone using mock data and running the adapter in stub mode, or as integrated. When running integrated, every effort is made to prevent environmental failures, however there is still a possibility.


### Unit Testing

Unit Testing includes testing basic adapter functionality as well as error conditions that are triggered in the adapter prior to any integration. There are two ways to run unit tests. The prefered method is to use the testRunner script; however, both methods are provided here.


```json
node utils/testRunner --unit

npm run test:unit
```

To add new unit tests, edit the `test/unit/adapterTestUnit.js` file. The tests that are already in this file should provide guidance for adding additional tests.

### Integration Testing - Standalone

Standalone Integration Testing requires mock data to be provided with the entities. If this data is not provided, standalone integration testing will fail. When the adapter is set to run in stub mode (setting the stub property to true), the adapter will run through its code up to the point of making the request. It will then retrieve the mock data and return that as if it had received that data as the response from Kafka. It will then translate the data so that the adapter can return the expected response to the rest of the Itential software. Standalone is the default integration test.

Similar to unit testing, there are two ways to run integration tests. Using the testRunner script is better because it prevents you from having to edit the test script; it will also resets information after testing is complete so that credentials are not saved in the file.

```json
node utils/testRunner
  answer no at the first prompt

npm run test:integration
```

To add new integration tests, edit the `test/integration/adapterTestIntegration.js` file. The tests that are already in this file should provide guidance for adding additional tests.

### Integration Testing

Integration Testing requires connectivity to Kafka. By using the testRunner script it prevents you from having to edit the integration test. It also resets the integration test after the test is complete so that credentials are not saved in the file.

> **Note**: These tests have been written as a best effort to make them work in most environments. However, the Adapter Builder often does not have the necessary information that is required to set up valid integration tests. For example, the order of the requests can be very important and data is often required for `creates` and `updates`. Hence, integration tests may have to be enhanced before they will work (integrate) with Kafka. Even after tests have been set up properly, it is possible there are environmental constraints that could result in test failures. Some examples of possible environmental issues are customizations that have been made within Kafka which change order dependencies or required data.

```json
node utils/testRunner
answer yes at the first prompt
answer all other questions on connectivity and credentials
```

Test should also be written to clean up after themselves. However, it is important to understand that in some cases this may not be possible. In addition, whenever exceptions occur, test execution may be stopped, which will prevent cleanup actions from running. It is recommended that tests be utilized in dev and test labs only.

> **Reminder**: Do not check in code with actual credentials to systems.

## [Configuration](./PROPERTIES.md)

## [Using this Adapter](./CALLS.md)

## [Troubleshoot](./TROUBLESHOOT.md)


## [Contributing](./CONTRIBUTING.md)


## License & Maintainers


```json
Itential Product Adapters are maintained by the Itential Adapter Team.
Itential OpenSource Adapters are maintained by the community at large.
Custom Adapters are maintained by other sources.
```

### Product License

[Apache 2.0](./LICENSE)