Chapter 107. OpenSearch Since Camel 4.0 Only producer is supported The OpenSearch component allows you to interface with an OpenSearch 2.8.x API using the Java API Client library. 107.1. Dependencies When using bean with Red Hat build of Camel Spring Boot make sure to use the following Maven dependency to have support for auto configuration: <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-opensearch-starter</artifactId> </dependency> 107.2. URI format 107.3. Configuring Options Camel components are configured on two levels: Component level Endpoint level 107.3.1. Component Level Options The component level is the highest level. The configurations you define at this level are inherited by all the endpoints. For example, a component can have security settings, credentials for authentication, urls for network connection, and so on. Since components typically have pre-configured defaults for the most common cases, you may need to only configure a few component options, or maybe none at all. You can configure components with Component DSL in a configuration file ( application.properties or yaml ), directly with Java code. 107.3.2. Endpoint Level Options At the Endpoint level you have many options, which you can use to configure what you want the endpoint to do. The options are categorized according to whether the endpoint is used as a consumer ( from ) or as a producer ( to ) or used for both. You can configure endpoints directly in the endpoint URI as path and query parameters. You can also use Endpoint DSL and DataFormat DSL as type safe ways of configuring endpoints and data formats in Java. When configuring options, use Property Placeholders for urls, port numbers, sensitive information, and other settings. Property placeholders provide a few benefits: They help prevent using hardcoded urls, port numbers, sensitive information, and other settings. They allow externalizing the configuration from the code. They help the code to become more flexible and reusable. 107.4. Component Options The OpenSearch component supports 13 options, which are listed below. Name Description Default Type connectionTimeout (producer) The time in ms to wait before connection will time out. 30000 int hostAddresses (producer) Comma separated list with ip:port formatted remote transport addresses to use. The ip and port options must be left blank for hostAddresses to be considered instead. String lazyStartProducer (producer) Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. false boolean maxRetryTimeout (producer) The time in ms before retry. 30000 int socketTimeout (producer) The timeout in ms to wait before the socket will time out. 30000 int autowiredEnabled (advanced) Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. true boolean client (advanced) Autowired To use an existing configured OpenSearch client, instead of creating a client per endpoint. This allows customizing the client with specific settings. RestClient enableSniffer (advanced) Enable automatically discover nodes from a running OpenSearch cluster. If this option is used in conjunction with Spring Boot, then it's managed by the Spring Boot configuration (see: Disable Sniffer in Spring Boot). false boolean sniffAfterFailureDelay (advanced) The delay of a sniff execution scheduled after a failure (in milliseconds). 60000 int snifferInterval (advanced) The interval between consecutive ordinary sniff executions in milliseconds. Will be honoured when sniffOnFailure is disabled or when there are no failures between consecutive sniff executions. 300000 int enableSSL (security) Enable SSL. false boolean password (security) Password for authenticating. String user (security) Basic authenticate user. String 107.5. Endpoint Options The OpenSearch endpoint is configured using URI syntax: With the following path and query parameters: 107.5.1. Path Parameters (1 parameters) Name Description Default Type clusterName (producer) Required Name of the cluster. String 107.5.2. Query Parameters (19 parameters) Name Description Default Type connectionTimeout (producer) The time in ms to wait before connection will time out. 30000 int disconnect (producer) Disconnect after it finishes calling the producer. false boolean from (producer) Starting index of the response. Integer hostAddresses (producer) Comma separated list with ip:port formatted remote transport addresses to use. String indexName (producer) The name of the index to act against. String maxRetryTimeout (producer) The time in ms before retry. 30000 int operation (producer) What operation to perform. Enum values: Index Update Bulk GetById MultiGet MultiSearch Delete DeleteIndex Search Exists Ping OpensearchOperation scrollKeepAliveMs (producer) Time in ms during which OpenSearch will keep search context alive. 60000 int size (producer) Size of the response. Integer socketTimeout (producer) The timeout in ms to wait before the socket will time out. 30000 int useScroll (producer) Enable scroll usage. false boolean waitForActiveShards (producer) Index creation waits for the right consistency number of shards to be available. 1 int lazyStartProducer (producer (advanced)) Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. false boolean documentClass (advanced) The class to use when deserializing the documents. ObjectNode Class enableSniffer (advanced) Enable automatically discover nodes from a running OpenSearch cluster. If this option is used in conjunction with Spring Boot, then it's managed by the Spring Boot configuration (see: Disable Sniffer in Spring Boot). false boolean sniffAfterFailureDelay (advanced) The delay of a sniff execution scheduled after a failure (in milliseconds). 60000 int snifferInterval (advanced) The interval between consecutive ordinary sniff executions in milliseconds. Will be honoured when sniffOnFailure is disabled or when there are no failures between consecutive sniff executions. 300000 int certificatePath (security) The certificate that can be used to access the ES Cluster. It can be loaded by default from classpath, but you can prefix with classpath:, file:, or http: to load the resource from different systems. String enableSSL (security) Enable SSL. false boolean 107.6. Message Headers The OpenSearch component supports 9 message header(s), listed below: Name Description Default Type operation (producer) Constant: PARAM_OPERATION The operation to perform. Enum values: Index Update Bulk GetById MultiGet MultiSearch Delete DeleteIndex Search Exists Ping OpensearchOperation indexId (producer) Constant: PARAM_INDEX_ID The id of the indexed document. String indexName (producer) Constant: PARAM_INDEX_NAME The name of the index to act against. String documentClass (producer) Constant: PARAM_DOCUMENT_CLASS The full qualified name of the class of the document to unmarshall. ObjectNode Class waitForActiveShards (producer) Constant: PARAM_WAIT_FOR_ACTIVE_SHARDS The index creation waits for the write consistency number of shards to be available. Integer scrollKeepAliveMs (producer) Constant: PARAM_SCROLL_KEEP_ALIVE_MS The starting index of the response. Integer useScroll (producer) Constant: PARAM_SCROLL Set to true to enable scroll usage. Boolean size (producer) Constant: PARAM_SIZE The size of the response. Integer from (producer) Constant: PARAM_FROM The starting index of the response. Integer 107.7. Usage 107.7.1. Message Operations The following OpenSearch operations are currently supported. Set an endpoint URI option or exchange header with a key of "operation" and a value set to one of the following. Some operations also require other parameters or the message body to be set. operation message body description Index Map , String , byte[] , Reader , InputStream or IndexRequest.Builder content to index Adds content to an index and returns the content's indexId in the body. You can set the name of the target index by setting the message header with the key "indexName". You can set the indexId by setting the message header with the key "indexId". GetById String or GetRequest.Builder index id of content to retrieve Retrieves the document corresponding to the given index id and returns a GetResponse object in the body. You can set the name of the target index by setting the message header with the key "indexName". You can set the type of document by setting the message header with the key "documentClass". Delete String or DeleteRequest.Builder index id of content to delete Deletes the specified indexName and returns a Result object in the body. You can set the name of the target index by setting the message header with the key "indexName". DeleteIndex String or DeleteIndexRequest.Builder index name of the index to delete Deletes the specified indexName and returns a status code in the body. You can set the name of the target index by setting the message header with the key "indexName". Bulk Iterable or BulkRequest.Builder of any type that is already accepted (DeleteOperation.Builder for delete operation, UpdateOperation.Builder for update operation, CreateOperation.Builder for create operation, byte[], InputStream, String, Reader, Map or any document type for index operation) Adds/Updates/Deletes content from/to an index and returns a List<BulkResponseItem> object in the body You can set the name of the target index by setting the message header with the key "indexName". Search Map , String or SearchRequest.Builder Search the content with the map of query string. You can set the name of the target index by setting the message header with the key "indexName". You can set the number of hits to return by setting the message header with the key "size". You can set the starting document offset by setting the message header with the key "from". MultiSearch MsearchRequest.Builder Multiple search in one MultiGet Iterable<String> or MgetRequest.Builder the id of the document to retrieve Multiple get in one You can set the name of the target index by setting the message header with the key "indexName". Exists None Checks whether the index exists or not and returns a Boolean flag in the body. You must set the name of the target index by setting the message header with the key "indexName". Update byte[] , InputStream , String , Reader , Map or any document type content to update Updates content to an index and returns the content's indexId in the body. You can set the name of the target index by setting the message header with the key "indexName". You can set the indexId by setting the message header with the key "indexId". Ping 107.7.2. Configure the component and enable basic authentication To use the OpenSearch component, it has to be configured with a minimum configuration. OpensearchComponent opensearchComponent = new OpensearchComponent(); opensearchComponent.setHostAddresses("opensearch-host:9200"); camelContext.addComponent("opensearch", opensearchComponent); For basic authentication with OpenSearch or using reverse http proxy in front of the OpenSearch cluster, simply setup basic authentication and SSL on the component like the example below OpenSearchComponent opensearchComponent = new OpenSearchComponent(); opensearchComponent.setHostAddresses("opensearch-host:9200"); opensearchComponent.setUser("opensearchuser"); opensearchComponent.setPassword("secure!!"); camelContext.addComponent("opensearch", opensearchComponent); 107.7.3. Document type For all the search operations, it is possible to indicate the type of document to retrieve to get the result already unmarshalled with the expected type. The document type can be set using the header "documentClass" or via the uri parameter of the same name. 107.8. Examples 107.8.1. Index Example Below is a simple INDEX example from("direct:index") .to("opensearch://opensearch?operation=Index&indexName=twitter"); <route> <from uri="direct:index"/> <to uri="opensearch://opensearch?operation=Index&amp;indexName=twitter"/> </route> Note For this operation, you'll need to specify an indexId header. A client only needs to pass a body message containing a Map to the route. The result body contains the indexId created. Map<String, String> map = new HashMap<String, String>(); map.put("content", "test"); String indexId = template.requestBody("direct:index", map, String.class); 107.8.2. Search Example Searching on specific field(s) and value use the Operation \u0301Search \u0301. Pass in the query JSON String or the Map from("direct:search") .to("opensearch://opensearch?operation=Search&indexName=twitter"); <route> <from uri="direct:search"/> <to uri="opensearch://opensearch?operation=Search&amp;indexName=twitter"/> </route> String query = "{\"query\":{\"match\":{\"doc.content\":\"new release of ApacheCamel\"}}}"; HitsMetadata<?> response = template.requestBody("direct:search", query, HitsMetadata.class); Search on specific field(s) using Map. Map<String, Object> actualQuery = new HashMap<>(); actualQuery.put("doc.content", "new release of ApacheCamel"); Map<String, Object> match = new HashMap<>(); match.put("match", actualQuery); Map<String, Object> query = new HashMap<>(); query.put("query", match); HitsMetadata<?> response = template.requestBody("direct:search", query, HitsMetadata.class); Search using OpenSearch scroll api to fetch all results. from("direct:search") .to("opensearch://opensearch?operation=Search&indexName=twitter&useScroll=true&scrollKeepAliveMs=30000"); <route> <from uri="direct:search"/> <to uri="opensearch://opensearch?operation=Search&amp;indexName=twitter&amp;useScroll=true&amp;scrollKeepAliveMs=30000"/> </route> String query = "{\"query\":{\"match\":{\"doc.content\":\"new release of ApacheCamel\"}}}"; try (OpenSearchScrollRequestIterator response = template.requestBody("direct:search", query, OpenSearchScrollRequestIterator.class)) { // do something smart with results } Split EIP can also be used. from("direct:search") .to("opensearch://opensearch?operation=Search&indexName=twitter&useScroll=true&scrollKeepAliveMs=30000") .split() .body() .streaming() .to("mock:output") .end(); 107.8.3. MultiSearch Example MultiSearching on specific field(s) and value uses the Operation MultiSearch . Pass in the MultiSearchRequest instance from("direct:multiSearch") .to("opensearch://opensearch?operation=MultiSearch"); <route> <from uri="direct:multiSearch"/> <to uri="opensearch://opensearch?operation=MultiSearch"/> </route> MultiSearch on specific field(s) MsearchRequest.Builder builder = new MsearchRequest.Builder().index("twitter").searches( new RequestItem.Builder().header(new MultisearchHeader.Builder().build()) .body(new MultisearchBody.Builder().query(b -> b.matchAll(x -> x)).build()).build(), new RequestItem.Builder().header(new MultisearchHeader.Builder().build()) .body(new MultisearchBody.Builder().query(b -> b.matchAll(x -> x)).build()).build()); List<MultiSearchResponseItem<?>> response = template.requestBody("direct:multiSearch", builder, List.class);

Commands:
<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-opensearch-starter</artifactId> </dependency>
opensearch://clusterName[?options]
opensearch:clusterName
OpensearchComponent opensearchComponent = new OpensearchComponent(); opensearchComponent.setHostAddresses("opensearch-host:9200"); camelContext.addComponent("opensearch", opensearchComponent);
OpenSearchComponent opensearchComponent = new OpenSearchComponent(); opensearchComponent.setHostAddresses("opensearch-host:9200"); opensearchComponent.setUser("opensearchuser"); opensearchComponent.setPassword("secure!!"); camelContext.addComponent("opensearch", opensearchComponent);
from("direct:index") .to("opensearch://opensearch?operation=Index&indexName=twitter");
<route> <from uri="direct:index"/> <to uri="opensearch://opensearch?operation=Index&amp;indexName=twitter"/> </route>
Map<String, String> map = new HashMap<String, String>(); map.put("content", "test"); String indexId = template.requestBody("direct:index", map, String.class);
from("direct:search") .to("opensearch://opensearch?operation=Search&indexName=twitter");
<route> <from uri="direct:search"/> <to uri="opensearch://opensearch?operation=Search&amp;indexName=twitter"/> </route>
String query = "{\"query\":{\"match\":{\"doc.content\":\"new release of ApacheCamel\"}}}"; HitsMetadata<?> response = template.requestBody("direct:search", query, HitsMetadata.class);
Map<String, Object> actualQuery = new HashMap<>(); actualQuery.put("doc.content", "new release of ApacheCamel"); Map<String, Object> match = new HashMap<>(); match.put("match", actualQuery); Map<String, Object> query = new HashMap<>(); query.put("query", match); HitsMetadata<?> response = template.requestBody("direct:search", query, HitsMetadata.class);
from("direct:search") .to("opensearch://opensearch?operation=Search&indexName=twitter&useScroll=true&scrollKeepAliveMs=30000");
<route> <from uri="direct:search"/> <to uri="opensearch://opensearch?operation=Search&amp;indexName=twitter&amp;useScroll=true&amp;scrollKeepAliveMs=30000"/> </route>
String query = "{\"query\":{\"match\":{\"doc.content\":\"new release of ApacheCamel\"}}}"; try (OpenSearchScrollRequestIterator response = template.requestBody("direct:search", query, OpenSearchScrollRequestIterator.class)) { // do something smart with results }
from("direct:search") .to("opensearch://opensearch?operation=Search&indexName=twitter&useScroll=true&scrollKeepAliveMs=30000") .split() .body() .streaming() .to("mock:output") .end();
from("direct:multiSearch") .to("opensearch://opensearch?operation=MultiSearch");
<route> <from uri="direct:multiSearch"/> <to uri="opensearch://opensearch?operation=MultiSearch"/> </route>
MsearchRequest.Builder builder = new MsearchRequest.Builder().index("twitter").searches( new RequestItem.Builder().header(new MultisearchHeader.Builder().build()) .body(new MultisearchBody.Builder().query(b -> b.matchAll(x -> x)).build()).build(), new RequestItem.Builder().header(new MultisearchHeader.Builder().build()) .body(new MultisearchBody.Builder().query(b -> b.matchAll(x -> x)).build()).build()); List<MultiSearchResponseItem<?>> response = template.requestBody("direct:multiSearch", builder, List.class);