[Elasticsearch] The Server Is Not A Supported Distribution Of Elasticsearch
By: Tony Li Xu
Tags: Elasticsearch, DevOps, Opensearch, Python3

Python Elasticsearch v7.14.0 Error

If you recently upgraded your Python elasticsearch client version to 7.14.0 , you may find yourself running into the following error when trying to connect to your existing AWS Elasticsearch Service (My version is v7.9 )

20210824 08:06:40.249330 [MainThread:INFO] run_app: es_log_level=WARNING

20210824 08:06:40.884733 [MainThread:CRITICAL] __init__: Uncaught exception

Traceback (most recent call last):

File "/home/xxxx/src/run.py", line 26, in <module>

root()

File "/home/xxxx/venv/lib/python3.7/site-packages/click/core.py",

...

File "/home/xxxx/venv/lib/python3.7/site-packages/elasticsearch/transport.py", line 413, in perform_request

_ProductChecker.raise_error(self._verified_elasticsearch)

File "/xxxx/venv/lib/python3.7/site-packages/elasticsearch/transport.py", line 630, in raise_error

raise UnsupportedProductError(message)

elasticsearch.exceptions.UnsupportedProductError: The client noticed that the server is not a supported distribution of Elasticsearch

Error highlight:

“ The client noticed that the server is not a supported distribution of Elasticsearch"

What Happened?

Elastic has modified the official Python client (Almost all its clients) for its Elasticsearch database not to work with forked versions and closed the GitHub issue to comments. Their office comment for Python clients is

https://github.com/elastic/elasticsearch-py/issues/1667

The Elasticsearch clients go through rigorous testing and validation against the official Elasticsearch distribution coming from Elastic. Since the AWS Elasticsearch Service has incompatible APIs with Elasticsearch itself, either by missing APIs or has incompatible format, we do not support it. Please consider using Elastic Cloud for an official and formally supported hosted Elasticsearch distribution.

Elasticsearch was an open-source product, but in January changed its licensing in response to what the company saw as unfair usage by public cloud platforms, AWS in particular. AWS forked Elasticsearch, before the license change, to create the Open Distro for Elasticsearch, which later evolved into OpenSearch, which had its 1.0 release last month. Casualties of this corporate battle include open-source developers who contributed to Elasticsearch but now feel let down.

So when you upgrade your Elasticsearch client version to v7.14.0 , you picked up on this quiet change to the source code, which checks that the database is Elastic’s product and not a fork. If there is no X-Elastic-Product HTTP header in the response or if the X-Elastic-Product HTTP header value isn’t Elasticsearch raise an error , is one of the changes.

How to Resolve This Issue?

Elasticsearch official answer/solution is:

Please consider using Elastic Cloud for an official and formally supported hosted Elasticsearch distribution.

I don’t have much confidence in it, they don’t really have to make this move, and because of this move, how much trust should the community put into their product going forward??

Anyway, I have the following two solutions for you, if you run into this issue like me:

Short term solution: You don’t have other choices but revert back your Elasticsearch client version to <v7.14.0 , unfortunately. So your current production environment can still work.

Long-term solution: Have a strategic conversation with your manager and dev team, make a decision on what Elasticsearch product you should use, basically Elasitcsearch vs OpenSearch (AWS). Then make medium/long-term plans for migration if you do decide to go with Elasticsearch. But I do believe that great open source software is built together, with a diverse community of contributors. So my personal preference is to stick with AWS OpenSearch and wait for their release of the OpenSearch clients. They already forked the main client branch and developing new clients. The new clients will offer the same APIs and functionality they use today.

https://aws.amazon.com/blogs/opensource/keeping-clients-of-opensearch-and-elasticsearch-compatible-with-open-source/