TypeError: __init__() missing 1 required positional argument: 'scheme'
Author: amirj
State: closed
Labels: name: bug

## Describe the bug
Based on [this tutorial](https://huggingface.co/docs/datasets/faiss_es#elasticsearch) the provided code should add Elasticsearch index but raised the following error, probably the new Elasticsearch version is not compatible though the tutorial doesn't provide any information about the supporting Elasticsearch version.

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
from datasets import load_dataset
squad = load_dataset('squad', split='validation')
squad.add_elasticsearch_index("context", host="localhost", port="9200")
```

## Expected results
[Creating an elastic index based on the provided tutorial](https://huggingface.co/docs/datasets/faiss_es#elasticsearch)

## Actual results
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-8fb51aa33961> in <module>
      1 from datasets import load_dataset
      2 squad = load_dataset('squad', split='validation')
----> 3 squad.add_elasticsearch_index("context", host="localhost", port="9200")

~/opt/anaconda3/lib/python3.8/site-packages/datasets/arrow_dataset.py in add_elasticsearch_index(self, column, index_name, host, port, es_client, es_index_name, es_index_config)
   3777         """
   3778         with self.formatted_as(type=None, columns=[column]):
-> 3779             super().add_elasticsearch_index(
   3780                 column=column,
   3781                 index_name=index_name,

~/opt/anaconda3/lib/python3.8/site-packages/datasets/search.py in add_elasticsearch_index(self, column, index_name, host, port, es_client, es_index_name, es_index_config)
    587         """
    588         index_name = index_name if index_name is not None else column
--> 589         es_index = ElasticSearchIndex(
    590             host=host, port=port, es_client=es_client, es_index_name=es_index_name, es_index_config=es_index_config
    591         )

~/opt/anaconda3/lib/python3.8/site-packages/datasets/search.py in __init__(self, host, port, es_client, es_index_name, es_index_config)
    123         from elasticsearch import Elasticsearch  # noqa: F811
    124 
--> 125         self.es_client = es_client if es_client is not None else Elasticsearch([{"host": host, "port": str(port)}])
    126         self.es_index_name = (
    127             es_index_name

~/opt/anaconda3/lib/python3.8/site-packages/elasticsearch/_sync/client/__init__.py in __init__(self, hosts, cloud_id, api_key, basic_auth, bearer_auth, opaque_id, headers, connections_per_node, http_compress, verify_certs, ca_certs, client_cert, client_key, ssl_assert_hostname, ssl_assert_fingerprint, ssl_version, ssl_context, ssl_show_warn, transport_class, request_timeout, node_class, node_pool_class, randomize_nodes_in_pool, node_selector_class, dead_node_backoff_factor, max_dead_node_backoff, serializer, serializers, default_mimetype, max_retries, retry_on_status, retry_on_timeout, sniff_on_start, sniff_before_requests, sniff_on_node_failure, sniff_timeout, min_delay_between_sniffing, sniffed_node_callback, meta_header, timeout, randomize_hosts, host_info_callback, sniffer_timeout, sniff_on_connection_fail, http_auth, maxsize, _transport)
    310 
    311         if _transport is None:
--> 312             node_configs = client_node_configs(
    313                 hosts,
    314                 cloud_id=cloud_id,

~/opt/anaconda3/lib/python3.8/site-packages/elasticsearch/_sync/client/utils.py in client_node_configs(hosts, cloud_id, **kwargs)
     99     else:
    100         assert hosts is not None
--> 101         node_configs = hosts_to_node_configs(hosts)
    102 
    103     # Remove all values which are 'DEFAULT' to avoid overwriting actual defaults.

~/opt/anaconda3/lib/python3.8/site-packages/elasticsearch/_sync/client/utils.py in hosts_to_node_configs(hosts)
    142 
    143         elif isinstance(host, Mapping):
--> 144             node_configs.append(host_mapping_to_node_config(host))
    145         else:
    146             raise ValueError(

~/opt/anaconda3/lib/python3.8/site-packages/elasticsearch/_sync/client/utils.py in host_mapping_to_node_config(host)
    209         options["path_prefix"] = options.pop("url_prefix")
    210 
--> 211     return NodeConfig(**options)  # type: ignore
    212 
    213 

TypeError: __init__() missing 1 required positional argument: 'scheme'
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.0.0
- Platform: Mac
- Python version: 3.8.0
- PyArrow version: 7.0.0
- ElaticSearch Info: 
{
  "name" : "byname",
  "cluster_name" : "elasticsearch_brew",
  "cluster_uuid" : "9xkjrltiQIG0J95ciWhqRA",
  "version" : {
    "number" : "7.10.2-SNAPSHOT",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "unknown",
    "build_date" : "2021-01-16T01:41:27.115673Z",
    "build_snapshot" : true,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}


Comments: Hi @amirj, thanks for reporting.

At first sight, your issue seems a version incompatibility between your Elasticsearch client and your Elasticsearch server.

Feel free to have a look at Elasticsearch client docs: https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/overview.html#_compatibility
> Language clients are forward compatible; meaning that clients support communicating with greater or equal minor versions of Elasticsearch. Elasticsearch language clients are only backwards compatible with default distributions and without guarantees made.

@albertvillanova It doesn't seem a version incompatibility between the client and server, since the following code is working:

```
from elasticsearch import Elasticsearch
es_client = Elasticsearch("http://localhost:9200")
dataset.add_elasticsearch_index(column="e1", es_client=es_client, es_index_name="e1_index")
```

Hi @amirj, 

I really think it is a version incompatibility issue between your Elasticsearch client and server:
- Your Elasticsearch server NodeConfig expects a positional argument named 'scheme'
- Whereas your Elasticsearch client passes only keyword arguments: `NodeConfig(**options)`

Moreover:
- Looking at your stack trace, I deduce you are using Elasticsearch client **"8"** major version:
  - the Elasticsearch file "elasticsearch/_sync/client/utils.py" was created in version "8.0.0a1": https://github.com/elastic/elasticsearch-py/commit/21fa13b0f03b7b27ace9e19a1f763d40bd2e2ba4
  - you can check your Elasticsearch client version by running this Python code:
    ```python
    import elasticsearch
    print(elasticsearch.__version__)
    ```

- However, in the *Environment info*, you informed that the major version of your Eleasticsearch cluster server is **"7"** ("7.10.2-SNAPSHOT")

Could you please align the Elasticsearch client/server major versions (as pointed out in Elasticsearch docs) and check if the problem persists?

I'm closing this issue, @amirj.

Feel free to re-open it if the problem persists. 



```
from elasticsearch import Elasticsearch
es = Elasticsearch([{'host': 'localhost', 'port': 9200}])
```
```
TypeError                                 Traceback (most recent call last)
<ipython-input-8-675c6ffe5293> in <module>
      1 #es = Elasticsearch([{'host':'localhost', 'port':9200}])
      2 from elasticsearch import Elasticsearch
----> 3 es = Elasticsearch([{'host': 'localhost', 'port': 9200}])

C:\ProgramData\Anaconda3\lib\site-packages\elasticsearch\_sync\client\__init__.py in __init__(self, hosts, cloud_id, api_key, basic_auth, bearer_auth, opaque_id, headers, connections_per_node, http_compress, verify_certs, ca_certs, client_cert, client_key, ssl_assert_hostname, ssl_assert_fingerprint, ssl_version, ssl_context, ssl_show_warn, transport_class, request_timeout, node_class, node_pool_class, randomize_nodes_in_pool, node_selector_class, dead_node_backoff_factor, max_dead_node_backoff, serializer, serializers, default_mimetype, max_retries, retry_on_status, retry_on_timeout, sniff_on_start, sniff_before_requests, sniff_on_node_failure, sniff_timeout, min_delay_between_sniffing, sniffed_node_callback, meta_header, timeout, randomize_hosts, host_info_callback, sniffer_timeout, sniff_on_connection_fail, http_auth, maxsize, _transport)
    310 
    311         if _transport is None:
--> 312             node_configs = client_node_configs(
    313                 hosts,
    314                 cloud_id=cloud_id,

C:\ProgramData\Anaconda3\lib\site-packages\elasticsearch\_sync\client\utils.py in client_node_configs(hosts, cloud_id, **kwargs)
     99     else:
    100         assert hosts is not None
--> 101         node_configs = hosts_to_node_configs(hosts)
    102 
    103     # Remove all values which are 'DEFAULT' to avoid overwriting actual defaults.

C:\ProgramData\Anaconda3\lib\site-packages\elasticsearch\_sync\client\utils.py in hosts_to_node_configs(hosts)
    142 
    143         elif isinstance(host, Mapping):
--> 144             node_configs.append(host_mapping_to_node_config(host))
    145         else:
    146             raise ValueError(

C:\ProgramData\Anaconda3\lib\site-packages\elasticsearch\_sync\client\utils.py in host_mapping_to_node_config(host)
    209         options["path_prefix"] = options.pop("url_prefix")
    210 
--> 211     return NodeConfig(**options)  # type: ignore
    212 
    213 

TypeError: __init__() missing 1 required positional argument: 'scheme'
```

I am facing the same issue, and version is same for the both i.e(8.1.3)

@raj713335, thanks for reporting.

Please note that in your code example, you are not using our `datasets` library. 

Thus, I think you should report that issue to `elasticsearch` library: https://github.com/elastic/elasticsearch-py



it is simple hack which shock you just replace    https to http in scheme

**In My Case:** ->

`es = Elasticsearch([{'host': 'localhost', 'port': 9200, "scheme": "http"}])
    if es.ping():
        print('Connected to ES!')
    else:
        print('Could not connect!')
        sys.exit()`