Repository: erikbern/ann-benchmarks
Issue: Runtime errors for some algorithms on glove-25-angular dataset
Root cause: Environment Incompatibility

Hello, I've followed these [instructions ](https://github.com/erikbern/ann-benchmarks#running). I wanted to quickly evaluate the algorithms on a single dataset via ```python run.py --dataset glove-25-angular```. However, I get the following errors for ```Elasticsearch```,  ```opensearchknn```, ```FaissIVFPQfs``` and ```vald```.  Is this expected? Do certain algorithms 'fail' for certain datasets?
Other algorithms seem to run fine (still running). 

I have a Intel(R) Xeon(R) CPU E5-1607 v2 @ 3.00GHz  workstation with 128 GB RAM. 

> Trying to instantiate ann_benchmarks.algorithms.elasticsearch.ElasticsearchScriptScoreQuery(['angular', 25])
Waiting for elasticsearch health endpoint...
Elasticsearch is ready
got a train set of size (1183514 * 25)
got 10000 queries
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 281, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

> ←[31m['angular', 25, {'M': 8, 'efConstruction': 500}]
Trying to instantiate ann_benchmarks.algorithms.opensearchknn.OpenSearchKNN(['angular', 25, {'M': 8, 'efConstruction': 500}])
Waiting for elasticsearch health endpoint...
Elasticsearch is ready
got a train set of size (1183514 * 25)
got 10000 queries
Uploading data to the Index: os-m-8-efconstruction-500
100%|##########| 1183514/1183514 [03:48<00:00, 5183.86it/s]
Force Merge...
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 281, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

> Trying to instantiate ann_benchmarks.algorithms.faiss.FaissIVFPQfs(['angular', 4096])
got a train set of size (1183514 * 25)
got 10000 queries
Traceback (most recent call last):
  File "run_algorithm.py", line 3, in <module>
    run_from_cmdline()
  File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
    run(definition, args.dataset, args.count, args.runs, args.batch)
  File "/home/app/ann_benchmarks/runner.py", line 122, in run
    algo.fit(X_train)
  File "/home/app/ann_benchmarks/algorithms/faiss.py", line 99, in fit
    index = faiss.index_factory(d, factory_string, faiss_metric)
  File "/root/anaconda3/lib/python3.8/site-packages/faiss/swigfaiss.py", line 5636, in index_factory
    return _swigfaiss.index_factory(*args)
RuntimeError: Error in void faiss::ProductQuantizer::set_derived_values() at /opt/conda/conda-bld/faiss-pkg_1623024438023/work/faiss/impl/ProductQuantizer.cpp:186: Error: 'd % M == 0' failed: The dimension of the vector (d) should be a multiple of the number of subquantizers (M)
←[0m

> ←[31m['angular', 'Float', {'edge': 20, 'searchedge': 60, 'bulk': 100}]
Trying to instantiate ann_benchmarks.algorithms.vald.Vald(['angular', 'Float', {'edge': 20, 'searchedge': 60, 'bulk': 100}])
got a train set of size (1183514 * 25)
got 10000 queries
←[32m2022-06-01 02:17:32        [INFO]: maxprocs: Leaving GOMAXPROCS=1: CPU quota undefined←[39m
←[33m2022-06-01 02:17:32        [WARN]: failed to setup option :        github.com/vdaas/vald/pkg/agent/core/ngt/service/vqueue.WithInsertBufferPoolSize.func1: invalid option, name: insertBufferPoolSize, val: 0←[39m
←[33m2022-06-01 02:17:32        [WARN]: failed to setup option :        github.com/vdaas/vald/pkg/agent/core/ngt/service/vqueue.WithDeleteBufferPoolSize.func1: invalid option, name: deleteBufferPoolSize, val: 0←[39m
←[33m2022-06-01 02:17:32        [WARN]: failed to setup option :        github.com/vdaas/vald/pkg/agent/core/ngt/handler/grpc.WithStreamConcurrency.func1: invalid option, name: streamConcurrency, val: 0←[39m
←[32m2022-06-01 02:17:32        [INFO]: service agent ngt(version: v0.0.0)starting...←[39m
←[32m2022-06-01 02:17:32        [INFO]: executing daemon pre-start function←[39m
←[32m2022-06-01 02:17:32        [INFO]: executing daemon start function←[39m
←[32m2022-06-01 02:17:32        [INFO]: server agent-grpc executing preStartFunc←[39m
←[32m2022-06-01 02:17:32        [INFO]: gRPC server agent-grpc starting on unix:///var/run/vald.sock←[39m
←[32m2022-06-01 02:17:32        [INFO]: REST server readiness starting on tcp://127.0.0.1:3001←[39m
←[32m2022-06-01 02:17:32        [INFO]: create index operation started, uncommitted indexes = 100←[39m
SIGILL: illegal instruction
PC=0xedb8ac m=7 sigcode=2
instruction bytes: 0xc4 0xe3 0x7d 0x38 0xc1 0x1 0xc5 0xfe 0x7f 0x43 0x30 0xc5 0xf8 0x77 0x48 0x8d


For faiss, the dimensionality has divisible by 4. The other three problems are probably implementation specific:

@alexklibisz Can you help with the elasticsearch and opensearchknn?
@kmrmt Is this expected to happen for vald on a 25-dimensional dataset?

I can't make much out of these logs. Without more info, I guess I can only say that you should try running them individually (i.e., not all algos at once). If they pass then it's probably some sort of resource constraint that occurs when you run multiple at once. If they fail, then at least the logs will hopefully be more understandable.

A bunch of algos will fail on every dataset unfortunately – I'd say 80-90%+ will succeed though.

Like @maumueller said, in the case of Glove it might be because the dimensionality is not divisible by 4

Hi @maumueller, @alexklibisz and @erikbern  please find the terminal output. This is again with ```python run.py --dataset glove-25-angular```. I'll update the thread with individual runs. 

> (ann-env) PS ann-benchmarks> python run.py --dataset glove-25-angular
> 2022-05-31 16:05:56,364 - annb - INFO - not all docker images available, only: {'ann-benchmarks-elasticsearch', 'ann-benchmarks-sklearn', 'ann-benchmarks-ngt', 'ann-benchmarks-mih', 'ann-benchmarks-rpforest', 'ann-benchmarks-kgraph', 'ann-benchmarks-sptag', 'ann-benchmarks-vald', 'ann-benchmarks-nmslib', 'ann-benchmarks-nearpy', 'ann-benchmarks-dolphinn', 'ann-benchmarks-mrpt', 'ann-benchmarks-n2', 'docker/getting-started', 'ann-benchmarks-hnswlib', 'ann-benchmarks-faiss', 'ann-benchmarks-datasketch', 'ann-benchmarks-elastiknn', 'ann-benchmarks-flann', 'ann-benchmarks-scipy', 'ann-benchmarks-vespa', 'ann-benchmarks-annoy', 'ann-benchmarks-opensearchknn', 'ann-benchmarks', 'ann-benchmarks-pynndescent', 'ann-benchmarks-milvus'}
> 2022-05-31 16:05:56,364 - annb - INFO - missing docker images: {'ann-benchmarks-diskann', 'ann-benchmarks-scann', 'ann-benchmarks-puffinn', 'ann-benchmarks-vearch', 'ann-benchmarks-diskann_pq'}
> 2022-05-31 16:05:56,365 - annb - INFO - Not running disabled algorithms [Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 200], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 10, 20], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 10], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 14, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 2048], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 512], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[50], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 10, 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 1], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 1000], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[1000], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 15], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 1], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 40], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[10], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 20], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 1], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 20], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 1024], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 100], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 10], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 14, 5], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 128], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 100], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 10], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 25], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 20], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[2000], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 200], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 30], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 400], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 200], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 100], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 10, 5], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 4096], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 12, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 12, 5], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[200], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 10, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 1], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 14, 20], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 64], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 14, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 5], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 100], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 12, 20], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 256], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 200], query_argument_groups=[], disabled=True), Definition(algorithm='dummy-algo-st', constructor='DummyAlgoSt', module='ann_benchmarks.algorithms.dummy_algo', docker_tag='ann-benchmarks-sklearn', arguments=['angular'], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 32], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 100], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 1], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 200], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 12, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 5], query_argument_groups=[], disabled=True), Definition(algorithm='bruteforce', constructor='BruteForce', module='ann_benchmarks.algorithms.bruteforce', docker_tag='ann-benchmarks-sklearn', arguments=['angular'], query_argument_groups=[], disabled=True), Definition(algorithm='elastiknn-exact', constructor='Exact', module='ann_benchmarks.algorithms.elastiknn', docker_tag='ann-benchmarks-elastiknn', arguments=['angular', 25], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 100], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 200], query_argument_groups=[], disabled=True), Definition(algorithm='dummy-algo-mt', constructor='DummyAlgoMt', module='ann_benchmarks.algorithms.dummy_algo', docker_tag='ann-benchmarks-sklearn', arguments=['angular'], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[100], query_argument_groups=[], disabled=True)]
> 2022-05-31 16:05:56,369 - annb - INFO - Order: [Definition(algorithm='ckdtree', constructor='CKDTree', module='ann_benchmarks.algorithms.ckdtree', docker_tag='ann-benchmarks-scipy', arguments=['angular', 20], query_argument_groups=[], disabled=False), Definition(algorithm='elasticsearch', constructor='ElasticsearchScriptScoreQuery', module='ann_benchmarks.algorithms.elasticsearch', docker_tag='ann-benchmarks-elasticsearch', arguments=['angular', 25], query_argument_groups=[], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 40, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 1.5, 'leaf_size': 25}], query_argument_groups=[[0.0], [0.02], [0.04], [0.06], [0.08], [0.1], [0.12], [0.14], [0.16]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.97}, False], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.5}, False], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.85}, False], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.8}, False], query_argument_groups=[], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 512], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 4, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='opensearchknn', constructor='OpenSearchKNN', module='ann_benchmarks.algorithms.opensearchknn', docker_tag='ann-benchmarks-opensearchknn', arguments=['angular', 25, {'M': 8, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='faiss-ivfpqfs', constructor='FaissIVFPQfs', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 4096], query_argument_groups=[[1, 0], [1, 10], [1, 100], [1, 1000], [5, 0], [5, 10], [5, 100], [5, 1000], [10, 0], [10, 10], [10, 100], [10, 1000], [50, 0], [50, 10], [50, 100], [50, 1000], [100, 0], [100, 10], [100, 100], [100, 1000], [200, 0], [200, 10], [200, 100], [200, 1000]], disabled=False), Definition(algorithm='vald(NGT-panng)', constructor='Vald', module='ann_benchmarks.algorithms.vald', docker_tag='ann-benchmarks-vald', arguments=['angular', 'Float', {'edge': 20, 'searchedge': 60, 'bulk': 100}], query_argument_groups=[[0.6], [0.8], [0.9], [1.0], [1.02], [1.05], [1.1], [1.2]], disabled=False), Definition(algorithm='annoy', constructor='Annoy', module='ann_benchmarks.algorithms.annoy', docker_tag='ann-benchmarks-annoy', arguments=['angular', 100], query_argument_groups=[[100], [200], [400], [1000], [2000], [4000], [10000], [20000], [40000], [100000], [200000], [400000]], disabled=False), Definition(algorithm='ckdtree', constructor='CKDTree', module='ann_benchmarks.algorithms.ckdtree', docker_tag='ann-benchmarks-scipy', arguments=['angular', 40], query_argument_groups=[], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 4096], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 80, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 2.0, 'leaf_size': 20}], query_argument_groups=[[0.0], [0.02], [0.04], [0.08], [0.12], [0.16], [0.2], [0.24]], disabled=False), Definition(algorithm='hnsw(vespa)', constructor='VespaHnsw', module='ann_benchmarks.algorithms.vespa', docker_tag='ann-benchmarks-vespa', arguments=['angular', 25, {'M': 12, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='SW-graph(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'sw-graph', {'NN': 3}, False], query_argument_groups=[[120], [80], [60], [40], [20], [10], [8], [4], [2]], disabled=False), Definition(algorithm='hnsw(vespa)', constructor='VespaHnsw', module='ann_benchmarks.algorithms.vespa', docker_tag='ann-benchmarks-vespa', arguments=['angular', 25, {'M': 24, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 3], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 10], query_argument_groups=[], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 48, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 40], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 300], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 100], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 40], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 100], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='hnsw(vespa)', constructor='VespaHnsw', module='ann_benchmarks.algorithms.vespa', docker_tag='ann-benchmarks-vespa', arguments=['angular', 25, {'M': 8, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 16, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 1000], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 1000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 3], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.95}, False], query_argument_groups=[], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.97], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'hnsw', {'M': 48, 'post': 2, 'efConstruction': 800}, False], query_argument_groups=[[50], [70], [90], [120], [160], [200], [400], [600], [700], [800], [1000], [1400], [1600], [2000]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 80, 'diversify_prob': 0.25, 'pruning_degree_multiplier': 2.0, 'leaf_size': 30}], query_argument_groups=[[0.08], [0.12], [0.16], [0.2], [0.24], [0.28], [0.32], [0.36]], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 10000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 40], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 3000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 36, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 1024], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.2], query_argument_groups=[], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 96, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 120, 'diversify_prob': 0.125, 'pruning_degree_multiplier': 2.5, 'leaf_size': 35}], query_argument_groups=[[0.16], [0.2], [0.24], [0.28], [0.32], [0.36]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 96, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 12, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='annoy', constructor='Annoy', module='ann_benchmarks.algorithms.annoy', docker_tag='ann-benchmarks-annoy', arguments=['angular', 400], query_argument_groups=[[100], [200], [400], [1000], [2000], [4000], [10000], [20000], [40000], [100000], [200000], [400000]], disabled=False), Definition(algorithm='NGT-onng', constructor='ONNG', module='ann_benchmarks.algorithms.onng_ngt', docker_tag='ann-benchmarks-ngt', arguments=['angular', 'Float', 0.1, {'edge': 100, 'outdegree': 10, 'indegree': 120, 'tree': False, 'refine': True}], query_argument_groups=[[[0.995, 40]], [[0.998, 40]], [[1.0, 40]], [[1.005, 40]], [[1.01, 40]], [[1.015, 40]], [[1.02, 40]]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.4}, False], query_argument_groups=[], disabled=False), Definition(algorithm='NGT-panng', constructor='PANNG', module='ann_benchmarks.algorithms.panng_ngt', docker_tag='ann-benchmarks-ngt', arguments=['angular', 'Float', {'edge': 20, 'pathadj': 40, 'searchedge': 60}], query_argument_groups=[[0.6], [0.8], [0.9], [1.0], [1.02], [1.05], [1.1], [1.2]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 40, 'diversify_prob': 0.5, 'pruning_degree_multiplier': 1.5, 'leaf_size': 25}], query_argument_groups=[[0.0], [0.02], [0.04], [0.06], [0.08], [0.1], [0.12], [0.14], [0.16]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 100], query_argument_groups=[], disabled=False), Definition(algorithm='mrpt', constructor='MRPT', module='ann_benchmarks.algorithms.mrpt', docker_tag='ann-benchmarks-mrpt', arguments=['angular', 10], query_argument_groups=[[0.1], [0.2], [0.3], [0.4], [0.5], [0.6], [0.7], [0.8], [0.85], [0.9], [0.925], [0.95], [0.97], [0.98], [0.99], [0.995]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 16, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.6}, False], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.9}, False], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 12, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='opensearchknn', constructor='OpenSearchKNN', module='ann_benchmarks.algorithms.opensearchknn', docker_tag='ann-benchmarks-opensearchknn', arguments=['angular', 25, {'M': 16, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 8, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 2048], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='opensearchknn', constructor='OpenSearchKNN', module='ann_benchmarks.algorithms.opensearchknn', docker_tag='ann-benchmarks-opensearchknn', arguments=['angular', 25, {'M': 24, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='ckdtree', constructor='CKDTree', module='ann_benchmarks.algorithms.ckdtree', docker_tag='ann-benchmarks-scipy', arguments=['angular', 100], query_argument_groups=[], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.95], query_argument_groups=[], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 48, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 24, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 10], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 100], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'hnsw', {'M': 20, 'post': 0, 'efConstruction': 800}, False], query_argument_groups=[[2], [5], [10], [15], [20], [30], [40], [50], [70], [80]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 64, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 20, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 0.5, 'leaf_size': 20}], query_argument_groups=[[0.0], [0.02], [0.04], [0.06], [0.08], [0.1], [0.12]], disabled=False), Definition(algorithm='ckdtree', constructor='CKDTree', module='ann_benchmarks.algorithms.ckdtree', docker_tag='ann-benchmarks-scipy', arguments=['angular', 400], query_argument_groups=[], disabled=False), Definition(algorithm='NGT-qg', constructor='QG', module='ann_benchmarks.algorithms.qg_ngt', docker_tag='ann-benchmarks-ngt', arguments=['angular', 'Float', 0.1, {'edge': 100, 'outdegree': 64, 'indegree': 120, 'max_edge': 96}], query_argument_groups=[[[1.2, 0.9]], [[1.2, 0.95]], [[1.2, 0.98]], [[1.2, 1.0]], [[1.2, 1.02]], [[1.5, 0.9]], [[1.5, 0.95]], [[1.5, 0.98]], [[1.5, 1.0]], [[1.5, 1.02]], [[2.0, 0.9]], [[2.0, 0.95]], [[2.0, 0.98]], [[2.0, 1.0]], [[2.0, 1.02]], [[3.0, 0.9]], [[3.0, 0.95]], [[3.0, 0.98]], [[3.0, 1.0]], [[3.0, 1.02]], [[5, 1.0]], [[10, 1.0]], [[20, 1.0]], [[2, 1.04]], [[3, 1.04]], [[5, 1.04]], [[8, 1.04]]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 120, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 2.5, 'leaf_size': 20}], query_argument_groups=[[0.0], [0.04], [0.08], [0.16], [0.2], [0.24], [0.28], [0.32]], disabled=False), Definition(algorithm='annoy', constructor='Annoy', module='ann_benchmarks.algorithms.annoy', docker_tag='ann-benchmarks-annoy', arguments=['angular', 200], query_argument_groups=[[100], [200], [400], [1000], [2000], [4000], [10000], [20000], [40000], [100000], [200000], [400000]], disabled=False), Definition(algorithm='opensearchknn', constructor='OpenSearchKNN', module='ann_benchmarks.algorithms.opensearchknn', docker_tag='ann-benchmarks-opensearchknn', arguments=['angular', 25, {'M': 36, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnsw(vespa)', constructor='VespaHnsw', module='ann_benchmarks.algorithms.vespa', docker_tag='ann-benchmarks-vespa', arguments=['angular', 25, {'M': 64, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 400], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 4, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 400], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 400], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(vespa)', constructor='VespaHnsw', module='ann_benchmarks.algorithms.vespa', docker_tag='ann-benchmarks-vespa', arguments=['angular', 25, {'M': 16, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 10], query_argument_groups=[], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 96, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 30000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 10], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 10], query_argument_groups=[], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 32], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='faiss-ivfpqfs', constructor='FaissIVFPQfs', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 1204], query_argument_groups=[[1, 0], [1, 10], [1, 100], [1, 1000], [5, 0], [5, 10], [5, 100], [5, 1000], [10, 0], [10, 10], [10, 100], [10, 1000], [50, 0], [50, 10], [50, 100], [50, 1000], [100, 0], [100, 10], [100, 100], [100, 1000], [200, 0], [200, 10], [200, 100], [200, 1000]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 100], query_argument_groups=[], disabled=False), Definition(algorithm='SW-graph(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'sw-graph', {'NN': 30}, False], query_argument_groups=[[700], [650], [550], [450], [350], [275], [200], [150], [120], [80], [50], [30]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 100], query_argument_groups=[], disabled=False), Definition(algorithm='bruteforce-blas', constructor='BruteForceBLAS', module='ann_benchmarks.algorithms.bruteforce', docker_tag='ann-benchmarks-sklearn', arguments=['angular'], query_argument_groups=[], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.8], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'hnsw', {'M': 32, 'post': 2, 'efConstruction': 800}, False], query_argument_groups=[[10], [20], [30], [40], [50], [60], [70], [80], [90], [100], [120], [140], [160], [200], [300], [400], [600], [700], [800], [1000], [1200], [1400], [1600], [2000]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 64, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 20, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 1.0, 'leaf_size': 20}], query_argument_groups=[[0.0], [0.02], [0.04], [0.06], [0.08], [0.1], [0.12]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 40], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 400], query_argument_groups=[], disabled=False), Definition(algorithm='faiss-ivfpqfs', constructor='FaissIVFPQfs', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 2048], query_argument_groups=[[1, 0], [1, 10], [1, 100], [1, 1000], [5, 0], [5, 10], [5, 100], [5, 1000], [10, 0], [10, 10], [10, 100], [10, 1000], [50, 0], [50, 10], [50, 100], [50, 1000], [100, 0], [100, 10], [100, 100], [100, 1000], [200, 0], [200, 10], [200, 100], [200, 1000]], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 100], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 3], query_argument_groups=[], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 4, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 36, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.5], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 10000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.1}, False], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 1000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='SW-graph(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'sw-graph', {'NN': 15}, False], query_argument_groups=[[80], [50], [30], [20]], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 30000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.3}, False], query_argument_groups=[], disabled=False), Definition(algorithm='kgraph', constructor='KGraph', module='ann_benchmarks.algorithms.kgraph', docker_tag='ann-benchmarks-kgraph', arguments=['angular', {'reverse': -1, 'K': 200, 'L': 300, 'S': 20}, False], query_argument_groups=[[1], [2], [3], [4], [5], [10], [20], [30], [40], [50], [60], [70], [80], [90], [100]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 24, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 10], query_argument_groups=[], disabled=False), Definition(algorithm='opensearchknn', constructor='OpenSearchKNN', module='ann_benchmarks.algorithms.opensearchknn', docker_tag='ann-benchmarks-opensearchknn', arguments=['angular', 25, {'M': 4, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='ckdtree', constructor='CKDTree', module='ann_benchmarks.algorithms.ckdtree', docker_tag='ann-benchmarks-scipy', arguments=['angular', 10], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 36, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='faiss-ivfpqfs', constructor='FaissIVFPQfs', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 512], query_argument_groups=[[1, 0], [1, 10], [1, 100], [1, 1000], [5, 0], [5, 10], [5, 100], [5, 1000], [10, 0], [10, 10], [10, 100], [10, 1000], [50, 0], [50, 10], [50, 100], [50, 1000], [100, 0], [100, 10], [100, 100], [100, 1000], [200, 0], [200, 10], [200, 100], [200, 1000]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 16, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 12, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 100], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.2}, False], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(vespa)', constructor='VespaHnsw', module='ann_benchmarks.algorithms.vespa', docker_tag='ann-benchmarks-vespa', arguments=['angular', 25, {'M': 36, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 256], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 3], query_argument_groups=[], disabled=False), Definition(algorithm='sptag', constructor='Sptag', module='ann_benchmarks.algorithms.sptag', docker_tag='ann-benchmarks-sptag', arguments=['angular', 'KDT'], query_argument_groups=[[100], [200], [400], [1000], [2000], [4000]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 40], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 300], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 400], query_argument_groups=[], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 8192], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 8, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnsw(vespa)', constructor='VespaHnsw', module='ann_benchmarks.algorithms.vespa', docker_tag='ann-benchmarks-vespa', arguments=['angular', 25, {'M': 48, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 24, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='ckdtree', constructor='CKDTree', module='ann_benchmarks.algorithms.ckdtree', docker_tag='ann-benchmarks-scipy', arguments=['angular', 200], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.99}, False], query_argument_groups=[], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 8, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnsw(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'hnsw', {'M': 12, 'post': 0, 'efConstruction': 800}, False], query_argument_groups=[[1], [2], [5], [10], [15], [20], [30], [40], [50], [70], [80]], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.9], query_argument_groups=[], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 200], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(vespa)', constructor='VespaHnsw', module='ann_benchmarks.algorithms.vespa', docker_tag='ann-benchmarks-vespa', arguments=['angular', 25, {'M': 4, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.7}, False], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(vespa)', constructor='VespaHnsw', module='ann_benchmarks.algorithms.vespa', docker_tag='ann-benchmarks-vespa', arguments=['angular', 25, {'M': 96, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 64], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='sptag', constructor='Sptag', module='ann_benchmarks.algorithms.sptag', docker_tag='ann-benchmarks-sptag', arguments=['angular', 'BKT'], query_argument_groups=[[100], [200], [400], [1000], [2000], [4000]], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 3000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='opensearchknn', constructor='OpenSearchKNN', module='ann_benchmarks.algorithms.opensearchknn', docker_tag='ann-benchmarks-opensearchknn', arguments=['angular', 25, {'M': 12, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='ckdtree', constructor='CKDTree', module='ann_benchmarks.algorithms.ckdtree', docker_tag='ann-benchmarks-scipy', arguments=['angular', 1000], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 3], query_argument_groups=[], disabled=False), Definition(algorithm='opensearchknn', constructor='OpenSearchKNN', module='ann_benchmarks.algorithms.opensearchknn', docker_tag='ann-benchmarks-opensearchknn', arguments=['angular', 25, {'M': 48, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 48, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 40], query_argument_groups=[], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 64, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 20], query_argument_groups=[], disabled=False), Definition(algorithm='NGT-onng', constructor='ONNG', module='ann_benchmarks.algorithms.onng_ngt', docker_tag='ann-benchmarks-ngt', arguments=['angular', 'Float', 0.1, {'edge': 100, 'outdegree': 10, 'indegree': 120}], query_argument_groups=[[[0.6, -2]], [[0.9, -2]], [[1.0, -2]], [[1.02, -2]], [[1.03, -2]], [[1.04, -2]], [[1.05, -2]], [[1.07, -2]], [[1.1, -2]], [[1.2, -2]]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 128], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 400], query_argument_groups=[], disabled=False)]
> ←[31m * Starting Elasticsearch Server
> sysctl: setting key "vm.max_map_count": Read-only file system
>    ...done.
> ['angular', 25]
> Trying to instantiate ann_benchmarks.algorithms.elasticsearch.ElasticsearchScriptScoreQuery(['angular', 25])
> Waiting for elasticsearch health endpoint...
> Elasticsearch is ready
> got a train set of size (1183514 * 25)
> got 10000 queries
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 449, in _make_request
>     six.raise_from(e, None)
>   File "<string>", line 3, in raise_from
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 444, in _make_request
>     httplib_response = conn.getresponse()
>   File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse
>     response.begin()
>   File "/usr/lib/python3.6/http/client.py", line 320, in begin
>     version, status, reason = self._read_status()
>   File "/usr/lib/python3.6/http/client.py", line 281, in _read_status
>     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
>   File "/usr/lib/python3.6/socket.py", line 586, in readinto
>     return self._sock.recv_into(b)
> socket.timeout: timed out
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 246, in perform_request
>     method, url, body, retries=Retry(False), headers=request_headers, **kw
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 786, in urlopen
>     method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 525, in increment
>     raise six.reraise(type(error), error, _stacktrace)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 770, in reraise
>     raise value
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 710, in urlopen
>     chunked=chunked,
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 451, in _make_request
>     self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 341, in _raise_timeout
>     self, url, "Read timed out. (read timeout=%s)" % timeout_value
> urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 122, in run
>     algo.fit(X_train)
>   File "/home/app/ann_benchmarks/algorithms/elasticsearch.py", line 79, in fit
>     self.es.indices.forcemerge(self.index, max_num_segments=1)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/utils.py", line 152, in _wrapped
>     return func(*args, params=params, headers=headers, **kwargs)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/indices.py", line 1088, in forcemerge
>     "POST", _make_path(index, "_forcemerge"), params=params, headers=headers
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 392, in perform_request
>     raise e
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 365, in perform_request
>     timeout=timeout,
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 257, in perform_request
>     raise ConnectionTimeout("TIMEOUT", str(e), e)
> elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10))
> ←[0m
> Child process for container 958d376b57returned exit code 1 with message None
> ←[31m['angular', 25, {'M': 8, 'efConstruction': 500}]
> Trying to instantiate ann_benchmarks.algorithms.opensearchknn.OpenSearchKNN(['angular', 25, {'M': 8, 'efConstruction': 500}])
> Waiting for elasticsearch health endpoint...
> Elasticsearch is ready
> got a train set of size (1183514 * 25)
> got 10000 queries
> Uploading data to the Index: os-m-8-efconstruction-500
> 100%|##########| 1183514/1183514 [03:48<00:00, 5183.86it/s]
> Force Merge...
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 449, in _make_request
>     six.raise_from(e, None)
>   File "<string>", line 3, in raise_from
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 444, in _make_request
>     httplib_response = conn.getresponse()
>   File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse
>     response.begin()
>   File "/usr/lib/python3.6/http/client.py", line 320, in begin
>     version, status, reason = self._read_status()
>   File "/usr/lib/python3.6/http/client.py", line 281, in _read_status
>     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
>   File "/usr/lib/python3.6/socket.py", line 586, in readinto
>     return self._sock.recv_into(b)
> socket.timeout: timed out
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 252, in perform_request
>     method, url, body, retries=Retry(False), headers=request_headers, **kw
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 786, in urlopen
>     method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 525, in increment
>     raise six.reraise(type(error), error, _stacktrace)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 770, in reraise
>     raise value
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 710, in urlopen
>     chunked=chunked,
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 451, in _make_request
>     self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 341, in _raise_timeout
>     self, url, "Read timed out. (read timeout=%s)" % timeout_value
> urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=1000)
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 122, in run
>     algo.fit(X_train)
>   File "/home/app/ann_benchmarks/algorithms/opensearchknn.py", line 69, in fit
>     self.es.indices.forcemerge(self.name, max_num_segments=1, request_timeout=1000)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/utils.py", line 168, in _wrapped
>     return func(*args, params=params, headers=headers, **kwargs)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/indices.py", line 1125, in forcemerge
>     "POST", _make_path(index, "_forcemerge"), params=params, headers=headers
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 415, in perform_request
>     raise e
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 388, in perform_request
>     timeout=timeout,
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 265, in perform_request
>     raise ConnectionTimeout("TIMEOUT", str(e), e)
> elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=1000))
> ←[0m
> Child process for container bcbae95dfcreturned exit code 1 with message None
> ←[31m['angular', 4096]
> Trying to instantiate ann_benchmarks.algorithms.faiss.FaissIVFPQfs(['angular', 4096])
> got a train set of size (1183514 * 25)
> got 10000 queries
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 122, in run
>     algo.fit(X_train)
>   File "/home/app/ann_benchmarks/algorithms/faiss.py", line 99, in fit
>     index = faiss.index_factory(d, factory_string, faiss_metric)
>   File "/root/anaconda3/lib/python3.8/site-packages/faiss/swigfaiss.py", line 5636, in index_factory
>     return _swigfaiss.index_factory(*args)
> RuntimeError: Error in void faiss::ProductQuantizer::set_derived_values() at /opt/conda/conda-bld/faiss-pkg_1623024438023/work/faiss/impl/ProductQuantizer.cpp:186: Error: 'd % M == 0' failed: The dimension of the vector (d) should be a multiple of the number of subquantizers (M)
> ←[0m
> Child process for container 1d4dd65e97returned exit code 1 with message None
> ←[31m['angular', 'Float', {'edge': 20, 'searchedge': 60, 'bulk': 100}]
> Trying to instantiate ann_benchmarks.algorithms.vald.Vald(['angular', 'Float', {'edge': 20, 'searchedge': 60, 'bulk': 100}])
> got a train set of size (1183514 * 25)
> got 10000 queries
> ←[32m2022-06-01 02:17:32        [INFO]: maxprocs: Leaving GOMAXPROCS=1: CPU quota undefined←[39m
> ←[33m2022-06-01 02:17:32        [WARN]: failed to setup option :        github.com/vdaas/vald/pkg/agent/core/ngt/service/vqueue.WithInsertBufferPoolSize.func1: invalid option, name: insertBufferPoolSize, val: 0←[39m
> ←[33m2022-06-01 02:17:32        [WARN]: failed to setup option :        github.com/vdaas/vald/pkg/agent/core/ngt/service/vqueue.WithDeleteBufferPoolSize.func1: invalid option, name: deleteBufferPoolSize, val: 0←[39m
> ←[33m2022-06-01 02:17:32        [WARN]: failed to setup option :        github.com/vdaas/vald/pkg/agent/core/ngt/handler/grpc.WithStreamConcurrency.func1: invalid option, name: streamConcurrency, val: 0←[39m
> ←[32m2022-06-01 02:17:32        [INFO]: service agent ngt(version: v0.0.0)starting...←[39m
> ←[32m2022-06-01 02:17:32        [INFO]: executing daemon pre-start function←[39m
> ←[32m2022-06-01 02:17:32        [INFO]: executing daemon start function←[39m
> ←[32m2022-06-01 02:17:32        [INFO]: server agent-grpc executing preStartFunc←[39m
> ←[32m2022-06-01 02:17:32        [INFO]: gRPC server agent-grpc starting on unix:///var/run/vald.sock←[39m
> ←[32m2022-06-01 02:17:32        [INFO]: REST server readiness starting on tcp://127.0.0.1:3001←[39m
> ←[32m2022-06-01 02:17:32        [INFO]: create index operation started, uncommitted indexes = 100←[39m
> SIGILL: illegal instruction
> PC=0xedb8ac m=7 sigcode=2
> instruction bytes: 0xc4 0xe3 0x7d 0x38 0xc1 0x1 0xc5 0xfe 0x7f 0x43 0x30 0xc5 0xf8 0x77 0x48 0x8d
> 
> goroutine 0 [idle]:
> runtime: unknown pc 0xedb8ac
> stack: frame={sp:0x7fecc67fbaa0, fp:0x0} stack=[0x7fecc5ffc490,0x7fecc67fc090)
> 0x00007fecc67fb9a0:  0x000000000000001f  0x000000000100c5fd
> 0x00007fecc67fb9b0:  0x00007f9e8f8f01c0  0x0000000000000000
> 0x00007fecc67fb9c0:  0x00007fec00000021  0x00007fecb4000080
> 0x00007fecc67fb9d0:  0x0000000000000200  0x00007fecb40c9d00
> 0x00007fecc67fb9e0:  0x0000000000001000  0x00007fecb4000020
> 0x00007fecc67fb9f0:  0x0000000000000008  0x0000003800000021
> 0x00007fecc67fba00:  0x0000000000000001  0x0000000000000000
> 0x00007fecc67fba10:  0x0000000000000000  0x000000770000007c
> 0x00007fecc67fba20:  0x0000005c0000006e  0x00007fecb4000080
> 0x00007fecc67fba30:  0x0000000000000040  0x000000000000001f
> 0x00007fecc67fba40:  0x0000000000000200  0x00007fecb4000020
> 0x00007fecc67fba50:  0x0000000000000000  0x00007fecb4000f38
> 0x00007fecc67fba60:  0x00007fecc67fbbe0  0x000000000100e0b1
> 0x00007fecc67fba70:  0x0000000000000200  0x00007fecc67fbaf0
> 0x00007fecc67fba80:  0x00007fecc67fbbe0  0x0000000000f03659
> 0x00007fecc67fba90:  0x00007fecc67fbcc8  0x0000000000edb868
> 0x00007fecc67fbaa0: <0x0000000000000000  0x00000000b4000020
> 0x00007fecc67fbab0:  0x0000000000000000  0x00007fecb4006584
> 0x00007fecc67fbac0:  0x00007fecf3b40018  0x00007fecc67fbcc0
> 0x00007fecc67fbad0:  0x000000000000000f  0x00007fecc67fbbb4
> 0x00007fecc67fbae0:  0x000000000387b2c0  0x00007fecc67fbbe0
> 0x00007fecc67fbaf0:  0x000000000387b020  0x0000000000eb9c05
> 0x00007fecc67fbb00:  0x00007fecc67fbb30  0x0000000000000000
> 0x00007fecc67fbb10:  0x00000000013343a0  0x00007fecc67fbcc8
> 0x00007fecc67fbb20:  0x00007fecc67fbc68  0x0000000000002710
> 0x00007fecc67fbb30:  0x00007fecc67fbe30  0x000000c0002fa000
> 0x00007fecc67fbb40:  0x00000000013343a0  0x000000000387b020
> 0x00007fecc67fbb50:  0x00007fecc67fbcc0  0x0000000000ec1584
> 0x00007fecc67fbb60:  0x0000000000000000  0x00007fecc67fbc00
> 0x00007fecc67fbb70:  0x00007fecc67fbc68  0x00007fecc67fbbb4
> 0x00007fecc67fbb80:  0x00007fecc67fbbe0  0x0000000000000000
> 0x00007fecc67fbb90:  0x0000000000000000  0x00000000000186a0
> runtime: unknown pc 0xedb8ac
> stack: frame={sp:0x7fecc67fbaa0, fp:0x0} stack=[0x7fecc5ffc490,0x7fecc67fc090)
> 0x00007fecc67fb9a0:  0x000000000000001f  0x000000000100c5fd
> 0x00007fecc67fb9b0:  0x00007f9e8f8f01c0  0x0000000000000000
> 0x00007fecc67fb9c0:  0x00007fec00000021  0x00007fecb4000080
> 0x00007fecc67fb9d0:  0x0000000000000200  0x00007fecb40c9d00
> 0x00007fecc67fb9e0:  0x0000000000001000  0x00007fecb4000020
> 0x00007fecc67fb9f0:  0x0000000000000008  0x0000003800000021
> 0x00007fecc67fba00:  0x0000000000000001  0x0000000000000000
> 0x00007fecc67fba10:  0x0000000000000000  0x000000770000007c
> 0x00007fecc67fba20:  0x0000005c0000006e  0x00007fecb4000080
> 0x00007fecc67fba30:  0x0000000000000040  0x000000000000001f
> 0x00007fecc67fba40:  0x0000000000000200  0x00007fecb4000020
> 0x00007fecc67fba50:  0x0000000000000000  0x00007fecb4000f38
> 0x00007fecc67fba60:  0x00007fecc67fbbe0  0x000000000100e0b1
> 0x00007fecc67fba70:  0x0000000000000200  0x00007fecc67fbaf0
> 0x00007fecc67fba80:  0x00007fecc67fbbe0  0x0000000000f03659
> 0x00007fecc67fba90:  0x00007fecc67fbcc8  0x0000000000edb868
> 0x00007fecc67fbaa0: <0x0000000000000000  0x00000000b4000020
> 0x00007fecc67fbab0:  0x0000000000000000  0x00007fecb4006584
> 0x00007fecc67fbac0:  0x00007fecf3b40018  0x00007fecc67fbcc0
> 0x00007fecc67fbad0:  0x000000000000000f  0x00007fecc67fbbb4
> 0x00007fecc67fbae0:  0x000000000387b2c0  0x00007fecc67fbbe0
> 0x00007fecc67fbaf0:  0x000000000387b020  0x0000000000eb9c05
> 0x00007fecc67fbb00:  0x00007fecc67fbb30  0x0000000000000000
> 0x00007fecc67fbb10:  0x00000000013343a0  0x00007fecc67fbcc8
> 0x00007fecc67fbb20:  0x00007fecc67fbc68  0x0000000000002710
> 0x00007fecc67fbb30:  0x00007fecc67fbe30  0x000000c0002fa000
> 0x00007fecc67fbb40:  0x00000000013343a0  0x000000000387b020
> 0x00007fecc67fbb50:  0x00007fecc67fbcc0  0x0000000000ec1584
> 0x00007fecc67fbb60:  0x0000000000000000  0x00007fecc67fbc00
> 0x00007fecc67fbb70:  0x00007fecc67fbc68  0x00007fecc67fbbb4
> 0x00007fecc67fbb80:  0x00007fecc67fbbe0  0x0000000000000000
> 0x00007fecc67fbb90:  0x0000000000000000  0x00000000000186a0
> 
> goroutine 188 [syscall]:
> runtime.cgocall(0xe67c80, 0xc0002f9430)
>         runtime/cgocall.go:156 +0x5c fp=0xc0002f9408 sp=0xc0002f93d0 pc=0x41847c
> github.com/vdaas/vald/internal/core/algorithm/ngt._Cfunc_ngt_create_index(0x387b020, 0x2710, 0x387ab90)
>         _cgo_gotypes.go:197 +0x49 fp=0xc0002f9430 sp=0xc0002f9408 pc=0xdffea9
> github.com/vdaas/vald/internal/core/algorithm/ngt.(*ngt).CreateIndex.func1(0x12e65a3, 0xe, 0xc00045cf50)
>         github.com/vdaas/vald/internal/core/algorithm/ngt/ngt.go:503 +0x72 fp=0xc0002f9478 sp=0xc0002f9430 pc=0xe04b92
> github.com/vdaas/vald/internal/core/algorithm/ngt.(*ngt).CreateIndex(0xc00031e6c0, 0x12e65a3)
>         github.com/vdaas/vald/internal/core/algorithm/ngt/ngt.go:503 +0x59 fp=0xc0002f94b0 sp=0xc0002f9478 pc=0xe04a99
> github.com/vdaas/vald/pkg/agent/core/ngt/service.(*ngt).CreateIndex(0xc000337560, {0x152b638, 0xc0000ed950}, 0x2710)
>         github.com/vdaas/vald/pkg/agent/core/ngt/service/ngt.go:689 +0x773 fp=0xc0002f9708 sp=0xc0002f94b0 pc=0xe17293
> github.com/vdaas/vald/pkg/agent/core/ngt/handler/grpc.(*server).CreateIndex(0xc00051c550, {0x152b638, 0xc0000ed950}, 0xc0000ed980)
>         github.com/vdaas/vald/pkg/agent/core/ngt/handler/grpc/handler.go:1693 +0x135 fp=0xc0002f9b70 sp=0xc0002f9708 pc=0xe5fa35
> github.com/vdaas/vald/apis/grpc/v1/agent/core._Agent_CreateIndex_Handler({0x12c1500, 0xc00051c550}, {0x152b638, 0xc0000ed950}, 0xc0004f72c0, 0x0)
>         github.com/vdaas/vald/apis/grpc/v1/agent/core/agent_vtproto.pb.go:141 +0x170 fp=0xc0002f9bc8 sp=0xc0002f9b70 pc=0xb0f2b0
> google.golang.org/grpc.(*Server).processUnaryRPC(0xc0002cafc0, {0x153bc58, 0xc0003b2180}, 0xc000336120, 0xc00053c030, 0x1cfda60, 0x0)
>         google.golang.org/grpc@v1.40.0/server.go:1279 +0xccf fp=0xc0002f9e48 sp=0xc0002f9bc8 pc=0xa3656f
> google.golang.org/grpc.(*Server).handleStream(0xc0002cafc0, {0x153bc58, 0xc0003b2180}, 0xc000336120, 0x0)
>         google.golang.org/grpc@v1.40.0/server.go:1608 +0xa2a fp=0xc0002f9f68 sp=0xc0002f9e48 pc=0xa3a38a
> google.golang.org/grpc.(*Server).serveStreams.func1.2()
>         google.golang.org/grpc@v1.40.0/server.go:923 +0x98 fp=0xc0002f9fe0 sp=0xc0002f9f68 pc=0xa34058
> runtime.goexit()
>         runtime/asm_amd64.s:1581 +0x1 fp=0xc0002f9fe8 sp=0xc0002f9fe0 pc=0x47e001
> created by google.golang.org/grpc.(*Server).serveStreams.func1
>         google.golang.org/grpc@v1.40.0/server.go:921 +0x294
> 
> goroutine 1 [select]:
> github.com/vdaas/vald/internal/runner.Run({0x152b5c8, 0xc000040030}, {0x1532558, 0xc00051c690}, {0x12e1497, 0x9})
>         github.com/vdaas/vald/internal/runner/runner.go:160 +0x3ec
> github.com/vdaas/vald/internal/runner.Do({0x152b5c8, 0xc000040030}, {0xc0006ffc88, 0x4, 0x0})
>         github.com/vdaas/vald/internal/runner/runner.go:137 +0x81d
> main.main.func1()
>         github.com/vdaas/vald/cmd/agent/core/ngt/main.go:40 +0x16e
> github.com/vdaas/vald/internal/safety.recoverFunc.func1()
>         github.com/vdaas/vald/internal/safety/safety.go:63 +0x73
> main.main()
>         github.com/vdaas/vald/cmd/agent/core/ngt/main.go:55 +0x42
> 
> goroutine 6 [select]:
> github.com/kpango/fastime.(*Fastime).StartTimerD.func1()
>         github.com/kpango/fastime@v1.0.17/fastime.go:192 +0x127
> created by github.com/kpango/fastime.(*Fastime).StartTimerD
>         github.com/kpango/fastime@v1.0.17/fastime.go:185 +0x147
> 
> goroutine 7 [select]:
> go.opencensus.io/stats/view.(*worker).start(0xc0000e6800)
>         go.opencensus.io@v0.23.0/stats/view/worker.go:276 +0xb9
> created by go.opencensus.io/stats/view.init.0
>         go.opencensus.io@v0.23.0/stats/view/worker.go:34 +0x92
> 
> goroutine 75 [syscall]:
> os/signal.signal_recv()
>         runtime/sigqueue.go:169 +0x98
> os/signal.loop()
>         os/signal/signal_unix.go:24 +0x19
> created by os/signal.Notify.func1.1
>         os/signal/signal.go:151 +0x2c
> 
> goroutine 76 [select]:
> os/signal.NotifyContext.func1()
>         os/signal/signal.go:288 +0x76
> created by os/signal.NotifyContext
>         os/signal/signal.go:287 +0x169
> 
> goroutine 77 [select]:
> github.com/vdaas/vald/pkg/agent/core/ngt/usecase.(*run).Start.func1()
>         github.com/vdaas/vald/pkg/agent/core/ngt/usecase/agentd.go:170 +0x259
> github.com/vdaas/vald/internal/safety.recoverFunc.func1()
>         github.com/vdaas/vald/internal/safety/safety.go:63 +0x73
> github.com/vdaas/vald/internal/errgroup.(*group).Go.func1()
>         github.com/vdaas/vald/internal/errgroup/group.go:131 +0x23b
> created by github.com/vdaas/vald/internal/errgroup.(*group).Go
>         github.com/vdaas/vald/internal/errgroup/group.go:115 +0x93
> 
> goroutine 78 [IO wait]:
> internal/poll.runtime_pollWait(0x7fecf3d4fe90, 0x72)
>         runtime/netpoll.go:229 +0x89
> internal/poll.(*pollDesc).wait(0xc00050de00, 0x20, 0x0)
>         internal/poll/fd_poll_runtime.go:84 +0x32
> internal/poll.(*pollDesc).waitRead(...)
>         internal/poll/fd_poll_runtime.go:89
> internal/poll.(*FD).Accept(0xc00050de00)
>         internal/poll/fd_unix.go:402 +0x22c
> net.(*netFD).accept(0xc00050de00)
>         net/fd_unix.go:173 +0x35
> net.(*UnixListener).accept(0xc000067c98)
>         net/unixsock_posix.go:167 +0x1c
> net.(*UnixListener).Accept(0xc00053c510)
>         net/unixsock.go:260 +0x3d
> google.golang.org/grpc.(*Server).Serve(0xc0002cafc0, {0x1523b30, 0xc00053c510})
>         google.golang.org/grpc@v1.40.0/server.go:779 +0x362
> github.com/vdaas/vald/internal/servers/server.(*server).ListenAndServe.func3()
>         github.com/vdaas/vald/internal/servers/server/server.go:310 +0x3e8
> github.com/vdaas/vald/internal/safety.recoverFunc.func1()
>         github.com/vdaas/vald/internal/safety/safety.go:63 +0x73
> github.com/vdaas/vald/internal/errgroup.(*group).Go.func1()
>         github.com/vdaas/vald/internal/errgroup/group.go:131 +0x23b
> created by github.com/vdaas/vald/internal/errgroup.(*group).Go
>         github.com/vdaas/vald/internal/errgroup/group.go:115 +0x93
> 
> goroutine 79 [IO wait]:
> internal/poll.runtime_pollWait(0x7fecf3d4fda8, 0x72)
>         runtime/netpoll.go:229 +0x89
> internal/poll.(*pollDesc).wait(0xc00050df80, 0x42c066, 0x0)
>         internal/poll/fd_poll_runtime.go:84 +0x32
> internal/poll.(*pollDesc).waitRead(...)
>         internal/poll/fd_poll_runtime.go:89
> internal/poll.(*FD).Accept(0xc00050df80)
>         internal/poll/fd_unix.go:402 +0x22c
> net.(*netFD).accept(0xc00050df80)
>         net/fd_unix.go:173 +0x35
> net.(*TCPListener).accept(0xc0003e9c38)
>         net/tcpsock_posix.go:140 +0x28
> net.(*TCPListener).Accept(0xc0003e9c38)
>         net/tcpsock.go:262 +0x3d
> net/http.(*Server).Serve(0xc0002e8540, {0x1523b00, 0xc0003e9c38})
>         net/http/server.go:3001 +0x394
> github.com/vdaas/vald/internal/servers/server.(*server).ListenAndServe.func3()
>         github.com/vdaas/vald/internal/servers/server/server.go:304 +0x352
> github.com/vdaas/vald/internal/safety.recoverFunc.func1()
>         github.com/vdaas/vald/internal/safety/safety.go:63 +0x73
> github.com/vdaas/vald/internal/errgroup.(*group).Go.func1()
>         github.com/vdaas/vald/internal/errgroup/group.go:131 +0x23b
> created by github.com/vdaas/vald/internal/errgroup.(*group).Go
>         github.com/vdaas/vald/internal/errgroup/group.go:115 +0x93
> 
> goroutine 80 [chan receive]:
> github.com/vdaas/vald/internal/servers.(*listener).ListenAndServe.func1()
>         github.com/vdaas/vald/internal/servers/servers.go:91 +0x78
> github.com/vdaas/vald/internal/safety.recoverFunc.func1()
>         github.com/vdaas/vald/internal/safety/safety.go:63 +0x73
> github.com/vdaas/vald/internal/errgroup.(*group).Go.func1()
>         github.com/vdaas/vald/internal/errgroup/group.go:131 +0x23b
> created by github.com/vdaas/vald/internal/errgroup.(*group).Go
>         github.com/vdaas/vald/internal/errgroup/group.go:115 +0x93
> 
> goroutine 84 [select]:
> google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc00051c820, 0x1)
>         google.golang.org/grpc@v1.40.0/internal/transport/controlbuf.go:406 +0x11b
> google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc00074c180)
>         google.golang.org/grpc@v1.40.0/internal/transport/controlbuf.go:533 +0x85
> google.golang.org/grpc/internal/transport.NewServerTransport.func2()
>         google.golang.org/grpc@v1.40.0/internal/transport/http2_server.go:321 +0xc6
> created by google.golang.org/grpc/internal/transport.NewServerTransport
>         google.golang.org/grpc@v1.40.0/internal/transport/http2_server.go:318 +0x17ef
> 
> goroutine 85 [select]:
> google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc0003b2180)
>         google.golang.org/grpc@v1.40.0/internal/transport/http2_server.go:1078 +0x259
> created by google.golang.org/grpc/internal/transport.NewServerTransport
>         google.golang.org/grpc@v1.40.0/internal/transport/http2_server.go:330 +0x1837
> 
> goroutine 86 [IO wait]:
> internal/poll.runtime_pollWait(0x7fecf3d4fcc0, 0x72)
>         runtime/netpoll.go:229 +0x89
> internal/poll.(*pollDesc).wait(0xc0000e6500, 0xc000766000, 0x0)
>         internal/poll/fd_poll_runtime.go:84 +0x32
> internal/poll.(*pollDesc).waitRead(...)
>         internal/poll/fd_poll_runtime.go:89
> internal/poll.(*FD).Read(0xc0000e6500, {0xc000766000, 0x8000, 0x8000})
>         internal/poll/fd_unix.go:167 +0x25a
> net.(*netFD).Read(0xc0000e6500, {0xc000766000, 0xc000062c78, 0x41a9e9})
>         net/fd_posix.go:56 +0x29
> net.(*conn).Read(0xc000010030, {0xc000766000, 0xc00074c420, 0x462574})
>         net/net.go:183 +0x45
> bufio.(*Reader).Read(0xc00074c120, {0xc0002e8660, 0x9, 0xc000062d30})
>         bufio/bufio.go:227 +0x1b4
> io.ReadAtLeast({0x150caa0, 0xc00074c120}, {0xc0002e8660, 0x9, 0x9}, 0x9)
>         io/io.go:328 +0x9a
> io.ReadFull(...)
>         io/io.go:347
> golang.org/x/net/http2.readFrameHeader({0xc0002e8660, 0x9, 0xc0003ed8c0}, {0x150caa0, 0xc00074c120})
>         golang.org/x/net@v0.0.0-20210825183410-e898025ed96a/http2/frame.go:237 +0x6e
> golang.org/x/net/http2.(*Framer).ReadFrame(0xc0002e8620)
>         golang.org/x/net@v0.0.0-20210825183410-e898025ed96a/http2/frame.go:498 +0x95
> google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc0003b2180, 0x1000000004aa101, 0x488200)
>         google.golang.org/grpc@v1.40.0/internal/transport/http2_server.go:558 +0xb2
> google.golang.org/grpc.(*Server).serveStreams(0xc0002cafc0, {0x153bc58, 0xc0003b2180})
>         google.golang.org/grpc@v1.40.0/server.go:907 +0x142
> google.golang.org/grpc.(*Server).handleRawConn.func1()
>         google.golang.org/grpc@v1.40.0/server.go:847 +0x46
> created by google.golang.org/grpc.(*Server).handleRawConn
>         google.golang.org/grpc@v1.40.0/server.go:846 +0x185
> 
> rax    0x7fecb4388f50
> rbx    0x7fecc67fbcc8
> rcx    0x7fecb4389150
> rdx    0x7fecb4000f40
> rdi    0x1d0d4c0
> rsi    0x7fecb4389150
> rbp    0x7fecc67fbaf0
> rsp    0x7fecc67fbaa0
> r8     0x7fecb4388f50
> r9     0x7fecb4388f50
> r10    0x230
> r11    0x206
> r12    0x7fecc67fbbe0
> r13    0x0
> r14    0x7fecb4000f38
> r15    0x7fecc67fbbe0
> rip    0xedb8ac
> rflags 0x10212
> cs     0x33
> fs     0x0
> gs     0x0
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 122, in run
>     algo.fit(X_train)
>   File "/home/app/ann_benchmarks/algorithms/vald.py", line 131, in fit
>     pool_size=10000))
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 946, in __call__
>     return _end_unary_response_blocking(state, call, False, None)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
>     raise _InactiveRpcError(state)
> grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
>         status = StatusCode.UNAVAILABLE
>         details = "Socket closed"
>         debug_error_string = "{"created":"@1654049853.438688700","description":"Error received from peer unix:/var/run/vald.sock","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
> >
> ←[0m
> Child process for container 4fe6dfddcbreturned exit code 1 with message None
> ←[31m/home/app/entrypoint.sh: line 2:    10 Illegal instruction     /var/lib/milvus/bin/milvus_server -d -c /var/lib/milvus/conf/server_config.yaml -l /var/lib/milvus/conf/log_config.conf
> ['angular', 'IVF_SQ8', 300]
> Trying to instantiate ann_benchmarks.algorithms.milvus.Milvus(['angular', 'IVF_SQ8', 300])
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 212, in connect
>     grpc.channel_ready_future(self._channel).result(timeout=timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 139, in result
>     self._block(timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 85, in _block
>     raise grpc.FutureTimeoutError()
> grpc.FutureTimeoutError
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 99, in run
>     algo = instantiate_algorithm(definition)
>   File "/home/app/ann_benchmarks/algorithms/definitions.py", line 24, in instantiate_algorithm
>     return constructor(*definition.arguments)
>   File "/home/app/ann_benchmarks/algorithms/milvus.py", line 14, in __init__
>     self._milvus.connect(host='localhost', port='19530')
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/stub.py", line 45, in connect
>     return self._handler.connect(host, port, uri, timeout)
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 216, in connect
>     raise NotConnectError('Fail connecting to server on {}. Timeout'.format(self._uri))
> milvus.client.exceptions.NotConnectError: Fail connecting to server on localhost:19530. Timeout
> ←[0m
> Child process for container 535457d77breturned exit code 1 with message None
> ←[31m/home/app/entrypoint.sh: line 2:    10 Illegal instruction     /var/lib/milvus/bin/milvus_server -d -c /var/lib/milvus/conf/server_config.yaml -l /var/lib/milvus/conf/log_config.conf
> ['angular', 'IVF_FLAT', 100]
> Trying to instantiate ann_benchmarks.algorithms.milvus.Milvus(['angular', 'IVF_FLAT', 100])
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 212, in connect
>     grpc.channel_ready_future(self._channel).result(timeout=timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 139, in result
>     self._block(timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 85, in _block
>     raise grpc.FutureTimeoutError()
> grpc.FutureTimeoutError
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 99, in run
>     algo = instantiate_algorithm(definition)
>   File "/home/app/ann_benchmarks/algorithms/definitions.py", line 24, in instantiate_algorithm
>     return constructor(*definition.arguments)
>   File "/home/app/ann_benchmarks/algorithms/milvus.py", line 14, in __init__
>     self._milvus.connect(host='localhost', port='19530')
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/stub.py", line 45, in connect
>     return self._handler.connect(host, port, uri, timeout)
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 216, in connect
>     raise NotConnectError('Fail connecting to server on {}. Timeout'.format(self._uri))
> milvus.client.exceptions.NotConnectError: Fail connecting to server on localhost:19530. Timeout
> ←[0m
> Child process for container e948999b37returned exit code 1 with message None
> ←[31m/home/app/entrypoint.sh: line 2:    10 Illegal instruction     /var/lib/milvus/bin/milvus_server -d -c /var/lib/milvus/conf/server_config.yaml -l /var/lib/milvus/conf/log_config.conf
> ['angular', 'IVF_FLAT', 1000]
> Trying to instantiate ann_benchmarks.algorithms.milvus.Milvus(['angular', 'IVF_FLAT', 1000])
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 212, in connect
>     grpc.channel_ready_future(self._channel).result(timeout=timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 139, in result
>     self._block(timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 85, in _block
>     raise grpc.FutureTimeoutError()
> grpc.FutureTimeoutError
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 99, in run
>     algo = instantiate_algorithm(definition)
>   File "/home/app/ann_benchmarks/algorithms/definitions.py", line 24, in instantiate_algorithm
>     return constructor(*definition.arguments)
>   File "/home/app/ann_benchmarks/algorithms/milvus.py", line 14, in __init__
>     self._milvus.connect(host='localhost', port='19530')
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/stub.py", line 45, in connect
>     return self._handler.connect(host, port, uri, timeout)
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 216, in connect
>     raise NotConnectError('Fail connecting to server on {}. Timeout'.format(self._uri))
> milvus.client.exceptions.NotConnectError: Fail connecting to server on localhost:19530. Timeout
> ←[0m
> Child process for container 7e33c765e7returned exit code 1 with message None
> ←[31m/home/app/entrypoint.sh: line 2:     6 Illegal instruction     /var/lib/milvus/bin/milvus_server -d -c /var/lib/milvus/conf/server_config.yaml -l /var/lib/milvus/conf/log_config.conf
> ['angular', 'IVF_SQ8', 10000]
> Trying to instantiate ann_benchmarks.algorithms.milvus.Milvus(['angular', 'IVF_SQ8', 10000])
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 212, in connect
>     grpc.channel_ready_future(self._channel).result(timeout=timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 139, in result
>     self._block(timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 85, in _block
>     raise grpc.FutureTimeoutError()
> grpc.FutureTimeoutError
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 99, in run
>     algo = instantiate_algorithm(definition)
>   File "/home/app/ann_benchmarks/algorithms/definitions.py", line 24, in instantiate_algorithm
>     return constructor(*definition.arguments)
>   File "/home/app/ann_benchmarks/algorithms/milvus.py", line 14, in __init__
>     self._milvus.connect(host='localhost', port='19530')
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/stub.py", line 45, in connect
>     return self._handler.connect(host, port, uri, timeout)
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 216, in connect
>     raise NotConnectError('Fail connecting to server on {}. Timeout'.format(self._uri))
> milvus.client.exceptions.NotConnectError: Fail connecting to server on localhost:19530. Timeout
> ←[0m
> Child process for container c77b00f613returned exit code 1 with message None
> ←[31m/home/app/entrypoint.sh: line 2:     9 Illegal instruction     /var/lib/milvus/bin/milvus_server -d -c /var/lib/milvus/conf/server_config.yaml -l /var/lib/milvus/conf/log_config.conf
> ['angular', 'IVF_FLAT', 3000]
> Trying to instantiate ann_benchmarks.algorithms.milvus.Milvus(['angular', 'IVF_FLAT', 3000])
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 212, in connect
>     grpc.channel_ready_future(self._channel).result(timeout=timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 139, in result
>     self._block(timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 85, in _block
>     raise grpc.FutureTimeoutError()
> grpc.FutureTimeoutError
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 99, in run
>     algo = instantiate_algorithm(definition)
>   File "/home/app/ann_benchmarks/algorithms/definitions.py", line 24, in instantiate_algorithm
>     return constructor(*definition.arguments)
>   File "/home/app/ann_benchmarks/algorithms/milvus.py", line 14, in __init__
>     self._milvus.connect(host='localhost', port='19530')
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/stub.py", line 45, in connect
>     return self._handler.connect(host, port, uri, timeout)
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 216, in connect
>     raise NotConnectError('Fail connecting to server on {}. Timeout'.format(self._uri))
> milvus.client.exceptions.NotConnectError: Fail connecting to server on localhost:19530. Timeout
> ←[0m
> Child process for container 98a1094dc1returned exit code 1 with message None
> ←[31m['angular', 25, {'M': 16, 'efConstruction': 500}]
> Trying to instantiate ann_benchmarks.algorithms.opensearchknn.OpenSearchKNN(['angular', 25, {'M': 16, 'efConstruction': 500}])
> Waiting for elasticsearch health endpoint...
> Elasticsearch is ready
> got a train set of size (1183514 * 25)
> got 10000 queries
> Uploading data to the Index: os-m-16-efconstruction-500
> 100%|##########| 1183514/1183514 [04:01<00:00, 4908.78it/s]
> Force Merge...
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 449, in _make_request
>     six.raise_from(e, None)
>   File "<string>", line 3, in raise_from
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 444, in _make_request
>     httplib_response = conn.getresponse()
>   File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse
>     response.begin()
>   File "/usr/lib/python3.6/http/client.py", line 320, in begin
>     version, status, reason = self._read_status()
>   File "/usr/lib/python3.6/http/client.py", line 281, in _read_status
>     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
>   File "/usr/lib/python3.6/socket.py", line 586, in readinto
>     return self._sock.recv_into(b)
> socket.timeout: timed out
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 252, in perform_request
>     method, url, body, retries=Retry(False), headers=request_headers, **kw
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 786, in urlopen
>     method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 525, in increment
>     raise six.reraise(type(error), error, _stacktrace)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 770, in reraise
>     raise value
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 710, in urlopen
>     chunked=chunked,
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 451, in _make_request
>     self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 341, in _raise_timeout
>     self, url, "Read timed out. (read timeout=%s)" % timeout_value
> urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=1000)
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 122, in run
>     algo.fit(X_train)
>   File "/home/app/ann_benchmarks/algorithms/opensearchknn.py", line 69, in fit
>     self.es.indices.forcemerge(self.name, max_num_segments=1, request_timeout=1000)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/utils.py", line 168, in _wrapped
>     return func(*args, params=params, headers=headers, **kwargs)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/indices.py", line 1125, in forcemerge
>     "POST", _make_path(index, "_forcemerge"), params=params, headers=headers
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 415, in perform_request
>     raise e
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 388, in perform_request
>     timeout=timeout,
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 265, in perform_request
>     raise ConnectionTimeout("TIMEOUT", str(e), e)
> elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=1000))
> ←[0m
> Child process for container 22321304c9returned exit code 1 with message None
> ←[31m['angular', 25, {'M': 24, 'efConstruction': 500}]
> Trying to instantiate ann_benchmarks.algorithms.opensearchknn.OpenSearchKNN(['angular', 25, {'M': 24, 'efConstruction': 500}])
> Waiting for elasticsearch health endpoint...
> Elasticsearch is ready
> got a train set of size (1183514 * 25)
> got 10000 queries
> Uploading data to the Index: os-m-24-efconstruction-500
> 100%|##########| 1183514/1183514 [03:52<00:00, 5096.40it/s]
> Force Merge...
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 449, in _make_request
>     six.raise_from(e, None)
>   File "<string>", line 3, in raise_from
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 444, in _make_request
>     httplib_response = conn.getresponse()
>   File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse
>     response.begin()
>   File "/usr/lib/python3.6/http/client.py", line 320, in begin
>     version, status, reason = self._read_status()
>   File "/usr/lib/python3.6/http/client.py", line 281, in _read_status
>     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
>   File "/usr/lib/python3.6/socket.py", line 586, in readinto
>     return self._sock.recv_into(b)
> socket.timeout: timed out
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 252, in perform_request
>     method, url, body, retries=Retry(False), headers=request_headers, **kw
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 786, in urlopen
>     method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 525, in increment
>     raise six.reraise(type(error), error, _stacktrace)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 770, in reraise
>     raise value
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 710, in urlopen
>     chunked=chunked,
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 451, in _make_request
>     self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 341, in _raise_timeout
>     self, url, "Read timed out. (read timeout=%s)" % timeout_value
> urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=1000)
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 122, in run
>     algo.fit(X_train)
>   File "/home/app/ann_benchmarks/algorithms/opensearchknn.py", line 69, in fit
>     self.es.indices.forcemerge(self.name, max_num_segments=1, request_timeout=1000)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/utils.py", line 168, in _wrapped
>     return func(*args, params=params, headers=headers, **kwargs)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/indices.py", line 1125, in forcemerge
>     "POST", _make_path(index, "_forcemerge"), params=params, headers=headers
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 415, in perform_request
>     raise e
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 388, in perform_request
>     timeout=timeout,
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 265, in perform_request
>     raise ConnectionTimeout("TIMEOUT", str(e), e)
> elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=1000))
> ←[0m
> Child process for container 280e8265bfreturned exit code 1 with message None
> ←[31m['angular', 25, {'M': 36, 'efConstruction': 500}]
> Trying to instantiate ann_benchmarks.algorithms.opensearchknn.OpenSearchKNN(['angular', 25, {'M': 36, 'efConstruction': 500}])
> Waiting for elasticsearch health endpoint...
> Elasticsearch is ready
> got a train set of size (1183514 * 25)
> got 10000 queries
> Uploading data to the Index: os-m-36-efconstruction-500
> 100%|##########| 1183514/1183514 [03:57<00:00, 4973.80it/s]
> Force Merge...
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 449, in _make_request
>     six.raise_from(e, None)
>   File "<string>", line 3, in raise_from
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 444, in _make_request
>     httplib_response = conn.getresponse()
>   File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse
>     response.begin()
>   File "/usr/lib/python3.6/http/client.py", line 320, in begin
>     version, status, reason = self._read_status()
>   File "/usr/lib/python3.6/http/client.py", line 281, in _read_status
>     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
>   File "/usr/lib/python3.6/socket.py", line 586, in readinto
>     return self._sock.recv_into(b)
> socket.timeout: timed out
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 252, in perform_request
>     method, url, body, retries=Retry(False), headers=request_headers, **kw
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 786, in urlopen
>     method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 525, in increment
>     raise six.reraise(type(error), error, _stacktrace)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 770, in reraise
>     raise value
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 710, in urlopen
>     chunked=chunked,
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 451, in _make_request
>     self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
>   File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 341, in _raise_timeout
>     self, url, "Read timed out. (read timeout=%s)" % timeout_value
> urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=1000)
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 122, in run
>     algo.fit(X_train)
>   File "/home/app/ann_benchmarks/algorithms/opensearchknn.py", line 69, in fit
>     self.es.indices.forcemerge(self.name, max_num_segments=1, request_timeout=1000)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/utils.py", line 168, in _wrapped
>     return func(*args, params=params, headers=headers, **kwargs)
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/client/indices.py", line 1125, in forcemerge
>     "POST", _make_path(index, "_forcemerge"), params=params, headers=headers
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 415, in perform_request
>     raise e
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/transport.py", line 388, in perform_request
>     timeout=timeout,
>   File "/usr/local/lib/python3.6/dist-packages/elasticsearch/connection/http_urllib3.py", line 265, in perform_request
>     raise ConnectionTimeout("TIMEOUT", str(e), e)
> elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=1000))
> ←[0m
> Child process for container d329a3e4careturned exit code 1 with message None
> ←[31m/home/app/entrypoint.sh: line 2:    10 Illegal instruction     /var/lib/milvus/bin/milvus_server -d -c /var/lib/milvus/conf/server_config.yaml -l /var/lib/milvus/conf/log_config.conf
> ['angular', 'IVF_FLAT', 30000]
> Trying to instantiate ann_benchmarks.algorithms.milvus.Milvus(['angular', 'IVF_FLAT', 30000])
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 212, in connect
>     grpc.channel_ready_future(self._channel).result(timeout=timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 139, in result
>     self._block(timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 85, in _block
>     raise grpc.FutureTimeoutError()
> grpc.FutureTimeoutError
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 99, in run
>     algo = instantiate_algorithm(definition)
>   File "/home/app/ann_benchmarks/algorithms/definitions.py", line 24, in instantiate_algorithm
>     return constructor(*definition.arguments)
>   File "/home/app/ann_benchmarks/algorithms/milvus.py", line 14, in __init__
>     self._milvus.connect(host='localhost', port='19530')
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/stub.py", line 45, in connect
>     return self._handler.connect(host, port, uri, timeout)
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 216, in connect
>     raise NotConnectError('Fail connecting to server on {}. Timeout'.format(self._uri))
> milvus.client.exceptions.NotConnectError: Fail connecting to server on localhost:19530. Timeout
> ←[0m
> Child process for container a7130ba538returned exit code 1 with message None
> ←[31m['angular', 1204]
> Trying to instantiate ann_benchmarks.algorithms.faiss.FaissIVFPQfs(['angular', 1204])
> got a train set of size (1183514 * 25)
> got 10000 queries
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 122, in run
>     algo.fit(X_train)
>   File "/home/app/ann_benchmarks/algorithms/faiss.py", line 99, in fit
>     index = faiss.index_factory(d, factory_string, faiss_metric)
>   File "/root/anaconda3/lib/python3.8/site-packages/faiss/swigfaiss.py", line 5636, in index_factory
>     return _swigfaiss.index_factory(*args)
> RuntimeError: Error in void faiss::ProductQuantizer::set_derived_values() at /opt/conda/conda-bld/faiss-pkg_1623024438023/work/faiss/impl/ProductQuantizer.cpp:186: Error: 'd % M == 0' failed: The dimension of the vector (d) should be a multiple of the number of subquantizers (M)
> ←[0m
> Child process for container a50221517freturned exit code 1 with message None
> ←[31m['angular', 2048]
> Trying to instantiate ann_benchmarks.algorithms.faiss.FaissIVFPQfs(['angular', 2048])
> got a train set of size (1183514 * 25)
> got 10000 queries
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 122, in run
>     algo.fit(X_train)
>   File "/home/app/ann_benchmarks/algorithms/faiss.py", line 99, in fit
>     index = faiss.index_factory(d, factory_string, faiss_metric)
>   File "/root/anaconda3/lib/python3.8/site-packages/faiss/swigfaiss.py", line 5636, in index_factory
>     return _swigfaiss.index_factory(*args)
> RuntimeError: Error in void faiss::ProductQuantizer::set_derived_values() at /opt/conda/conda-bld/faiss-pkg_1623024438023/work/faiss/impl/ProductQuantizer.cpp:186: Error: 'd % M == 0' failed: The dimension of the vector (d) should be a multiple of the number of subquantizers (M)
> ←[0m
> Child process for container a0750667bereturned exit code 1 with message None
> ←[31m/home/app/entrypoint.sh: line 2:    10 Illegal instruction     /var/lib/milvus/bin/milvus_server -d -c /var/lib/milvus/conf/server_config.yaml -l /var/lib/milvus/conf/log_config.conf
> ['angular', 'IVF_SQ8', 100]
> Trying to instantiate ann_benchmarks.algorithms.milvus.Milvus(['angular', 'IVF_SQ8', 100])
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 212, in connect
>     grpc.channel_ready_future(self._channel).result(timeout=timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 139, in result
>     self._block(timeout)
>   File "/usr/local/lib/python3.6/dist-packages/grpc/_utilities.py", line 85, in _block
>     raise grpc.FutureTimeoutError()
> grpc.FutureTimeoutError
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "run_algorithm.py", line 3, in <module>
>     run_from_cmdline()
>   File "/home/app/ann_benchmarks/runner.py", line 211, in run_from_cmdline
>     run(definition, args.dataset, args.count, args.runs, args.batch)
>   File "/home/app/ann_benchmarks/runner.py", line 99, in run
>     algo = instantiate_algorithm(definition)
>   File "/home/app/ann_benchmarks/algorithms/definitions.py", line 24, in instantiate_algorithm
>     return constructor(*definition.arguments)
>   File "/home/app/ann_benchmarks/algorithms/milvus.py", line 14, in __init__
>     self._milvus.connect(host='localhost', port='19530')
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/stub.py", line 45, in connect
>     return self._handler.connect(host, port, uri, timeout)
>   File "/usr/local/lib/python3.6/dist-packages/milvus/client/grpc_handler.py", line 216, in connect
>     raise NotConnectError('Fail connecting to server on {}. Timeout'.format(self._uri))
> milvus.client.exceptions.NotConnectError: Fail connecting to server on localhost:19530. Timeout
> ←[0m
> Child process for container 809ee34a9areturned exit code 1 with message None
> 

> Force Merge...
> ...
> urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=1000)

I think it's timing out on the force merge step. So all vectors have been indexed, it's trying to, basically, "optimize" the index, but it's taking too long.

@fcakir @maumueller Hi, I am the maintainer of the Vald project.
Unfortunately, current Vald docker image does not support non-AVX2 environments.
The CPU Xeon E5-1607 v2 does not support AVX2, so it has been confirmed that Vald does not work properly.

The successor CPU, Xeon E5-1607 v3, supports AVX2 and will work.
FYI: https://technical.city/en/cpu/Xeon-E5-1607-v3-vs-Xeon-E5-1607-v2

Since we reused the official Vald docker image for this part when running the ann-benchmark, I will submit a pull request for the change at a later date, since it is necessary to rebuild from source code when using Vald in a non-AVX2 environment.