Elasticsearch Vector Store
Source: llama_index

<a href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/docs/examples/vector_stores/ElasticsearchIndexDemo.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a># Elasticsearch Vector StoreElasticsearch is a distributed, RESTful search and analytics engine built on top of Apache Lucene. It offers different retrieval options including dense vector retrieval, sparse vector retrieval, keyword search and hybrid search.[Sign up](https://cloud.elastic.co/registration?utm_source=llama-index&utm_content=documentation) for a free trial of Elastic Cloud or run a local server like described below.Requires Elasticsearch 8.9.0 or higher and AIOHTTP.```python%pip install -qU llama-index-vector-stores-elasticsearch llama-index openai``````pythonimport getpassimport osimport openaios.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")openai.api_key = os.environ["OPENAI_API_KEY"]```## Running and connecting to ElasticsearchTwo ways to setup an Elasticsearch instance for use with:### Elastic CloudElastic Cloud is a managed Elasticsearch service. [Sign up](https://cloud.elastic.co/registration?utm_source=llama-index&utm_content=documentation) for a free trial.### LocallyGet started with Elasticsearch by running it locally. The easiest way is to use the official Elasticsearch Docker image. See the Elasticsearch Docker documentation for more information.```bashdocker run -p 9200:9200 \  -e "discovery.type=single-node" \  -e "xpack.security.enabled=false" \  -e "xpack.license.self_generated.type=trial" \  docker.elastic.co/elasticsearch/elasticsearch:8.13.2```## Configuring ElasticsearchStoreThe ElasticsearchStore class is used to connect to an Elasticsearch instance. It requires the following parameters:        - index_name: Name of the Elasticsearch index. Required.        - es_client: Optional. Pre-existing Elasticsearch client.        - es_url: Optional. Elasticsearch URL.        - es_cloud_id: Optional. Elasticsearch cloud ID.        - es_api_key: Optional. Elasticsearch API key.        - es_user: Optional. Elasticsearch username.        - es_password: Optional. Elasticsearch password.        - text_field: Optional. Name of the Elasticsearch field that stores the text.        - vector_field: Optional. Name of the Elasticsearch field that stores the                    embedding.        - batch_size: Optional. Batch size for bulk indexing. Defaults to 200.        - distance_strategy: Optional. Distance strategy to use for similarity search.                        Defaults to "COSINE".### Example: Connecting locally```pythonfrom llama_index.vector_stores.elasticsearch import ElasticsearchStorees = ElasticsearchStore(    index_name="my_index",    es_url="http://localhost:9200",)```### Example: Connecting to Elastic Cloud with username and password```pythonfrom llama_index.vector_stores.elasticsearch import ElasticsearchStorees = ElasticsearchStore(    index_name="my_index",    es_cloud_id="<cloud-id>", # found within the deployment page    es_user="elastic"    es_password="<password>" # provided when creating deployment. Alternatively can reset password.)```### Example: Connecting to Elastic Cloud with API Key```pythonfrom llama_index.stores.elasticsearch import ElasticsearchStorees = ElasticsearchStore(    index_name="my_index",    es_cloud_id="<cloud-id>", # found within the deployment page    es_api_key="<api-key>" # create an API key within Kibana (Security -> API Keys))```#### Example data```pythonfrom llama_index.core.schema import TextNodemovies = [    TextNode(        text="The lives of two mob hitmen, a boxer, a gangster and his wife, and a pair of diner bandits intertwine in four tales of violence and redemption.",        metadata={"title": "Pulp Fiction"},    ),    TextNode(        text="When the menace known as the Joker wreaks havoc and chaos on the people of Gotham, Batman must accept one of the greatest psychological and physical tests of his ability to fight injustice.",        metadata={"title": "The Dark Knight"},    ),    TextNode(        text="An insomniac office worker and a devil-may-care soapmaker form an underground fight club that evolves into something much, much more.",        metadata={"title": "Fight Club"},    ),    TextNode(        text="A thief who steals corporate secrets through the use of dream-sharing technology is given the inverse task of planting an idea into thed of a C.E.O.",        metadata={"title": "Inception"},    ),    TextNode(        text="A computer hacker learns from mysterious rebels about the true nature of his reality and his role in the war against its controllers.",        metadata={"title": "The Matrix"},    ),    TextNode(        text="Two detectives, a rookie and a veteran, hunt a serial killer who uses the seven deadly sins as his motives.",        metadata={"title": "Se7en"},    ),    TextNode(        text="An organized crime dynasty's aging patriarch transfers control of his clandestine empire to his reluctant son.",        metadata={"title": "The Godfather", "theme": "Mafia"},    ),]```## Retrieval ExamplesThis section shows the different retrieval options available through the `ElasticsearchStore` and make use of them via a VectorStoreIndex.```pythonfrom llama_index.core import StorageContext, VectorStoreIndexfrom llama_index.vector_stores.elasticsearch import ElasticsearchStore```We first define a helper function to retrieve and print results for user query input:```pythondef print_results(results):    for rank, result in enumerate(results, 1):        print(            f"{rank}. title={result.metadata['title']} score={result.get_score()} text={result.get_text()}"        )def search(    vector_store: ElasticsearchStore, nodes: list[TextNode], query: str):    storage_context = StorageContext.from_defaults(vector_store=vector_store)    index = VectorStoreIndex(nodes, storage_context=storage_context)    print(">>> Documents:")    retriever = index.as_retriever()    results = retriever.retrieve(query)    print_results(results)    print("\n>>> Answer:")    query_engine = index.as_query_engine()    response = query_engine.query(query)    print(response)```### Dense retrievalHere we use embeddings from OpenAI to search.```pythonfrom llama_index.vector_stores.elasticsearch import AsyncDenseVectorStrategydense_vector_store = ElasticsearchStore(    es_url="http://localhost:9200",  # for Elastic Cloud authentication see above    index_name="movies_dense",    retrieval_strategy=AsyncDenseVectorStrategy(),)search(dense_vector_store, movies, "which movie involves dreaming?")```    >>> Documents:    1. title=Inception score=1.0 text=A thief who steals corporate secrets through the use of dream-sharing technology is given the inverse task of planting an idea into thed of a C.E.O.        >>> Answer:    InceptionThis is also the default retrieval strategy:```pythondefault_store = ElasticsearchStore(    es_url="http://localhost:9200",  # for Elastic Cloud authentication see above    index_name="movies_default",)search(default_store, movies, "which movie involves dreaming?")```    >>> Documents:    1. title=Inception score=1.0 text=A thief who steals corporate secrets through the use of dream-sharing technology is given the inverse task of planting an idea into thed of a C.E.O.        >>> Answer:    Inception### Sparse retrievalFor this example you first need to [deploy the ELSER model](https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-elser.html) version two in your Elasticsearch deployment.```pythonfrom llama_index.vector_stores.elasticsearch import AsyncSparseVectorStrategysparse_vector_store = ElasticsearchStore(    es_url="http://localhost:9200",  # for Elastic Cloud authentication see above    index_name="movies_sparse",    retrieval_strategy=AsyncSparseVectorStrategy(model_id=".elser_model_2"),)search(sparse_vector_store, movies, "which movie involves dreaming?")```    >>> Documents:    1. title=Inception score=1.0 text=A thief who steals corporate secrets through the use of dream-sharing technology is given the inverse task of planting an idea into thed of a C.E.O.        >>> Answer:    Inception### Keyword retrievalTo use classic full-text search, you can use the BM25 strategy.```pythonfrom llama_index.vector_stores.elasticsearch import AsyncBM25Strategybm25_store = ElasticsearchStore(    es_url="http://localhost:9200",  # for Elastic Cloud authentication see above    index_name="movies_bm25",    retrieval_strategy=AsyncBM25Strategy(),)search(bm25_store, movies, "joker")```    >>> Documents:    1. title=The Dark Knight score=1.0 text=When the menace known as the Joker wreaks havoc and chaos on the people of Gotham, Batman must accept one of the greatest psychological and physical tests of his ability to fight injustice.        >>> Answer:    The Joker is a menacing character who wreaks havoc and chaos on the people of Gotham, posing a significant challenge for Batman to combat injustice.### Hybrid retrievalCombining dense retrieval and keyword search for hybrid retrieval can be enabled by setting a flag.```pythonfrom llama_index.vector_stores.elasticsearch import AsyncDenseVectorStrategyhybrid_store = ElasticsearchStore(    es_url="http://localhost:9200",  # for Elastic Cloud authentication see above    index_name="movies_hybrid",    retrieval_strategy=AsyncDenseVectorStrategy(hybrid=True),)search(hybrid_store, movies, "which movie involves dreaming?")```    >>> Documents:    1. title=Inception score=0.36787944117144233 text=A thief who steals corporate secrets through the use of dream-sharing technology is given the inverse task of planting an idea into thed of a C.E.O.        >>> Answer:    "Inception" is the movie that involves dreaming.### Metadata FiltersWe can also apply filters to the query engine based on the metadata of our documents.```pythonfrom llama_index.core.vector_stores import ExactMatchFilter, MetadataFiltersmetadata_store = ElasticsearchStore(    es_url="http://localhost:9200",  # for Elastic Cloud authentication see above    index_name="movies_metadata",)storage_context = StorageContext.from_defaults(vector_store=metadata_store)index = VectorStoreIndex(movies, storage_context=storage_context)# Metadata filterfilters = MetadataFilters(    filters=[ExactMatchFilter(key="theme", value="Mafia")])retriever = index.as_retriever(filters=filters)results = retriever.retrieve("What is inception about?")print_results(results)```    1. title=The Godfather score=1.0 text=An organized crime dynasty's aging patriarch transfers control of his clandestine empire to his reluctant son.## Custom Filters and overriding Query The elastic search implementation only supports ExactMatchFilters provided from LlamaIndex at the moment. Elasticsearch itself supports a wide range of filters, including range filters, geo filters, and more. To use these filters, you can pass them in as a list of dictionaries to the `es_filter` parameter.```pythondef custom_query(query, query_str):    print("custom query", query)    return queryquery_engine = index.as_query_engine(    vector_store_kwargs={        "es_filter": [{"match": {"title": "matrix"}}],        "custom_query": custom_query,    })query_engine.query("what is this movie about?")```    custom query {'knn': {'filter': [{'match': {'title': 'matrix'}}], 'field': 'embedding', 'k': 2, 'num_candidates': 20, 'query_vector': [0.00446691969409585, -0.038953110575675964, -0.023963095620274544, -0.024891795590519905, -0.016729693859815598, 0.017200583592057228, -0.002360992832109332, -0.012622482143342495, -0.009980263188481331, -0.026108263060450554, 0.02950914017856121, 0.018626336008310318, -0.016154160723090172, -0.012099270708858967, 0.03777588531374931, 0.006209868937730789, 0.03539527207612991, -0.011746102944016457, 0.0029888467397540808, -0.022066453471779823, -0.02290359139442444, -0.011752642691135406, -0.018744058907032013, -0.015251620672643185, 0.0034074161667376757, 0.00014756205200683326, 0.022955913096666336, -0.02264198660850525, 0.002032350515946746, -0.021778685972094536, 0.012164671905338764, -0.015055416151881218, 0.006543416064232588, -0.009509372524917126, -0.008632993325591087, -0.006814832333475351, 0.011765723116695881, -0.01788076013326645, 0.00166691979393363, 0.002267795614898205, 0.015460905618965626, -0.016533490270376205, -0.014401402324438095, -0.0142836794257164, -0.020863065496087074, -0.01714826375246048, 0.027913343161344528, -0.032962337136268616, -0.016546569764614105, 0.0019947446417063475, 0.026304468512535095, 0.011706861667335033, -0.03733115270733833, -0.02812262810766697, -0.01879638060927391, 0.003232467221096158, -0.01393051166087389, -0.00879649631679058, 0.018469372764229774, -0.006402803119271994, 0.016481168568134308, 0.009149664081633091, -0.023492205888032913, -0.0024296643678098917, -0.00589921185746789, 0.0089338393881917, -0.016755854710936546, -0.0016309489728882909, -0.011726481840014458, -0.004267445299774408, 0.03162814676761627, 0.04190925508737564, 0.015513226389884949, 0.0019260731060057878, 0.027050044387578964, -0.03780204430222511, -0.038220614194869995, -0.008155561983585358, 0.01085010264068842, 0.021333957090973854, 0.026042861863970757, -0.008122861385345459, -0.021137751638889313, 0.02192256972193718, 0.0020094600040465593, -0.0013415474677458405, -0.0063570220954716206, 0.02903824858367443, 0.006232759449630976, 0.0072072409093379974, 0.00309185404330492, 0.0014167592162266374, 0.03691258653998375, 0.023034395650029182, -0.005048993043601513, 0.025336526334285736, 0.0038652264047414064, 0.009254306554794312, -0.007560409139841795, -0.02906440943479538, -0.00708951847627759, 0.0022220145910978317, -0.016808176413178444, -0.008305985480546951, -0.01105938758701086, -0.0062752701342105865, 0.011602219194173813, -0.011621839366853237, 0.04303416237235069, 0.0014069488970562816, -0.025179563090205193, 0.009293547831475735, 0.01714826375246048, -0.030110832303762436, -0.0007423065835610032, -0.023740731179714203, 0.019594278186559677, 0.0014004088006913662, 0.02074534446001053, -0.01884870044887066, 0.020876146852970123, 0.026814598590135574, 0.01534318272024393, -0.005971153266727924, 0.012328175827860832, -0.014074394479393959, -0.025192642584443092, -0.008855357766151428, 0.006697109900414944, -0.020091328769922256, 0.03050324134528637, -0.0019080876372754574, 0.0057128178887069225, 0.004175883252173662, -0.02783486247062683, 0.0035022483207285404, -0.0032242920715361834, 0.02075842395424843, -0.028253432363271713, -0.024237781763076782, 0.029352176934480667, 0.02979690581560135, -0.027573255822062492, -0.0004287883348297328, -0.02974458411335945, 0.004221664275974035, 0.01598411798477173, 0.02167404443025589, 0.020915387198328972, -0.00558855477720499, 0.011105168610811234, 0.009888701140880585, 0.007318423595279455, 0.0031539855990558863, 0.018992584198713303, 0.016729693859815598, -0.016559649258852005, -0.01837781071662903, -0.01570943184196949, 0.0026340438053011894, -0.01831240952014923, 0.0038815767038613558, -0.001384058385156095, -0.004784116987138987, 0.011281752027571201, 0.04175229370594025, 0.01274674478918314, -0.012328175827860832, -0.01595795713365078, -0.0022449051029980183, -0.011994628235697746, 0.03693874552845955, -0.030817169696092606, 0.01998668722808361, -0.007651971187442541, 0.009561693295836449, 0.01595795713365078, 0.019620439037680626, -0.013256876729428768, 0.012073109857738018, -0.011301372200250626, 0.0334070660173893, 0.028436556458473206, 0.024237781763076782, -0.030869489535689354, 0.023479124531149864, -0.013538102619349957, -0.009130043908953667, 0.00642569363117218, -0.0003075912536587566, -0.005300788674503565, 0.006984876003116369, 0.011092088185250759, -0.007822014391422272, -0.6659438610076904, -0.017449110746383667, 0.009332788176834583, 0.01096128486096859, 0.0040254597552120686, 0.007292263209819794, 0.01466300804167986, -0.006278540473431349, -0.01713518239557743, 0.02048373781144619, -0.007233401760458946, -0.004983591381460428, 0.012236613780260086, -0.01638960652053356, -0.027023883536458015, -0.01574867218732834, -0.0021141022443771362, -0.0309479720890522, -0.010941664688289165, 0.008842277340590954, -0.03544759377837181, 0.03526446968317032, -0.021307796239852905, -0.004947620443999767, 0.02236730046570301, 0.011491036973893642, 0.013001810759305954, -0.019241109490394592, -0.0025997080374509096, -0.003065693425014615, -0.008914219215512276, 0.038194455206394196, 0.0071549201384186745, -0.007625810336321592, 0.03589232265949249, -0.01926727034151554, -0.02520572394132614, 0.019816642627120018, 0.012949489057064056, 0.03740963712334633, -0.004728525876998901, -0.006651328876614571, 0.013603503815829754, 0.004447299521416426, -0.011870365589857101, -0.006517255678772926, 0.037226513028144836, -0.01930651068687439, 0.0057128178887069225, -0.004241284914314747, -0.0015598249156028032, 0.021582482382655144, 0.006111766677349806, 0.0034106862731277943, 0.00404508039355278, -0.007691211998462677, 0.029456818476319313, -0.006926015019416809, 0.007743533235043287, 0.002228554803878069, -0.004136642441153526, 0.028462715446949005, -0.0034957081079483032, -0.001543474500067532, -0.007815474644303322, 0.028384234756231308, 0.01809004507958889, 0.02330908179283142, 0.006497635040432215, -0.039868731051683426, 0.011144408956170082, 0.020352935418486595, 0.00372134312056005, -0.010039124637842178, 0.011595679447054863, 0.015395504422485828, 0.026121344417333603, 0.0022776059340685606, 0.0036199709866195917, -0.010071825236082077, 0.008652613498270512, 0.004401518497616053, -0.02258966490626335, 0.003544759238138795, 0.01854785531759262, 0.01670353300869465, -0.026840759441256523, 0.009457051753997803, -0.01225623395293951, -0.011399474926292896, -0.0077958544716238976, 0.009587854146957397, -0.017802277579903603, -0.021961810067296028, 0.01908414624631405, 0.01880946010351181, 0.00333383958786726, -0.00784817524254322, 0.011641460470855236, -0.016481168568134308, 0.01155643817037344, -0.000591474468819797, 0.022812029346823692, 0.017213664948940277, 0.005814190022647381, 0.0015320292441174388, 0.012472058646380901, 0.009993343614041805, 0.03045092150568962, -0.011079007759690285, 0.008240584284067154, 0.0021909489296376705, -0.013269956223666668, -0.01379970833659172, -0.013034511357545853, -0.02718084678053856, 0.03291001543402672, 0.011085547506809235, 0.02521880343556404, -0.029169052839279175, 0.01404823362827301, -0.020052088424563408, -0.0023217517882585526, 0.00589921185746789, -0.005967883393168449, 0.009751358069479465, -0.007854715920984745, 0.006389722693711519, -0.03152350336313248, 0.01085664238780737, -0.007004496641457081, 0.008992700837552547, 0.0036134307738393545, -0.026971563696861267, -0.0030591534450650215, 0.015513226389884949, 0.010438073426485062, -0.0037802045699208975, -0.0002462773700244725, -0.015840234234929085, -0.023008234798908234, 0.004136642441153526, 0.009365489706397057, 0.0065270657651126385, -0.010496934875845909, -0.014859212562441826, 0.007102598901838064, -0.01344654057174921, -0.017252905294299126, 0.00399275915697217, -0.008960000239312649, -0.03926703706383705, -0.023165198042988777, 0.029142891988158226, 0.0001587007282068953, -0.00047702190931886435, -0.007135299500077963, -0.01595795713365078, -0.005026102531701326, -0.019411154091358185, 0.008122861385345459, -0.01598411798477173, -0.022498102858662605, -0.005441401619464159, 0.02355760708451271, -0.008279824629426003, 0.012458978220820427, 0.024891795590519905, 0.003610160667449236, -0.01858709566295147, -0.01298219058662653, -0.002843328518792987, -0.018403971567749977, 0.036834102123975754, -0.006638248451054096, 0.03045092150568962, 0.009685956872999668, 0.011392934247851372, -0.007488467264920473, 0.0013554452452808619, 0.0029070950113236904, 0.0019195328932255507, -0.028515037149190903, 0.007298802956938744, 0.007540788501501083, -0.014414481818675995, 0.03837757930159569, 0.008861898444592953, -0.017069781199097633, 0.024120058864355087, -0.0069063943810760975, 0.03636321425437927, -0.014819971285760403, 0.00019395620620343834, -0.01598411798477173, -0.004290335811674595, 0.020287534222006798, 0.014584526419639587, 0.004594452679157257, 0.01689973846077919, 0.04899877682328224, -0.016481168568134308, 0.02599054016172886, -0.01641576737165451, -0.008188262581825256, -0.01739678904414177, 0.011026686057448387, -0.02950914017856121, 0.02094154804944992, 0.010863183066248894, 0.023152116686105728, -0.016742773354053497, -0.005405430682003498, -0.00673635071143508, 0.006932554766535759, 0.023452963680028915, -0.018691737204790115, 0.025846658274531364, -0.0356568768620491, 0.0031539855990558863, -0.010680058971047401, -0.002248175209388137, -0.008508729748427868, -0.004120292142033577, -0.01022878848016262, 0.022040292620658875, 0.01165454089641571, 0.008848818019032478, 0.02809646725654602, -0.03243912383913994, -0.021072350442409515, -0.002400233643129468, -0.027076205238699913, 0.008410627953708172, -0.0034924380015581846, -0.005984233692288399, 0.0010202628327533603, 0.01760607399046421, 0.014492964372038841, -0.01880946010351181, -0.029352176934480667, 0.04672280326485634, 0.032255999743938446, -0.024891795590519905, 0.023623008280992508, -0.02308671548962593, 0.03152350336313248, 0.027913343161344528, 0.002375708194449544, 0.026016701012849808, -0.024407826364040375, 0.012334715574979782, -0.004807007499039173, 0.009921401739120483, -0.027704060077667236, -0.011464876122772694, -0.01047731377184391, -0.020889226347208023, 0.028462715446949005, 0.022511182352900505, -0.0015238540945574641, 0.00047211680794134736, 0.008462948724627495, -0.0050849635154008865, 0.0148853724822402, -0.022537343204021454, 0.002784467302262783, -0.002269430784508586, 0.006317781284451485, -0.01760607399046421, 0.002864584093913436, -0.020562220364809036, 0.009790598414838314, -0.00941781047731638, -0.0030035621020942926, 0.006239299662411213, 0.01466300804167986, 0.010830482468008995, 0.011111708357930183, 0.0101241460070014, -0.017985401675105095, -0.03288385644555092, 0.022772789001464844, 0.011876905336976051, -0.022458862513303757, -0.026971563696861267, -0.029456818476319313, -0.024237781763076782, -0.014623766764998436, 0.03780204430222511, 0.002068321220576763, 0.02592513896524906, 0.001957138767465949, -0.0013399124145507812, -0.011988087557256222, 0.01106592733412981, 0.014022073708474636, 0.008724555373191833, 0.019646599888801575, -0.027965664863586426, -0.0009671241277828813, -0.014087474904954433, -0.003140905173495412, -0.03937168046832085, 0.011092088185250759, -0.0017265985952690244, 0.015918714925646782, -0.025598132982850075, 0.013969752006232738, -0.0055460440926253796, -0.009901781566441059, -0.009908321313560009, -0.017082862555980682, 0.0030280877836048603, -0.016742773354053497, -0.005696467123925686, -0.015539387241005898, 0.0006401168066076934, 0.010071825236082077, -0.0026144233997911215, 0.0011984817683696747, -0.03045092150568962, -0.016572730615735054, -0.0025441169273108244, 0.10736303776502609, 0.03238680213689804, 0.0078023942187428474, 0.03045092150568962, 0.00031617519562132657, -0.02235421910881996, -0.011275212280452251, -0.025872819125652313, 0.011582599021494389, 0.009188905358314514, 0.0075015476904809475, 0.0010889343684539199, 0.005905752070248127, -0.0003386569442227483, 0.013511941768229008, 0.016494248062372208, 0.010346511378884315, 0.015670189633965492, -0.0006842627772130072, -0.029639942571520805, -0.0026700147427618504, 0.00369191262871027, 0.02885512448847294, 0.025061840191483498, -0.01860017515718937, -0.030320117250084877, 0.007436146028339863, 0.001064408803358674, -0.006854073144495487, 0.01379970833659172, -0.026343708857893944, -0.007449226453900337, -0.007782774046063423, -0.008960000239312649, -0.003358365036547184, 0.016507329419255257, 0.018992584198713303, 0.03024163655936718, 0.05313214659690857, -0.009672876447439194, -0.005974423605948687, -0.003001927165314555, 0.0035185986198484898, -0.0178676787763834, 0.005886131431907415, -0.02736397087574005, -0.004924729932099581, 0.02551965042948723, 0.03549991175532341, -0.012753285467624664, 0.03479357808828354, 0.010307270102202892, -0.02694540284574032, 0.001117547508329153, 0.027285490185022354, -0.009306628257036209, -0.002506511053070426, 0.0007181898108683527, -0.0020454307086765766, 0.02664455585181713, -0.010418453253805637, -0.03160198777914047, 0.007357664406299591, -0.003590540261939168, 0.00369191262871027, -0.04980975389480591, -0.0014412846649065614, -0.03220367804169655, -0.0038848468102514744, -0.015251620672643185, -0.013786627911031246, -0.02733781188726425, -0.03165430575609207, -0.030110832303762436, 0.012485139071941376, 0.004244554787874222, 0.004892029333859682, -0.007043737452477217, -0.008999241515994072, 0.029221372678875923, -0.019201869145035744, -0.02406773716211319, -0.0036722919903695583, -0.01670353300869465, 0.003288058564066887, 0.02449938841164112, -0.0011584233725443482, 0.0019538686610758305, -0.0035120584070682526, 0.020784584805369377, 0.006697109900414944, 0.02669687755405903, 0.010202627629041672, -0.019594278186559677, 0.027076205238699913, -0.003312584012746811, 0.017265986651182175, 0.0226158257573843, 0.01404823362827301, -0.02528420463204384, 0.010431532748043537, -0.028253432363271713, -0.006563036702573299, -0.022772789001464844, -0.006628438364714384, -0.013616584241390228, -0.01357734389603138, 0.01619340106844902, -0.01619340106844902, -0.012328175827860832, -0.024224702268838882, -0.02238037995994091, -0.012223533354699612, -0.004493080545216799, 0.012210452929139137, 0.012864467687904835, -0.004918189719319344, 0.0427987165749073, -0.0030787738505750895, 0.0016489343252032995, 0.013191474601626396, -0.02621290646493435, 0.010156846605241299, 0.012177752330899239, -0.007534248288720846, -0.013865109533071518, 0.015774833038449287, -0.043975941836833954, 0.010876263491809368, 0.002223649760708213, -0.012884087860584259, 0.033459387719631195, 0.0023887883871793747, -0.016036437824368477, -0.026069022715091705, -0.012347796000540257, -0.0190449059009552, 0.03071252629160881, -0.01809004507958889, -0.025794336572289467, -0.007612730376422405, -0.011687241494655609, 0.005745518486946821, -0.018233926966786385, 0.017802277579903603, -0.025912059471011162, -0.015853313729166985, -0.022223416715860367, -0.0014429197181016207, 0.011392934247851372, -0.012321635149419308, 0.0031392702367156744, -0.003937168046832085, -0.03000619076192379, 0.006703649647533894, -0.046094950288534164, -0.002777927089482546, 0.00033599999733269215, 0.011249051429331303, 0.011406014673411846, 0.02759941667318344, 0.00559836532920599, 0.01781535893678665, 0.01403515413403511, -0.010627737268805504, 0.023034395650029182, -0.01179188396781683, 0.004846248310059309, -0.03555223345756531, 0.037488117814064026, 0.028384234756231308, -0.00025710949557833374, 0.017566831782460213, -0.01165454089641571, 0.015251620672643185, 0.01976432092487812, 0.020549139007925987, 0.012406657449901104, -0.02092846855521202, -0.037304993718862534, -0.014349080622196198, 0.0015230365097522736, -0.01347270142287016, -0.007455766666680574, -0.014375241473317146, -0.024604029953479767, 0.002305401489138603, 0.00023564964067190886, -0.009365489706397057, -0.019136467948555946, 0.04091515392065048, -0.0013824234483763576, 0.0012303650146350265, -0.004139912314713001, 0.004957430996000767, 0.01022878848016262, -0.0014968760078772902, -0.0032749781385064125, 0.005742248147726059, -0.005523153580725193, 0.008449869230389595, 0.012001167982816696, -0.001093839411623776, 0.011811504140496254, 0.0018786570290103555, 0.01955503784120083, -0.004257635213434696, -0.008305985480546951, 0.0025751825887709856, 0.000532204401679337, -0.0029430657159537077, -0.00808362103998661, -0.018220847472548485, -0.03424420580267906, 0.0008780146017670631, -0.013642745092511177, -0.008946919813752174, 0.00917582493275404, 0.006978335790336132, -0.01877021975815296, 0.009313168004155159, -0.0001295766414841637, 0.05195492133498192, 0.020666861906647682, 0.023897694423794746, 0.009757897816598415, -0.004771036561578512, -0.0031196498312056065, 0.007011036854237318, 0.0031703358981758356, 0.009574773721396923, 0.02308671548962593, 0.020562220364809036, 0.0075865695253014565, 0.005052262917160988, -0.01621956191956997, -0.005961343180388212, -0.0334070660173893, -0.01534318272024393, 0.0018590365070849657, 0.027573255822062492, 0.00916928518563509, -0.011504117399454117, -0.02807030826807022, 0.008404088206589222, 0.037043388932943344, -0.003065693425014615, 0.007815474644303322, -0.021857168525457382, -0.03861302137374878, -0.023296000435948372, -0.01810312457382679, -0.004702365025877953, 0.022550424560904503, -0.0034172264859080315, -0.024826394394040108, 0.016716614365577698, -0.021543242037296295, 0.006422423757612705, -0.0148853724822402, -0.013969752006232738, 0.027573255822062492, -0.006553226616233587, 0.011157489381730556, 0.015160058625042439, -0.002540846820920706, -0.0028220731765031815, 0.002995386952534318, 0.012714044190943241, 0.025323446840047836, -0.022668147459626198, 0.007115678861737251, 0.008384467102587223, 0.00738382525742054, 0.016141081228852272, -0.004381897859275341, -0.022275738418102264, -0.0357615202665329, 0.0017674744594842196, 0.007213781122118235, 0.01807696372270584, 0.006749430671334267, -0.009888701140880585, -0.008737634867429733, -0.01788076013326645, -0.00024464234593324363, -0.025166481733322144, -0.01226277370005846, -0.011746102944016457, -0.016036437824368477, -0.02830575220286846, -0.026539912447333336, 0.014846132136881351, 0.018233926966786385, -0.00168408767785877, 0.00941781047731638, 0.027050044387578964, 0.04905109480023384, -0.027520935982465744, -0.010451153852045536, -0.009725197218358517, 0.022275738418102264, -0.02026137337088585, 0.005258277524262667, 0.0011862190440297127, -0.016585810109972954, 0.03497670218348503, -0.010287649929523468, -0.002465635072439909, -0.00798551831394434, -0.0032390074338763952, -0.0062752701342105865, -0.0017020730301737785, -0.01760607399046421, -0.002083036582916975, 0.013250336050987244, -0.013734307140111923, -0.007233401760458946, -0.003227562177926302, 0.011752642691135406, -0.018233926966786385, 0.012622482143342495, 0.010666978545486927, 0.001635854016058147, 0.012105810455977917, -0.005163445603102446, 0.004391707945615053, -0.01641576737165451, -0.02979690581560135, -0.013995912857353687, 0.0040614306926727295, 0.01557862851768732, 0.001187854097224772, 0.00878995656967163, -0.020169811323285103, -0.014754570089280605, -0.008135941810905933, 0.020143650472164154, -0.007514628116041422, 0.0005816642660647631, 0.014519124291837215, 0.03021547570824623, -0.00906464271247387, 0.007776233833283186, 0.008430248126387596, 0.010653898119926453, -0.018194686621427536, -0.018443211913108826, -0.027233168482780457, -0.0356307178735733, -0.010052205063402653, 0.018011562526226044, 0.01357734389603138, -0.015892555937170982, -0.014375241473317146, -0.009685956872999668, -0.016114920377731323, 0.003227562177926302, 0.0010055474704131484, 0.01645500771701336, 0.03950248286128044, -0.0003386569442227483, -0.0027059854473918676, 0.008750715292990208, 0.000965489074587822, 0.01812928542494774, -0.009790598414838314, -0.006160817574709654, 0.017684554681181908, -0.017252905294299126, 0.0009115329012274742, 0.014257518574595451, -0.01952887699007988, 0.0010227153543382883, 0.004937810357660055, 0.004182423464953899, -0.006272000260651112, 0.016795095056295395, -0.01931959204375744, -0.0028809343930333853, 0.010156846605241299, 0.007893956266343594, 0.006265460047870874, -0.004110482055693865, -0.02194873057305813, -0.025781257078051567, 0.007266102358698845, 0.012871007435023785, 0.009254306554794312, -0.0017020730301737785, -0.02308671548962593, 0.03476741537451744, -0.01621956191956997, 0.00966633576899767, -0.023191358894109726, -0.03604928404092789, 0.001555737224407494, -0.008874977938830853, 0.011046307161450386, 0.017069781199097633, 0.014924613758921623, 0.014061314053833485, 0.008626452647149563, -0.025585051625967026, -0.004676204640418291, 0.005127474665641785, -0.004169343039393425, -0.011242511682212353, -0.007560409139841795, -0.010869722813367844, 0.0032945985440164804, -0.0056310659274458885, 0.02120315469801426, -0.014689167961478233, 0.0038652264047414064, -0.012066570110619068, 0.0050359126180410385, -0.028985928744077682, 0.02480023354291916, 0.0015287591377273202, -0.01382586918771267, -0.00820134300738573, -0.006697109900414944, -0.0031310950871556997, -0.007128759287297726, -0.006448584143072367, 0.011562978848814964, -0.012517839670181274, -0.007246482186019421, 0.004800467286258936, -0.00540870102122426, -0.028724322095513344, 0.009672876447439194, -0.01831240952014923, -0.015186219476163387, 0.029404496774077415, 0.2164003551006317, 0.019398072734475136, -0.015787912532687187, 0.02193565107882023, 0.004630423616617918, 0.027390131726861, 0.03243912383913994, 0.0043230364099144936, -0.007213781122118235, 0.009365489706397057, 0.005915562156587839, 0.021870248019695282, 0.017239825800061226, 0.0007799124578014016, -0.007279182784259319, -0.035709198564291, -0.008266745135188103, -0.01574867218732834, -0.0014788905391469598, -0.038220614194869995, -0.009332788176834583, 0.003646131604909897, -0.026762278750538826, -0.014702248387038708, 0.026003621518611908, -0.0004108029243070632, 0.01662505231797695, -0.020771503448486328, 0.00620005838572979, 0.013564263470470905, 0.0013399124145507812, -0.04067970812320709, 0.007429606281220913, -0.010778160765767097, -0.01712210290133953, -0.007298802956938744, -0.006641518324613571, -0.012766364961862564, 0.019371913745999336, 0.002719065872952342, -0.0026356789749115705, -0.009195445105433464, 0.003201401559635997, 0.004342657048255205, 0.002818803070113063, 0.007449226453900337, -0.005784759297966957, -0.0009802044369280338, -0.0049508907832205296, -0.005732438061386347, -0.024590950459241867, -0.01085664238780737, 0.015552467666566372, 0.04578102380037308, -0.011020146310329437, 0.011517197825014591, 0.0006262189708650112, -0.010948204435408115, -0.0008943650173023343, -0.0015025986358523369, -0.004326306749135256, 0.03068636544048786, -0.026618395000696182, 0.00846948940306902, -0.0018802920822054148, 0.02622598595917225, -0.032072875648736954, 0.008986161090433598, 0.008868438191711903, -0.014008993282914162, -0.01344654057174921, -0.014453723095357418, -0.01662505231797695, -0.004558481741696596, 0.001986569492146373, 0.009221605956554413, 0.010065284557640553, 0.014322919771075249, 0.01884870044887066, 0.012557080946862698, -0.018286248669028282, -0.01298873033374548, -0.005461022257804871, -0.011131328530609608, -0.023466045036911964, -0.02427702210843563, 0.005833810195326805, -0.021137751638889313, 0.0045257811434566975, -0.016507329419255257, 0.004182423464953899, -0.03000619076192379, -0.008953460492193699, 0.007429606281220913, 0.006965255830436945, 0.014336000196635723, 0.023439884185791016, 0.04190925508737564, -0.013420379720628262, -0.0024018685799092054, -0.03353786841034889, -0.00998680293560028, 0.012458978220820427, 0.0007145109702832997, -0.018887942656874657, -0.01663813181221485, -0.012661723420023918, 0.026814598590135574, 0.016036437824368477, -0.009816759265959263, 0.008063999935984612, 0.009195445105433464, 0.017213664948940277, -0.017305226996541023, -0.0006294890772551298, 0.006170628126710653, 0.008966539986431599, -0.01784151792526245, 0.020836906507611275, -0.03693874552845955, 0.001570452586747706, -0.011229431256651878, -0.019816642627120018, -0.008855357766151428, 0.005006481893360615, -0.011948847211897373, 0.00018894890672527254, 0.004411328583955765, 0.00012375182996038347, -0.017436029389500618, 0.004878948908299208, -0.012812145985662937, 0.010333430953323841, -0.002864584093913436, -0.0007157372310757637, 0.007233401760458946, 0.012942949309945107, 0.015591708943247795, -0.010045664384961128, 0.027468614280223846, 0.01239357702434063, -0.01026148907840252, 0.0020650511141866446, 0.006464934442192316, 0.003727883333340287, -0.031104935333132744, 0.021556321531534195, -0.008417167700827122, -0.002374073024839163, -0.022511182352900505, -0.009509372524917126, -0.039450161159038544, -0.001117547508329153, -0.017946161329746246, 0.003747503738850355, -0.045833345502614975, -0.022432701662182808, -0.03073868714272976, -0.01688665710389614, 0.021320875734090805, -0.00500321201980114, 0.02478715404868126, 0.009339328855276108, -0.0006818102556280792, -0.02045757696032524, 0.000785226293373853, -0.16763702034950256, 0.0033354745246469975, 0.009352409280836582, -0.0072007011622190475, 0.02617366425693035, -0.0004385985666885972, 0.012478599324822426, 0.032543767243623734, -0.011092088185250759, -0.0011968467151746154, 0.00987562071532011, 0.012799066491425037, -0.027520935982465744, 0.0038554160855710506, -0.003242277540266514, 0.001795270130969584, 0.01806388422846794, 0.007063358090817928, 0.011641460470855236, 0.0030885839369148016, 0.009725197218358517, 0.007455766666680574, 0.01415287610143423, 0.0039306278340518475, 0.012622482143342495, -0.014741489663720131, 0.027547094970941544, 0.013250336050987244, 0.003544759238138795, -0.005510073155164719, -0.009090803563594818, -0.0010799416340887547, 0.0021860438864678144, 0.024120058864355087, 0.009201985783874989, 0.009803678840398788, 0.0025604672264307737, -0.011543357744812965, -0.023910773918032646, 0.009260847233235836, 0.013564263470470905, -0.004247825127094984, 0.009901781566441059, -0.016311123967170715, 0.014231357723474503, 0.03432268649339676, 0.01476765051484108, 0.00977097824215889, 0.024486307054758072, -0.007351124193519354, 0.018874861299991608, -0.03215136006474495, -0.0032210219651460648, -0.014571445994079113, 0.027704060077667236, 0.021974891424179077, 0.0022972263395786285, 0.005294248461723328, -0.004110482055693865, 0.00220729922875762, -0.030660204589366913, 0.0006115036667324603, 0.004283795598894358, -0.0020912117324769497, -0.014924613758921623, -0.022066453471779823, 0.0053334892727434635, 0.00809016078710556, -0.01716134324669838, 0.009228146634995937, -0.009025401435792446, 0.010536175221204758, -0.010084905661642551, -0.005722627975046635, 0.0071680000983178616, 0.020091328769922256, -0.01903182454407215, 0.022812029346823692, 0.01574867218732834, 0.016795095056295395, -0.006638248451054096, 0.03842989727854729, -0.010660437867045403, 0.007056817878037691, 0.027573255822062492, 0.0021255475003272295, -0.008737634867429733, -0.025585051625967026, 0.005163445603102446, -0.008273284882307053, -0.006997956428676844, -0.02956146001815796, -0.021765606477856636, -0.013289577327668667, 0.004663124214857817, 0.016088759526610374, 0.009757897816598415, 0.017920000478625298, 0.016363445669412613, -0.01831240952014923, -0.0032504526898264885, -0.007573489099740982, -0.0048691388219594955, 0.008547971025109291, 0.022105693817138672, 0.02733781188726425, 0.02498335763812065, 0.0007770511438138783, 0.01226277370005846, -0.011948847211897373, 0.006039824802428484, 0.000842043838929385, 0.024185460060834885, 0.03911007568240166, -0.0009736642823554575, 0.0018770219758152962, 0.010980905033648014, -0.00540870102122426, 0.009201985783874989, 0.0022612556349486113, 0.050882335752248764, 0.007972437888383865, -0.020771503448486328, -0.0014911533799022436, -0.007115678861737251, -0.027677899226546288, -0.08392315357923508, -0.011353693902492523, -0.01620648242533207, 0.019685840234160423, -0.02019597217440605, 0.01975124143064022, -0.005895941983908415, 0.006814832333475351, 0.004159532953053713, 0.037749722599983215, -0.004581372253596783, -0.03542143106460571, 0.004231474362313747, -0.007455766666680574, 0.002687999978661537, -0.009234686382114887, -0.03314546123147011, -0.014492964372038841, -0.011634919792413712, 0.010738920420408249, -0.02950914017856121, -0.007972437888383865, -0.014414481818675995, -0.02214493416249752, -0.004963970743119717, -0.007691211998462677, -0.024250861257314682, 0.004205313976854086, 0.008391007781028748, 0.03045092150568962, -0.01759299263358116, -0.024002335965633392, 0.0018574015703052282, -0.037723563611507416, 0.021373197436332703, -0.009221605956554413, -0.01224969420582056, -0.007612730376422405, 0.02141243778169155, -0.04366201534867287, 0.008168642409145832, 0.02405465766787529, 0.007861255668103695, -0.025781257078051567, 0.011504117399454117, -0.007625810336321592, -0.021621722728013992, 0.029613781720399857, 0.01763223484158516, -0.03136654198169708, -0.032491445541381836, -0.007560409139841795, -0.027233168482780457, -0.011366774328052998, 0.030607884749770164, -0.0030051972717046738, 0.018273169174790382, -0.0065728467889130116, -0.008456408977508545, 0.002208934398368001, 0.0014036789070814848, 0.008116321638226509, -0.012779445387423038, 0.007893956266343594, 0.0067755915224552155, -0.020052088424563408, 0.0061804382130503654, 0.014623766764998436, 0.02383229322731495, -0.0130933728069067, -0.02070610225200653, 0.006281810346990824, -0.009443971328437328, -0.007161459885537624, -0.024420905858278275, -0.006912934593856335, -0.026343708857893944, -0.0018099854933097959, 0.012949489057064056, -0.0357876792550087, -0.010725839994847775, -0.011759182438254356, 0.03288385644555092, -0.017279066145420074, 0.003080408787354827, 0.007808934431523085, 0.0032406423706561327, 0.006648058537393808, 0.004411328583955765, 0.0008649343508295715, -0.01836473122239113, 0.017710715532302856, -0.00261769350618124, -0.013721226714551449, -0.029953869059681892, 0.02383229322731495, -0.008711474947631359, -0.0020536058582365513, 0.006242569535970688, 0.02622598595917225, -0.016782015562057495, -0.0034074161667376757, -0.06001238152384758, 0.011975008063018322, 0.00673635071143508, -0.022498102858662605, 0.005320408847182989, 0.0032651680521667004, 0.03761892020702362, -0.021503999829292297, 0.011334073729813099, 0.017671475186944008, -0.005045722704380751, 0.016311123967170715, 0.01928035169839859, 0.004633693490177393, -0.01225623395293951, -0.02572893537580967, -0.005290978122502565, -0.017422949895262718, 0.0012050219811499119, 0.002128817606717348, -0.015447825193405151, -0.036310892552137375, 0.015800993889570236, 0.009587854146957397, -0.006033285055309534, -0.006729810498654842, -0.005487182643264532, 0.01025494933128357, -0.04217086359858513, -0.0017674744594842196, 0.008057460188865662, -0.00018905110482592136, 0.007887416519224644, 0.013511941768229008, 0.017305226996541023, -0.0024296643678098917, -0.012086190283298492, 0.022785868495702744, 0.01621956191956997, 0.02670995704829693, 0.005614715628325939, -0.010398832149803638, 0.01810312457382679, -0.004947620443999767, 0.00903848186135292, 0.016847416758537292, -0.03026779741048813, -0.008286365307867527, 0.03280537202954292, 0.0016701897839084268, 0.03165430575609207, 0.00577821908518672, -0.02019597217440605, -0.034453488886356354, -0.007828555069863796, 0.006801751907914877, 0.023439884185791016, -0.014597605913877487, -0.007861255668103695, 0.0030133724212646484, 0.026239067316055298, 0.006249109748750925, -0.001480525592342019, 0.009535533376038074, 0.012942949309945107, 0.02546732872724533, -0.003217751858755946, 0.007599649950861931, 0.0005305693484842777, -0.035866159945726395, 0.016795095056295395, 0.004349197261035442, 0.0056670368649065495, 0.0017642044695094228, 0.013328817673027515, 0.01082394178956747, 0.000770510989241302, -0.017959240823984146, -0.005068613216280937, 0.005922102369368076, 0.022079532966017723, -0.015526306815445423, -0.034191884100437164, 0.005807649809867144, 0.04005185514688492, 0.008077080361545086, -0.004100671503692865, 0.019123386591672897, 0.001824700739234686, 0.013551183044910431, -0.01107246708124876, 0.01998668722808361, 0.014401402324438095, 0.00214353296905756, 0.0015025986358523369, 0.0017102481797337532, -0.01810312457382679, -0.00434592692181468, 0.007298802956938744, 0.025781257078051567, 0.00952899269759655, -0.0023381023202091455, -0.004584642592817545, -0.030058512464165688, -0.0009752992773428559, 0.0035545695573091507, -0.005768408998847008, -0.03073868714272976, -0.0007038832409307361, 0.013041051104664803, 0.01975124143064022, 0.010686598718166351, -0.02285127155482769, 0.012563620693981647, -0.03259608894586563, -0.005817459896206856, 0.020313693210482597, -0.0006372554926201701, -0.010725839994847775, 0.032543767243623734, -0.005068613216280937, 0.00079013139475137, 0.023518364876508713, -0.0057978397235274315, 0.022001052275300026, 0.0022727008908987045, 0.024682512506842613, -0.019960526376962662, -0.02312595769762993, 0.010189548134803772, 0.0015827154275029898, 0.01359042339026928, -0.020182890817523003, 0.0031310950871556997, -0.003384525654837489, -0.020653782412409782, -0.007828555069863796, 0.022275738418102264, -0.026788439601659775, 0.05140554904937744, 0.003551299450919032, 0.019188789650797844, -0.01239357702434063, -0.022288817912340164, -0.00019835037528537214, 0.008057460188865662, 0.01141909509897232, -0.027442453429102898, -0.010143767111003399, 0.010516555048525333, 0.004578102380037308, 0.005199416074901819, 0.002365897875279188, -0.02354452572762966, 0.004826627671718597, -0.018247008323669434, 0.00843678880482912, -0.0029185402672737837, -0.016023358330130577, 0.007239941973239183, 0.0027108904905617237, 0.005670306738466024, -0.011366774328052998, -0.0008412263123318553, 0.008266745135188103, 0.035107504576444626, 0.00035316788125783205, -0.0018443212611600757, -0.03696490451693535, -0.0069521754048764706, -0.0019048175308853388, -0.006569576915353537, -0.015591708943247795, 0.03283153474330902, -0.006455124355852604, -0.0005694014835171402, -0.006971796043217182, -0.009430890902876854, 0.013420379720628262, 0.008129402063786983, 0.014375241473317146, -0.017331387847661972, -0.015853313729166985, 0.018678657710552216, -0.008711474947631359, -0.0154739860445261, -0.026788439601659775, -0.030634045600891113]}}