Saravanan666/pypi-org huggingface dataset:
Package: elasticsearch-opentracing
This package enables distributed tracing for the Python elasticsearch-py and elasticsearch-dsl libraries.InstalationRun the following command:$ pip install elasticsearch_opentracingGetting startedPlease see the examples directory. Overall, usage requires that a tracer gets set, and initialize the Elasticsearch client specifyingTracingTransportas tracing class, and optionally set an active span (to be used as parent span when tracing the actual Elasticsearch statements):importelasticsearch_opentracingelasticsearch_opentracing.init_tracing(tracer)# An OpenTracing compatible tracer.es=Elasticsearch(transport_class=elasticsearch_opentracing.TracingTransport)elasticsearch_opentracing.set_active_span(main_span)# Optional.es.index(index='test-index',doc_type='tweet',id=99,body={'author':'linus','text':'Hello there','timestamp':datetime.now(),})res=es.get(index='test-index',doc_type='tweet',id=99)elasticsearch_opentracing.clear_active_span()By default, all Elasticsearch requests are traced. It’s possible to have it set to false when initializing the library, and callenable_tracinganddisable_tracingto explicitly trace statements happening within that section:elasticsearch_opentracing.init_tracing(tracer,trace_all_requests=False)elasticsearch_opentracing.enable_tracing()res1=es.get(index='test-index',doc_type='tweet',id=99)res2=es.get(index='test-index',doc_type='user',id=666)elasticsearch_opentracing.disable_tracing()When usingtrace_all_requests, any calls made toenable_tracinganddisable_tracingare ignored.In case of an exception happening under this block, an implicit call todisable_tracingwill take place, with the request causing the error including error information with it.DSLWhen using theelasticsearch-dsllibrary (which runs on top ofelasticsearch-py), the same semantics and calls are used. When creating a default connection, the transport can be specified as well:# elasticsearch_dsl.connections.connectionsconnections.create_connection(hosts=['127.0.0.1'],transport_class=elasticsearch_opentracing.TracingTransport)MultithreadingTracing and parent span data is kept as thread local data, which means that applications using many threads (Django, Flask, Pyramid, etc) will work just fine.Further informationIf you’re interested in learning more about the OpenTracing standard, please visitopentracing.ioorjoin the mailing list. If you would like to implement OpenTracing in your project and need help, feel free to send us a note atcommunity@opentracing.io.
Package: elasticsearch_parse
#elasticsearch_parse=================首先注明下本项目是由来,学习Elasticsearch本身是有些痛苦的, 我们可以熟练的编写SQL 、 Mongodb语句，但对于Elasticsearch Dsl语法总是蒙头，一个劲的翻找笔记...简单说 Elasticsearch Parse可以让你更容易的上手ES, 他的功能主要是语句映射, 有点ORM的意思...此项目继承于`elasticsearch-dsl-py`,本来是想fork子项目,开发好后提交pull request, 但是想到我这语法映射的功能, 对于官方来说不是很稀罕,索性直接砍掉了 70% 代码,然后中间又加了一些佐料, 最终只是为了DSL语句映射功能 !模块安装方法:```pip install elasticsearch_parse```下面我们来体验下封装后的es语法解释器.首先用原始的DSL语法操作,一眼望去会有些麻烦, 手写起来会更麻烦.```from elasticsearch import Elasticsearchclient = Elasticsearch()response = client.search(index="my-index",body={"query": {"filtered": {"query": {"bool": {"must": [{"match": {"title": "python"}}],"must_not": [{"match": {"description": "beta"}}]}},"filter": {"term": {"category": "search"}}}},"aggs" : {"per_tag": {"terms": {"field": "tags"},"aggs": {"max_lines": {"max": {"field": "lines"}}}}}})```下面是使用Elasticsearch_parse的用法,要多简单就多简单```from elasticsearch_parse import Search, Qs = Search(index="my-index") \.filter("term", blog="xiaorui.cc") \.query("match", author="ruifengyun")   \.query(~Q("match", face="good"))s.aggs.bucket('per_tag', 'terms', field='tags')response = s.execute()```我们得到的结果是:```{"query": {"filtered": {"filter": {"term": {"blog": "xiaorui.cc"}},"query": {"bool": {"must_not": [{"match": {"face": "good"}}],"must": [{"match": {"author": "ruifengyun"}}]}}}},"aggs": {"per_tag": {"terms": {"field": "tags"}}}}```我们把语法的用法给过一遍.```s = search.Search()```通过match查询,f字段值为55的数据```s.query('match', f=55)```时间范围```s.query('range', ** {'@timestamp': {'lt': 'now'}})```外围的size的控制```s = s.query('match', f=42)s[3].to_dict() {'query': {'match_all': {}}, 'from': 3, 'size': 1}``````assert s.to_dict(size=10) == {"query": {"match": {'f': 42}}, "size": 10}```嵌入内部size控制```s = search.Search.from_dict({"size": 5})assert {"query": {"match_all": {}},"size": 5} == s.to_dict()```对于aggs的聚合的使用```s = s.query('match', f=42)assert {"query": {"match": {'f': 42}}} == s.to_dict()assert {"query": {"match": {'f': 42}}, "size": 10} == s.to_dict(size=10)s.aggs.bucket('per_tag', 'terms', field='f').metric('max_score', 'max', field='score')d = {'aggs': {'per_tag': {'terms': {'field': 'f'},'aggs': {'max_score': {'max': {'field': 'score'}}}}}```...
Package: elasticsearch-pii-redacter
Did you find PII (Personally Identifiable Information) in your Elasticsearch
indices that doesn’t belong there? This is the tool for you!The elastic-pii-redacter can help you redact information from even Searchable
Snapshot mounted indices. It works with deeply nested fields, too!Client ConfigurationThe tool connects using thees_clientPython module.The configuration file structure requireselasticsearchat the root level:---
elasticsearch:
  client:
    hosts: https://10.11.12.13:9200
    cloud_id:
    bearer_auth:
    opaque_id:
    request_timeout: 60
    http_compress:
    verify_certs:
    ca_certs:
    client_cert:
    client_key:
    ssl_assert_hostname:
    ssl_assert_fingerprint:
    ssl_version:
  other_settings:
    master_only:
    skip_version_test:
    username:
    password:
    api_key:
      id:
      api_key:
      token:

logging:
  loglevel: INFO
  logfile: /path/to/file.log
  logformat: default
  blacklist: []REDACTIONS_FILEConfiguration---
redactions:
  - job_name_20230731_redact_hot:
      pattern: hot-*
      query: {'match': {'message': 'message1'}}
      fields: ['message']
      message: REDACTED
      expected_docs: 1
      restore_settings: {'index.routing.allocation.include._tier_preference': 'data_warm,data_hot,data_content'}
  - job_name_20230731_redact_cold:
      pattern: restored-cold-*
      query: {'match': {'nested.key': 'nested19'}}
      fields: ['nested.key']
      message: REDACTED
      expected_docs: 1
      restore_settings: {'index.routing.allocation.include._tier_preference': 'data_warm,data_hot,data_content'}
  - job_name_20230731_redact_frozen:
      pattern: partial-frozen-*
      query: {'range': {'number': {'gte': 8, 'lte': 11}}}
      fields: ['deep.l1.l2.l3']
      message: REDACTED
      expected_docs: 4
Package: elasticsearch-prometheus
Environment variablesELASTICSEARCH_QUERY_RESULTS_NAME
Name of prometheus gauge (default: ‘elasticsearch_query_results’)ELASTICSEARCH_QUERY_RESULTS_TEXT
Text of prometheus gauge (default: ‘Results from Elasticsearch query’)ES_QUERY_TIME_NAME
Name of total Elasticsearch query time summary (default: ‘es_query_time_seconds’)ES_QUERY_TIME_TEXT
Text of total Elasticsearch query time summary (default: ‘Time spent Querying Elasticsearchi’)ES_HOST_LIST
Comma-separated list of Elasticsearch endpoints (default: ‘http://elasticsearch.jamiehouse:9200’)ES_MAX_CONNECTIONS
Max connections (default: 25)ES_SEARCH_INDEX
Elasticsearch search index (default: tempsensor-*)ES_SEARCH_BODY
Elasticsearch search body (default: ‘{“query”:{“range”:{“@timestamp”:{“gt”:”now-15m”}}},”aggs”:{“avg_temp”:{“avg”:{“field”:”temp”}}}}’)ES_SEARCH_RESULT_FIELD_PATH
Dot-separated path to index result JSON for single-value to store in ELASTICSEARCH_QUERY_RESULTS gauge (default: ‘aggregations.avg_temp.value’)SLEEP_TIME
Seconds to sleep between queries (default: 10)
Package: elasticsearch-query
elasticsearch-queryRun queries against Kibana's Elasticsearch that gets logs from Logstash. Forked fromWikia'skibana.py.pip install elasticsearch-queryBasic Usagefromelasticsearch_queryimportElasticsearchQueryes_query=ElasticsearchQuery(es_host='es.prod',since=12345,period=900,index_prefix='logstash-my-app')es_hostneeds to be specified with a host of Elasticsearch instance to connect.Provide eithersince(absolute timestamp) orperiod(last N seconds):since: UNIX timestamp data should be fetched since (if None, then period specifies the last n seconds).period: period (in seconds) before now() to be used when since is empty (defaults to last 15 minutes).index_prefixargument will be used to build indices names to query in.
They should follow theindex-name-YYYY.MM.DDnaming convention, e.g.logstash-my-app-2014.08.19.get_rowsReturns data matching the given query (provided as adict).es_query.get_rows(match={"tags":'edge-cache-requestmessage'},limit=2000)match: query to be run against log messages (ex. {"@message": "Foo Bar DB queries"}).limit: the number of results (defaults to 10).query_by_stringReturns data matching the given query string (provided as aLucene query).es_query.query_by_string(query='@message:"^PHP Fatal"',limit=2000)es_query.query_by_string(query='@message:"^PHP Fatal"',fields=['@message','@es_query_host'],limit=2000)query: query string to be run against log messages (ex.@message:"^PHP Fatal").fields: optional list of fields to fetchlimit: the number of results (defaults to 10).query_by_sqlReturns data matching the givenSQL query.This featurerequiresnon-OSS versionof Elasticsearch ("To be clear, while the X-Pack source code is now available in the public repositories, it isn’t under an Open Source license").es_query.query_by_sql(sql='SELECT host FROM "app-requests" WHERE host =\'app2.prod\'')sql:SQL queryto be runcountReturns number of matching entrieses_query.count(query='@message:"^PHP Fatal"')Integration testselasticsearch-querycomes with integration tests suite..travis.ymlwill install elasticsearch OSS version and run them.But you can also run it locally. Simply passES_TEST_HOST=<elasticsearch IP>env variable when running tests (make test).
Package: elasticsearch-reindex
Elasticsearch ReindexDescriptionThis tool can be used to transfer Elasticsearch indexes between different servers.Installingpipinstallelasticsearch-reindexHow to useWhitelistsource_hostElasticsearch  indest_hostElasticsearch.You should edit Elasticsearch YML config:Config path:$ /etc/elasticsearch/elasticsearch.ymlAdd setting to the end of the file:$ reindex.remote.whitelist: <es-source-host>:<es-source-port>Use CLI for run migration data:$ elasticsearch_reindex \
    --source_host http(s)://es-source-host:es-source-port \
    --dest_host http(s)://es-dest-host:es-dest-port \
    --check_interval 5 \
    --concurrent_tasks 3 \
    -i test_index_1 -i test_index_2CLI Params description (example):Required fields:source_host- Elasticsearch endpoint where data will be extracted.dest_host- Elasticsearch endpoint where data will be transfered.Optional fields:check_interval- Time period (in second) to check task success status.Default value-10(seconds)concurrent_tasks- How many parallel task Elasticsearch will process.Default value-1(sync mode)indexes- List of user ES indexes to migrate instead of all source indexes.Run library from Python script:fromelasticsearch_reindeximportManagerdefmain()->None:"""Example reindex function."""config={"source_host":"http://localhost:9201","dest_host":"http://localhost:9202","check_interval":20,"concurrent_tasks":5,}manager=Manager.from_dict(data=config)manager.start_reindex()if__name__=="__main__":main()With custom user indexes:fromelasticsearch_reindeximportManagerdefmain()->None:"""Example reindex function."""config={"source_host":"http://localhost:9201","dest_host":"http://localhost:9202","check_interval":20,"concurrent_tasks":5,"indexes":["es-index-1","es-index-2","es-index-n"]}manager=Manager.from_dict(data=config)manager.start_reindex()if__name__=="__main__":main()Local installSetup and activate a python3 virtualenv via your preferred method. e.g. and install production requirements:$ make veTo remove virtualenv:$ make cleanTo install github hooks:$ make install_hooksCreate .env file and fill the data:$ cp .env.example .envExport env variables:$ export $(xargs < .env)Env variables description:Variable for enable testing:ENV- variable for enable testing mode.
For activate test mode set to value -test.Elasticsearch docker settings:ES_SOURCE_PORT- Source Elasticsearch portES_DEST_PORT- Destination Elasticsearch portES_VERSION- Elasticsearch versionLOCAL_IP- Address of you local host machine in LAN like192.168.4.106.You can find it in:Mac OS:$ ifconfigLinux (find it in response):$ ip rTestsFirstly up docker-compose services with 2 nodes of ElasticSearch:$ docker-compose up -dEnsure that Elasticsearch nodes started correctly.Env variables set from.envfile.For Source Elasticsearch:$ curl -X GET $LOCAL_IP:$ES_SOURCE_PORTFor destination Elasticsearch:$ curl -X GET $LOCAL_IP:$ES_DEST_PORTExport toPYTHONPATHenv variable:$ export PYTHONPATH="."For run tests withpytestuse:$ pytest ./tests
Package: elasticsearch-serverless
The official Python client for Elasticsearch Serverless.FeaturesTranslating basic Python data types to and from JSONConfigurable automatic discovery of cluster nodesPersistent connectionsLoad balancing (with pluggable selection strategy) across available nodesFailed connection penalization (time based - failed connections won’t be
retried until a timeout is reached)Support for TLS and HTTP authenticationThread safety across requestsPluggable architectureHelper functions for idiomatically using APIs togetherInstallationInstall theelasticsearch-serverlesspackage withpip:$ python -m pip install elasticsearch-serverlessIf your application uses async/await in Python you can install with
theasyncextra:$ python -m pip install elasticsearch-serverless[async]Read more abouthow to use asyncio with this project.CompatibilityTODODocumentationDocumentation for the client isavailable on elastic.coandRead the Docs.Quick Start# Import the client from the 'elasticsearch' module>>>fromelasticsearch_serverlessimportElasticsearch# Instantiate a client instance>>>client=Elasticsearch("http://localhost:9200")# Call an API, in this example `info()`>>>resp=client.info()# View the result>>>resp{"name":"instance-name","cluster_name":"cluster-name","cluster_uuid":"cluster-uuid","version":{"number":"7.14.0",...},"tagline":"You know, for Search"}You can read more aboutconfiguring the clientin the documentation.LicenseCopyright 2023 Elasticsearch B.V. Licensed under the Apache License, Version 2.0.
Package: elasticsearch-test-py
Start Elasticsearch using Python.Settingselasticsearch_test.ElasticsearchTest(
  port=1234
  es_path='/home/maria/elasticsearch-5.2.4'
  data_dir='/home/maria/es-data'
  es_java_opts='-Xms512m -Xmx512m'
)portDefault:9200es_pathPath to the Elasticsearch directory (home of/bin). Default: uses$ES_HOMEes_java_optsSets$ES_JAVA_OPTSfor process to this value if present.data_dirDirectory to place/dataand/logsin. Default: usestempfile.Runningimport time
import elasticsearch_test
instance = elasticsearch_test.ElasticsearchTest()

# This blocks until either an error is found
# or Elasticsearch has been initialized.

instance.start()

# or

instance.start(block=False)

while True:
    if instance.is_started():
        break
    time.sleep(1)TestingUse it as a resource::import elasticsearch_test
import elasticsearch

with elasticsearch_test.ElasticsearchTest() as instance:
    connection_info = instance.get_connection_info()
    client = elasticsearch.Elasticsearch(connection_info)Use it as a Pytest fixture::import elasticsearch_test

@pytest.fixture(scope="session")
def es():
    instance = elasticsearch_test.ElasticsearchTest()
    instance.start()
    yield instance
    instance.stop()When usingdata_dir, if there are preexisting data in the directory,
you should also wait for the data to be initialized::with elasticsearch_test.ElasticsearchTest(data_dir='my_data') as instance:
    while not instance.is_data_initialized():
        time.sleep(1)
    # do something!Test this packageNeeded:pyenvand the pluginpyenv-virtualenv.Create a normal virtualenv and activate itInstall extraspip install .[testing]Install the needed Python versions usingpyenv:3.8,3.9,3.10,3.11Setup virtualenvs for all but the Python version you are using, for example if you use3.8$ pyenv virtualenv -p python3.9 3.9.8 py39
$ pyenv virtualenv -p python3.10 3.10.8 py310
$ pyenv virtualenv -p python3.11 3.11.0 py311Setup path to Elasticsearch$ export ES_HOME=/opt/elasticsearch-6.8.23/Activate everything and runtox:$ pyenv shell py39 py310 py311
$ source ./venv/bin/activate
$ tox
Package: elasticsearch-tool
ht_es介绍python es的再封装主要借鉴SQLAlchemy的方式使用说明简单使用"""搜索测试工具,数据填充脚本"""importrandom,timefromelasticsearch_tool.elements.documentimportDoc,Fieldsfromelasticsearch_tool.base_element.operateimportdatetime_tool,NOT,ORfromdatetimeimportdatetimefromelasticsearch_tool.configimportConfig# 配置es服务器地址信息Config.set_host('localhost',port='9200')words=['单词','词汇','检索','我了','艾克','维护费','没理解','接是','咯怕','那么','行风','奶茶店','全网通','雨天','末尾','已收到','就好撒大家看法','是你们','你们','舞女','不','容易','一样','是你的','玩儿一天']definsert_indices():text=''word=[]id=int(time.time()*1000)num=random.randint(000,999)for_inrange(100):n=random.randint(0,len(words)-1)text+=words[n]for_inrange(5):n=random.randint(0,len(words)-1)word.append(words[n])return{'text':text,'word':word,'id':id,'num':num,'date':datetime.today(),'has_go':num%2==1,'height':time.time()}classDocTry(Doc):# 配置文档的索引值,类型值,以及文档id的字段__indices__='fifth'__types__='docs'__pk__='id'# 设定文档字段和类型text=Fields.Stringword=Fields.Listid=Fields.Integernum=Fields.Integerdate=Fields.Datetimehas_go=Fields.Booleanheight=Fields.Floatif__name__=='__main__':for_inrange(15):time.sleep(0.01)data=insert_indices()doc=DocTry()fork,vindata.items():doc.__setattr__(k,v)top=doc.text>'str'nop=doc.num>20doc.save()DocTry().search(text='检索').all()all_re=DocTry().filter(DocTry.num>50,DocTry.text=='单词',DocTry.text!='一样',OR(DocTry.text=='知识',DocTry.id>100,DocTry.word.in_(['你们','那么','已收到','一样']),),NOT(DocTry.has_go==False)).all()forresultinall_re:print(result.query_to_dict())更多使用方法,请见:参与贡献联系642641850@qq.com
Package: elasticsearch-xpack
This is an addon to the official elasticsearch python client that adds
functionality for theX-Pack extensionsInstallationYou can install this addon usingpip:pip install elasticsearch-xpackUsageYou can use this client alone:fromelasticsearchimportElasticsearchfromelasticsearch_xpackimportXPackClientclient=Elasticsearch()xpack=XPackClient(client)xpack.info()Or you can add thexpacknamespace to the official client to mimic the
behaviors of other namespaces:XPackClient.infect_client(client)client.xpack.info()LicenseCopyright 2015 ElasticsearchLicensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Package: elastic-ssh
AWS EC2 SSH UtilityThis is a command line utility to assist in connecting to EC2 via SSH using IAM andEC2 Instance Connect.  It also allows connecting to private instances via a Bastion Host if you have one configured in your VPC. This currently only works on *nix command lines with the OpenSSH client installed at /usr/bin/ssh.AWS SetupFollow Amazon'sdocumentationon setting up Instance Connect.InstallationInstall via you're favorite method for installing Python-based CLI tools:pip,pipenv,easy_install, etc.  I recommend usingpipenv, creating an environment just for this tool and then creating a symlink from the virtualenv bin directory to a directory on your PATH.$pipenvinstall-egit+https://github.com/goodwillaz/elastic-ssh\#egg=elastic-ssh
$sudoln-s`pipenv--venv`/bin/aws-ec2/usr/local/bin/aws-ec2Once you've installed the utility, you'll need to configure it.  It uses AWS's Boto library which means it will look in the standard places for AWS Credentials.  To configure the utility with an optional profile name (default isdefault):$aws-ec2[--profile<profile>]configureNote that the private key you are prompted for does not need to be the key-pair you used to create your EC2 instance.  You can use any RSA based key-pair. The public key must sit in the same folder as the private key and have a.pubsuffix to it.UsageSimply typeaws-ec2 sshon the command line; you'll be provided with a list of instances (and their names) for easy selection.  Select the instance you'd like to connect to and you'll instantly be connected!For help, simply runaws-ec2 --helporaws-ec2 <command> --helpEnvironment Variable SupportAll of the command line flags can be specified as environment variables (if you need to):AWS_EC2_DEBUGAWS_EC2_PROFILEAWS_EC2_SSH_USERAWS_EC2_SSH_PORTAWS_EC2_SSH_KEYAWS_EC2_SSH_COMMANDLicenseSee theLICENSEfile for license rights and limitations (BSD 3-clause).
Package: elastictalk
ElastictalkElastictalk is a AWS ebcli extend libraryUsage# Show commands
et

# Show global argument setting
et --help

# Show commands' argument setting
# et <command> --help
# Ex:
et save_env_var --help

# Use command with arguments
elastictalk.py take_rds_snapshot rds_id --rds_snapshot_id=new_rds_snapshot_id
Package: elastictools
Useful tools to work with Elastic stack in PythonDescriptionIt consists of two main modules:indextools: tools to work with Elastic indicesdoctools: tools to work with Elastic documentsInstallationNormal installationpipinstallelastictoolsDevelopment installationgitclonehttps://github.com/ncthuc/elastictools.gitcdelastictools
pipinstall--editable.Change log[0.2.3] - 2019-06-19Fix multiple doc_type in mapping[0.2.2] - 2019-06-18Fix multiple error related to doc_type in ES7Update elasticsearch-py to 7.0.2Fix bug related to doctype while indexing document in ES 7[0.2.1] - 2019-06-18Update elasticsearch-py to 7.0.2Fix bug related to doctype while indexing document in ES 7[0.1.4] - 2018-12-07AddedDocTools.dump()DocTools.bulk_insert_from_json()[0.1.3] - 2018-11-09AddedDocTools.bulk()DocTools.bulk_insert_from_csv()IndexTools.create_if_not_exists()IndexTools.truncate()[0.1.2] - 2018-11-08AddedHISTORY.mdlong_descriptionin setup.py that contains content ofREADME.mdandHISTORY.mdChangedN/ARemovedN/A[0.0.1] - 2018-11-07AddedDocTools.make_search_body()DocTools.search()DocTools.msearch()ChangedRemoved