Saravanan666/pypi-org huggingface dataset:
Package: elasticsearch-async
This is an adapter forelasticsearch-pyproviding a transport layer based on
Python’sasynciomodule. All API calls now return a future wrapping the
response.Sniffing (when requested) is also done via a scheduled coroutine.Example for python 3.5+importasynciofromelasticsearch_asyncimportAsyncElasticsearchclient=AsyncElasticsearch(hosts=['localhost','other-host'])asyncdefprint_info():info=awaitclient.info()print(info)loop=asyncio.get_event_loop()loop.run_until_complete(print_info())loop.run_until_complete(client.transport.close())loop.close()Example for python 3.4importasynciofromelasticsearch_asyncimportAsyncElasticsearchclient=AsyncElasticsearch(hosts=['localhost','other-host'])@asyncio.coroutinedefprint_info():info=yield fromclient.info()print(info)loop=asyncio.get_event_loop()loop.run_until_complete(print_info())loop.run_until_complete(client.transport.close())loop.close()Example with SSL Contextimportasynciofromelasticsearch_asyncimportAsyncElasticsearchfromelasticsearch.connection.http_urllib3importcreate_ssl_contextcontext=create_ssl_context(cafile="/certs/ca/ca.crt")client=AsyncElasticsearch(hosts=['elasticsearch-xpack'],ssl_context=context,http_auth=('elastic','changeme'))@asyncio.coroutinedefprint_info():info=yield fromclient.info()print(info)loop=asyncio.get_event_loop()loop.run_until_complete(print_info())loop.run_until_complete(client.transport.close())loop.close()AsyncElasticsearchintroduces one extra parameterloopwhich can be
used to pass in an event loop you wish the client to use. By defaultasyncio.get_event_loop()will be used.Installationelasticsearch-asyncis available via PyPI so you can install it using pippipinstallelasticsearch-asyncLicenseCopyright 2015 ElasticsearchLicensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Package: elasticsearchcloudwatch
elasticsearch-cloudwatch-python=============================Elasticsearch monitoring scripts for CloudWatch.### NOTE: This is just a template from which to make your own scripts. ###Do not attempt to download and run.  You will need to modify the scripts tofit your own needs.Requirements------------- Python >= 2.6- Boto >= 2.33.0- Boto3 >= 1.2.2- Elasticsearch >= 2.1.0Installation------------Optionally create a virtual environment and activate it. Then just run`pip install elasticsearchcloudwatch`. This will install the scripts in /usr/local/bin folder.For script usage, run:get_web_reqeust.py --helpput_web_request.py --helpExamples--------Edit the /etc/elasticsearchcloudwatch/escw.config file and and put the appropriate parameters.To perform a simple test run without posting data to Amazon CloudWatch* * * * * /usr/bin/get_web_request.py  --query WebRequest_Test  --verboseTo post the data to Amazon CloudWatch* * * * * /usr/bin/put_web_request.py  --query WebRequest_Test --from-cron  --verboseConfiguration-------------To allow an EC2 instance to read and post metric data to Amazon CloudWatch,this IAM policy is required:{"Statement": [{"Action": ["elasticsearchcloudwatch:WebRequest"],"Effect": "Allow","Resource": "*"}]}If the policy is configured via an IAM role that is assigned to the EC2server this script runs on, you're done.Otherwise you can configure the policy for a user account and exportthe credentials before running the script:export AWS_ACCESS_KEY_ID=[Your AWS Access Key ID]export AWS_SECRET_ACCESS_KEY=[Your AWS Secret Access Key]Third option is to create a _~/.boto_ file with this content:[Credentials]aws_access_key_id = Your AWS Access Key IDaws_secret_access_key = Your AWS Secret Access KeyCopyright---------Copyright 2015 Will RubelBased on Python-Version of CloudWatch Monitoring Scripts for Linux -Copyright 2015 Oliver Siegmar. All Rights Reserved.Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.
Package: elasticSearch-collections
ElasticSearchCollectionsCollections of ElasticSearch pyscripts for humanExamplefromelasticSearch_collections.elasticSearch_collectionsimportElasticSearchCollectionsfromtypingimportDict,List# Connect to es hostses=ElasticSearchCollections(["https://127.0.0.1"],http_auth=("username","password"),ca_certs=False,verify_certs=False,timeout=600)deffn_data_update(_data:List[Dict]):actions=[]foriin_data:#  do somethingaction={"_op_type":"update","_index":i["_index"],"_type":"_doc","_id":i["_id"],"doc":{# update logic}}actions.append(action)ifactions:es.bulk(actions)es_search_info={"index":'_index',"scroll":'1m',"size":1000,"body":"""{"query": {"match_all":{}}}""","timeout":"10m"}_res_list=es.scroll_source_generator(es_search_info)# run multiple threads with threadofromthreado.simple_thread_runnerimportSimpleThreadsRunner# Define the num_workers and batch_size# refer to https://github.com/AlexNg9527/threadosr=SimpleThreadsRunner(fn=fn_data_update)sr.run_threads(num_workers=10,iter_data=_res_list,batch_size=10)# clear scroll_ides.delete_scroll_id()Installing and Supported VersionselasticSearch_collections is available on PyPI:$python-mpipinstallelasticSearch_collections
Package: elasticsearch-curator
Have indices in Elasticsearch? This is the tool for you!Like a museum curator manages the exhibits and collections on display,
Elasticsearch Curator helps you curate, or manage your indices.ANNOUNCEMENTCurator is breaking into version dependent releases. Curator 6.x will work with
Elasticsearch 6.x, Curator 7.x will work with Elasticsearch 7.x, and when it is
released, Curator 8.x will work with Elasticsearch 8.x.Watch this space for updates when that is coming.New Client ConfigurationCurator now connects using thees_clientPython module. This separation makes it much easier
to update the client connection portion separate from Curator. It is largely derived from the
original Curator client configuration, but with important updates.The updated configuration file structure requireselasticsearchat the root level:---
elasticsearch:
  client:
    hosts: https://10.11.12.13:9200
    cloud_id:
    bearer_auth:
    opaque_id:
    request_timeout: 60
    http_compress:
    verify_certs:
    ca_certs:
    client_cert:
    client_key:
    ssl_assert_hostname:
    ssl_assert_fingerprint:
    ssl_version:
  other_settings:
    master_only:
    skip_version_test:
    username:
    password:
    api_key:
      id:
      api_key:

logging:
  loglevel: INFO
  logfile: /path/to/file.log
  logformat: default
  blacklist: []Action File ConfigurationAction file structure is unchanged, for now. A few actions may have had the options modified a bit.
Package: elasticsearch-curator-api
NOTE:this package is a fork ofElasticsearch Curatorwithout the CLI functionality (and without a dependency onClick). Only the API is exposed for library usage.Curator APIHave indices in Elasticsearch? This is the library for you!Like a museum curator manages the exhibits and collections on display,
Elasticsearch Curator helps you curate, or manage your indices.Build StatusBranchStatusMaster5.xPyPI:Curator API DocumentationCurator ships with both an API and a wrapper script (which is actually defined
as an entry point).  The API allows you to write your own scripts to accomplish
similar goals, or even new and different things with theCurator API, and
theElasticsearch Python API.Getting StartedSee theInstallation guideand thecommand-line usage guideRunningcurator--helpwill also show usage information.Contributingfork the repomake changes in your forkadd tests to cover your changes (if necessary)run testssign theCLAsend a pull request!To run from source, use therun_curator.pyscript in the root directory of
the project.Running TestsTo run the test suite just runpython setup.py testWhen changing code, contributing new code or fixing a bug please make sure you
include tests in your PR (or mark it as without tests so that someone else can
pick it up to add the tests). When fixing a bug please make sure the test
actually tests the bug - it should fail without the code changes and pass after
they’re applied (it can still be one commit of course).The tests will try to connect to your local elasticsearch instance and run
integration tests against it. This will delete all the data stored there! You
can use the env variableTEST_ES_SERVERto point to a different instance
(for example, ‘otherhost:9203’).OriginsCurator was first calledclearESindices.py[1] and was almost immediately
renamed tologstash_index_cleaner.py[1].  After a time it was migrated
under the [logstash](https://github.com/elastic/logstash) repository asexpire_logs.  Soon thereafter, Jordan Sissel was hired by Elasticsearch, as
was the original author of this tool.  It became Elasticsearch Curator after
that and is now hosted at <https://github.com/elastic/curator>[1] <https://logstash.jira.com/browse/LOGSTASH-211>
Package: elasticsearch-curator-internal
Have indices in Elasticsearch? This is the tool for you!Like a museum curator manages the exhibits and collections on display,
Elasticsearch Curator helps you curate, or manage your indices.Curator API DocumentationCurator ships with both an API and a wrapper script (which is actually defined
as an entry point).  The API allows you to write your own scripts to accomplish
similar goals, or even new and different things with theCurator API, and
theElasticsearch Python API.Curator CLI DocumentationTheCurator CLI Documentationis now a part of the document repository athttp://elastic.co/guideathttp://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.htmlGetting StartedSee theInstallation guideand thecommand-line usage guideRunningcurator--helpwill also show usage information.Contributingfork the repomake changes in your forkadd tests to cover your changes (if necessary)run testssign theCLAsend a pull request!To run from source, use therun_curator.pyscript in the root directory of
the project.Running TestsTo run the test suite just runpython setup.py testWhen changing code, contributing new code or fixing a bug please make sure you
include tests in your PR (or mark it as without tests so that someone else can
pick it up to add the tests). When fixing a bug please make sure the test
actually tests the bug - it should fail without the code changes and pass after
they’re applied (it can still be one commit of course).The tests will try to connect to your local elasticsearch instance and run
integration tests against it. This will delete all the data stored there! You
can use the env variableTEST_ES_SERVERto point to a different instance
(for example, ‘otherhost:9203’).OriginsCurator was first calledclearESindices.py[1]and was almost immediately
renamed tologstash_index_cleaner.py[1].  After a time it was migrated
under thelogstashrepository asexpire_logs.  Soon thereafter, Jordan Sissel was hired by Elasticsearch, as
was the original author of this tool.  It became Elasticsearch Curator after
that and is now hosted atelastic/curator.[1](1,2)LOGSTASH-211.
Package: elasticsearch-fixtures
Elasticsearch FixturesThis tool is heavily inspired byklen's Mixer.This tool is optimized to be used withDjangoandpytest.It allows you to add documents to a test-index in yout Elasticsearch instance
when you run your unittests.Withelasticsearch_fixturesyou can do something like this:fromelasticsearch_fixtures.es_mixerimportESMixeres_mixer=ESMixer(host='http://localhost:9200/')deftest_something():doc1=es_mixer.blend('indexname',id=1,title='test')assertdoc1['source']['title']=='test'Note: If you provide anidand a document with thatidalready exists, that
document will not be updated, but fully replaced.You can also create documents without providing anid. Elasticsearch will then
auto-create anid:deftest_something():doc1=es_mixer.blend('indexname',title='test')print(doc1['id'])And you can update an existing document that is already in the index:deftest_something():es_mixer.blend('indexname',id=1,title='test')doc1=es_mixer.update('indexname',id=1,title='new title')assertdoc1['source']['title']=='new_title'Big word of warning: We are no ES experts. We have only started using ES around
June 2019 and one of the first problems that we had was that we did not know how
to properly write unittests for our Django views and Graphene resolver
functions. There are probably much better ways to do this. If you know one, we
would love to hear about it (just open an issue and tell us about it!).Installationpip install elasticsearch-fixturesConfigurationIn order to make sure that your tests wipe the index at the very beginning and
also after each test, create the followingconftest.py:"""Global settings for pytest."""importpytestfromdjango.confimportsettingsfromelasticsearch_fixtures.es_mixerimportESMixeres_mixer=ESMixer(host=settings.ELASTICSEARCH_HOST)index=settings.ELASTICSEARCH_INDEX@pytest.fixture(scope='session',autouse=True)defsetup_elasticsearch():es_mixer.wipe_index(index)@pytest.fixture(autouse=True)defcleanup_elasticsearch():yieldes_mixer.wipe_index(index)As you can see, in this example we also use a Django setting calledELASTICSEARCH_HOST(with a trailing slash) andELASTICSEARCH_INDEX. Of
course you can name your own settings however you like.
Package: elasticsearch-follow
Follow Elasticsearch continuouslyOverviewelasticsearch_follow is library helping to query Elasticsearch continuously.It needshttps://github.com/elastic/elasticsearch-pyas a dependency.elasticsearch_follow acts as a wrapper for elasticsearch-py and handles various
use-cases, like following logs by polling elasticsearch continuously and fetching
loglines via a generator. It is possible to easily fetch lines surrounding a
given logline.How to poll Elasticsearch continuouslyThe polling logic is implemented in the class ElasticsearchFollow, which needs
an Elasircsearch object from elasticsearch-py. The class Follower takes an
ElasticsearchFollow-object and has a method to create a generator which yields
loglines until all elements of a query have been returned. After this a new
generator has to be created and used.How to fetch log-lines from ElasticsearchTo just fetch loglines, one can use ElasticsearchFetch which has
a search_surrounding. This returns a list of lists, where each list contains
the queried loglines and the lines before and after as requested by the parameters
num_before and num_after.InstallationYou can install theelasticsearchpackage with pip:pipinstallelasticsearch_followSee also:https://pypi.org/project/elasticsearch-follow/Example usageThis package introduces the command line tooles_tailwhich can be used for
following logs written to Elasticsearch and directly fetching log lines by a query.
It is possible to configure the output via an format string.# Follow the logs written to the indexes starting with logstash.# Print the fieds @timestamp and messagees_tail-c"http://localhost:9200"tail--index"logstash*"-f"{@timestamp} {message}"# Fetch all logs in the last hour with the field loglevel contains ERROR and fetch the two lines before and after.# Print the fieds @timestamp and messagees_tail-c"http://localhost:9200"fetch--index"logstash"-f"{@timestamp} {message}"--queryloglevel:ERROR-A2-B2-F"now-1h"# It is also possible to print nested fieldses_tail-c"http://localhost:9200"fetch--index"logstash"-f"{@timestamp} {message} {kv[field]} {kv[nested][field]}"-F"now-1h"The command line options can also be given via environment variables by using the prefixES_TAIL.
For exampleexportES_TAIL_USERNAME='username'exportES_TAIL_PASSWORD='password'es_tail-chttp://localhost:9200tailExample usage of the libraryfromelasticsearchimportElasticsearchfromelasticsearch_followimportElasticsearchFollow,Followeres=Elasticsearch()es_follow=ElasticsearchFollow(elasticsearch=es)# The Follower is used to get a generator which yields new# elements until it runs out. time_delta give the number of# seconds to look into the past.follower=Follower(elasticsearch_follow=es_follow,index='some-index',time_delta=60)whileTrue:entries=follower.generator()forentryinentries:print(entry)time.sleep(0.1)
Package: elasticsearch-haystack
Elasticsearch Document StoreDocument Store for Haystack 2.x, supports ElasticSearch 8.Installationpip install elasticsearch-haystackTestingTo run tests first start a Docker container running ElasticSearch. We provide a utilitydocker-compose.ymlfor that:docker-compose upThen run tests:hatch run testLicenseelasticsearch-haystackis distributed under the terms of theApache-2.0license.
Package: elasticsearch-kernel
Elasticsearch_kernelElasticsearch  Kernel for Jupyter中文介绍Installationstep1:pip3 install elasticsearch_kernelTo get the newest one from this repo (note that we are in the alpha stage, so there may be frequent updates), type:pip3 install git+git://github.com/Hourout/elasticsearch_kernel.gitstep2:Add kernel to your jupyter:python3 -m elasticsearch_kernel.installALL DONE! 🎉🎉🎉Uninstallstep1:View and remove elasticsearch kerneljupyter kernelspec list
jupyter kernelspec remove elasticsearchstep2:uninstall elasticsearch kernel:pip3 uninstall elasticsearch-kernelALL DONE! 🎉🎉🎉Usingjupyter notebookstep1: you should set elasticsearch server (host and port)step2: write your elasticsearch sqlQuotekernel logohttps://www.elastic.co/cn/elasticsearch
Package: elasticsearch-kibana-cli
Elasticsearch Kibana CLI (eskbcli)ElasticSearch Kibana CLI (eskbcli) provides a shell interface to query
an ElasticSearch backend via the Kibana frontend which is useful in
situations where the ElasticSearch backend is not otherwise accessible.ElasticSearch Kibana CLI makes it possible to copy-paste query expressions
directly from the Kibana user-interface and then easily access very large
sets of result data.  This makes theeskbcliuseful in SecOps situations
where the ability to rapidly move from a Kibana query to raw data is
valued.Configuration options are available to adjust http-headers so-as-to enable
access to Kibana in situations that require complex user-authentication
such as when Kibana exists behind an OAuth reverse proxy or other session-
based authentication arrangement.Install / Upgradeuser@computer:~$pipinstall[--upgrade]elasticsearch-kibana-cliDocumentationDocumentation is available athttps://elasticsearch-kibana-cli.readthedocs.ioUsagesearch- Execute the named search configuration.summary- Summary report for search result datafile; use "-" to pipe stdin.show- Show the named eskbcli search configuration.list- List the available eskbcli search names.Config filesRefer to the worked exampleconfig fileswith descriptions and details.ProjectGithub -github.com/ndejong/elasticsearch_kibana_cliPyPI -pypi.python.org/pypi/elasticsearch-kibana-cliReadTheDocs -elasticsearch-kibana-cli.readthedocs.ioCopyright © 2021 Nicholas de Jong
Package: elasticsearch-logger
elasticsearch-loggerSimple package for logging to Elasticsearch. Expected usage is to use this package inside your k8s cluster.Installationpip install elasticsearch_loggerExample of usageIn this example we write 3 messages (1-3) to buffer and after that write to Elasticsearch running on address 127.0.0.   :9200 one-by-one (that should be changed to bulk write). Logger has 4 levels of severity (DEBUG, INFO, WARNING, ERROR) and writes messages in the following format:appName:default-app-name appName.keyword:default-app-name env:default-env-name env.keyword:default-env-name payload:Hi 3 severity:INFO severity.keyword:INFO timestamp:Jun 30, 2021 @ 18:46:54.620 _index:app-name-logs-default-app-name-30-06-2021 _score: - _type:doc.from elasticsearch_logger import ElasticLogger
mh_logger = ElasticLogger(
    buffer_size=3,
    index='app-name-logs',
    app_name='app_name',
    env='dev',
    port=9200,
    ip_addr='127.0.0.1'
)

mh_logger.info('Hi! 1')
mh_logger.info('Hi! 2')
mh_logger.info('Hi! 3')
mh_logger.info('Hi! 4')
Package: elasticsearch-logging-handler
elasticsearch-logging-handlerThis is a minimalistic Elasticsearch logging handler for python. This handler uses only url as an authentication method. For more robust authentication considerCMRESHandler.Package on PyPI -https://pypi.org/project/elasticsearch-logging-handler/Parameters forElasticHandler:NameTypeRequiredDefaultDescriptionExamplehoststrTrue-Url for Elasticsearch cluster. Currently, this handler support only basic authentication through providing user and password in the url.https://user:password@my-cluster.es:9200,https://my-cluster.es:9200indexstrTrue-Name of the Index to write in.preprocessing-logslevelintFalse0 (NOTSET)Minimal logging level for the handler. Handler will only process messages with level larger than this parameter.20, logging.WARNINGbatch_sizeintFalse1000Size of the buffer that stores logs before sending to Elasticsearch. If buffer is full, send batch immediately, without waiting for theflush_period2000 - send batches of maximum size 2000flush_periodfloatFalse1.0Period during which handler will accumulate logs into batch before sending it ot the cluster.10.0 - wait 10 seconds before sendingtimezonestrFalseNoneTimezone for which to transform @timestamp for the record.'Europe/Amsterdam', 'Australia/Sydney'ElasticHandleris nonblocking, meaning any logging call e.g.logging.exceptionwill not block calling thread. which is useful in the case of high logging load.
Package: elasticsearch-ltr
Python Elasticsearch Learning to Rank clientThis is an add-on toofficial Python Elasticsearch clientadding support
forElasticsearch Learning to Rankplugin API.Installationpython -m pip install elasticsearch_ltrUsagefrom elasticsearch import Elasticsearch
from elasticsearch_ltr import LTRClient

client = Elasticsearch()
LTRClient.infect_client(client)

client.ltr.create_feature_store()
...For more code you may check outtests/folder.Running testsYou'll have to run Elasticsearch onlocalhost:9200with LTR plugin installed. Then just dopython -m pytest tests/
Package: elasticsearch-nvidia-metrics
Environment variablesNODE_NAME          - Hostname to report to elasticsearch (Default: localhost)
ES_HOST_LIST       - List of Elasticsearch targets (Default:http://elasticsearch.jamiehouse:9200)
ES_MAX_CONNECTIONS - Max Elasticsearch connections (Default: 25)
ES_INDEX           - Elasticsearch index to write to (Default: ‘nvidia-metrics-YYYY.MM.DD’
SLEEP_TIME         - Reporting interval (Default: 10)Publish new version: make clean build publishcontainer upload