Package: escli
EscliEscli is a tool for interacting with an Elasticsearch service via the command line.
It can also be used with Enterprise App Search, by use of the--appflag.This project began as an experimental side project during November 2021, implementing a limited set of functionality.
It is currently considered prototypical, and not suitable for production use.InstallationTo installescliin the current virtualenv or for the entire system, simply usepip:$pipinstallescliTo instead install just for the current user, include the--useroption:$pipinstall--userescliVersionThe current installed version ofesclican be shown using theescli versioncommand.Quick Search Example$exportESCLI_CLOUD_ID=xxxx:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
$exportESCLI_PASSWORD=XXXXXXXXXXXXXXXXXXXX
$esclisearchkibana_sample_data_flights-fgithub-i"FlightNum,OriginAirportID,DestAirportID"-n15|FlightNum|DestAirportID|OriginAirportID||-------------|-----------------|-------------------||9HY9SWR|SYD|FRA||X98CCZO|VE05|CPT||UFK2WIZ|VE05|VE05||EAYQW69|TV01|NA01||58U013N|XIY|AICM||XEJ78I2|GE01|CYEG||EVARI8I|ZRH|ZRH||1IRBW25|YOW|RM12||M05KE88|HYD|MI11||SNI3M1Z|TV01|SVO||JQ2XXQ5|HEL|ABQ||V30ITD0|VIE|VE05||P0WMFH7|PVG|AICM||VT9O2KD|YOW|NA01||NRHSVG8|SJU|RM12|Operating ModesBy default,esclioperates in Elasticsearch mode, which expects a regular Elasticsearch service to be available.To switch to Enterprise App Search mode instead, useescli -aorescli --app.Connectivity & AuthenticationTheesclitool relies on connection details and credentials supplied through environment variables.
For a defaultElastic Clouddeployment, only theESCLI_CLOUD_IDandESCLI_PASSWORDvariables will generally need to be set.
Other variables are available for use with local, on-prem, and other customised deployments.The following variables are accepted:ESCLI_CLOUD_IDTheCloud IDof an Elastic Cloud deployment.
If using a local or on-prem deployment, this can remain unset.ESCLI_ADDRThe host names or URLs to which to connect.
This does not need to be set ifESCLI_CLOUD_IDis set.The string may contain one or more individual values, each separated by commas.
Each value can be of the formhost,host:portorscheme://host:port.
Bothhttpandhttpsschemes are valid here.ESCLI_USERThe name of the user, used for authentication over HTTP.
If this value is not set,elasticis used as a default.ESCLI_PASSWORDThe password used for authentication over HTTP.
If no password is set,escliassumes no HTTP auth is intended, and connects without.VerbosityVerbosity can be increased using the-vcommand line option and decreased using the-qoption.
These options can be passed multiple times (e.g.-vv) for a higher level of detail with eachvorqincreasing or decreasing the level respectively.
Any-vand-qoptions passed must be includedbeforethe command, i.e.escli -v COMMAND ARGS...The table below shows the available verbosity levels and the options required to select each.
Verbosity level zero is the default and does not require any explicit options to be passed.
Note that critical errors cannot be hidden.VerbosityOptionsDEBUGINFOWARNINGERRORCRITICAL+2-vvshowshowshowshowshow+1-vhideshowshowshowshow0hidehideshowshowshow-1-qhidehidehideshowshow-2-qqhidehidehidehideshowSearchingA search can be performed using theescli searchcommand.
Each search operation requires a target index and search criteria in the formFIELD=VALUE.
Column selection and output format can also be tuned by command line options.If no criteria are passed, a 'match_all' search will be carried out.
The example below searches thekibana_sample_data_flightsindex, returning theFlightNum,OriginandDestfields for the first 5 hits.$esclisearch-i=FlightNum,Origin,Dest-n=5kibana_sample_data_flights
FlightNumOriginDest
-----------------------------------------------------------------------------------------------------
9HY9SWRFrankfurtamMainAirportSydneyKingsfordSmithInternationalAirport
X98CCZOCapeTownInternationalAirportVeniceMarcoPoloAirport
UFK2WIZVeniceMarcoPoloAirportVeniceMarcoPoloAirport
EAYQW69NaplesInternationalAirportTreviso-Sant'Angelo Airport58U013N      Licenciado Benito Juarez International Airport  Xi'anXianyangInternationalAirportThis second example applies criteria to select only those results with "London" within theOriginCityName.$esclisearch-i=FlightNum,Origin,Dest-n=5kibana_sample_data_flightsOriginCityName=London
FlightNumOriginDest
----------------------------------------------------------------------------------------
46J5N4YLondonGatwickAirportOttawaMacdonald-CartierInternationalAirport
R0JFGVCLondonLutonAirportStockholm-ArlandaAirport
X8NT4WOLondonGatwickAirportNewChitoseAirport
T0939V5LondonGatwickAirportLondonGatwickAirport
AGZPJJ3LondonLutonAirportMontreal/PierreElliottTrudeauInternationalAirportOutput FormatsEscli supports a number of different output formats for search results.
Theescli formatscommand shows the full list of formats available.$escliformats
Outputformatsforsearchresults:csvcsv_unixfancy_gridfancy_outlinegithubgridhtmljiralatexlatex_booktabslatex_longtablelatex_rawmediawikimoinmoinndjsonorgtblpipeplainprestoprettypsqlrstsimpletextiletsvunsafehtmlyoutrackThis list includes all formats supported bytabulatewhich is used internally by Escli.SortingResults can be sorted using the-sor--sortoption.
To this can be passed the name of a field by which to sort.
To sort in reverse order, prefix the field name with a tilde '~' symbol.PaginationSearch results are automatically paginated.
By default, the first 10 results from page 1 are returned, but this can be tuned using the-nand-poptions respectively.The-noption (long form--page-size) is used to determine the number of results returned per page.The-poption (long form--page-number) is used to select a page number to return.
All results on earlier pages will be skipped, returning only the results for the desired page.The example below shows an App Search query against thenational-parks-demodata set, returning only page 3 of results.$escli--appsearch-i=id,title,date_established-s=title-p=3national-parks-demo
date_establishedtitleid
------------------------------------------------------------------------1910-05-11T05:00:00+00:00Glacierpark_glacier1980-12-02T06:00:00+00:00GlacierBaypark_glacier-bay1919-02-26T06:00:00+00:00GrandCanyonpark_grand-canyon1929-02-26T06:00:00+00:00GrandTetonpark_grand-teton1986-10-27T06:00:00+00:00GreatBasinpark_great-basin2004-09-13T05:00:00+00:00GreatSandDunespark_great-sand-dunes1934-06-15T05:00:00+00:00GreatSmokyMountainspark_great-smoky-mountains1966-10-15T05:00:00+00:00GuadalupeMountainspark_guadalupe-mountains1916-08-01T05:00:00+00:00Haleakalapark_haleakala1916-08-01T05:00:00+00:00HawaiiVolcanoespark_hawaii-volcanoesIngestionTo ingest data, use theescli ingestcommand.
One or more JSON-formatted files can be supplied with the document content, or data can be read fromstdin.A simple import fromstdinmight look like this:$echo'{"name": "Alice", "age": 33}'|escli-vingestpeople
INFO:[elasticsearch]GEThttp://localhost:9200/[status:200request:0.002s]INFO:[elasticsearch]POSThttp://localhost:9200/people/_doc[status:201request:0.177s]INFO:[escli.commands.ingest]IngestedJSONdatafromfile'<stdin>'withresult{...}Whereas an import from a file would look like this:$escli-vingestpeoplebob.json
INFO:[elasticsearch]GEThttp://localhost:9200/[status:200request:0.002s]INFO:[elasticsearch]POSThttp://localhost:9200/people/_doc[status:201request:0.008s]INFO:[escli.commands.ingest]IngestedJSONdatafromfile'bob.json'withresult{...}A quick search shows that the documents have been successfully ingested:$esclisearchpeople
nameage
-----------
Alice33Bob44While JSON is the default format required for source data, the-foption allows for explicit selection of any one of the available formats, listed below:csv- Excel-compatible CSVcsv_unix- Unix-compatible CSVjson- single document JSONndjson- newline-delimited JSONtsv- Tab-separated valuesNote that most formats allow one document per line, whereasjsononly allows one document per file, by design.
CSV and TSV formats require a header line to be included, containing the names of the fields.Chaining Input and OutputNewline-delimited JSON (ndjson) is supported as both an input and an output format.
This allows data to be easily streamed out of oneescliprocess and into another.The example below extracts five documents from thekibana_sample_data_flightsindex and feeds them directly into theflights2index.$esclisearchkibana_sample_data_flights-n=5-f=ndjson|escli-vingestflights2-f=ndjson
INFO:[elasticsearch]GEThttp://localhost:9200/[status:200request:0.002s]INFO:[elasticsearch]POSThttp://localhost:9200/flights2/_doc[status:201request:0.150s]INFO:[escli.commands.ingest]IngestedJSONdatafromfile'<stdin>',line1withresult{...}INFO:[elasticsearch]POSThttp://localhost:9200/flights2/_doc[status:201request:0.005s]INFO:[escli.commands.ingest]IngestedJSONdatafromfile'<stdin>',line2withresult{...}INFO:[elasticsearch]POSThttp://localhost:9200/flights2/_doc[status:201request:0.004s]INFO:[escli.commands.ingest]IngestedJSONdatafromfile'<stdin>',line3withresult{...}INFO:[elasticsearch]POSThttp://localhost:9200/flights2/_doc[status:201request:0.004s]INFO:[escli.commands.ingest]IngestedJSONdatafromfile'<stdin>',line4withresult{...}INFO:[elasticsearch]POSThttp://localhost:9200/flights2/_doc[status:201request:0.005s]INFO:[escli.commands.ingest]IngestedJSONdatafromfile'<stdin>',line5withresult{...}Note that-f ndjsonis used for format selection for both thesearchandingestprocesses.