Package: pandagg
What is it?pandaggis a Python package providing a simple interface to manipulate ElasticSearch queries and aggregations. Its goal is to make it
the easiest possible to explore data indexed in an Elasticsearch cluster.Some of its interactive features are inspired bypandaslibrary, hence the namepandaggwhich aims to applypandas to Elasticsearchaggregations.pandaggis also greatly inspired by the official high level python clientelasticsearch-dsl,
and is intended to make it more convenient to deal with deeply nested queries and aggregations.Why another librarypandaggprovides the following features:interactive mode for cluster discoveryricher aggregations syntax, and aggregations parsing featuresdeclarative indicesbulk ORM operationstyping annotationsDocumentationFull documentation and user-guide are availablehere on read-the-docs.Installationpip install pandaggDependenciesHard dependency:ligthtreeSoft dependency: to parse aggregation results as tabular dataframe:pandasQuick demoDiscover indices on cluster with matching pattern:>>>fromelasticsearchimportElasticsearch>>>frompandagg.discoveryimportdiscover>>>client=Elasticsearch(hosts=['localhost:9200'])>>>indices=discover(client,"mov*")>>>indices<Indices>['movies','movies_fake']Explore index mappings:>>>movies=indices.movies>>>movies.imappings<Mappings>_├──directors[Nested]│├──director_idKeyword│├──first_nameText││└──raw~Keyword│├──full_nameText││└──raw~Keyword│├──genresKeyword│└──last_nameText│└──raw~Keyword├──genresKeyword├──movie_idKeyword├──nameText...>>>movies.imappings.roles<Mappingssubpart:roles>roles[Nested]├──actor_idKeyword├──first_nameText│└──raw~Keyword├──full_nameText│└──raw~Keyword├──genderKeyword├──last_nameText│└──raw~Keyword└──roleKeywordExecute aggregation on field:>>>movies.imappings.roles.gender.a.terms()doc_countkeyM2296792MF1135174FBuild search request:>>>search=movies.search().size(2).groupby('decade','histogram',interval=10,field='year').groupby('genres',size=3).agg('avg_rank','avg',field='rank').agg('avg_nb_roles','avg',field='nb_roles').filter('range',year={"gte":1990})>>>search.to_dict(){'aggs':{'decade':{u'aggs':{'genres':{u'aggs':{'avg_nb_roles':{u'avg':{'field':'nb_roles'}},'avg_rank':{u'avg':{'field':'rank'}}},'terms':{'field':'genres','size':3}}},'histogram':{'field':'year','interval':10}}},'query':{'bool':{u'filter':[{'range':{'year':{'gte':1990}}}]}},'size':2}Execute it:>>>response=search.execute()>>>response<Response>took52ms,success:True,totalresult>=10000,contains2hitsParse it in tabular format:>>>response.aggregations.to_dataframe()avg_nb_rolesavg_rankdoc_countdecadegenres1990.0Documentary3.7789826.5170938393Drama18.5180675.98142912232Short3.0232846.311326121972000.0Documentary5.5814336.9808988639Drama14.3853916.26967511500Short4.0530826.83625313451DisclaimersIt does not ensure retro-compatible with previous versions of elasticsearch (intended to work with >=7). It is part
of the roadmap to tagpandaggversions according to the ElasticSearch versions they are related to (ie
v7.1.4 would work with Elasticsearch v7.X.X).ContributingAll contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome.Roadmap prioritiesclean and proper documentationpackage versions for different ElasticSearch versionsonboard new contributors