Package: athena-usage-metrics-extractor
Athena Usage Extractor[]Athena helps you analyze unstructured, semi-structured, and structured data stored in Amazon S3. Examples include CSV, JSON, or columnar data formats such as Apache Parquet and Apache ORC. You can use Athena to run ad-hoc queries using ANSI SQL, without the need to aggregate or load the data into Athena.Athena usage is simple python library that allows you to extract all usage informationInstallation[]AthenaUsageExtractorAthena helps you analyze unstructured, semi-structured, and structured data stored in Amazon S3. Examples include CSV, JSON, or columnar data formats such as Apache Parquet and Apache ORC. You can use Athena to run ad-hoc queries using ANSI SQL, without the need to aggregate or load the data into Athena.Athena usage is simple python library that allows you to extract all usage informationInstallationacpipinstallathena-usage-metrics-extractorUsageimportsysfromAthenaUsageExtractorimportAthenaUsageExtractordefmain():helper=AthenaUsageExtractor(aws_region='us-east-1',aws_access_key='XXXXX',aws_secret_key='XXXXX')response=helper.get_usage_for_date(date='2022-08-12',workgroup='primary')whileTrue:try:data=next(response)print(data)exceptStopIterationase:breakexceptExceptionase:breakmain()Json format Returned{"QueryExecutionId":"490024e6-3e89-4ec4-9ffd-b1302a77d33d","Query":"<YOU WILL GET THE QUERY USER RAN >","StatementType":"DML","WorkGroup":"primary","OutputLocation":"<AWS S3 Output Path >","Database":"default","SelectedEngineVersion":"AUTO","EffectiveEngineVersion":"Athena engine version 2","EngineExecutionTimeInMillis":"14045","DataScannedInBytes":"59597591861","TotalExecutionTimeInMillis":"14292","QueryQueueTimeInMillis":"214","QueryPlanningTimeInMillis":"960","ServiceProcessingTimeInMillis":"33","State":"SUCCEEDED","SubmissionDateTime":"2022-08-12 13:56:07.837000-04:00","CompletionDateTime":"2022-08-12 13:56:22.129000-04:00"}AuthorsSoumil Nitin ShahSoumil Nitin ShahBachelor in Electronic Engineering |
Masters in Electrical Engineering |
Master in Computer Engineering |Website :https://soumilshah.herokuapp.comGithub:https://github.com/soumilshah1995Linkedin:https://www.linkedin.com/in/shah-soumil/Blog:https://soumilshah1995.blogspot.com/Youtube :https://www.youtube.com/channel/UC_eOodxvwS_H7x2uLQa-svw?view_as=subscriberFacebook Page :https://www.facebook.com/soumilshah1995/Email :shahsoumil519@gmail.comprojects :https://soumilshah.herokuapp.com/projectI earned a Bachelor of Science in Electronic Engineering and a double masterâ€™s in Electrical and Computer Engineering. I have extensive expertise in developing scalable and high-performance software applications in Python. I have a YouTube channel where I teach people about Data Science, Machine learning, Elastic search, and AWS. I work as data collection and processing Team Lead at Jobtarget where I spent most of my time developing Ingestion Framework and creating microservices and scalable architecture on AWS. I have worked with a massive amount of data which includes creating data lakes (1.2T) optimizing data lakes query by creating a partition and using the right file format and compression. I have also developed and worked on a streaming application for ingesting real-time streams data via kinesis and firehose to elastic searchLicenseThis project is licensed under the MIT License - see theLICENSE.mdfile for details