TAMIL AWS S3 SELECT | InterviewDOT
Channel: Interview DOT
Date: 2019-07-06

Click here - https://www.youtube.com/channel/UCd0U_xlQxdZynq09knDszXA?sub_confirmation=1 to get notifications. TAMIL AWS S3 SELECT

https://github.com/net-vinothkumar/awss3select/blob/master/README.md

AWS S3 Bucket :
Amazon Simple Storage Service (S3) stores data for millions of applications used by market leaders in every industry.

With S3, I can store as many objects as I want and individual objects can be as large as 5 terabytes. Data in object storage have traditionally been accessed as a whole entities, meaning when you ask for a 5 gigabyte object you get all 5 gigabytes. It’s the nature of object storage.

S3 Select enables applications to retrieve only a subset of data from an object by using simple SQL expressions.
By using S3 Select to retrieve only the data needed by your application, you can achieve drastic performance increases – in many cases you can get as much as a 400% improvement.

When to use S3-Select?
S3 Select is extremely useful when:
You have a large amount of structured data that can be queried upon.
Only a small portion of the object is relevant to your current needs.
You want to have partial data retrieval ability and get filtered data for user. You want to pre-filter S3 objects before performing additional analysis with tools like Spark or Presto. You want to reduce the volume of data that has to be loaded and processed by your applications thus, improving performance and thus, reducing cost.

Cost of S3-Select
The cost of S3 Select depends on two things — DATA SCANNED TO QUERY and DATA RETURNED. For example, if S3 query scans 5GB data and returns 2GB data, the cost for us-east-1 region would be :
Data Scanned by S3 SELECT : $0.002 / GB * 5 = $0.01
Data Returned by S3 SELECT : $0.0007/GB * 2 = $0.0014
Total = $0.0114 per S3 SELECT request.

Terminology to write S3-Select query
To use S3 Select, your data must be structured in either CSV or JSON format with UTF-8 encoding. You can also compress your files with GZIP or BZIP2 before sending to S3 to save on object size.

aws s3 pricing : https://aws.amazon.com/s3/pricing/?nc1=h_ls

To learn more in english - simplilearn: https://www.youtube.com/watch?v=XGcoeEyt2UM