Snowflake Architecture, Edition & Pricing Overview
By: Rajiv Gupta
Tags: Certification, Guides And Tutorials, Snowflake, Architecture, Snow

Before I start with Snowflake architecture I would like to create some background why Snowflake was created. Legacy data warehouses are based on technology that is, at its core, decades old. They were designed in a time when data was simpler, and the number of people in an organization with the need or desire to access the database were few. Day by day not only volume of data is changing, also source and format of data are also increasing. To meet the demands and opportunities of today and tomorrow, data warehouses will need to fundamentally change. Hence, Snowflake was born which combines a completely new SQL query engine with an innovative architecture natively designed for the cloud.

Snowflake uses a hybrid MPP architecture that combines shared disk and shared-nothing architectures. It uses a single copy of data similar to the shared disk architecture but also provides the ability to run multiple compute engines on top of that data. This hybrid approach provides the benefit of single disk storage but also allows scaling up and down as needed.

Snowflake utilizes micro-partitions to securely and efficiently store data. Snowflake automatically splits data into modest-sized micro-partitions, and metadata is extracted to enable efficient query processing. The micro-partitions are then columnar compressed and fully encrypted using a secure-key hierarchy.

Snowflake is cloud-agnostic, meaning it sits on top of 3 major cloud service providers(i.e. AWS, Azure & Google) and provide the same user experience.

Snowflake is a true SaaS offering. More specifically:

You don't need to buy or install or configure any hardware to use Snowflake. All are taken care by Snowflake.

You don’t need to buy or install or configure or manage any Software. All are taken care by Snowflake.

No headache of hardware maintenance, software upgrades, database tuning, server upgrade and patching etc. All are handled by Snowflake.

Snowflake architecture is divided into 3 layers. Also known as 3 layer architecture.

Cloud Services: Cloud service layer is also known as metadata layer sometime. As this layer is the top most layer which takes care of all metadata stuff. This layer has a collection of other services which help coordinates activities across Snowflake. This is sometimes referred to as “Brain of system” as this layer handles crucial data management functionality. Services managed in this layer include:

Authentication

Infrastructure management

Metadata management

Query parsing and optimization

Access control

2. Query Processing: This layer is also known as Compute layer or Virtual Warehouse layer or Muscles layer. All query execution is performed in the processing layer. Snowflake processes queries using “virtual warehouses”. Each virtual warehouse is an MPP compute cluster composed of multiple compute nodes allocated by Snowflake from a cloud provider. Each Virtual warehouse has its own compute cluster and does not share with any other warehouse. Snowflake Virtual warehouse is an MPP compute cluster which is a combination of multiple compute nodes provided by an underline cloud provider. Compute layer is separate from Storage layer in snowflake which means you can independently scale up/down without disturbing storage layer. The Virtual Warehouse comes in the following eight different sizes. Warehouses can be started and stopped at any time. They can also be resized at any time, even while running, to accommodate the need for more or less compute resources, based on the type of operations being performed by the warehouse.

(Source: Snowflake)

Snowflake supports auto-scale multi-cluster warehouses for up to 10 Virtual Warehouses. A multi-cluster warehouse mode is enabled starting Enterprise Edition and above by specifying different values for maximum and minimum clusters. In this mode, Snowflake starts and stops clusters as needed to dynamically manage the load on the warehouse.

3. Database Storage: Snowflake storage layer is the hybrid columnar storage. Storage in snowflake is shared, meaning multiple warehouse can access same storage at a time on scale. When data is loaded into Snowflake, Snowflake reorganizes that data into its internal optimized, compressed, columnar format. Snowflake stores this optimized data in cloud storage. Snowflake takes care of the organization, file size, structure, compression, metadata, statistics, and other aspects of data storage. Data in Snowflake is stored in any one of the supported cloud provider in its proprietary format which is not disclosed to users. It is also worth noting that compression is automatically applied when data is converted by Snowflake into a columnar format, resulting in increased efficiency and reduced storage costs.

You can connect to Snowflake in various ways mentioned below:

Source: Snowflake

Snowflake Editions

Snowflake offers a multiple edition to choose from. Organization can choose one based on their usage and needs. Each successive edition builds on the previous edition through the addition of edition-specific features and/or higher levels of service. Based on which edition the organization chooses determines the unit costs for the credits and the data storage. Region is also an important factor which derives the cost. Storage cost depends on what option you have taken for. Snowflake offer below two:

On Demand: Usage-based pricing with no long-term licensing requirements.

Usage-based pricing with no long-term licensing requirements. Capacity: Discounted pricing based on an up-front Capacity commitment.

High Level overview of Editions and their features:

Source: Snowflake

Snowflake Pricing

All snowflake pricing is based on how much storage, Compute (Virtual warehouse) and Cloud Services resource you are using.

1> Usage for data storage is calculated on the daily average amount of data (in bytes) stored in the system for:

Files staged for bulk data loading/unloading (can be stored compressed or uncompressed).

Database tables, including historical data for Time Travel (always compressed by Snowflake).

Fail-safe for database tables (always compressed by Snowflake).

Clones of database tables that reference data deleted in the table that owns the clones.

2> Usage for virtual warehouses (compute) resources is calculated based on the number of Snowflake credits consumed by virtual warehouses for executing queries, loading/unloading data, and performing other DML operations.

3> Usage for cloud services is charged only if the daily consumption of cloud services for the account exceeds 10% of the daily usage of the compute resources. The charge is calculated daily (in the UTC time zone). This ensures that the 10% adjustment is accurately applied each day, at the credit price for that day.

Snowflake provides numerous methods, in both the web interface and SQL, for monitoring and controlling your storage, credit, and cloud services usage.

Pricing details based on Provider & Region of your choice along with choice of edition can be seen here.

Hope this blog help you to get insight on Overview of Snowflake Architecture, Edition & Pricing. Feel free to ask a question in the comment section if you have any doubts regarding this. Give a clap if you like the blog. Stay connected to see many more such cool stuff. Thanks for your support.

You Can Find me:

Follow me on Medium: https://rajivgupta780184.medium.com/

Follow me on Twitter: https://twitter.com/RAJIVGUPTA780

Connect with me in LinkedIn: https://www.linkedin.com/in/rajiv-gupta-618b0228/

Subscribe to my YouTube Channel: https://www.youtube.com/channel/UC8Fwkdf2d6-hnNvcrzovktg

#Keep learning #Keep Sharing #Everyday Learning.

References:-