Azure DP-900 Short Notes: Explore Core Data Concepts
Tags: azure, cloud, dp900, certification
Published: 2021-02-13 03:17:01


 👉 [Learn Module: Explore Core Data Concepts](https://docs.microsoft.com/en-us/learn/modules/explore-core-data-concepts/)
<br>

## Identify the need for data solutions

👉Data is a collection of facts such as numbers, descriptions, and observations used in making of decision. 

👉Three types of data

1. Structured data
    - Tabular data that is represented by rows and columns in a database.
    - Tables in this form are called relational databases.
2. Semi-structured data
    - Information that doesn't reside in a relational database but still has some structure to it.
    - Ex: JSON, key-value stores and graph databases
3. Unstructured data
    - Data with no proper structure.
    - Ex: Audio, Video , Binary dat files

👉Based on the type of data, there are multiple ways to store and access data in Azure cloud.

👉Stored data needs to be processed. There are two types of data processing solutions.

1. Transaction processing systems
    - primary function of business computing.
    - work performed by transactional systems is often referred to as Online Transactional Processing (OLTP).
    - Data is divided into small pieces for faster processing.
    - For example in a database  tables are split out into separate groups of columns and this is called normalization.
2. Analytical systems
    - Support business users who need to query data and gain a big picture view.
    - Capturing raw data and generate insights to make future business decisions.
    - Common tasks of a analysis system
        1. Data Ingestion - Capturing the raw data.
        2. Data Processing - Converting captured data into a common format to be processed.
        3. Data Querying - Querying data to analyze.
        4. Data Visualization - Generating charts such as bar charts, line charts out of queried data in order.

---

## Identify types of data and data storage

👉Relational Data and Non-relational Data have different characteristics.

1. Relational Data
    - Most well-understood model for holding data.
    - Data normalization helps to reduce any data redundancy within the database.

2. Non-relational Data
    - Store data in a format that more closely matches the original structure.
    - Data duplication present which increases the storage required.
    - Due to data duplication, any data modification may cause to update data present at multiple locations.

👉Two different types of workloads.

1. Transactional workloads
    - Transaction is a sequence of operations that are atomic.
    - Mostly commonly use relational databases.
    - A transactional database must adhere to the ACID.
        1. Atomicity = A transaction is treated as a single unit, which either succeeds completely, or fails.
        2. Consistency = A transaction can only take the data in the database from one valid state to another.
        3. Isolation = Concurrent execution of transactions leaves the database in the same state that would have been obtained if the transactions were executed sequentially.
        4. Durability = Once a transaction has been committed, it will remain committed even if there's a system failure.

2. Analytical workloads
    - Read-only systems that store vast volumes of historical data or business metrics.
    - Used for data analysis and decision making.

---

## Describe the difference between batch and streaming data

👉Data processing is converting data into meaningful information.

👉There are two types of data processing.

1. Batch Processing
    - New data elements are collected into a group and the whole group is then processed at a future time as a batch.
    - Data Scope = Process all the data in the dataset.
    - Data Size = large datasets.
    - Performance = latency is a few hours.
    - Analysis = performing complex analytics.
2. Streaming and real-time data
    - In stream processing, each new piece of data is processed when it arrives.
    - Beneficial for dynamic data.
    - Ideal for time-critical operations that require an instant real-time response.
    - Data Scope =  Access to the most recent data received.
    - Data Size = Individual records or micro batches.
    - Performance = latency in the order of seconds or milliseconds.
    - Analysis = simple response functions.