# Smart Grid Data Management in a Heterogeneous Environment with a Hybrid Load Forecasting Model
## 4 The Proposed System
### Data Tier Structure  
The data tier consists of structured and unstructured databases. The structured database will be concerned with many processes such as historical data cleaning and pre-processing. Aside from that, online data that can directly be injected by API Gateway, as shown in Figure 1, will be dedicated to subscriber information management only. In contrast, the second part of the data tier contains the time-series database (No-Sql) unstructured database where the data are stored and retrieved in both hot and cold form from the API for all objects connected to our systems. Data are kept independent from the analytics, application, and presentation tiers [46]. The data tier works on the data  
Figure 1: PIAS Tiers Diagram.  
persistence mechanisms that include database servers, file shares, etc. Data persistence mechanisms are enclosed by the data access layer that reveals the data. The data access layer provides an API to the application tier that shows the methods of managing stored data without revealing or creating dependencies on the data storage mechanisms. This allows the data tier to be updated or changed without affecting the application tier clients [47]. However, the cost is incurred by the separation of tier to bring further scalability and maintainability improvements. These costs include the model's implementation costs and operational costs that ensure the model's higher accuracy and efficiency. The proposed framework was built using an SQL server database that employs the following operations to operate the proposed structure. These processes allow the data to be ready for further processes in the analytics tier and application tier, so the proposed model can achieve its objectives and is easily scalable to more resources and information. The following steps have been applied in the data tier using the structured query language (SQL):  
* Step 1: Database Scheme creation: Structure definition, data format, and correlation among the tables
* Step 2: Data Acquisition: The collected data can be inserted into the database using many techniques, such as API Gateway, links to other databases as well as direct import to the data files, and direct reading from smart meters through PLC, the Data Concentrator Unit (DCU), and GPRS, in addition to reading offline data (mechanical meters) through the PIAS mobile application, as shown in Figure 2a,b. This mobile app is designed to work offline, connected to our system through API Gateway with the capability of reading the meter's value from the image of mechanical meter measurement. The identified measurement value is transferred to a digital value and stored in the mobile devices which were used at the time of reading. Then, the stored data are sent along with the unique ID of each subscriber and meter ID to the PIAS through API Gateway when the mobile device is connected to any internet network. This will effectively reduce human errors in reading the values and reduce the costs required for this process.
* Step 3: Data manipulation: Create, read, update, and delete (CRUD) operations of any data from the database.
* Step 4: Querying: Retrieval of stored data to be used by the analytics and application tier.
* Step 5: Integration of security modules: Authorizing access to the data and ensuring which data to reveal.  
Figure 2: _Cont_.