## 4 Practical Application of the E-Commerce Platform
### Big E-Commerce Data Platform Construction  
Considering that it satisfies the big data requirements for e-commerce, the Infrastructure-as-a-Service (IaaS) cloud computing platform was considered. Hardware services, including virtual machines and storage, offer dependability, scalability, cost-effectiveness, and service provision. For this implementation, the platform was hosted on Google Cloud computing. Six machines were used to construct a big e-commerce-data platform cluster. There was one master node and five worker nodes in the cluster. Each worker node has four vCPUs and 15 GB of RAM, running a 64-bit Linux operating system, whilst the master node has eight vCPUs. A secure shell (SSH) is used to create a secure encrypted connection between cluster nodes. E-commerce data resources are proactively transferred to the master node's IP address once a secure connection has been made. It should be noted that providing a secure operating environment is a continuous process; however, this is beyond the scope of this study. All five worker node machines ran CentOS Linux and were deployed on the Hadoop platform. As for the remaining master machine running the Windows OS for performing the analytical tasks. The IP address and host names of the nodes were identified at each node in the respective host configuration. Figure 3 presents the cluster setup, and the HDFS distributes file blocks among the cluster nodes. The work involved in any analysis task is distributed to worker nodes. Each involved node runs the assigned task on its block from the file, and the results are collated and digested into a single result after each involved block has been analyzed.  
Figure 3: Cluster setup and HDFS distributes file blocks among cluster nodes.  
To construct the big e-commerce data platform, the following components were set up on the cluster nodes:  
* Flume: data acquisition.
* Hadoop platform: comprises the storing component HDFS and processing components MapReduce and resource negotiator Yarn [29].
* The SparkCore [22]: includes components for scheduling, fault recovery, memory management, and interaction with the storage system.
* SparkSQL: facilitates reading, writing, and managing data stored in the data lake repository and querying the data [22; 26].  
The data acquisition component, Flume, is set up on the master node to actively poll data and is responsible for sinking the data into a specified zone in the data lake repository with respect to the data resource. As the e-commerce data need to be organized to avoid a \"Data Swamp\", which is a term used to describe data that have lost processes and standards, and the data became difficult to find, manipulate, and inevitably analyze, multiple Flume agents are configured. Once the data were available for the data lake, the analytical tools were connected to a big e-commerce data platform via a network connection. The analytical tools perform analytical tasks by sending queries through Spark, and the querying components return the outcome to the analytical tool. Thus, the analytical tools work at the top of the big e-commerce data platform. In this implementation, the Cloudera Manager Hadoop distribution was adopted because it provides an easy development environment for practicing users to develop tools for e-commerce-related applications. The Cloudera Manager automates the installation of the required configurations, debugging, Structured Query Language (SQL) database, Cloudera Distributed Hadoop (CDH) agents, and other necessary components for this platform.  
Users can use SparkSQL's built-in libraries (e.g., SQL and Dataframes) or third-party integrations (e.g., Zeppelin [30]). These tools can be leveraged to perform data pre-processing tasks such as missing-values augmentation, PCA, detection, and data cleaning tasks.