# A Multidata Connection Scheme for Big Data High-Dimension Using the Data Connection Coefficient

Authors: Yoon-su Jeong; Seung-soo Shin
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

In the era of big data and cloud computing, sources and types of data vary, and the volume and flow of data are massive and continuous. With the widespread use of mobile devices and the Internet, huge volumes of data distributed over heterogeneous networks move forward and backward across networks. In order to meet the demands of big data service providers and customers, efficient technologies for processing and transferring big data over networks are needed. This paper proposes a multidata connection (MDC) scheme that decreases the amount of power and time necessary for information to be communicated between the content server and the mobile users (i.e., the content consumers who are moving freely across different networks while using their mobile devices). MDC scheme is an approach to data validation that requires the presentation of two or more pieces of data in a heterogeneous environment of big data. The MDC transmits the difference of the consecutive data sequences instead of sending the data itself to the receiver, thus increasing transmission throughput and speed.


---

## Body

## 1. Introduction

The increase in data sources, including social networks and mobile devices, has brought the exponential growth, availability, and use of information [1, 2]. This trend to larger data sets, commonly referred to as big data, entails new technologies and practices of collecting, storing, and analyzing data sets as traditional data processing applications cannot handle them efficiently. Advanced big data technologies can uncover hidden values or additional information from large volumes of data, providing more predictive, tailored information that can lead to substantial innovation gains [3, 4].Although the immense amount of data is generated on a regular basis via a variety of channels (e.g., business processes, machines, and human interaction with things like social media sites, and mobile devices), information services reflecting specific individual and community needs and interests are scare. This indicates that technologies that can uncover the insights and meaning of the underlying data are essential to gain value from big data.Big data implies enormous volumes of data. For example, a single data file in scientific applications is of terabyte size and it can take hours, days, or even longer to traverse the network. However, volume is not the only aspect that characterizes big data. Big data is often defined along three dimensions—volume, velocity, and variety [5]. That is, big data is high volume, high velocity, and high variety information assets.Big data has the power to radically change every aspect of our lives, including politics, economy, culture, and science. While it provides new opportunities for prosperity and innovation, there is always the question of privacy or security, particularly with customer data [6]. Online searches, store purchases, Facebook posts, cell phone usage, and so forth are creating a flood of data that when organized and analyzed, reveals trends and habits about individuals and society at large. This is why big data is sometimes nicknamed “modern day version of Big Brother.” The threats of data breach and leakage need to be understood and protected through the use of appropriate security controls.Big data is being transmitted over communications networks with increasing frequency, causing the problem of machine and communication overhead. The transmission of huge data files is not the only challenge. Real-time data transmission is also becoming increasingly commonplace. For example, mobile users may request big data services using their smartphones while moving across different networks. An efficient technique that can process and move large amounts of data quickly and easily in such a setting is needed.The MDC scheme proposed in this paper minimizes the time necessary for information to be communicated between the content server and the mobile user so that the user moving across networks can receive big data services without interruption. MDC scheme is an approach to data validation that requires the presentation of two or more pieces of data in a heterogeneous environment of big data. The MDC transmits the difference between the consecutive data sequences (the locations of the data in the content server, more precisely) rather than the data itself. This allows achieving low latency and reliable throughput, thus avoiding interruptions in service. In addition, the MDC delivers sufficient random bits to be used for validation checks.The remainder of the paper is organized as follows. Section2 describes the definition and characteristics of big data. Section 3 presents the proposed MDC scheme. In Section 4, the performance of the proposed scheme is evaluated. The final section provides concluding remarks and directions for future research.


## 2. Related Work

### 2.1. Big Data

Big data refers to the expanding volume of high velocity, complex, and diverse types of data, both structured and unstructured [1]. Big data comes in the form of emails, photos, monitoring devices, audio, cell phone GPS signals, and many more. With the widespread use of mobile devices and the Internet, data is routinely generated by human interaction on systems like social media, causing the exponential growth of the volume of data to be analyzed. The advance of machine to machine (M2M) also accelerates the exponential growth of data volumes [7, 8].In today’s world, data comes from everywhere and it is different from the data in conventional database systems in many aspects. Online content produced by Internet users, commonly known as the UCC or UGC, is an example of the unstructured data that creates problems for storage, mining, and analyzing data. With the available data on social media platforms, it is possible to know many personal things about individuals (e.g., demographic information, preferences, purchase habits, and friends). CCTVs that are installed almost everywhere (e.g., roads, buildings, and inside the residence elevator) are gathering huge volumes of video data every day. Along with the private sector, the public sector is generating and managing large amounts of data associated with census, health and welfare services, national pensions, and so forth [9].


### 2.2. Characteristics of Big Data

Table1 shows the four major elements of big data—volume, velocity, variety, and complexity. Today, big data is recognized as a government and national asset in many countries but it requires innovative forms of information processing for enhanced insight discovery, decision making, and process optimization. The size of the data sets within the data analysis and velocity with which they need to be analyzed has outpaced the current abilities of standard tools and methods of analysis.Table 1
Four elements of big data.
Element
Descriptions
Volume increases
(i)  Digital information is growing exponentially along with the advance of IT(ii) Enter the zettabyte era
Variety increases
(i) Data collection can be from varied sources and forms (e.g., activity logs, text posted on social media, spatial data, and physical event data)(ii) The diversity of unstructured data grows (e.g., multimedia data)
Complexity increases
(i) Data that does not fit the structures of formal database architectures, incompatible data storage formats, redundancy, and so forth(ii) The complexity of data management grows due to the diversity of data types, forms, and sources (iii) Advanced techniques are required to efficiently manage and process the data of high complexity
Velocity increases
(i) The increase of real-time data (object data, sensor monitoring data, streaming data, etc.)(ii) The speed of data generations and flows are fast-paced(iii) Processing and analysis speed is vital to deal with huge volumes of data and to provide volatile information in timeTechnologies such as distributed storage, parallel processing, and cloud computing allow fast processing and analysis of massive data sets. With such technologies, industries and digital marketers can gather immediate feedback from tweet updates, Facebook comments, or social media discussions and predict and analyze their business circumstances within a tolerable elapsed time. Thus, companies can provide a more appropriate and prompt response to the customers, increasing their chance of success.In the market, there are open source and/or free software solutions and distributed, scalable, and flexible hardware infrastructures for the management and analysis of large quantities of data. For example, Hadoop, R packages, and cloud computing are popular big data technologies. That is, it is possible to realize benefits from the use of big data without procuring expensive storages or a data warehouse [4, 10].


## 2.1. Big Data

Big data refers to the expanding volume of high velocity, complex, and diverse types of data, both structured and unstructured [1]. Big data comes in the form of emails, photos, monitoring devices, audio, cell phone GPS signals, and many more. With the widespread use of mobile devices and the Internet, data is routinely generated by human interaction on systems like social media, causing the exponential growth of the volume of data to be analyzed. The advance of machine to machine (M2M) also accelerates the exponential growth of data volumes [7, 8].In today’s world, data comes from everywhere and it is different from the data in conventional database systems in many aspects. Online content produced by Internet users, commonly known as the UCC or UGC, is an example of the unstructured data that creates problems for storage, mining, and analyzing data. With the available data on social media platforms, it is possible to know many personal things about individuals (e.g., demographic information, preferences, purchase habits, and friends). CCTVs that are installed almost everywhere (e.g., roads, buildings, and inside the residence elevator) are gathering huge volumes of video data every day. Along with the private sector, the public sector is generating and managing large amounts of data associated with census, health and welfare services, national pensions, and so forth [9].


## 2.2. Characteristics of Big Data

Table1 shows the four major elements of big data—volume, velocity, variety, and complexity. Today, big data is recognized as a government and national asset in many countries but it requires innovative forms of information processing for enhanced insight discovery, decision making, and process optimization. The size of the data sets within the data analysis and velocity with which they need to be analyzed has outpaced the current abilities of standard tools and methods of analysis.Table 1
Four elements of big data.
Element
Descriptions
Volume increases
(i)  Digital information is growing exponentially along with the advance of IT(ii) Enter the zettabyte era
Variety increases
(i) Data collection can be from varied sources and forms (e.g., activity logs, text posted on social media, spatial data, and physical event data)(ii) The diversity of unstructured data grows (e.g., multimedia data)
Complexity increases
(i) Data that does not fit the structures of formal database architectures, incompatible data storage formats, redundancy, and so forth(ii) The complexity of data management grows due to the diversity of data types, forms, and sources (iii) Advanced techniques are required to efficiently manage and process the data of high complexity
Velocity increases
(i) The increase of real-time data (object data, sensor monitoring data, streaming data, etc.)(ii) The speed of data generations and flows are fast-paced(iii) Processing and analysis speed is vital to deal with huge volumes of data and to provide volatile information in timeTechnologies such as distributed storage, parallel processing, and cloud computing allow fast processing and analysis of massive data sets. With such technologies, industries and digital marketers can gather immediate feedback from tweet updates, Facebook comments, or social media discussions and predict and analyze their business circumstances within a tolerable elapsed time. Thus, companies can provide a more appropriate and prompt response to the customers, increasing their chance of success.In the market, there are open source and/or free software solutions and distributed, scalable, and flexible hardware infrastructures for the management and analysis of large quantities of data. For example, Hadoop, R packages, and cloud computing are popular big data technologies. That is, it is possible to realize benefits from the use of big data without procuring expensive storages or a data warehouse [4, 10].


## 3. Multidata Connection Scheme

As the volumes and types of data in clouds increase, techniques for efficiently handling big data stored in heterogeneous devices of different networks are required. The MDC scheme proposed in this paper increases the throughput of cloud-based big data by predicting the data to be sent at later times and sending the difference of the consecutive data sequences, not the data itself, to the receiving node.


### 3.1. Overview

The MDC scheme transfers the difference between the consecutive data members, rather than the data member itself, in order to increase throughput and speed. Letm[k] be the data member of a data set transmitted in a big data environment. Instead of sending m[k] to the receiver, the MDC sends d[k], the difference of the two consecutive data members, as represented in (1). k denotes the number of data members in the data set transmitted in a big data environment:(1)dk=mk-mk-1.Whend[k] and the previous data member m[k-1] are known, m[k] can be derived. Thus, the receiver is able to create m[k] with the received difference d[k].As presented in Figure1, the MDC increases throughput by predicting m[k], the kth data member. Given that an estimate of m[k] is m˙[k], the difference with a data estimation error, that is, d[k]=m[k]-m˙[k], will be transferred. The receiver predicts m˙[k] with the past data members and creates m[k] by adding the received d[k] to the predicted m˙[k].Figure 1
A multidata connection structure using the prediction coefficient.Suppose that the all-order derivatives of the data member at a random time pointt, m[t], are known. Using a Taylor series, m(t+TS) is expressed in (2). TS denotes the time at which server S generates the data member:(2)mt+TS=mt+TSm˙t+TS22!m˙t+TS33!m˙t+⋯(3)≈mt+TSm˙tfor smallTS.In (2), if the data member and its derivative values at time t are known, the data member at a later time t+TS can be predicted. Even if only the first derivative is known, an approximate estimation of the future data member is possible, as expressed in (3). Let m[k] be the kth data of M(t). If m(kTS)=m[k], then m(kTS±TS)=m[k+1]. With t=kTS in (3), m˙[kTS]~[m(kTS)-m(kTS-TS)]/TS is produced. This leads to (4)mk+1~mk+TSmk-mk-1TS=2mk-mk-1.An approximate valuem(t)+TSm˙(t) in (3) is further refined by increasing the number of terms on the right side. To perform the differentials of higher order, more past data are needed. The accuracy of prediction increases as the number of previous data members increases. The prediction is carried out using(5)mk≈a1mk-1+a2mk-2+⋯+aNmk-N.In (5), a1m[k-1]+a2m[k-2]+⋯+aNm[k-N] denotes m˙[k], that is, an estimate of m[k]. m˙[k] is expressed in (6)m˙k=a1mk-1+a2mk-2+⋯+aNmk-N.AsN increases, the accuracy of prediction increases. The data member sent by the server is m[k] and the data member received by the receiver is m˙[k].


### 3.2. Linear Mean Square Error Estimation

When two estimation errorsx and y of the data member of interest are related, one of them can be used to derive the other. That is, y can be estimated based on the knowledge of x. An estimate y^ of y is also likely to have an estimation error, making it not quite the same as the actual estimation error y. The MDC finds the best estimate of the data member estimation error using the minimum mean square error (MMSE) criterion, ε2¯. This is expressed in (7)ε2¯=y-y^2¯.In general,y^, the best estimate of the data member estimation error y, is a nonlinear function of x. Assuming that x^=0, y^ can be restricted to a linear function of x, as represented in (8)y^=ax.ε
2
¯ is computed using (9)ε2¯=y-y^2¯=y-ax2¯=y2¯+a2x2¯-2axy¯.To minimizeε2¯ of (9), (10) is executed. Consider(10)∂ε2¯∂a=2ax2¯-2xy¯=0.a is computed using (11)a=xy¯x2¯=RxyRxx.In the equation above,Rxy=xy¯, Rxx=x2¯, and Ryy=y2¯. By applying a of (11) to (12), y-Rxyx/Rxx is produced. Consider(12)ε=y-ax=y-RxyxRxx.As shown in (13), xε¯ is x(y-(Rxy/Rxx)x)¯=xy¯-(Rxy/Rxx)x2¯. Consider(13)xε¯=xy-RxyRxxx¯=xy¯-RxyRxxx2¯.In (13), xy¯=Rxy and xx¯=x2¯=Rxx. With this, xε¯ is rewritten as (14)xε¯=Rxy-Rxy.Equation (14) is in line with the orthogonality principle. x is orthogonal (“not related”) to ε when the mean square error (MSE) is the minimum.By the orthogonality theorem, (15) computes the MMSE. Consider(15)ε2¯=y-ax2¯=y-axy¯-a·εx¯=y-axy¯=y2¯-a·yx¯=Ryy-a·Rxy.


### 3.3. Estimation of the Data Members

If an estimate of the data member of interest,x0, is related with other n estimates of the data member, x1+x2,…,xN, then x0 can be estimated via the linear summation of x1+x2,…,xN. This is represented in(16)x^0=a1x1+a2x2+⋯+anxn=∑aixi.As expressed in (17), ε2¯ is computed using [x0-(a1x1+a2x2+⋯+anxn)]2¯. Consider(17)ε2¯=x0-a1x1+a2x2+⋯+anxn2¯.To minimizeε2¯ of (17), (18) is performed. Consider(18)∂ε2¯∂a1=∂ε2¯∂a2=⋯=∂ε2¯∂an=0.∂
ε
2
¯
/
∂
a
i takes the derivative of ε2¯ with respect to ai, as represented in (19)∂ε2¯∂ai=∂∂aix0-a1x1+a2x2+⋯+anxn2¯=0.∂
ε
2
¯
/
∂
a
i is rewritten in (20) by rearranging the differential calculus and the average. Consider(20)∂ε2¯∂ai=-2x0-a1x1+a2x2+⋯+anxnxi¯=0.Equation (20) is rewritten in (21)ε·xi¯=0i=1,2,…,n.Equation (21) can be transformed to (22) but the condition in (23) must be satisfied. Consider(22)R0i=a1Ri1+a2Ri2+⋯+anRin,but(23)Rij=xixj¯.By differentiatingε2¯ with regard to a1,a2,…,an and equating the result to zero, n equations are obtained. Constant a1,a2,…,an can be obtained using the inverse matrix, as shown in (24)a1a2⋯aN=R11R12⋯R1nR21R22⋯R2n⋯⋯⋯Rn1Rn2⋯RnnR01R02⋯R0n.In (20) and (21), the error ε is orthogonal to the data member estimates x1,x2,…,xn so as to have the best estimate of the data member. The MSE in this optimum condition is expressed in (25)ε2¯=εε¯=εx0-a1x1+a2x2+⋯+anxn¯.In (25), εxi¯ is 0 (i=1,2,…,n). ε2¯ is rewritten in (26)ε2¯=εx0¯=x0x0-a1x1+a2x2+⋯+anxn¯=R00-a1R01+a2R02+⋯+anR0n.


## 3.1. Overview

The MDC scheme transfers the difference between the consecutive data members, rather than the data member itself, in order to increase throughput and speed. Letm[k] be the data member of a data set transmitted in a big data environment. Instead of sending m[k] to the receiver, the MDC sends d[k], the difference of the two consecutive data members, as represented in (1). k denotes the number of data members in the data set transmitted in a big data environment:(1)dk=mk-mk-1.Whend[k] and the previous data member m[k-1] are known, m[k] can be derived. Thus, the receiver is able to create m[k] with the received difference d[k].As presented in Figure1, the MDC increases throughput by predicting m[k], the kth data member. Given that an estimate of m[k] is m˙[k], the difference with a data estimation error, that is, d[k]=m[k]-m˙[k], will be transferred. The receiver predicts m˙[k] with the past data members and creates m[k] by adding the received d[k] to the predicted m˙[k].Figure 1
A multidata connection structure using the prediction coefficient.Suppose that the all-order derivatives of the data member at a random time pointt, m[t], are known. Using a Taylor series, m(t+TS) is expressed in (2). TS denotes the time at which server S generates the data member:(2)mt+TS=mt+TSm˙t+TS22!m˙t+TS33!m˙t+⋯(3)≈mt+TSm˙tfor smallTS.In (2), if the data member and its derivative values at time t are known, the data member at a later time t+TS can be predicted. Even if only the first derivative is known, an approximate estimation of the future data member is possible, as expressed in (3). Let m[k] be the kth data of M(t). If m(kTS)=m[k], then m(kTS±TS)=m[k+1]. With t=kTS in (3), m˙[kTS]~[m(kTS)-m(kTS-TS)]/TS is produced. This leads to (4)mk+1~mk+TSmk-mk-1TS=2mk-mk-1.An approximate valuem(t)+TSm˙(t) in (3) is further refined by increasing the number of terms on the right side. To perform the differentials of higher order, more past data are needed. The accuracy of prediction increases as the number of previous data members increases. The prediction is carried out using(5)mk≈a1mk-1+a2mk-2+⋯+aNmk-N.In (5), a1m[k-1]+a2m[k-2]+⋯+aNm[k-N] denotes m˙[k], that is, an estimate of m[k]. m˙[k] is expressed in (6)m˙k=a1mk-1+a2mk-2+⋯+aNmk-N.AsN increases, the accuracy of prediction increases. The data member sent by the server is m[k] and the data member received by the receiver is m˙[k].


## 3.2. Linear Mean Square Error Estimation

When two estimation errorsx and y of the data member of interest are related, one of them can be used to derive the other. That is, y can be estimated based on the knowledge of x. An estimate y^ of y is also likely to have an estimation error, making it not quite the same as the actual estimation error y. The MDC finds the best estimate of the data member estimation error using the minimum mean square error (MMSE) criterion, ε2¯. This is expressed in (7)ε2¯=y-y^2¯.In general,y^, the best estimate of the data member estimation error y, is a nonlinear function of x. Assuming that x^=0, y^ can be restricted to a linear function of x, as represented in (8)y^=ax.ε
2
¯ is computed using (9)ε2¯=y-y^2¯=y-ax2¯=y2¯+a2x2¯-2axy¯.To minimizeε2¯ of (9), (10) is executed. Consider(10)∂ε2¯∂a=2ax2¯-2xy¯=0.a is computed using (11)a=xy¯x2¯=RxyRxx.In the equation above,Rxy=xy¯, Rxx=x2¯, and Ryy=y2¯. By applying a of (11) to (12), y-Rxyx/Rxx is produced. Consider(12)ε=y-ax=y-RxyxRxx.As shown in (13), xε¯ is x(y-(Rxy/Rxx)x)¯=xy¯-(Rxy/Rxx)x2¯. Consider(13)xε¯=xy-RxyRxxx¯=xy¯-RxyRxxx2¯.In (13), xy¯=Rxy and xx¯=x2¯=Rxx. With this, xε¯ is rewritten as (14)xε¯=Rxy-Rxy.Equation (14) is in line with the orthogonality principle. x is orthogonal (“not related”) to ε when the mean square error (MSE) is the minimum.By the orthogonality theorem, (15) computes the MMSE. Consider(15)ε2¯=y-ax2¯=y-axy¯-a·εx¯=y-axy¯=y2¯-a·yx¯=Ryy-a·Rxy.


## 3.3. Estimation of the Data Members

If an estimate of the data member of interest,x0, is related with other n estimates of the data member, x1+x2,…,xN, then x0 can be estimated via the linear summation of x1+x2,…,xN. This is represented in(16)x^0=a1x1+a2x2+⋯+anxn=∑aixi.As expressed in (17), ε2¯ is computed using [x0-(a1x1+a2x2+⋯+anxn)]2¯. Consider(17)ε2¯=x0-a1x1+a2x2+⋯+anxn2¯.To minimizeε2¯ of (17), (18) is performed. Consider(18)∂ε2¯∂a1=∂ε2¯∂a2=⋯=∂ε2¯∂an=0.∂
ε
2
¯
/
∂
a
i takes the derivative of ε2¯ with respect to ai, as represented in (19)∂ε2¯∂ai=∂∂aix0-a1x1+a2x2+⋯+anxn2¯=0.∂
ε
2
¯
/
∂
a
i is rewritten in (20) by rearranging the differential calculus and the average. Consider(20)∂ε2¯∂ai=-2x0-a1x1+a2x2+⋯+anxnxi¯=0.Equation (20) is rewritten in (21)ε·xi¯=0i=1,2,…,n.Equation (21) can be transformed to (22) but the condition in (23) must be satisfied. Consider(22)R0i=a1Ri1+a2Ri2+⋯+anRin,but(23)Rij=xixj¯.By differentiatingε2¯ with regard to a1,a2,…,an and equating the result to zero, n equations are obtained. Constant a1,a2,…,an can be obtained using the inverse matrix, as shown in (24)a1a2⋯aN=R11R12⋯R1nR21R22⋯R2n⋯⋯⋯Rn1Rn2⋯RnnR01R02⋯R0n.In (20) and (21), the error ε is orthogonal to the data member estimates x1,x2,…,xn so as to have the best estimate of the data member. The MSE in this optimum condition is expressed in (25)ε2¯=εε¯=εx0-a1x1+a2x2+⋯+anxn¯.In (25), εxi¯ is 0 (i=1,2,…,n). ε2¯ is rewritten in (26)ε2¯=εx0¯=x0x0-a1x1+a2x2+⋯+anxn¯=R00-a1R01+a2R02+⋯+anR0n.


## 4. Experiments

### 4.1. Performance Measures

In the experiments, MATLAB was used to evaluate the proposed MDC scheme withε2¯, the MMSE of data member estimations. For simplification, it was assumed that the data member m[k] has the same attribute function. With this assumption, all data members have the same size and the receiver needs the same storage space for m[k]. If an estimate of the data member m[k] is m˙[k], the difference with an estimation error, d[k]=m[k]-m˙[k], is transmitted.The performance of the MDC was evaluated in terms of data member prediction accuracy measured usingPm[k], the probability of mean squared data prediction error. To evaluate the accuracy of the received difference information, two random nodes ni and nj share the MSEs associated with the data differences. Here, the difference can be between two consecutive data members, denoted as d[1]  (=m[1]-m[0]), and between as many as q (=d[q]). Pm[k] is calculated using (27)Pmk=1-1-1km.k denotes the predicted location of the data member used in the MSE. m denotes the number of data differences used in the MSE. m denotes the size of m˙[k]:(28)Probd¯=j=d-1j-1Pmkj-11-Pmkd-j,(29)Probd¯=j=djPmkj1-Pmkd-j.In the equations above,d- denotes the probability of MSE of m˙[k]. d- chooses m of m˙[k] that satisfies 1≤j≤d. Using j, 1+d-1·Pm[k] is produced. d- of m˙[k] is j that obtains the actual data member m[k] and satisfies 0≤j≤d.To measure the performance of the proposed MDC scheme,ε2¯, m˙[k], and d- were used.


### 4.2. Experiment Results

Figure2 shows the data accuracy in terms of ε2¯ where Pm[k] is 0.5 and k varies form 50,000, 100,000, 250,000, up to 500,000. Compared to the previous scheme that sends the data member itself, the MDC decreases the amount of power necessary for the data to be sent from the server to the receiver by 28%. As the number of data members increases, the throughput and storage space efficiency of the MDC increases.Figure 2
Accuracy in terms of the linear MSE.Figure3 shows Pm[k] of the data members that are randomly selected from the networks of different sizes. The results show that Pm[k] increases as the network size increases. These results were obtained by analyzing many different ε2¯, m˙[k], and d- values. In Figure 3, Pm[k] is proportional to 2/k.Figure 3
The probability of data prediction errors.Figure4 presents the delays in obtaining the data at the receiver based on the probability of prediction errors. Compared to the previous scheme in which the data is sent directly by the server to the receiver, the MDC decreases the delay by 28.6%. This indicates that transferring the data difference instead of the data itself contributes to providing seamless big data services to the end users.Figure 4
Delays in obtaining the data.Figure5 shows the service delays with regard to the number of hops between source (the content server) and destination (the user). In comparison with the previous scheme, the proposed MDC has a constant service delay irrespective of the number of network hops. The MDC reduces the amount of data to be transferred by delivering only the difference of data sequences, and thus it is less affected by different network conditions.Figure 5
Service delays with regard to hop counts.


## 4.1. Performance Measures

In the experiments, MATLAB was used to evaluate the proposed MDC scheme withε2¯, the MMSE of data member estimations. For simplification, it was assumed that the data member m[k] has the same attribute function. With this assumption, all data members have the same size and the receiver needs the same storage space for m[k]. If an estimate of the data member m[k] is m˙[k], the difference with an estimation error, d[k]=m[k]-m˙[k], is transmitted.The performance of the MDC was evaluated in terms of data member prediction accuracy measured usingPm[k], the probability of mean squared data prediction error. To evaluate the accuracy of the received difference information, two random nodes ni and nj share the MSEs associated with the data differences. Here, the difference can be between two consecutive data members, denoted as d[1]  (=m[1]-m[0]), and between as many as q (=d[q]). Pm[k] is calculated using (27)Pmk=1-1-1km.k denotes the predicted location of the data member used in the MSE. m denotes the number of data differences used in the MSE. m denotes the size of m˙[k]:(28)Probd¯=j=d-1j-1Pmkj-11-Pmkd-j,(29)Probd¯=j=djPmkj1-Pmkd-j.In the equations above,d- denotes the probability of MSE of m˙[k]. d- chooses m of m˙[k] that satisfies 1≤j≤d. Using j, 1+d-1·Pm[k] is produced. d- of m˙[k] is j that obtains the actual data member m[k] and satisfies 0≤j≤d.To measure the performance of the proposed MDC scheme,ε2¯, m˙[k], and d- were used.


## 4.2. Experiment Results

Figure2 shows the data accuracy in terms of ε2¯ where Pm[k] is 0.5 and k varies form 50,000, 100,000, 250,000, up to 500,000. Compared to the previous scheme that sends the data member itself, the MDC decreases the amount of power necessary for the data to be sent from the server to the receiver by 28%. As the number of data members increases, the throughput and storage space efficiency of the MDC increases.Figure 2
Accuracy in terms of the linear MSE.Figure3 shows Pm[k] of the data members that are randomly selected from the networks of different sizes. The results show that Pm[k] increases as the network size increases. These results were obtained by analyzing many different ε2¯, m˙[k], and d- values. In Figure 3, Pm[k] is proportional to 2/k.Figure 3
The probability of data prediction errors.Figure4 presents the delays in obtaining the data at the receiver based on the probability of prediction errors. Compared to the previous scheme in which the data is sent directly by the server to the receiver, the MDC decreases the delay by 28.6%. This indicates that transferring the data difference instead of the data itself contributes to providing seamless big data services to the end users.Figure 4
Delays in obtaining the data.Figure5 shows the service delays with regard to the number of hops between source (the content server) and destination (the user). In comparison with the previous scheme, the proposed MDC has a constant service delay irrespective of the number of network hops. The MDC reduces the amount of data to be transferred by delivering only the difference of data sequences, and thus it is less affected by different network conditions.Figure 5
Service delays with regard to hop counts.


## 5. Conclusion

This paper presented the MDC scheme that improves throughput by transferring the difference of the consecutive data sequences, instead of the data itself, in heterogeneous big data environments. The MDC minimizes the data transfer time between the content server and the user so that the user who moves across different networks can receive big data services without interruption. As it sends only the difference of the consecutive data members, high throughput is achieved irrespective of the types, functionalities, and characteristics of the data.In comparison with the previous scheme, it was observed that the MDC decreases the amount of power required to communicate the data by 28%. This processing efficiency increases as the number of data members to be delivered increases, which leads to increased storage space efficiency. The MDC decreases the delay in obtaining the data at the receiving node by 28.6%. In addition, the MDC maintains a constant service delay irrespective of the number of network hops. In the future, the performance of the MDC will be examined in an environment where two or more networks are interacting with, and its resilience against various security attacks will be studied.



---

*Source: 931352-2015-10-19.xml*