Research on massive data storage in virtual roaming system based on RAID5

A massive data management method was proposed based on RAID5 against limitations of data expansion, and a lack of automatic data recovery capability in current virtual roaming system: the panoramic data is separated from the roaming system as an independent panoramic data service system (PDSS), and distributed to multiple nodes to performance parallel service taking advantage of disk parallelism of RAID5. In this way, we can enhance the data storage capacity and fault tolerance of the server in a large extend. Finally, a prototype virtual roaming system was designed based on PDSS. The results showed that this method had a good application prospect. Introduction Virtual roaming technology based on the Panorama and combined with the advantage of GIS is an important approach to realize the visualization of GIS. With the continuous areal expansion covered by the virtual roaming and greater detailed panorama spatial information, the storage nodes in data center under the massive data storage environment are very large,its efficiency and fault tolerance influence the efficiency and stability of virtual roaming system. It is difficult to meet the practical demands , though there are some solutions such as data compression or the improvement of the broadband to release the pressure of the server. These methods are not the best way to solve these problems, because the P2P structure has a large number of nodes and complex encryption algorithm.Redundant Arrays of Independent Disks(RAID) puts a plurality of disks together to form a unified logical storage device, the common methods of which were disk imaging, striping and error correction. For example, RAID5 scatters data stored in different disks arrays for the purpose of massive storage and high transmission rate, data security and storage cost, so it has been widely used . In this paper, a new method was proposed to solve the massive data problem in virtual roaming system based on RAID5. Data storage structure and fault-tolerance Mechanism Taking one section of the road for example, supposing that there are i panoramic data and n isomorphic corresponding storage units, the throughput is k. The loading time is differ under the same network due to the different size of the panoramic data, named 1 2 , ,. n t t t , the whole loading time is 1 2 , ,. n T t t t = ; the average number of the requesting access is λ, theoretically, the rejection rate is: 1 (1 ) r r N T k T r λ = = − + × (1) Nr represents the number of data rejections, Tr the total access time. Therefore, to reduce the rejection rate ρ, we need to reduce T. If the panoramic data are stored on n different storage units, the service time of each units would reduce to T/n, the acceptance successful request rate would rise from k/(1+λ×T) up to k/(1+λ×T/n), which means that the actual parallelism of the application server is nearly n times compared to single disk.Hence, in order to improve data services performance, a server structure was adopted which is similar to the features of RAID5 . As is shown in Fig1, the panoramic data is separated from the roaming system and distributed to multiple nodes to establish a data server connected by high speed network between service nodes and control nodes. The service nodes are used to dynamically store panoramic data, while the control nodes International Conference on Advances in Mechanical Engineering and Industrial Informatics (AMEII 2015) © 2015. The authors Published by Atlantis Press 1408 used to response to the request from the roaming system to assign the service node. Data nodes store and send the data to service nodes via the high speed network according to the instruction from the service nodes. Fig.1 Structure of clustered parallel panoramic data service system The Fig.2 shows the panorama data unit and the parity unit block. In each row, P represents parity section information, and the data in each node are stored in the remaining modules. The panorama data are divided into several StripeUnit, and a row data unit and verification units are combined into one stripe. The local panorama data are stored in data unit, and the parity unit which is useful in data recovery is used to store information on other nodes. We take advantage of random allocation algorithm to assign the panorama data, in other words, the storage unit are deposited to each node through a pseudo random function to solve the problem of node overload. Fig.2 Storage configuration for parallel panoramic data The data and the system are often combined together in traditional virtual roaming system which will reduce the stability and fault tolerance, that is to say the whole system will be paralyzed once fatal error or disk damage occurs on the local server. EVENODD code [10] is a dual fault-tolerance coding algorithm which is commonly used in array storage system. If the error occurs on any two disks, the redundant information of two disks in the system will be increased to recover the lost data. EVENODD code is based on the array of (p-1)×(p+2), and the first P columns is used to store raw data, and the last two columns which is the checking column is used to store the redundancy checking data. di,j(0≤i≤p-2,0≤j≤p+1) represents the line i data of disk j, The column P is called for row-best checking column. The checking block di,p was gained by exclusive OR operation with all the original data in the line i; The p+1 column is diagonal-best checking list, we can get the checking block through the exclusive or between the regulators and the corresponding raw data blocks. The structural formula is as follows: