# A Distributed File-Based Storage System for Improving High Availability of Space Weather Data
## 4 Implementation
### File Distribution  
The IPFS relies on the BitSwap protocol to exchange blocks between nodes and a distributed hash table (DHT) to store the pointers of peers who have actual locations of files and node information [22]. Basically, our proposed system was developed from the existing system and focused only on changing the storage system model. In the IPFS configuration, there was a useful parameter, called _storageMax_, to set the upper limit for the size of the IPFS repository that was stored in our local disk. The IPFS-Cluster established a priority for block replication based on the maximum free space of the peer. Therefore, setting the configuration of _storageMax_ for storing the IPFS repository on the peer leader as the highest level was considered. This peer must store all blocks from all nodes due to having the primary user and being located in the main data center. Next, the configuration of _storageMax_ was set by selecting other peers, which collected the lowest size of total daily data, considering which peers had a more stable connection. Thus, the system would execute a sequence to replicate them according to those configurations. A file that was uploaded to the IPFS network would map into an object or called block with fixed size using a combination of a hash function and base encoding. The SHA256 hash function had been applied and made the size of each block at most 256 KB because the IPFS uses the Rabin fingerprint method for chunking files. The Rabin fingerprint method uses content-defined chunking that identifies each fingerprint based on a 20-byte Secure Hash Algoritms (SHA) hash value [23]. The content calculation was used to create a hash name for each block, so no duplication occurred with the same content stored at the node. Each file that was uploaded to the IPFS had each hash name as a content identifier (CID) in the network. The data structure in this system used a directed acyclic graph (DAG), forming Merkle trees based on the CID that interconnected objects as linked hashes. A file that was added to the IPFS might become one block, or several chunks of blocks, depending on its size. This means that a file smaller than 256 KB would become a block with one unique hash name that has the size like its original. If a block had a size of more than 256 KB, it was broken into several chunks of the block according to its total size. Data was uploaded from a January folder (Jan.) from the SN-4000 instrument containing the folders dated from 1 January to 31 January; each date's folder also included files with sizes greater than 256 KB. All blocks derived from the root of the block had a different hash name linked as a Merkle tree of the block. Moreover, if we investigated its data structure, the Link part of that file contained all chunks of block hashes that were connected to the root hash with a maximum size of 256 KB (Figure 9). This system relied on the DHT and BitSwap protocol for the block exchange process. DHT provides two essential functions [24]. Peer routing, which is used to identify other peers on the network based on PeerID (nodeID). This information only listed the online node to be announced to others. Another function is content routing, which serves information about data published by nodes and records file locations. When a peer queried the DHT based on a requested hash, the system would check the DHT to determine whether the requested hash was referenced or not. If not, these peers would get the data directly from DHT, and the system would update the DHT. Otherwise, the DHT would route the requester and connect to the peer that stored the hash (provider) to obtain the hash from the provider. Finally, the system stored new records in the DHT. BitSwap manages two processes: the first acquires a set of blocks (_want_list_) that have been requested by a peer, and the second sends a set of blocks (_have_list_) that the peer requests.