## 5 Cross-Referencing Bloom-Filter-Based Data Integrity Framework
### Implementation Details  
The communication sequence and the messages that are exchanged between data transfer endpoints are illustrated in Figure 6 and Listing 1, respectively.  
When data transfer is initiated, the following steps occur:  
1. The source initiates a connection to the sink by sending a MSG_CONNECT request. Upon successful connection, the sink responds with a MSG_SUCCESS message.
2. The source creates a list of files that need to be transferred.
3. If the transfer is resumed from a failed transfer, then go to Step 12.
4. For each file, the source end sends a MSG_NEWFILE request to the sink end. The request message contains file metadata information. The sink end opens the file  
Figure 6: Communication sequence of the CRBF-based integrity framework.  
specified in the request message and responds with a MSG_FILEID message. The response message includes the file descriptor used in the sink end.
5. The source queries the DLBF to see if the object has already been transferred. If it has not, the source initiates the object transfer by sending a MSG_NEWOBJ request to the sink. Upon receiving the data, the sink writes the object data to the PFS. Once the object has been successfully written, the sink computes the block hash and compares it to the hash that was received in the MSG_NEWOBJ request. If the hashes match, the sink responds with a MSG_OBJSYNC message.
6. If the object integrity verification is successful, the file-based DLBF is aggregated at both the source and sink endpoints. However, if the verification fails, the source endpoint marks the object as failed and schedules it for re-transfer.
7. Steps 5 and 6 are repeated for all file objects.
8. Upon receiving all objects of a file, the sink endpoint validates the integrity of the file by comparing the computed file hash with the file hash received from the source endpoint. If the hashes match, the sink endpoint acknowledges the completion of the file transfer by sending a MSG_FILECLOSE message.
9. If the file integrity verification is successful, both the dataset Bloom filter (DSBF) and the metadata Bloom filter (MDBF) at both ends of the data transfer are aggregated with the file information. However, if the file integrity verification fails, the source endpoint schedules a re-transfer of the file.
10. For each file in the dataset, Steps 3 to 9 are repeated.
11. After transferring all files in the dataset, the source endpoint performs a dataset integrity verification. If the verification is successful, it sends a MSG_DISCONNECT message to the sink endpoint. However, if the verification fails, the source endpoint repeats Steps 2 to 11.
12. For each file, the source endpoint queries the DSBF to check whether the file is already transferred and whether the file integrity is maintained.
13. If the DSBF query result is negative, then go to Step 4.
14. If the DSBF query result is positive, the source endpoint issues a FILE_CHECK request with the file metadata. The sink endpoint queries the sink-end MDBF and responds with CHECK_RSP by including the MDBF query result.
15. If the sink-end MDBF query result is positive, the source endpoint queries the source-end MDBF. If the result is positive, the file is considered successfully transferred and skipped from the transfer. Otherwise, go to Step 4.