wangkx/HPCC-Platform — comment on commit e464501e64b58a095796dd876f9efccce72770d7
File:  line 0
By wangkx:

@richardkchapman / @jakesmith :

This is my attempt to fix one problem for gh-560. Please take a look and let me know whether I am in correct direction and if any mistake.

As I mentioned in my comments for gh-560, I think that ECLwatch does have the function to delete a file from a thor (or roxie) while keeping the same file for a roxie (or thor). But, the existing code does not work if both roxie and thor runs in the same computer. After the replicated file is deleted and I re-do the replicate, the second file is created, but, the state of the DFU WU shows 'failed'. By taking a look inside the data folders, I found that the original file (for example, in hpcc-data/thor and hpcc-mirror/thor) is also deleted when I delete the replicated file (in hpcc-data/roxie). I traced into the code for deleting the replicated file . When the code (in dali/base) is deleting a file for a cluster (for exmple, mythor), it filters while files to be deleted based on IP. Since both mythor and myroxie have the same IP, the files in both hpcc-data/thor and hpcc-data/roxie are deleted.

I am trying to fix that problem using the code here. Inside the removePhysicalPartFiles(), I suggest adding dirfilter in addition to grpfilter. The dirfilter contains the BasDir and ReplicateDir for the specified cluster. In the casyncfor::Do(), we check whether the filePath is in the dirfilter or not. If not in, the file should not be removed. After the requested files are deleted, we may need to update the directory attribute for the logical file. Inside the CFileClusterOwner::saveClusters(), we may update the directory using the BaseDir of the first cluster (which is myroxie) after the selected cluster (which is mythor) is removed.

I just tested my new code in my local environment (one thor and one roxie in the same PC) and looks like the code works.