HPCC-10620 : Ensure logical file meta data is checked

Regression in HPCC-8624, caused the logical file meta info the
query supplies to no longer to be checked against the published
DFS meta info. i.e. formatCRC, compression, grouping

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-10594 : Block access to foreign dali from logicalfilelist

Fileservice calls can bypass the local Dali (and therefore the
check from slaves) and go directly to a foreign Dali.
This is highly undesirable, for the same reason it was blocked
by default from the local environment's Dali. It causes a flood
of requests that can usually be recoded more efficiently by
ensuring the request is called once or outside slaves.

LogicalFileList was the only call that in FileServices that
directly called a DFS call that sent a message directly.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-10517 Fix Package subfile nodes sometimes added to DALI as empty

Signed-off-by: Anthony Fishbeck <anthony.fishbeck@lexisnexis.com>
Merge pull request #5269 from afishbeck/emptyPackageSubfileNodes

HPCC-10517 Fix Package subfile nodes sometimes added to DALI as empty

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-10610 Add DBGLOG trace of call into createSingleFileClone

Signed-off-by: Anthony Fishbeck <anthony.fishbeck@lexisnexis.com>
Merge pull request #5270 from afishbeck/createSingleFileCloneTrace

HPCC-10610 Add DBGLOG trace of call into createSingleFileClone

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #5260 from jakesmith/hpcc-10620

HPCC-10620 : Ensure logical file meta data is checked

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #5263 from jakesmith/hpcc-10594

HPCC-10594 : Block access to foreign dali from logicalfilelist

Reviewed-By: Gavin Halliday <gavin.halliday@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-10592 Roxie needs to cache dali lookups that fail to match

Add some tracing

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #5272 from richardkchapman/roxie-lookup-tracing

HPCC-10592 Roxie needs to cache dali lookups that fail to match

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-10650 createSingleFileClone don't remove files unless changed

When overwrite is specified, cloning a remote file should not delete
a local instance unless the file has changed in a significant way.

cloneFrom info should be updated in case the user is trying to change
where the file gets copied from (source dfs ip, cluster, etc).

Signed-off-by: Anthony Fishbeck <anthony.fishbeck@lexisnexis.com>
HPCC-10651 Roxie dali cache ineffective on subfiles resolved via package

The cache was holding IResolvedFile objects, but the CResolvedFile for the
superfile was holding on to the IDistributedFile from the subfile, not the
IResolvedFile that contained it, so the subfiles kept getting added to the
cache then removed again as they were released.

Also adds some tracing.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #5274 from richardkchapman/roxie-dali-cache-again

HPCC-10651 Roxie dali cache ineffective on subfiles resolved via package

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Merge pull request #5273 from afishbeck/cloneOverwriteChangedOnly

HPCC-10650 createSingleFileClone don't remove files unless changed

Reviewed-By: Gavin Halliday <gavin.halliday@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge branch 'closedown-4.2.x' into candidate-4.2.2

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Community Edition 4.2.2-rc6 Release Candidate 6

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge remote-tracking branch 'origin/candidate-4.2.2' into closedown-4.2.x

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>