hpcc-systems/HPCC-Platform — Issue #1816: Superfiles replication issue
Author: jakesmith
Created: 2012-03-09 15:29:01+00:00
State: open

If subfiles are mixed, some replicated, some not.
The super file will appear to be unreplicated when dealing with the file descriptor.

Consequently, if any primary is missing, even on a subfile that is replicated, the engines will not see it/try to read it.

To avoid this the engines, could avoid using the super IFileDescritpors where most of the subflle meta info has been lost, Or pull out specific info from each sub, but that would be shame and make supers hard to deal with + make them less efficient to serialize to all slaves etc.

At the moment, we preserve some of the meta info of supers in a super file descriptor (ISuperFileDescriptor), but attribute information from the subfiles has been lost.

Needs some more thought, but it would be nice if ISuperFileDescriptor allowed access to subfile information.
On the whole most of it will be identical, so only need [internally] hold the differences to the super attributes, but provide access to a representation of each subfiles attributes.

@rengolin, @richardkchapman