hpcc-systems/HPCC-Platform — Issue #879: PromoteSuperFileList record count for datasets
Author: rengolin
Created: 2011-11-01 16:00:20+00:00
State: open

Reproducing bug 78420 here for easy access:

FileServices.PromoteSuperFileList does not correctly populate record count for datasets with variable length fields in addhead parameter

FileServices.PromoteSuperFileList is not properly populating the record count
(in dali?). This is annoying when looking at the superfile though ECLWatch and
fatal when trying to deploy the superfile to Roxie. Also, the owner field is
not being populated either.

At the bottom of this descriptionm I have created an ECL code example that
compares what happens when a PromoteSuperFileList operation is done
('...superfile1') vs. when an AddSuperfile operation is done ('...superfile2').
The result of the LogicalFileList operations show (I removed the 'modified'
and 'cluster' fields):

name size rowcount owner
superfile_promotesuperfilelist_test::superfile1 true 8 -1
superfile_promotesuperfilelist_test::superfile2 true 8 1 cellajr

The rowcount for superfile1 displays as 0 through ECLWatch. (Let me know if you
want a screen shot.)

The sample code below was created on a dev environment, but should be reproducible on
any system. This envirnonment is currently on build 702_32, but I have seen
the problem in earlier 702 builds (such as 702_16b).

// ----- Start ECL Code ---------------
file_prefix := '~superfile_PromoteSuperFileList_test::';
my_superfile1 := file_prefix + 'superfile1';
my_superfile2 := file_prefix + 'superfile2';
my_subfile := file_prefix + 'subfile';

rec := record
string word {maxlength(40)};
end;

sequential(
FileServices.DeleteSuperFile(my_superfile1,true),
FileServices.DeleteSuperFile(my_superfile2,true),
output(dataset([{'word'}],rec),,my_subfile,overwrite),
FileServices.PromoteSuperFileList([my_superfile1],my_subfile),
if(FileServices.SuperFileExists(my_superfile2),
FileServices.ClearSuperFile(my_superfile2,true),
FileServices.CreateSuperFile(my_superfile2)),
FileServices.StartSuperFileTransaction(),
FileServices.AddSuperFile(my_superfile2, my_subfile),
FileServices.FinishSuperFileTransaction(),
output(FileServices.LogicalFileList(file_prefix[2..] + '*',,true))
);
// ----- End ECL Code ---------------
Note, if I change the record layout in the example to:

rec := record
string word {maxlength(40)};
end;

and the dataset created to:

dataset([{1}],rec)

then the rowcount is populated correctly (although the owner is still empty).