Collecting joint representations for the validation set