Filter out buttons which aren't the submitter while constructing the dataset