Experiment with separately weighting the empty and filled similarity.