Cache vertex similarity and calculate both within cluster and between clusters simiarity base on it.