Yes. For example if it was a four-player game and there was a white six on the silver tray, all three passive players could choose that same die as their choice for scoring.