It was probably my idea, but another problem with the old approach is that if another object occluded the backfaces but not the front, this object would not get picked (or picked first with drill picking).