Don't allow panning if one or fewer facets

There is no point in rotation/panning if there is only one face.