Only need the Vary: Accept when there are multiple formats

The Vary: Accept header tells caches that the result depends on content
negotiation and so to store one copy per format. We only need this when
we have multiple formats.