Avoid repeatedly checking specific post format.

Will only check for a specific post format if it’s a post format
archive page.

See https://core.trac.wordpress.org/changeset/30854
Fixes #664.