At any rate, the country vicarage had retreated into the background again.
The country vicarage is always in the foreground and never retreats. 
mnli hypothesis: The country vicarage is always in the foreground and never retreats.  premise: At any rate, the country vicarage had retreated into the background again.