Make request bodies caching configurable, close #1237