TestRunner: Add option to keep existing caches intact when running tests

Change-Id: I09fc724c47ddf01444fc603cbd14fdb032ec0c14