Revert "Freeze the testing runtime configs at a latter stage"

This reverts commit 5da3ad341b6b4ffb6c947916dfa47664cef4f1cc.