Cleanup config cache
Consider number of layers in a project can be higher than 100 for caching