#9435 - enforce rules for using EC/tiered pools with CephFS

Why are we checking for read tier here? EC pools support random reads; we need the cache to absorb our writes.