These two factors sets a upper limit on how many clients a single cache can handle, especially the filedescriptor limit.
Fault tolerance is also a issue.
If you have one single large cache, then you have a single point of failure of the service for all clients. Another issue is upgrade path. It is feasible if the cache can be upgraded in incremental steps when the demand increases. To overcome this one can deploy multiple caches. Clients distributed using some kind of load balancing.