  We consider distributed caching of content across several small base stations
(SBSs) in a wireless network, where the content is encoded using a maximum
distance separable code. Specifically, we apply soft time-to-live (STTL) cache
management policies, where coded packets may be evicted from the caches at
periodic times. We propose a reinforcement learning (RL) approach to find coded
STTL policies minimizing the overall network load. We demonstrate that such
caching policies achieve almost the same network load as policies obtained
through optimization, where the latter assumes perfect knowledge of the
distribution of times between file requests as well the distribution of the
number of SBSs within communication range of a user placing a request. We also
suggest a multi-agent RL (MARL) framework for the scenario of non-uniformly
distributed requests in space. For such a scenario, we show that MARL caching
policies achieve lower network load as compared to optimized caching policies
assuming a uniform request placement. We also provide convincing evidence that
synchronous updates offer a lower network load than asynchronous updates for
spatially homogeneous renewal request processes due to the memory of the
renewal processes.
