  Frequency control is an important problem in modern recommender systems. It
dictates the delivery frequency of recommendations to maintain product quality
and efficiency. For example, the frequency of delivering promotional
notifications impacts daily metrics as well as the infrastructure resource
consumption (e.g. CPU and memory usage). There remain open questions on what
objective we should optimize to represent business values in the long term
best, and how we should balance between daily metrics and resource consumption
in a dynamically fluctuating environment. We propose a personalized methodology
for the frequency control problem, which combines long-term value optimization
using reinforcement learning (RL) with a robust volume control technique we
termed "Effective Factor". We demonstrate statistically significant improvement
in daily metrics and resource efficiency by our method in several notification
applications at a scale of billions of users. To our best knowledge, our study
represents the first deep RL application on the frequency control problem at
such an industrial scale.
