Repository: tokio-rs/tokio
File: tokio/src/sync/semaphore.rs (Rust)

Code before:
/// proceeding to open another file.
/// ```
/// use std::io::Result;
/// use tokio::fs::File;
/// use tokio::sync::Semaphore;
/// use tokio::io::AsyncWriteExt;
///
/// static PERMITS: Semaphore = Semaphore::const_new(100);
///
/// async fn write_to_file(message: &[u8]) -> Result<()> {
///     let _permit = PERMITS.acquire().await.unwrap();
///     let mut buffer = File::create("example.txt").await?;
///     buffer.write_all(message).await?;
///     Ok(()) // Permit goes out of scope here, and is available again for acquisition
/// }
/// ```
///
/// Implement a simple token bucket for rate limiting
///
/// Many applications and systems have constraints on the rate at which certain
/// operations should occur. Exceeding this rate can result in suboptimal
/// performance or even errors.
///
/// The provided example uses a `TokenBucket`, implemented using a semaphore, that
/// limits operations to a specific rate. The token bucket will be refilled gradually.
/// When the rate is exceeded, the `acquire` method will await until a token is available.
/// ```
/// use std::sync::Arc;
/// use tokio::sync::{AcquireError, Semaphore, SemaphorePermit};
/// use tokio::time::{sleep, Duration};
///
/// struct TokenBucket {
///     sem: Arc<Semaphore>,
///     jh: tokio::task::JoinHandle<()>,
/// }
///
/// impl TokenBucket {
///     fn new(rate: usize) -> Self {
///         let sem = Arc::new(Semaphore::new(rate));
///
///         // refills the permits each 1/rate seconds.
///         let jh = tokio::spawn({
///             let sem = sem.clone();
///
///             async move {
///                 let time_slice = 1.0 / (rate as f32);
///
///                 loop {
///                     sleep(Duration::from_secs_f32(time_slice)).await;
///
///                     let cap = rate - sem.available_permits();

Review comment (suggestion):
We should mention two things here:

 * Token buckets allow short bursts that are faster than the allowed rate. In fact, this is part of the point of token buckets.
 * This implementation is suboptimal when the rate is large, because it consumes a lot of cpu constantly looping and sleeping.

Code after:
///
/// With a token bucket, each incoming request consumes a token, and the tokens are refilled at a
/// certain rate that defines the rate limit. When a burst of requests arrives, tokens are
/// immediately given out until the bucket is empty. Once the bucket is empty, requests will have to
/// wait for new tokens to be added.
///
/// Unlike the example that limits how many requests can be handled at the same time, we do not add
/// tokens back when we finish handling a request. Instead, tokens are added only by a timer task.
///
/// Note that this implementation is suboptimal when the duration is small, because it consumes a
/// lot of cpu constantly looping and sleeping.
///
/// [token bucket]: https://en.wikipedia.org/wiki/Token_bucket
/// ```
/// use std::sync::Arc;
/// use tokio::sync::{AcquireError, Semaphore};
/// use tokio::time::{interval, Duration};
///
/// struct TokenBucket {
///     sem: Arc<Semaphore>,
///     jh: tokio::task::JoinHandle<()>,
/// }
///
/// impl TokenBucket {
///     fn new(duration: Duration, capacity: usize) -> Self {
///         let sem = Arc::new(Semaphore::new(capacity));
///
///         // refills the tokens at the end of each interval
///         let jh = tokio::spawn({
///             let sem = sem.clone();
///             let mut interval = interval(duration);
///             interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
///
///             async move {
///                 loop {
///                     interval.tick().await;
///
///                     if sem.available_permits() < capacity {
///                         sem.add_permits(1);
///                     }
///                 }
///             }
///         });
///
///         Self { jh, sem }
///     }
///
///     async fn acquire(&self) -> Result<(), AcquireError> {
///         self.sem.acquire().await.map(|p| p.forget())
///     }
///