gRPC - Deadlines, Cancellations, and Retries
Published: 2023-07-13 16:03:09

## Background

- Why do you want to cancel your request?
    
    Senario 1
    
    - Imagine client application calling multiple services
    - There is a thread pool from which each request is assigned a worker thread
    - If any service is slow/unresponsive, the request will block that thread
    - Slowly all those requests will capture more and more threads and the application becomes unresponsive
    
    Scenario 2
    
    - User has already moved on and doesn’t want the result
    

- But should we always cancel the timed out request?
    
    Maybe not. 
    
    - The request almost completed and if we are using cache then better save the result for next time
    - maybe more reasons 🤔

- Can you cancel an HTTP request?
    - Short Answer is `NO`
    - Request can be cancelled in client. But is it really cancellation?
    - `The Problem:` HTTP is stateless so there is no way we can cancel the request running in the server, sad!
    - But if we can add some state on top of HTTP in application (it is not easy)
        - Cookies
        - Store state in DB (like Redis)
        - Pub/Sub or queues

### gRPC and its support

- gRPC is built on top of HTTP2 but it is stateful
- It already supports deadlines, cancellations and retries and it is easy to use
- Either the client or server can terminate the RPC
- When one party cancels the RPC, the other party can determine it by checking the context
- Based on this, the cancellation request can be cascaded to other services. That’s powerful

## Deadlines vs Timeouts

**Deadline:**
a fixed point in time by which the RPC should complete

**Timeout:**
a duration of time after which the RPC times out

- `Example`
    
    *Deadline:* finish at 7:15:35PM
    *Timeout:* finish in 10 seconds
    
- Timeouts are not much encouraged because it is hard to translate them to the servers
- Timeouts cannot be directly applied for the entire life cycle of the request, in case when there are multiple downstream RPCs

## Deadlines

- Deadline configuration:
    - A deadline is configured using `context.WithDeadline()` when a call is made.
    - There is no default deadline value. gRPC calls aren't time limited unless a deadline is specified.
    - If a past or current time is used then the call immediately exceeds the deadline.
    - The deadline is sent with the gRPC call to the service and is independently tracked by both the client and the service. It is possible that a gRPC call completes on one machine, but by the time the response has returned to the client the deadline has been exceeded.

If a deadline is exceeded, the client and service have different behavior:

- The client immediately aborts the underlying HTTP request and throws a `DeadlineExceeded` error. The client app can choose to catch the error and display a timeout message to the user.
- On the server, the executing HTTP request is aborted. Although the HTTP request is aborted, the gRPC call continues to run on the server until the method completes.
- It's important that the cancellation token is passed to async methods so they are cancelled along with the call. For example, passing a cancellation token to async database queries and HTTP requests.
- Passing a cancellation token allows the canceled call to complete quickly on the server and free up resources for other calls.
- When a gRPC call is configured with retry fault handling and a deadline, the deadline tracks time across all retries for a gRPC call. If the deadline is exceeded, a gRPC call immediately aborts the underlying HTTP request, skips any remaining retries, and throws a  `DeadlineExceeded` error.

Refer this for 

- [https://grpc.io/blog/deadlines/](https://grpc.io/blog/deadlines/)
- [https://docs.microsoft.com/en-us/aspnet/core/grpc/deadlines-cancellation?view=aspnetcore-6.0](https://docs.microsoft.com/en-us/aspnet/core/grpc/deadlines-cancellation?view=aspnetcore-6.0)

## Cancellations

- Cancellation allows a gRPC client to cancel long running calls that are no longer needed. For example, a gRPC call that streams realtime updates is started when the user visits a page on a website. The stream should be canceled when the user navigates away from the page.
- The server application can check whether the current context is canceled by using `stream.Context().Err() == context.Canceled`

## Retries

- gRPC retries is a feature that allows gRPC clients to automatically retry failed calls.
- gRPC calls can be interrupted by transient faults. Transient faults include:
    - Momentary loss of network connectivity.
    - Temporary unavailability of a service.
    - Timeouts due to server load.
- `Example`
    - [https://github.com/grpc/grpc-go/blob/master/examples/features/retry/README.md](https://github.com/grpc/grpc-go/blob/master/examples/features/retry/README.md)
- **When retries are valid**
    - Calls are retried when:
        - The failing status code matches a value in `RetryableStatusCodes`.
        - The previous number of attempts is less than `MaxAttempts`.
        - The call hasn't been committed.
        - The deadline hasn't been exceeded.
    - A gRPC call becomes committed in two scenarios:
        - The client receives response headers. Response headers are sent by the server when `ServerCallContext.WriteResponseHeadersAsync` is called, or when the first message is written to the server response stream.
        - The client's outgoing message (or messages if streaming) has exceeded the client's maximum buffer size. `MaxRetryBufferSize` and `MaxRetryBufferPerCallSize` are [configured on the channel](https://docs.microsoft.com/en-us/aspnet/core/grpc/configuration?view=aspnetcore-6.0#configure-client-options).
    - Committed calls won't retry, regardless of the status code or the previous number of attempts.

### Resources

- [https://docs.microsoft.com/en-us/aspnet/core/grpc/retries?view=aspnetcore-6.0](https://docs.microsoft.com/en-us/aspnet/core/grpc/retries?view=aspnetcore-6.0)
- [https://docs.microsoft.com/en-us/azure/architecture/best-practices/transient-faults](https://docs.microsoft.com/en-us/azure/architecture/best-practices/transient-faults)
- [https://github.com/grpc/proposal/blob/master/A6-client-retries.md](https://github.com/grpc/proposal/blob/master/A6-client-retries.md)