Implementing Health Checks in ASP.NET Core
By: Christopher Laine
Tags: DevOps, Programming, Docker, Software Development, Microservices

Implementing Health Checks in ASP.NET Core

Ensuring your services are healthy has never been easier

Photo by Jair Lázaro on Unsplash

When partial failure attacks

At one of my companies, I’ve been working on a DDD system written in ASP.NET for quite some time now (nearly 10 years, by my reckoning). In that time, we’ve gone from a BBOM (Big Ball of Mud) to a large number of distinct Bounded Contexts. This, clearly, is a good thing. We no longer have the monolith, and we have a lovely distributed system which is far easier to scale and deploy.

And then there’s the times when things go pear-shaped.

One day, a whole lot of event messaging traffic just ground to a halt. Not all of the traffic, just some percentage of it (insert sour face here). This, as any seasoned engineer will tell you, can be a far worse a problem to work through than if everything stopped, especially if your logging is not as clear as you or your DevOps team would like it to be (note: it’s NEVER as clear as you’d like it to be).

When some of the data is not processing, it means that you have a sick whozits in your system somewhere which is failing to do its job properly, and now the race is on to find out who’s healthy and who’s not.

In the end, of course, we found it: A minor security change had been made on the database, but not ALL of the services affected had been updated / restarted to take this into account. So, one of these services was the sick puppy, while it’s siblings were still doing their job happily. Thus the intermittent success and failure.

Logging helped, but it took far longer than we wanted to work out who was failing and what the issue was. The outage lasted 28 minutes, and none of us were happy with that result.

These kinds of ‘partial failure’ scenarios do happen, and they can cause big headaches, especially as your platform splits out into more and more discrete microservices. Knowing the health of any one microservice instance can be critical to solving such a partial failure quickly.

Health Checks to the rescue

A common pattern one sees these days is the use of some form of a “health check”. This is usually an endpoint exposed by your microservice (often in the form of a REST resource) which allows it to tell a caller its overall health, so that the caller can act appropriately if the microservice instance comes back as unhealthy.

This is useful in and of itself, as it is nice to be able to “ask” your microservice instance how it is doing health-wise.

However, this kind of health checking is critical in many microservice architectures. In container orchestration and coordination systems such as Kubernetes or AWS ECS Fargate, it is expected that a health endpoint is exposed by your microservice, so that the orchestration systems can ensure your microservice is working as expected.

Why is this? When you build a container cluster, you will end up with 1…x number of running instances of your containerised microservice. It is critical for the health of your application that any given instance can be checked periodically for its overall health. It isn’t realistic for you to do so manually, particularly as the number of microservices grows over time.

Modern microservice platforms make it possible for any sick instances of your microservice to be removed from the cluster automatically and replaced with a new, healthy instance.

That’s cool!

And it’s all made possible by using effective Health Checks.

ASP.NET Core Health Checks

In ASP.NET Core, health checks are baked in from the ground up. We’re going to implement some basic health checking logic, so you can see how easy it can be to expose this kind of functionality.

Let’s get started. I’ve added my code for this to github, so grab a copy and follow along :)

Install the following package in your application

install-package Microsoft.AspNetCore.Diagnostics.HealthChecks

Open your startup.cs file. In here, we will add the basic health check logic to get us started. Edit (or add) your ConfigureServices / Configure methods to add the following lines.

public void ConfigureServices(IServiceCollection services)

{

services.AddHealthChecks();

} public void Configure(IApplicationBuilder app, IHostingEnvironment env)

{

//other code

app.UseHealthChecks("/health");

//other code

}

This is dead simple, but of course does nothing. If you kick the tires on this via a REST call, all you get is this every time

REQUEST:

GET /health HTTP/1.1

Host: localhost:54411

Accept: */*

Cache-Control: no-cache

Host: localhost:54411

accept-encoding: gzip, deflate

Connection: keep-alive

cache-control: no-cache RESPONSE:

200 OK

Healthy

For two lines of code, not too bad. However, we can do better

We’ll add a new class to our web api which implements a simple check for health. Create a new class called ApiHealthCheck, which implements IHealthCheck interface

public class ApiHealthCheck : IHealthCheck

{

public Task<HealthCheckResult> CheckHealthAsync(

HealthCheckContext context,

CancellationToken cancellationToken = new CancellationToken())

{

//TODO: Implement your own healthcheck logic here

var isHealthy = true;

if(isHealthy)

{

return Task.FromResult(HealthCheckResult.Healthy("I am one healthy microservice API"));

}



return Task.FromResult(HealthCheckResult.Unhealthy("I am the sad, unhealthy microservice API"));

}

}

Now, let’s wire this in back in our Startup.cs

public void ConfigureServices(IServiceCollection services)

{

services.AddHealthChecks()

.AddCheck<ApiHealthCheck>("api");

So, rather than a generic health check for the service itself, we can now add logic to our health check.

Customising HealthCheck output

Now, if you ran up the health check after we implemented our ApiHealthCheck class, you’ll note that you all you still get for a response is the word “Healthy” when you call the REST endpoint. Uhhhh, okay. That’s good, I guess. And what if it’s unhealthy? You might want to know more about the details of the health / sickness.

By default, all ASP.NET Core health check will tell you is one of the three primary statuses (Healthy, Unhealthy or Degraded), which is not the best if you want to know what is making your microservice sick. So, you can add some additional logic in your Startup.cs to get something more meaningful returned.

Go back to your Startup class’s Configure method, and add the following method (you’ll need to install json.net, if it’s not already there)

private static Task WriteHealthCheckResponse(HttpContext httpContext,

HealthReport result)

{ httpContext.Response.ContentType = “application/json”; var json = new JObject(

new JProperty(“status”, result.Status.ToString()),

new JProperty(“results”, new JObject(result.Entries.Select(pair =>

new JProperty(pair.Key, new JObject(

new JProperty(“status”, pair.Value.Status.ToString()),

new JProperty(“description”, pair.Value.Description),

new JProperty(“data”, new JObject(pair.Value.Data.Select(

p => new JProperty(p.Key, p.Value))))))))));

return httpContext.Response.WriteAsync(

json.ToString(Formatting.Indented)); }

In your Startup’s Configure method, change your UseHealthCheck configuration to look like this

app.UseHealthChecks(“/health”, new HealthCheckOptions()

{

//that's to the method you created

ResponseWriter = WriteHealthCheckResponse

});

Now what do we get when we navigate to our health check? Something far more descriptive.

{

"status": "Healthy",

"results": {

"api": {

"status": "Healthy",

"description": "I am one healthy microservice API",

"data": {}

}

}

}

Now our microservice is telling us something meaningful.

Chaining Health Checks

One powerful feature in ASP.NET Core health checks is that you can add as many health checks as you require in a chain. This allows you to separate your health check logic into discreet logical boundaries, and the health check system will automatically roll these up. Let’s try it.

We’re going to make a degraded scenario. A degraded health check response is useful if your microservice is not quite dead, but not doing its best either.

Add a new class called SecondaryHealthCheck.cs

public class SecondaryHealthCheck : IHealthCheck

{

public Task<HealthCheckResult> CheckHealthAsync(

HealthCheckContext context,

CancellationToken cancellationToken = new CancellationToken())

{

return Task.FromResult(HealthCheckResult.Degraded("Not feeling so well, boss!"));

}

}

And add a reference to this new health check into the startup’s health check chain inside the ConfigureServices method

services.AddHealthChecks()

.AddCheck<ApiHealthCheck>("api")

.AddCheck<SecondaryHealthCheck>("secondary");

Now what do we get?

{

"status": "Degraded",

"results": {

"api": {

"status": "Healthy",

"description": "I am one healthy microservice API",

"data": {}

},

"secondary": {

"status": "Degraded",

"description": "Not feeling so well, boss!",

"data": {}

}

}

}

As you can see, ASP.NET Core rolls up the health checks so that the least healthy status is reported at the top of the stack, with detailed information for each health check outlined below.

Pre-canned Health Checks

A service is only as good as its required resources. Database connectivity (say Sql Server, Postgres or Mysql), Redis Cache server, even the UI could count as resources required by your microservice to be considered healthy and able to do its job.

GitHub contributors, particularly the team at Xabaril/AspNetCore.Diagnostics.HealthChecks offer a goodly list of pre-canned health check nuget packages, to help you make health checking far easier to implement.

While you more than likely have other health check needs beyond these, it does mean you can save a lot of time by using at least some of them to get the benefits of health checks without having to write a lot of custom code.

Conclusion

If you are getting started building containerised microservices for .NET, I think you will find that health checks are not just a ‘nice to have’ feature to tell you and your co-workers if your app is healthy or not. In a cloud-native containerised application, health checks mean that you and yout team can know and act upon the health of your services in a robust and standardised way.

ASP.NET Core has really added first-class support for health checks, and made it very easy to build and customise them at the same time.

Hope this helps

For more info, have a starting gander here on Microsoft’s documentation page.