How To Troubleshoot a Silent Webpage (A Docker Misadventure)
Tags: docker, web, devops, architecture
Published: 2024-01-15 10:35:56


## Introduction:

Ever faced a webpage that refuses to say a word? Join me in unraveling a recent hiccup in our Docker adventures, where an innocuous port-curling expedition turned into an unexpected outage. In this blog post, we'll navigate through the timelines, actions taken, and the crucial lessons learned from this episode.


## The Incident: A Webpage Goes Mute

It all started on a regular `Monday morning at 10:47 AM (WAT)`. Eager to check on the latest Docker container, I initiated a simple `curl` command on port 8080 mapped to the container's port 80. To my surprise, instead of a cheerful page, I was greeted with the ominous message: "empty reply from server."


## The Impact: Silent Users and Empty Pages

The repercussions were swift. Our webpage turned into a mute spectator, rendering nothing but an empty response. Quick estimates suggested about 5% of our users were caught in this silent storm.


## Root Cause: The Vanishing Server Start

The detective work began. The culprit? A server that mysteriously forgot to start after a recent operation. A simple oversight, yet one that sent ripples through our digital landscape.


## Timeline: The Race Against Time

At `10:47 AM`, the silent alarm was raised, not by a user complaint, but by our vigilant monitoring alert. The race against time had begun.


## Actions Taken: Unveiling the Silent Server

The first move was to inspect the server's whereabouts. What port was it listening to? Was it even awake? A thorough check revealed the dormant state of our crucial server.


## Resolution: Reviving the Silent Server

With the issue laid bare, the solution was straightforward: start the server! However, to prevent such muteness in the future, a Puppet script was introduced. Now, every server update triggers an automatic start, silencing any potential disruptions.

Initially using a bash script, this was used:

```
#!/usr/bin/env bash
# get Apache to run on the container and to return a page

service apache2 start
```

With Puppet for automation, let's create a Puppet manifest file. Let's call it `apache_service.pp`:
```
# File: apache_service.pp

# Define a class for managing the Apache service
class apache_service {
  service { 'apache2':
    ensure => 'running',
    enable => true,
    subscribe => File['/etc/apache2/httpd.conf'], # Trigger the service restart when the Apache configuration file is updated
  }
}

# Apply the class to ensure it is included when Puppet runs
include apache_service
```


## Root Cause and Resolution: Learning from the Silence

The root cause was a temporarily faulty server, fixed but left in silence. The resolution? A proactive decision to script server starts, ensuring they speak up after every update.


## Corrective and Preventative Measures: Scripts and Vigilance

To fortify our defenses, every engineer is now armed with mandatory pre and post-server operation scripts. Additionally, the Quality Assurance team is on high alert, (re)educated on the nuances of our vigilant monitoring tools.


## Conclusion: Lessons from Silence

In the ever-evolving landscape of Docker and servers, even a momentary silence can be deafening. This incident taught us the importance of vigilance, automation, and proactive measures. Let this be a reminder to all: in the realm of webpages, silence is not always golden.