How to use the new Symfony Maker command to work with GitHub Webhooks
Tags: symfony, webhook, github, php
Published: 2024-05-31 07:14:51

Recently I've been working on a tool that would gather some open-source contribution metrics from our teams. We mostly focus on contributions on GitHub, so I started studying the [API](https://docs.github.com/en/rest) to see how I could get the relevant data I needed to process. If I wanted to get fresh data on a regular basis, I would have sent requests periodically. But polling API endpoints is [not ideal](https://docs.github.com/en/rest/using-the-rest-api/best-practices-for-using-the-rest-api#avoid-polling), especially when the services you are accessing can **come to you instead** !

## Enter Webhooks !

Webhooks are a pretty common way for services from the outside world to communicate with your own application. It is quite similar to the event subscriber in its design :
- A remote service declares a list of **steps in its lifecycle** (for github: an issue has been opened, a comment has been made on a PR, ...), and for each of theses steps it will **dispatch** an event containing relevant data.
- You can **subscribe** to any of these events, and you'll get notified when they are dispatched.

The main difference with your local event based system resides in the **transport** : events are sent over the network to a custom endpoint where you implement your own logic to handle the events.

Nowadays, Webhooks are widely used for a lot of different purposes (getting information on a mail delivery, get notified of the steps of a payment process, ...), and the process to create a webhook is somehow always the same :
- Expose an endpoint.
- Check if the request should be processed.
- Check if the request if authorized and well formed.
- Process the request.

To make things easier for developers, Symfony released the [Webhook and RemoteEvent](https://symfony.com/blog/new-in-symfony-6-3-webhook-and-remoteevent-components) components  in **[Symfony 6.3](https://symfony.com/blog/category/living-on-the-edge/6.3)**. The **Webhook** component focuses on making the creation of endpoint and validation of request easy, while **RemoteEvent** is about making the event's payload transit on Messenger and be handled by a RemoteEventConsumer, where your logic will live.

To install these components, run :
```shell
$ composer require symfony/webhook
```

## How does this work ?

Okay, now we've installed the component, where should we start ?

First of all, let's set up our webhook so that we can effectively handle the requests that will be sent to us by GitHub.

At the time of writing, the component's documentation isn't fully released yet, so it may be a little bit confusing at first. But don't sweat : to make your life easier, a new Maker command was introduced !

To create a new Webhook, run :
```shell
$ symfony console make:webhook
```

The maker will ask you for the webhook name. It will be used to generate the webhook url (https://example.com/webhook/the_name_goes_here). Let’s call it “**github**”.  
Next you'll be asked you for the RequestMatchers to use. For GitHub, we know that the events are sent via POST requests and the format is JSON, so we'll add `MethodRequestMatcher` and `IsJsonRequestMatcher`.

![Screenshot of the command output indicating the success and creation of 3 new files](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b45mo6xrah7t3kfdvqe3.gif)

Now we can see that the command added some config to `config/packages/webhook.yaml` and created two files in our project source dir : `src/Webhook/GithubRequestParser.php` and `src/RemoteEvent/GithubWebhookConsumer`.
Hooray 🎉 ! Now we have some basis to work on.

## Tweaking the code

Let's dive into the generated class `src/Webhook/GithubRequestParser.php` and see what changes we have to make to fit our needs.

```php
<?php
// src/Webhook/GithubRequestParser.php
namespace App\Webhook;

use Symfony\Component\HttpFoundation\ChainRequestMatcher;
use Symfony\Component\HttpFoundation\Exception\JsonException;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\RequestMatcher\IsJsonRequestMatcher;
use Symfony\Component\HttpFoundation\RequestMatcher\MethodRequestMatcher;
use Symfony\Component\HttpFoundation\RequestMatcherInterface;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\RemoteEvent\RemoteEvent;
use Symfony\Component\Webhook\Client\AbstractRequestParser;
use Symfony\Component\Webhook\Exception\RejectWebhookException;

final class GithubRequestParser extends AbstractRequestParser
{
    protected function getRequestMatcher(): RequestMatcherInterface
    {
        return new ChainRequestMatcher([
            new IsJsonRequestMatcher(),
            new MethodRequestMatcher('POST'),
        ]);
    }

    /**
     * @throws JsonException
     */
    protected function doParse(
        Request $request,
        #[\SensitiveParameter] string $secret
    ): ?RemoteEvent
    {
        // TODO: Adapt or replace the content of this method to fit your need.

        // Validate the request against $secret.
        $authToken = $request->headers->get('X-Authentication-Token');
        if ($authToken !== $secret) {
            throw new RejectWebhookException(Response::HTTP_UNAUTHORIZED, 'Invalid authentication token.');
        }

        // Validate the request payload.
        if (!$request->getPayload()->has('name')
            || !$request->getPayload()->has('id')) {
            throw new RejectWebhookException(Response::HTTP_BAD_REQUEST, 'Request payload does not contain required fields.');
        }

        // Parse the request payload and return a RemoteEvent object.
        $payload = $request->getPayload()->all();

        return new RemoteEvent(
            $payload['name'],
            $payload['id'],
            $payload,
        );
    }
}
```

We can see that the RequestMatchers previously selected were added to a `ChainRequestMatcher`. We don't have anything else to do in this method 🥳.  

Now in the `doParse` method, we see that three main steps are hinted by the comments : 
- **Validating** the request against the secret.
- Check the request is **well formed** (mandatory fields are present, the expected format is respected ...).
- Returning a **remote event** holding the payload.

GitHub has an [interesting documentation](https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries) on request validation, with some snippets in Ruby, JavaScript, Python ... but no PHP 😢. No worries, I made the translation for you :

```php
$signature = $request->headers->get('X-Hub-Signature-256');  
if (!is_string($signature)  
    || !str_starts_with($signature, 'sha256=')  
    || !hash_equals('sha256='.hash_hmac('sha256', $request->getContent(), $secret), $signature)) {  
    throw new RejectWebhookException(Response::HTTP_UNAUTHORIZED, 'Invalid authentication token.');  
}
```

Now, for the validation : we're expecting a variety of events to knock at our webhook's door, so we won't be too strict on format validation.  
To create a `RemoteEvent` we'll need a **name** (action) and an **id**. That will be our minimum requirements :
```php
if (!$request->getPayload()->has('action') || !$request->getPayload()->has('number')) {  
    throw new RejectWebhookException(Response::HTTP_BAD_REQUEST, 'Request payload does not contain required fields.');  
}
```

Then all that's left to do is to create and return a `RemoteEvent` :
```php
$payload = $request->getPayload()->all();  
  
return new RemoteEvent(  
    $payload['action'],  
    $payload['number'],  
    $payload,  
);
```

This event will be passed over to Messenger, that in turn will pass it to your `GithubWebhookEventConsumer` (thanks to the `#[AsRemoteEventConsumer('github')]` attribute on the class).

```php
<?php
// src/RemoteEvent/GithubWebhookConsumer.php
namespace App\RemoteEvent;

use Symfony\Component\RemoteEvent\Attribute\AsRemoteEventConsumer;
use Symfony\Component\RemoteEvent\Consumer\ConsumerInterface;
use Symfony\Component\RemoteEvent\RemoteEvent;

#[AsRemoteEventConsumer('github')]
final class GithubWebhookConsumer implements ConsumerInterface
{
    public function __construct()
    {
    }

    public function consume(RemoteEvent $event): void
    {
        // Implement your own logic here
    }
}
```

Here is where you'll put your **custom logic** : mapping to DTO, persisting to database, ... whatever fits your needs.

Finally, head to `config/packages/webhook.yaml` and set up a secret (a random string of text with high entropy). This being sensitive data, it should be referenced here but stored in an environment variable ([see Symfony documentation](https://symfony.com/doc/current/configuration/secrets.html)).

```yaml
# config/packages/webhook.yaml
framework:
  webhook:
    routing:
      github:
        service: App\Webhook\GithubRequestParser
        secret: '%env(GITHUB_WEBHOOK_SECRET)%'
```

## Call me back

Now that we're ready to handle requests, all we need to do is ask GitHub to send us some. 
The [official documentation](https://docs.github.com/en/webhooks/using-webhooks/creating-webhooks) is really good so we won't detail the process here. You'll need the endpoint url (https://example.com/webhook/github) and your secret. Just be aware that you can only create webhooks for **resources that you own**. If you want to be notified on actions performed on a repository you don't own, you'll have to ask the owner to set it up for you. If this is not an option, you'll have to rely on good old [API polling](https://en.wikipedia.org/wiki/Polling_(computer_science)).


## Going further

### Is that even useful ?

You may be tempted to say that all we've done is exposing an endpoint to process a request, and that it could have been done without the webhook component.
And you would be right : you can achieve the same result with a custom controller.  
But let's see the benefits of doing it the way we did :
- A single conf file with minimal and simple configuration for all our exposed endpoints.
- A clean implementation of `AbstractRequestParser` to handle request authorization, validation and event dispatching.
- Any service can be turned into a remote event consumer just by using `#[AsRemoteEventConsumer]` and `ConsumerInterface`.
- A seamless integration with Symfony Messenger, Notifier, Mailer (and more to come).
- All of the above was done by **running a single command and writing less than 10 lines of code** 🤯.

### What's next ?

You may want to take a look at Github's [Best practices for using webhooks](https://docs.github.com/en/webhooks/using-webhooks/best-practices-for-using-webhooks) .
That could make you want to :
- Add an `IpsRequestMatcher` to check if the request is sent from one of GitHub's official IPs.
- Use an async transport to reduce the request process time.
- Handle re-deliveries
- ...

You may even want to [contribute](https://symfony.com/doc/current/contributing/index.html) to Symfony by improving the component or the maker, creating a Bridge to spare some trouble to future developers, ... It's all up to you to make Symfony even better !

### Tips : Local webhooks

When developing, you may want to receive requests to test your code. You'll need a webhook proxy for this. I would suggest using [smee.io](https://smee.io/) :
- Start a new channel.
- Copy the link to the "Payload URL" in the GitHub webhook configuration form.
- Download the smee client on your local machine.
- Run `smee -u https://smee.io/thispartisrandom --port 8000 --path /webhhok/github` \*

\**For a Symfony application running on localhost:8000*

Accessing the smee url from your browser will allow you to visualize webhooks deliveries : headers, payload, ... and you'll be able to **replay them**.
You can take advantage of this and copy the payloads and headers to **create fixtures to test your webhooks** !