HTTP(S) proxy server in Crystal in less than 50 lines of code
Tags: crystal, proxy, tutorial
Published: 2020-06-24 12:34:04

---
title: HTTP(S) proxy server in Crystal in less than 50 lines of code
published: true
description: 
tags: crystal, proxy, tutorial
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/zcrds6u94ch9ahgqa4is.jpg
---

The purpose of this tutorial is to implement a proxy server for HTTP and HTTPS in pure [Crystal](https://crystal-lang.org/). 

Handling of HTTP is a matter of parsing request, passing such request to the destination server, reading response, and passing it back to the client. 

HTTPS is different as it'll use a technique called [HTTP CONNECT tunneling](https://en.wikipedia.org/wiki/HTTP_tunnel). At first, the client sends a request using HTTP `CONNECT` method to set up the tunnel between the client and destination server. When such a tunnel consisting of two TCP connections is ready, the client starts a regular TLS handshake with the destination server to establish a secure connection and later send requests and receive responses.

All we need for that is a built-in Crystal HTTP server and client from [HTTP](https://crystal-lang.org/api/latest/HTTP.html) module.

## HTTP

To support HTTP we will use a built-in HTTP server and client. The role of the proxy is to handle HTTP requests, pass a request to the desired destination, and send a response back to the client.

![HTTP Proxy](https://dev-to-uploads.s3.amazonaws.com/i/2l55kcnnxudgsupgapan.png)

## HTTP CONNECT tunneling

If a client wants to use HTTPS to talk to the server. The client is aware of using a proxy. A simple HTTP request/response flow cannot be used since the client needs to establish a secure connection with the server (HTTPS). The technique which works is to use the HTTP CONNECT method.

In this mechanism, the client asks a proxy server to forward the TCP connection to the desired destination. The server then proceeds to make the connection on behalf of the client. Once the connection has been established by the server, the proxy server continues to proxy the TCP stream to and from the client. Only the initial connection request is HTTP - after that, the server simply proxies the established TCP connection.

![HTTPS Proxy](https://dev-to-uploads.s3.amazonaws.com/i/zinmm92sj7lq64czqp9c.png)

## Implementation

> Keep in mind the presented code is not a production-grade solution.

```crystal
require "http"

class HTTP::ProxyHandler
  include HTTP::Handler

  def call(context)
    case context.request.method
    when "CONNECT"
      handle_tunneling(context)
    else
      handle_http(context)
    end
  end

  private def handle_tunneling(context)
    host, port = context.request.resource.split(":", 2)
    upstream = TCPSocket.new(host, port)

    context.response.upgrade do |downstream|
      channel = Channel(Nil).new(2)

      downstream = downstream.as(TCPSocket)
      downstream.sync = true

      spawn do
        transfer(upstream, downstream, channel)
        transfer(downstream, upstream, channel)
      end

      2.times { channel.receive }
    end
  end

  private def transfer(destination, source, channel)
    spawn do
      IO.copy(destination, source)
    rescue ex
      Log.error(exception: ex) { "Unhandled exception on HTTP::ProxyHandler" }
    ensure
      channel.send(nil)
    end
  end

  private def handle_http(context)
    uri = URI.parse(context.request.resource)
    client = HTTP::Client.new(uri)
    response = client.exec(context.request)

    context.response.headers.merge!(response.headers)
    context.response.status_code = response.status_code
    context.response.puts(response.body)
  end
end

proxy_handler = HTTP::ProxyHandler.new
server = HTTP::Server.new([proxy_handler])
address = server.bind_tcp(8080)
puts "Listening on http://#{address}"
server.listen
```

Our server while getting a request will take one of two paths:

* handling HTTP
* handling HTTP CONNECT tunneling.

This is done with:

```crystal
def call(context)
  case context.request.method
  when "CONNECT"
    handle_tunneling(context)
  else
    handle_http(context)
  end
end
```

Function to handle HTTP — `handle_http` is self-explanatory so let's focus on handling tunneling.

The first part of `handle_tunneling` is about setting connection to destination server:

```crystal
host, port = context.request.resource.split(":", 2)
upstream = TCPSocket.new(host, port)
```

All the magic happens inside the upgrade handler block:

```crystal
context.response.upgrade do |downstream|
  ...
end
```

Once we've two TCP connections (client → proxy, proxy → destination server) we need to set tunnel up:

```crystal
transfer(upstream, downstream, channel)
transfer(downstream, upstream, channel)
```

Next, we spawning two fibers and waiting for data are copied in two directions: from the client to the destination server and backward.

A buffered channel of capacity 2 is used to communicate that both fibers ended. So execution goes to the main fiber.

More info about Crystal concurrency you can find in the official [documentation](https://crystal-lang.org/reference/guides/concurrency.html)

And finally, we run our proxy server.

```crystal
proxy_handler = HTTP::ProxyHandler.new
server = HTTP::Server.new([proxy_handler])
address = server.bind_tcp(8080)
puts "Listening on http://#{address}"
server.listen
```

The above code will initialize a server with our proxy handler.
The port of the HTTP server is set by using the method `bind_tcp` on the object `HTTP::Server` (the port set to 8080).

For more information, check `HTTP::Server` [documentation](https://crystal-lang.org/api/latest/HTTP/Server.html)

## Testing

To test our proxy you can use e.g. Chromium:

```
chromium-browser --proxy-server=http://localhost:8080
```

or curl:

```
curl -Lv --proxy http://localhost:8080 https://httpbin.org/get
```

Testing the Proxy speeds on my DigitalOcean instance:

![Speedtest](https://dev-to-uploads.s3.amazonaws.com/i/e63zhly5i7xni7ur3ss6.png)

Happy Crystalling!