Running Rack: How Ruby HTTP servers run Rails apps
Tags: ruby, rails, programming, coding
Published: 2018-12-18 13:18:35

---
title: "Running Rack: How Ruby HTTP servers run Rails apps"
description: After starting work on a simple HTTP server a while back, we're diving back in to learn how to handle POST requests and what we need to do to serve a Rails application.
cover_image: https://thepracticaldev.s3.amazonaws.com/i/5g5lbaiy1m5q8kkm5tvr.jpg
tags: ruby, rails, programming, coding
series: HTTP servers in Ruby
canonical_url: https://blog.appsignal.com/2017/01/24/ruby-magic-building-a-ruby-http-server-part-2-running-a-rails-app.html
published: true
---

*Originally published at [blog.appsignal.com/2017/01/24/ruby-magic-building-a-ruby-http-server-part-2-running-a-rails-app.html](https://blog.appsignal.com/2017/01/24/ruby-magic-building-a-ruby-http-server-part-2-running-a-rails-app.html) on Jan 24, 2017.*

_In the Ruby Magic series we love to take software apart to learn how it functions under the hood. It's all about the process; the end result isn't something you'd use in production, we learn about the internal workings of the Ruby language and its popular libraries. We publish a new article about once a month, so be sure to [subscribe to our newsletter](https://appsignal.com/ruby-magic) if you're into this sort of thing too._

In an earlier edition of Ruby Magic we [implemented a 30-line HTTP server in Ruby](http://blog.appsignal.com/2016/11/23/ruby-magic-building-a-30-line-http-server-in-ruby.html). Without having to write a lot of code, we were able to handle HTTP GET requests and serve a simple Rack application. This time, we'll take our home made server a bit further. When we're done, we'll have a web server that can serve Rails' famous fifteen minute blog that allows you to create, update and delete posts.

## Where we left off

Last time, we  implemented just enough of a server to have it serve [Rack::Lobster](https://github.com/rack/rack/blob/master/lib/rack/lobster.rb) as an example application.

1. Our implementation opened a TCP server and waited for a request to come in.
2. When that happened, the request-line (`GET /?flip=left HTTP/1.1\r\n`) was parsed to get the request method (`GET`), the path (`/`), and the query parameters (`flip=left`).
3. The request method, the path and the query string were passed to the Rack app, which returned a triplet with a status, some response headers and the response body.
4. Using those, we were able to build an HTTP response to send back to the browser, before closing the connection to wait for a new request to come in.

``` ruby
# http_server.rb
require 'socket'
require 'rack'
require 'rack/lobster'

app = Rack::Lobster.new
server = TCPServer.new 5678

#1
while session = server.accept
  request = session.gets
  puts request

  #2
  method, full_path = request.split(' ')
  path, query = full_path.split('?')

  #3
  status, headers, body = app.call({
    'REQUEST_METHOD' => method,
    'PATH_INFO' => path,
    'QUERY_STRING' => query
  })

  #4
  session.print "HTTP/1.1 #{status}\r\n"
  headers.each do |key, value|
    session.print "#{key}: #{value}\r\n"
  end
  session.print "\r\n"
  body.each do |part|
    session.print part
  end
  session.close
end
```

We'll be continuing with the code we wrote last time. If you want to follow along, here's the [code we ended up with](https://gist.github.com/jeffkreeftmeijer/7f08d1f7e381b9c552666750914925eb).

## Rack and Rails

Ruby frameworks like Rails and Sinatra are built on top of the Rack interface. Just like the instance of `Rack::Lobster` we're using to test our server right now, Rails' `Rails.application` is a Rack application object. In theory, this would mean that our server should already be able to serve a Rails application.

To test that, I've prepared [a simple Rails application](https://github.com/jeffkreeftmeijer/wups). Let's clone that into the same directory as our server.

```
$ ls
http_server.rb
$ git clone https://github.com/jeffkreeftmeijer/wups.git blog
Cloning into 'blog'...
remote: Counting objects: 162, done.
remote: Compressing objects: 100% (112/112), done.
remote: Total 162 (delta 32), reused 162 (delta 32), pack-reused 0
Receiving objects: 100% (162/162), 29.09 KiB | 0 bytes/s, done.
Resolving deltas: 100% (32/32), done.
Checking connectivity... done.
$ ls
blog           http_server.rb
```

Then, in our server, require the Rails application's environment file instead of `rack` and `rack/lobster`, and put the `Rails.application` in the `app` variable instead of `Rack::Lobster.new`.

``` ruby
# http_server.rb
require 'socket'
require_relative 'blog/config/environment'

app = Rails.application
server = TCPServer.new 5678
# ...
```

Starting the server (`ruby http_server.rb`) and opening http://localhost:5678 shows us we're not quite there yet. The server doesn't crash, but we're greeted with an internal server error in the browser.

![500 Internal Server Error. If you are the administrator of this website, then please read this web application's log file and/or the web server's log file to find out what went wrong.](https://blog.appsignal.com/images/blog/2017-01/500.png)

Checking our server's logs, we can see that we're missing something called `rack.input`. It turns out that we've been lazy while implementing our server last time, so there's more work to do before we can get this Rails application to work.

```
$ ruby http_server.rb
GET / HTTP/1.1
Error during failsafe response: Missing rack.input
  ...
  http_server.rb:15:in `<main>'
```

## The Rack environment

Back when we implemented our server, we glossed over [the Rack environment](http://www.rubydoc.info/github/rack/rack/file/SPEC#The_Environment) and ignored most of the variables that are required to properly serve Rack applications. We ended up only implementing the `REQUEST_METHOD`, `PATH_INFO`, and `QUERY_STRING` variables, as those were sufficient for our simple Rack app.

As we've already seen from the exception when we tried to start our new application, Rails needs `rack.input`, which is used as an input stream for raw HTTP POST data. Besides that, there are some more variables we need to pass, like the server's port number, and the request cookie data.

Luckily, Rack provides [`Rack::Lint`](https://github.com/rack/rack/blob/master/lib/rack/lint.rb<Paste>) to help make sure all variables in the Rack environment are present and valid. We can use it to test our server by wrapping our Rails app in it by calling `Rack::Lint.new` and passing the `Rails.application`.

``` ruby
# http_server.rb
require 'socket'
require_relative 'blog/config/environment'

app = Rack::Lint.new(Rails.application)
server = TCPServer.new 5678
# ...
```

`Rack::Lint` will throw an exception when a variable in the environment is missing or invalid. Right now, starting our server again and opening http://localhost:5678 will crash the server and `Rack::Lint` will notify us of the first error: the `SERVER_NAME` variable wasn't set.

```
~/Appsignal/http-server (master) $ ruby http_server.rb
GET / HTTP/1.1
/Users/jeff/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/rack-2.0.1/lib/rack/lint.rb:20:in `assert': env missing required key SERVER_NAME (Rack::Lint::LintError)
        ...
        from http_server.rb:15:in `<main>'
```

By fixing each error that is thrown at us, we can keep adding variables until `Rack::Lint` stops crashing our server. Let's go over each of the variables `Rack::Lint` requires.

- `SERVER_NAME`: the server's hostname. We're only running this server locally right now, so we'll use "localhost".
- `SERVER_PORT`: the port our server is running on. We've hardcoded the port number (5678), so we'll just pass that to the Rack environment.
- `rack.version`: the targeted Rack _protocol_ version number as an array of integers. [`[1,3]`](https://github.com/rack/rack/blob/master/lib/rack.rb#L14) at the time of writing.
- `rack.input`: the input stream containing the raw HTTP post data. We'll get to this later, but we'll pass an empty `StringIO` instance (with an ASCII-8BIT encoding) for now.
- `rack.errors`: the error stream for `Rack::Logger` to write to. We're using `$stderr`.
- `rack.multithread`: our server is single-threaded, so this can be set to `false`.
- `rack.multiprocess`: our server is running in a single process, so this can be set to `false` as well.
- `rack.run_once`: our server can handle multiple sequential requests in one process, so this is `false` too.
- `rack.url_scheme`: no SSL support, so this can be set to "http" instead of "https".

After adding all missing variables, `Rack::Lint` will notify us of one more problem in our environment.

```
$ ruby http_server.rb
GET / HTTP/1.1
/Users/jeff/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/rack-2.0.1/lib/rack/lint.rb:20:in `assert': env variable QUERY_STRING has non-string value nil (Rack::Lint::LintError)
        ...
        from http_server.rb:18:in `<main>'
```

When there's no query string in the request, we'll now pass `nil` as the `QUERY_STRING`, which is not allowed. In that case, Rack expects an empty string instead. After implementing the missing variables and updating the query string, this is what our environment looks like:

``` ruby
# http_server.rb
# ...
  method, full_path = request.split(' ')
  path, query = full_path.split('?')

  input = StringIO.new
  input.set_encoding 'ASCII-8BIT'

  status, headers, body = app.call({
    'REQUEST_METHOD' => method,
    'PATH_INFO' => path,
    'QUERY_STRING' => query || '',
    'SERVER_NAME' => 'localhost',
    'SERVER_PORT' => '5678',
    'rack.version' => [1,3],
    'rack.input' => input,
    'rack.errors' => $stderr,
    'rack.multithread' => false,
    'rack.multiprocess' => false,
    'rack.run_once' => false,
    'rack.url_scheme' => 'http'
  })

  session.print "HTTP/1.1 #{status}\r\n"
# ...
```

Restarting the server and visiting http://localhost:5678 again, we'll be greeted with Rails' "You're on Rails!"-page, meaning we're now running an actual Rails application on our home made server!

![Yay! You're on Rails!](https://blog.appsignal.com/images/blog/2017-01/youre-on-rails.png)

## Parsing HTTP POST bodies

This application is more than just that index page. Visiting http://localhost:5678/posts will display an empty list of posts. If we try to create a new post by filling in the new post form and pressing "Create post", we're greeted by an `ActionController::InvalidAuthenticityToken` exception.

The authenticity token is sent along when posting a form and is used to check if the request came from a trusted source. Our server is completely ignoring POST data right now, so the token isn't sent, and the request can't be verified.

Back when we first implemented our HTTP server, we used `session.gets` to get the first line (called the [Request-Line](https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1)), and parsed the HTTP method and path from that. Besides parsing the Request-Line, we ignored the rest of the request.

To be able to extract the POST data, we'll first need to understand how an HTTP request is structured. Looking at an example, we can see that the structure resembles an HTTP response:

```
POST /posts HTTP/1.1\r\n
Host: localhost:5678\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n
Accept-Encoding: gzip, deflate\r\n
Accept-Language: en-us\r\n
Content-Type: application/x-www-form-urlencoded\r\n
Origin: http://localhost:5678\r\n
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/602.2.14 (KHTML, like Gecko) Version/10.0.1 Safari/602.2.14\r\n
Cookie: _wups_session=LzE0Z2hSZFNseG5TR3dEVEwzNE52U0lFa0pmVGlQZGtZR3AveWlyMEFvUHRPeXlQUzQ4L0xlKzNLVWtqYld2cjdiWkpmclZIaEhJd1R6eDhaZThFbVBlN2p6QWpJdllHL2F4Z3VseUZ6NU1BRTU5Y1crM2lLRVY0UzdSZkpwYkt2SGFLZUQrYVFvaFE0VjZmZlIrNk5BPT0tLUpLTHQvRHQ0T3FycWV0ZFZhVHZWZkE9PQ%3D%3D--4ef4508c936004db748da10be58731049fa190ee\r\n
Connection: keep-alive\r\n
Upgrade-Insecure-Requests: 1\r\n
Referer: http://localhost:5678/posts/new\r\n
Content-Length: 369\r\n
\r\n
utf8=%E2%9C%93&authenticity_token=3fu7e8v70K0h9o%2FGNiXxaXSVg3nZ%2FuoL60nlhssUEHpQRz%2BM4ZIHjQduQMexvXrNoC2pjmhNPI4xNNA0Qkh5Lg%3D%3D&post%5Btitle%5D=My+first+post&post%5Bcreated_at%281i%29%5D=2017&post%5Bcreated_at%282i%29%5D=1&post%5Bcreated_at%283i%29%5D=23&post%5Bcreated_at%284i%29%5D=18&post%5Bcreated_at%285i%29%5D=47&post%5Bbody%5D=It+works%21&commit=Create+Post
```

Much like a response, an HTTP request consists of:

- A Request-Line (`POST /posts HTTP/1.1\r\n`), consisting of a method token (`POST`), a request URI (`/posts/`), and the HTTP version (`HTTP/1.1`), followed by a CRLF (a carriage return: \r, followed by line feed: \n) to indicate the end of the line
- Header lines (`Host: localhost:5678\r\n`). The header key, followed by a colon, then the value, and a CRLF.
- A newline (or a double CRLF) to separate the request line and headers from the body: (`\r\n\r\n`)
- The URL encoded POST body

After using `session.gets` to take the first line of the request (the Request-Line), we're left with some header lines and a body. To get the header lines, we need to retrieve lines from the session until we find a newline (`\r\n`).

For each header line, we'll split on the first colon. Everything before the colon is the key, and everything after is the value. We `#strip` the value to remove the newline from the end.

To know how many bytes we need to read from the request to get the body, we use the "Content-Length" header, which the browser automatically includes when sending a request.

``` ruby
# http_server.rb
# ...
  headers = {}
  while (line = session.gets) != "\r\n"
    key, value = line.split(':', 2)
    headers[key] = value.strip
  end

  body = session.read(headers["Content-Length"].to_i)
# ...
```

Now, instead of sending an empty object, we'll send a `StringIO` instance with the body we received via the request. Also, since we're now parsing the cookies from the request's header, we can add them to the Rack environment in the `HTTP_COOKIE` variable to pass the request authenticity check.

``` ruby
# http_server.rb
# ...
  status, headers, body = app.call({
    # ...
    'REMOTE_ADDR' => '127.0.0.1',
    'HTTP_COOKIE' => headers['Cookie'],
    'rack.version' => [1,3],
    'rack.input' => StringIO.new(body),
    'rack.errors' => $stderr,
    # ...
  })
# ...
```

There we go. If we restart the server and try to submit the form again, you'll see that we successfully created the first post on our blog!

![Post was successfully created.](https://blog.appsignal.com/images/blog/2017-01/first-post.png)

We seriously upgraded our web server this time. Instead of just accepting GET requests from a Rack app, we're now serving a complete Rails app that handles POST requests. And we still haven't written more than fifty lines of code in total!

If you want to play around with our new and improved server, [here's the code](https://gist.github.com/jeffkreeftmeijer/43d46bca63de648436d6018c73bd436b). Let us know if you want to know more, or have a specific question!