HELP NEEDED: Understanding Rails ActionController::Live Module (and Async Limitations)
Tags: help, rails
Published: 2019-11-11 06:38:52

---
title: "HELP NEEDED: Understanding Rails ActionController::Live Module (and Async Limitations)"
published: true
description: help needed to understand the async limitations of rails actioncontroller::live module while attempting to stream data from another rails server
tags: help, rails
---

# The Situation: Practicing Data Ingestion in Rails (and Trying to be Async)

So, for reasons that (**ahem**) may or may not be job-interview-related, I'm practicing building a Rails server for ingesting, parsing, and storing data in a database.

My goal was to make two Rails apps, and have them talk to each other:
1. A server that **broadcasts hashes** containing randomly-generated characters (ideally repeating every 1 second, indefinitely)
2. A server that **listens for and receives the hashes,** and parses them as they're received to store in a database

I intended to implement this behavior asynchronously with the [**ActionController::Live**](https://api.rubyonrails.org/classes/ActionController/Live.html) module. The intended behavior is for the **Broadcast server** to emit a hash *(as a string)* every 1 second, and for the **Receiver server** to parse and store each hash as they come in. (For my tests, I have this looping 5 times.)

**My problem is that the character-hashes are rendered 1-by-1 when testing in my browser and in the Broadcast server's console...**

![gif of hashes being rendered in Chrome browser one by one, with a 1 second delay](https://thepracticaldev.s3.amazonaws.com/i/2sqkbkcwq08ta3uvni7x.gif)
*working fine in Chrome...*

![gif of hashes being rendered in console one by one, with a 1 second delay](https://thepracticaldev.s3.amazonaws.com/i/0vklf508fs3l7kxmj97u.gif)
*working fine in console...*

**...but in the Receiver server's console, all the character-hashes come at once in the HTTP response!**

![gif of hashes being returned as one http res.body response in Rails console](https://thepracticaldev.s3.amazonaws.com/i/q0n3kyfool02ip4qhwlx.gif)
*all the JSON is received as one response body!*

### So why is the async behavior (apparently) working in some places, and not in others?


# Broadcaster server
The Broadcaster server is a simple Rails app that uses a `BroadcasterController` with `ActionController::Live` and its [server-side event (`SSE`) module](https://api.rubyonrails.org/classes/ActionController/Live/SSE.html). 

The `index` method generates a random `character_hash`, writes it to the current SSE `response.stream` in the variable `sse`, and pauses for 1 second to illustrate async behavior.

```ruby
# Broadcaster app
# /app/controllers/broadcaster_controller.rb

class BroadcasterController < ApplicationController
    include ActionController::Live

    def index
        name_array = ["Ryu", "Peco", "Rei", "Momo", "Garr", "Nina"]
        hp_array = [132, 71, 15, 1, 0, 325]
        magic_array = ["Frost", "Typhoon", "Magic Ball", "Ascension", "Rejuvinate", "Weretiger"]

        response.headers['Content-Type'] = "text/event-stream"
        sse = SSE.new(response.stream)

        begin
            5.times do      
                character_hash = {
                    "uuid": SecureRandom.uuid,
                    "name": name_array.sample,
                    "hp": hp_array.sample,
                    "magic": magic_array.sample
                }
                sse.write({ character: character_hash })
                sleep 1
            end
        rescue IOError
            # client disconnected
        ensure
            sse.close
        end
    end
end
```
```ruby
# Broadcaster app
# /config/routes.rb

Rails.application.routes.draw do
  get 'broadcaster' => 'broadcaster#index'
end
```
Once we start the server with `rails s`, we can use `curl -i http://localhost:3000/broadcaster` in the command line to send a Get request to the `index` method. The response will return each character with a 1 second delay in-between:

![gif of hashes being rendered in console one by one, with a 1 second delay](https://thepracticaldev.s3.amazonaws.com/i/0vklf508fs3l7kxmj97u.gif)

Since navigating to `http://localhost:3000/broadcaster` in the browser will also send a Get request, we see the same behavior here in Chrome:

![gif of hashes being rendered in Chrome browser one by one, with a 1 second delay](https://thepracticaldev.s3.amazonaws.com/i/2sqkbkcwq08ta3uvni7x.gif)

So far, so good...




# Receiver server
The other Rails app is a Receiver server that sends a Get request to the Broadcaster at `http://localhost:3000/broadcaster`, and parses its response to store the received characters in the database. 

We also have it `puts` a readout to show us the `res.body` characters arriving all at once, instead of asynchronously as we saw above.

```ruby
# Receiver app
# /app/controllers/listener_controller.rb

require 'net/http'

class ListenerController < ApplicationController
    def index
        url = URI.parse('http://localhost:3000/broadcaster')
        req = Net::HTTP::Get.new(url.to_s)
        res = Net::HTTP.start(url.host, url.port) { |http| http.request(req) }

        puts <<-READOUT

    res.body:
    ==============================
    #{res.body}

        READOUT

        char_array = res.body.split("\n\n")
        char_array.each do |data_str|
            data_hash = eval(data_str.slice!(6..-1))    # slice to remove leading "data: " substring
            char_hash = data_hash[:character]
            Character.create("uuid": char_hash[:uuid], "name": char_hash[:name], "hp": char_hash[:hp], "magic": char_hash[:magic])
        end
    end
end
```

```ruby
# Receiver app
# /config/routes.rb

Rails.application.routes.draw do
  get 'listener' => 'listener#index'
end
```

Thus, when we start the server with `rails server -p 3001` and send a Get request with `curl -i http://localhost:3001/listener`, we call the ListenerController's `index` method. 

Here, `index` sends a Get request to our Broadcaster server at `localhost:3000/broadcaster`. **But instead of seeing the asynchronous behavior we saw before, it all arrives at once:**

![gif of hashes being returned as one http res.body response in Rails console](https://thepracticaldev.s3.amazonaws.com/i/q0n3kyfool02ip4qhwlx.gif)

So, instead of parsing each character *as they come in as separate objects*, we have to split the `res.body` into separate strings. **And of course, we have to wait until all 5 characters are finished generating before we receive them**. So much for scaling it up to send an unlimited number of characters!


# Where I'm At

From the research I've done, I *think* the async behavior is being limited by Rails' use of the standard HTTP request/response cycle as the basis for `ActionController::Live`. As such, **each request only gets one response,** and that's why all the characters have to come back as one `res.body` string!

Per [this excellent article by Eric Bidelman covering SSEs in HTML5](https://medium.com/@fbazzarella/streaming-data-with-server-sent-events-b0694e6e0b32), I *thought* I was moving toward implementing **long polling**...but apparently not.

Further, the tutorials I'm following usually expect us to build a JavaScript event listener to catch the async data from the Broadcaster server. **So, is it just browser-magic that's making the ActionController::Live async behavior work in Chrome?** 

But then, why do the characters still **appear** to be coming in asynchronously when we use `curl` directly on `localhost:3000/broadcaster`...

...and **NOT** when using `curl` __*indirectly*__ through `localhost:3001/listener`?

## Any help, advice, or insight is greatly appreciated! <3

### Links/Tutorials used:
*  **Main tutorial followed:** https://tenderlovemaking.com/2012/07/30/is-it-live.html

* https://stackoverflow.com/questions/17678068/long-polling-push-events-between-rails-and-ruby-client/17707334#17707334

* https://stackoverflow.com/questions/35057075/rails-actioncontrollerlive-sends-everything-at-once-instead-of-async

* https://stackoverflow.com/questions/21763527/asynchronous-responses-in-ruby-on-rails-4-using-sse

* https://www.html5rocks.com/en/tutorials/eventsource/basics/

* https://medium.com/@fbazzarella/streaming-data-with-server-sent-events-b0694e6e0b32