Rails 7 pagination with Turbo Frame / Streams / Infinite Scrolling / Stimulus JS
Tags: rails, tutorial, webdev, ruby
Published: 2022-11-15 01:00:42

After working on a few projects using Rails 7 and Turbo, I noticed that they all used [Turbo Frames](https://turbo.hotwired.dev/reference/frames) and [Turbo Streams](https://turbo.hotwired.dev/reference/streams) in different ways.

Now that I have some perspective on this subject, I wanted to write down a guide on how to have a modular pagination system (effortlessly switching between next/prev+next/infinite scrolling)  using Turbo Frames and Turbo Streams, and then add a tiny bit of javascript using [Stimulus JS](https://stimulus.hotwired.dev/) to polish things up ✨.


As of October 2022, I'd recommend reading those two amazing blog posts as we all used different approaches to achieve our goal:
👉 [Infinite scrolling pagination with Rails, Hotwire, and Turbo](https://www.bearer.com/blog/infinite-scrolling-pagination-hotwire) from Bearer (❤️ what they're doing). 
👉 [Pagination and infinite scrolling with Rails and the Hotwire stack](https://www.colby.so/posts/pagination-and-infinite-scrolling-with-hotwire) from Colby.so

---
##Read-bait 

If you want to achieve manual paging with **prev/next** buttons (carousel style), manual **append-only** or  **infinite scrolling** in less than **100 lines of code** (I told you it was a bait), you're at the right place.

![infinite scrolling demo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eumeaobftl6hzicwybn3.gif)

---
## Table Of Contents

  * [1. Setup](#setup)
  * [2. Styling with TailwindCSS](#styling)
  * [3. Paginating with Pagy](#pagy)
  * [4. Navigating with Turbo Frames and Turbo Streams](#navigating)
  * [5. Next page only pagination](#paging_with_next)
  * [6. Prev/Next page pagination](#paging_with_prev_next)
  * [7. Infinite scrolling](#paging_with_infinite_scrolling)
  * [8. History.pushState](#history_push_state)
  * [9. Non-technical conclusion](#conclusion)

---
## <a name="setup">1. Setup</a>

Generate a new Rails application using `esbuild` and `tailwindcss` to make things look fancier than the default scaffold. Make sure you're using a version of `@hotwired/turbo` >= 7.2.0 (via `turbo-rails` or directly) as Turbo Streams using GET requests [were introduced](https://github.com/hotwired/turbo/pull/612) on 7.2.0

```shell
$ rails new rails-demo-turbo-stream-pagination --javascript=esbuild --css=tailwind
```

```shell
$ rails -v
7.0.4
```

Add these two well-known gem in the Rails community to our `Gemfile`: 
```
# Gemfile
gem "faker" # to populate our seed data
gem "pagy" # pagination gem
```

Scaffold our `Article` model
```shell
rails g scaffold Article title:string cover_url:string body:text
```

And create our `Article` instances
```ruby
# seeds.rb
100.times do
  Article.create(
    title: Faker::Book.title,
    body: Faker::Lorem.paragraph,
    cover_url: "https://source.unsplash.com/random/800x600?book"
  )
end
```

```shell
$ rake db:create
$ rake db:migrate
$ rake db:seed
$ bin/dev
```

Take a trip to `localhost:3000/articles`, and your screen should look like this 👇

![default scaffold for articles/index](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1q3lp4jx8f1rfipmfund.png)
Looking good isn't it?


---
## <a name="styling">2. Styling with TailwindCSS</a>

Let's setup a minimal set of styles so we can pretend to work on a real-life application. Thanks to [TailwindCSS](https://tailwindcss.com/), **we can get something decent in less than 2 minutes**, just by updating those 3 files:  
`application.html.erb`
```erb
<body class="bg-gradient-to-r from-pink-50 to-sky-50 container px-12 xl:px-0 py-24 mx-auto max-w-5xl antialiased">
  <%= yield %>
</body>
```

`articles/index.html.erb`
```erb

<div class="flex justify-between items-center mb-12">
  <h1 class="text-2xl font-bold font-mono">Articles</h1>
  <%= link_to "Write an article ✍️ ", new_article_path, class: "px-6 py-3 bg-white text-black rounded shadow border border-black" %>
</div>

<div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
  <% @articles.each do |article| %>
    <%= render "article" %>
  <% end %>
</div>

```

`articles/article.html.erb`
```erb
<div id="<%= dom_id article %>" class="bg-white border border-black overflow-hidden rounded shadow flex flex-col">
  <%= image_tag article.cover_url, class: "w-full object-fill h-64" %>
  <div class="px-8 py-6 flex-1">
    <h2 class="font-mono text-xl font-bold"><%= article.title %></h2>
    <p class="mt-4"><%= article.body.truncate(128) %></p>
  </div>
  <p class="px-8 pb-6 text-gray-500">#<%= article.id %> @ <%= l(article.created_at, format: :short) %></p>
</div>
```

Hit refresh and you're now working on that look waaay nicer! Still can't believe this took us less than 5 min and a few lines of code.


![article instances scaffold with tailwindcss classes](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gpcbv319nf455zuia5el.png)



---
## <a name="pagy">3. Paginating with Pagy</a>

We will follow the setup instructions from [Pagy repository](https://github.com/ddnexus/pagy) to keep going: 

`app/controllers/application_controller.rb`
```ruby
include Pagy::Backend
```

`app/helpers/application_helper.rb`
```ruby
include Pagy::Frontend
```

Update `app/controllers/articles_controller.rb` to use pagy. I picked 3 as I wanted @articles to fit within my viewport. 
```ruby
def index
  @pagy, @articles = pagy(Article.all, items: 3)
end
```

And finally, update `app/views/articles/index.html.erb` add the `pagy_nav` helper method that will display our pagination links (we won't wast our time styling it cause we're going to get rid of it very soon!).

Your `/app/views/articles/index.html.erb` should now look like this: 

```erb
<div class="flex justify-between items-center mb-12">
  <h1 class="text-2xl font-bold font-mono">Articles</h1>
  <%= link_to "Write an article ✍️ ", new_article_path, class: "px-6 py-3 bg-white text-black rounded shadow border border-black" %>
</div>

<div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
  <% @articles.each do |article| %>
    <%= render "article" %>
  <% end %>
</div>

<div class="flex mt-12 items-center justify-center space-x-4 w-full">
  <%== pagy_nav(@pagy) %>
</div>
<p class="text-center mt-8 text-gray-500"><%== pagy_info @pagy %></p>
```

You should now have a page with a fully functioning  pagination and extra information about the pagination cursor.

![articles/index with pagination helpers](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a0irx1hxryexyhki4dk5.png)

---
## <a name="navigating">4. Navigating with Turbo Frames and Turbo Streams</a>

I'll skip the whole part where you could just use Turbo Frame to avoid full page reload as it is extremely well described in both Bearer and Colby.so articles mentioned in the introduction.

Instead, I will focus on how to make our pagination system as flexible as possible, as you might need different types of pagination within the same application. Whether we'll be using infinite scrolling, carousel, or append-only pagination, <strong>we'll only need 2 distinct Turbo Frames</strong> 💥. 

### Placeholder/Skeleton

To begin, we'll add what is called a placeholder, or skeleton (If you're reading this article from 2010, this is just a replacement for what we use to name "spinner.gif" 😂) to keep our users waiting while our server will be fetching its response.

Create a new file `article_placeholder.html.erb`, then add the following code to create a blank article card. 

`app/views/articles/article_placeholder.html.erb`
```erb
<div class="bg-white border border-black overflow-hidden rounded shadow flex flex-col">
  <div class="bg-gray-100 w-full h-64 block"></div>
  <div class="px-8 py-6 flex-1">
    <h2 class="font-mono text-xl font-bold"><span class="w-3/4 py-2.5 rounded animate-pulse bg-gray-200 block" /></h2>
    <div class="mt-8 space-y-4">
      <p class="w-full py-1.5 rounded animate-pulse bg-gray-200 block"></p>
      <p class="w-5/6 py-1.5 rounded animate-pulse bg-gray-200 block"></p>
      <p class="w-full py-1.5 rounded animate-pulse bg-gray-200 block"></p>
    </div>
    <p class="mt-8 w-5/6 py-1.5 rounded animate-pulse bg-gray-200 block"></p>
  </div>
</div>
```

Keep in mind that you want your placeholder to be meaningful to your user and extremely fast to load (do not load external resources). Hint: add `sleep(2)` to your `ArticleController#index` to observe your placeholders/skeletons. 

Once again, thanks to TailwindCSS we can fire those things up super easily with a few lines of code. Rendering this partial should give you this exact article card placeholder.

![article card placeholder image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s7c0xjny0l6nogoq6v2x.png)

### Turbo Frame #1: Article collection 

**Role:**
👉 Using manual pagination (with a next and/or prev button), we'll use this frame to `append` or `replace` our paginated articles collection.
👉 Using infinite scrolling, we'll just `append` our paginated articles into this frame.

**Code:**
We'll replace our `@articles` collection iteration from `app/views/articles/index.html.erb` with a `<turbo-frame>` tag using [lazy loading](https://turbo.hotwired.dev/reference/frames#lazy-loaded-frame) to fetch articles as a GET turbo_stream request using the following lines:

We're replacing 

```erb
<div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
  <% @articles.each do |article| %>
    <%= render "article" %>
  <% end %>
</div>
```

By this

```erb
<%= turbo_frame_tag "articles_list", src: articles_url(format: :turbo_stream, page: params[:page]), loading: "lazy" do %>
  <div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8" id="articles_placeholder">
    <!-- Adjust this to your Pagy default item count -->
    <% 3.times do %>
      <%= render "article_placeholder" %>
    <% end %>
  </div>
<% end %>
```

### Turbo Frame #2: Pagination

**Role:**
👉 Using manual pagination (with a next and/or prev button), we'll use this frame to append or replace paginated articles. 
👉 Using infinite scrolling, we'll just append paginated articles into this frame (we'll get back to this later).

**Code:**
Add to `app/articles/index.html.erb`
```erb
<%= turbo_frame_tag "articles_pagination" %>
```

Our `app/views/articles/index.html.erb` should looks like this:

```erb
<div class="flex justify-between items-center mb-12">
  <h1 class="text-2xl font-bold font-mono">Articles</h1>
  <%= link_to "Write an article ✍️ ", new_article_path, class: "px-6 py-3 bg-white text-black rounded shadow border border-black" %>
</div>

<%= turbo_frame_tag "articles_list", src: articles_url(format: :turbo_stream, page: params[:page]), loading: "lazy" do %>
  <div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8" id="articles_placeholder">
    <!-- Adjust this to your Pagy default item count -->
    <% 3.times do %>
      <%= render "article_placeholder" %>
    <% end %>
  </div>
<% end %>

<%= turbo_frame_tag "articles_pagination" %>
```

### Turbo Stream Response

Now that our `app/views/articles/index.html.erb` is setup with Turbo Frames, we're ready to accept a Turbo Stream format response from ArticlesController.

`app/controllers/articles_controller.rb`

```ruby 
def index
  @pagy, @articles = pagy(Article.all, items: 3)
    
  respond_to do |format|
    format.html 
    format.turbo_stream
  end
end
```

Simple enough right? That's because the magic 🪄 will happen in `format.turbo_stream`, which will resolve to `app/views/articles/index.turbo_stream.erb`. 

```shell
$ touch app/views/articles/index.turbo_stream.erb 
```

--- 
## <a name="paging_with_next">5. Next page only pagination</a>

![append-only pagination](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8jdcqzd63fct4b5msbjp.gif)

To achieve an append-only pagination, we will add the following code to the newly created `app/views/articles/index.turbo_stream.erb`: 

```erb
<!-- remove placeholders -->
<%= turbo_stream.remove("articles_placeholder") %>
<!-- append @articles to the #article_list turbo_frame -->
<%= turbo_stream.append("articles_list") do %>
  <div class="mt-8 md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
    <% @articles.each do |article| %>
      <%= render article %>
    <% end %>
  </div> 
<% end %>
<!-- update the pagination turbo_frame with the current pagination offset -->
<%= turbo_stream.update "articles_pagination" do %>
  <% if @pagy.next %>
    <div class="text-center mt-12">
      <%= link_to "Next", articles_url(page: @pagy.next), data: { turbo_stream: true }, class: "bg-white rounded px-6 py-3 border border-black shadow" %>
      <p class="mt-8 text-gray-500"><%== pagy_info @pagy %></p>
    </div>
  <% end %>
<% end %> 
```

We're using 3 Turbo Streams here to: 
1. Remove skeletons
2. Append records to our `#article_list` Turbo Frame
3. Update pagination with the latests offsets and informations. 

--- 
## 6. <a name="paging_with_prev_next">Prev/Next page pagination</a>


![prev/next navigation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b4zs8mdnyzh6t74xpd8l.gif)

To achieve a prev/next pagination, we will add the following code to `app/views/articles/index.turbo_stream.erb`: 

```erb 
<!-- update @articles collection -->
<%= turbo_stream.update("articles_list") do %>
  <div class="mt-8 md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
    <% @articles.each do |article| %>
      <%= render article %>
    <% end %>
  </div>
<% end %>
<!-- update the pagination turbo_frame with the current pagination offset -->
<%= turbo_stream.update "articles_pagination" do %>
  <div class="flex mt-12 items-center justify-center space-x-4 w-full">
    <% if @pagy.prev %>
      <%= link_to "Prev", articles_url(page: @pagy.prev), data: { turbo_stream: true }, class: "bg-white rounded px-6 py-3 border border-black shadow" %>
    <% end %>
    <% if @pagy.next %>
      <%= link_to "Next", articles_url(page: @pagy.next), data: { turbo_stream: true }, class: "bg-white rounded px-6 py-3 border border-black shadow" %>
    <% end %> 
  </div>
  <p class="text-center mt-8 text-gray-500"><%== pagy_info @pagy %></p>
<% end %>  
```

We're using 2 Turbo Streams here to: 
1. Update records from `#article_list` Turbo Frame. Since update is used, skeletons will get replaced!
2. Update pagination with the latests offsets and informations. 

--- 
## 7. <a name="paging_with_infinite_scrolling">Infinite scrolling</a>

![infinite scrolling navigation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sbirq2egy5rx4twx8n9h.gif)

Last but not least, we can achieve an infinite scroll pagination pretty easily thanks to the `loading="lazy"` attribute on a `<turbo-frame>` tag. 
According to the [documentation](https://turbo.hotwired.dev/reference/frames#lazy-loaded-frame), our frame content will **only** be loaded if the frame is visible. 
Let's use this to perform the following actions when the frame gets visible (or in other words, when we reach the bottom of our page):  
👉 Add new content to our article collection Turbo Frame
👉 Paginate to the next offset as soon as we reach the bottom of the page. 

Good news is that we've already wrote this piece of code, we just need to call it!

`app/views/articles/index.html.erb`: 
```erb
<!-- remove skeletons -->
<%= turbo_stream.remove("articles_placeholder") %>
<!-- append @articles to the #article_list turbo_frame -->
<%= turbo_stream.append("articles_list") do %>
  <div class="mt-8 md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
    <% @articles.each do |article| %>
      <%= render article %>
    <% end %>
  </div>  
<% end %>
<!-- replace #article_pagination frame by our content loader when it gets visible (our turbo_frame_tag with loading="lazy" attribute -->
<% if @pagy.next.present? %>
  <%= turbo_stream.replace "articles_pagination" do %>
    <%= turbo_frame_tag "articles_pagination", src: articles_url(page: @pagy.next, format: "turbo_stream"), loading: "lazy" %>
  <% end %>
<% end %> 
```

And we're done!

--- 
## 8. <a name="history_push_state">History.pushState</a>

In order to perfect our paging system with Turbo Streams and Turbo Frames, we need to behave as close as possible to a traditional paging system. When clicking on a page link `/articles?page=2`, we expect our browser to update both its URL and history state. Let's see how we can bring back this behavior.

Problem: Our pagination links are 'framed' by our `#article_pagination` Turbo Frame. Every time click a link inside the frame, it will update the frame src attribute instead of our browser URL 🤷‍♀️. 

Thankfully [@Intrepidd](https://github.com/Intrepidd) wrote a [Stimulus Controller](https://gist.github.com/Intrepidd/ac68cb7dfd17d422374807efb6bf2f42?permalink_comment_id=3733333#gistcomment-3733333) doing the amazing job of observing an element attribute mutation, then push its `src` attribute to our history state using `@hotwired/turbo` `navigator` api. 

Add the controller to your code base. You can name it just like his author [did](https://gist.github.com/Intrepidd/ac68cb7dfd17d422374807efb6bf2f42?permalink_comment_id=4269474#gistcomment-4269474) TurboFrameHistory / turbo_frame_history_controller (don't forget to register it in your in your Stimulus controllers)

and replace 

```erb
<%= turbo_frame_tag "articles_pagination" %>
```
by 
```erb
<%= turbo_frame_tag "articles_pagination", data: { controller: "turbo-frame-history" } %>
```

in `app/articles/index.html.erb` and there you go! Now every time a link from the `#pagination_article` Turbo Frame is clicked, following chain will happen: 
1. the frame `src`'s attribute is updated 
2. `mutate(entries)` from our Stimulus controller is called 
3. [navigator](https://github.com/hotwired/turbo/blob/main/src/core/drive/navigator.ts).[history](https://github.com/hotwired/turbo/blob/main/src/core/drive/history.ts).push is called


Note: This technique will **NOT** work with the infinite scroll code snippet as we're not triggering any click from inside the `#article_pagination` Turbo Frame. 

If you really want the same behavior when using the infinite scroll, a solution could be to wrap the `#article_pagination` Turbo Frame and observe its [childList](https://developer.mozilla.org/fr/docs/Web/API/MutationObserver) mutation. Another solution could be to set a data-attribute on the replaced pagination turbo frame and manually trigger a mutation.

```erb
<%= turbo_stream.remove("articles_placeholder") %>
<%= turbo_stream.append("articles_list") do %>
  <div class="mt-8 md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
    <% @articles.each do |article| %>
      <%= render article %>
    <% end %>
  </div>  
<% end %>
<% if @pagy.next.present? %>
  <%= turbo_stream.replace "articles_pagination" do %>
    <%= turbo_frame_tag "articles_pagination", src: articles_url(page: @pagy.next, format: "turbo_stream"), loading: "lazy", data: { mutate_on_connect: true } %>
  <% end %>
<% end %>   
```

```js
// turbo_frame_history.js
connect () {
  useMutation(this, { attributes: true })
    if(this.element.dataset.mutateOnConnect) {
      this.mutate([{ type: 'attributes', attributeName: 'src' }])
    }
  }
}
```

And you should be good to go!

--- 
## <a name="conclusion">9. Non-technical conclusion</a>

On a final note, here's an article on [how Etsy.com purchases dropped by 22%](https://www.abtestcases.com/testcases/changing-pagination-to-infinite-scrolling-on-the-listing-page/) when switching from traditional prev/next/pages to infinite scroll on their product listing page. 

Whether you run an e-commerce store or a blog about cats, users behavior has a strategic influence on your website metrics (good metrics is a SEO boost, as well as a boost when using ads engines like Adwords or Facebook Ads), and so your revenue. 

Don't just assume that X pagination style will be the best for your website just because it's fancier to code or because it's the trend. **A/B testing is the way to go** if you want to figure things out and we just saw how Rails with Turbo made it feels like a breeze. 


🧑‍💻 You'll find the github repository of this article right here: [https://github.com/Chwistophe/rails-demo-turbo-frame-stream-pagination](https://github.com/Chwistophe/rails-demo-turbo-frame-stream-pagination)

Cheers,