Repository: rails/solid_queue (Ruby)

Issue:
Enqueued jobs do not respect queue priorities
Hey there, I'm testing out the queue priorities in solid queue. My `queue.yml` is as follows:

```yml
default: &default
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues: [urgent*, semi_urgent*, default*, low*]
      threads: 1
      processes: 1
      polling_interval: 0.1

development:
  <<: *default

test:
  <<: *default

production:
  <<: *default

```

I also have 3 test jobs for each queue priority level - A, B and C:

```ruby
# Testing a low priority job A
module TestSqJobs
  class LowPriorityJobA < ActiveJob::Base
    self.queue_adapter = :solid_queue
    queue_as :low_A
    def priority
      0
    end

    def perform
      sleep(1)
    end
  end
end
```

Finally, I execute each job 5 times using the following ruby code:
```ruby
job_classes = [
      TestSqJobs::SemiUrgentPriorityJobA,
      TestSqJobs::SemiUrgentPriorityJobB,
      TestSqJobs::SemiUrgentPriorityJobC,
      TestSqJobs::UrgentPriorityJobA,
      TestSqJobs::UrgentPriorityJobB,
      TestSqJobs::UrgentPriorityJobC,
      TestSqJobs::DefaultPriorityJobA,
      TestSqJobs::DefaultPriorityJobB,
      TestSqJobs::DefaultPriorityJobC,
      TestSqJobs::LowPriorityJobA,
      TestSqJobs::LowPriorityJobB,
      TestSqJobs::LowPriorityJobC
    ]

    job_classes.each do |job_class|
      5.times do |i|
        begin
          job_class.perform_later
          puts "Enqueued #{job_class} instance #{i + 1}"
        rescue ArgumentError => e
          puts "Skipping #{job_class} due to missing arguments: #{e.message}"
        end
      end
    end

    puts "Successfully enqueued 5 instances of each TestSqJobs job where possible."
```

I've observed this on mission control, and it seems like not all the jobs with the urgent prefix gets fulfilled first. Here's a screenshot of my observations:

<img width="1436" alt="image" src="https://github.com/user-attachments/assets/73353709-04cf-43b2-8ce4-39e5dc476031">

We can see that jobs are taking from the `default` and `semi_urgent` queue instead of clearing out the job queues with the `urgent` prefix.


Discussion:
🤔  You're enqueuing the jobs sequentially, no? And the ones enqueued first are these: 
```
      TestSqJobs::SemiUrgentPriorityJobA,
      TestSqJobs::SemiUrgentPriorityJobB,
      TestSqJobs::SemiUrgentPriorityJobC,
```

So they're going to be run before 
```
      TestSqJobs::UrgentPriorityJobA,
      TestSqJobs::UrgentPriorityJobB,
      TestSqJobs::UrgentPriorityJobC,
```
because when they're enqueued, they're the only ones there. 
Ah, I see what you mean. Yes, there's a bug there in the order of queues when they use prefixes. The order is the one returned from the DB from this query: 

```ruby
 relation.where(([ "queue_name LIKE ?" ] * prefixes.count).join(" OR "), *prefixes).distinct(:queue_name).pluck(:queue_name)
```
Which doesn't need to match the order in the `LIKE` clause. These should be reordered to match the order in the clause and the config. 

This should work fine if you don't use prefixes, though (which is what I'd recommend in this case, anyway, collapsing all your `low_`, `urgent_`, etc. queues into one). I'll fix the case with prefixes.

I see, thanks for the info @rosa! I was thinking to use wildcards together with the priority keywords so that I still can have separate queues for specific jobs while still having the benefit of using queue priorities. 

Can I confirm what will be the scheduling algorithm used within jobs that match the same wildcard queue name? So, for example my `queue.yml` file would have queues defined as: `[urgent*, semi_urgent*, default*, low*]`, and I have `urgent_A`, `urgent_B` and `urgent_C` jobs enqueued. Will the selection of these jobs be FCFS? Or would it be by random selection? Thanks!
Within the same wildcard name, `urgent_A, urgent_B, urgent_C`, the selection would be non-deterministic, it'll depend on the order the DB returns the queue names from `SELECT queue_name FROM solid_queue_ready_executions WHERE queue_name LIKE 'urgent_%'`). 

If you use exact queue names, polling will be faster, and the order will be deterministic. 
The fix for this was released in version 1.0.2, thanks again for the report!



Patch:
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index adb6b9c8..eef17fa5 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -53,5 +53,3 @@ jobs:
           bin/rails db:setup
       - name: Run tests
         run: bin/rails test
-      - name: Run tests with separate connection
-        run: SEPARATE_CONNECTION=1 bin/rails test
diff --git a/.gitignore b/.gitignore
index b1dd6310..147c05f2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -5,7 +5,7 @@
 /tmp/
 /test/dummy/db/*.sqlite3
 /test/dummy/db/*.sqlite3-*
-/test/dummy/log/*.log
+/test/dummy/log/*.log*
 /test/dummy/tmp/
 
 # Folder for JetBrains IDEs
diff --git a/.rubocop.yml b/.rubocop.yml
index d395e1e9..0269c78b 100644
--- a/.rubocop.yml
+++ b/.rubocop.yml
@@ -7,3 +7,4 @@ AllCops:
   TargetRubyVersion: 3.0
   Exclude:
     - "test/dummy/db/schema.rb"
+    - "test/dummy/db/queue_schema.rb"
diff --git a/README.md b/README.md
index f4b04364..a032270a 100644
--- a/README.md
+++ b/README.md
@@ -149,6 +149,9 @@ Here's an overview of the different options:
   This will create a worker fetching jobs from all queues starting with `staging`. The wildcard `*` is only allowed on its own or at the end of a queue name; you can't specify queue names such as `*_some_queue`. These will be ignored.
 
   Finally, you can combine prefixes with exact names, like `[ staging*, background ]`, and the behaviour with respect to order will be the same as with only exact names.
+
+  Check the sections below on [how queue order behaves combined with priorities](#queue-order-and-priorities), and [how the way you specify the queues per worker might affect performance](#queues-specification-and-performance).
+
 - `threads`: this is the max size of the thread pool that each worker will have to run jobs. Each worker will fetch this number of jobs from their queue(s), at most and will post them to the thread pool to be run. By default, this is `3`. Only workers have this setting.
 - `processes`: this is the number of worker processes that will be forked by the supervisor with the settings given. By default, this is `1`, just a single process. This setting is useful if you want to dedicate more than one CPU core to a queue or queues with the same configuration. Only workers have this setting.
 - `concurrency_maintenance`: whether the dispatcher will perform the concurrency maintenance work. This is `true` by default, and it's useful if you don't use any [concurrency controls](#concurrency-controls) and want to disable it or if you run multiple dispatchers and want some of them to just dispatch jobs without doing anything else.
@@ -164,6 +167,67 @@ This is useful when you run jobs with different importance or urgency in the sam
 
 We recommend not mixing queue order with priorities but either choosing one or the other, as that will make job execution order more straightforward for you.
 
+### Queues specification and performance
+
+To keep polling performant and ensure a covering index is always used, Solid Queue only does two types of polling queries:
+```sql
+-- No filtering by queue
+SELECT job_id
+FROM solid_queue_ready_executions
+ORDER BY priority ASC, job_id ASC
+LIMIT ?
+FOR UPDATE SKIP LOCKED;
+
+-- Filtering by a single queue
+SELECT job_id
+FROM solid_queue_ready_executions
+WHERE queue_name = ?
+ORDER BY priority ASC, job_id ASC
+LIMIT ?
+FOR UPDATE SKIP LOCKED;
+```
+
+The first one (no filtering by queue) is used when you specify
+```yml
+queues: *
+```
+and there aren't any queues paused, as we want to target all queues.
+
+In other cases, we need to have a list of queues to filter by, in order, because we can only filter by a single queue at a time to ensure we use an index to sort. This means that if you specify your queues as:
+```yml
+queues: beta*
+```
+
+we'll need to get a list of all existing queues matching that prefix first, with a query that would look like this:
+```sql
+SELECT DISTINCT(queue_name)
+FROM solid_queue_ready_executions
+WHERE queue_name LIKE 'beta%';
+```
+
+This type of `DISTINCT` query on a column that's the leftmost column in an index can be performed very fast in MySQL thanks to a technique called [Loose Index Scan](https://dev.mysql.com/doc/refman/8.0/en/group-by-optimization.html#loose-index-scan). PostgreSQL and SQLite, however, don't implement this technique, which means that if your `solid_queue_ready_executions` table is very big because your queues get very deep, this query will get slow. Normally your `solid_queue_ready_executions` table will be small, but it can happen.
+
+Similarly to using prefixes, the same will happen if you have paused queues, because we need to get a list of all queues with a query like
+```sql
+SELECT DISTINCT(queue_name)
+FROM solid_queue_ready_executions
+```
+
+and then remove the paused ones. Pausing in general should be something rare, used in special circumstances, and for a short period of time. If you don't want to process jobs from a queue anymore, the best way to do that is to remove it from your list of queues.
+
+💡 To sum up, **if you want to ensure optimal performance on polling**, the best way to do that is to always specify exact names for them, and not have any queues paused.
+
+Do this:
+
+```yml
+queues: background, backend
+```
+
+instead of this:
+```yml
+queues: back*
+```
+
 
 ### Threads, processes and signals
 
diff --git a/app/models/solid_queue/queue_selector.rb b/app/models/solid_queue/queue_selector.rb
index 77849056..24f6a6ad 100644
--- a/app/models/solid_queue/queue_selector.rb
+++ b/app/models/solid_queue/queue_selector.rb
@@ -34,7 +34,7 @@ def queue_names
       def eligible_queues
         if include_all_queues? then all_queues
         else
-          exact_names + prefixed_names
+          in_raw_order(exact_names + prefixed_names)
         end
       end
 
@@ -42,8 +42,12 @@ def include_all_queues?
         "*".in? raw_queues
       end
 
+      def all_queues
+        relation.distinct(:queue_name).pluck(:queue_name)
+      end
+
       def exact_names
-        raw_queues.select { |queue| !queue.include?("*") }
+        raw_queues.select { |queue| exact_name?(queue) }
       end
 
       def prefixed_names
@@ -54,15 +58,41 @@ def prefixed_names
       end
 
       def prefixes
-        @prefixes ||= raw_queues.select { |queue| queue.ends_with?("*") }.map { |queue| queue.tr("*", "%") }
+        @prefixes ||= raw_queues.select { |queue| prefixed_name?(queue) }.map { |queue| queue.tr("*", "%") }
       end
 
-      def all_queues
-        relation.distinct(:queue_name).pluck(:queue_name)
+      def exact_name?(queue)
+        !queue.include?("*")
+      end
+
+      def prefixed_name?(queue)
+        queue.ends_with?("*")
       end
 
       def paused_queues
         @paused_queues ||= Pause.all.pluck(:queue_name)
       end
+
+      def in_raw_order(queues)
+        # Only need to sort if we have prefixes and more than one queue name.
+        # Exact names are selected in the same order as they're found
+        if queues.one? || prefixes.empty?
+          queues
+        else
+          queues = queues.dup
+          raw_queues.flat_map { |raw_queue| delete_in_order(raw_queue, queues) }.compact
+        end
+      end
+
+      def delete_in_order(raw_queue, queues)
+        if exact_name?(raw_queue)
+          queues.delete(raw_queue)
+        elsif prefixed_name?(raw_queue)
+          prefix = raw_queue.tr("*", "")
+          queues.select { |queue| queue.start_with?(prefix) }.tap do |matches|
+            queues -= matches
+          end
+        end
+      end
   end
 end
diff --git a/bin/setup b/bin/setup
index b3e163ab..5fbe3e57 100755
--- a/bin/setup
+++ b/bin/setup
@@ -2,11 +2,19 @@
 set -eu
 cd "$(dirname "${BASH_SOURCE[0]}")"
 
-docker-compose up -d --remove-orphans
-docker-compose ps
+if docker compose version &> /dev/null; then
+  DOCKER_COMPOSE_CMD="docker compose"
+else
+  DOCKER_COMPOSE_CMD="docker-compose"
+fi
+
+$DOCKER_COMPOSE_CMD up -d --remove-orphans
+$DOCKER_COMPOSE_CMD ps
 
 bundle
 
 echo "Creating databases..."
 
-rails db:reset
+rails db:reset TARGET_DB=sqlite
+rails db:reset TARGET_DB=mysql
+rails db:reset TARGET_DB=postgres


Test patch:
diff --git a/test/dummy/bin/jobs b/test/dummy/bin/jobs
new file mode 100755
index 00000000..dcf59f30
--- /dev/null
+++ b/test/dummy/bin/jobs
@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+
+require_relative "../config/environment"
+require "solid_queue/cli"
+
+SolidQueue::Cli.start(ARGV)
diff --git a/test/dummy/config/application.rb b/test/dummy/config/application.rb
index 502f70cb..8fa76e8c 100644
--- a/test/dummy/config/application.rb
+++ b/test/dummy/config/application.rb
@@ -28,9 +28,5 @@ class Application < Rails::Application
     # config.eager_load_paths << Rails.root.join("extras")
 
     config.active_job.queue_adapter = :solid_queue
-
-    if ENV["SEPARATE_CONNECTION"] && ENV["TARGET_DB"] != "sqlite"
-      config.solid_queue.connects_to = { database: { writing: :primary, reading: :replica } }
-    end
   end
 end
diff --git a/test/dummy/config/database.yml b/test/dummy/config/database.yml
index 85f80b36..027f6706 100644
--- a/test/dummy/config/database.yml
+++ b/test/dummy/config/database.yml
@@ -35,19 +35,19 @@ default: &default
 development:
   primary:
     <<: *default
-    database: <%= database_name_from("solid_queue_development") %>
-  replica:
+    database: <%= database_name_from("development") %>
+  queue:
     <<: *default
-    database: <%= database_name_from("solid_queue_development") %>
-    replica: true
+    database: <%= database_name_from("development_queue") %>
+    migrations_paths: db/queue_migrate
 
 test:
   primary:
     <<: *default
     pool: 20
-    database: <%= database_name_from("solid_queue_test") %>
-  replica:
+    database: <%= database_name_from("test") %>
+  queue:
     <<: *default
     pool: 20
-    database: <%= database_name_from("solid_queue_test") %>
-    replica: true
+    database: <%= database_name_from("test_queue") %>
+    migrations_paths: db/queue_migrate
diff --git a/test/dummy/config/environments/development.rb b/test/dummy/config/environments/development.rb
index 4d9f1d46..da4baad4 100644
--- a/test/dummy/config/environments/development.rb
+++ b/test/dummy/config/environments/development.rb
@@ -51,6 +51,10 @@
   # Raises error for missing translations.
   # config.i18n.raise_on_missing_translations = true
 
+  # Replace the default in-process and non-durable queuing backend for Active Job.
+  config.active_job.queue_adapter = :solid_queue
+  config.solid_queue.connects_to = { database: { writing: :queue } }
+
   # Annotate rendered view with file names.
   # config.action_view.annotate_rendered_view_with_filenames = true
 
diff --git a/test/dummy/config/environments/production.rb b/test/dummy/config/environments/production.rb
index d08a6f9d..9323229a 100644
--- a/test/dummy/config/environments/production.rb
+++ b/test/dummy/config/environments/production.rb
@@ -44,8 +44,10 @@
   # Use a different cache store in production.
   # config.cache_store = :mem_cache_store
 
-  # Use a real queuing backend for Active Job (and separate queues per environment).
-  # config.active_job.queue_adapter     = :resque
+  # Replace the default in-process and non-durable queuing backend for Active Job.
+  config.active_job.queue_adapter = :solid_queue
+  config.solid_queue.connects_to = { database: { writing: :queue } }
+
   # config.active_job.queue_name_prefix = "dummy_production"
 
   # Enable locale fallbacks for I18n (makes lookups for any locale fall back to
diff --git a/test/dummy/config/environments/test.rb b/test/dummy/config/environments/test.rb
index df832026..a5a99232 100644
--- a/test/dummy/config/environments/test.rb
+++ b/test/dummy/config/environments/test.rb
@@ -47,6 +47,10 @@
   # Raises error for missing translations.
   # config.i18n.raise_on_missing_translations = true
 
+  # Replace the default in-process and non-durable queuing backend for Active Job.
+  config.active_job.queue_adapter = :solid_queue
+  config.solid_queue.connects_to = { database: { writing: :queue } }
+
   # Annotate rendered view with file names.
   # config.action_view.annotate_rendered_view_with_filenames = true
 
diff --git a/test/dummy/db/queue_schema.rb b/test/dummy/db/queue_schema.rb
new file mode 100644
index 00000000..697c2e92
--- /dev/null
+++ b/test/dummy/db/queue_schema.rb
@@ -0,0 +1,141 @@
+# This file is auto-generated from the current state of the database. Instead
+# of editing this file, please use the migrations feature of Active Record to
+# incrementally modify your database, and then regenerate this schema definition.
+#
+# This file is the source Rails uses to define your schema when running `bin/rails
+# db:schema:load`. When creating a new database, `bin/rails db:schema:load` tends to
+# be faster and is potentially less error prone than running all of your
+# migrations from scratch. Old migrations may fail to apply correctly if those
+# migrations use external dependencies or application code.
+#
+# It's strongly recommended that you check this file into your version control system.
+
+ActiveRecord::Schema[7.1].define(version: 1) do
+  create_table "solid_queue_blocked_executions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.bigint "job_id", null: false
+    t.string "queue_name", null: false
+    t.integer "priority", default: 0, null: false
+    t.string "concurrency_key", null: false
+    t.datetime "expires_at", null: false
+    t.datetime "created_at", null: false
+    t.index ["concurrency_key", "priority", "job_id"], name: "index_solid_queue_blocked_executions_for_release"
+    t.index ["expires_at", "concurrency_key"], name: "index_solid_queue_blocked_executions_for_maintenance"
+    t.index ["job_id"], name: "index_solid_queue_blocked_executions_on_job_id", unique: true
+  end
+
+  create_table "solid_queue_claimed_executions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.bigint "job_id", null: false
+    t.bigint "process_id"
+    t.datetime "created_at", null: false
+    t.index ["job_id"], name: "index_solid_queue_claimed_executions_on_job_id", unique: true
+    t.index ["process_id", "job_id"], name: "index_solid_queue_claimed_executions_on_process_id_and_job_id"
+  end
+
+  create_table "solid_queue_failed_executions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.bigint "job_id", null: false
+    t.text "error"
+    t.datetime "created_at", null: false
+    t.index ["job_id"], name: "index_solid_queue_failed_executions_on_job_id", unique: true
+  end
+
+  create_table "solid_queue_jobs", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.string "queue_name", null: false
+    t.string "class_name", null: false
+    t.text "arguments"
+    t.integer "priority", default: 0, null: false
+    t.string "active_job_id"
+    t.datetime "scheduled_at"
+    t.datetime "finished_at"
+    t.string "concurrency_key"
+    t.datetime "created_at", null: false
+    t.datetime "updated_at", null: false
+    t.index ["active_job_id"], name: "index_solid_queue_jobs_on_active_job_id"
+    t.index ["class_name"], name: "index_solid_queue_jobs_on_class_name"
+    t.index ["finished_at"], name: "index_solid_queue_jobs_on_finished_at"
+    t.index ["queue_name", "finished_at"], name: "index_solid_queue_jobs_for_filtering"
+    t.index ["scheduled_at", "finished_at"], name: "index_solid_queue_jobs_for_alerting"
+  end
+
+  create_table "solid_queue_pauses", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.string "queue_name", null: false
+    t.datetime "created_at", null: false
+    t.index ["queue_name"], name: "index_solid_queue_pauses_on_queue_name", unique: true
+  end
+
+  create_table "solid_queue_processes", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.string "kind", null: false
+    t.datetime "last_heartbeat_at", null: false
+    t.bigint "supervisor_id"
+    t.integer "pid", null: false
+    t.string "hostname"
+    t.text "metadata"
+    t.datetime "created_at", null: false
+    t.string "name", null: false
+    t.index ["last_heartbeat_at"], name: "index_solid_queue_processes_on_last_heartbeat_at"
+    t.index ["name", "supervisor_id"], name: "index_solid_queue_processes_on_name_and_supervisor_id", unique: true
+    t.index ["supervisor_id"], name: "index_solid_queue_processes_on_supervisor_id"
+  end
+
+  create_table "solid_queue_ready_executions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.bigint "job_id", null: false
+    t.string "queue_name", null: false
+    t.integer "priority", default: 0, null: false
+    t.datetime "created_at", null: false
+    t.index ["job_id"], name: "index_solid_queue_ready_executions_on_job_id", unique: true
+    t.index ["priority", "job_id"], name: "index_solid_queue_poll_all"
+    t.index ["queue_name", "priority", "job_id"], name: "index_solid_queue_poll_by_queue"
+  end
+
+  create_table "solid_queue_recurring_executions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.bigint "job_id", null: false
+    t.string "task_key", null: false
+    t.datetime "run_at", null: false
+    t.datetime "created_at", null: false
+    t.index ["job_id"], name: "index_solid_queue_recurring_executions_on_job_id", unique: true
+    t.index ["task_key", "run_at"], name: "index_solid_queue_recurring_executions_on_task_key_and_run_at", unique: true
+  end
+
+  create_table "solid_queue_recurring_tasks", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.string "key", null: false
+    t.string "schedule", null: false
+    t.string "command", limit: 2048
+    t.string "class_name"
+    t.text "arguments"
+    t.string "queue_name"
+    t.integer "priority", default: 0
+    t.boolean "static", default: true, null: false
+    t.text "description"
+    t.datetime "created_at", null: false
+    t.datetime "updated_at", null: false
+    t.index ["key"], name: "index_solid_queue_recurring_tasks_on_key", unique: true
+    t.index ["static"], name: "index_solid_queue_recurring_tasks_on_static"
+  end
+
+  create_table "solid_queue_scheduled_executions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.bigint "job_id", null: false
+    t.string "queue_name", null: false
+    t.integer "priority", default: 0, null: false
+    t.datetime "scheduled_at", null: false
+    t.datetime "created_at", null: false
+    t.index ["job_id"], name: "index_solid_queue_scheduled_executions_on_job_id", unique: true
+    t.index ["scheduled_at", "priority", "job_id"], name: "index_solid_queue_dispatch_all"
+  end
+
+  create_table "solid_queue_semaphores", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
+    t.string "key", null: false
+    t.integer "value", default: 1, null: false
+    t.datetime "expires_at", null: false
+    t.datetime "created_at", null: false
+    t.datetime "updated_at", null: false
+    t.index ["expires_at"], name: "index_solid_queue_semaphores_on_expires_at"
+    t.index ["key", "value"], name: "index_solid_queue_semaphores_on_key_and_value"
+    t.index ["key"], name: "index_solid_queue_semaphores_on_key", unique: true
+  end
+
+  add_foreign_key "solid_queue_blocked_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
+  add_foreign_key "solid_queue_claimed_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
+  add_foreign_key "solid_queue_failed_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
+  add_foreign_key "solid_queue_ready_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
+  add_foreign_key "solid_queue_recurring_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
+  add_foreign_key "solid_queue_scheduled_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
+end
diff --git a/test/models/solid_queue/job_test.rb b/test/models/solid_queue/job_test.rb
index a9b3cc59..ed8ba090 100644
--- a/test/models/solid_queue/job_test.rb
+++ b/test/models/solid_queue/job_test.rb
@@ -249,21 +249,19 @@ class NonOverlappingGroupedJob2 < NonOverlappingJob
     end
   end
 
-  if ENV["SEPARATE_CONNECTION"] && ENV["TARGET_DB"] != "sqlite"
-    test "uses a different connection and transaction than the one in use when connects_to is specified" do
-      assert_difference -> { SolidQueue::Job.count } do
-        assert_no_difference -> { JobResult.count } do
-          JobResult.transaction do
-            JobResult.create!(queue_name: "default", value: "this will be rolled back")
-            StoreResultJob.perform_later("enqueued inside a rolled back transaction")
-            raise ActiveRecord::Rollback
-          end
+  test "enqueue successfully inside a rolled-back transaction in the app DB" do
+    assert_difference -> { SolidQueue::Job.count } do
+      assert_no_difference -> { JobResult.count } do
+        JobResult.transaction do
+          JobResult.create!(queue_name: "default", value: "this will be rolled back")
+          StoreResultJob.perform_later("enqueued inside a rolled back transaction")
+          raise ActiveRecord::Rollback
         end
       end
-
-      job = SolidQueue::Job.last
-      assert_equal "enqueued inside a rolled back transaction", job.arguments.dig("arguments", 0)
     end
+
+    job = SolidQueue::Job.last
+    assert_equal "enqueued inside a rolled back transaction", job.arguments.dig("arguments", 0)
   end
 
   private
diff --git a/test/models/solid_queue/ready_execution_test.rb b/test/models/solid_queue/ready_execution_test.rb
index 9f904c89..dd9269ca 100644
--- a/test/models/solid_queue/ready_execution_test.rb
+++ b/test/models/solid_queue/ready_execution_test.rb
@@ -49,28 +49,51 @@ class SolidQueue::ReadyExecutionTest < ActiveSupport::TestCase
     end
   end
 
-  test "queue order and then priority is respected when using a list of queues" do
+  test "claim jobs using a wildcard" do
     AddToBufferJob.perform_later("hey")
-    job = SolidQueue::Job.last
-    assert_equal "background", job.queue_name
 
-    assert_claimed_jobs(3) do
-      SolidQueue::ReadyExecution.claim(%w[ background backend ], 3, 42)
+    assert_claimed_jobs(6) do
+      SolidQueue::ReadyExecution.claim("*", SolidQueue::Job.count + 1, 42)
     end
+  end
 
-    assert job.reload.claimed?
-    @jobs.first(2).each do |job|
-      assert_not job.reload.ready?
-      assert job.claimed?
+  test "claim jobs using queue prefixes" do
+    AddToBufferJob.perform_later("hey")
+
+    assert_claimed_jobs(1) do
+      SolidQueue::ReadyExecution.claim("backgr*", SolidQueue::Job.count + 1, 42)
     end
+
+    assert @jobs.none?(&:claimed?)
   end
 
-  test "claim jobs using a wildcard" do
+  test "claim jobs using a wildcard and having paused queues" do
     AddToBufferJob.perform_later("hey")
 
-    assert_claimed_jobs(6) do
+    SolidQueue::Queue.find_by_name("backend").pause
+
+    assert_claimed_jobs(1) do
       SolidQueue::ReadyExecution.claim("*", SolidQueue::Job.count + 1, 42)
     end
+
+    @jobs.each(&:reload)
+    assert @jobs.none?(&:claimed?)
+  end
+
+  test "claim jobs using both exact names and a prefix" do
+    AddToBufferJob.perform_later("hey")
+
+    assert_claimed_jobs(6) do
+      SolidQueue::ReadyExecution.claim(%w[ backe* background ], SolidQueue::Job.count + 1, 42)
+    end
+  end
+
+  test "claim jobs for queue without jobs at the moment using prefixes" do
+    AddToBufferJob.perform_later("hey")
+
+    assert_claimed_jobs(0) do
+      SolidQueue::ReadyExecution.claim(%w[ none* ], SolidQueue::Job.count + 1, 42)
+    end
   end
 
   test "priority order is used when claiming jobs using a wildcard" do
@@ -88,43 +111,61 @@ class SolidQueue::ReadyExecutionTest < ActiveSupport::TestCase
     end
   end
 
-  test "claim jobs using queue prefixes" do
+  test "queue order and then priority is respected when using a list of queues" do
     AddToBufferJob.perform_later("hey")
+    job = SolidQueue::Job.last
+    assert_equal "background", job.queue_name
 
-    assert_claimed_jobs(1) do
-      SolidQueue::ReadyExecution.claim("backgr*", SolidQueue::Job.count + 1, 42)
+    assert_claimed_jobs(3) do
+      SolidQueue::ReadyExecution.claim(%w[ background backend ], 3, 42)
     end
 
-    assert @jobs.none?(&:claimed?)
+    assert job.reload.claimed?
+    @jobs.first(2).each do |job|
+      assert_not job.reload.ready?
+      assert job.claimed?
+    end
   end
 
-  test "claim jobs using a wildcard and having paused queues" do
-    AddToBufferJob.perform_later("hey")
+  test "queue order is respected when using prefixes" do
+    %w[ queue_b1 queue_b2 queue_a2 queue_a1 queue_b1 queue_a2 queue_b2 queue_a1 ].each do |queue_name|
+      AddToBufferJob.set(queue: queue_name).perform_later(1)
+    end
 
-    SolidQueue::Queue.find_by_name("backend").pause
+    # Claim 8 jobs
+    claimed_jobs = []
+    4.times do
+      assert_claimed_jobs(2) do
+        SolidQueue::ReadyExecution.claim(%w[ queue_b* queue_a* ], 2, 42)
+      end
 
-    assert_claimed_jobs(1) do
-      SolidQueue::ReadyExecution.claim("*", SolidQueue::Job.count + 1, 42)
+      claimed_jobs += SolidQueue::ClaimedExecution.order(:id).last(2).map(&:job)
     end
 
-    @jobs.each(&:reload)
-    assert @jobs.none?(&:claimed?)
+    # Check claim order
+    assert_equal %w[ queue_b1 queue_b1 queue_b2 queue_b2 queue_a1 queue_a1 queue_a2 queue_a2 ],
+      claimed_jobs.map(&:queue_name)
   end
 
-  test "claim jobs using both exact names and a prefixes" do
-    AddToBufferJob.perform_later("hey")
 
-    assert_claimed_jobs(6) do
-      SolidQueue::ReadyExecution.claim(%w[ backe* background ], SolidQueue::Job.count + 1, 42)
+  test "queue order is respected when mixing exact names with prefixes" do
+    %w[ queue_b1 queue_b2 queue_a2 queue_c2 queue_a1 queue_c1 queue_b1 queue_a2 queue_b2 queue_a1 ].each do |queue_name|
+      AddToBufferJob.set(queue: queue_name).perform_later(1)
     end
-  end
 
-  test "claim jobs for queue without jobs at the moment using prefixes" do
-    AddToBufferJob.perform_later("hey")
+    # Claim 10 jobs
+    claimed_jobs = []
+    5.times do
+      assert_claimed_jobs(2) do
+        SolidQueue::ReadyExecution.claim(%w[ queue_a2 queue_c1 queue_b* queue_c2 queue_a* ], 2, 42)
+      end
 
-    assert_claimed_jobs(0) do
-      SolidQueue::ReadyExecution.claim(%w[ none* ], SolidQueue::Job.count + 1, 42)
+      claimed_jobs += SolidQueue::ClaimedExecution.order(:id).last(2).map(&:job)
     end
+
+    # Check claim order
+    assert_equal %w[ queue_a2 queue_a2 queue_c1 queue_b1 queue_b1 queue_b2 queue_b2 queue_c2 queue_a1 queue_a1 ],
+      claimed_jobs.map(&:queue_name)
   end
 
   test "discard all" do