【Rails】RSpecでのシステムテスト作成に大苦戦

dockerを使用してRailsのWebアプリを作成中。
RSpecでシステムテストを作成、`bundle exec rspec`でテストを行うとエラー発生。
```
Selenium::WebDriver::Error::UnknownError: 
unknown error: Chrome failed to start: exited abnormally.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
```
`DevToolsActivePort file doesn't exist`はChromeブラウザがSeleniumと連携するために必要なポートを開放できていないことを意味している。

#### 関連ファイルまとめ
- docker-compose.yml
  複数のサービス（コンテナ）を定義する設定ファイル。アプリケーションに必要なコンテナを一度に起動できるようにする。
  - db
    PostgreSQLデータベースを提供。
    healthcheckによってdbが正常に稼働しているかを確認している。
  - web
    Railsアプリケーションをホストする。
    dockerfile: Dockerfile.devによって、開発環境用のDockerfileを使用してビルドする。
    depends_onでdbが健康であることを確認した後に起動。
  - selenium_chrome
    SeleniumとChromeブラウザを含むコンテナ。
    このコンテナがテスト中に使用され、4444ポートを通じてWebDriverが通信する。
    ```diff_ruby
    # dompose.yml
    
    services:
      db:
        image: postgres
        restart: always
        environment:
          TZ: Asia/Tokyo
          POSTGRES_PASSWORD: password
        volumes:
          - postgresql_data:/var/lib/postgresql
        ports:
          - 5432:5432
        healthcheck:
          test: ["CMD-SHELL", "pg_isready -d myapp_development -U postgres"]
          interval: 10s
          timeout: 5s
          retries: 5
      web:
        build:
          context: .
          dockerfile: Dockerfile.dev
        command: bash -c "bundle install && bundle exec rails db:prepare && rm -f tmp/pids/server.pid && ./bin/dev"
        tty: true
        stdin_open: true
        volumes:
          - .:/myapp
          - bundle_data:/usr/local/bundle:cached
          - node_modules:/myapp/node_modules
        environment:
          TZ: Asia/Tokyo
        ports:
          - "3000:3000"
        depends_on:
          db:
            condition: service_healthy
      selenium_chrome:
        image: selenium/standalone-chrome:latest
        ports:
          - "4444:4444"
    volumes:
      bundle_data:
      postgresql_data:
      node_modules:
      ```
    
- rails_helper.rb
  Rspecをセットアップするための設定ファイル。システムテストでは、このファイルでCapybaraとSeleniumを設定。
  - Capybara.javascript_driverを :selenium_remote_chromeに設定することで、JavaScriptテストにSeleniumを使用し、リモートで実行するように設定。
  - selenium_remote_chromeは、Dockerコンテナ内で起動しているChromeインスタンス（selenium_chrome）を利用する。
  - RSpec.configure内
    - driven_by :selenium_chromeで、RSpecのシステムテストにSeleniumのChromeドライバを使用することを指定。
    - Capybara.server_hostやCapybara.server_portは、Capybaraがサーバーにアクセスする際のホストとポートを設定する。ここでは、テストが実行されるコンテナのホストを指定。
    ```diff_ruby
    # rails_helper.rb
    
    # This file is copied to spec/ when you run 'rails generate rspec:install'
    require 'spec_helper'
    ENV['RAILS_ENV'] ||= 'test'
    require_relative '../config/environment'
    # Prevent database truncation if the environment is production
    abort("The Rails environment is running in production mode!") if Rails.env.production?
    # Uncomment the line below in case you have `--require rails_helper` in the `.rspec` file
    # that will avoid rails generators crashing because migrations haven't been run yet
    # return unless Rails.env.test?
    require 'rspec/rails'
    require 'selenium-webdriver'
    require 'capybara/rspec'
    
    Capybara.javascript_driver = :selenium_remote_chrome
    
    # Add additional requires below this line. Rails is not loaded until this point!
    
    # Requires supporting ruby files with custom matchers and macros, etc, in
    # spec/support/ and its subdirectories. Files matching `spec/**/*_spec.rb` are
    # run as spec files by default. This means that files in spec/support that end
    # in _spec.rb will both be required and run as specs, causing the specs to be
    # run twice. It is recommended that you do not name files matching this glob to
    # end with _spec.rb. You can configure this pattern with the --pattern
    # option on the command line or in ~/.rspec, .rspec or `.rspec-local`.
    #
    # The following line is provided for convenience purposes. It has the downside
    # of increasing the boot-up time by auto-requiring all files in the support
    # directory. Alternatively, in the individual `*_spec.rb` files, manually
    # require only the support files necessary.
    #
    Dir[Rails.root.join('spec', 'support', '**', '*.rb')].sort.each { |f| require f }
    
    # Checks for pending migrations and applies them before tests are run.
    # If you are not using ActiveRecord, you can remove these lines.
    begin
      ActiveRecord::Migration.maintain_test_schema!
    rescue ActiveRecord::PendingMigrationError => e
      abort e.to_s.strip
    end
    RSpec.configure do |config|    
      # Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
      config.fixture_paths = [
        Rails.root.join('spec/fixtures')
      ]
    
      # If you're not using ActiveRecord, or you'd prefer not to run each of your
      # examples within a transaction, remove the following line or assign false
      # instead of true.
      config.use_transactional_fixtures = true
    
      # You can uncomment this line to turn off ActiveRecord support entirely.
      # config.use_active_record = false
    
      # RSpec Rails uses metadata to mix in different behaviours to your tests,
      # for example enabling you to call `get` and `post` in request specs. e.g.:
      #
      #     RSpec.describe UsersController, type: :request do
      #       # ...
      #     end
      #
      # The different available types are documented in the features, such as in
      # https://rspec.info/features/7-0/rspec-rails
      #
      # You can also this infer these behaviours automatically by location, e.g.
      # /spec/models would pull in the same behaviour as `type: :model` but this
      # behaviour is considered legacy and will be removed in a future version.
      #
      # To enable this behaviour uncomment the line below.
      # config.infer_spec_type_from_file_location!
    
      # Filter lines from Rails gems in backtraces.
      config.filter_rails_from_backtrace!
      # arbitrary gems may also be filtered via:
      # config.filter_gems_from_backtrace("gem name")
    
      config.include FactoryBot::Syntax::Methods
      config.before(:each, type: :system) do
        driven_by :selenium_chrome
        Capybara.server_host = IPSocket.getaddress(Socket.gethostname)
        Capybara.server_port = 4444
        Capybara.app_host = "http://#{Capybara.server_host}:#{Capybara.server_port}"
        Capybara.ignore_hidden_elements = false
      end
    end
    ```
- capybara.rb
  Capybaraの設定を行うファイル。ここでSeleniumのリモートWebDriverを設定し、Chromeのヘッドレスモードでテストを実行する設定にしている。
    - driven_by :selenium によって、Seleniumを使ったシステムテストを実行。
    - url: 'http://selenium_chrome:4444/wd/hub' で、selenium_chrome コンテナと通信するためのURLを指定。
    - headless_chrome を指定することで、ブラウザのUIを表示せずにテストを実行する。
        ```diff_ruby
        # capybara.rb
        
        RSpec.configure do |config|
          Webdrivers::Chromedriver.required_version =  '114.0.5735.90'
          config.before(:each, type: :system) do
            driven_by :selenium, using: :headless_chrome, options: {
            browser: :remote,
            url: 'http://selenium_chrome:4444/wd/hub'
            }
            Capybara.server_host = 'web'
          end
        end
        ```

### 取り組んだこと
- `selenium/standalone-chrome`のイメージの確認
  `selenium/standalone-chrome`は通常、ChromeブラウザとChromeDriverを含むが、バージョンや依存関係によってエラーが発生することがある。正しく起動しているか以下のコマンドを実行し確認。
    <details><summary>docker logs graduate_app-selenium_chrome-1</summary>
    
    ```
    $ docker logs graduate_app-selenium_chrome-1
    2024-12-01 09:13:14,331 INFO Included extra file "/etc/supervisor/conf.d/chrome-cleanup.conf" during parsing
    2024-12-01 09:13:14,331 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
    2024-12-01 09:13:14,339 INFO RPC interface 'supervisor' initialized
    2024-12-01 09:13:14,340 INFO supervisord started with pid 8
    2024-12-01 09:13:15,346 INFO spawned: 'xvfb' with pid 9
    2024-12-01 09:13:15,347 INFO spawned: 'vnc' with pid 10
    2024-12-01 09:13:15,349 INFO spawned: 'novnc' with pid 11
    2024-12-01 09:13:15,350 INFO spawned: 'selenium-standalone' with pid 12
    2024-12-01 09:13:15,359 INFO success: xvfb entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
    2024-12-01 09:13:15,359 INFO success: vnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
    2024-12-01 09:13:15,359 INFO success: novnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
    2024-12-01 09:13:15,359 INFO success: selenium-standalone entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
    Appending Selenium option: --heartbeat-period 30
    Appending Selenium option: --log-level INFO
    Appending Selenium option: --http-logs false
    Appending Selenium option: --structured-logs false
    Appending Selenium option: --reject-unsupported-caps true
    Setting up SE_NODE_GRID_URL...
    Selenium Grid Standalone configuration:
    [network]
    relax-checks = true
    
    [node]
    session-timeout = "300"
    override-max-sessions = false
    detect-drivers = false
    drain-after-session-count = 0
    max-sessions = 1
    
    [[node.driver-configuration]]
    display-name = "chrome"
    stereotype = '{"browserName": "chrome", "browserVersion": "131.0", "platformName": "Linux", "goog:chromeOptions": {"binary": "/usr/bin/google-chrome"}, "se:containerName": ""}'
    max-sessions = 1
    
    Starting Selenium Grid Standalone...
    Tracing is enabled
    Classpath will be enriched with these external jars :  --ext /external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-exporter-otlp/1.44.1/opentelemetry-exporter-otlp-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-netty/1.68.1/grpc-netty-1.68.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-codec-http/4.1.115.Final/netty-codec-http-4.1.115.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-trace/1.44.1/opentelemetry-sdk-trace-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-metrics/1.44.1/opentelemetry-sdk-metrics-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-logs/1.44.1/opentelemetry-sdk-logs-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-exporter-otlp-common/1.44.1/opentelemetry-exporter-otlp-common-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-exporter-sender-okhttp/1.44.1/opentelemetry-exporter-sender-okhttp-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-extension-autoconfigure-spi/1.44.1/opentelemetry-sdk-extension-autoconfigure-spi-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-api/1.68.1/grpc-api-1.68.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-codec-http2/4.1.110.Final/netty-codec-http2-4.1.110.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-core/1.68.1/grpc-core-1.68.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-handler-proxy/4.1.110.Final/netty-handler-proxy-4.1.110.Final.jar:/external_jars/https/repo1.maven.org/maven2/com/google/guava/guava/33.2.1-android/guava-33.2.1-android.jar:/external_jars/https/repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.28.0/error_prone_annotations-2.28.0.jar:/external_jars/https/repo1.maven.org/maven2/io/perfmark/perfmark-api/0.27.0/perfmark-api-0.27.0.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-transport-native-unix-common/4.1.115.Final/netty-transport-native-unix-common-4.1.115.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-util/1.68.1/grpc-util-1.68.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-common/4.1.115.Final/netty-common-4.1.115.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-buffer/4.1.115.Final/netty-buffer-4.1.115.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-transport/4.1.115.Final/netty-transport-4.1.115.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-codec/4.1.115.Final/netty-codec-4.1.115.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-handler/4.1.115.Final/netty-handler-4.1.115.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-api/1.44.1/opentelemetry-api-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-common/1.44.1/opentelemetry-sdk-common-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-api-incubator/1.44.1-alpha/opentelemetry-api-incubator-1.44.1-alpha.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-exporter-common/1.44.1/opentelemetry-exporter-common-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/com/squareup/okhttp3/okhttp/4.12.0/okhttp-4.12.0.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk/1.44.1/opentelemetry-sdk-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar:/external_jars/https/repo1.maven.org/maven2/com/google/code/gson/gson/2.11.0/gson-2.11.0.jar:/external_jars/https/repo1.maven.org/maven2/com/google/android/annotations/4.1.1.4/annotations-4.1.1.4.jar:/external_jars/https/repo1.maven.org/maven2/org/codehaus/mojo/animal-sniffer-annotations/1.24/animal-sniffer-annotations-1.24.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-context/1.68.1/grpc-context-1.68.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-codec-socks/4.1.110.Final/netty-codec-socks-4.1.110.Final.jar:/external_jars/https/repo1.maven.org/maven2/com/google/guava/failureaccess/1.0.2/failureaccess-1.0.2.jar:/external_jars/https/repo1.maven.org/maven2/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/external_jars/https/repo1.maven.org/maven2/org/checkerframework/checker-qual/3.42.0/checker-qual-3.42.0.jar:/external_jars/https/repo1.maven.org/maven2/com/google/j2objc/j2objc-annotations/3.0.0/j2objc-annotations-3.0.0.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-resolver/4.1.115.Final/netty-resolver-4.1.115.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-context/1.44.1/opentelemetry-context-1.44.1.jar:/external_jars/https/repo1.maven.org/maven2/com/squareup/okio/okio/3.6.0/okio-3.6.0.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-jdk8/1.9.10/kotlin-stdlib-jdk8-1.9.10.jar:/external_jars/https/repo1.maven.org/maven2/com/squareup/okio/okio-jvm/3.6.0/okio-jvm-3.6.0.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib/1.9.10/kotlin-stdlib-1.9.10.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-jdk7/1.9.10/kotlin-stdlib-jdk7-1.9.10.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-common/1.9.10/kotlin-stdlib-common-1.9.10.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/annotations/13.0/annotations-13.0.jar
    List arguments for OpenTelemetry:  -Dotel.resource.attributes=service.name=selenium-standalone -Dotel.traces.exporter=otlp -Dotel.java.global-autoconfigure.enabled=true
    09:13:16.702 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
    09:13:16.717 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
    09:13:17.880 INFO [NodeOptions.getSessionFactories] - Detected 2 available processors
    09:13:17.958 INFO [NodeOptions.report] - Adding chrome for {"browserName": "chrome","browserVersion": "131.0","goog:chromeOptions": {"binary": "\u002fusr\u002fbin\u002fgoogle-chrome"},"platformName": "linux","se:containerName": "","se:noVncPort": 7900,"se:vncEnabled": true} 1 times
    09:13:17.997 INFO [Node.<init>] - Binding additional locator mechanisms: relative
    09:13:18.034 INFO [GridModel.setAvailability] - Switching Node 1ab27371-6330-4f2e-afcb-07e6933ef611 (uri: http://192.168.32.2:4444) from DOWN to UP
    09:13:18.036 INFO [LocalDistributor.add] - Added node 1ab27371-6330-4f2e-afcb-07e6933ef611 at http://192.168.32.2:4444. Health check every 120s
    09:13:18.324 INFO [Standalone.execute] - Started Selenium Standalone 4.27.0 (revision d6e718d): http://192.168.32.2:4444
    ```
    </details>
    
    - `09:13:18.324 INFO [Standalone.execute] - Started Selenium Standalone 4.27.0 (revision d6e718d): http://192.168.32.2:4444`
      使用しているselenium/standalone-chromeイメージのバージョンは4.27.0
    - `2024-12-01 09:13:15,359 INFO success: selenium-standalone entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)`
      selenium-standaloneが正常に「RUNNING」状態になったことが記録されていることから、selenium/standaloneのプロセスが正常に開始されていることを確認
     - `09:13:17.958 INFO [NodeOptions.report] - Adding chrome for {"browserName": "chrome","browserVersion": "131.0","goog:chromeOptions": {"binary": "/usr/bin/google-chrome"},"platformName": "linux","se:containerName": "","se:noVncPort": 7900,"se:vncEnabled": true} 1 times`
       使用されているブラウザがChromeでバージョンは131.0
       google-chromeバイナリの場所やVNCが有効になっていることを確認
    - `2024-12-01 09:13:15,347 INFO spawned: 'vnc' with pid 10`
      `2024-12-01 09:13:15,359 INFO success: vnc entered RUNNING state`
      VNCサーバーが正常に起動していることを確認
      
  以上から`selenium/standalone-chromeは正しく起動していると考えられる。
- `--no-sandbox`オプションの追加
  `--no-sandbox`オプションはChromeが特定セキュリティ機能を無効化し、コンテナ内で正常に起動できるようになる。コンテナ内でChromeを動作させる際に必要な場合があり、今回は`--no-sandbox`と`--disable-gpu`のオプションを追加。
    ```diff_ruby
    # rails_helper.rb
    
    ...略...
    
    RSpec.configure do |config|
    + Webdrivers::Chromedriver.required_version = '114.0.5735.90'
    + config.before(:each, type: :system) do
    +   driven_by :selenium, using: :headless_chrome, options: {
    +     browser: :remote,
    +     url: 'http://selenium_chrome:4444/wd/hub',
    +     desired_capabilities: Selenium::WebDriver::Remote::Capabilities.chrome(
    +       'chromeOptions' => { 'args' => ['--headless', '--no-sandbox', '--disable-gpu'] }
    +     )
    +   }
    +   Capybara.server_host = 'web'
    + end
    end
    
    ...略...
    
    ```
  この状態で再度`bundle exec rspec`を実行。最初と異なるエラーが出現。
    ```
    Failure/Error:
           capabilities: Selenium::WebDriver::Remote::Capabilities.chrome(
             'chromeOptions' => { 'args' => ['--headless', '--no-sandbox', '--disable-gpu'] }
           ),
    
         NoMethodError:
           undefined method `chrome' for Selenium::WebDriver::Remote::Capabilities:Class
    ```
    メソッドが存在しないとエラーが出ている。Selenium WebdriverのバージョンによるAPIの変更で、最新バージョンでは`Selenium::WebDriver::Remote::Capabilities.chrome`の代わりに`Selenium::WebDriver::Options`を使ってオプション設定する方法が推奨されている。これより、`Selenium::WebDriver::Options`を使用して修正。
また、rails_helper.rbでCapybaraの設定が重複している部分を最適化。

    ```diff_ruby
    # rails_helper.rb
    
    ...略...
    
    RSpec.configure do |config|
      Webdrivers::Chromedriver.required_version = '114.0.5735.90'
    
      config.before(:each, type: :system) do
        driven_by :selenium, using: :headless_chrome, options: {
          browser: :remote,
          url: 'http://selenium_chrome:4444/wd/hub',
    -     capabilities: Selenium::WebDriver::Remote::Capabilities.chrome(
    -       'chromeOptions' => { 'args' => ['--headless', '--no-sandbox', '--disable-gpu'] }
    -     ),
    +     options: Selenium::WebDriver::Chrome::Options.new.tap { |opt|
    +       opt.add_argument('--headless')
    +       opt.add_argument('--no-sandbox')
    +       opt.add_argument('--disable-gpu')
    +     }
        }
    +   Capybara.server_host = IPSocket.getaddress(Socket.gethostname)
    +   Capybara.server_port = 4444
    +   Capybara.app_host = "http://#{Capybara.server_host}:#{Capybara.server_port}"
    +   Capybara.ignore_hidden_elements = false
      end
      
      ...略...
      
      config.include FactoryBot::Syntax::Methods
    - config.before(:each, type: :system) do
    -   driven_by :selenium_chrome
    -   Capybara.server_host = IPSocket.getaddress(Socket.gethostname)
    -   Capybara.server_port = 4444
    -   Capybara.app_host = "http://#{Capybara.server_host}:#{Capybara.server_port}"
    -   Capybara.ignore_hidden_elements = false
    - end
    end
    ```

### 感想
最新のSelenium WebDriverのバージョンでは、Chromeオプションを設定するために`Selenium::WebDriver::Remote::Capabilities.chrome`が使えないことが分かった。
今後、Chromeの起動オプションを指定する場合は`Selenium::WebDriver::Chrome::Options`を使用するといいと学んだ。
システムテストでこんなに躓くと思わず、心折れかけましたが、いろんな方の助けでここまで進めました！！
たくさんサイトを参考にさせていただきましたが、根本的な解決に至らなく、、、
最終的にchatGPTにお世話になり、なんとかできたっぽいです。
このままシステムテスト作成でうまくいきますように:pray: