Repository: pre-commit/pre-commit

Issue:
No such file or directory: '/github/home/.cache/pre-commit/repo4mrvfeou/rbenv-system/.install_state_v1staging'
### search you tried in the issue tracker

Found this one #1658

### describe your issue

Running pre-commit GitHub action in a [custom container](https://github.com/platform-engineering-org/helper/blob/main/Dockerfile).

Workflow is [broken](https://github.com/platform-engineering-org/bootstrap/actions/runs/4342905858/jobs/7584289627):
```
[INFO] Installing environment for https://github.com/markdownlint/markdownlint.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: FileNotFoundError: [Errno 2] No such file or directory: '/github/home/.cache/pre-commit/repo4mrvfeou/rbenv-system/.install_state_v1staging'
```



### pre-commit --version

pre-commit 2.20.0

### .pre-commit-config.yaml

```yaml
---
ci:
  skip: [terraform_fmt, terraform_validate, terragrunt_validate]

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      - id: end-of-file-fixer
      - id: trailing-whitespace
        args:
          - --markdown-linebreak-ext=md
      - id: check-docstring-first
      - id: requirements-txt-fixer
      - id: check-merge-conflict
      - id: no-commit-to-branch
        args:
          - "--branch"
          - "main"
      - id: check-symlinks
      - id: detect-private-key
      - id: detect-aws-credentials
        args:
          - --allow-missing-credentials
      - id: check-json
  - repo: https://github.com/markdownlint/markdownlint
    rev: v0.12.0
    hooks:
      - id: markdownlint
        additional_dependencies: [rake]
  - repo: https://github.com/maxbrunet/pre-commit-renovate
    rev: 34.157.1
    hooks:
      - id: renovate-config-validator
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.77.1
    hooks:
      - id: terraform_providers_lock
        args:
          - --tf-init-args=-upgrade
      - id: terraform_fmt
      - id: terraform_validate
      - id: terragrunt_validate
```


### ~/.cache/pre-commit/pre-commit.log (if present)

_No response_


Discussion:
you found an exact duplicate, why did you make an issue?
Because I didn't understand how to fix my problem with the information provided on the other closed issue
you've not shown what you've tried so I don't know how to help you.  right now it looks like you've tried nothing
I've tried to check rbenv is installed in the container. It is not. 
I've checked the ruby version. It is 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux].
What else can I try?
clearly that container is broken -- why do you need that container?  also looks like your CI is running both pre-commit.ci and github actions pre-commit -- which is kind of wasteful
> clearly that container is broken -- why do you need that container? also looks like your CI is running both pre-commit.ci and github actions pre-commit -- which is kind of wasteful

Agree. It is broken but I don't have enough ruby expriance to figure out what is broken and fix it. The container is built by me and it used for baking all my team tooling in a container image instead of installing/upgrading them on our laptops.
if your company wants paid support I'm happy to provide it -- but this seems like you've already got several solutions which don't involve that container
I am pocking the pre-commit code. It seems like the language ruby ENVIRONMENT_DIR is set to `rbenv` while the container image doesn't have it. The error is persistent even if `rbenv` is installed.
In repository.py line 108,  it tries to write a file in a directory which doesn't exist. Could you provide me a hint where it is created?
@asottile I'm seeing the same issue described here, and do not have `rbenv` installed so #1658 doesn't give a solution.

I've looked in `~/.cache/pre-commit/pre-commit.log` but I don't see any useful information.

Looking at the code:

https://github.com/pre-commit/pre-commit/blob/8d84a7a2702b074a8b46f5e38af28bd576291251/pre_commit/languages/ruby.py#L114
This seems to be responsible for creating the environment directory. If the version is `system`, we don't run `rbenv init -`. So are we expecting `gem build ...` to create the `rbenv-system` directory?

If the output of `gem build ...` contains some useful information on what's going wrong, it's being dropped:

https://github.com/pre-commit/pre-commit/blob/8d84a7a2702b074a8b46f5e38af28bd576291251/pre_commit/util.py#L89
This collects the stdout & stderr of the called process and returns them.

https://github.com/pre-commit/pre-commit/blob/8d84a7a2702b074a8b46f5e38af28bd576291251/pre_commit/lang_base.py#L91
This calls `cmd_output_b() but ignores the return values. Perhaps more verbose output is needed in the case of an error.

## Contents of `~/.cache/pre-commit/pre-commit.log`

### version information

```
pre-commit version: 3.2.2
git --version: git version 2.39.2
sys.version:
    3.10.10 (main, Mar 26 2023, 19:47:56) [GCC 12.2.1 20230121]
sys.executable: /home/pbarker/Projects/pbarker.dev/venv/bin/python
os.name: posix
sys.platform: linux
```

### error information

```
An unexpected error has occurred: FileNotFoundError: [Errno 2] No such file or directory: '/home/pbarker/.cache/pre-commit/repoiuzm_cni/rbenv-system/.install_state_v1staging'
```

```
Traceback (most recent call last):
  File "/home/pbarker/Projects/pbarker.dev/venv/lib/python3.10/site-packages/pre_commit/error_handler.py", line 73, in error_handler
    yield
  File "/home/pbarker/Projects/pbarker.dev/venv/lib/python3.10/site-packages/pre_commit/main.py", line 409, in main
    return run(args.config, store, args)
  File "/home/pbarker/Projects/pbarker.dev/venv/lib/python3.10/site-packages/pre_commit/commands/run.py", line 442, in run
    install_hook_envs(to_install, store)
  File "/home/pbarker/Projects/pbarker.dev/venv/lib/python3.10/site-packages/pre_commit/repository.py", line 248, in install_hook_envs
    _hook_install(hook)
  File "/home/pbarker/Projects/pbarker.dev/venv/lib/python3.10/site-packages/pre_commit/repository.py", line 111, in _hook_install
    with open(staging, 'w') as state_file:
FileNotFoundError: [Errno 2] No such file or directory: '/home/pbarker/.cache/pre-commit/repoiuzm_cni/rbenv-system/.install_state_v1staging'
```
I'm getting the same error on Windows, ruby 3.2.2, no rbenv.
[pre-commit.log](https://github.com/pre-commit/pre-commit/files/11664668/pre-commit.log)

Seeing same thing as https://github.com/pre-commit/pre-commit/issues/2799#issuecomment-1510465720

Occurs during `pre-commit install-hooks` with this hook:

```yaml
repos:
  - repo: https://github.com/rubocop/rubocop
    rev: v1.52.0
    hooks:
      - id: rubocop

```

### version information

```
pre-commit version: 3.3.2
git --version: git version 2.39.3
sys.version:
    3.11.2 (main, Mar  9 2023, 21:49:05) [GCC 11.3.1 20220421 (Red Hat 11.3.1-2)]
sys.executable: .../bin/python3.11
os.name: posix
sys.platform: linux
```

### error information

```
An unexpected error has occurred: FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/pre-commit/repo238a8i_c/rbenv-system/.install_state_v1staging'
```

```
Traceback (most recent call last):
  File ".../lib/python3.11/site-packages/pre_commit/error_handler.py", line 73, in error_handler
    yield
  File ".../lib/python3.11/site-packages/pre_commit/main.py", line 410, in main
    return install_hooks(args.config, store)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pre_commit/commands/install_uninstall.py", line 146, in install_hooks
    install_hook_envs(all_hooks(load_config(config_file), store), store)
  File ".../lib/python3.11/site-packages/pre_commit/repository.py", line 248, in install_hook_envs
    _hook_install(hook)
  File ".../lib/python3.11/site-packages/pre_commit/repository.py", line 111, in _hook_install
    with open(staging, 'w') as state_file:
         ^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/pre-commit/repo238a8i_c/rbenv-system/.install_state_v1staging'
```

(I've partially elided the Python paths above.)

There directory `/root/.cache/pre-commit/repo238a8i_c/rbenv-system` does NOT exist. I'm not sure what's supposed to create it.

Ruby is installed via `dnf -y -q install ruby ruby-devel`:

```
Installed:
  annobin-11.05-1.el9.x86_64
  dwz-0.14-3.el9.x86_64
  efi-srpm-macros-6-2.el9_0.0.1.noarch
  file-5.39-12.el9.x86_64
  fonts-srpm-macros-1:2.0.5-7.el9.1.noarch
  gcc-plugin-annobin-11.3.1-4.3.el9.alma.x86_64
  ghc-srpm-macros-1.5.0-6.el9.noarch
  glibc-gconv-extra-2.34-60.el9.x86_64
  go-srpm-macros-3.2.0-1.el9.noarch
  kernel-srpm-macros-1.0-12.el9.noarch
  lua-srpm-macros-1-6.el9.noarch
  ocaml-srpm-macros-6-6.el9.noarch
  openblas-srpm-macros-2-11.el9.noarch
  perl-srpm-macros-1-41.el9.noarch
  pyproject-srpm-macros-1.6.2-1.el9.noarch
  python-srpm-macros-3.9-52.el9.noarch
  qt5-srpm-macros-5.15.3-1.el9.noarch
  redhat-rpm-config-199-1.el9.alma.noarch
  ruby-3.0.4-160.el9_0.x86_64
  ruby-default-gems-3.0.4-160.el9_0.noarch
  ruby-devel-3.0.4-160.el9_0.x86_64
  ruby-libs-3.0.4-160.el9_0.x86_64
  rubygem-bigdecimal-3.0.0-160.el9_0.x86_64
  rubygem-bundler-2.2.33-160.el9_0.noarch
  rubygem-io-console-0.5.7-160.el9_0.x86_64
  rubygem-json-2.5.1-160.el9_0.x86_64
  rubygem-psych-3.3.2-160.el9_0.x86_64
  rubygem-rdoc-6.3.3-160.el9_0.noarch
  rubygems-3.2.33-160.el9_0.noarch
  rust-srpm-macros-17-4.el9.noarch
  zip-3.0-35.el9.x86_64
```

By editing the source code to get the gem output I'm seeing this:

```
[root@e736e09e3d9d foo]# pre-commit install-hooks
cmd=('git', 'rev-parse', '--show-cdup')
cmd=('git', 'rev-parse', '--is-inside-git-dir')
[INFO] Installing environment for https://github.com/rubocop/rubocop.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
cmd=('gem', 'build', 'rubocop.gemspec')
stdout:
  Successfully built RubyGem
  Name: rubocop
  Version: 1.52.0
  File: rubocop-1.52.0.gem

stderr:
WARNING:  open-ended dependency on parser (>= 3.2.0.0) is not recommended
  if parser is semantically versioned, use:
    add_runtime_dependency 'parser', '~> 3.2', '>= 3.2.0.0'
WARNING:  See https://guides.rubygems.org/specification-reference/ for help

cmd=('gem', 'install', '--no-document', '--no-format-executable', '--no-user-install', 'rubocop-1.52.0.gem')
stdout:
Successfully installed rubocop-1.52.0
1 gem installed

stderr:

An unexpected error has occurred: FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/pre-commit/repo98m3m8cs/rbenv-system/.install_state_v1staging'
cmd=('git', '--version')
Check the log at /root/.cache/pre-commit/pre-commit.log
```

Gem env:

```
RubyGems Environment:
  - RUBYGEMS VERSION: 3.2.33
  - RUBY VERSION: 3.0.4 (2022-04-12 patchlevel 208) [x86_64-linux]
  - INSTALLATION DIRECTORY: /usr/share/gems
  - USER INSTALLATION DIRECTORY: /root/.local/share/gem/ruby
  - RUBY EXECUTABLE: /usr/bin/ruby
  - GIT EXECUTABLE: /usr/bin/git
  - EXECUTABLE DIRECTORY: /usr/bin
  - SPEC CACHE DIRECTORY: /root/.local/share/gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /etc
  - RUBYGEMS PLATFORMS:
     - ruby
     - x86_64-linux
  - GEM PATHS:
     - /usr/share/gems
     - /root/.local/share/gem/ruby
     - /usr/local/share/gems
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
     - "gem" => "--install-dir=/usr/local/share/gems --bindir /usr/local/bin"
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /root/.local/bin
     - /root/bin
     - /opt/python/bin
     - /usr/local/sbin
     - /usr/local/bin
     - /usr/sbin
     - /usr/bin
     - /sbin
     - /bin
```
can you share your environment and gem and Ruby rc files?
@asottile I edited my comment above yours to add more information, including the stdout/stderr of the gem build commands and the output of `gem env`. 
your `--install-dir` option appears to be breaking gem -- any idea where that's coming from?  we use `GEM_HOME` to specify the target installation for gems
> your `--install-dir` option appears to be breaking gem -- any idea where that's coming from? we use `GEM_HOME` to specify the target installation for gems

Hmm, either Red Hat's packaging or possibly it's something with newer versions of gem?
In my experience, you have to set _both_ `GEM_HOME` and `GEM_PATH`.
`GEM_PATH` is for lookup, `GEM_HOME` is installation target -- only the latter is necessary for pre-commit since it intends to isolate from others (instead of cascading gem PATH lookup)
To be clear, the `gem env` output I shared is running it directly from the shell, not with `GEM_HOME` as set by pre-commit. 
yes of course -- the useful part from `gem env` appears to be this configuration override:

```
     - "gem" => "--install-dir=/usr/local/share/gems --bindir /usr/local/bin"
```

afaik that's coming from some config file -- are you sure this is an otherwise empty image?
Here's what `gem env` looks like when I run it from pre-commit:

```
cmd=('gem', 'env')
stdout:
RubyGems Environment:
  - RUBYGEMS VERSION: 3.2.33
  - RUBY VERSION: 3.0.4 (2022-04-12 patchlevel 208) [x86_64-linux]
  - INSTALLATION DIRECTORY: /root/.cache/pre-commit/repo98m3m8cs/rbenv-system/gems
  - USER INSTALLATION DIRECTORY: /root/.local/share/gem/ruby
  - RUBY EXECUTABLE: /usr/bin/ruby
  - GIT EXECUTABLE: /usr/bin/git
  - EXECUTABLE DIRECTORY: /root/.cache/pre-commit/repo98m3m8cs/rbenv-system/gems/bin
  - SPEC CACHE DIRECTORY: /root/.local/share/gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /etc
  - RUBYGEMS PLATFORMS:
     - ruby
     - x86_64-linux
  - GEM PATHS:
     - /root/.cache/pre-commit/repo98m3m8cs/rbenv-system/gems
     - /root/.local/share/gem/ruby
     - /usr/share/gems
     - /usr/local/share/gems
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
     - "gem" => "--install-dir=/usr/local/share/gems --bindir /usr/local/bin"
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /root/.cache/pre-commit/repo98m3m8cs/rbenv-system/gems/bin
     - /root/.local/bin
     - /root/bin
     - /opt/python/bin
     - /usr/local/sbin
     - /usr/local/bin
     - /usr/sbin
     - /usr/bin
     - /sbin
     - /bin
```

Is that not what we expect?

This image is based on Alma 9 and is customized for Python. Ruby is installed via `dnf`.
Hmm, you're right, rubocop is installing to `/usr/local/share/gems/gems/rubocop-1.52.`. I gather we want it to be installing to  `/root/.cache/pre-commit/repo98m3m8cs/rbenv-system/gems`.

Let me see if I can track that down.
This is apparently an issue with Alma Linux (aka Red Hat) 9.2. You can repro it with:

```
$ docker run -it almalinux:9.2 bash
# dnf -yq install git-core ruby-devel rubygems gcc make pip
# pip install pre-commit
# git init foo
# cd foo
# cat >.pre-commit-config.yaml<<EOF
repos:
  - repo: https://github.com/rubocop/rubocop
    rev: v1.52.0
    hooks:
      - id: rubocop
fail_fast: true
EOF
# pre-commit install-hooks
```

And I gather it's an issue with how Alma Linux packages up rubygems:

```
     - "gem" => "--install-dir=/usr/local/share/gems --bindir /usr/local/bin"
```
can you report it to them? unconditionally changing `--install-dir` seems wrong especially when the user is requesting a different one
I'm really not a fan of Ruby, but I don't think this is something RedHat is likely to fix. FWIW:

```
[root@f4dbfd9c2ee3 /]# irb
irb(main):001:0> Gem.operating_system_defaults
=> {"gem"=>"--install-dir=/usr/local/share/gems --bindir /usr/local/bin"}
```

That's where it comes from. RedHat's answer will probably be: "Pound sand. You shouldn't be using the system ruby. Install your own ruby."

RubyGems accommodated this issue:

https://github.com/rubygems/rubygems/pull/5566

Can pre-commit use `--install-dir` instead of `GEM_HOME`?
Work-around:

```
$ rm -f /usr/share/rubygems/rubygems/defaults/operating_system.rb
```

Blech. No more time for shaving yaks so that's the solution I'm going to live with.

This looks to be an intended use-case by RubyGems:

https://github.com/rubygems/rubygems/pull/2116

This means that pre-commit is not compatible with any packaged gem that uses this feature.

If you don't want to accommodate it I understand. Would you accept a PR that at least causes pre-commit to emit a more helpful error message?

And here's where upstream does their customization:

https://src.fedoraproject.org/rpms/ruby/blob/rawhide/f/ruby.spec#_736
https://src.fedoraproject.org/rpms/ruby/blob/rawhide/f/operating_system.rb
Note that Gentoo has similar customisation: https://github.com/gentoo/gentoo/blob/master/dev-ruby/rubygems/files/gentoo-defaults-5.rb
@jaysoffian did you want to try your idea from https://github.com/pre-commit/pre-commit/issues/2799#issuecomment-1581744060 ?
> @jaysoffian did you want to try your idea from [#2799 (comment)](https://github.com/pre-commit/pre-commit/issues/2799#issuecomment-1581744060) ?

@asottile this patch works for me under the environment described in https://github.com/pre-commit/pre-commit/issues/2799#issuecomment-1581700041

```diff
diff --git i/pre_commit/languages/ruby.py w/pre_commit/languages/ruby.py
index 76631f2532..13cc628d80 100644
--- i/pre_commit/languages/ruby.py
+++ w/pre_commit/languages/ruby.py
@@ -125,6 +125,7 @@ def install_environment(
             # Need to call this after installing to set up the shims
             lang_base.setup_cmd(prefix, ('rbenv', 'rehash'))
 
+    envdir = lang_base.environment_dir(prefix, ENVIRONMENT_DIR, version)
     with in_env(prefix, version):
         lang_base.setup_cmd(
             prefix, ('gem', 'build', *prefix.star('.gemspec')),
@@ -135,6 +136,8 @@ def install_environment(
                 'gem', 'install',
                 '--no-document', '--no-format-executable',
                 '--no-user-install',
+                '--install-dir', os.path.join(envdir, 'gems'),
+                '--bindir',  os.path.join(envdir, 'bin'),
                 *prefix.star('.gem'), *additional_dependencies,
             ),
         )
```

Would you like a PR? 
do you want to send a pr?

Resolving patch:
diff --git a/pre_commit/languages/ruby.py b/pre_commit/languages/ruby.py
--- a/pre_commit/languages/ruby.py
+++ b/pre_commit/languages/ruby.py
@@ -114,6 +114,8 @@ def _install_ruby(
 def install_environment(
         prefix: Prefix, version: str, additional_dependencies: Sequence[str],
 ) -> None:
+    envdir = lang_base.environment_dir(prefix, ENVIRONMENT_DIR, version)
+
     if version != 'system':  # pragma: win32 no cover
         _install_rbenv(prefix, version)
         with in_env(prefix, version):
@@ -135,6 +137,8 @@ def install_environment(
                 'gem', 'install',
                 '--no-document', '--no-format-executable',
                 '--no-user-install',
+                '--install-dir', os.path.join(envdir, 'gems'),
+                '--bindir', os.path.join(envdir, 'gems', 'bin'),
                 *prefix.star('.gem'), *additional_dependencies,
             ),
         )


Test patch: