Repository: ruby/rubygems (Ruby)

Issue:
Running gem generate_index --update does not clean up temporary directories
<!--
Before reporting an issue, make sure you're using the latest version of rubygems
by running `gem update --system`, and double check whether the issue has already
been fixed in the latest version.
-->

I'm having a problem or would like to suggest a feature.

My current problem is that running gem generate_index --update does not clean up temporary directories after itself.

This issue is related to:

  - [ ] Network problems
  - [ ] Installing a library
  - [ ] Publishing a library
  - [x] The command line `gem`
  - [ ] Other

Here are my current environment details:

```
$ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 3.3.7
  - RUBY VERSION: 3.1.2 (2022-04-12 patchlevel 20) [x86_64-linux-musl]
  - INSTALLATION DIRECTORY: /usr/lib/ruby/gems/3.1.0
  - USER INSTALLATION DIRECTORY: XXXXXXXXXXXXXXXXXXXX
  - RUBY EXECUTABLE: /usr/bin/ruby
  - GIT EXECUTABLE: /usr/bin/git
  - EXECUTABLE DIRECTORY: /usr/bin
  - SPEC CACHE DIRECTORY: XXXXXXXXXXXXXXXXXXXXXXXXXX
  - SYSTEM CONFIGURATION DIRECTORY: /etc
  - RUBYGEMS PLATFORMS:
     - ruby
     - x86_64-linux
  - GEM PATHS:
     - /usr/lib/ruby/gems/3.1.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /bin
     - /usr/bin
     - /usr/local/sbin
     - /usr/local/bin
     - /usr/sbin
     - /sbin
```

I will abide by the [code of conduct](https://github.com/rubygems/rubygems/blob/master/CODE_OF_CONDUCT.md).



Discussion:
In order to expedite a fix, could you show a series of commands that can be run in order to see the issue, and point out the list of unexpected files left around? Thanks!
Definitely :) First issue I'm reporting here, so I've tried to stick to the issue template as best as I could.

I hope this helps to illustrate the problem:

```
+$ mkdir /tmp/y
+$ cd /tmp/y
+$
+$ gem generate_index
Generating Marshal quick index gemspecs for 0 gems

Complete
Generated Marshal quick index gemspecs: 0.000s
Generating specs index
Generated specs index: 0.000s
Generating latest specs index
Generated latest specs index: 0.000s
Generating prerelease specs index
Generated prerelease specs index: 0.000s
Compressing indices
Compressed indices: 0.000s
+$ ls /tmp | grep gem_generate_index
+$
+$ gem generate_index --update
No new gems
+$ ls /tmp | grep gem_generate_index
gem_generate_index20220622-17821-yuue69
```
Thank you!
@graywolf-at-work Interestingly, I can't reproduce the issue running the commands you posted on my system. Can you provide a `Dockerfile` that reproduces this independently? Also, can you try upgrading RubyGems just in case? I don't recall any fixes related to this, but just in case.
Sure, no problem:

```
+   $ cat Containerfile
from alpine:3.16

run apk upgrade -Ua
run apk add ruby ruby-rdoc

run gem update --system -N

run mkdir /tmp/x

run gem generate_index --directory /tmp/x
run ! test -e /tmp/gem_generate_index*

run gem generate_index --update --directory /tmp/x
run ! test -e /tmp/gem_generate_index*
```

```
+   $ podman build -t repro .
STEP 1/9: FROM alpine:3.16
STEP 2/9: run apk upgrade -Ua
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
(1/4) Upgrading alpine-baselayout-data (3.2.0-r20 -> 3.2.0-r22)
(2/4) Upgrading busybox (1.35.0-r13 -> 1.35.0-r14)
Executing busybox-1.35.0-r14.post-upgrade
(3/4) Upgrading alpine-baselayout (3.2.0-r20 -> 3.2.0-r22)
Executing alpine-baselayout-3.2.0-r22.pre-upgrade
Executing alpine-baselayout-3.2.0-r22.post-upgrade
(4/4) Upgrading ssl_client (1.35.0-r13 -> 1.35.0-r14)
Executing busybox-1.35.0-r14.trigger
OK: 6 MiB in 14 packages
--> 024fd7f4841
STEP 3/9: run apk add ruby ruby-rdoc
(1/11) Installing ca-certificates (20211220-r0)
(2/11) Installing gmp (6.2.1-r2)
(3/11) Installing libffi (3.4.2-r1)
(4/11) Installing ncurses-terminfo-base (6.3_p20220521-r0)
(5/11) Installing ncurses-libs (6.3_p20220521-r0)
(6/11) Installing readline (8.1.2-r0)
(7/11) Installing libucontext (1.2-r0)
(8/11) Installing yaml (0.2.5-r0)
(9/11) Installing ruby-libs (3.1.2-r0)
(10/11) Installing ruby (3.1.2-r0)
(11/11) Installing ruby-rdoc (3.1.2-r0)
Executing busybox-1.35.0-r14.trigger
Executing ca-certificates-20211220-r0.trigger
OK: 25 MiB in 25 packages
--> cd08bef9774
STEP 4/9: run gem update --system -N
Updating rubygems-update
Successfully installed rubygems-update-3.3.16
Installing RubyGems 3.3.16
  Successfully built RubyGem
  Name: bundler
  Version: 2.3.16
  File: bundler-2.3.16.gem
Bundler 2.3.16 installed
RubyGems 3.3.16 installed
Regenerating binstubs
Regenerating plugins

# 3.3.16 / 2022-06-15

## Enhancements:

* Auto-fix and warn gem packages including a gemspec with `require_paths`
  as an array of arrays. Pull request #5615 by deivid-rodriguez
* Misc cargo builder improvements. Pull request #5459 by ianks
* Installs bundler 2.3.16 as a default gem.

## Bug fixes:

* Fix incorrect password redaction when there's an error in `gem source
  -a`. Pull request #5623 by deivid-rodriguez
* Fix another regression when loading old marshaled specs. Pull request
  #5610 by deivid-rodriguez

# 3.3.15 / 2022-06-01

## Enhancements:

* Support the change of did_you_mean about `Exception#detailed_message`.
  Pull request #5560 by mame
* Installs bundler 2.3.15 as a default gem.

## Bug fixes:

* Fix loading old marshaled specs including `YAML::PrivateType` constant.
  Pull request #5415 by deivid-rodriguez
* Fix rubygems update when non default `--install-dir` is configured. Pull
  request #5566 by deivid-rodriguez

# 3.3.14 / 2022-05-18

## Enhancements:

* Installs bundler 2.3.14 as a default gem.

# 3.3.13 / 2022-05-04

## Enhancements:

* Installs bundler 2.3.13 as a default gem.

## Bug fixes:

* Fix regression when resolving ruby constraints. Pull request #5486 by
  deivid-rodriguez

## Documentation:

* Clarify description of owner-flags. Pull request #5497 by kronn

# 3.3.12 / 2022-04-20

## Enhancements:

* Less error swallowing when installing gems. Pull request #5475 by
  deivid-rodriguez
* Stop considering `RUBY_PATCHLEVEL` for resolution. Pull request #5472 by
  deivid-rodriguez
* Bump vendored optparse to latest master. Pull request #5466 by
  deivid-rodriguez
* Installs bundler 2.3.12 as a default gem.

## Documentation:

* Fix formatting in docs. Pull request #5470 by peterzhu2118
* Fix a typo. Pull request #5401 by znz

# 3.3.11 / 2022-04-07

## Enhancements:

* Enable mfa on specific keys during gem signin. Pull request #5305 by
  aellispierce
* Prefer `__dir__` to `__FILE__`. Pull request #5444 by deivid-rodriguez
* Add cargo builder for rust extensions. Pull request #5175 by ianks
* Installs bundler 2.3.11 as a default gem.

## Documentation:

* Improve RDoc setup. Pull request #5398 by deivid-rodriguez

# 3.3.10 / 2022-03-23

## Enhancements:

* Installs bundler 2.3.10 as a default gem.

## Documentation:

* Enable `Gem::Package` example in RDoc documentation. Pull request #5399
  by nobu
* Unhide RDoc documentation from top level `Gem` module. Pull request
  #5396 by nobu

# 3.3.9 / 2022-03-09

## Enhancements:

* Installs bundler 2.3.9 as a default gem.

# 3.3.8 / 2022-02-23

## Enhancements:

* Installs bundler 2.3.8 as a default gem.


------------------------------------------------------------------------------

RubyGems installed the following executables:
        /usr/bin/gem
        /usr/bin/bundle
        /usr/bin/bundler

RubyGems system software updated
--> 2582edcb164
STEP 5/9: run mkdir /tmp/x
--> 5be746490f4
STEP 6/9: run gem generate_index --directory /tmp/x
Generating Marshal quick index gemspecs for 0 gems

Complete
Generated Marshal quick index gemspecs: 0.000s
Generating specs index
Generated specs index: 0.000s
Generating latest specs index
Generated latest specs index: 0.000s
Generating prerelease specs index
Generated prerelease specs index: 0.000s
Compressing indices
Compressed indices: 0.000s
--> b3df13a0d76
STEP 7/9: run ! test -e /tmp/gem_generate_index*
--> ab0911e7782
STEP 8/9: run gem generate_index --update --directory /tmp/x
No new gems
--> 6005791846b
STEP 9/9: run ! test -e /tmp/gem_generate_index*
Error: error building at STEP "RUN ! test -e /tmp/gem_generate_index*": error while running runtime: exit status 1
```

I think it is caused by https://github.com/rubygems/rubygems/blob/5b910cc4a78827c0571252dd93776b559cef608a/lib/rubygems/indexer.rb#L348 not having this ensure https://github.com/rubygems/rubygems/blob/5b910cc4a78827c0571252dd93776b559cef608a/lib/rubygems/indexer.rb#L275 block. However I'm not familiar enough with the code base to be sure of it.
Thanks @graywolf-at-work!

I couldn't repro the issue because my system has a different default temp folder that's not `/tmp`. Anyways, I think you're right. I think you can double check by manually adding the rescue clause to `update_index` and see if that fixes the issue. If yes, I'm happy to take a PR :)
Does rubygems require CLA in order to be able to make the PR?
You don't explicitly need to sign any CLA. By contributing you implicitly agree to our license, code of conduct, and so on. Is that ok for you?
Yeah that is fine, one less thing to run by our legal. Once I have it approved, I will send a PR.



Patch:
diff --git a/lib/rubygems/indexer.rb b/lib/rubygems/indexer.rb
index 6e8dade6407b..a5a86f41111c 100644
--- a/lib/rubygems/indexer.rb
+++ b/lib/rubygems/indexer.rb
@@ -401,6 +401,8 @@ def update_index
 
       File.utime newest_mtime, newest_mtime, dst_name
     end
+  ensure
+    FileUtils.rm_rf @directory
   end
 
   ##


Test patch:
diff --git a/test/rubygems/helper.rb b/test/rubygems/helper.rb
index 75eed7be6b12..7bd7dd0c3bef 100644
--- a/test/rubygems/helper.rb
+++ b/test/rubygems/helper.rb
@@ -119,6 +119,12 @@ def assert_directory_exists(path, msg = nil)
     assert File.directory?(path), msg
   end
 
+  def refute_directory_exists(path, msg = nil)
+    msg = build_message(msg, "Expected path '#{path}' not to be a directory")
+    assert_path_not_exist path
+    refute File.directory?(path), msg
+  end
+
   # https://github.com/seattlerb/minitest/blob/21d9e804b63c619f602f3f4ece6c71b48974707a/lib/minitest/assertions.rb#L188
   def _synchronize
     yield
diff --git a/test/rubygems/test_gem_indexer.rb b/test/rubygems/test_gem_indexer.rb
index 6653f29adf12..24afae5fcc7f 100644
--- a/test/rubygems/test_gem_indexer.rb
+++ b/test/rubygems/test_gem_indexer.rb
@@ -103,6 +103,8 @@ def test_generate_index
 
     assert_indexed @indexerdir, "latest_specs.#{@marshal_version}"
     assert_indexed @indexerdir, "latest_specs.#{@marshal_version}.gz"
+
+    refute_directory_exists @indexer.directory
   end
 
   def test_generate_index_modern
@@ -342,6 +344,8 @@ def test_update_index
 
       assert_includes pre_specs_index, @d2_1_a_tuple
       refute_includes pre_specs_index, @d2_1_tuple
+
+      refute_directory_exists @indexer.directory
     end
   end