Repository: gravitational/teleport
Pull request #52725: [teleport-update] Set umask 0022 for teleport-update to avoid errors on enable
Author: login: sclevine
Labels: name: size/sm
name: backport/branch/v17
Created: 2025-03-03T21:30:18Z | Merged: 2025-03-04T17:05:31Z

If the local shell has an unusual umask, teleport-update will refuse to complete `teleport-update enable`:

```
2025-03-03T20:24:36.890Z INFO [UPDATER]   Initiating installation. target_version:17.3.0-dev.hugoau.7+Enterprise active_version: agent/updater.go:362
2025-03-03T20:24:37.088Z INFO [UPDATER]   Downloading Teleport tarball. url:https://cdn.cloud.gravitational.io/teleport-ent-v17.3.0-dev.hugoau.7-linux-arm64-bin.tar.gz size:188521662 agent/installer.go:295
2025-03-03T20:24:37.876Z INFO [UPDATER]   Downloading file:teleport-ent-v17.3.0-dev.hugoau.7-linux-arm64-bin.tar.gz progress:20% agent/logger.go:49
2025-03-03T20:24:38.546Z INFO [UPDATER]   Downloading file:teleport-ent-v17.3.0-dev.hugoau.7-linux-arm64-bin.tar.gz progress:40% agent/logger.go:49
2025-03-03T20:24:39.051Z INFO [UPDATER]   Downloading file:teleport-ent-v17.3.0-dev.hugoau.7-linux-arm64-bin.tar.gz progress:60% agent/logger.go:49
2025-03-03T20:24:39.683Z INFO [UPDATER]   Downloading file:teleport-ent-v17.3.0-dev.hugoau.7-linux-arm64-bin.tar.gz progress:80% agent/logger.go:49
2025-03-03T20:24:40.215Z INFO [UPDATER]   Downloading file:teleport-ent-v17.3.0-dev.hugoau.7-linux-arm64-bin.tar.gz progress:100% agent/logger.go:49
2025-03-03T20:24:40.215Z INFO [UPDATER]   Download complete. duration:3.227484932s size:188521662 agent/installer.go:323
2025-03-03T20:24:43.273Z INFO [UPDATER]   Extracting Teleport tarball. path:/opt/teleport/default/versions/17.3.0-dev.hugoau.7_ent size:761835520 agent/installer.go:343
2025-03-03T20:24:48.235Z WARN [UPDATER]   Found unexpected non-executable file name:fdpass-teleport agent/validate.go:96
2025-03-03T20:24:48.236Z WARN [UPDATER]   Found unexpected non-executable file name:tbot agent/validate.go:96
2025-03-03T20:24:48.236Z WARN [UPDATER]   Found unexpected non-executable file name:tctl agent/validate.go:96
2025-03-03T20:24:48.236Z WARN [UPDATER]   Found unexpected non-executable file name:teleport agent/validate.go:96
2025-03-03T20:24:48.236Z WARN [UPDATER]   Found unexpected non-executable file name:teleport-update agent/validate.go:96
2025-03-03T20:24:48.236Z WARN [UPDATER]   Found unexpected non-executable file name:tsh agent/validate.go:96
2025-03-03T20:24:48.236Z ERRO [UPDATER]   Command failed. error:[
ERROR REPORT:
Original Error: *errors.errorString no binaries available to link
Stack Trace:
github.com/gravitational/teleport/lib/autoupdate/agent/insta
```

This PR changes the umask of the whole `teleport-update` process to ensure all operations are consistent with teleport-update when run by systemd (which always uses 0022). This would be undesirable for most CLIs, but `teleport-update` should never create files as a normal user, or create files in normal user directories.

As a workaround until this PR is merged, users can run `umask 0022` before `teleport-update enable`.

--

changelog: Allow teleport-update to be used in shells that set a restrictive umask

--

The `teleport-update` binary will be used to enable, disable, and trigger automatic Teleport agent updates. The new auto-updates system manages a local installation of the cluster-specified version of Teleport stored in `/opt/teleport`.

RFD: https://github.com/gravitational/teleport/pull/47126
Goal (internal): https://github.com/gravitational/cloud/issues/11856


Diff:
diff --git a/lib/autoupdate/agent/installer.go b/lib/autoupdate/agent/installer.go
index a3005ccd0406c..96b5d9c3fadd9 100644
--- a/lib/autoupdate/agent/installer.go
+++ b/lib/autoupdate/agent/installer.go
@@ -64,6 +64,7 @@ const (
 
 // LocalInstaller manages the creation and removal of installations
 // of Teleport.
+// SetRequiredUmask must be called before any methods are executed.
 type LocalInstaller struct {
 	// InstallDir contains each installation, named by version.
 	InstallDir string
diff --git a/lib/autoupdate/agent/updater.go b/lib/autoupdate/agent/updater.go
index f42579ea248a0..fbffe7a1510a8 100644
--- a/lib/autoupdate/agent/updater.go
+++ b/lib/autoupdate/agent/updater.go
@@ -30,6 +30,7 @@ import (
 	"path/filepath"
 	"runtime"
 	"slices"
+	"syscall"
 	"time"
 
 	"github.com/gravitational/trace"
@@ -57,6 +58,9 @@ const (
 	reservedFreeDisk = 10_000_000
 	// debugSocketFileName is the name of Teleport's debug socket in the data dir.
 	debugSocketFileName = "debug.sock" // 10 MB
+	// requiredUmask must be set before this package can be used.
+	// Use syscall.Umask to set when no other goroutines are running.
+	requiredUmask = 0o022
 )
 
 // Log keys
@@ -67,6 +71,20 @@ const (
 	errorKey  = "error"
 )
 
+// SetRequiredUmask sets the umask to match the systemd umask that the teleport-update service will execute with.
+// This ensures consistent file permissions.
+// NOTE: This must be run in main.go before any goroutines that create files are started.
+func SetRequiredUmask(ctx context.Context, log *slog.Logger) {
+	warnUmask(ctx, log, syscall.Umask(requiredUmask))
+}
+
+func warnUmask(ctx context.Context, log *slog.Logger, old int) {
+	if old&^requiredUmask != 0 {
+		log.WarnContext(ctx, "Restrictive umask detected. Umask has been changed to 0022 for teleport-update and all child processes.")
+		log.WarnContext(ctx, "All files created by teleport-update will have permissions set according to this umask.")
+	}
+}
+
 // NewLocalUpdater returns a new Updater that auto-updates local
 // installations of the Teleport agent.
 // The AutoUpdater uses an HTTP client with sane defaults for downloads, and
@@ -174,6 +192,7 @@ type LocalUpdaterConfig struct {
 }
 
 // Updater implements the agent-local logic for Teleport agent auto-updates.
+// SetRequiredUmask must be called before any methods are executed, except for Status.
 type Updater struct {
 	// Log contains a logger.
 	Log *slog.Logger
@@ -545,6 +564,7 @@ func isActiveOrEnabled(ctx context.Context, s Process) (bool, error) {
 
 // Status returns all available local and remote fields related to agent auto-updates.
 // Status is safe to run concurrently with other Updater commands.
+// Status does not write files, and therefore does not require SetRequiredUmask.
 func (u *Updater) Status(ctx context.Context) (Status, error) {
 	var out Status
 	// Read configuration from update.yaml.
diff --git a/lib/autoupdate/agent/updater_test.go b/lib/autoupdate/agent/updater_test.go
index cc789c20bca08..082b22e2ac162 100644
--- a/lib/autoupdate/agent/updater_test.go
+++ b/lib/autoupdate/agent/updater_test.go
@@ -19,9 +19,12 @@
 package agent
 
 import (
+	"bytes"
 	"context"
 	"encoding/json"
 	"errors"
+	"fmt"
+	"log/slog"
 	"net/http"
 	"net/http/httptest"
 	"os"
@@ -40,6 +43,40 @@ import (
 	"github.com/gravitational/teleport/lib/utils/testutils/golden"
 )
 
+func TestWarnUmask(t *testing.T) {
+	t.Parallel()
+
+	for _, tt := range []struct {
+		old  int
+		warn bool
+	}{
+		{old: 0o000, warn: false},
+		{old: 0o001, warn: true},
+		{old: 0o011, warn: true},
+		{old: 0o111, warn: true},
+		{old: 0o002, warn: false},
+		{old: 0o020, warn: false},
+		{old: 0o022, warn: false},
+		{old: 0o220, warn: true},
+		{old: 0o200, warn: true},
+		{old: 0o222, warn: true},
+		{old: 0o333, warn: true},
+		{old: 0o444, warn: true},
+		{old: 0o555, warn: true},
+		{old: 0o666, warn: true},
+		{old: 0o777, warn: true},
+	} {
+		t.Run(fmt.Sprintf("old umask %o", tt.old), func(t *testing.T) {
+			ctx := context.Background()
+			out := &bytes.Buffer{}
+			warnUmask(ctx, slog.New(slog.NewTextHandler(out,
+				&slog.HandlerOptions{ReplaceAttr: msgOnly},
+			)), tt.old)
+			assert.Equal(t, tt.warn, strings.Contains(out.String(), "detected"))
+		})
+	}
+}
+
 func TestUpdater_Disable(t *testing.T) {
 	t.Parallel()
 
diff --git a/tool/teleport-update/main.go b/tool/teleport-update/main.go
index f8a36e106ffbc..58ccb6bdd0584 100644
--- a/tool/teleport-update/main.go
+++ b/tool/teleport-update/main.go
@@ -184,6 +184,13 @@ func Run(args []string) int {
 		return 1
 	}
 
+	// Set required umask for commands that write files to system directories as root, and warn loudly if it changes.
+	switch command {
+	case statusCmd.FullCommand(), versionCmd.FullCommand():
+	default:
+		autoupdate.SetRequiredUmask(ctx, plog)
+	}
+
 	switch command {
 	case enableCmd.FullCommand():
 		ccfg.Enabled = true


Reviews:
user: login: vapopov
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T21:34:25Z
user: login: zmb3
body: I'm good with dropping the flag so long as we keep the warning message.

Just because it was decided that the updater is theoretically not a package manager doesn't mean it doesn't perform tasks similar to that of a package manager and doesn't mean we can't take inspiration from prior art.
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-04T04:03:17Z
user: login: fheinecke
body: Looks good enough to ship to fix customer breakage. Would appreciate if you reviewed the comments after the PR, and a fix is less urgently needed.

> I reached out the customer who reported this issue for feedback. Requiring the flag on each invocation would be disruptive UX, so I removed it.

Sounds great, thanks for asking them! If that's what customers want, then we should do it that way.

state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T05:07:04Z
user: login: sclevine
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-04T05:35:25Z
user: login: sclevine
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-04T05:43:50Z
user: login: hugoShaka
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T13:30:43Z
user: login: fheinecke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T19:21:20Z

Review comments:
diff_hunk: @@ -184,6 +184,13 @@ func Run(args []string) int {
 		return 1
 	}
 
+	// Set required umask for most commands, and warn loudly if it changes.
path: tool/teleport-update/main.go
user: login: fheinecke
body: Does this need to be set for all commands or just ones that install or update? I'm not familiar enough with the tool to know where this is and is not needed.

Alternatively, can this more tightly wrap the actual filesystem write calls? This would reduce the likelihood of a vulnerability in the future where an attacker is able to use the updater to write executable files.
created_at: 2025-03-04T04:57:44Z
author_association: CONTRIBUTOR
diff_hunk: @@ -67,6 +71,17 @@ const (
 	errorKey  = "error"
 )
 
+// SetRequiredUmask sets the umask to match the systemd umask that the teleport-update service will execute with.
+// This ensures consistent file permissions.
+// NOTE: This must be run in main.go before any goroutines that create files are started.
+func SetRequiredUmask(ctx context.Context, log *slog.Logger) {
path: lib/autoupdate/agent/updater.go
user: login: fheinecke
body: Can you add some tests around this and the updater itself to catch regressions?
created_at: 2025-03-04T05:05:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -184,6 +184,13 @@ func Run(args []string) int {
 		return 1
 	}
 
+	// Set required umask for most commands, and warn loudly if it changes.
path: tool/teleport-update/main.go
user: login: sclevine
body: > Does this need to be set for all commands or just ones that install or update?

It needs to be set for all commands that must run as root and write system files, so I scoped it to just those commands.
I updated the godoc to clarify this as well. We can make sure that it doesn't get set for user commands.

(This is similar to systemd, in a way. Systemd system services always get `Umask = 0022` unless they explicitly define their own umask. There is no way for root to change that behavior. However, user services respect the user umask.)

> Alternatively, can this more tightly wrap the actual filesystem write calls? 

This is tricky, because the umask is process-global, and changing it impacts all goroutines in a thread-unsafe manner. I agree with your point about `syscall.Umask` having important security implications, and I want to avoid calling it in importable code that could impact other goroutines by silently modifying their file permissions.
created_at: 2025-03-04T05:35:24Z
author_association: MEMBER
diff_hunk: @@ -67,6 +71,17 @@ const (
 	errorKey  = "error"
 )
 
+// SetRequiredUmask sets the umask to match the systemd umask that the teleport-update service will execute with.
+// This ensures consistent file permissions.
+// NOTE: This must be run in main.go before any goroutines that create files are started.
+func SetRequiredUmask(ctx context.Context, log *slog.Logger) {
path: lib/autoupdate/agent/updater.go
user: login: sclevine
body: I plan to add a case to @hugoShaka's Ansible integration test suite to ensure it's set appropriately throughout the whole workflow: https://github.com/gravitational/teleport.e/tree/hugo/autoupdate-ansible-playbook/tests/ansible-autoupdate-playbook (currently private link)

I'll add a unit test to the bitwise conditional on the warning. I'm hesitant to call `syscall.Umask` in unit tests, since it impacts all tests in parallel and could lead to very hard to debug flakes.

created_at: 2025-03-04T05:43:50Z
author_association: MEMBER
diff_hunk: @@ -184,6 +184,13 @@ func Run(args []string) int {
 		return 1
 	}
 
+	// Set required umask for most commands, and warn loudly if it changes.
path: tool/teleport-update/main.go
user: login: fheinecke
body: This might not be feasible, but could all functions that could be feasibly impacted by this pass around a mutex, where if a given function held the lock, it could set/unset the umask? Or maybe there could be two "groups" of functions - one privileged and one not - that would share this mutex.
created_at: 2025-03-04T19:21:20Z
author_association: CONTRIBUTOR

PR comments:
user: login: fheinecke
created_at: 2025-03-03T21:35:33Z
body: Is this really something that we should be setting? A umask value of 0022 is pretty standard, and if it's not 0022 then having it set to some other value is probably intentional.
author_association: CONTRIBUTOR
user: login: sclevine
created_at: 2025-03-03T21:52:03Z
body: > Is this really something that we should be setting? A umask value of 0022 is pretty standard, and if it's not 0022 then having it set to some other value is probably intentional.

I'm definitely on the fence about this. I went this direction because `teleport-update` never touches user files, just files in /opt and /usr/local, and it's behavior should always be consistent with the systemd service behavior. I could modify `utils.Extract` and many other places where teleport-update writes files to `os.Chmod` after creating, but I worry that future additions to `teleport-update` might miss this.
author_association: MEMBER
user: login: fheinecke
created_at: 2025-03-03T21:56:32Z
body: There's also some risk that if (not necessarily now but down the line as the code changes), an attacker may be able to exploit the updater in some way to create executable files. This would _probably_ be why an admin would remove the `x` bit from the umask in the first place. If this is the case, this would essentially be the `teleport-updater` saying "I know better/more about your environment than you" to the sysadmin, which is pretty much never going to be the case.

Personally I'm of the opinion that if most systems default to 0022, then we should build to be compatible with that. If a sysadmin changes the default, then it's up to them (for all applications) to ensure compatibility.
author_association: CONTRIBUTOR
user: login: sclevine
created_at: 2025-03-03T22:04:44Z
body: > Personally I'm of the opinion that if most systems default to 0022, then we should build to be compatible with that. If a sysadmin changes the default, then it's up to them (for all applications) to ensure compatibility.

This is an argument for not fixing the bug. We could just fail on unmask != 0022, with a more clear error.

For prior-art though, @espadolini points out that other package management tools do this:
https://github.com/rpm-software-management/rpm/blob/aab013a8a2c83e088f1c32ccabb44613afebb357/lib/transaction.cc#L1726-L1727
author_association: MEMBER
user: login: sclevine
created_at: 2025-03-03T22:14:49Z
body: I can definitely add a loud warning when we change the umask, if the old umask was different. Would that reduce concerns about unexpected behavior?
author_association: MEMBER
user: login: fheinecke
created_at: 2025-03-03T22:32:43Z
body: > For prior-art though, @espadolini points out that other package management tools do this

That's great, and is probably sufficient to set this, if we and customers are considering the updater a package manager. We  [had a discussion](https://github.com/gravitational/teleport/pull/40190#discussion_r1571300325) awhile back as to whether or not the new updater system is a package manager system. It was de facto decided that the updater is not a package manager, otherwise, we should be placing binaries `/usr/bin`.

> Would that reduce concerns about unexpected behavior?

It would, though not outright eliminate them. If we're considering the updater system to be a package manager system, then I think a warning + set would be fine. If not, IMO we should either warn and either continue or fail, and leave it to the sysadmin to.

On a more technical level, if we do want to change the mask, then limiting the scope of the umask change to specific logic (rather than the entire program) is probably the safest way to go.
author_association: CONTRIBUTOR
user: login: sclevine
created_at: 2025-03-03T22:53:26Z
body: Hm, we could fail unless you pass `--allow-ignore-umask`. Then I could move the umask change into the subcommands in `main.go` that write files, so that `status` doesn't require it. Any preference between warning and proceeding vs. requiring a flag? I do worry that the flag makes the UX worse, so I lean towards the first option.
author_association: MEMBER
user: login: fheinecke
created_at: 2025-03-03T23:00:21Z
body: Maybe a flag + env (with either one allowing for the umask override)? My concern is that the `teleport-update` will end up buried in a script, ansible playbook, etc. and nobody will see it, and assume that their changed umask is covering everything. Needing to explicitly tell the program to override it ensures that sysadmins are aware of what's going on.

Completely get that this user interface is a bit worse, but IMO if an admin is changing the umask then they're accepting that things will not always work properly the first time.

Of course I am just one datapoint. If you've heard otherwise from customers then we should probably defer to them.
author_association: CONTRIBUTOR
user: login: sclevine
created_at: 2025-03-03T23:46:34Z
body: @fheinecke how does that look?
author_association: MEMBER
user: login: sclevine
created_at: 2025-03-04T00:20:36Z
body: I reached out the customer who reported this issue for feedback. Requiring the flag on each invocation would be disruptive UX, so I removed it.
author_association: MEMBER
user: login: public-teleport-github-review-bot[bot]
created_at: 2025-03-04T17:06:48Z
body: 
@sclevine See the table below for backport results.

| Branch | Result |
|--------|--------|
| branch/v17 | [Create PR](https://github.com/gravitational/teleport/compare/branch/v17...bot/backport-52725-branch/v17?body=Backport+%2352725+to+branch%2Fv17%0A%0Achangelog%3A+Allow+teleport-update+to+be+used+in+shells+that+set+a+restrictive+umask%0D%0A&expand=1&title=%5Bv17%5D+%5Bteleport-update%5D+Set+umask+0022+for+teleport-update+to+avoid+errors+on+enable) |

author_association: NONE