Repository: canonical/lxd
Pull request #15534: Auth: Maintain permissions on instance move
Author: login: markylaing
Created: 2025-05-07T13:08:39Z | Merged: 2025-05-08T15:25:01Z

This PR fixes couple of issues in the auth subsystem when moving instances. 

Unfortunately, [maintaining instance ID's](https://github.com/canonical/lxd/issues/14932#issuecomment-2643099960) when moving them between pools/members became quite difficult - we'd need quite a large rewrite. Happy to talk further on this, but I'm not sure the pros of maintaining the instance ID outweigh the cons. The main benefit is not needing to update database entries that reference the instance by ID, whereas the main drawback is that move and copy would use different code paths, meaning more code to test and maintain.

In any case, for now, I've opted to update the permissions table to point to the new instance ID.

Closes #14932 

Linked issue #14932: Loss of instance permissions after storage pool migration
## LXD information
- lxd snap version: `git-2cc542a` (latest/edge)

## Description
When migrating an instance between storage pools, the associated permissions (such as `can_edit`) assigned via an auth group are lost. This occurs because the instance record is deleted and recreated, leading to the removal of permissions (per this [discussion](https://chat.canonical.com/canonical/pl/ugzb5p39wbbgd837hzh9k8resr)).

## Steps to reproduce
1. Assign an entitlement (e.g., can_edit) to a restricted user via an auth group for an instance.
2. Perform an instance migration to another storage pool.
3. Check the permissions for the instance after migration.

## Expected behaviour
The instance should retain its assigned permissions after migration.

## Actual behavior
The permissions are lost because the instance is recreated with a new record ID.

## Affected scenarios
- Moving an instance between storage pools.
- Moving an instance between cluster members.
Linked issue comments: user: login: markylaing
created_at: 2025-02-07T14:28:21Z
body: The fix for this should collect permissions that are directly related to the instance before it is deleted, then set those same permissions again on the newly created copy of the instance.
author_association: CONTRIBUTOR
user: login: markylaing
created_at: 2025-02-07T14:29:51Z
body: *Ideally* we would maintain the ID of the instance and update its' database record with the new cluster member ID.
author_association: CONTRIBUTOR
user: login: tomponline
created_at: 2025-02-07T14:30:04Z
body: > The fix for this should collect permissions that are directly related to the instance before it is deleted, then set those same permissions again on the newly created copy of the instance.

Thats one approach we could take.

But I'd also like to explore maintaining the ID (so not deleting the record), in order to avoid breaking other things that reference the record by ID (such as `warnings` and potentially replication schedules in the future).
author_association: MEMBER
user: login: mas-who
created_at: 2025-02-10T10:30:12Z
body: Additional note regarding moving an instance to a different cluster member:

When I tried moving an instance to a different cluster member, I get an error saying the instance cannot be found. The set of permissions I tried to do this is shown below:

![Image](https://github.com/user-attachments/assets/97aacb61-7fcd-47d6-b63a-34444ea43586)

Note I could migrate the instance with `instance_manager` entitlement for the project `default`, which I believe is equivalent to the set of instance permissions shown above. Am I doing something wrong with the permission required perhaps? I'm not entirely sure if this behaviour is 100% relevant to this issue, however, I think it's probably due to the sequence of operations required for instance migration (happy to open another issue if necessary).
author_association: NONE

Diff:
diff --git a/lxd/auth/drivers/openfga.go b/lxd/auth/drivers/openfga.go
index d88105396ed7..60746c97dff8 100644
--- a/lxd/auth/drivers/openfga.go
+++ b/lxd/auth/drivers/openfga.go
@@ -247,6 +247,7 @@ func (e *embeddedOpenFGA) CheckPermission(ctx context.Context, entityURL *api.UR
 		// If we have a not found error from the underlying OpenFGADatastore we should mask it to make requests consistent.
 		// (all not found errors returned before an access control decision is made are masked to prevent discovery).
 		if api.StatusErrorCheck(err, http.StatusNotFound) {
+			l.Debug("Entity not found", logger.Ctx{"http_code": http.StatusNotFound})
 			return api.NewGenericStatusError(http.StatusNotFound)
 		}
 
@@ -278,6 +279,7 @@ func (e *embeddedOpenFGA) CheckPermission(ctx context.Context, entityURL *api.UR
 				// If we have a not found error from the underlying OpenFGADatastore we should mask it to make requests consistent.
 				// (all not found errors returned before an access control decision is made are masked to prevent discovery).
 				if api.StatusErrorCheck(err, http.StatusNotFound) {
+					l.Debug("Entity not found", logger.Ctx{"http_code": http.StatusNotFound})
 					return api.NewGenericStatusError(http.StatusNotFound)
 				}
 
diff --git a/lxd/instance_post.go b/lxd/instance_post.go
index f53b59ea0c58..547268651cf5 100644
--- a/lxd/instance_post.go
+++ b/lxd/instance_post.go
@@ -618,6 +618,17 @@ func instancePostMigration(s *state.State, inst instance.Instance, req api.Insta
 		return err
 	}
 
+	// Update any permissions relating to the old instance to point to the new instance before it is deleted.
+	// Warnings relating to the old instance will be deleted.
+	err = s.DB.Cluster.Transaction(context.TODO(), func(ctx context.Context, tx *db.ClusterTx) error {
+		q := `UPDATE auth_groups_permissions SET entity_id = ? WHERE entity_type = ? AND entity_id = ?`
+		_, err = tx.Tx().ExecContext(ctx, q, targetInst.ID(), dbCluster.EntityType(entity.TypeInstance), inst.ID())
+		return err
+	})
+	if err != nil {
+		return fmt.Errorf("Failed to copy instance permissions: %w", err)
+	}
+
 	// Delete original instance.
 	err = inst.Delete(true)
 	if err != nil {
diff --git a/shared/entity/type.go b/shared/entity/type.go
index 77f50b3b09d9..a16cb39f655c 100644
--- a/shared/entity/type.go
+++ b/shared/entity/type.go
@@ -16,10 +16,23 @@ type typeInfo interface {
 	// specific, false if not.
 	requiresProject() bool
 
+	// requiresLocation() bool returns whether the Type requires a location to be uniquely specified, e.g. true if it is
+	// node specific, false if not.
+	requiresLocation() bool
+
 	// path returns the API path for the resource. The pathPlaceholder constant should be used in place of mux variables.
 	path() []string
 }
 
+// typeInfoCommon partially implements typeInfo and can be embedded in typeInfo
+// implementations for convenience.
+type typeInfoCommon struct{}
+
+// requiresLocation returns false by default.
+func (typeInfoCommon) requiresLocation() bool {
+	return false
+}
+
 const (
 	// TypeContainer represents container resources.
 	TypeContainer Type = "container"
@@ -180,7 +193,9 @@ func APIMetricsEntityTypes() []Type {
 	return metricsEntityTypes
 }
 
-type container struct{}
+type container struct {
+	typeInfoCommon
+}
 
 func (container) requiresProject() bool {
 	return true
@@ -190,7 +205,9 @@ func (container) path() []string {
 	return []string{"containers", pathPlaceholder}
 }
 
-type image struct{}
+type image struct {
+	typeInfoCommon
+}
 
 func (image) requiresProject() bool {
 	return true
@@ -200,7 +217,9 @@ func (image) path() []string {
 	return []string{"images", pathPlaceholder}
 }
 
-type profile struct{}
+type profile struct {
+	typeInfoCommon
+}
 
 func (profile) requiresProject() bool {
 	return true
@@ -210,7 +229,9 @@ func (profile) path() []string {
 	return []string{"profiles", pathPlaceholder}
 }
 
-type project struct{}
+type project struct {
+	typeInfoCommon
+}
 
 func (project) requiresProject() bool {
 	return false
@@ -220,7 +241,9 @@ func (project) path() []string {
 	return []string{"projects", pathPlaceholder}
 }
 
-type certificate struct{}
+type certificate struct {
+	typeInfoCommon
+}
 
 func (certificate) requiresProject() bool {
 	return false
@@ -230,7 +253,9 @@ func (certificate) path() []string {
 	return []string{"certificates", pathPlaceholder}
 }
 
-type instance struct{}
+type instance struct {
+	typeInfoCommon
+}
 
 func (instance) requiresProject() bool {
 	return true
@@ -240,7 +265,9 @@ func (instance) path() []string {
 	return []string{"instances", pathPlaceholder}
 }
 
-type instanceBackup struct{}
+type instanceBackup struct {
+	typeInfoCommon
+}
 
 func (instanceBackup) requiresProject() bool {
 	return true
@@ -250,7 +277,9 @@ func (instanceBackup) path() []string {
 	return []string{"instances", pathPlaceholder, "backups", pathPlaceholder}
 }
 
-type instanceSnapshot struct{}
+type instanceSnapshot struct {
+	typeInfoCommon
+}
 
 func (instanceSnapshot) requiresProject() bool {
 	return true
@@ -260,7 +289,9 @@ func (instanceSnapshot) path() []string {
 	return []string{"instances", pathPlaceholder, "snapshots", pathPlaceholder}
 }
 
-type network struct{}
+type network struct {
+	typeInfoCommon
+}
 
 func (network) requiresProject() bool {
 	return true
@@ -270,7 +301,9 @@ func (network) path() []string {
 	return []string{"networks", pathPlaceholder}
 }
 
-type networkACL struct{}
+type networkACL struct {
+	typeInfoCommon
+}
 
 func (networkACL) requiresProject() bool {
 	return true
@@ -280,7 +313,9 @@ func (networkACL) path() []string {
 	return []string{"network-acls", pathPlaceholder}
 }
 
-type clusterMember struct{}
+type clusterMember struct {
+	typeInfoCommon
+}
 
 func (clusterMember) requiresProject() bool {
 	return false
@@ -290,7 +325,9 @@ func (clusterMember) path() []string {
 	return []string{"cluster", "members", pathPlaceholder}
 }
 
-type operation struct{}
+type operation struct {
+	typeInfoCommon
+}
 
 func (operation) requiresProject() bool {
 	return false
@@ -300,7 +337,9 @@ func (operation) path() []string {
 	return []string{"operations", pathPlaceholder}
 }
 
-type storagePool struct{}
+type storagePool struct {
+	typeInfoCommon
+}
 
 func (storagePool) requiresProject() bool {
 	return false
@@ -320,6 +359,10 @@ func (storageVolume) path() []string {
 	return []string{"storage-pools", pathPlaceholder, "volumes", pathPlaceholder, pathPlaceholder}
 }
 
+func (storageVolume) requiresLocation() bool {
+	return true
+}
+
 type storageVolumeBackup struct{}
 
 func (storageVolumeBackup) requiresProject() bool {
@@ -330,6 +373,10 @@ func (storageVolumeBackup) path() []string {
 	return []string{"storage-pools", pathPlaceholder, "volumes", pathPlaceholder, pathPlaceholder, "backups", pathPlaceholder}
 }
 
+func (storageVolumeBackup) requiresLocation() bool {
+	return true
+}
+
 type storageVolumeSnapshot struct{}
 
 func (storageVolumeSnapshot) requiresProject() bool {
@@ -340,7 +387,13 @@ func (storageVolumeSnapshot) path() []string {
 	return []string{"storage-pools", pathPlaceholder, "volumes", pathPlaceholder, pathPlaceholder, "snapshots", pathPlaceholder}
 }
 
-type warning struct{}
+func (storageVolumeSnapshot) requiresLocation() bool {
+	return true
+}
+
+type warning struct {
+	typeInfoCommon
+}
 
 func (warning) requiresProject() bool {
 	return false
@@ -350,7 +403,9 @@ func (warning) path() []string {
 	return []string{"warnings", pathPlaceholder}
 }
 
-type clusterGroup struct{}
+type clusterGroup struct {
+	typeInfoCommon
+}
 
 func (clusterGroup) requiresProject() bool {
 	return false
@@ -360,7 +415,9 @@ func (clusterGroup) path() []string {
 	return []string{"cluster", "groups", pathPlaceholder}
 }
 
-type storageBucket struct{}
+type storageBucket struct {
+	typeInfoCommon
+}
 
 func (storageBucket) requiresProject() bool {
 	return true
@@ -370,7 +427,9 @@ func (storageBucket) path() []string {
 	return []string{"storage-pools", pathPlaceholder, "buckets", pathPlaceholder}
 }
 
-type server struct{}
+type server struct {
+	typeInfoCommon
+}
 
 func (server) requiresProject() bool {
 	return false
@@ -380,7 +439,9 @@ func (server) path() []string {
 	return []string{}
 }
 
-type imageAlias struct{}
+type imageAlias struct {
+	typeInfoCommon
+}
 
 func (imageAlias) requiresProject() bool {
 	return true
@@ -390,7 +451,9 @@ func (imageAlias) path() []string {
 	return []string{"images", "aliases", pathPlaceholder}
 }
 
-type networkZone struct{}
+type networkZone struct {
+	typeInfoCommon
+}
 
 func (networkZone) requiresProject() bool {
 	return true
@@ -400,7 +463,9 @@ func (networkZone) path() []string {
 	return []string{"network-zones", pathPlaceholder}
 }
 
-type identity struct{}
+type identity struct {
+	typeInfoCommon
+}
 
 func (identity) requiresProject() bool {
 	return false
@@ -410,7 +475,9 @@ func (identity) path() []string {
 	return []string{"auth", "identities", pathPlaceholder, pathPlaceholder}
 }
 
-type authGroup struct{}
+type authGroup struct {
+	typeInfoCommon
+}
 
 func (authGroup) requiresProject() bool {
 	return false
@@ -420,7 +487,9 @@ func (authGroup) path() []string {
 	return []string{"auth", "groups", pathPlaceholder}
 }
 
-type identityProviderGroup struct{}
+type identityProviderGroup struct {
+	typeInfoCommon
+}
 
 func (identityProviderGroup) requiresProject() bool {
 	return false
diff --git a/shared/entity/type_test.go b/shared/entity/type_test.go
index d004f1a074b1..4f36dfff5516 100644
--- a/shared/entity/type_test.go
+++ b/shared/entity/type_test.go
@@ -144,6 +144,16 @@ func TestURL(t *testing.T) {
 			expectedPathArgs:      []string{"my-instance"},
 			expectedErr:           nil,
 		},
+		{
+			name:                  "instances",
+			rawURL:                "/1.0/instances/my-instance?target=member01",
+			expectedNormalisedURL: "/1.0/instances/my-instance?project=default",
+			expectedEntityType:    TypeInstance,
+			expectedLocation:      "member01",
+			expectedProject:       api.ProjectDefaultName,
+			expectedPathArgs:      []string{"my-instance"},
+			expectedErr:           nil,
+		},
 		{
 			name:                  "instance backup",
 			rawURL:                "/1.0/instances/my-instance/backups/my-backup?project=my-project",
diff --git a/shared/entity/url.go b/shared/entity/url.go
index e91c0c5a0c17..4d2a058f253a 100644
--- a/shared/entity/url.go
+++ b/shared/entity/url.go
@@ -64,8 +64,11 @@ func (t Type) URL(projectName string, location string, pathArguments ...string)
 		u = u.WithQuery("project", projectName)
 	}
 
-	// Always set location if provided (empty or "none" locations are ignored).
-	u = u.Target(location)
+	// Only set location if required to uniquely identify the entity.
+	if info.requiresLocation() {
+		u = u.Target(location)
+	}
+
 	return u, nil
 }
 
diff --git a/test/suites/clustering_move.sh b/test/suites/clustering_move.sh
index bbd10c9bc8f2..849c97c67957 100644
--- a/test/suites/clustering_move.sh
+++ b/test/suites/clustering_move.sh
@@ -42,24 +42,37 @@ test_clustering_move() {
   LXD_DIR="${LXD_ONE_DIR}" lxc init testimage c2 --target node2
   LXD_DIR="${LXD_ONE_DIR}" lxc init testimage c3 --target node3
 
+  # Set up a fine-grained TLS identity.
+  token="$(LXD_DIR=${LXD_ONE_DIR} lxc auth identity create tls/test --quiet)"
+  lxc remote add cluster 10.1.1.101:8443 --token="${token}"
+  lxc remote set-url cluster https://10.1.1.102:8443
+
+  # Make the identity a member of a group that has minimal permissions for moving the instances.
+  LXD_DIR=${LXD_ONE_DIR} lxc auth group create instance-movers
+  LXD_DIR=${LXD_ONE_DIR} lxc auth identity group add tls/test instance-movers
+  LXD_DIR=${LXD_ONE_DIR} lxc auth group permission add instance-movers project default can_create_instances # Required, since a move constitutes an initial copy.
+  LXD_DIR=${LXD_ONE_DIR} lxc auth group permission add instance-movers project default can_view_events # Required for the CLI to watch the operation.
+  LXD_DIR=${LXD_ONE_DIR} lxc auth group permission add instance-movers instance c1 can_edit project=default
+  LXD_DIR=${LXD_ONE_DIR} lxc auth group permission add instance-movers instance c1 can_view project=default
+
   # Perform default move tests falling back to the built in logic of choosing the node
   # with the least number of instances when targeting a cluster group.
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target node2
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target @foobar1
-  LXD_DIR="${LXD_ONE_DIR}" lxc info c1 | grep -q "Location: node1"
+  lxc move cluster:c1 --target node2
+  lxc move cluster:c1 --target @foobar1
+  lxc info cluster:c1 | grep -q "Location: node1"
 
   # c1 can be moved within the same cluster group if it has multiple members
-  current_location="$(LXD_DIR="${LXD_ONE_DIR}" lxc query /1.0/instances/c1 | jq -r '.location')"
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target=@default
-  LXD_DIR="${LXD_ONE_DIR}" lxc query /1.0/instances/c1 | jq -re ".location != \"$current_location\""
-  current_location="$(LXD_DIR="${LXD_ONE_DIR}" lxc query /1.0/instances/c1 | jq -r '.location')"
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target=@default
-  LXD_DIR="${LXD_ONE_DIR}" lxc query /1.0/instances/c1 | jq -re ".location != \"$current_location\""
+  current_location="$(query cluster:/1.0/instances/c1 | jq -r '.location')"
+  lxc move cluster:c1 --target=@default
+  query cluster:/1.0/instances/c1 | jq -re ".location != \"$current_location\""
+  current_location="$(query cluster:/1.0/instances/c1 | jq -r '.location')"
+  lxc move cluster:c1 --target=@default
+  query cluster:/1.0/instances/c1 | jq -re ".location != \"$current_location\""
 
   # c1 cannot be moved within the same cluster group if it has a single member
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target=@foobar3
-  LXD_DIR="${LXD_ONE_DIR}" lxc info c1 | grep -q "Location: node3"
-  ! LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target=@foobar3 || false
+  lxc move cluster:c1 --target=@foobar3
+  lxc info cluster:c1 | grep -q "Location: node3"
+  ! lxc move cluster:c1 --target=@foobar3 || false
 
   # Perform standard move tests using the `scheduler.instance` cluster member setting.
   LXD_DIR="${LXD_ONE_DIR}" lxc cluster set node2 scheduler.instance=group
@@ -74,31 +87,35 @@ test_clustering_move() {
   # - c3 is deployed on node3
 
   # c1 can be moved to node2 by group targeting.
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target=@foobar2
-  LXD_DIR="${LXD_ONE_DIR}" lxc info c1 | grep -q "Location: node2"
+  lxc move cluster:c1 --target=@foobar2
+  lxc info cluster:c1 | grep -q "Location: node2"
 
   # c2 can be moved to node1 by manual targeting.
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c2 --target=node1
-  LXD_DIR="${LXD_ONE_DIR}" lxc info c2 | grep -q "Location: node1"
+  LXD_DIR=${LXD_ONE_DIR} lxc auth group permission add instance-movers instance c2 can_edit project=default
+  LXD_DIR=${LXD_ONE_DIR} lxc auth group permission add instance-movers instance c2 can_view project=default
+  lxc move cluster:c2 --target=node1
+  lxc info cluster:c2 | grep -q "Location: node1"
 
   # c1 cannot be moved to node3 by group targeting.
-  ! LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target=@foobar3 || false
+  ! lxc move cluster:c1 --target=@foobar3 || false
 
   # c2 can be moved to node2 by manual targeting.
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c2 --target=node2
+  lxc move cluster:c2 --target=node2
 
   # c3 can be moved to node1 by manual targeting.
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c3 --target=node1
-  LXD_DIR="${LXD_ONE_DIR}" lxc info c3 | grep -q "Location: node1"
+  LXD_DIR=${LXD_ONE_DIR} lxc auth group permission add instance-movers instance c3 can_edit project=default
+  LXD_DIR=${LXD_ONE_DIR} lxc auth group permission add instance-movers instance c3 can_view project=default
+  lxc move cluster:c3 --target=node1
+  lxc info cluster:c3 | grep -q "Location: node1"
 
   # c3 can be moved back to node by by manual targeting.
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c3 --target=node3
-  LXD_DIR="${LXD_ONE_DIR}" lxc info c3 | grep -q "Location: node3"
+  lxc move cluster:c3 --target=node3
+  lxc info cluster:c3 | grep -q "Location: node3"
 
   # Clean up
   LXD_DIR="${LXD_ONE_DIR}" lxc cluster unset node2 scheduler.instance
   LXD_DIR="${LXD_ONE_DIR}" lxc cluster unset node3 scheduler.instance
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target node1
+  lxc move cluster:c1 --target node1
 
   # Perform extended scheduler tests involving the `instance.placement.scriptlet` global setting.
   # Start by statically targeting node3 (index 0).
@@ -113,14 +130,14 @@ def instance_placement(request, candidate_members):
         return
 EOF
 
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target @foobar3
-  LXD_DIR="${LXD_ONE_DIR}" lxc info c1 | grep -q "Location: node3"
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c2 --target @foobar3
-  LXD_DIR="${LXD_ONE_DIR}" lxc info c2 | grep -q "Location: node3"
+  lxc move cluster:c1 --target @foobar3
+  lxc info cluster:c1 | grep -q "Location: node3"
+  lxc move cluster:c2 --target @foobar3
+  lxc info cluster:c2 | grep -q "Location: node3"
 
   # Ensure that setting an invalid target won't interrupt the move and fall back to the built in behavior.
   # Equally distribute the instances beforehand so that node1 will get selected.
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c2 --target node2
+  lxc move cluster:c2 --target node2
 
   cat << EOF | LXD_DIR="${LXD_ONE_DIR}" lxc config set instances.placement.scriptlet=-
 def instance_placement(request, candidate_members):
@@ -131,8 +148,8 @@ def instance_placement(request, candidate_members):
         return
 EOF
 
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target @foobar1
-  LXD_DIR="${LXD_ONE_DIR}" lxc info c1 | grep -q "Location: node1"
+  lxc move cluster:c1 --target @foobar1
+  lxc info cluster:c1 | grep -q "Location: node1"
 
   # If the scriptlet produces a runtime error, the move fails.
   cat << EOF | LXD_DIR="${LXD_ONE_DIR}" lxc config set instances.placement.scriptlet=-
@@ -143,7 +160,7 @@ def instance_placement(request, candidate_members):
         return
 EOF
 
-  ! LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target @foobar2 || false
+  ! lxc move cluster:c1 --target @foobar2 || false
 
   # If the scriptlet intentionally runs into an error, the move fails.
   cat << EOF | LXD_DIR="${LXD_ONE_DIR}" lxc config set instances.placement.scriptlet=-
@@ -153,7 +170,7 @@ def instance_placement(request, candidate_members):
         fail("Instance not allowed") # Fail to prevent instance creation.
 EOF
 
-  ! LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target @foobar2 || false
+  ! lxc move cluster:c1 --target @foobar2 || false
 
   # Cleanup
   LXD_DIR="${LXD_ONE_DIR}" lxc config unset instances.placement.scriptlet
@@ -170,19 +187,27 @@ EOF
   LXD_DIR="${LXD_ONE_DIR}" lxc project set default restricted=true restricted.networks.uplinks=${bridge}
   LXD_DIR="${LXD_ONE_DIR}" lxc project set default restricted.cluster.groups=foobar1,foobar2
 
-  # Moving to a node that is not a member of foobar1 or foobar2 will fail.
-  # The same applies for an unlisted group
-  ! LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target @foobar3 || false
-  ! LXD_DIR="${LXD_ONE_DIR}" lxc move c2 --target node3 || false
+  # Moving to an unlisted group fails.
+  ! lxc move cluster:c1 --target @foobar3 || false
+
+  # Moving directly to another node within the cluster group fails because the caller does not have
+  # can_override_cluster_target_restriction on server.
+  ! lxc move cluster:c2 --target node2 || false
 
-  # Moving instances in between the restricted groups
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c1 --target node2
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c2 --target @foobar1
-  LXD_DIR="${LXD_ONE_DIR}" lxc move c3 --target node1
+  # After adding the entitlement, moving to a cluster member that is not in the list of restricted cluster groups will fail.
+  LXD_DIR="${LXD_ONE_DIR}" lxc auth group permission add instance-movers server can_override_cluster_target_restriction
+  ! lxc move cluster:c2 --target node3 || false
+
+  # Moving instances in between the restricted groups (note that targeting members directly now works after adding the entitlement).
+  lxc move cluster:c1 --target node2
+  lxc move cluster:c2 --target @foobar1
+  lxc move cluster:c3 --target node1
 
   # Cleanup
+  lxc remote remove cluster
   LXD_DIR="${LXD_ONE_DIR}" lxc delete -f c1 c2 c3
-
+  LXD_DIR="${LXD_ONE_DIR}" lxc auth group delete instance-movers
+  LXD_DIR="${LXD_ONE_DIR}" lxc auth identity delete tls/test
   LXD_DIR="${LXD_THREE_DIR}" lxd shutdown
   LXD_DIR="${LXD_TWO_DIR}" lxd shutdown
   LXD_DIR="${LXD_ONE_DIR}" lxd shutdown


Reviews:
user: login: tomponline
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-07T13:28:38Z
user: login: tomponline
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-07T13:29:20Z
user: login: tomponline
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-07T13:29:56Z
user: login: tomponline
body: A few small suggestions, but overall LGTM, thanks!

Are there similar fixes need for custom volumes?
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-05-07T13:31:56Z
user: login: markylaing
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-07T13:53:54Z
user: login: markylaing
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-07T13:57:03Z
user: login: markylaing
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-07T14:30:46Z
user: login: tomponline
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-07T15:07:08Z
user: login: tomponline
state: DISMISSED
author_association: MEMBER
submitted_at: 2025-05-08T12:24:27Z
user: login: tomponline
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-08T13:32:47Z

Review comments:
diff_hunk: @@ -260,7 +289,9 @@ func (instanceSnapshot) path() []string {
 	return []string{"instances", pathPlaceholder, "snapshots", pathPlaceholder}
 }
 
-type network struct{}
+type network struct {
path: shared/entity/type.go
user: login: tomponline
body: @markylaing in general "managed" networks are not cluster member specific, so this is correct, but do we need to consider the `lxc network list` output that shows cluster member specific network interfaces?
created_at: 2025-05-07T13:28:37Z
author_association: MEMBER
diff_hunk: @@ -64,8 +64,11 @@ func (t Type) URL(projectName string, location string, pathArguments ...string)
 		u = u.WithQuery("project", projectName)
 	}
 
-	// Always set location if provided (empty or "none" locations are ignored).
-	u = u.Target(location)
+	// Only set location if required.
path: shared/entity/url.go
user: login: tomponline
body: this would be useful to expand this comment to say:

> // Only set location if required to uniquely identify the entity
created_at: 2025-05-07T13:29:19Z
author_association: MEMBER
diff_hunk: @@ -247,6 +247,7 @@ func (e *embeddedOpenFGA) CheckPermission(ctx context.Context, entityURL *api.UR
 		// If we have a not found error from the underlying OpenFGADatastore we should mask it to make requests consistent.
 		// (all not found errors returned before an access control decision is made are masked to prevent discovery).
 		if api.StatusErrorCheck(err, http.StatusNotFound) {
+			l.Info("Entity not found", logger.Ctx{"http_code": http.StatusNotFound})
path: lxd/auth/drivers/openfga.go
user: login: tomponline
body: should these be warnings? They shouldnt happen normally right?
created_at: 2025-05-07T13:29:55Z
author_association: MEMBER
diff_hunk: @@ -260,7 +289,9 @@ func (instanceSnapshot) path() []string {
 	return []string{"instances", pathPlaceholder, "snapshots", pathPlaceholder}
 }
 
-type network struct{}
+type network struct {
path: shared/entity/type.go
user: login: markylaing
body: This shouldn't be a problem as we have a catch all for unmanaged networks at the server level (`can_view_unmanaged_networks`), so permissions for them aren't checked using the `network` entity type. 
created_at: 2025-05-07T13:53:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -247,6 +247,7 @@ func (e *embeddedOpenFGA) CheckPermission(ctx context.Context, entityURL *api.UR
 		// If we have a not found error from the underlying OpenFGADatastore we should mask it to make requests consistent.
 		// (all not found errors returned before an access control decision is made are masked to prevent discovery).
 		if api.StatusErrorCheck(err, http.StatusNotFound) {
+			l.Info("Entity not found", logger.Ctx{"http_code": http.StatusNotFound})
path: lxd/auth/drivers/openfga.go
user: login: markylaing
body: This happens normally in many cases. For example with certificates, permissions are checked before we've attempted to access the certificate from the database. So if someone calls `PUT /1.0/certificates/not-a-certificate` we'll hit this code path.
created_at: 2025-05-07T13:57:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -64,8 +64,11 @@ func (t Type) URL(projectName string, location string, pathArguments ...string)
 		u = u.WithQuery("project", projectName)
 	}
 
-	// Always set location if provided (empty or "none" locations are ignored).
-	u = u.Target(location)
+	// Only set location if required.
path: shared/entity/url.go
user: login: markylaing
body: Fixed.
created_at: 2025-05-07T14:30:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -247,6 +247,7 @@ func (e *embeddedOpenFGA) CheckPermission(ctx context.Context, entityURL *api.UR
 		// If we have a not found error from the underlying OpenFGADatastore we should mask it to make requests consistent.
 		// (all not found errors returned before an access control decision is made are masked to prevent discovery).
 		if api.StatusErrorCheck(err, http.StatusNotFound) {
+			l.Info("Entity not found", logger.Ctx{"http_code": http.StatusNotFound})
path: lxd/auth/drivers/openfga.go
user: login: tomponline
body: hrm, ok so we probably dont want to log that as info then
created_at: 2025-05-07T15:07:08Z
author_association: MEMBER

PR comments:
user: login: markylaing
created_at: 2025-05-07T13:58:14Z
body: > Are there similar fixes need for custom volumes?

I'm not sure! I'll check.

Also need to investigate why tests are failing. I ran all the ones I thought pertinent locally.
author_association: CONTRIBUTOR
user: login: markylaing
created_at: 2025-05-07T13:59:19Z
body: > Also need to investigate why tests are failing. I ran all the ones I thought pertinent locally.

Ahh, just some missing test clean up :facepalm: 
author_association: CONTRIBUTOR
user: login: tomponline
created_at: 2025-05-08T08:51:29Z
body: @markylaing lets lose the l.Info lines as we dont want to be logging that stuff loudly if it occurs routinely. Thanks
author_association: MEMBER
user: login: markylaing
created_at: 2025-05-08T09:02:41Z
body: > @markylaing lets lose the l.Info lines as we dont want to be logging that stuff loudly if it occurs routinely. Thanks

Yep will do. Was just looking into doing this for storage volumes but I think that's a little more involved and can be a separate PR.
author_association: CONTRIBUTOR
user: login: markylaing
created_at: 2025-05-08T09:07:12Z
body: > @markylaing lets lose the l.Info lines as we dont want to be logging that stuff loudly if it occurs routinely. Thanks

Downgraded the log level to debug for those messages.
author_association: CONTRIBUTOR
user: login: markylaing
created_at: 2025-05-08T09:14:08Z
body: Tests failing on a documentation dependency: https://github.com/canonical/lxd/actions/runs/14902825148/job/41858754416#step:14:314

Not sure what's going on there.
author_association: CONTRIBUTOR
user: login: markylaing
created_at: 2025-05-08T13:45:34Z
body: Tests happy now :)
author_association: CONTRIBUTOR