Repository: AvaloniaUI/Avalonia

Issue:
XYFocus cannot easily be restricted to branches of the visual tree.
### Describe the bug

At present when using XYFocus, the search for valid candidate controls to focus next always starts at the TopLevel.  While the search does restrict its results to controls that allow XYFocus, there's no simple way for a developer to restrict results to specific branches of the Visual tree.  That means that at present "modal" dialogs like DialogHost.Avalonia can't trap XYFocus within a displayed dialog, as any XYFocus enabled control that is a descendant of the same TopLevel is returned as a valid focus candidate.  Tab based navigation can be trapped with KeyboardNavigation.TabNavigation="Contained", and pointer based focus is also intercepted by the DialogHost overlay to prevent pointer events reaching "disabled" controls.

### To Reproduce

1. Clone https://github.com/IanRawley/XYFocusSearchIssue.git and compile.
2. Open the dialog with the button.  Focus will be placed on the first button within the dialog.
3. Use XYFocus to navigate.  Any attempts to navigate down from inside the dialog will focus the TextBox outside the dialog which should not be possible.

### Expected behavior

Some API for a developer to limit the XYFocus search for candidates to a specific subtree of the Visual Tree.

### Avalonia version

11.1.0

### OS

_No response_

### Additional context

I've been thinking of how this would be implemented, and had a few ideas.
1. When searching for candidates, choose the highest level ancestor of the initiating control which has XYFocus enabled, but whose parent does not.  While this would work generally, I think there are valid cases for having XYFocus able to move between neighbouring branches on the Visual tree but not to their parent or other sibling branches which this would prevent.
2. Focus engagement.  From what I understand Focus Engagement as a mechanic is intended for individual controls rather than branches, and would probably cause issues with nested controls that require engagement to function properly.
3. An attached property to set a search boundary.  XYFocus.TryDirectionalFocus() would then search upwards from the element that initiated navigation, and use the first control with the property set as the search root, or the current behaviour of the KeyboardNavigationHandler owner or TopLevel as fallbacks.  A similar sort of idea to Grid.IsSharedSizeScope.  This is something I think I could implement as a PR.
4. FocusScopes?  But that seems like it's got a lot of other issues attached to it, and being an effectively internal API doesn't allow app developers much control at the moment.
5. The developer turns XYFocus on and off on branches as views change, having XYFocus enabled only on branches that should be navigable to at any given time.  This gets messy quickly if you start having multiple DialogHost controls and XYFocus enabled in various branches that cannot be aware of each other.

My current preference is option 3 above, the attached property, which as I said I can probably make a PR for if it's deemed an appropriate option.


Hints:
One more idea for me actually, implement ICustomKeyboardNavigation somewhere appropriate and use that to trap XYFocus within a tree.

EDIT: Won't work.  None of the XYFocus methods that would be needed are public.  They're all internal / private.
https://learn.microsoft.com/en-us/windows/apps/design/input/focus-navigation#enabled

Looks like UWP behavior is supposed to prevent XY navigation between 2 sibling XYFocusKeyboardNavigation="Enabled" subtrees if their parent is explicitly disabled.  See the second image in the linked section.  

I'm not sure I think that's the right way to do it.  Personally I feel having more explicit control over boundaries without having to disable XYFocus is the better way.

If desired though I could probably do a PR to implement UWP behaviour here as well.
Digging into WinUI 3 code it behaves the same way, bubbling an XY navigation event up from the source until it finds a candidate to focus or hits a parent that isn't XY navigation enabled.  See
https://github.com/microsoft/microsoft-ui-xaml/blob/66a7b0ae71c19f89c6a7d86a1986794ad1a1bf09/src/dxaml/xcp/core/dll/eventmgr.cpp#L1385C9-L1401C6

and here
https://github.com/microsoft/microsoft-ui-xaml/blob/66a7b0ae71c19f89c6a7d86a1986794ad1a1bf09/src/dxaml/xcp/components/focus/inc/FocusSelection.h#L77-L80

Patch:
diff --git a/src/Avalonia.Base/Input/Navigation/XYFocus.Impl.cs b/src/Avalonia.Base/Input/Navigation/XYFocus.Impl.cs
index 867e80f1762..b2a79aa3b96 100644
--- a/src/Avalonia.Base/Input/Navigation/XYFocus.Impl.cs
+++ b/src/Avalonia.Base/Input/Navigation/XYFocus.Impl.cs
@@ -92,6 +92,11 @@ internal void UpdateManifolds(
             return null;
         }
 
+        if(!(XYFocusHelpers.FindXYSearchRoot(inputElement, keyDeviceType) is InputElement searchRoot))
+        {
+            return null;
+        }
+
         _instance.SetManifoldsFromBounds(bounds);
 
         return _instance.GetNextFocusableElement(direction, inputElement, engagedControl, true, new XYFocusOptions
@@ -99,7 +104,7 @@ internal void UpdateManifolds(
             KeyDeviceType = keyDeviceType,
             FocusedElementBounds = bounds,
             UpdateManifold = true,
-            SearchRoot = owner as InputElement ?? inputElement.GetVisualRoot() as InputElement
+            SearchRoot = searchRoot
         });
     }
 
diff --git a/src/Avalonia.Base/Input/Navigation/XYFocusHelpers.cs b/src/Avalonia.Base/Input/Navigation/XYFocusHelpers.cs
index 1914f6f6c63..7bdcfc8fb9e 100644
--- a/src/Avalonia.Base/Input/Navigation/XYFocusHelpers.cs
+++ b/src/Avalonia.Base/Input/Navigation/XYFocusHelpers.cs
@@ -1,4 +1,5 @@
 ﻿using System;
+using Avalonia.VisualTree;
 
 namespace Avalonia.Input;
 
@@ -13,11 +14,25 @@ private static bool IsAllowedXYNavigationMode(XYFocusNavigationModes modes, KeyD
     {
         return keyDeviceType switch
         {
-            null => true, // programmatic input, allow any subtree.
+            null => !modes.Equals(XYFocusNavigationModes.Disabled), // programmatic input, allow any subtree except Disabled.
             KeyDeviceType.Keyboard => modes.HasFlag(XYFocusNavigationModes.Keyboard),
             KeyDeviceType.Gamepad => modes.HasFlag(XYFocusNavigationModes.Gamepad),
             KeyDeviceType.Remote => modes.HasFlag(XYFocusNavigationModes.Remote),
             _ => throw new ArgumentOutOfRangeException(nameof(keyDeviceType), keyDeviceType, null)
         };
     }
+
+    internal static InputElement? FindXYSearchRoot(this InputElement visual, KeyDeviceType? keyDeviceType)
+    {
+        InputElement candidate = visual;
+        InputElement? candidateParent = visual.FindAncestorOfType<InputElement>();
+
+        while (candidateParent is not null && candidateParent.IsAllowedXYNavigationMode(keyDeviceType))
+        {
+            candidate = candidateParent;
+            candidateParent = candidate.FindAncestorOfType<InputElement>();
+        }
+
+        return candidate;
+    }
 }


Test patch:
diff --git a/tests/Avalonia.Base.UnitTests/Input/KeyboardNavigationTests_XY.cs b/tests/Avalonia.Base.UnitTests/Input/KeyboardNavigationTests_XY.cs
index 2ea3a51908f..fa5286a8d80 100644
--- a/tests/Avalonia.Base.UnitTests/Input/KeyboardNavigationTests_XY.cs
+++ b/tests/Avalonia.Base.UnitTests/Input/KeyboardNavigationTests_XY.cs
@@ -418,4 +418,45 @@ public void Can_Focus_Any_Element_If_Nothing_Was_Focused()
 
         Assert.Equal(candidate, FocusManager.GetFocusManager(window)!.GetFocusedElement());
     }
+
+    [Fact]
+    public void Cannot_Focus_Across_XYFocus_Boundaries()
+    {
+        using var _ = UnitTestApplication.Start(TestServices.FocusableWindow);
+
+        var current = new Button() { Height = 20 };
+        var candidate = new Button() { Height = 20 };
+        var currentParent = new StackPanel
+        {
+            [XYFocus.NavigationModesProperty] = XYFocusNavigationModes.Enabled,
+            Orientation = Orientation.Vertical,
+            Spacing = 20,
+            Children = { current }
+        };
+        var candidateParent = new StackPanel
+        {
+            [XYFocus.NavigationModesProperty] = XYFocusNavigationModes.Enabled,
+            Orientation = Orientation.Vertical,
+            Spacing = 20,
+            Children = { candidate }
+        };
+
+        var grandparent = new StackPanel
+        {
+            [XYFocus.NavigationModesProperty] = XYFocusNavigationModes.Disabled,
+            Orientation = Orientation.Vertical,
+            Spacing = 20,
+            Children = { currentParent, candidateParent }
+        };
+
+        var window = new Window
+        {
+            [XYFocus.NavigationModesProperty] = XYFocusNavigationModes.Enabled,
+            Content = grandparent,
+            Height = 300
+        };
+        window.Show();
+
+        Assert.Null(KeyboardNavigationHandler.GetNext(current, NavigationDirection.Down));
+    }
 }