Repository: nvaccess/nvda

Issue:
UIA handler: introduce UIA drag/drop event handlers for use with announcing items that were rearranged on screen
Hi,

I believe the following was asked before, but figured it would be best to put it in a systematic way:

### Background:
When rearranging UIA objects such as Start menu tiles and Action Center items (Windows 10), UIA can raise drag drop events. This is checked by Narrator (at least) to announce the new object location. For example, when rearranging Action Center items by pressing Alt+Shift+left/right arrow keys, screen readers may announce the new location of the just dragged object. This feature will become useful in future Windows 10 releases as latest Windows Insider Preview build allows one to rearragne virtual desktops and UIA drag drop events are fired.

### Is your feature request related to a problem? Please describe.
Currently NVDA does not announce items or new locations when items are rearranged. Technially, it does through windows 10 app Essentials add-on, but it would be better to bring this to core.

### Describe the solution you'd like
Let NVDA listen to UIA drag/drop events and announce the rearrange item and new location by calling obj.reportFocus method.

### Describe alternatives you've considered
Leave it up to add-ons to do this.

### Additional context
For a practical example and justifications, see:
https://blogs.windows.com/windows-insider/2021/03/17/announcing-windows-10-insider-preview-build-21337/

### Implementation strategy:
UIA drag/drop event handler code from Windows 10 App Essentials add-on will be ported to NVDA Core and will correspond to gain focus event unless folks believe a separate set of UIA events should be created. There are six events related to drag and drop, but the events we are interested in are "drag dropped" and "drag complete".

Thanks.


Discussion:
Hi,

Actually, gain focus event won't work directly as NVDA (through UIA handler thread) says focus event was ignored by the dragged object. So it appears dedicated events should be created in UIA objects to queue gain focus event for focused objects.

Thanks.
This would be very helpful for the highlighter as well.
Adding this comment here instead of on the closed PR #12428:

>2022 update: see josephsl/wintenapps#75 as to why this PR should be taken care of sooner than later, as without this, users must rely on Windows App Essentials add-on to rearrange tiles and folders in Windows 11 Version 22H2 start menu.

_Originally posted by @josephsl in https://github.com/nvaccess/nvda/issues/12428#issuecomment-1221601039_
I guess to get the work going, https://github.com/nvaccess/nvda/pull/12428 can be split up in parts so that at least the basic reporting of states is there.
Hi, in this case, at least two PR’s will be created: one to handle drag complete event, the other to handle drop target event. A possible third PR might be introduced to handle drag drop effect property which might be the easiest to handle. Thanks.


Hi,

Based on code examiantion, looks like at least four PR's are needed:

1. Report draggable state in UIA objects.
2. Handle drag start/complete events.
3. Report drag drop effect property and its changes. This is the one Quinn is asking for.
4. Handle drop target dropped event.

I can come up with these PR's within a week.

Thanks.

Resolving patch:
diff --git a/source/NVDAObjects/UIA/__init__.py b/source/NVDAObjects/UIA/__init__.py
--- a/source/NVDAObjects/UIA/__init__.py
+++ b/source/NVDAObjects/UIA/__init__.py
@@ -2055,6 +2055,37 @@ def event_UIA_itemStatus(self):
 			ui.message(f"{self.name} {itemStatus}")
 		self._itemStatusCache = itemStatus
 
+	def event_UIA_dragDropEffect(self):
+		# UIA drag drop effect was introduced in Windows 8.
+		try:
+			ui.message(self._getUIACacheablePropertyValue(UIAHandler.UIA_DragDropEffectPropertyId))
+		except COMError:
+			pass
+
+	def event_UIA_dropTargetEffect(self):
+		# UIA drop target effect property was introduced in Windows 8.
+		try:
+			dropTargetEffect = self._getUIACacheablePropertyValue(
+				UIAHandler.UIA_DropTargetDropTargetEffectPropertyId
+			)
+		except COMError:
+			dropTargetEffect = ""
+		# Sometimes drop target effect text is empty as it comes from a different object.
+		if not dropTargetEffect:
+			for element in reversed(api.getFocusAncestors()):
+				if not isinstance(element, UIA):
+					continue
+				try:
+					dropTargetEffect = element._getUIACacheablePropertyValue(
+						UIAHandler.UIA_DropTargetDropTargetEffectPropertyId
+					)
+				except COMError:
+					dropTargetEffect = ""
+				if dropTargetEffect:
+					break
+		if dropTargetEffect:
+			ui.message(dropTargetEffect)
+
 
 class TreeviewItem(UIA):
 
diff --git a/source/UIAHandler/__init__.py b/source/UIAHandler/__init__.py
--- a/source/UIAHandler/__init__.py
+++ b/source/UIAHandler/__init__.py
@@ -169,19 +169,22 @@
 }
 
 UIAPropertyIdsToNVDAEventNames={
-	UIA_NamePropertyId:"nameChange",
-	UIA_HelpTextPropertyId:"descriptionChange",
-	UIA_ExpandCollapseExpandCollapseStatePropertyId:"stateChange",
-	UIA_ToggleToggleStatePropertyId:"stateChange",
-	UIA_IsEnabledPropertyId:"stateChange",
-	UIA_ValueValuePropertyId:"valueChange",
-	UIA_RangeValueValuePropertyId:"valueChange",
-	UIA_ControllerForPropertyId:"UIA_controllerFor",
-	UIA_ItemStatusPropertyId:"UIA_itemStatus",
+	UIA.UIA_NamePropertyId: "nameChange",
+	UIA.UIA_HelpTextPropertyId: "descriptionChange",
+	UIA.UIA_ExpandCollapseExpandCollapseStatePropertyId: "stateChange",
+	UIA.UIA_ToggleToggleStatePropertyId: "stateChange",
+	UIA.UIA_IsEnabledPropertyId: "stateChange",
+	UIA.UIA_ValueValuePropertyId: "valueChange",
+	UIA.UIA_RangeValueValuePropertyId: "valueChange",
+	UIA.UIA_ControllerForPropertyId: "UIA_controllerFor",
+	UIA.UIA_ItemStatusPropertyId: "UIA_itemStatus",
+	UIA.UIA_DragDropEffectPropertyId: "UIA_dragDropEffect",
+	UIA.UIA_DropTargetDropTargetEffectPropertyId: "UIA_dropTargetEffect",
 }
 
 globalEventHandlerGroupUIAPropertyIds = {
-	UIA.UIA_RangeValueValuePropertyId
+	UIA.UIA_RangeValueValuePropertyId,
+	UIA.UIA_DropTargetDropTargetEffectPropertyId,
 }
 
 localEventHandlerGroupUIAPropertyIds = (


Test patch: