Repository: nvaccess/nvda

Issue:
Emulate mouse on touch screens
**Reported by k_kolev1985 on 2014-02-14 13:53**
At present, on a touch screen, NVDA can interact with things on the screen only via object navigation. If an object is well designed, it is OK. But if not - problems arise. For example, if an object has no default action assigned, there is no way (at least as far as I know) to interact with it, only via the touch screen - NVDA will say "no action" and nothing will happen. For other cases in witch object exploring fails when working only with a touch screen, see tickets #3779 and #3780. To read text on a touch screen with gestures, there is the so called "text mode", To switch between those 2 modes (text and objects), the user must tap with 3 fingers on the touch screen.

So, to improve the touch screen experience with NVDA, I suggest that a third mode of screen exploration is added to NVDA. The mode will emulate the mouse cursor. When this new mode is activated and the user explores the screen with his finger, NVDA will behave as if the mouse cursor is following the user's finger and NVDA will read things on the screen as if the user is moving the mouse cursor on the screen with "mouse tracking" enabled. Gestures for performing various actions with the mouse buttons must be added, but I think that at least some of them (eg. left single click, right single click, mouse buttons lock) are already available and only gestures for those actions must be assigned. Also, gestures for left double click, middle button single click and wheel scrolling must also be added.

I don't know however if my suggestion of mouse emulation is even possible to implement, but I wont know until I ask - don't I (?) :).



Discussion:

Resolving patch:
diff --git a/source/globalCommands.py b/source/globalCommands.py
--- a/source/globalCommands.py
+++ b/source/globalCommands.py
@@ -2228,6 +2228,37 @@ def script_touch_hoverUp(self,gesture):
 				obj.doAction()
 	script_touch_hoverUp.category=SCRCAT_TOUCH
 
+	def script_touch_rightClick(self, gesture):
+		obj = api.getNavigatorObject()
+		# Ignore invisible or offscreen objects as they cannot even be navigated with touch gestures.
+		if controlTypes.STATE_INVISIBLE in obj.states or controlTypes.STATE_OFFSCREEN in obj.states:
+			return
+		try:
+			p = api.getReviewPosition().pointAtStart
+		except (NotImplementedError, LookupError):
+			p = None
+		if p:
+			x = p.x
+			y = p.y
+		else:
+			try:
+				(left, top, width, height) = obj.location
+			# Flake8/E722: stems from object location script.
+			except: # noqa
+				# Translators: Reported when the object has no location for the mouse to move to it.
+				ui.message(_("object has no location"))
+				return
+			# Don't bother clicking when parts or the entire object is offscreen.
+			if min(left, top, width, height) < 0:
+				return
+			x = left + (width // 2)
+			y = top + (height // 2)
+		winUser.setCursorPos(x, y)
+		self.script_rightMouseClick(gesture)
+	# Translators: Input help mode message for touch right click command.
+	script_touch_rightClick.__doc__ = _("Clicks the right mouse button at the current touch position. This is generally used to activate a context menu.") # noqa Flake8/E501
+	script_touch_rightClick.category = SCRCAT_TOUCH
+
 	def script_activateConfigProfilesDialog(self, gesture):
 		wx.CallAfter(gui.mainFrame.onConfigProfilesCommand, None)
 	# Translators: Describes the command to open the Configuration Profiles dialog.
@@ -2389,6 +2420,8 @@ def script_toggleScreenCurtain(self, gesture):
 		"ts:3finger_tap":"touch_changeMode",
 		"ts:2finger_double_tap":"showGui",
 		"ts:hoverUp":"touch_hoverUp",
+		"ts:tapAndHold": "touch_rightClick", # noqa (Flake8/ET121)
+
 		# Review cursor
 		"kb:shift+numpad7": "review_top",
 		"kb(laptop):NVDA+control+home": "review_top",


Test patch: