Repository: nvaccess/nvda

Issue:
Add support for Windows Terminal
<!--
Please thoroughly read NVDA's wiki article on how to fill in this template, including how to provide the required files.
Issues may be closed if the required information is not present.
https://github.com/nvaccess/nvda/wiki/Github-issue-template-explanation-and-examples
-->

### Is your feature request related to a problem? Please describe.
The new Windows Terminal is inaccessible.

### Describe the solution you'd like
Once UIA is fully implemented upstream, the `NVDAObjects.UIA.WinConsoleUIA` class should provide support.

### Describe alternatives you've considered
None.

### Additional context

Blocked by microsoft/terminal#2447
Terminal programs: "speak typed words" provides incorrect results in some cases when "report dynamic content changes" is disabled
<!--
Please thoroughly read NVDA's wiki article on how to fill in this template, including how to provide the required files.
Issues may be closed if the required information is not present.
https://github.com/nvaccess/nvda/wiki/Github-issue-template-explanation-and-examples
-->

### Steps to reproduce:
1. Disable "report dynamic content changes" (NVDA+5) and enable "speak typed words" (NVDA+3).
2. Open a Windows Console, UIA or legacy using the new typed character support.
3. In a git repository, type `git log`.
4. Press q to close the pager.
5. Type "git" and press space.

### Actual behaviour:
NVDA says "qgit".

### Expected behaviour:
NVDA says "git".

### System configuration
#### NVDA installed/portable/running from source:
Installed.

#### NVDA version:
Latest master.

#### Windows version:
Windows 10 1909.

#### Name and version of other software in use when reproducing the issue:
Windows Console.

#### Other information about your system:
N/A

### Other questions
#### Does the issue still occur after restarting your computer?
N/A

#### Have you tried any other versions of NVDA? If so, please report their behaviors.
N/A

#### If addons are disabled, is your problem still occuring?
N/A

#### Did you try to run the COM registry fixing tool in NVDA menu / tools?
N/A



Discussion:
> Once UIA is fully implemented upstream, the `NVDAObjects.UIA.WinConsoleUIA` class should provide support.

Mm, note that `NVDAObjects.UIA.WinConsoleUIA` has some Windows Console specific workarounds in it. The Microsoft Console will most likely have a different UIA implementation that is hopefully more close to the default implementation in our UIATextInfo.



Resolving patch:
diff --git a/source/NVDAObjects/UIA/__init__.py b/source/NVDAObjects/UIA/__init__.py
--- a/source/NVDAObjects/UIA/__init__.py
+++ b/source/NVDAObjects/UIA/__init__.py
@@ -1,8 +1,9 @@
-#NVDAObjects/UIA/__init__.py
-#A part of NonVisual Desktop Access (NVDA)
-#This file is covered by the GNU General Public License.
-#See the file COPYING for more details.
-#Copyright (C) 2009-2019 NV Access Limited, Joseph Lee, Mohammad Suliman, Babbage B.V., Leonard de Ruijter
+# NVDAObjects/UIA/__init__.py
+# A part of NonVisual Desktop Access (NVDA)
+# This file is covered by the GNU General Public License.
+# See the file COPYING for more details.
+# Copyright (C) 2009-2020 NV Access Limited, Joseph Lee, Mohammad Suliman,
+# Babbage B.V., Leonard de Ruijter, Bill Dengler
 
 """Support for UI Automation (UIA) controls."""
 
@@ -948,6 +949,10 @@ def findOverlayClasses(self,clsList):
 		):
 			from . import winConsoleUIA
 			winConsoleUIA.findExtraOverlayClasses(self, clsList)
+		elif UIAClassName == "TermControl":
+			from . import winConsoleUIA
+			clsList.append(winConsoleUIA.WinTerminalUIA)
+
 		# Add editableText support if UIA supports a text pattern
 		if self.TextInfo==UIATextInfo:
 			if UIAHandler.autoSelectDetectionAvailable:
diff --git a/source/NVDAObjects/UIA/winConsoleUIA.py b/source/NVDAObjects/UIA/winConsoleUIA.py
--- a/source/NVDAObjects/UIA/winConsoleUIA.py
+++ b/source/NVDAObjects/UIA/winConsoleUIA.py
@@ -2,7 +2,7 @@
 # A part of NonVisual Desktop Access (NVDA)
 # This file is covered by the GNU General Public License.
 # See the file COPYING for more details.
-# Copyright (C) 2019 Bill Dengler
+# Copyright (C) 2019-2020 Bill Dengler
 
 import ctypes
 import NVDAHelper
@@ -13,19 +13,113 @@
 from comtypes import COMError
 from UIAUtils import isTextRangeOffscreen
 from . import UIATextInfo
-from ..behaviors import KeyboardHandlerBasedTypedCharSupport
+from ..behaviors import EnhancedTermTypedCharSupport, KeyboardHandlerBasedTypedCharSupport
 from ..window import Window
 
 
 class consoleUIATextInfo(UIATextInfo):
-
 	def __init__(self, obj, position, _rangeObj=None):
+		collapseToEnd = None
 		# We want to limit  textInfos to just the visible part of the console.
 		# Therefore we specifically handle POSITION_FIRST, POSITION_LAST and POSITION_ALL.
+		if not _rangeObj and position in (
+			textInfos.POSITION_FIRST,
+			textInfos.POSITION_LAST,
+			textInfos.POSITION_ALL
+		):
+			try:
+				_rangeObj, collapseToEnd = self._getBoundingRange(obj, position)
+			except (COMError, RuntimeError):
+				# We couldn't bound the console.
+				from logHandler import log
+				log.warning("Couldn't get bounding range for console", exc_info=True)
+				# Fall back to presenting the entire buffer.
+				_rangeObj, collapseToEnd = None, None
+		super(consoleUIATextInfo, self).__init__(obj, position, _rangeObj)
+		if collapseToEnd is not None:
+			self.collapse(end=collapseToEnd)
+
+	def _getBoundingRange(self, obj, position):
+		"""Returns the UIA text range to which the console should be bounded,
+		and whether the textInfo should be collapsed after instantiation."""
+		# microsoft/terminal#4495: In newer consoles,
+		# IUIAutomationTextRange::getVisibleRanges returns a reliable contiguous range.
+		_rangeObj = obj.UIATextPattern.GetVisibleRanges().GetElement(0)
+		collapseToEnd = None
+		if position == textInfos.POSITION_FIRST:
+			collapseToEnd = False
+		elif position == textInfos.POSITION_LAST:
+			# We must pull back the end by one character otherwise when we collapse to end,
+			# a console bug results in a textRange covering the entire console buffer!
+			# Strangely the *very* last character is a special blank point
+			# so we never seem to miss a real character.
+			_rangeObj.MoveEndpointByUnit(
+				UIAHandler.TextPatternRangeEndpoint_End,
+				UIAHandler.NVDAUnitsToUIAUnits['character'],
+				-1
+			)
+			collapseToEnd = True
+		return (_rangeObj, collapseToEnd)
+
+	def move(self, unit, direction, endPoint=None):
+		oldInfo = None
+		if self.basePosition != textInfos.POSITION_CARET:
+			# Ensure we haven't gone beyond the visible text.
+			# UIA adds thousands of blank lines to the end of the console.
+			boundingInfo = self.obj.makeTextInfo(textInfos.POSITION_ALL)
+			oldInfo = self.copy()
+		res = self._move(unit, direction, endPoint)
+		# Console textRanges have access to the entire console buffer.
+		# However, we want to limit ourselves to onscreen text.
+		# Therefore, if the textInfo was originally visible,
+		# but we are now above or below the visible range,
+		# Restore the original textRange and pretend the move didn't work.
+		if oldInfo:
+			try:
+				if (
+					(
+						self.compareEndPoints(boundingInfo, "startToStart") < 0
+						or self.compareEndPoints(boundingInfo, "startToEnd") >= 0
+					)
+					and not (
+						oldInfo.compareEndPoints(boundingInfo, "startToStart") < 0
+						or oldInfo.compareEndPoints(boundingInfo, "startToEnd") >= 0
+					)
+				):
+					self._rangeObj = oldInfo._rangeObj
+					return 0
+			except (COMError, RuntimeError):
+				pass
+		return res
+
+	def _move(self, unit, direction, endPoint=None):
+		"Perform a move without respect to bounding."
+		return super(consoleUIATextInfo, self).move(unit, direction, endPoint)
+
+	def __ne__(self, other):
+		"""Support more accurate caret move detection."""
+		return not self == other
+
+	def _get_text(self):
+		# #10036: return a space if the text range is empty.
+		# Consoles don't actually store spaces, the character is merely left blank.
+		res = super(consoleUIATextInfo, self)._get_text()
+		if not res:
+			return ' '
+		else:
+			return res
+
+
+class consoleUIATextInfoPre21H1(consoleUIATextInfo):
+	"""Fixes expand/collapse on end inclusive UIA text ranges, uses rangeFromPoint
+	instead of broken GetVisibleRanges for bounding, and implements word
+	movement support."""
+	def _getBoundingRange(self, obj, position):
 		# We could use IUIAutomationTextRange::getVisibleRanges, but it seems very broken in consoles
 		# once more than a few screens worth of content has been written to the console.
 		# Therefore we resort to using IUIAutomationTextPattern::rangeFromPoint
 		# for the top left, and bottom right of the console window.
+		_rangeObj = None
 		if position is textInfos.POSITION_FIRST:
 			_rangeObj = self.__class__(obj, obj.location.topLeft)._rangeObj
 		elif position is textInfos.POSITION_LAST:
@@ -46,13 +140,13 @@ def __init__(self, obj, position, _rangeObj=None):
 			last = self.__class__(obj, textInfos.POSITION_LAST)
 			first.setEndPoint(last, "endToEnd")
 			_rangeObj = first._rangeObj
-		super(consoleUIATextInfo, self).__init__(obj, position, _rangeObj)
+		return (_rangeObj, None)
 
 	def collapse(self, end=False):
-		"""Works around a UIA bug on Windows 10 1803 and later."""
-		# When collapsing, consoles seem to incorrectly push the start of the
-		# textRange back one character.
-		# Correct this by bringing the start back up to where the end is.
+		"""Works around a UIA bug on Windows 10 versions before 21H1.
+		When collapsing, consoles seem to incorrectly push the start of the
+		textRange back one character.
+		Correct this by bringing the start back up to where the end is."""
 		oldInfo = self.copy()
 		super(consoleUIATextInfo, self).collapse(end=end)
 		if not end:
@@ -62,15 +156,65 @@ def collapse(self, end=False):
 				UIAHandler.TextPatternRangeEndpoint_Start
 			)
 
-	def move(self, unit, direction, endPoint=None):
-		oldInfo = None
-		if self.basePosition != textInfos.POSITION_CARET:
-			# Insure we haven't gone beyond the visible text.
-			# UIA adds thousands of blank lines to the end of the console.
-			boundingInfo = self.obj.makeTextInfo(textInfos.POSITION_ALL)
-			oldInfo = self.copy()
-		if unit == textInfos.UNIT_WORD and direction != 0:
+	def compareEndPoints(self, other, which):
+		"""Works around a UIA bug on Windows 10 versions before 21H1.
+		Even when a console textRange's start and end have been moved to the
+		same position, the console incorrectly reports the end as being
+		past the start.
+		Compare to the start (not the end) when collapsed."""
+		selfEndPoint, otherEndPoint = which.split("To")
+		if selfEndPoint == "end" and self._isCollapsed():
+			selfEndPoint = "start"
+		if otherEndPoint == "End" and other._isCollapsed():
+			otherEndPoint = "Start"
+		which = f"{selfEndPoint}To{otherEndPoint}"
+		return super().compareEndPoints(other, which=which)
+
+	def setEndPoint(self, other, which):
+		"""Override of L{textInfos.TextInfo.setEndPoint}.
+		Works around a UIA bug on Windows 10 versions before 21H1 that means we can
+		not trust the "end" endpoint of a collapsed (empty) text range
+		for comparisons.
+		"""
+		selfEndPoint, otherEndPoint = which.split("To")
+		# In this case, there is no need to check if self is collapsed
+		# since the point of this method is to change its text range, modifying the "end" endpoint of a collapsed
+		# text range is fine.
+		if otherEndPoint == "End" and other._isCollapsed():
+			otherEndPoint = "Start"
+		which = f"{selfEndPoint}To{otherEndPoint}"
+		return super().setEndPoint(other, which=which)
+
+	def expand(self, unit):
+		if unit == textInfos.UNIT_WORD:
 			# UIA doesn't implement word movement, so we need to do it manually.
+			lineInfo = self.copy()
+			lineInfo.expand(textInfos.UNIT_LINE)
+			offset = self._getCurrentOffsetInThisLine(lineInfo)
+			start, end = self._getWordOffsetsInThisLine(offset, lineInfo)
+			wordEndPoints = (
+				(offset - start) * -1,
+				end - offset - 1
+			)
+			if wordEndPoints[0]:
+				self._rangeObj.MoveEndpointByUnit(
+					UIAHandler.TextPatternRangeEndpoint_Start,
+					UIAHandler.NVDAUnitsToUIAUnits[textInfos.UNIT_CHARACTER],
+					wordEndPoints[0]
+				)
+			if wordEndPoints[1]:
+				self._rangeObj.MoveEndpointByUnit(
+					UIAHandler.TextPatternRangeEndpoint_End,
+					UIAHandler.NVDAUnitsToUIAUnits[textInfos.UNIT_CHARACTER],
+					wordEndPoints[1]
+				)
+		else:
+			return super(consoleUIATextInfo, self).expand(unit)
+
+	def _move(self, unit, direction, endPoint=None):
+		if unit == textInfos.UNIT_WORD and direction != 0:
+			# On Windows 10 versions before 21H1, UIA doesn't implement word
+			# movement, so we need to do it manually.
 			# Relative to the current line, calculate our offset
 			# and the current word's offsets.
 			lineInfo = self.copy()
@@ -128,98 +272,8 @@ def move(self, unit, direction, endPoint=None):
 			# after moving.
 			# Therefore manually collapse.
 			self.collapse()
-		# Console textRanges have access to the entire console buffer.
-		# However, we want to limit ourselves to onscreen text.
-		# Therefore, if the textInfo was originally visible,
-		# but we are now above or below the visible range,
-		# Restore the original textRange and pretend the move didn't work.
-		if oldInfo:
-			try:
-				if (
-					(
-						self.compareEndPoints(boundingInfo, "startToStart") < 0
-						or self.compareEndPoints(boundingInfo, "startToEnd") >= 0
-					)
-					and not (
-						oldInfo.compareEndPoints(boundingInfo, "startToStart") < 0
-						or oldInfo.compareEndPoints(boundingInfo, "startToEnd") >= 0
-					)
-				):
-					self._rangeObj = oldInfo._rangeObj
-					return 0
-			except (COMError, RuntimeError):
-				pass
 		return res
 
-	def expand(self, unit):
-		if unit == textInfos.UNIT_WORD:
-			# UIA doesn't implement word movement, so we need to do it manually.
-			lineInfo = self.copy()
-			lineInfo.expand(textInfos.UNIT_LINE)
-			offset = self._getCurrentOffsetInThisLine(lineInfo)
-			start, end = self._getWordOffsetsInThisLine(offset, lineInfo)
-			wordEndPoints = (
-				(offset - start) * -1,
-				end - offset - 1
-			)
-			if wordEndPoints[0]:
-				self._rangeObj.MoveEndpointByUnit(
-					UIAHandler.TextPatternRangeEndpoint_Start,
-					UIAHandler.NVDAUnitsToUIAUnits[textInfos.UNIT_CHARACTER],
-					wordEndPoints[0]
-				)
-			if wordEndPoints[1]:
-				self._rangeObj.MoveEndpointByUnit(
-					UIAHandler.TextPatternRangeEndpoint_End,
-					UIAHandler.NVDAUnitsToUIAUnits[textInfos.UNIT_CHARACTER],
-					wordEndPoints[1]
-				)
-		else:
-			return super(consoleUIATextInfo, self).expand(unit)
-
-	def compareEndPoints(self, other, which):
-		"""Works around a UIA bug on Windows 10 1803 and later."""
-		# Even when a console textRange's start and end have been moved to the
-		# same position, the console incorrectly reports the end as being
-		# past the start.
-		# Compare to the start (not the end) when collapsed.
-		selfEndPoint, otherEndPoint = which.split("To")
-		if selfEndPoint == "end" and self._isCollapsed():
-			selfEndPoint = "start"
-		if otherEndPoint == "End" and other._isCollapsed():
-			otherEndPoint = "Start"
-		which = f"{selfEndPoint}To{otherEndPoint}"
-		return super().compareEndPoints(other, which=which)
-
-	def setEndPoint(self, other, which):
-		"""Override of L{textInfos.TextInfo.setEndPoint}.
-		Works around a UIA bug on Windows 10 1803 and later that means we can
-		not trust the "end" endpoint of a collapsed (empty) text range
-		for comparisons.
-		"""
-		selfEndPoint, otherEndPoint = which.split("To")
-		# In this case, there is no need to check if self is collapsed
-		# since the point of this method is to change its text range, modifying the "end" endpoint of a collapsed
-		# text range is fine.
-		if otherEndPoint == "End" and other._isCollapsed():
-			otherEndPoint = "Start"
-		which = f"{selfEndPoint}To{otherEndPoint}"
-		return super().setEndPoint(other, which=which)
-
-	def _isCollapsed(self):
-		"""Works around a UIA bug on Windows 10 1803 and later that means we
-		cannot trust the "end" endpoint of a collapsed (empty) text range
-		for comparisons.
-		Instead we check to see if we can get the first character from the
-		text range. A collapsed range will not have any characters
-		and will return an empty string."""
-		return not bool(self._rangeObj.getText(1))
-
-	def _get_isCollapsed(self):
-		# To decide if the textRange is collapsed,
-		# Check if it has no text.
-		return self._isCollapsed()
-
 	def _getCurrentOffsetInThisLine(self, lineInfo):
 		"""
 		Given a caret textInfo expanded to line, returns the index into the
@@ -258,18 +312,19 @@ def _getWordOffsetsInThisLine(self, offset, lineInfo):
 			min(end.value, max(1, lineTextLen - 2))
 		)
 
-	def __ne__(self, other):
-		"""Support more accurate caret move detection."""
-		return not self == other
+	def _isCollapsed(self):
+		"""Works around a UIA bug on Windows 10 versions before 21H1 that means we
+		cannot trust the "end" endpoint of a collapsed (empty) text range
+		for comparisons.
+		Instead we check to see if we can get the first character from the
+		text range. A collapsed range will not have any characters
+		and will return an empty string."""
+		return not bool(self._rangeObj.getText(1))
 
-	def _get_text(self):
-		# #10036: return a space if the text range is empty.
-		# Consoles don't actually store spaces, the character is merely left blank.
-		res = super(consoleUIATextInfo, self)._get_text()
-		if not res:
-			return ' '
-		else:
-			return res
+	def _get_isCollapsed(self):
+		# To decide if the textRange is collapsed,
+		# Check if it has no text.
+		return self._isCollapsed()
 
 
 class consoleUIAWindow(Window):
@@ -297,12 +352,19 @@ def _get_windowThreadID(self):
 			threadID = super().windowThreadID
 		return threadID
 
+	def _get_is21H1Plus(self):
+		"Returns whether this is a newer version of Windows Console with an improved UIA implementation."
+		# microsoft/terminal#4495: In newer consoles,
+		# IUIAutomationTextRange::getVisibleRanges returns one visible range.
+		return self.UIATextPattern.GetVisibleRanges().length == 1
+
 	def _get_TextInfo(self):
 		"""Overriding _get_TextInfo and thus the TextInfo property
 		on NVDAObjects.UIA.UIA
-		consoleUIATextInfo fixes expand/collapse, implements word movement, and
-		bounds review to the visible text."""
-		return consoleUIATextInfo
+		consoleUIATextInfo bounds review to the visible text.
+		ConsoleUIATextInfoPre21H1 fixes expand/collapse and implements word
+		movement."""
+		return consoleUIATextInfo if self.is21H1Plus else consoleUIATextInfoPre21H1
 
 	def _getTextLines(self):
 		# This override of _getTextLines takes advantage of the fact that
@@ -317,3 +379,8 @@ def findExtraOverlayClasses(obj, clsList):
 		clsList.append(WinConsoleUIA)
 	elif obj.UIAElement.cachedAutomationId == "Console Window":
 		clsList.append(consoleUIAWindow)
+
+
+class WinTerminalUIA(EnhancedTermTypedCharSupport):
+	def _get_TextInfo(self):
+		return consoleUIATextInfo
diff --git a/source/NVDAObjects/behaviors.py b/source/NVDAObjects/behaviors.py
--- a/source/NVDAObjects/behaviors.py
+++ b/source/NVDAObjects/behaviors.py
@@ -3,7 +3,7 @@
 # A part of NonVisual Desktop Access (NVDA)
 # This file is covered by the GNU General Public License.
 # See the file COPYING for more details.
-# Copyright (C) 2006-2019 NV Access Limited, Peter Vágner, Joseph Lee, Bill Dengler
+# Copyright (C) 2006-2020 NV Access Limited, Peter Vágner, Joseph Lee, Bill Dengler
 
 """Mix-in classes which provide common behaviour for particular types of controls across different APIs.
 Behaviors described in this mix-in include providing table navigation commands for certain table rows, terminal input and output support, announcing notifications and suggestion items and so on.
@@ -390,17 +390,11 @@ def _get_caretMovementDetectionUsesEvents(self):
 		return False
 
 
-class KeyboardHandlerBasedTypedCharSupport(Terminal):
-	"""A Terminal object that also provides typed character support for
-	console applications via keyboardHandler events.
-	These events are queued from NVDA's global keyboard hook.
-	Therefore, an event is fired for every single character that is being typed,
-	even when a character is not written to the console (e.g. in read only console applications).
-	This approach is an alternative to monitoring the console output for
-	characters close to the caret, or injecting in-process with NVDAHelper.
-	This class relies on the toUnicodeEx Windows function, and in particular
-	the flag to preserve keyboard state available in Windows 10 1607
-	and later."""
+class EnhancedTermTypedCharSupport(Terminal):
+	"""A Terminal object with keyboard support enhancements for console applications.
+	Notably, it suppresses duplicate typed character announcements and can
+	hold typed characters in a queue and only dispatch once the screen updates.
+	This is useful for suppression of passwords, etc."""
 	#: Whether this object quickly and reliably sends textChange events
 	#: when its contents update.
 	#: Timely and reliable textChange events are required
@@ -422,6 +416,9 @@ def _reportNewLines(self, lines):
 			and len(lines[0].strip()) < max(len(speech.curWordChars) + 1, 3)
 		):
 			return
+		# Clear the typed word buffer for new text lines.
+		speech.clearTypedWordBuffer()
+		self._queuedChars = []
 		super()._reportNewLines(lines)
 
 	def event_typedCharacter(self, ch):
@@ -465,16 +462,6 @@ def script_flush_queuedChars(self, gesture):
 		speech.clearTypedWordBuffer()
 		gesture.send()
 
-	def _calculateNewText(self, newLines, oldLines):
-		hasNewLines = (
-			self._findNonBlankIndices(newLines)
-			!= self._findNonBlankIndices(oldLines)
-		)
-		if hasNewLines:
-			# Clear the typed word buffer for new text lines.
-			speech.clearTypedWordBuffer()
-			self._queuedChars = []
-		return super()._calculateNewText(newLines, oldLines)
 
 	def _dispatchQueue(self):
 		"""Sends queued typedCharacter events through to NVDA."""
@@ -482,12 +469,20 @@ def _dispatchQueue(self):
 			ch = self._queuedChars.pop(0)
 			super().event_typedCharacter(ch)
 
-	def _findNonBlankIndices(self, lines):
-		"""
-		Given a list of strings, returns a list of indices where the strings
-		are not empty.
-		"""
-		return [index for index, line in enumerate(lines) if line]
+
+class KeyboardHandlerBasedTypedCharSupport(EnhancedTermTypedCharSupport):
+	"""An EnhancedTermTypedCharSupport object that provides typed character support for
+	console applications via keyboardHandler events.
+	These events are queued from NVDA's global keyboard hook.
+	Therefore, an event is fired for every single character that is being typed,
+	even when a character is not written to the console (e.g. in read only console applications).
+	This approach is an alternative to monitoring the console output for
+	characters close to the caret, or injecting in-process with NVDAHelper.
+	This class does not implement any specific functionality by itself.
+	Rather, it instructs keyboardHandler to use the toUnicodeEx Windows function, in particular
+	the flag to preserve keyboard state available in Windows 10 1607
+	and later."""
+	pass
 
 
 class CandidateItem(NVDAObject):


Test patch: