Repository: nvaccess/nvda

Issue:
The speaking of the typed characters does not work in Universal apps in the latest build of the Windows Insider Preview
I've noticed that, with the latest build of the Windows Insider preview, NVDA does not speak the typed characters in the so called universal apps. That includes the search field in the Start menu. If I use the arrow keys to review the typed characters - it works as expected. If I use the DELETE or BACKSPACE keys to delete characters, it also works as expected. The only issue is with the characters not being spoken when entering them from the keyboard (keyboard echo). Both Windows and NVDA are the latest test builds.

Steps to reproduce it:
1. Update the Windows Insider Preview to its latest build via Windows Update.
2. Update the installed copy of NVDA to its latest "next" snapshot.
3. Open one of the so called "universal" apps. The Microsoft Edge browser will do it.
4. Make sure that the option "Speak typed characters" of NVDA is turned on.
5. Find an edit field in that app and start typing text in it.

Expected results:
NVDA should speak the typed characters.

Actual results:
NVDA stays silent.

Test environment:
- Operating system: Windows 10 Pro Insider Preview (build 14352.1002)), 32-bit, in English with all locale settings set to "Bulgarian", in a virtual machine in VMWare Player.
- Input languages: "Bulgarian" and "English US".
- VMware® Workstation 12 Player version: 12.1.1 build-3770994.
- NVDA version: next-13343,73ee6b2a.
- Processor: Intel Core i5-2320 at 3.00GHz.
- RAM Memory: 3.00GB.



Discussion:
Am I the only one that can reproduce this? Does it work OK on your machines?

No; I confirmed this, but neglected to comment here. The issue is that 
Microsoft have completely blocked NVDA (and all other assistive 
technology software) from injecting into UWP apps. Previously, only 
installed software with uiAccess privileges (including NVDA) could 
inject, but now, even this has been blocked. This has been done in the 
name of security, but the problem is that Microsoft have not provided a 
suitable alternative which enables AT to access typed characters in a 
way which works properly for all languages. We've raised this with 
Microsoft, but they've yet to provide a useful solution or response.


Resolving patch:
diff --git a/source/keyboardHandler.py b/source/keyboardHandler.py
--- a/source/keyboardHandler.py
+++ b/source/keyboardHandler.py
@@ -7,11 +7,15 @@
 
 """Keyboard support"""
 
+import ctypes
+import sys
 import time
 import re
 import wx
+import winVersion
 import winUser
 import vkCodes
+import eventHandler
 import speech
 import ui
 from keyLabels import localizedKeyLabels
@@ -73,6 +77,7 @@ def isNVDAModifierKey(vkCode,extended):
 def internal_keyDownEvent(vkCode,scanCode,extended,injected):
 	"""Event called by winInputHook when it receives a keyDown.
 	"""
+	gestureExecuted=False
 	try:
 		global lastNVDAModifier, lastNVDAModifierReleaseTime, bypassNVDAModifier, passKeyThroughCount, lastPassThroughKeyDown, currentModifiers, keyCounter, stickyNVDAModifier, stickyNVDAModifierLocked
 		# Injected keys should be ignored in some cases.
@@ -146,6 +151,7 @@ def internal_keyDownEvent(vkCode,scanCode,extended,injected):
 
 		try:
 			inputCore.manager.executeGesture(gesture)
+			gestureExecuted=True
 			trappedKeys.add(keyCode)
 			if canModifiersPerformAction(gesture.generalizedModifiers):
 				# #3472: These modifiers can perform an action if pressed alone
@@ -162,6 +168,22 @@ def internal_keyDownEvent(vkCode,scanCode,extended,injected):
 				return False
 	except:
 		log.error("internal_keyDownEvent", exc_info=True)
+	finally:
+		# #6017: handle typed characters for UWP apps in Win10 RS2 and above where we can't detect typed characters in-process 
+		# This code must be in the 'finally' block as code above returns in several places yet we still want to execute this particular code.
+		focus=api.getFocusObject()
+		if winVersion.winVersion.build>=14986 and not gestureExecuted and not isNVDAModifierKey(vkCode,extended) and not vkCode in KeyboardInputGesture.NORMAL_MODIFIER_KEYS and focus.windowClassName.startswith('Windows.UI.Core'):
+			keyStates=(ctypes.c_byte*256)()
+			for k in xrange(256):
+				keyStates[k]=ctypes.windll.user32.GetKeyState(k)
+			charBuf=ctypes.create_unicode_buffer(5)
+			hkl=ctypes.windll.user32.GetKeyboardLayout(focus.windowThreadID)
+			# In previous Windows builds, calling ToUnicodeEx would destroy keyboard buffer state and therefore cause the app to not produce the right WM_CHAR message.
+			# However, ToUnicodeEx now can take a new flag of 0x4, which stops it from destroying keyboard state, thus allowing us to safely call it here.
+			res=ctypes.windll.user32.ToUnicodeEx(vkCode,scanCode,keyStates,charBuf,len(charBuf),0x4,hkl)
+			if res>0:
+				for ch in charBuf[:res]: 
+					eventHandler.queueEvent("typedCharacter",focus,ch=ch)
 	return True
 
 def internal_keyUpEvent(vkCode,scanCode,extended,injected):


Test patch: