Repository: nvaccess/nvda

Issue:
Native braille display drivers: provide a generic way to handle acknowledgement packets.
### Issue summary
For several braille displays, when sending a packet, the display acknowledges the receipt of a packet. This is a mechanism to make sure that a packet has been delivered to a braille display in a correct way. Several displays have such a feature, including Handy Tech, Eurobraille and Freedom Scientific displays.

### Proposal
Since there are multiple drivers with the need of proper ACK handling, I propose creating a generic system for this.

1. Add three extra properties to the braille.BrailleDisplayDriver class.
    ```
	#: Whether displays for this driver return acknowledgements for sent packets.
	#: L{_handleAck} should be called when an ACK is received.
	#: Note that thread safety is required for the generic implementation to function properly.
	#: If a display is not thread safe, a driver should manually implement ACK processing.
	#: @type: bool
	receivesAckPackets = False
	#: Whether this driver is awaiting an Ack for a connected display.
	#: This is set to C{True} after displaying cells when L{receivesAckPackets} is True,
	#: and set to C{False} by L{_handleAck} or when C{timeout} has elapsed.
	awaitingAck = False
	#: Maximum timeout to use for communication with a device (in seconds).
	#: This can be used for serial connections.
	#: Furthermore, it is used by L{braille._BgThread} to stop waiting for missed acknowledgement packets.
	#: @type: float
	timeout = 0.2
    ```

Note that we can also put awaitingAck on the background thread.
2. in BrailleHandler._writeCells, check for not self.display.awaitingAck, and only queue the executor if we aren't waiting for an ACK and there is no queued write.
3. The executor should set awaitingAck to True whenever a succesful write has been made. Furthermore, it should start a waitable timer that, if not canceled by BrailleDisplayDriver._handleAck, logs the fact that an ACK has been missed, resets awaitingAck to False and queues the executor APC to the background thread.
4. BrailleDisplayDriver._handleAck should set awaitingAck to False and queue the executor APC to the background thread.

Thoughts from @Jcsteh, @michaeldcurran and @bramd are appreciated.


Discussion:
Conceptually, this sounds reasonable to me. A couple of clarifications:

1. You suggest that both the executor and _handleAck should "queue the
executor APC to the background thread". These functions (the executor and
_handleAck) will be running in the background thread, right? So we're
actually queuing to the same thread we're in. There's no problem with that;
I just want to be super clear.
2. I assume awaitingAck should never be touched by braille display drivers
themselves; i.e. it is just used by core for managing state? If so, perhaps
it should be underscore prefixed and the docstring amended to reflect that
this is for internal use by core code only, not the driver.

[**@jcsteh**](https://github.com/jcsteh) commented on [3 Nov 2017, 05:35 CET](https://github.com/nvaccess/nvda/issues/7721#issuecomment-341621086 "2017-11-03T04:35:38Z - Replied by Github Reply Comments"):
> 1. You suggest that both the executor and _handleAck should "queue the
> executor APC to the background thread". These functions (the executor and
> _handleAck) will be running in the background thread, right? So we're
> actually queuing to the same thread we're in. There's no problem with that;
> I just want to be super clear.

That is true, but I was a bit reluctant to call a PAPCFUNC like a regular python function, so that's why I decided to queue here.

> 2. I assume awaitingAck should never be touched by braille display drivers
> themselves; i.e. it is just used by core for managing state? If so, perhaps
> it should be underscore prefixed and the docstring amended to reflect that
> this is for internal use by core code only, not the driver.

That is true, I will do that.

Resolving patch:
diff --git a/source/braille.py b/source/braille.py
--- a/source/braille.py
+++ b/source/braille.py
@@ -1590,7 +1590,7 @@ def _writeCells(self, cells):
 		# we just replace the data.
 		# This means that if multiple writes occur while an earlier write is still in progress,
 		# we skip all but the last.
-		if not alreadyQueued:
+		if not alreadyQueued and not self.display._awaitingAck:
 			# Queue a call to the background thread.
 			_BgThread.queueApc(_BgThread.executor)
 
@@ -1833,6 +1833,7 @@ def start(cls):
 		thread.daemon = True
 		thread.start()
 		cls.handle = ctypes.windll.kernel32.OpenThread(winKernel.THREAD_SET_CONTEXT, False, thread.ident)
+		cls.ackTimerHandle = winKernel.createWaitableTimer()
 
 	@classmethod
 	def queueApc(cls, func):
@@ -1843,6 +1844,10 @@ def stop(cls):
 		if not cls.thread:
 			return
 		cls.exit = True
+		if not ctypes.windll.kernel32.CancelWaitableTimer(cls.ackTimerHandle):
+			raise ctypes.WinError()
+		winKernel.closeHandle(cls.ackTimerHandle)
+		cls.ackTimerHandle = None
 		# Wake up the thread. It will exit when it sees exit is True.
 		cls.queueApc(cls.executor)
 		cls.thread.join()
@@ -1856,6 +1861,9 @@ def executor(param):
 		if _BgThread.exit:
 			# func will see this and exit.
 			return
+		if handler.display._awaitingAck:
+			# Do not write cells when we are awaiting an ACK
+			return
 		with _BgThread.queuedWriteLock:
 			data = _BgThread.queuedWrite
 			_BgThread.queuedWrite = None
@@ -1866,6 +1874,22 @@ def executor(param):
 		except:
 			log.error("Error displaying cells. Disabling display", exc_info=True)
 			handler.setDisplayByName("noBraille", isFallback=True)
+		else:
+			if handler.display.receivesAckPackets:
+				handler.display._awaitingAck = True
+				winKernel.setWaitableTimer(
+					_BgThread.ackTimerHandle,
+					int(handler.display.timeout*2000),
+					0,
+					_BgThread.ackTimeoutResetter
+				)
+
+	@winKernel.PAPCFUNC
+	def ackTimeoutResetter(param):
+		if handler.display.receivesAckPackets and handler.display._awaitingAck:
+			log.debugWarning("Waiting for %s ACK packet timed out"%handler.display.name)
+			handler.display._awaitingAck = False
+			_BgThread.queueApc(_BgThread.executor)
 
 	@classmethod
 	def func(cls):
@@ -1932,6 +1956,22 @@ class BrailleDisplayDriver(baseObject.AutoPropertyObject):
 	#: This is also required to use the L{hwIo} module.
 	#: @type: bool
 	isThreadSafe = False
+	#: Whether displays for this driver return acknowledgements for sent packets.
+	#: L{_handleAck} should be called when an ACK is received.
+	#: Note that thread safety is required for the generic implementation to function properly.
+	#: If a display is not thread safe, a driver should manually implement ACK processing.
+	#: @type: bool
+	receivesAckPackets = False
+	#: Whether this driver is awaiting an Ack for a connected display.
+	#: This is set to C{True} after displaying cells when L{receivesAckPackets} is True,
+	#: and set to C{False} by L{_handleAck} or when C{timeout} has elapsed.
+	#: This is for internal use by NVDA core code only and shouldn't be touched by a driver itself.
+	_awaitingAck = False
+	#: Maximum timeout to use for communication with a device (in seconds).
+	#: This can be used for serial connections.
+	#: Furthermore, it is used by L{_BgThread} to stop waiting for missed acknowledgement packets.
+	#: @type: float
+	timeout = 0.2
 
 	@classmethod
 	def check(cls):
@@ -2009,6 +2049,15 @@ def _getModifierGestures(cls):
 					if emuGesture.isModifier:
 						yield set(gesture.split(":")[1].split("+")), set(emuGesture._keyNamesInDisplayOrder)
 
+	def _handleAck(self):
+		"""Base implementation to handle acknowledgement packets."""
+		if not self.receivesAckPackets:
+			raise NotImplementedError("This display driver does not support ACK packet handling")
+		if not ctypes.windll.kernel32.CancelWaitableTimer(_BgThread.ackTimerHandle):
+			raise ctypes.WinError()
+		self._awaitingAck = False
+		_BgThread.queueApc(_BgThread.executor)
+
 class BrailleDisplayGesture(inputCore.InputGesture):
 	"""A button, wheel or other control pressed on a braille display.
 	Subclasses must provide L{source} and L{id}.
diff --git a/source/brailleDisplayDrivers/handyTech.py b/source/brailleDisplayDrivers/handyTech.py
--- a/source/brailleDisplayDrivers/handyTech.py
+++ b/source/brailleDisplayDrivers/handyTech.py
@@ -20,7 +20,6 @@
 from logHandler import log
 
 
-TIMEOUT = 0.2
 BAUD_RATE = 19200
 PARITY = serial.PARITY_ODD
 
@@ -485,6 +484,8 @@ class BrailleDisplayDriver(braille.BrailleDisplayDriver, ScriptableObject):
 	# Translators: The name of a series of braille displays.
 	description = _("Handy Tech braille displays")
 	isThreadSafe = True
+	receivesAckPackets = True
+	timeout = 0.2
 
 	@classmethod
 	def check(cls):
@@ -543,8 +544,6 @@ def __init__(self, port="auto"):
 		self._ignoreKeyReleases = False
 		self._keysDown = set()
 		self._brailleInput = False
-		self._pendingCells = []
-		self._awaitingACK = False
 		self._hidSerialBuffer = ""
 
 		if port == "auto":
@@ -566,7 +565,7 @@ def __init__(self, port="auto"):
 						self._dev.write(HT_HID_RPT_InCommand + HT_HID_CMD_FlushBuffers)
 				else:
 					self._dev = hwIo.Serial(port, baudrate=BAUD_RATE, parity=PARITY,
-						timeout=TIMEOUT, writeTimeout=TIMEOUT, onReceive=self._onReceive)
+						timeout=self.timeout, writeTimeout=self.timeout, onReceive=self._onReceive)
 			except EnvironmentError:
 				log.debugWarning("", exc_info=True)
 				continue
@@ -574,7 +573,7 @@ def __init__(self, port="auto"):
 			self.sendPacket(HT_PKT_RESET)
 			for _i in xrange(3):
 				# An expected response hasn't arrived yet, so wait for it.
-				self._dev.waitForRead(TIMEOUT)
+				self._dev.waitForRead(self.timeout)
 				if self.numCells and self._model:
 					break
 
@@ -643,13 +642,6 @@ def _handleKeyRelease(self):
 		# being released, so they should be ignored.
 		self._ignoreKeyReleases = True
 
-	def _handleAck(self):
-		if not self._awaitingACK:
-			return
-		self._awaitingACK = False
-		if self._pendingCells:
-			self.display(self._pendingCells)
-
 	# pylint: disable=R0912
 	# Pylint complains about many branches, might be worth refactoring
 	def _onReceive(self, data):
@@ -761,13 +753,8 @@ def _onReceive(self, data):
 
 
 	def display(self, cells):
-		if not self._awaitingACK:
-			# cells will already be padded up to numCells.
-			self._model.display(cells)
-			self._awaitingACK = True
-			self._pendingCells = []
-		else:
-			self._pendingCells = cells
+		# cells will already be padded up to numCells.
+		self._model.display(cells)
 
 	scriptCategory = SCRCAT_BRAILLE
 
diff --git a/source/winKernel.py b/source/winKernel.py
--- a/source/winKernel.py
+++ b/source/winKernel.py
@@ -55,6 +55,58 @@ def CreateFile(fileName,desiredAccess,shareMode,securityAttributes,creationDispo
 	return res
 
 
+def createWaitableTimer(securityAttributes=None, manualReset=False, name=None):
+	"""Wrapper to the kernel32 CreateWaitableTimer function.
+	Consult https://msdn.microsoft.com/en-us/library/windows/desktop/ms682492.aspx for Microsoft's documentation.
+	In contrast with the original function, this wrapper assumes the following defaults.
+	@param securityAttributes: Defaults to C{None};
+		The timer object gets a default security descriptor and the handle cannot be inherited.
+		The ACLs in the default security descriptor for a timer come from the primary or impersonation token of the creator.
+	@type securityAttributes: pointer to L{SECURITY_ATTRIBUTES}
+	@param manualReset: Defaults to C{False} which means the timer is a synchronization timer.
+		If C{True}, the timer is a manual-reset notification timer.
+	@type manualReset: bool
+	@param name: Defaults to C{None}, the timer object is created without a name.
+	@type name: unicode
+	"""
+	res = kernel32.CreateWaitableTimerW(securityAttributes, manualReset, name)
+	if res==0:
+		raise ctypes.WinError()
+	return res
+
+def setWaitableTimer(handle, dueTime, period=0, completionRoutine=None, arg=None, resume=False):
+	"""Wrapper to the kernel32 SETWaitableTimer function.
+	Consult https://msdn.microsoft.com/en-us/library/windows/desktop/ms686289.aspx for Microsoft's documentation.
+	@param handle: A handle to the timer object.
+	@type handle: int
+	@param dueTime: Relative time (in miliseconds).
+		Note that the original function requires relative time to be supplied as a negative nanoseconds value.
+	@type dueTime: int
+	@param period: Defaults to 0, timer is only executed once.
+		Value should be supplied in miliseconds.
+	@type period: int
+	@param completionRoutine: The function to be executed when the timer elapses.
+	@type completionRoutine: L{PAPCFUNC}
+	@param arg: Defaults to C{None}; a pointer to a structure that is passed to the completion routine.
+	@type arg: L{ctypes.c_void_p}
+	@param resume: Defaults to C{False}; the system is not restored.
+		If this parameter is TRUE, restores a system in suspended power conservation mode 
+		when the timer state is set to signaled.
+	@type resume: bool
+	"""
+	res = kernel32.SetWaitableTimer(
+		handle,
+		# due time is in 100 nanosecond intervals, relative time should be negated.
+		byref(LARGE_INTEGER(dueTime*-10000)),
+		period,
+		completionRoutine,
+		arg,
+		resume
+	)
+	if res==0:
+		raise ctypes.WinError()
+	return True
+
 
 def openProcess(*args):
 	return kernel32.OpenProcess(*args)


Test patch: