Repository: nvaccess/nvda

Issue:
OneCore Voices: Support faster speech rates, greater pitch range, etc.
Compared with NVDA, Narrator is able to speak at faster rates with the OneCore Voices. It can also access a much wider pitch range. In addition, the rate set in Narrator is not affected by the rate set in Windows Settings, whereas NVDA is affected by this. This is because Narrator used an API which was previously private. That API has now been made public, so NVDA will be able to use it.

See the [Options property on the SpeechSynthesizer class](https://docs.microsoft.com/en-us/uwp/api/windows.media.speechsynthesis.speechsynthesizer#Windows_Media_SpeechSynthesis_SpeechSynthesizer_Options_) and the [SpeechSynthesizerOptions class](https://docs.microsoft.com/en-us/uwp/api/windows.media.speechsynthesis.speechsynthesizeroptions). Note that AudioPitch, AudioVolume and SpeakingRate (the properties we want) were only introduced in Windows 10 Insider 10.0.16257.0.

Unfortunately, we can't use these just yet for a few reasons:

1. We need a newer version of the Windows 10 SDK, but versions after 10.0.15063 only work with Visual Studio 2017. We can't yet build with Visual Studio 2017. So, this is blocked by our move to VS 2017, which is in turn blocked by support in SCons.
2. This is still a preview version of the SDK. Even once it's out of preview, it might be a while before it gets installed on AppVeyor.

Implementation notes:

1. Because this will only be available in later builds of Windows 10, we'll need to test for support of this new API at runtime. I know UWP provides a way to do this, but I've never done it myself.
2. Aside from testing for the API, we'll probably still need to keep the code around which adjusts rate, etc. with SSML to work with earlier Windows 10 builds. We might just be able to pass the default rate, volume, etc. to `_OcSsmlConverter` if the new API is supported instead of passing the cached user settings, but I'm not certain. Either way, this part of the code is going to get a bit ugly because we have to support these two cases.


Discussion:
Hi, the API itself became public since build 16225 and later (according to UWP docs). Thanks.

 

From: James Teh [mailto:notifications@github.com] 
Sent: Monday, August 14, 2017 10:31 PM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Subject: [nvaccess/nvda] OneCore Voices: Support faster speech rates, greater pitch range, etc. (#7498)

 

Compared with NVDA, Narrator is able to speak at faster rates with the OneCore Voices. It can also access a much wider pitch range. In addition, the rate set in Narrator is not affected by the rate set in Windows Settings, whereas NVDA is affected by this. This is because Narrator used an API which was previously private. That API has now been made public, so NVDA will be able to use it.

See the Options property on the SpeechSynthesizer class <https://docs.microsoft.com/en-us/uwp/api/windows.media.speechsynthesis.speechsynthesizer#Windows_Media_SpeechSynthesis_SpeechSynthesizer_Options_>  and the SpeechSynthesizerOptions class <https://docs.microsoft.com/en-us/uwp/api/windows.media.speechsynthesis.speechsynthesizeroptions> . Note that AudioPitch, AudioVolume and SpeakingRate (the properties we want) were only introduced in Windows 10 Insider 10.0.16257.0.

Unfortunately, we can't use these just yet for a few reasons:

1.	We need a newer version of the Windows 10 SDK, but versions after 10.0.15063 only work with Visual Studio 2017. We can't yet build with Visual Studio 2017. So, this is blocked by our move to VS 2017, which is in turn blocked by support in SCons.
2.	This is still a preview version of the SDK. Even once it's out of preview, it might be a while before it gets installed on AppVeyor.

Implementation notes:

1.	Because this will only be available in later builds of Windows 10, we'll need to test for support of this new API at runtime. I know UWP provides a way to do this, but I've never done it myself.
2.	Aside from testing for the API, we'll probably still need to keep the code around which adjusts rate, etc. with SSML to work with earlier Windows 10 builds. We might just be able to pass the default rate, volume, etc. to _OcSsmlConverter if the new API is supported instead of passing the cached user settings, but I'm not certain. Either way, this part of the code is going to get a bit ugly because we have to support these two cases.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <https://github.com/nvaccess/nvda/issues/7498> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkBmaCqvOKDtgVYkPhCk96JWEGQmtks5sYS0cgaJpZM4O3NCA> .


There's a bit of confusion as to when this became public. The documentation I linked to (which is supposedly the official UWP documentation) says 16257. However, I've also seen references which suggest earlier. It's safe to say that it's now available in Insider builds and the preview SDK.
Seeing a possible glitch here, in latest next build on latest windows 10 stable, changing the tts settings in either control panel or Narrator Settings does not appear to influence NVDA when speaking through oneCore, where I have observed this happening a few months ago. Has something changed in regards to this?


Narrator Settings would never affect it, only Windows Speech Settings.

Nothing has changed in the driver. Something might have changed in Windows
I guess. Even if that is so, there's nothing we can do to work around it
until this issue is implemented (which is currently blocked by quite a few
things as noted above).

Found it already. There's quite a few places where speech rates can be set and only one of them works, e.g. the ones in narrator settings and control panel's tts settings don't work, only the settings in another part of the windows+i settings dialog seem to take :)
@jcsteh: I belief all blocking cases for this issue are resolved, right? If so, we can safely remove the blocking label and set a priority for this.
As far as I know, yes, this issue is no longer blocked. It's not as simple to fix as it seems, though, since you have to be mindful of the driver's internal speech queue; you can't just set the param.. 


Resolving patch:
diff --git a/source/synthDriverHandler.py b/source/synthDriverHandler.py
--- a/source/synthDriverHandler.py
+++ b/source/synthDriverHandler.py
@@ -3,7 +3,7 @@
 #A part of NonVisual Desktop Access (NVDA)
 #This file is covered by the GNU General Public License.
 #See the file COPYING for more details.
-#Copyright (C) 2006-2019 NV Access Limited, Peter Vágner, Aleksey Sadovoy, Joseph Lee, Arnold Loubriat
+#Copyright (C) 2006-2019 NV Access Limited, Peter Vágner, Aleksey Sadovoy, Joseph Lee, Arnold Loubriat, Leonard de Ruijter
 
 from copy import deepcopy
 import os
@@ -264,6 +264,15 @@ def RateSetting(cls,minStep=1):
 		# Translators: Label for a setting in synth settings ring.
 		displayName=pgettext('synth setting','Rate'))
 	@classmethod
+	def RateBoostSetting(cls):
+		"""Factory function for creating rate boost setting."""
+		# Translators: This is the name of the rate boost voice toggle
+		# which further increases the speaking rate when enabled.
+		return BooleanSynthSetting("rateBoost",_("Rate boos&t"),
+		# Translators: Label for a setting in synth settings ring.
+		displayName=pgettext('synth setting','Rate boost'),
+		availableInSynthSettingsRing=True)
+	@classmethod
 	def VolumeSetting(cls,minStep=1):
 		"""Factory function for creating volume setting."""
 		# Translators: Label for a setting in voice settings dialog.
diff --git a/source/synthDrivers/espeak.py b/source/synthDrivers/espeak.py
--- a/source/synthDrivers/espeak.py
+++ b/source/synthDrivers/espeak.py
@@ -1,7 +1,7 @@
 # -*- coding: UTF-8 -*-
 #synthDrivers/espeak.py
 #A part of NonVisual Desktop Access (NVDA)
-#Copyright (C) 2007-2015 NV Access Limited, Peter Vágner, Aleksey Sadovoy
+#Copyright (C) 2007-2019 NV Access Limited, Peter Vágner, Aleksey Sadovoy, Leonard de Ruijter
 #This file is covered by the GNU General Public License.
 #See the file COPYING for more details.
 
@@ -23,9 +23,7 @@ class SynthDriver(SynthDriver):
 		SynthDriver.VoiceSetting(),
 		SynthDriver.VariantSetting(),
 		SynthDriver.RateSetting(),
-		# Translators: This is the name of the rate boost voice toggle
-		# which further increases the speaking rate when enabled.
-		BooleanSynthSetting("rateBoost",_("Rate boos&t")),
+		SynthDriver.RateBoostSetting(),
 		SynthDriver.PitchSetting(),
 		SynthDriver.InflectionSetting(),
 		SynthDriver.VolumeSetting(),
diff --git a/source/synthDrivers/oneCore.py b/source/synthDrivers/oneCore.py
--- a/source/synthDrivers/oneCore.py
+++ b/source/synthDrivers/oneCore.py
@@ -1,6 +1,6 @@
 #synthDrivers/oneCore.py
 #A part of NonVisual Desktop Access (NVDA)
-#Copyright (C) 2016-2018 Tyler Spivey, NV Access Limited, James Teh
+#Copyright (C) 2016-2019 Tyler Spivey, NV Access Limited, James Teh, Leonard de Ruijter
 #This file is covered by the GNU General Public License.
 #See the file COPYING for more details.
 
@@ -33,24 +33,6 @@
 
 class _OcSsmlConverter(speechXml.SsmlConverter):
 
-	def __init__(self, defaultLanguage, rate, pitch, volume):
-		super(_OcSsmlConverter, self).__init__(defaultLanguage)
-		self._rate = rate
-		self._pitch = pitch
-		self._volume = volume
-
-	def generateBalancerCommands(self, speechSequence):
-		commands = super(_OcSsmlConverter, self).generateBalancerCommands(speechSequence)
-		# The EncloseAllCommand from SSML must be first.
-		yield next(commands)
-		# OneCore doesn't provide a way to set base prosody values.
-		# Therefore, the base values need to be set using SSML.
-		yield self.convertRateCommand(speech.RateCommand(multiplier=1))
-		yield self.convertVolumeCommand(speech.VolumeCommand(multiplier=1))
-		yield self.convertPitchCommand(speech.PitchCommand(multiplier=1))
-		for command in commands:
-			yield command
-
 	def _convertProsody(self, command, attr, default, base):
 		if command.multiplier == 1 and base == default:
 			# Returning to synth default.
@@ -80,20 +62,37 @@ def convertLangChangeCommand(self, command):
 			return None
 		return super(_OcSsmlConverter, self).convertLangChangeCommand(command)
 
+class _OcPreAPI5SsmlConverter(_OcSsmlConverter):
+
+	def __init__(self, defaultLanguage, rate, pitch, volume):
+		super(_OcPreAPI5SsmlConverter, self).__init__(defaultLanguage)
+		self._rate = rate
+		self._pitch = pitch
+		self._volume = volume
+
+	def generateBalancerCommands(self, speechSequence):
+		commands = super(_OcPreAPI5SsmlConverter, self).generateBalancerCommands(speechSequence)
+		# The EncloseAllCommand from SSML must be first.
+		yield next(commands)
+		# OneCore didn't provide a way to set base prosody values before API version 5.
+		# Therefore, the base values need to be set using SSML.
+		yield self.convertRateCommand(speech.RateCommand(multiplier=1))
+		yield self.convertVolumeCommand(speech.VolumeCommand(multiplier=1))
+		yield self.convertPitchCommand(speech.PitchCommand(multiplier=1))
+		for command in commands:
+			yield command
+
 class SynthDriver(SynthDriver):
+
+	MIN_PITCH = 0.0
+	MAX_PITCH = 2.0
+	MIN_RATE = 0.5
+	DEFAULT_MAX_RATE = 1.5
+	BOOSTED_MAX_RATE = 6.0
+
 	name = "oneCore"
 	# Translators: Description for a speech synthesizer.
 	description = _("Windows OneCore voices")
-	supportedSettings = (
-		SynthDriver.VoiceSetting(),
-		SynthDriver.RateSetting(),
-		SynthDriver.PitchSetting(),
-		SynthDriver.VolumeSetting(),
-	)
-	# These are all controlled via SSML, so we only need attributes, not properties.
-	rate = None
-	pitch = None
-	volume = None
 
 	@classmethod
 	def check(cls):
@@ -105,10 +104,38 @@ def check(cls):
 		# For binary copies, only present this as an available synth if this is Windows 10.
 		return winVersion.winVersion.major >= 10
 
+	def _get_supportsProsodyOptions(self):
+		self.supportsProsodyOptions = self._dll.ocSpeech_supportsProsodyOptions()
+		return self.supportsProsodyOptions
+
+	def _get_supportedSettings(self):
+		self.supportedSettings = settings = [
+			SynthDriver.VoiceSetting(),
+			SynthDriver.RateSetting(),
+		]
+		if self.supportsProsodyOptions:
+			settings.append(SynthDriver.RateBoostSetting())
+		settings.extend([
+			SynthDriver.PitchSetting(),
+			SynthDriver.VolumeSetting(),
+		])
+		return settings
+
 	def __init__(self):
 		super(SynthDriver, self).__init__()
 		self._dll = NVDAHelper.getHelperLocalWin10Dll()
 		self._dll.ocSpeech_getCurrentVoiceLanguage.restype = ctypes.c_wchar_p
+		if self.supportsProsodyOptions:
+			self._dll.ocSpeech_getPitch.restype = ctypes.c_double
+			self._dll.ocSpeech_getVolume.restype = ctypes.c_double
+			self._dll.ocSpeech_getRate.restype = ctypes.c_double
+		else:
+			log.debugWarning("Prosody options not supported")
+			# Set initial values for parameters that can't be queried.
+			# This initialises our cache for the value.
+			self._rate = 50
+			self._pitch = 50
+			self._volume = 100
 		self._handle = self._dll.ocSpeech_initialize()
 		self._callbackInst = ocSpeech_Callback(self._callback)
 		self._dll.ocSpeech_setCallback(self._handle, self._callbackInst)
@@ -121,11 +148,6 @@ def __init__(self):
 		self._isProcessing = False
 		# Initialize the voice to a sane default
 		self.voice=self._getDefaultVoice()
-		# Set initial values for parameters that can't be queried.
-		# This initialises our cache for the value.
-		self.rate = 50
-		self.pitch = 50
-		self.volume = 100
 
 	def _maybeInitPlayer(self, wav):
 		"""Initialize audio playback based on the wave header provided by the synthesizer.
@@ -156,12 +178,20 @@ def cancel(self):
 		self._wasCancelled = True
 		log.debug("Cancelling")
 		# There might be more text pending. Throw it away.
-		self._queuedSpeech = []
+		if self.supportsProsodyOptions:
+			# In this case however, we must keep any parameter changes.
+			self._queuedSpeech = [item for item in self._queuedSpeech
+				if not isinstance(item, basestring)]
+		else:
+			self._queuedSpeech = []
 		if self._player:
 			self._player.stop()
 
 	def speak(self, speechSequence):
-		conv = _OcSsmlConverter(self.language, self.rate, self.pitch, self.volume)
+		if self.supportsProsodyOptions:
+			conv = _OcSsmlConverter(self.language)
+		else:
+			conv = _OcPreAPI5SsmlConverter(self.language, self._rate, self._pitch, self._volume)
 		text = conv.convertToXml(speechSequence)
 		# #7495: Calling WaveOutOpen blocks for ~100 ms if called from the callback
 		# when the SSML includes marks.
@@ -177,6 +207,65 @@ def _queueSpeech(self, item):
 		if not self._isProcessing:
 			self._processQueue()
 
+	@classmethod
+	def _percentToParam(self, percent, min, max):
+		"""Overrides SynthDriver._percentToParam to return floating point parameter values.
+		"""
+		return float(percent) / 100 * (max - min) + min
+
+	def _get_pitch(self):
+		if not self.supportsProsodyOptions:
+			return self._pitch
+		rawPitch = self._dll.ocSpeech_getPitch(self._handle)
+		return self._paramToPercent(rawPitch, self.MIN_PITCH, self.MAX_PITCH)
+
+	def _set_pitch(self, pitch):
+		if not self.supportsProsodyOptions:
+			self._pitch = pitch
+			return
+		rawPitch = self._percentToParam(pitch, self.MIN_PITCH, self.MAX_PITCH)
+		self._queuedSpeech.append((self._dll.ocSpeech_setPitch, rawPitch))
+
+	def _get_volume(self):
+		if not self.supportsProsodyOptions:
+			return self._volume
+		rawVolume = self._dll.ocSpeech_getVolume(self._handle)
+		return int(rawVolume * 100)
+
+	def _set_volume(self, volume):
+		if not self.supportsProsodyOptions:
+			self._volume = volume
+			return
+		rawVolume = volume / 100.0
+		self._queuedSpeech.append((self._dll.ocSpeech_setVolume, rawVolume))
+
+	def _get_rate(self):
+		if not self.supportsProsodyOptions:
+			return self._rate
+		rawRate = self._dll.ocSpeech_getRate(self._handle)
+		maxRate = self.BOOSTED_MAX_RATE if self._rateBoost else self.DEFAULT_MAX_RATE
+		return self._paramToPercent(rawRate, self.MIN_RATE, maxRate)
+
+	def _set_rate(self, rate):
+		if not self.supportsProsodyOptions:
+			self._rate = rate
+			return
+		maxRate = self.BOOSTED_MAX_RATE if self._rateBoost else self.DEFAULT_MAX_RATE
+		rawRate = self._percentToParam(rate, self.MIN_RATE, maxRate)
+		self._queuedSpeech.append((self._dll.ocSpeech_setRate, rawRate))
+
+	_rateBoost = False
+
+	def _get_rateBoost(self):
+		return self._rateBoost
+
+	def _set_rateBoost(self, enable):
+		if enable == self._rateBoost:
+			return
+		rate = self.rate
+		self._rateBoost = enable
+		self.rate = rate
+
 	def _processQueue(self):
 		if not self._queuedSpeech:
 			# There are no more queued utterances at this point, so call idle.
@@ -184,8 +273,15 @@ def _processQueue(self):
 			# so by the time this is done, there might be something queued.
 			log.debug("Calling idle on audio player")
 			self._player.idle()
-		if self._queuedSpeech:
+		while self._queuedSpeech:
 			item = self._queuedSpeech.pop(0)
+			if isinstance(item, tuple):
+				# Parameter change.
+				# Note that, if prosody otions aren't supported, this code will never be executed.
+				func, value = item
+				value = ctypes.c_double(value)
+				func(self._handle, value)
+				continue
 			self._wasCancelled = False
 			log.debug("Begin processing speech")
 			self._isProcessing = True


Test patch: