Repository: nvaccess/nvda

Issue:
Automatic language switching fails with eSpeak
<!-- Please read the text in this edit field before filling it in.
Please thoroughly read NVDA's wiki article on how to fill in this template, including how to provide the required files.
Issues may be closed if the required information is not present.
https://github.com/nvaccess/nvda/blob/master/devDocs/githubIssueTemplateExplanationAndExamples.md
Please also note that the NVDA project has a Citizen and Contributor Code of Conduct which can be found at https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV Access expects that all contributors and other community members read and abide by the rules set out in this document while participating or contributing to this project. This includes creating or commenting on issues and pull requests. 

Each of the questions and sections below start with multiple hash symbols (#). Place your answers and information on the blank line below each question.
-->

### Steps to reproduce:
1. Go to http://forum.audiogames.net
2. Switch to a different language other than English, make sure automatic language dialect switching is on
3. Read through the page.
### Actual behavior:
NVDA does not switch languages
### Expected behavior:
NVDA must switch languages automatically, as it used to do in the version before 2022.1.
### System configuration
#### NVDA installed/portable/running from source:
Installed
#### NVDA version:
2022.1
#### Windows version:
Windows10 21h2
#### Name and version of other software in use when reproducing the issue:
NVDA, Google Chrome
#### Other information about your system:

### Other questions
#### Does the issue still occur after restarting your computer?
Yes
#### Have you tried any other versions of NVDA? If so, please report their behaviors.
Yes, the version before 2022.1, I forgot.
#### If NVDA add-ons are disabled, is your problem still occurring?
Yes
#### Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?
Yes


Discussion:
Hello

In some way, I can reproduce it, at least partly.

Preliminary notes:
* Windows French, NVDA language default (thus French)
* In the list of OneCore voices, I have French voices (Hortense, Julie, Paul) and German voices (Stefan, Katja, Hedda) but no English voice. I do not remember if I had never had one, if I have uninstalled it (what would seem strange but why not for tests), or if this voice has disappeared without my willing (e.g. during an update or an installation of something).

STR:
* Ensure NVDA has automatic language switching enabled (during my tests, automatic dialect switching is disabled however)
* Open the following URL: `data:text/html,<p>Hello</p><p lang="en">world</p>`
* Read the page

Actual result:
* With NVDA 2021.3.4 (portable), the whole page is read in French since I have no English OneCore voice.
* With NVDA 2022.1 (installed), the word "world" is not read and an error is logged.

Here are the logs for both versions of NVDA:
* [OneCoreWithNVDA2021.3.4.log](https://github.com/nvaccess/nvda/files/8755212/OneCoreWithNVDA2021.3.4.log): OK
* [OneCoreWithNVDA2022.1.log](https://github.com/nvaccess/nvda/files/8755214/OneCoreWithNVDA2022.1.log): Error

Cc @michaelDCurran, @feerrenrut, @seanbudd for information on critical issue on 2022.1 release.



@burakyuksek your initial description is lacking of details. So maybe what I have reported is not the same issue. If this is the case, sorry for hijacking your issue. Just let me know and I will open a new dedicated issue.

To clarify your issue, could you precise the following points:
>     2. Switch to a different language other than English, make sure automatic language dialect switching is on
* Which synthesizer and language are you using?
* Which option do you enable in speech settings? Only "Automatic language switching", only "Automatic dialect switching" or both?

> NVDA does not switch languages
And what does it do? Read nothing instead of what should be read? Or read with wrong voice/language the website's content?


It's important to note that not all synthesizers will support the switching of languages/dialects. For example, a synth I use on a daily basis is RHVoice (they're compatible with 2022.1 by the way). Language and dialect switching, at least to my knowledge, does not work correctly at all, even with both those options checked. The synth you'll want to try out for automatic language/dialect detection is eSpeak because it can natively switch languages.

Also, this page is a bad example of reproducing automatic language/dialect detection because of the HTML code I just saw when viewing the page source; It has no markup to say what language the homepage was in, so the synth couldn't switch to it either way.

I'd suggest using a site that you know can change languages (for example, a site in a language outside of English, perhaps), or, alternatively, probably using Google Translate as that also switches language tags for the translated text.

Following up...

After trying the Google Translate site, automatic language detection does appear to work correctly in NVDA (at least with ESpeak). If anyone else cannot reproduce this, I'd be happy to see this issue closed.


@CyrilleB79 - would you be able to open a new issue if this is separate from @burakyuksek's issue.

@burakyuksek - can you confirm what synthesizer you are using?
Please reproduce this issue and provide a [log file](https://github.com/nvaccess/nvda/wiki/LogFilesAndCrashDumps) of the behaviour. Ensure your log level is set to debug in general preferences.
I am using espeak, the primary language I use is Turkish. I toggle both automatic language and dialect switching on. When I enter the site I mentioned, in previous versions, NVDA would automatically switch to English, but now it does not. The log is attached.
[nvda.log](https://github.com/nvaccess/nvda/files/8761809/nvda.log)

I have opened a new specific issue for OneCore issue: #13732.
This was broken in espeak around this commit https://github.com/espeak-ng/espeak-ng/commit/e91d0a58612a01954687d34c7417cf0fbd3cadaa.

An issue should be written on [the espeak repository](https://github.com/espeak-ng/espeak-ng/issues/new).

Resolving patch:
diff --git a/source/languageHandler.py b/source/languageHandler.py
--- a/source/languageHandler.py
+++ b/source/languageHandler.py
@@ -501,3 +501,11 @@ def useImperialMeasurements() -> bool:
 	return buf.value == '1'
 
 
+def stripLocaleFromLangCode(langWithOptionalLocale: str) -> str:
+	"""
+	Get the lang code eg "en" for "en-au" or "chr" for "chr-US-Qaaa-x-west".
+	@param langWithOptionalLocale: may already be language only, or include locale specifier
+	(e.g. "en" or "en-au").
+	@return The language only part, before the first dash.
+	"""
+	return langWithOptionalLocale.split("-")[0]
diff --git a/source/speech/commands.py b/source/speech/commands.py
--- a/source/speech/commands.py
+++ b/source/speech/commands.py
@@ -144,6 +144,15 @@ def __init__(self, lang: Optional[str]):
 	def __repr__(self):
 		return "LangChangeCommand (%r)"%self.lang
 
+	def __eq__(self, __o: object) -> bool:
+		if __o is self:
+			# __o is a reference to the same object.
+			# Check performed first for performance reasons.
+			return True
+		if isinstance(__o, LangChangeCommand):
+			return self.lang == __o.lang
+		return super().__eq__(__o)
+
 class BreakCommand(SynthCommand):
 	"""Insert a break between words.
 	"""
diff --git a/source/synthDrivers/espeak.py b/source/synthDrivers/espeak.py
--- a/source/synthDrivers/espeak.py
+++ b/source/synthDrivers/espeak.py
@@ -1,19 +1,23 @@
 # -*- coding: UTF-8 -*-
-#synthDrivers/espeak.py
-#A part of NonVisual Desktop Access (NVDA)
-#Copyright (C) 2007-2019 NV Access Limited, Peter Vágner, Aleksey Sadovoy, Leonard de Ruijter
-#This file is covered by the GNU General Public License.
-#See the file COPYING for more details.
+# A part of NonVisual Desktop Access (NVDA)
+# Copyright (C) 2007-2022 NV Access Limited, Peter Vágner, Aleksey Sadovoy, Leonard de Ruijter
+# This file is covered by the GNU General Public License.
+# See the file COPYING for more details.
 
 import os
 from collections import OrderedDict
+from typing import Optional, Set
+
 from . import _espeak
-import threading
-import languageHandler
+from languageHandler import (
+	getLanguage,
+	stripLocaleFromLangCode,
+)
 from synthDriverHandler import SynthDriver, VoiceInfo, synthIndexReached, synthDoneSpeaking
 import speech
 from logHandler import log
 
+from speech.types import SpeechSequence
 from speech.commands import (
 	IndexCommand,
 	CharacterModeCommand,
@@ -50,6 +54,153 @@ class SynthDriver(SynthDriver):
 	}
 	supportedNotifications = {synthIndexReached, synthDoneSpeaking}
 
+	# A mapping of commonly used language codes to eSpeak languages.
+	# Introduced due to eSpeak issue: https://github.com/espeak-ng/espeak-ng/issues/1200
+	# These are used when eSpeak doesn't support a given language code
+	# but a default alias is appropriate.
+	_defaultLangToLocale = {
+		# Languages without locale that aren't supported in eSpeak 7e5457f91e10,
+		# with a language with locale that is supported.
+		# Found via:
+		# set(stripLocaleFromLangCode(lang) for lang in self.availableLanguages).difference(self.availableLanguages)
+		"en": "en-gb",
+		"chr": "chr-US-Qaaa-x-west",
+		"fr": "fr-fr",
+	}
+
+	availableLanguages: Set[Optional[str]]
+	"""
+	For eSpeak commit 7e5457f91e10, this is equivalent to:
+	{
+		'ia',
+		'ru',
+		'cy',
+		'ms',
+		'af',
+		'fi',
+		'fr-fr',
+		'nog',
+		'gu',
+		'hu',
+		'eu',
+		'om',
+		'en-029',
+		'de',
+		'es',
+		'kk',
+		'an',
+		'nci',
+		'uk',
+		'vi-vn-x-south',
+		'grc',
+		'it',
+		'vi-vn-x-central',
+		'bg',
+		'piqd',
+		'ug',
+		'ar',
+		'da',
+		'mi',
+		'mr',
+		'pt-br',
+		'fr-ch',
+		'py',
+		'uz',
+		'en-gb',
+		'sw',
+		'as',
+		'shn',
+		'vi',
+		'nl',
+		'bs',
+		'ga',
+		'pap',
+		'sv',
+		'kn',
+		'gn',
+		'th',
+		'tr',
+		'pa',
+		'mt',
+		'chr-US-Qaaa-x-west',
+		'eo',
+		'kok',
+		'ky',
+		'lfn',
+		'is',
+		'pt',
+		'en-gb-x-gbcwmd',
+		'en-gb-x-rp',
+		'ht',
+		'bpy',
+		'fr-be',
+		'nb',
+		'lt',
+		'ja',
+		'te',
+		'tn',
+		'es-419',
+		'gd',
+		'sjn',
+		'he',
+		'hyw',
+		'et',
+		'ro',
+		'ru-lv',
+		'sq',
+		'quc',
+		'am',
+		'hr',
+		'qya',
+		'ka',
+		'el',
+		'tt',
+		'or',
+		'pl',
+		'qu',
+		'ba',
+		'ta',
+		'cmn',
+		'io',
+		'en-us',
+		'ur',
+		'hi',
+		'en-gb-scotland',
+		'fa',
+		'kl',
+		'tk',
+		'ku',
+		'si',
+		'cv',
+		'ca',
+		'qdb',
+		'hak',
+		'fa-latn',
+		'lv',
+		'en-gb-x-gbclan',
+		'ltg',
+		'ne',
+		'sl',
+		'az',
+		'yue',
+		'sk',
+		'hy',
+		'my',
+		'ko',
+		'mk',
+		'smj',
+		'ml',
+		'cmn-latn-pinyin',
+		'id',
+		'la',
+		'sr',
+		'bn',
+		'sd',
+		'cs',
+		'jbo',
+		'haw'
+	}
+	"""
 	@classmethod
 	def check(cls):
 		return True
@@ -57,7 +208,7 @@ def check(cls):
 	def __init__(self):
 		_espeak.initialize(self._onIndexReached)
 		log.info("Using eSpeak NG version %s" % _espeak.info())
-		lang=languageHandler.getLanguage()
+		lang = getLanguage()
 		_espeak.setVoiceByLanguage(lang)
 		self._language=lang
 		self._variantDict=_espeak.getVariantDict()
@@ -90,8 +241,76 @@ def _processText(self, text):
 			0x5B: u" [", # [: [[ indicates phonemes
 		})
 
-	def speak(self,speechSequence):
-		defaultLanguage=self._language
+	def _normalizeLangCommand(self, command: LangChangeCommand) -> LangChangeCommand:
+		"""
+		Checks if a LangChangeCommand language is compatible with eSpeak.
+		If not, find a default mapping occurs in L{_defaultLangToLocale}.
+		Otherwise, finds a language of a different dialect exists (e.g. ru-ru to ru).
+		Returns an eSpeak compatible LangChangeCommand.
+		"""
+		lowerCaseAvailableLangs = set(lang.lower() for lang in self.availableLanguages)
+		# Use default language if no command.lang is supplied
+		langWithLocale = command.lang if command.lang else self._language
+		langWithLocale = langWithLocale.lower().replace('_', '-')
+
+		langWithoutLocale: Optional[str] = stripLocaleFromLangCode(langWithLocale)
+
+		# Check for any language where the language code matches, regardless of dialect: e.g. ru-ru to ru
+		matchingLangs = filter(
+			lambda lang: stripLocaleFromLangCode(lang) == langWithoutLocale,
+			lowerCaseAvailableLangs
+		)
+		anyLocaleMatchingLang = next(matchingLangs, None)
+		
+		# Check from a list of known default mapping locales: e.g. en to en-gb
+		# Created due to eSpeak issue: https://github.com/espeak-ng/espeak-ng/issues/1200
+		knownDefaultLang = self._defaultLangToLocale.get(langWithoutLocale, None)
+		if knownDefaultLang is not None and knownDefaultLang not in self.availableLanguages:
+			# This means eSpeak has changed and we need to update the mapping
+			log.error(f"Default mapping unknown to eSpeak {knownDefaultLang} not in {self.availableLanguages}")
+			knownDefaultLang = None
+
+		if langWithLocale in lowerCaseAvailableLangs:
+			eSpeakLang = langWithLocale
+		elif knownDefaultLang is not None:
+			eSpeakLang = knownDefaultLang
+		elif langWithoutLocale in lowerCaseAvailableLangs:
+			eSpeakLang = langWithoutLocale
+		elif anyLocaleMatchingLang is not None:
+			eSpeakLang = anyLocaleMatchingLang
+		else:
+			log.debugWarning(f"Unable to find an eSpeak language for '{langWithLocale}'")
+			eSpeakLang = None
+		return LangChangeCommand(eSpeakLang)
+
+	def _handleLangChangeCommand(
+			self,
+			langChangeCommand: LangChangeCommand,
+			langChanged: bool,
+	) -> str:
+		"""Get language xml tags needed to handle a lang change command.
+			- if a language change has already been handled for this speech,
+			close the open voice tag.
+			- if the language is supported by eSpeak, switch to that language.
+			- otherwise, switch to the default synthesizer language.
+		"""
+		langChangeCommand = self._normalizeLangCommand(langChangeCommand)
+		voiceChangeXML = ""
+		if langChanged:
+			# Close existing voice tag
+			voiceChangeXML += "</voice>"
+		if langChangeCommand.lang is not None:
+			# Open new voice tag using eSpeak compatible language
+			voiceChangeXML += f'<voice xml:lang="{langChangeCommand.lang}">'
+		else:
+			# Open new voice tag using default voice
+			voiceChangeXML += "<voice>"
+		return voiceChangeXML
+
+	# C901 'speak' is too complex
+	# Note: when working on speak, look for opportunities to simplify
+	# and move logic out into smaller helper functions.
+	def speak(self, speechSequence: SpeechSequence):  # noqa: C901
 		textList=[]
 		langChanged=False
 		prosody={}
@@ -106,10 +325,9 @@ def speak(self,speechSequence):
 			elif isinstance(item, CharacterModeCommand):
 				textList.append("<say-as interpret-as=\"characters\">" if item.state else "</say-as>")
 			elif isinstance(item, LangChangeCommand):
-				if langChanged:
-					textList.append("</voice>")
-				textList.append("<voice xml:lang=\"%s\">"%(item.lang if item.lang else defaultLanguage).replace('_','-'))
-				langChanged=True
+				langChangeXML = self._handleLangChangeCommand(item, langChanged)
+				textList.append(langChangeXML)
+				langChanged = True
 			elif isinstance(item, BreakCommand):
 				textList.append('<break time="%dms" />' % item.time)
 			elif type(item) in self.PROSODY_ATTRS:


Test patch:
diff --git a/tests/unit/test_synthDrivers/__init__.py b/tests/unit/test_synthDrivers/__init__.py
new file mode 100644
diff --git a/tests/unit/test_synthDrivers/test_espeak.py b/tests/unit/test_synthDrivers/test_espeak.py
new file mode 100644
--- /dev/null
+++ b/tests/unit/test_synthDrivers/test_espeak.py
@@ -0,0 +1,141 @@
+# A part of NonVisual Desktop Access (NVDA)
+# This file is covered by the GNU General Public License.
+# See the file COPYING for more details.
+# Copyright (C) 2022 NV Access Limited.
+
+"""Unit tests for the eSpeak synth driver submodule.
+"""
+
+import logging
+import unittest
+
+from languageHandler import stripLocaleFromLangCode
+import logHandler
+from speech.commands import LangChangeCommand
+from synthDrivers.espeak import SynthDriver
+from synthDrivers._espeak import _setVoiceByLanguage
+import nvwave
+
+
+class FakeESpeakSynthDriver:
+	_language = "default"
+	_defaultLangToLocale = {"default": "en-gb"}
+	availableLanguages = {"fr", "fr-fr", "en-gb", "ta-ta"}
+
+
+class TestSynthDriver_Logic(unittest.TestCase):
+	"""Testing of internal logic for the eSpeak driver."""
+	def test_normalizeLangCommand(self):
+		"""Test cases for determining a supported eSpeak language from a LangChangeCommand."""
+		self.assertEqual(
+			LangChangeCommand("en-gb"),
+			SynthDriver._normalizeLangCommand(FakeESpeakSynthDriver, LangChangeCommand(None)),
+			msg="Default language used if language code not provided"
+		)
+		self.assertEqual(
+			LangChangeCommand("fr-fr"),
+			SynthDriver._normalizeLangCommand(FakeESpeakSynthDriver, LangChangeCommand("fr_FR")),
+			msg="Language with locale used when available"
+		)
+		self.assertEqual(
+			LangChangeCommand("en-gb"),
+			SynthDriver._normalizeLangCommand(FakeESpeakSynthDriver, LangChangeCommand("default")),
+			msg="Default eSpeak language mappings used"
+		)
+		self.assertEqual(
+			LangChangeCommand("fr"),
+			SynthDriver._normalizeLangCommand(FakeESpeakSynthDriver, LangChangeCommand("fr_FAKE")),
+			msg="Language without locale used when available"
+		)
+		self.assertEqual(
+			LangChangeCommand("ta-ta"),
+			SynthDriver._normalizeLangCommand(FakeESpeakSynthDriver, LangChangeCommand("ta-gb")),
+			msg="Language with any locale used when available"
+		)
+		with self.assertLogs(logHandler.log, level=logging.DEBUG) as logContext:
+			self.assertEqual(
+				LangChangeCommand(None),
+				SynthDriver._normalizeLangCommand(FakeESpeakSynthDriver, LangChangeCommand("fake")),
+				msg="No matching available language returns None"
+			)
+		self.assertIn(
+			"Unable to find an eSpeak language for 'fake'",
+			logContext.output[0]
+		)
+
+
+class TestSynthDriver_Integration(unittest.TestCase):
+	"""Perform integration testing for the eSpeak synth driver."""
+
+	def setUp(self) -> None:
+		# Prevent NVWave from causing an exception
+		# when running unit tests in AppVeyor
+		self._nvwaveOpenOld = nvwave.WavePlayer.open
+		nvwave.WavePlayer.open = lambda self: None
+		self._driver = SynthDriver()
+	
+	def tearDown(self) -> None:
+		self._driver.terminate()
+		nvwave.WavePlayer.open = self._nvwaveOpenOld
+
+	def test_defaultMappingAvailableLanguage(self):
+		"""Confirms language codes remapped by default are supported by eSpeak and are comprehensive."""
+		eSpeakAvailableLangs = self._driver.availableLanguages
+		mappedDefaultLanguages = set(self._driver._defaultLangToLocale.values())
+		unexpectedUnsupportedDefaultLanguages = mappedDefaultLanguages.difference(eSpeakAvailableLangs)
+		self.assertEqual(
+			set(),
+			unexpectedUnsupportedDefaultLanguages,
+			msg=(
+				"Languages mapped by default are no longer supported by eSpeak: "
+				f"{unexpectedUnsupportedDefaultLanguages}"
+			)
+		)
+
+		expectedUnsupportedMappedLanguages = set(self._driver._defaultLangToLocale.keys())
+		unexpectedSupportedMappedLanguages = expectedUnsupportedMappedLanguages.intersection(eSpeakAvailableLangs)
+		self.assertEqual(
+			set(),
+			unexpectedSupportedMappedLanguages,
+			msg=(
+				"Languages mapped to eSpeak defaults are now supported by eSpeak: "
+				f"{unexpectedSupportedMappedLanguages}"
+			)
+		)
+
+		supportedLanguagesWithLocaleStripped = set(stripLocaleFromLangCode(lang) for lang in eSpeakAvailableLangs)
+		unsupportedLanguagesWithoutLocale = supportedLanguagesWithLocaleStripped.difference(eSpeakAvailableLangs)
+		self.assertEqual(
+			expectedUnsupportedMappedLanguages,
+			unsupportedLanguagesWithoutLocale,
+			msg=(
+				"eSpeak has a language with locale supported"
+				"but the language without locale is unsupported."
+				"Update _defaultLangToLocale to include a new mapping."
+			)
+		)
+
+	def test_availableLanguagesWithoutLocale(self):
+		"""
+		Confirms that eSpeak can manually be switched to all of its supported languages with the locale removed.
+		This doesn't test automatic language switching.
+		"""
+		availableLangsWithoutLocale = set(stripLocaleFromLangCode(lang) for lang in self._driver.availableLanguages)
+		for langWithoutLocale in availableLangsWithoutLocale:
+			_setVoiceByLanguage(langWithoutLocale)
+			self.assertEqual(
+				langWithoutLocale,
+				self._driver.voice.split("\\")[-1],  # Language code is the last item, e.g. (gmw\en, roa\fr-CH)
+				msg="Language without locale not supported by eSpeak"
+			)
+
+	def test_fallbackToBritishEnglish(self):
+		"""
+		Confirms that eSpeak falls back to en-gb when manually switching to an unknown language
+		"""
+		_setVoiceByLanguage("fake-lang")
+		self.assertEqual(
+			"gmw\\en",
+			self._driver.voice,
+			msg="Language without locale not supported by eSpeak"
+		)