Repository: nvaccess/nvda

Issue:
Tab characters should be considered blank
### Steps to reproduce:

Read the two lines between "start" and "end":

```
start
    
		
end
```

(Note: The first line after start has 4 spaces and the next line has 2 tabs)

### Actual behavior:

They are read as:

```
start
blank
*nothing is read*
end
```

Further, if indentation reporting is on, then they are read as:

```
start
4 space blank
2 tab blank
end
```

### Expected behavior:

I would expect the line with the 2 tabs should be read as "blank" when indentation reporting is turned off, since it is a blank line when indentation reporting is turned on. Nothing being read at all causes confusion when reading code, since it can seem as if NVDA has stopped responding.

### System configuration
#### NVDA installed/portable/running from source: installed

#### NVDA version: 2021.3.3

### Cause of problem

In `speech.py`, whether text is considered blank is determine by this code:

```
#: If a chunk of text contains only these characters, it will be considered blank.
BLANK_CHUNK_CHARS = frozenset((" ", "\n", "\r", "\0", u"\xa0"))
def isBlank(text):
	"""Determine whether text should be reported as blank.
	@param text: The text in question.
	@type text: str
	@return: C{True} if the text is blank, C{False} if not.
	@rtype: bool
	"""
	return not text or set(text) <= BLANK_CHUNK_CHARS
```

The tab character is missing from `BLANK_CHUNK_CHARS `, which is why  it isn't considered blank. The solution would be to also include it in the set.

Not sure if there is something I am missing and this is actually the intended behavior, but the inconsistency between the line being blank when indentation reporting is on/off makes me think it is a bug.

If it is agreed this is a bug, I would be happy to provide the PR as my first contribution to the NVDA project.


Discussion:
Accidently submitted before writing anything, editing it now...
Edited it now with full context
I realized one reason why tab is probably not in `BLANK_CHUNK_CHARS`. When symbol level is set to All, tab is read, however the characters in `Blank_chunk_chars` are never read at any symbol level.

I experimented with adding tab to `BLANK_CHUNK_CHARS` and it does cause the line with tabs to be read as "blank", however when symbol level is All, the line is still read as "blank", even though the expected behavior would be to read the tabs.

Because of this, I no longer believe the correct solution is to add tab to `BLANK_CHUNK_CHARS`, but I also believe the behavior I described in the first comment needs to be fixed, since I don't think NVDA should never just speak nothing when all the characters in a text are not in the current symbol level

I think the correct solution could be to make it so, when an empty speech sequence is produced because none of the characters are read in the current symbol level, then the empty speech sequence should be changed to "blank"
Experimenting with the idea that if text is empty after applying dictionary substitutions and symbol level, then it should be read as "blank", I achieved the desired behavior when I altered `speech.speech.processText` like this:

```
 def processText(locale,text,symbolLevel):
 	text = speechDictHandler.processText(text)
 	text = characterProcessing.processSpeechSymbols(locale, text, symbolLevel)
 	text = RE_CONVERT_WHITESPACE.sub(u" ", text)
-	return text.strip()
+	text = text.strip()
+	return text if text else _("blank")
```

With that experimental change, a line with just tabs is read as "blank" when symbol level is not All and as "tab tab" when symbol level is All. Also aline with other punctuation that is only read at a certain symbol level is also read as "blank" if symbol level is switched to something lower.

There is one remaining somewhat unexpected behavior. If indentation reporting is on, the two tab line is read as "2 tab blank" or "*beep* blank", whereas when indentation reporting is off, it is read as "tab tab". Perhaps although slightly unintuitive, it might actually be the prefferential behavior.

Either way, I think the changes I proposed with a little cleanup would improve over the current behavior. Please let me know what you think and I can then create a PR
Hi @SamKacer
Since you have already a precise idea on how to implement a fix for this issue, would you mind submitting a pull request for it?

cc @SamKacer 
@CyrilleB79 yes, glad to help out and provide the PR. When I next have the time, I will put it together

Resolving patch:
diff --git a/source/speech/__init__.py b/source/speech/__init__.py
--- a/source/speech/__init__.py
+++ b/source/speech/__init__.py
@@ -4,6 +4,7 @@
 # Copyright (C) 2006-2021 NV Access Limited, Peter Vágner, Aleksey Sadovoy, Babbage B.V., Bill Dengler,
 # Julien Cochuyt
 
+import functools
 from .speech import (
 	_extendSpeechSequence_addMathForTextInfo,
 	_getSpellingSpeechAddCharMode,
@@ -155,7 +156,9 @@ def initialize():
 	synthDriverHandler.setSynth(config.conf["speech"]["synth"])
 	speechInitialize()
 	sayAllInitialize(
-		speak,
+		# override speak func to always have suppressBlanks=True,
+		# since blanks should never be spoken in say all
+		functools.partial(speak, suppressBlanks=True),
 		speakObject,
 		getTextInfoSpeech,
 		SpeakTextInfoState,
diff --git a/source/speech/speech.py b/source/speech/speech.py
--- a/source/speech/speech.py
+++ b/source/speech/speech.py
@@ -238,7 +238,7 @@ def speakSpelling(
 		locale=locale,
 		useCharacterDescriptions=useCharacterDescriptions
 	))
-	speak(seq, priority=priority)
+	speak(seq, priority=priority, suppressBlanks=True)
 
 
 def _getSpellingSpeechAddCharMode(
@@ -878,12 +878,14 @@ def getIndentationSpeech(indentation: str, formatConfig: Dict[str, bool]) -> Spe
 def speak(  # noqa: C901
 		speechSequence: SpeechSequence,
 		symbolLevel: Optional[int] = None,
-		priority: Spri = Spri.NORMAL
+		priority: Spri = Spri.NORMAL,
+		suppressBlanks: bool = False
 ):
 	"""Speaks a sequence of text and speech commands
 	@param speechSequence: the sequence of text and L{SpeechCommand} objects to speak
 	@param symbolLevel: The symbol verbosity level; C{None} (default) to use the user's configuration.
 	@param priority: The speech priority.
+	@param suppressBlanks: Whether to not append "blank" to the speech even if considered blank
 	"""
 	logBadSequenceTypes(speechSequence)
 	# in case priority was explicitly passed in as None, set to default.
@@ -943,9 +945,22 @@ def speak(  # noqa: C901
 		if autoLanguageSwitching and isinstance(item,LangChangeCommand):
 			curLanguage=item.lang
 		if isinstance(item,str):
-			speechSequence[index]=processText(curLanguage,item,symbolLevel)
-			if not inCharacterMode:
-				speechSequence[index]+=CHUNK_SEPARATOR
+			text = processText(curLanguage, item, symbolLevel)
+			if not inCharacterMode and text:
+				text += CHUNK_SEPARATOR
+			speechSequence[index] = text
+	# speech sequence should be considered blank if:
+	# 1. it contains strings
+	# 2. all strings are blank after processing
+	if (
+		not suppressBlanks
+		and any(isinstance(i, str) for i in speechSequence)
+		# for checking if blank, just check if empty instead of isBlank(),
+		# since whitespace has been stripped during processing
+		and all(not s for s in speechSequence if isinstance(s, str))
+	):
+		# Translators: This is spoken when the speech sequence is considered blank.
+		speechSequence.append(_("blank"))
 	_manager.speak(speechSequence, priority)
 
 
@@ -964,7 +979,7 @@ def speakPreselectedText(
 	"""
 	seq = getPreselectedTextSpeech(text)
 	if seq:
-		speak(seq, symbolLevel=None, priority=priority)
+		speak(seq, symbolLevel=None, priority=priority, suppressBlanks=True)
 
 
 def getPreselectedTextSpeech(
@@ -1013,7 +1028,7 @@ def speakSelectionMessage(
 ):
 	seq = _getSelectionMessageSpeech(message, text)
 	if seq:
-		speak(seq, symbolLevel=None, priority=priority)
+		speak(seq, symbolLevel=None, priority=priority, suppressBlanks=True)
 
 
 def _getSelectionMessageSpeech(
@@ -1230,7 +1245,7 @@ def speakTextInfo(
 
 	speechGen = GeneratorWithReturn(speechGen)
 	for seq in speechGen:
-		speak(seq, priority=priority)
+		speak(seq, priority=priority, suppressBlanks=suppressBlanks)
 	return speechGen.returnValue
 
 


Test patch:
diff --git a/tests/system/robot/chromeTests.py b/tests/system/robot/chromeTests.py
--- a/tests/system/robot/chromeTests.py
+++ b/tests/system/robot/chromeTests.py
@@ -205,21 +205,16 @@ def test_mark_aria_details_role():
 		# The role "ROLE_LIST_ITEM" is used instead
 		"has details",
 		"doc-endnote,",
-		"",  # space between spans
 		"has foot note",
 		"doc-footnote,",
-		"",  # space between spans
 		"has comment",
 		"comment,",
-		"",  # space between spans
 		# the role definition is unsupported as an IA2 role
 		# The role "ROLE_PARAGRAPH" is used instead
 		"has details",
 		"definition,",
-		"",  # space between spans
 		"has details",
 		"definition,",
-		"",  # space between spans
 		# The role "form" is deliberately unsupported
 		"has details",
 		"form",
@@ -862,7 +857,7 @@ def test_pr11606():
 	actualSpeech = _chrome.getSpeechAfterKey("tab")
 	_asserts.strings_match(
 		actualSpeech,
-		"section  multi line  editable  list  bullet  link  A    link  B"
+		"section  multi line  editable  list  bullet  link  A  link  B"
 	)
 	# move past the end of the first link.
 	# This should not be affected due to pr #11606.
@@ -886,7 +881,7 @@ def test_pr11606():
 	actualSpeech = _chrome.getSpeechAfterKey("NVDA+upArrow")
 	_asserts.strings_match(
 		actualSpeech,
-		"bullet  link  A    link  B"
+		"bullet  link  A  link  B"
 	)
 
 
@@ -1214,14 +1209,14 @@ def test_ariaRoleDescription_inline_contentEditable():
 	actualSpeech = _chrome.getSpeechAfterKey("downArrow")
 	_asserts.strings_match(
 		actualSpeech,
-		"Start  drawing  Our logo    End"
+		"Start  drawing  Our logo  End"
 	)
 	# When reading the line by word,
 	# Both entering and exiting the custom role should be reported.
 	actualSpeech = _chrome.getSpeechAfterKey("control+rightArrow")
 	_asserts.strings_match(
 		actualSpeech,
-		"drawing  Our logo    out of drawing"
+		"drawing  Our logo  out of drawing"
 	)
 	actualSpeech = _chrome.getSpeechAfterKey("control+rightArrow")
 	_asserts.strings_match(
diff --git a/tests/system/robot/symbolPronunciationTests.py b/tests/system/robot/symbolPronunciationTests.py
--- a/tests/system/robot/symbolPronunciationTests.py
+++ b/tests/system/robot/symbolPronunciationTests.py
@@ -147,10 +147,10 @@ def test_moveByWord():
 			"don't",  # Expected: mid-word symbol
 			'right-pointing arrow', 't-shirt',
 			# todo: There should not be any "empty" words. Expect 'bar bar', and 'at  caret  star  line'
-			'1', 'bar', '2', '', '3', '', '4',
+			'1', 'bar', '2', 'blank', '3', 'blank', '4',
 			# end of first line
 			'blank',  # single space and newline
-			'',  # tab and newline  todo: There should not be any "empty" words.
+			'blank',  # tab and newline  todo: There should not be any "empty" words.
 			'blank',  # 4 spaces and newline
 			'right-pointing arrow',
 			't-shirt',
@@ -195,14 +195,14 @@ def test_moveByLine():
 		symbolLevel=SymLevel.NONE,
 		expectedSpeech=[
 			'Say', '(quietly)', 'Hello,', 'Jim .', "don't",  # Expect:
-			'',  # todo: Expect 'right-pointing arrow'
+			'blank',  # todo: Expect 'right-pointing arrow'
 			't-shirt',
-			'',  # todo: Expect 'right-pointing arrow'
+			'blank',  # todo: Expect 'right-pointing arrow'
 			't-shirt',
 			't-shirt',  # todo: Expect 'right-pointing arrow t-shirt'
 			'1   2    3      4',  # todo: Should symbols be passed to synth, i.e. "1 | 2 || 3 etc"?
 			'blank',  # single space
-			'',  # tab  # todo: There should not be any "empty" lines.
+			'blank',  # tab  # todo: There should not be any "empty" lines.
 			'blank',  # four spaces
 			'blank',  # end of doc
 		]