Repository: nvaccess/nvda

Issue:
Unexpected behavior with table and container navigation for nested tables
I've seen the case below in a corporate environment. I agree it isn't common style of coding.

### Steps to reproduce:
1. Execute the following from a Firefox location bar:
    ```
    data:text/html, <p>Prepare for a table</p><table><tr><th>Header 1</th><th>Header 2</th></tr><tr><td>Outer table cell 1</td><td>Outer table cell 2</td></tr><tr><td><table><tr><td>Inner table cell 1</td><td>Inner table cell 2</td></tr></table></td><td>Outer table cell 4</td></tr><tr><td>Outer table cell 5</td><td>Outer table cell 6</td></tr></table>
    ```

2. Use table navigation inside the table. Everything works fine, until you reach outer table cell 3, which consists of a table itself with two cells.

### Expected behavior:
A. When in the inner table, pressing comma moves you out of it to outer table cell 4
B. Table navigation doesn't get trapped into the inner table as soon as you are in it.

### Actual behavior:
A. When in the inner table, pressing comma moves you out of it to outer table cell 6, so the bottom of the outer table
B. Table navigation gets trapped into the inner table as soon as you are in it.

### Technical details
The outer table is treated as a content table, the inner table is treated as a layout table. This is why pressing comma in the inner table moves you to the edge of the outer table, since layout tables aren't treated as containers. If you enable "Include layout tables" in browse mode settings, issue A goes away.

### Points to discuss:
At least there is an inconsistency here, as table nav works for layout tables, but they aren't treated as containers.

* Should we disable table navigation altogether for layout tables inside content tables? I think we should consider this. May be we should even disable table navigation in all layout tables
* If the former is too much, how about treating layout tables as containers, so if you get trapped with table nav, you can navigate out of the container to the outer table?


Discussion:
I've always believed table navigation shouldn't consider layout tables when inclusion of layout tables is disabled. However, I believe @MichaelDCurran made a case against this, hence the current behaviour. Mick, is this (still?) correct, and if so, can you explain why? @QChristensen, thoughts?
I may have thought this at one time, but  today I would be fine with ignoring layout tables. I think from memory the issue was more around the complexity to do this, especially with Gecko as we use a hybrid of virtualBuffer and out-of-proc IA2. having said this, some of that code was slightly refactored when abstracting table support for browseMode, so it is worth looking into again. I certainly agree the difference between table navigation and container jumping is bad.
P2 because this can be quite problematic and layout tables are still (unfortunately) rather common.
Ok, let me investigate the complexity of this later this week
I'll take this, as I am also working on some ARIA grid stuff which may directly affect this code.
Ah, that works too.

Resolving patch:
diff --git a/source/NVDAObjects/IAccessible/__init__.py b/source/NVDAObjects/IAccessible/__init__.py
--- a/source/NVDAObjects/IAccessible/__init__.py
+++ b/source/NVDAObjects/IAccessible/__init__.py
@@ -1022,6 +1022,8 @@ def getChild(self, index):
 		return self.correctAPIForRelation(IAccessible(IAccessibleObject=child[0], IAccessibleChildID=child[1]))
 
 	def _get_IA2Attributes(self):
+		if not isinstance(self.IAccessibleObject,IAccessibleHandler.IAccessible2):
+			return {}
 		try:
 			attribs = self.IAccessibleObject.attributes
 		except COMError as e:
@@ -1035,7 +1037,7 @@ def event_IA2AttributeChange(self):
 		# We currently only care about changes to the accessible drag and drop attributes, which we map to states, so treat this as a stateChange.
 		self.event_stateChange()
 
-	def _get_rowNumber(self):
+	def _get_IA2PhysicalRowNumber(self):
 		table=self.table
 		if table:
 			if self.IAccessibleTableUsesTableCellIndexAttrib:
@@ -1052,7 +1054,15 @@ def _get_rowNumber(self):
 				log.debugWarning("IAccessibleTable::rowIndex failed", exc_info=True)
 		raise NotImplementedError
 
-	def _get_columnNumber(self):
+	def _get_rowNumber(self):
+		index=self.IA2Attributes.get('rowindex')
+		if index is None and isinstance(self.parent,IAccessible):
+			index=self.parent.IA2Attributes.get('rowindex')
+		if index is None:
+			index=self.IA2PhysicalRowNumber
+		return index
+
+	def _get_IA2PhysicalColumnNumber(self):
 		table=self.table
 		if table:
 			if self.IAccessibleTableUsesTableCellIndexAttrib:
@@ -1069,7 +1079,13 @@ def _get_columnNumber(self):
 				log.debugWarning("IAccessibleTable::columnIndex failed", exc_info=True)
 		raise NotImplementedError
 
-	def _get_rowCount(self):
+	def _get_columnNumber(self):
+		index=self.IA2Attributes.get('colindex')
+		if index is None:
+			index=self.IA2PhysicalColumnNumber
+		return index
+
+	def _get_IA2PhysicalRowCount(self):
 		if hasattr(self,'IAccessibleTableObject'):
 			try:
 				return self.IAccessibleTableObject.nRows
@@ -1077,7 +1093,13 @@ def _get_rowCount(self):
 				log.debugWarning("IAccessibleTable::nRows failed", exc_info=True)
 		raise NotImplementedError
 
-	def _get_columnCount(self):
+	def _get_rowCount(self):
+		count=self.IA2Attributes.get('rowcount')
+		if count is None:
+			count=self.IA2PhysicalRowCount
+		return count
+
+	def _get_IA2PhysicalColumnCount(self):
 		if hasattr(self,'IAccessibleTableObject'):
 			try:
 				return self.IAccessibleTableObject.nColumns
@@ -1085,6 +1107,12 @@ def _get_columnCount(self):
 				log.debugWarning("IAccessibleTable::nColumns failed", exc_info=True)
 		raise NotImplementedError
 
+	def _get_columnCount(self):
+		count=self.IA2Attributes.get('colcount')
+		if count is None:
+			count=self.IA2PhysicalColumnCount
+		return count
+
 	def _get__IATableCell(self):
 		# Permanently cache the result.
 		try:
diff --git a/source/browseMode.py b/source/browseMode.py
--- a/source/browseMode.py
+++ b/source/browseMode.py
@@ -1539,6 +1539,11 @@ def script_movePastEndOfContainer(self,gesture):
 	# Translators: Description for the Move past end of container command in browse mode. 
 	script_movePastEndOfContainer.__doc__=_("Moves past the end  of the container element, such as a list or table")
 
+	#: The controlField attribute name that should be used as the row number when navigating in a table. By default this is the same as the presentational attribute name
+	navigationalTableRowNumberAttributeName="table-rownumber"
+	#: The controlField attribute name that should be used as the column number when navigating in a table. By default this is the same as the presentational attribute name
+	navigationalTableColumnNumberAttributeName="table-columnnumber"
+
 	def _getTableCellCoords(self, info):
 		"""
 		Fetches information about the deepest table cell at the given position.
@@ -1551,17 +1556,28 @@ def _getTableCellCoords(self, info):
 		if info.isCollapsed:
 			info = info.copy()
 			info.expand(textInfos.UNIT_CHARACTER)
-		for field in reversed(info.getTextWithFields()):
+		fields=list(info.getTextWithFields())
+		# First record the ID of all layout tables so that we can skip them when searching for the deepest table
+		layoutIDs=set()
+		for field in fields:
+			if isinstance(field, textInfos.FieldCommand) and field.command == "controlStart" and field.field.get('table-layout'):
+				tableID=field.field.get('table-id')
+				if tableID is not None:
+					layoutIDs.add(tableID)
+		for field in reversed(fields):
 			if not (isinstance(field, textInfos.FieldCommand) and field.command == "controlStart"):
 				# Not a control field.
 				continue
 			attrs = field.field
-			if "table-id" in attrs and "table-rownumber" in attrs:
+			tableID=attrs.get('table-id')
+			if tableID is None or tableID in layoutIDs:
+				continue
+			if self.navigationalTableColumnNumberAttributeName in attrs and not attrs.get('table-layout'):
 				break
 		else:
 			raise LookupError("Not in a table cell")
 		return (attrs["table-id"],
-			attrs["table-rownumber"], attrs["table-columnnumber"],
+			attrs[self.navigationalTableRowNumberAttributeName], attrs[self.navigationalTableColumnNumberAttributeName],
 			attrs.get("table-rowsspanned", 1), attrs.get("table-columnsspanned", 1))
 
 	def _getTableCellAt(self,tableID,startPos,row,column):
@@ -1576,13 +1592,16 @@ def _getTableCellAt(self,tableID,startPos,row,column):
 		@type column: int
 		@returns: the table cell's position in the document
 		@rtype: L{textInfos.TextInfo}
+		@raises: LookupError if the cell does not exist 
 		"""
 		raise NotImplementedError
 
+	_missingTableCellSearchLimit=3 #: The number of missing  cells L{_getNearestTableCell} is allowed to skip over to locate the next available cell
 	def _getNearestTableCell(self, tableID, startPos, origRow, origCol, origRowSpan, origColSpan, movement, axis):
 		"""
 		Locates the nearest table cell relative to another table cell in a given direction, given its coordinates.
 		For example, this is used to move to the cell in the next column, previous row, etc.
+		This method will skip over missing table cells (where L{_getTableCellAt} raises LookupError), up to the number of times set by _missingTableCellSearchLimit set on this instance.
 		@param tableID: the ID of the table
 		@param startPos: the position in the document to start searching from.
 		@type startPos: L{textInfos.TextInfo}
@@ -1612,19 +1631,28 @@ def _getNearestTableCell(self, tableID, startPos, origRow, origCol, origRowSpan,
 		elif axis == "column":
 			destCol += origColSpan if movement == "next" else -1
 
-		if destCol < 1 or destRow<1:
-			# Optimisation: We're definitely at the edge of the column or row.
-			raise LookupError
-
-		return self._getTableCellAt(tableID,startPos,destRow,destCol)
+		# Try and fetch the cell at these coordinates, though  if a  cell is missing, try  several more times moving the coordinates on by one cell each time
+		limit=self._missingTableCellSearchLimit
+		while limit>0:
+			limit-=1
+			if destCol < 1 or destRow<1:
+				# Optimisation: We're definitely at the edge of the column or row.
+				raise LookupError
+			try:
+				return self._getTableCellAt(tableID,startPos,destRow,destCol)
+			except LookupError:
+				pass
+			if axis=="row":
+				destRow+=1 if movement=="next" else -1
+			else:
+				destCol+=1 if movement=="next" else -1
+		raise LookupError
 
 	def _tableMovementScriptHelper(self, movement="next", axis=None):
 		if isScriptWaiting():
 			return
 		formatConfig=config.conf["documentFormatting"].copy()
 		formatConfig["reportTables"]=True
-		#  For now, table movement includes layout tables even if reporting of layout tables is disabled.
-		formatConfig["includeLayoutTables"]=True
 		try:
 			tableID, origRow, origCol, origRowSpan, origColSpan = self._getTableCellCoords(self.selection)
 		except LookupError:
diff --git a/source/virtualBuffers/gecko_ia2.py b/source/virtualBuffers/gecko_ia2.py
--- a/source/virtualBuffers/gecko_ia2.py
+++ b/source/virtualBuffers/gecko_ia2.py
@@ -23,6 +23,11 @@
 class Gecko_ia2_TextInfo(VirtualBufferTextInfo):
 
 	def _normalizeControlField(self,attrs):
+		for attr in ("table-physicalrownumber","table-physicalcolumnnumber","table-physicalrowcount","table-physicalcolumncount"):
+			attrVal=attrs.get(attr)
+			if attrVal is not None:
+				attrs[attr]=int(attrVal)
+
 		current = attrs.get("IAccessible2::attribute_current")
 		if current is not None:
 			attrs['current']= current
@@ -260,12 +265,20 @@ def event_scrollingStart(self, obj, nextHandler):
 			return nextHandler()
 	event_scrollingStart.ignoreIsReady = True
 
+	# NVDA exposes IAccessible2 table interface row and column numbers as table-physicalrownumber and table-physicalcolumnnumber respectively.
+	# These should be used when navigating the physical table (I.e. these values should be provided to the table interfaces).
+	# The presentational table-columnnumber and table-rownumber attributes are normally duplicates of the physical ones, but are overridden  by the values of aria-rowindex and aria-colindex if present.
+	navigationalTableRowNumberAttributeName="table-physicalrownumber"
+	navigationalTableColumnNumberAttributeName="table-physicalcolumnnumber"
+
 	def _getTableCellAt(self,tableID,startPos,destRow,destCol):
 		docHandle = self.rootDocHandle
 		table = self.getNVDAObjectFromIdentifier(docHandle, tableID)
 		try:
 			cell = table.IAccessibleTableObject.accessibleAt(destRow - 1, destCol - 1).QueryInterface(IAccessible2)
 			cell = NVDAObjects.IAccessible.IAccessible(IAccessibleObject=cell, IAccessibleChildID=0)
+			if cell.IA2Attributes.get('hidden'):
+				raise LookupError("Found hidden cell") 
 			return self.makeTextInfo(cell)
 		except (COMError, RuntimeError):
 			raise LookupError


Test patch: