Repository: thousandbrainsproject/tbp.monty
Pull request #305: fix: SurfacePolicy touch_object and dynamic_call no longer interfere
Author: login: tristanls
Labels: name: triaged
description: This issue or pull request was triaged
Created: 2025-05-09T17:35:55Z | Merged: 2025-05-12T15:20:20Z

This pull request removes the interference between SurfacePolicy's `dynamic_call` implementation and its embedded `touch_object` positioning procedure, which finds the object if the sensor loses it. Additionally, it resets the `touch_object` state between each object finding attempt. Previously, the `touch_object` procedure would resume from the random state it ended in from the previous attempt to find the object.

The `touch_object` state reset is done by having the data loader reset the `touch_object` positioning procedure on success. This is not great, but for now, `touch_object` might have succeeded and not know it. In the future, when it is refactored to the `TouchObject` positioning procedure, the reset will be contained within the positioning procedure itself.

The interference is removed by having `touch_object` no longer save its actions in `SurfacePolicy`'s `self.action`.

The test is updated with the example sequence of new expected behavior, demonstrating non-interference and resetting the `touch_object` state between each object finding attempt.

Lastly, return types of related methods are made specific instead of generic per best practice.

## Benchmark results
Multiple decimal places results here, the table updates are in the benchmark result changeset

| Code | Experiment | Correct (%) | Used MLH (%) | Num Match Steps | Rotation Error | WandB |
|---|---|---|---|---|---|---|
| new | base_config_10distinctobj_surf_agent | 100 | 0 | 27.642857142857142 | 0.20275 | [run](https://wandb.ai/thousand-brains-project/Monty/runs/4d51r1go/overview)
| new | randrot_noise_10distinctobj_surf_agent | 100 | 1 | 28.46 | 0.3758580000000001 | [run](https://wandb.ai/thousand-brains-project/Monty/runs/rhddvorw/overview)
| new | randrot_10distinctobj_surf_agent | 100 | 1 | 28.08 | 0.352194 | [run](https://wandb.ai/thousand-brains-project/Monty/runs/bn6cwftb/overview)
| new | base_10simobj_surf_agent | 94.28571428571428 | 11.428571428571429 | 86.30714285714286 | 0.24581287878787875 | [run](https://wandb.ai/thousand-brains-project/Monty/runs/elslasu1/overview)
| new | randrot_noise_10simobj_surf_agent | 93 | 36 | 181.35 | 0.5434129032258065 | [run](https://wandb.ai/thousand-brains-project/Monty/runs/8dr5cmeu/overview)
| new | randomrot_rawnoise_10distinctobj_surf_agent | 69 | 74 | 15.19 | 1.945234782608695 | [run](https://wandb.ai/thousand-brains-project/Monty/runs/rxy360ug/overview)
| new | base_77obj_surf_agent | 99.13419913419914 | 7.35930735930736 | 57.064935064935064 | 0.2236707423580786 | [run](https://wandb.ai/thousand-brains-project/Monty/runs/47nc07bu/overview)
| new | randrot_noise_77obj_surf_agent | 93.5064935064935 | 25.108225108225103 | 122.70995670995671 | 0.6740199074074075 | [run](https://wandb.ai/thousand-brains-project/Monty/runs/t7te0nhh/overview)
| new | unsupervised_inference_distinctobj_surf_agent | 3.488372093023256 | 40.69767441860465 | 100 | 1.5507666666666668 | [run](https://wandb.ai/thousand-brains-project/Monty/runs/4ffdsqfb/overview)

| Code | Experiment | Correct - 1st Epoch (%) | Correct - >1st Epoch (%) | Mean Objects per Graph | Mean Graphs per Object |
|---|---|---|---|---|---|
| new | surf_agent_unsupervised_10distinctobj | 80.00 | 87.78 | 1.22 | 1.1
| new | surf_agent_unsupervised_10distinctobj_noise | 70.00 | 72.22 | 1.07 | 1.75
| new | surf_agent_unsupervised_10simobj | 40.00 | 80.00 | 2.33 | 1.4


Diff:
diff --git a/benchmarks/results/ycb_10objs.csv b/benchmarks/results/ycb_10objs.csv
index 2338f23c6..ede526fd4 100644
--- a/benchmarks/results/ycb_10objs.csv
+++ b/benchmarks/results/ycb_10objs.csv
@@ -1,13 +1,13 @@
 Experiment,Correct (%)|align right,Used MLH (%)|align right,Num Match Steps|align right,Rotation Error (degrees)|align right,Run Time (mins)|align right,Episode Run Time (s)|align right
 base_config_10distinctobj_dist_agent,100.00,0.00,37,16.04,5,19
-base_config_10distinctobj_surf_agent,100.00,0.00,28,11.66,3,12
+base_config_10distinctobj_surf_agent,100.00,0.00,28,11.62,2,10
 randrot_noise_10distinctobj_dist_agent,100.00,3.00,55,22.35,5,33
 randrot_noise_10distinctobj_dist_on_distm,99.00,1.00,38,12.03,4,28
-randrot_noise_10distinctobj_surf_agent,100.00,0.00,29,21.27,3,19
-randrot_10distinctobj_surf_agent,100.00,1.00,28,18.61,2,11
+randrot_noise_10distinctobj_surf_agent,100.00,1.00,28,21.53,3,19
+randrot_10distinctobj_surf_agent,100.00,1.00,28,20.17,2,10
 randrot_noise_10distinctobj_5lms_dist_agent,100.00,0.00,57,45.26,10,84
-base_10simobj_surf_agent,94.29,11.43,84,12.74,6,28
+base_10simobj_surf_agent,94.29,11.43,86,14.08,6,31
 randrot_noise_10simobj_dist_agent,84.00,40.00,237,34.95,18,147
-randrot_noise_10simobj_surf_agent,93.00,34.00,178,30.60,15,134
-randomrot_rawnoise_10distinctobj_surf_agent,68.00,73.00,16,114.56,3,6
+randrot_noise_10simobj_surf_agent,93.00,36.00,181,31.14,17,146
+randomrot_rawnoise_10distinctobj_surf_agent,69.00,74.00,15,111.45,4,6
 base_10multi_distinctobj_dist_agent,79.29,10.71,31,20.05,43,1
diff --git a/benchmarks/results/ycb_77objs.csv b/benchmarks/results/ycb_77objs.csv
index 99770a47e..8a17c8ac7 100644
--- a/benchmarks/results/ycb_77objs.csv
+++ b/benchmarks/results/ycb_77objs.csv
@@ -1,6 +1,6 @@
 Experiment,Correct (%)|align right,Used MLH (%)|align right,Num Match Steps|align right,Rotation Error (degrees)|align right,Run Time (mins)|align right,Episode Run Time (s)|align right
 base_77obj_dist_agent,93.51,12.99,108,16.04,62,212
-base_77obj_surf_agent,99.13,6.06,54,10.74,12,33
+base_77obj_surf_agent,99.13,7.36,57,12.82,12,35
 randrot_noise_77obj_dist_agent,89.61,22.51,152,37.24,85,308
-randrot_noise_77obj_surf_agent,91.34,25.11,119,39.21,32,115
+randrot_noise_77obj_surf_agent,93.51,25.11,123,38.61,35,128
 randrot_noise_77obj_5lms_dist_agent,88.31,0.00,70,58.44,32,862
diff --git a/benchmarks/results/ycb_unsupervised.csv b/benchmarks/results/ycb_unsupervised.csv
index 878e02549..0139f644d 100644
--- a/benchmarks/results/ycb_unsupervised.csv
+++ b/benchmarks/results/ycb_unsupervised.csv
@@ -1,4 +1,4 @@
 Experiment, Correct - 1st Epoch(%)|align right, Correct - >1st Epoch (%)|align right, Mean Objects per Graph|align right, Mean Graphs per Object|align right, Run Time (mins)|align right, Episode Run Time (s)|align right
-surf_agent_unsupervised_10distinctobj,70.00,84.44,1.43,1.11,23,14
-surf_agent_unsupervised_10distinctobj_noise,70.00,67.78,1.19,2.11,27,16
-surf_agent_unsupervised_10simobj,40.00,72.22,2.43,1.7,35,21
+surf_agent_unsupervised_10distinctobj,80.00,87.78,1.22,1.1,16,10
+surf_agent_unsupervised_10distinctobj_noise,70.00,72.22,1.07,1.75,25,15
+surf_agent_unsupervised_10simobj,40.00,80.00,2.33,1.4,35,21
diff --git a/benchmarks/results/ycb_unsupervised_inference.csv b/benchmarks/results/ycb_unsupervised_inference.csv
index 0699e9167..6573fe2d1 100644
--- a/benchmarks/results/ycb_unsupervised_inference.csv
+++ b/benchmarks/results/ycb_unsupervised_inference.csv
@@ -1,3 +1,3 @@
 Experiment,Correct (%)|align right,Num Match Steps|align right,Run Time (mins)|align right,Episode Run Time (s)|align right
 unsupervised_inference_distinctobj_dist_agent,12.00,100,5,3.0
-unsupervised_inference_distinctobj_surf_agent,5.00,98,15,9
+unsupervised_inference_distinctobj_surf_agent,3.00,100,15,9
diff --git a/src/tbp/monty/frameworks/environments/embodied_data.py b/src/tbp/monty/frameworks/environments/embodied_data.py
index 8807c6d0c..ce3949226 100644
--- a/src/tbp/monty/frameworks/environments/embodied_data.py
+++ b/src/tbp/monty/frameworks/environments/embodied_data.py
@@ -476,6 +476,14 @@ def __next__(self):
                     view_sensor_id="view_finder",
                     state=self.motor_system._state,
                 )
+            else:
+                # TODO: Encapsulate this reset inside TouchObject positioning
+                #       procedure once it exists.
+                #       This is a hack to reset the current touch_object
+                #       positioning procedure state so that the next time
+                #       SurfacePolicy falls off the object, it will try to find
+                #       the object using its full repertoire of actions.
+                self.motor_system._policy.touch_search_amount = 0
 
             self._observation, proprioceptive_state = self.dataset[self._action]
             motor_system_state = MotorSystemState(proprioceptive_state)
diff --git a/src/tbp/monty/frameworks/models/motor_policies.py b/src/tbp/monty/frameworks/models/motor_policies.py
index ddf4f88e4..fd276ab35 100644
--- a/src/tbp/monty/frameworks/models/motor_policies.py
+++ b/src/tbp/monty/frameworks/models/motor_policies.py
@@ -1107,7 +1107,7 @@ def pre_episode(self):
 
     def touch_object(
         self, raw_observation, view_sensor_id: str, state: MotorSystemState
-    ) -> Action:
+    ) -> MoveForward | OrientHorizontal | OrientVertical:
         """The surface agent's policy for moving onto an object for sensing it.
 
         Like the distant agent's get_good_view, this is called at the beginning
@@ -1126,11 +1126,11 @@ def touch_object(
 
         Args:
             raw_observation: The raw observation from the simulator.
-            view_sensor_id: The ID of the viewfinder sensor.
-            state: The current state of the motor system.
+            view_sensor_id (str): The ID of the viewfinder sensor.
+            state (MotorSystemState): The current state of the motor system.
 
         Returns:
-            Action to take.
+            (MoveForward | OrientHorizontal | OrientVertical): Action to take.
         """
         # If the viewfinder sees the object within range, then move to it
         depth_at_center = self.get_depth_at_center(raw_observation, view_sensor_id)
@@ -1144,8 +1144,7 @@ def touch_object(
             )
             logging.debug(f"Move to touch visible object, forward by {distance}")
 
-            self.action = MoveForward(agent_id=self.agent_id, distance=distance)
-            return self.action
+            return MoveForward(agent_id=self.agent_id, distance=distance)
 
         logging.debug("Surface policy searching for object...")
 
@@ -1207,26 +1206,28 @@ def touch_object(
         self.touch_search_amount += rotation_degrees  # Accumulate total rotations
         # for touch-search
 
-        if orientation == "vertical":
-            self.action = OrientVertical(
+        return (
+            OrientVertical(
                 agent_id=self.agent_id,
                 rotation_degrees=rotation_degrees,
                 down_distance=move_lat_amount,
                 forward_distance=move_forward_amount,
             )
-        else:
-            self.action = OrientHorizontal(
+            if orientation == "vertical"
+            else OrientHorizontal(
                 agent_id=self.agent_id,
                 rotation_degrees=rotation_degrees,
                 left_distance=move_lat_amount,
                 forward_distance=move_forward_amount,
             )
-        return self.action
+        )
 
     ###
     # Methods that define behavior of __call__
     ###
-    def dynamic_call(self, state: Optional[MotorSystemState] = None) -> Action:
+    def dynamic_call(
+        self, state: Optional[MotorSystemState] = None
+    ) -> OrientHorizontal | OrientVertical | MoveTangentially | MoveForward | None:
         """Return the next action to take.
 
         This requires self.processed_observations to be updated at every step
@@ -1238,7 +1239,8 @@ def dynamic_call(self, state: Optional[MotorSystemState] = None) -> Action:
                 Defaults to None.
 
         Returns:
-            (Action): The action to take.
+            (OrientHorizontal | OrientVertical | MoveTangentially | MoveForward | None):
+                The action to take.
         """
         # Check if we have poor visualization of the object
         if self.processed_observations.get_feature_by_name("object_coverage") < 0.1:
@@ -1269,7 +1271,7 @@ def dynamic_call(self, state: Optional[MotorSystemState] = None) -> Action:
 
         return self.get_next_action(state)
 
-    def _orient_horizontal(self, state: MotorSystemState) -> Action:
+    def _orient_horizontal(self, state: MotorSystemState) -> OrientHorizontal:
         """Orient the agent horizontally.
 
         Args:
@@ -1289,7 +1291,7 @@ def _orient_horizontal(self, state: MotorSystemState) -> Action:
             forward_distance=forward_distance,
         )
 
-    def _orient_vertical(self, state: MotorSystemState) -> Action:
+    def _orient_vertical(self, state: MotorSystemState) -> OrientVertical:
         """Orient the agent vertically.
 
         Args:
@@ -1309,7 +1311,7 @@ def _orient_vertical(self, state: MotorSystemState) -> Action:
             forward_distance=forward_distance,
         )
 
-    def _move_tangentially(self, state: MotorSystemState) -> Action:
+    def _move_tangentially(self, state: MotorSystemState) -> MoveTangentially:
         """Move tangentially along the object surface.
 
         Args:
@@ -1338,7 +1340,7 @@ def _move_tangentially(self, state: MotorSystemState) -> Action:
 
         return action
 
-    def _move_forward(self) -> Action:
+    def _move_forward(self) -> MoveForward:
         """Move forward to touch the object at the right distance.
 
         Returns:
@@ -1353,7 +1355,9 @@ def _move_forward(self) -> Action:
         )
         return action
 
-    def get_next_action(self, state: MotorSystemState):
+    def get_next_action(
+        self, state: MotorSystemState
+    ) -> OrientHorizontal | OrientVertical | MoveTangentially | MoveForward | None:
         """Retrieve next action from a cycle of four actions.
 
         First move forward to touch the object at the right distance
diff --git a/tests/unit/policy_test.py b/tests/unit/policy_test.py
index f8028c7f8..d03b74a07 100644
--- a/tests/unit/policy_test.py
+++ b/tests/unit/policy_test.py
@@ -892,58 +892,116 @@ def test_surface_policy_moves_back_to_object(self):
             for loader_step, observation in enumerate(exp.dataloader):
                 exp.model.step(observation)
 
-                # |Step| Action           | Motor-only? | Obs processed? | Source
-                # |----|------------------|-------------|----------------|-------------
-                # | 13 | OrientHorizontal | True        | False          | touch_object
-                # | 14 | OrientHorizontal | True        | False          | touch_object
-                # | 15 | OrientHorizontal | True        | False          | touch_object
-                # | 16 | OrientHorizontal | True        | False          | touch_object
-                # | 17 | OrientHorizontal | True        | False          | touch_object
-                # | 18 | OrientHorizontal | True        | False          | touch_object
-                # | 19 | OrientHorizontal | True        | False          | touch_object
-                # | 20 | OrientHorizontal | True        | False          | touch_object
-                # | 21 | OrientHorizontal | True        | False          | touch_object
-                # | 22 | OrientHorizontal | True        | False          | touch_object
-                # | 23 | OrientHorizontal | True        | False          | touch_object
-                # | 24 | OrientHorizontal | True        | False          | touch_object
-                # | 25 | OrientVertical   | True        | False          | touch_object
-                # | 26 | MoveTangentially | True        | False          | dynamic_call
-                # | 27 | OrientVertical   | True        | False          | touch_object
-                # | 28 | MoveForward      | True        | False          | touch_object
-                # | 29 | OrientHorizontal | True        | False          | dynamic_call
-                # | 30 | OrientVertical   | True        | True           | dynamic_call
-                #
-                # Note that 26 (MoveTangentially) is a bug, in that SurfacePolicy
-                # dynamic_call assumes that it is in the middle of the
-                # MoveForward->OrientHorizontal->OrientVertical->MoveTangentially
-                # cycle, but in fact it is not.
-                # As a result, the sensor falls off the object again.
-
-                if 13 <= loader_step <= 25:  # Motor-only touch_object steps
+                #  Step | Action           | Motor-only? | Obs processed? | Source
+                # ------|------------------|-------------|----------------|-------------
+                #  1    | MoveForward      | True        | False          | dynamic_call
+                #  2    | OrientHorizontal | True        | False          | dynamic_call
+                #  3    | OrientVertical   | False       | True           | dynamic_call
+                #  4    | MoveTangentially | True        | False          | dynamic_call
+                #  5    | MoveForward      | True        | False          | dynamic_call
+                #  6    | OrientHorizontal | True        | False          | dynamic_call
+                #  7    | OrientVertical   | False       | True           | dynamic_call
+                #  8    | MoveTangentially | True        | False          | dynamic_call
+                #  9    | MoveForward      | True        | False          | dynamic_call
+                #  10   | OrientHorizontal | True        | False          | dynamic_call
+                #  11   | OrientVertical   | False       | True           | dynamic_call
+                #  12   | MoveTangentially | True        | False          | dynamic_call
+                # falls off object
+                #  13   | OrientHorizontal | True        | False          | touch_object
+                #  14   | OrientHorizontal | True        | False          | touch_object
+                #  15   | OrientHorizontal | True        | False          | touch_object
+                #  16   | OrientHorizontal | True        | False          | touch_object
+                #  17   | OrientHorizontal | True        | False          | touch_object
+                #  18   | OrientHorizontal | True        | False          | touch_object
+                #  19   | OrientHorizontal | True        | False          | touch_object
+                #  20   | OrientHorizontal | True        | False          | touch_object
+                #  21   | OrientHorizontal | True        | False          | touch_object
+                #  22   | OrientHorizontal | True        | False          | touch_object
+                #  23   | OrientHorizontal | True        | False          | touch_object
+                #  24   | OrientHorizontal | True        | False          | touch_object
+                #  25   | OrientVertical   | True        | False          | touch_object
+                # back on object
+                #  26   | OrientHorizontal | True        | False          | dynamic_call
+                #  27   | OrientVertical   | False       | True           | dynamic_call
+                #  28   | MoveTangentially | True        | False          | dynamic_call
+                # falls off object
+                #  29   | OrientHorizontal | True        | False          | touch_object
+                #  30   | OrientHorizontal | True        | False          | touch_object
+                #  31   | OrientHorizontal | True        | False          | touch_object
+                #  32   | OrientHorizontal | True        | False          | touch_object
+                #  33   | OrientHorizontal | True        | False          | touch_object
+                #  34   | OrientHorizontal | True        | False          | touch_object
+                #  35   | OrientHorizontal | True        | False          | touch_object
+                #  36   | OrientHorizontal | True        | False          | touch_object
+                #  37   | OrientHorizontal | True        | False          | touch_object
+                #  38   | OrientHorizontal | True        | False          | touch_object
+                #  39   | OrientHorizontal | True        | False          | touch_object
+                #  40   | OrientHorizontal | True        | False          | touch_object
+                #  41   | OrientVertical   | True        | False          | touch_object
+                #  42   | MoveForward      | True        | False          | touch_object
+                # back on object
+                #  43   | OrientHorizontal | True        | False          | dynamic_call
+                #  44   | OrientVertical   | False       | True           | dynamic_call
+                #  45   | MoveTangentially | True        | False          | dynamic_call
+                # falls off object
+                #  46   | OrientHorizontal | True        | False          | touch_object
+                #  47   | OrientHorizontal | True        | False          | touch_object
+                #  48   | OrientHorizontal | True        | False          | touch_object
+                #  49   | OrientHorizontal | True        | False          | touch_object
+                #  50   | OrientHorizontal | True        | False          | touch_object
+                #  51   | OrientHorizontal | True        | False          | touch_object
+                #  52   | OrientHorizontal | True        | False          | touch_object
+                #  53   | OrientHorizontal | True        | False          | touch_object
+                #  54   | OrientHorizontal | True        | False          | touch_object
+                #  55   | OrientHorizontal | True        | False          | touch_object
+                #  56   | OrientHorizontal | True        | False          | touch_object
+                #  57   | OrientHorizontal | True        | False          | touch_object
+                #  58   | OrientVertical   | True        | False          | touch_object
+                # back on object
+                #  59   | OrientHorizontal | True        | False          | dynamic_call
+                #  60   | OrientVertical   | False       | True           | dynamic_call
+                #  61   | MoveTangentially | True        | False          | dynamic_call
+                # falls off object
+                #  62   | OrientHorizontal | True        | False          | touch_object
+
+                # Motor-only touch_object steps
+                if (
+                    13 <= loader_step <= 25
+                    or 29 <= loader_step <= 42
+                    or 46 <= loader_step <= 58
+                    or loader_step == 62
+                ):
                     assert not exp.model.learning_modules[
                         0
-                    ].buffer.get_last_obs_processed(), "Should be off object"
+                    ].buffer.get_last_obs_processed(), (
+                        "Should be off object, motor-only step"
+                    )
+                if loader_step == 62:
+                    break  # Finish test
 
-                if loader_step == 26:  # Incorrect motor-only MoveTangentially
+                # First on-object steps are always OrientHorizontal motor-only steps
+                if loader_step in [26, 43, 59]:
                     assert not exp.model.learning_modules[
                         0
-                    ].buffer.get_last_obs_processed(), "Should be off object"
+                    ].buffer.get_last_obs_processed(), (
+                        "Should be on object, motor-only step"
+                    )
 
-                if 27 <= loader_step <= 28:  # Motor-only touch_object steps
-                    assert not exp.model.learning_modules[
+                # Second on-object steps are always OrientVertical that send data to LM
+                if loader_step in [27, 44, 60]:
+                    assert exp.model.learning_modules[
                         0
-                    ].buffer.get_last_obs_processed(), "Should be off object"
+                    ].buffer.get_last_obs_processed(), (
+                        "Should be on object, sending data to LM"
+                    )
 
-                if loader_step == 29:  # OrientHorizontal from SurfacePolicy cycle start
+                # Third on-object steps are always MoveTangentially motor-only steps
+                if loader_step in [28, 45, 61]:
                     assert not exp.model.learning_modules[
                         0
-                    ].buffer.get_last_obs_processed(), "Should be off object"
-
-                if loader_step == 30:  # OrientVertical from SurfacePolicy cycle
-                    assert exp.model.learning_modules[
-                        0
-                    ].buffer.get_last_obs_processed(), "Should be back on object"
-                    break  # Don't go into exploratory mode
+                    ].buffer.get_last_obs_processed(), (
+                        "Should be on object, motor-only step"
+                    )
 
     def test_surface_policy_orientation(self):
         """Test ability of surface agent to orient to a point-normal.


Reviews:
user: login: vkakerbeck
body: Nice catch and fix!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T10:40:05Z
user: login: tristanls
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T15:00:03Z
user: login: tristanls
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T15:00:56Z

Review comments:
diff_hunk: @@ -892,58 +892,116 @@ def test_surface_policy_moves_back_to_object(self):
             for loader_step, observation in enumerate(exp.dataloader):
                 exp.model.step(observation)
 
-                # |Step| Action           | Motor-only? | Obs processed? | Source
-                # |----|------------------|-------------|----------------|-------------
-                # | 13 | OrientHorizontal | True        | False          | touch_object
-                # | 14 | OrientHorizontal | True        | False          | touch_object
-                # | 15 | OrientHorizontal | True        | False          | touch_object
-                # | 16 | OrientHorizontal | True        | False          | touch_object
-                # | 17 | OrientHorizontal | True        | False          | touch_object
-                # | 18 | OrientHorizontal | True        | False          | touch_object
-                # | 19 | OrientHorizontal | True        | False          | touch_object
-                # | 20 | OrientHorizontal | True        | False          | touch_object
-                # | 21 | OrientHorizontal | True        | False          | touch_object
-                # | 22 | OrientHorizontal | True        | False          | touch_object
-                # | 23 | OrientHorizontal | True        | False          | touch_object
-                # | 24 | OrientHorizontal | True        | False          | touch_object
-                # | 25 | OrientVertical   | True        | False          | touch_object
-                # | 26 | MoveTangentially | True        | False          | dynamic_call
-                # | 27 | OrientVertical   | True        | False          | touch_object
-                # | 28 | MoveForward      | True        | False          | touch_object
-                # | 29 | OrientHorizontal | True        | False          | dynamic_call
-                # | 30 | OrientVertical   | True        | True           | dynamic_call
-                #
-                # Note that 26 (MoveTangentially) is a bug, in that SurfacePolicy
-                # dynamic_call assumes that it is in the middle of the
-                # MoveForward->OrientHorizontal->OrientVertical->MoveTangentially
-                # cycle, but in fact it is not.
-                # As a result, the sensor falls off the object again.
-
-                if 13 <= loader_step <= 25:  # Motor-only touch_object steps
+                #  Step | Action           | Motor-only? | Obs processed? | Source
path: tests/unit/policy_test.py
user: login: vkakerbeck
body: Are motor_only and obs_processed ever both True or both False?  Do we need both columns here?
created_at: 2025-05-12T10:38:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -892,58 +892,116 @@ def test_surface_policy_moves_back_to_object(self):
             for loader_step, observation in enumerate(exp.dataloader):
                 exp.model.step(observation)
 
-                # |Step| Action           | Motor-only? | Obs processed? | Source
-                # |----|------------------|-------------|----------------|-------------
-                # | 13 | OrientHorizontal | True        | False          | touch_object
-                # | 14 | OrientHorizontal | True        | False          | touch_object
-                # | 15 | OrientHorizontal | True        | False          | touch_object
-                # | 16 | OrientHorizontal | True        | False          | touch_object
-                # | 17 | OrientHorizontal | True        | False          | touch_object
-                # | 18 | OrientHorizontal | True        | False          | touch_object
-                # | 19 | OrientHorizontal | True        | False          | touch_object
-                # | 20 | OrientHorizontal | True        | False          | touch_object
-                # | 21 | OrientHorizontal | True        | False          | touch_object
-                # | 22 | OrientHorizontal | True        | False          | touch_object
-                # | 23 | OrientHorizontal | True        | False          | touch_object
-                # | 24 | OrientHorizontal | True        | False          | touch_object
-                # | 25 | OrientVertical   | True        | False          | touch_object
-                # | 26 | MoveTangentially | True        | False          | dynamic_call
-                # | 27 | OrientVertical   | True        | False          | touch_object
-                # | 28 | MoveForward      | True        | False          | touch_object
-                # | 29 | OrientHorizontal | True        | False          | dynamic_call
-                # | 30 | OrientVertical   | True        | True           | dynamic_call
-                #
-                # Note that 26 (MoveTangentially) is a bug, in that SurfacePolicy
-                # dynamic_call assumes that it is in the middle of the
-                # MoveForward->OrientHorizontal->OrientVertical->MoveTangentially
-                # cycle, but in fact it is not.
-                # As a result, the sensor falls off the object again.
-
-                if 13 <= loader_step <= 25:  # Motor-only touch_object steps
+                #  Step | Action           | Motor-only? | Obs processed? | Source
path: tests/unit/policy_test.py
user: login: tristanls
body: > Are motor_only and obs_processed ever both True or both False?

Yes, when a sensor thinks it should be sending data to the learning module `(OrientVertical,motor_only_step=False`), but observes no data, the result will be `motor_only_step=False` and `obs_processed=False`.
created_at: 2025-05-12T15:00:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -892,58 +892,116 @@ def test_surface_policy_moves_back_to_object(self):
             for loader_step, observation in enumerate(exp.dataloader):
                 exp.model.step(observation)
 
-                # |Step| Action           | Motor-only? | Obs processed? | Source
-                # |----|------------------|-------------|----------------|-------------
-                # | 13 | OrientHorizontal | True        | False          | touch_object
-                # | 14 | OrientHorizontal | True        | False          | touch_object
-                # | 15 | OrientHorizontal | True        | False          | touch_object
-                # | 16 | OrientHorizontal | True        | False          | touch_object
-                # | 17 | OrientHorizontal | True        | False          | touch_object
-                # | 18 | OrientHorizontal | True        | False          | touch_object
-                # | 19 | OrientHorizontal | True        | False          | touch_object
-                # | 20 | OrientHorizontal | True        | False          | touch_object
-                # | 21 | OrientHorizontal | True        | False          | touch_object
-                # | 22 | OrientHorizontal | True        | False          | touch_object
-                # | 23 | OrientHorizontal | True        | False          | touch_object
-                # | 24 | OrientHorizontal | True        | False          | touch_object
-                # | 25 | OrientVertical   | True        | False          | touch_object
-                # | 26 | MoveTangentially | True        | False          | dynamic_call
-                # | 27 | OrientVertical   | True        | False          | touch_object
-                # | 28 | MoveForward      | True        | False          | touch_object
-                # | 29 | OrientHorizontal | True        | False          | dynamic_call
-                # | 30 | OrientVertical   | True        | True           | dynamic_call
-                #
-                # Note that 26 (MoveTangentially) is a bug, in that SurfacePolicy
-                # dynamic_call assumes that it is in the middle of the
-                # MoveForward->OrientHorizontal->OrientVertical->MoveTangentially
-                # cycle, but in fact it is not.
-                # As a result, the sensor falls off the object again.
-
-                if 13 <= loader_step <= 25:  # Motor-only touch_object steps
+                #  Step | Action           | Motor-only? | Obs processed? | Source
path: tests/unit/policy_test.py
user: login: tristanls
body: I will rework this test and this event flow in the following pull request when I implement the `TouchObject` positioning procedure.
created_at: 2025-05-12T15:00:56Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-05-09T18:37:51Z
body: <!--pin-comment-marker-docs-preview-->
📚 **Documentation Preview**

✅ A preview of the documentation changes in this PR is available for maintainers at:
https://thousandbrainsproject.readme.io/v0.5-tristanls-surface-policy-bug

Last updated: 2025-05-12 15:14 UTC


author_association: NONE
user: login: tristanls
created_at: 2025-05-09T23:12:03Z
body: @ramyamounir, FYI, it looks like unsupervised training benchmarks got a bump in a good direction, at least on my laptop.
author_association: CONTRIBUTOR