Repository: sonic-net/sonic-platform-daemons
Pull request #500: [xcvrd] DPB support on platforms with CmisManagerTask enabled
Author: login: ishidawataru
Created: 2024-06-14T02:53:07Z | Merged: 2025-03-27T03:06:08Z

<!-- Provide a general summary of your changes in the Title above -->


#### Description

This commit fixes DPB support with CMIS transceivers.

CmisManagerTask's `port_dict` must be updated according to the port add/remove events.
This commit removes the `port_mapping` field from CmisManagerTask as
`port_mapping` was mostly used just for storing `asic_id` information
and that can be simply done by `port_dict` instead.

Added a helper method `get_asic_id()` method to CmisManagerTask for
getting `asic_id` from `logical_port`.

#### Motivation and Context

fixes https://github.com/sonic-net/sonic-buildimage/issues/18893

#### How Has This Been Tested?

I tested on the VS environment.

#### Additional Information (Optional)



Diff:
diff --git a/sonic-xcvrd/tests/test_xcvrd.py b/sonic-xcvrd/tests/test_xcvrd.py
index fe4a2dc19..9a03049c0 100644
--- a/sonic-xcvrd/tests/test_xcvrd.py
+++ b/sonic-xcvrd/tests/test_xcvrd.py
@@ -239,6 +239,26 @@ def test_CmisManagerTask_task_run_with_exception(self):
         assert("sonic-xcvrd/xcvrd/xcvrd.py" in str(trace))
         assert("wait_for_port_config_done" in str(trace))
 
+        port_change_event = PortChangeEvent('Ethernet0', 1, 0, PortChangeEvent.PORT_ADD)
+        port_mapping.handle_port_change_event(port_change_event)
+        cmis_manager = CmisManagerTask(DEFAULT_NAMESPACE, port_mapping, stop_event)
+        cmis_manager.wait_for_port_config_done = MagicMock() #no-op
+        cmis_manager.update_port_transceiver_status_table_sw_cmis_state = MagicMock(side_effect = NotImplementedError)
+        exception_received = None
+        trace = None
+        try:
+            cmis_manager.start()
+            cmis_manager.join()
+        except Exception as e1:
+            exception_received = e1
+            trace = traceback.format_exc()
+
+        assert not cmis_manager.is_alive()
+        assert(type(exception_received) == NotImplementedError)
+        assert("NotImplementedError" in str(trace) and "effect" in str(trace))
+        assert("sonic-xcvrd/xcvrd/xcvrd.py" in str(trace))
+        assert("update_port_transceiver_status_table_sw_cmis_state" in str(trace))
+
     @patch('xcvrd.xcvrd.PortChangeObserver', MagicMock(handle_port_update_event=MagicMock()))
     @patch('xcvrd.xcvrd.CmisManagerTask.wait_for_port_config_done', MagicMock())
     @patch('xcvrd.xcvrd.is_fast_reboot_enabled', MagicMock(return_value=(False)))
@@ -268,7 +288,7 @@ def test_CmisManagerTask_get_xcvr_api_exception(self, mock_platform_chassis, moc
         mock_sfp.get_xcvr_api = MagicMock(side_effect=NotImplementedError)
         task.task_worker()
         assert mock_log_exception_traceback.call_count == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_FAILED
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_FAILED
 
         # Case 2: is_flat_memory() raises AttributeError. In this case, CMIS SM should transition to READY state
         mock_xcvr_api = MagicMock()
@@ -277,7 +297,7 @@ def test_CmisManagerTask_get_xcvr_api_exception(self, mock_platform_chassis, moc
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.on_port_update_event(port_change_event)
         task.task_worker()
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_READY
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_READY
 
         # Case 2.5: get_module_type_abbreviation() returns unsupported module type. In this case, CMIS SM should transition to READY state
         mock_xcvr_api.is_flat_memory = MagicMock(return_value=False)
@@ -285,7 +305,7 @@ def test_CmisManagerTask_get_xcvr_api_exception(self, mock_platform_chassis, moc
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.on_port_update_event(port_change_event)
         task.task_worker()
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_READY
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_READY
 
         # Case 3: get_cmis_application_desired() raises an exception
         mock_xcvr_api.is_flat_memory = MagicMock(return_value=False)
@@ -296,7 +316,7 @@ def test_CmisManagerTask_get_xcvr_api_exception(self, mock_platform_chassis, moc
         task.get_cmis_host_lanes_mask = MagicMock()
         task.task_worker()
         assert mock_log_exception_traceback.call_count == 2
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_FAILED
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_FAILED
         assert task.get_cmis_host_lanes_mask.call_count == 0
 
     @patch('xcvrd.xcvrd_utilities.port_event_helper.subscribe_port_config_change', MagicMock(side_effect = NotImplementedError))
@@ -361,7 +381,7 @@ def test_SfpStateUpdateTask_task_run_with_exception(self):
     @patch('xcvrd.xcvrd.SffManagerTask.join')
     def test_DaemonXcvrd_run_with_exception(self, mock_task_join_sff, mock_task_join_sfp,
                                             mock_task_join_dom, mock_init, mock_os_kill):
-        mock_init.return_value = (PortMapping(), set())
+        mock_init.return_value = PortMapping()
         xcvrd = DaemonXcvrd(SYSLOG_IDENTIFIER)
         xcvrd.enable_sff_mgr = True
         xcvrd.load_feature_flags = MagicMock()
@@ -1640,7 +1660,7 @@ def mock_get_sfp(port):
     @patch('xcvrd.xcvrd.DomInfoUpdateTask.join')
     @patch('xcvrd.xcvrd.SfpStateUpdateTask.join')
     def test_DaemonXcvrd_run(self, mock_task_stop1, mock_task_stop2, mock_task_run1, mock_task_run2, mock_deinit, mock_init):
-        mock_init.return_value = (PortMapping(), set())
+        mock_init.return_value = PortMapping()
         xcvrd = DaemonXcvrd(SYSLOG_IDENTIFIER)
         xcvrd.load_feature_flags = MagicMock()
         xcvrd.stop_event.wait = MagicMock()
@@ -1933,12 +1953,25 @@ def test_CmisManagerTask_handle_port_change_event(self):
 
         port_change_event = PortChangeEvent('Ethernet0', 1, 0, PortChangeEvent.PORT_DEL)
         task.on_port_update_event(port_change_event)
+        assert len(task.port_dict) == 0
+
+        port_dict = {'speed':'400000', 'lanes':'1,2,3,4,5,6,7,8'}
+        port_change_event = PortChangeEvent('Ethernet0', 1, 0, PortChangeEvent.PORT_SET, port_dict)
+        task.on_port_update_event(port_change_event)
         assert len(task.port_dict) == 1
 
-        port_change_event = PortChangeEvent('Ethernet0', 1, 0, PortChangeEvent.PORT_SET)
+        # STATE_DB DEL event doesn't remove port from port_dict
+        # this happens when transceiver is plugged-out or DPB is used
+        port_change_event = PortChangeEvent('Ethernet0', 1, 0, PortChangeEvent.PORT_DEL, {}, db_name='STATE_DB')
         task.on_port_update_event(port_change_event)
         assert len(task.port_dict) == 1
 
+        # CONFIG_DB DEL event removes port from port_dict
+        # this happens when DPB is used
+        port_change_event = PortChangeEvent('Ethernet0', 1, 0, PortChangeEvent.PORT_DEL, {}, db_name='CONFIG_DB', table_name='PORT')
+        task.on_port_update_event(port_change_event)
+        assert len(task.port_dict) == 0
+
     @patch('xcvrd.xcvrd.XcvrTableHelper')
     def test_CmisManagerTask_get_configured_freq(self, mock_table_helper):
         port_mapping = PortMapping()
@@ -2431,16 +2464,15 @@ def test_CmisManagerTask_task_worker(self, mock_chassis, mock_get_status_tbl):
         mock_chassis.get_sfp = MagicMock(return_value=mock_sfp)
 
         port_mapping = PortMapping()
+        port_mapping.handle_port_change_event(PortChangeEvent('Ethernet0', 1, 0, PortChangeEvent.PORT_ADD))
         stop_event = threading.Event()
         task = CmisManagerTask(DEFAULT_NAMESPACE, port_mapping, stop_event)
-        task.port_mapping.logical_port_list = ['Ethernet0']
         task.xcvr_table_helper = XcvrTableHelper(DEFAULT_NAMESPACE)
         task.xcvr_table_helper.get_status_tbl.return_value = mock_get_status_tbl
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_UNKNOWN
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_UNKNOWN
 
-        task.port_mapping.logical_port_list = MagicMock()
         port_change_event = PortChangeEvent('PortConfigDone', -1, 0, PortChangeEvent.PORT_SET)
         task.on_port_update_event(port_change_event)
         assert task.isPortConfigDone
@@ -2449,7 +2481,7 @@ def test_CmisManagerTask_task_worker(self, mock_chassis, mock_get_status_tbl):
                                             {'speed':'400000', 'lanes':'1,2,3,4,5,6,7,8'})
         task.on_port_update_event(port_change_event)
         assert len(task.port_dict) == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_INSERTED
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_INSERTED
 
         task.get_host_tx_status = MagicMock(return_value='true')
         task.get_port_admin_status = MagicMock(return_value='up')
@@ -2463,51 +2495,50 @@ def test_CmisManagerTask_task_worker(self, mock_chassis, mock_get_status_tbl):
         mock_xcvr_api.decommission_all_datapaths = MagicMock(return_value=True)
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_DP_DEINIT
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_DP_DEINIT
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
         assert mock_xcvr_api.set_datapath_deinit.call_count == 1
         assert mock_xcvr_api.tx_disable_channel.call_count == 1
         assert mock_xcvr_api.set_lpmode.call_count == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_AP_CONF
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_AP_CONF
 
         # Case 2: DP_DEINIT --> AP Configured
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
         assert mock_xcvr_api.set_application.call_count == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_DP_INIT
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_DP_INIT
 
         # Case 3: AP Configured --> DP_INIT
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
         assert mock_xcvr_api.set_datapath_init.call_count == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_DP_TXON
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_DP_TXON
 
         # Case 4: DP_INIT --> DP_TXON
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
         assert mock_xcvr_api.tx_disable_channel.call_count == 2
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_DP_ACTIVATE
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_DP_ACTIVATE
 
         # Case 5: DP_TXON --> DP_ACTIVATION
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.post_port_active_apsel_to_db = MagicMock()
         task.task_worker()
         assert task.post_port_active_apsel_to_db.call_count == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_READY
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_READY
 
         # Fail test coverage - Module Inserted state failing to reach DP_DEINIT
         port_mapping = PortMapping()
+        port_mapping.handle_port_change_event(PortChangeEvent('Ethernet1', 1, 0, PortChangeEvent.PORT_ADD))
         stop_event = threading.Event()
         task = CmisManagerTask(DEFAULT_NAMESPACE, port_mapping, stop_event)
-        task.port_mapping.logical_port_list = ['Ethernet1']
         task.xcvr_table_helper = XcvrTableHelper(DEFAULT_NAMESPACE)
         task.xcvr_table_helper.get_status_tbl.return_value = mock_get_status_tbl
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
-        assert get_cmis_state_from_state_db('Ethernet1', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet1'))) == CMIS_STATE_UNKNOWN
+        assert get_cmis_state_from_state_db('Ethernet1', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet1'))) == CMIS_STATE_UNKNOWN
 
-        task.port_mapping.logical_port_list = MagicMock()
         port_change_event = PortChangeEvent('PortConfigDone', -1, 0, PortChangeEvent.PORT_SET)
         task.on_port_update_event(port_change_event)
         assert task.isPortConfigDone
@@ -2516,7 +2547,7 @@ def test_CmisManagerTask_task_worker(self, mock_chassis, mock_get_status_tbl):
                                             {'speed':'400000', 'lanes':'1,2,3,4,5,6,7,8'})
         task.on_port_update_event(port_change_event)
         assert len(task.port_dict) == 1
-        assert get_cmis_state_from_state_db('Ethernet1', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet1'))) == CMIS_STATE_INSERTED
+        assert get_cmis_state_from_state_db('Ethernet1', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet1'))) == CMIS_STATE_INSERTED
 
         task.get_host_tx_status = MagicMock(return_value='true')
         task.get_port_admin_status = MagicMock(return_value='up')
@@ -2530,7 +2561,7 @@ def test_CmisManagerTask_task_worker(self, mock_chassis, mock_get_status_tbl):
         mock_xcvr_api.decommission_all_datapaths = MagicMock(return_value=False)
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
-        assert not get_cmis_state_from_state_db('Ethernet1', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet1'))) == CMIS_STATE_DP_DEINIT
+        assert not get_cmis_state_from_state_db('Ethernet1', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet1'))) == CMIS_STATE_DP_DEINIT
 
     @patch('xcvrd.xcvrd.XcvrTableHelper.get_status_tbl')
     @patch('xcvrd.xcvrd.platform_chassis')
@@ -2635,16 +2666,15 @@ def test_CmisManagerTask_task_worker_fastboot(self, mock_chassis, mock_get_statu
         mock_chassis.get_sfp = MagicMock(return_value=mock_sfp)
 
         port_mapping = PortMapping()
+        port_mapping.handle_port_change_event(PortChangeEvent('Ethernet0', 1, 0, PortChangeEvent.PORT_ADD))
         stop_event = threading.Event()
         task = CmisManagerTask(DEFAULT_NAMESPACE, port_mapping, stop_event)
-        task.port_mapping.logical_port_list = ['Ethernet0']
         task.xcvr_table_helper = XcvrTableHelper(DEFAULT_NAMESPACE)
         task.xcvr_table_helper.get_status_tbl.return_value = mock_get_status_tbl
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_UNKNOWN
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_UNKNOWN
 
-        task.port_mapping.logical_port_list = MagicMock()
         port_change_event = PortChangeEvent('PortConfigDone', -1, 0, PortChangeEvent.PORT_SET)
         task.on_port_update_event(port_change_event)
         assert task.isPortConfigDone
@@ -2653,7 +2683,7 @@ def test_CmisManagerTask_task_worker_fastboot(self, mock_chassis, mock_get_statu
                                             {'speed':'400000', 'lanes':'1,2,3,4,5,6,7,8'})
         task.on_port_update_event(port_change_event)
         assert len(task.port_dict) == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_INSERTED
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_INSERTED
 
         task.get_host_tx_status = MagicMock(return_value='false')
         task.get_port_admin_status = MagicMock(return_value='up')
@@ -2668,7 +2698,7 @@ def test_CmisManagerTask_task_worker_fastboot(self, mock_chassis, mock_get_statu
 
         assert mock_xcvr_api.tx_disable_channel.call_count == 1
         assert task.post_port_active_apsel_to_db.call_count == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_READY
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_READY
 
     @patch('xcvrd.xcvrd.XcvrTableHelper.get_status_tbl')
     @patch('xcvrd.xcvrd.platform_chassis')
@@ -2773,16 +2803,15 @@ def test_CmisManagerTask_task_worker_host_tx_ready_false(self, mock_chassis, moc
         mock_chassis.get_sfp = MagicMock(return_value=mock_sfp)
 
         port_mapping = PortMapping()
+        port_mapping.handle_port_change_event(PortChangeEvent('Ethernet0', 1, 0, PortChangeEvent.PORT_ADD))
         stop_event = threading.Event()
         task = CmisManagerTask(DEFAULT_NAMESPACE, port_mapping, stop_event)
-        task.port_mapping.logical_port_list = ['Ethernet0']
         task.xcvr_table_helper = XcvrTableHelper(DEFAULT_NAMESPACE)
         task.xcvr_table_helper.get_status_tbl.return_value = mock_get_status_tbl
         task.task_stopping_event.is_set = MagicMock(side_effect=[False, False, True])
         task.task_worker()
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_UNKNOWN
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_UNKNOWN
 
-        task.port_mapping.logical_port_list = MagicMock()
         port_change_event = PortChangeEvent('PortConfigDone', -1, 0, PortChangeEvent.PORT_SET)
         task.on_port_update_event(port_change_event)
         assert task.isPortConfigDone
@@ -2791,7 +2820,7 @@ def test_CmisManagerTask_task_worker_host_tx_ready_false(self, mock_chassis, moc
                                             {'speed':'400000', 'lanes':'1,2,3,4,5,6,7,8'})
         task.on_port_update_event(port_change_event)
         assert len(task.port_dict) == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_INSERTED
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_INSERTED
 
         task.get_host_tx_status = MagicMock(return_value='false')
         task.get_port_admin_status = MagicMock(return_value='up')
@@ -2806,7 +2835,7 @@ def test_CmisManagerTask_task_worker_host_tx_ready_false(self, mock_chassis, moc
 
         assert task.post_port_active_apsel_to_db.call_count == 1
         assert mock_xcvr_api.tx_disable_channel.call_count == 1
-        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.port_mapping.get_asic_id_for_logical_port('Ethernet0'))) == CMIS_STATE_READY
+        assert get_cmis_state_from_state_db('Ethernet0', task.xcvr_table_helper.get_status_tbl(task.get_asic_id('Ethernet0'))) == CMIS_STATE_READY
 
     @pytest.mark.parametrize("lport, expected_dom_polling", [
         ('Ethernet0', 'disabled'),
diff --git a/sonic-xcvrd/xcvrd/xcvrd.py b/sonic-xcvrd/xcvrd/xcvrd.py
index b712122c8..8ad3f227c 100644
--- a/sonic-xcvrd/xcvrd/xcvrd.py
+++ b/sonic-xcvrd/xcvrd/xcvrd.py
@@ -604,12 +604,12 @@ def __init__(self, namespaces, port_mapping, main_thread_stop_event, skip_cmis_m
         self.exc = None
         self.task_stopping_event = threading.Event()
         self.main_thread_stop_event = main_thread_stop_event
-        self.port_dict = {}
-        self.port_mapping = copy.deepcopy(port_mapping)
+        self.port_dict = {k: {"asic_id": v} for k, v in port_mapping.logical_to_asic.items()}
         self.isPortInitDone = False
         self.isPortConfigDone = False
         self.skip_cmis_mgr = skip_cmis_mgr
         self.namespaces = namespaces
+        self.xcvr_table_helper = XcvrTableHelper(self.namespaces)
 
     def log_debug(self, message):
         helper_logger.log_debug("CMIS: {}".format(message))
@@ -620,9 +620,11 @@ def log_notice(self, message):
     def log_error(self, message):
         helper_logger.log_error("CMIS: {}".format(message))
 
+    def get_asic_id(self, lport):
+        return self.port_dict.get(lport, {}).get("asic_id", -1)
+
     def update_port_transceiver_status_table_sw_cmis_state(self, lport, cmis_state_to_set):
-        asic_index = self.port_mapping.get_asic_id_for_logical_port(lport)
-        status_table = self.xcvr_table_helper.get_status_tbl(asic_index)
+        status_table = self.xcvr_table_helper.get_status_tbl(self.get_asic_id(lport))
         if status_table is None:
             helper_logger.log_error("status_table is None while updating "
                                     "sw CMIS state for lport {}".format(lport))
@@ -654,15 +656,12 @@ def on_port_update_event(self, port_change_event):
         if pport is None:
             return
 
-        # Skip if the port/cage type is not a CMIS
-        # 'index' can be -1 if STATE_DB|PORT_TABLE
-        if lport not in self.port_dict:
-            self.port_dict[lport] = {}
-
         if port_change_event.port_dict is None:
             return
 
         if port_change_event.event_type == port_change_event.PORT_SET:
+            if lport not in self.port_dict:
+                self.port_dict[lport] = {"asic_id": port_change_event.asic_id}
             if pport >= 0:
                 self.port_dict[lport]['index'] = pport
             if 'speed' in port_change_event.port_dict and port_change_event.port_dict['speed'] != 'N/A':
@@ -681,8 +680,45 @@ def on_port_update_event(self, port_change_event):
                 self.port_dict[lport]['subport'] = int(port_change_event.port_dict['subport'])
 
             self.force_cmis_reinit(lport, 0)
-        else:
-            self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+
+        elif port_change_event.event_type == port_change_event.PORT_DEL:
+            # In handling the DEL event, the following two scenarios must be considered:
+            # 1. PORT_DEL event due to transceiver plug-out
+            # 2. PORT_DEL event due to Dynamic Port Breakout (DPB)
+            #
+            # Scenario 1 is simple, as only a STATE_DB|TRANSCEIVER_INFO PORT_DEL event occurs,
+            # so we just need to set SW_CMIS_STATE to CMIS_STATE_REMOVED.
+            #
+            # Scenario 2 is a bit more complex. First, for the port(s) before DPB, a CONFIG_DB|PORT PORT_DEL
+            # and a STATE_DB|PORT_TABLE PORT_DEL event occur. Next, for the port(s) after DPB,
+            # a CONFIG_DB|PORT PORT_SET and a STATE_DB|PORT_TABLE PORT_SET event occur.
+            # After that (after a short delay), a STATE_DB|TRANSCEIVER_INFO PORT_DEL event
+            # occurs for the port(s) before DPB, and finally, a STATE_DB|TRANSCEIVER_INFO
+            # PORT_SET event occurs for the port(s) after DPB.
+            #
+            # Below is the event sequence when configuring Ethernet0 from "2x200G" to "1x400G"
+            # (based on actual logs).
+            #
+            # 1.  SET Ethernet0 CONFIG_DB|PORT
+            # 2.  DEL Ethernet2 CONFIG_DB|PORT
+            # 3.  DEL Ethernet0 CONFIG_DB|PORT
+            # 4.  DEL Ethernet0 STATE_DB|PORT_TABLE
+            # 5.  DEL Ethernet2 STATE_DB|PORT_TABLE
+            # 6.  SET Ethernet0 CONFIG_DB|PORT
+            # 7.  SET Ethernet0 STATE_DB|PORT_TABLE
+            # 8.  SET Ethernet0 STATE_DB|PORT_TABLE
+            # 9.  DEL Ethernet2 STATE_DB|TRANSCEIVER_INFO
+            # 10. DEL Ethernet0 STATE_DB|TRANSCEIVER_INFO
+            # 11. SET Ethernet0 STATE_DB|TRANSCEIVER_INFO
+            #
+            # To handle both scenarios, if the lport exists in port_dict for any DEL EVENT,
+            # set SW_CMIS_STATE to REMOVED. Additionally, for DEL EVENTS from CONFIG_DB due to DPB,
+            # remove the lport from port_dict.
+            if lport in self.port_dict:
+                self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+
+            if port_change_event.db_name == 'CONFIG_DB' and port_change_event.table_name == 'PORT':
+                self.port_dict.pop(lport)
 
     def get_cmis_dp_init_duration_secs(self, api):
         return api.get_datapath_init_duration()/1000
@@ -946,8 +982,7 @@ def get_configured_laser_freq_from_db(self, lport):
            Return the Tx power configured by user in CONFIG_DB's PORT table
         """
         freq = 0
-        asic_index = self.port_mapping.get_asic_id_for_logical_port(lport)
-        port_tbl = self.xcvr_table_helper.get_cfg_port_tbl(asic_index)
+        port_tbl = self.xcvr_table_helper.get_cfg_port_tbl(self.get_asic_id(lport))
 
         found, port_info = port_tbl.get(lport)
         if found and 'laser_freq' in dict(port_info):
@@ -959,8 +994,7 @@ def get_configured_tx_power_from_db(self, lport):
            Return the Tx power configured by user in CONFIG_DB's PORT table
         """
         power = 0
-        asic_index = self.port_mapping.get_asic_id_for_logical_port(lport)
-        port_tbl = self.xcvr_table_helper.get_cfg_port_tbl(asic_index)
+        port_tbl = self.xcvr_table_helper.get_cfg_port_tbl(self.get_asic_id(lport))
 
         found, port_info = port_tbl.get(lport)
         if found and 'tx_power' in dict(port_info):
@@ -970,8 +1004,7 @@ def get_configured_tx_power_from_db(self, lport):
     def get_host_tx_status(self, lport):
         host_tx_ready = 'false'
 
-        asic_index = self.port_mapping.get_asic_id_for_logical_port(lport)
-        state_port_tbl = self.xcvr_table_helper.get_state_port_tbl(asic_index)
+        state_port_tbl = self.xcvr_table_helper.get_state_port_tbl(self.get_asic_id(lport))
 
         found, port_info = state_port_tbl.get(lport)
         if found and 'host_tx_ready' in dict(port_info):
@@ -981,8 +1014,7 @@ def get_host_tx_status(self, lport):
     def get_port_admin_status(self, lport):
         admin_status = 'down'
 
-        asic_index = self.port_mapping.get_asic_id_for_logical_port(lport)
-        cfg_port_tbl = self.xcvr_table_helper.get_cfg_port_tbl(asic_index)
+        cfg_port_tbl = self.xcvr_table_helper.get_cfg_port_tbl(self.get_asic_id(lport))
 
         found, port_info = cfg_port_tbl.get(lport)
         if found:
@@ -1060,8 +1092,7 @@ def post_port_active_apsel_to_db(self, api, lport, host_lanes_mask, reset_apsel=
                 tuple_list.append(('host_lane_count', 'N/A'))
                 tuple_list.append(('media_lane_count', 'N/A'))
 
-        asic_index = self.port_mapping.get_asic_id_for_logical_port(lport)
-        intf_tbl = self.xcvr_table_helper.get_intf_tbl(asic_index)
+        intf_tbl = self.xcvr_table_helper.get_intf_tbl(self.get_asic_id(lport))
         if not intf_tbl:
             helper_logger.log_warning("Active ApSel db update: TRANSCEIVER_INFO table not found for {}".format(lport))
             return
@@ -1095,16 +1126,6 @@ def wait_for_port_config_done(self, namespace):
                 break
 
     def task_worker(self):
-        self.xcvr_table_helper = XcvrTableHelper(self.namespaces)
-
-        self.log_notice("Waiting for PortConfigDone...")
-        for namespace in self.namespaces:
-            self.wait_for_port_config_done(namespace)
-
-        logical_port_list = self.port_mapping.logical_port_list
-        for lport in logical_port_list:
-            self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_UNKNOWN)
-
         is_fast_reboot = is_fast_reboot_enabled()
 
         # APPL_DB for CONFIG updates, and STATE_DB for insertion/removal
@@ -1123,7 +1144,7 @@ def task_worker(self):
                 if lport not in self.port_dict:
                     continue
 
-                state = get_cmis_state_from_state_db(lport, self.xcvr_table_helper.get_status_tbl(self.port_mapping.get_asic_id_for_logical_port(lport)))
+                state = get_cmis_state_from_state_db(lport, self.xcvr_table_helper.get_status_tbl(self.get_asic_id(lport)))
                 if state in CMIS_TERMINAL_STATES or state == CMIS_STATE_UNKNOWN:
                     if state != CMIS_STATE_READY:
                         self.port_dict[lport]['appl'] = 0
@@ -1457,6 +1478,14 @@ def run(self):
             return
 
         try:
+
+            self.log_notice("Waiting for PortConfigDone...")
+            for namespace in self.namespaces:
+                self.wait_for_port_config_done(namespace)
+
+            for lport in self.port_dict.keys():
+                self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_UNKNOWN)
+
             self.task_worker()
         except Exception as e:
             helper_logger.log_error("Exception occured at {} thread due to {}".format(threading.current_thread().getName(), repr(e)))
@@ -2138,20 +2167,13 @@ def init(self):
         # Load new platform api class
         try:
             import sonic_platform.platform
-            import sonic_platform_base.sonic_sfp.sfputilhelper
             platform_chassis = sonic_platform.platform.Platform().get_chassis()
             self.log_info("chassis loaded {}".format(platform_chassis))
-            # we have to make use of sfputil for some features
-            # even though when new platform api is used for all vendors.
-            # in this sense, we treat it as a part of new platform api.
-            # we have already moved sfputil to sonic_platform_base
-            # which is the root of new platform api.
-            platform_sfputil = sonic_platform_base.sonic_sfp.sfputilhelper.SfpUtilHelper()
         except Exception as e:
             self.log_warning("Failed to load chassis due to {}".format(repr(e)))
 
         # Load platform specific sfputil class
-        if platform_chassis is None or platform_sfputil is None:
+        if platform_chassis is None:
             try:
                 platform_sfputil = self.load_platform_util(PLATFORM_SPECIFIC_MODULE_NAME, PLATFORM_SPECIFIC_CLASS_NAME)
             except Exception as e:


Reviews:
user: login: prgeor
body: @ishidawataru There is not enough information for me to understand the problem. As long as CONFIG_DB port table has the right number of lanes and Speed specified, Xcvrd must be able to select the best matching application from the list of advertised application by the module. Do you see any issue in DPB port deletion path or DPB port creation path?
state: CHANGES_REQUESTED
author_association: COLLABORATOR
submitted_at: 2024-07-05T16:55:05Z
user: login: longhuan-cisco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-08-07T19:07:19Z
user: login: longhuan-cisco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-08-07T20:54:06Z
user: login: longhuan-cisco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-08-07T21:30:37Z
user: login: mihirpat1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-08-14T21:33:32Z
user: login: ishidawataru
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-09-13T07:21:06Z
user: login: ishidawataru
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-09-17T01:58:19Z
user: login: ishidawataru
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-09-17T01:59:06Z
user: login: chiourung
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-09-26T07:05:46Z
user: login: ishidawataru
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-09-26T07:41:28Z
user: login: ishidawataru
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2024-10-03T03:44:53Z
user: login: longhuan-cisco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T20:50:19Z
user: login: prgeor
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-25T03:20:29Z
user: login: ishidawataru
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T04:03:03Z
user: login: longhuan-cisco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-26T22:55:50Z
user: login: longhuan-cisco
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-26T22:56:07Z
user: login: prgeor
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-27T03:06:00Z

Review comments:
diff_hunk: @@ -896,7 +897,13 @@ def on_port_update_event(self, port_change_event):
 
             self.force_cmis_reinit(lport, 0)
         else:
+            # PORT_DEL event for the same lport happens 3 times because
+            # we are subscribing to CONFIG_DB, STATE_DB|TRANSCEIVER_INFO, and STATE_DB|PORT_TABLE.
+            # We only handle the first one and ignore the rest.
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: longhuan-cisco
body: Instead of introducing a new attribute `deleted_ports` and its corresponding update logic, you may be able to simply rely on port_change_event's existing `db_name`(/`table_name`) to only proceed to do `update_port_transceiver_status_table_sw_cmis_state` for the case of STATE_DB|TRANSCEIVER_INFO (since `update_port_transceiver_status_table_sw_cmis_state` is updating SW state of transceiver for the case of transceiver removal)?
https://github.com/sonic-net/sonic-platform-daemons/blob/bf865c6b711833347d3c57e9d84cd366bcd1b776/sonic-xcvrd/xcvrd/xcvrd_utilities/port_event_helper.py#L30-L31


created_at: 2024-08-07T19:07:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1621,6 +1623,17 @@ def handle_cmis_state_machine(self, lport, info, is_fast_reboot):
             log_exception_traceback()
             self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_FAILED)
 
+    def do_task(self, is_fast_reboot):
+        for lport, info in self.port_dict.items():
+            if self.task_stopping_event.is_set():
+                break
+
+            self.handle_cmis_state_machine(lport, info, is_fast_reboot)
+
+        for event in self.deleted_ports.values():
+            self.port_dict.pop(event.port_name)
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: longhuan-cisco
body: Would it make the flow/logic simpler by just directly deleting `self.port_dict[lport]` entry at the PORT_DEL event of CONFIG_DB in `on_port_update_event()`?
created_at: 2024-08-07T20:54:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1300,354 +1302,360 @@ def wait_for_port_config_done(self, namespace):
             if key in ["PortConfigDone", "PortInitDone"]:
                 break
 
-    def task_worker(self):
-        self.xcvr_table_helper = XcvrTableHelper(self.namespaces)
-
-        self.log_notice("Waiting for PortConfigDone...")
-        for namespace in self.namespaces:
-            self.wait_for_port_config_done(namespace)
-
-        logical_port_list = self.port_mapping.logical_port_list
-        for lport in logical_port_list:
-            self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_UNKNOWN)
-
-        is_fast_reboot = is_fast_reboot_enabled()
-
-        # APPL_DB for CONFIG updates, and STATE_DB for insertion/removal
-        port_change_observer = PortChangeObserver(self.namespaces, helper_logger,
-                                                  self.task_stopping_event,
-                                                  self.on_port_update_event)
-
-        while not self.task_stopping_event.is_set():
-            # Handle port change event from main thread
-            port_change_observer.handle_port_update_event()
-
-            for lport, info in self.port_dict.items():
-                if self.task_stopping_event.is_set():
-                    break
-
-                if lport not in self.port_dict:
-                    continue
-
-                state = get_cmis_state_from_state_db(lport, self.xcvr_table_helper.get_status_tbl(self.port_mapping.get_asic_id_for_logical_port(lport)))
-                if state in CMIS_TERMINAL_STATES or state == CMIS_STATE_UNKNOWN:
-                    if state != CMIS_STATE_READY:
-                        self.port_dict[lport]['appl'] = 0
-                        self.port_dict[lport]['host_lanes_mask'] = 0
-                    continue
+    def handle_cmis_state_machine(self, lport, info, is_fast_reboot):
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: longhuan-cisco
body: The change here seems to be refactoring today's cmis_mgr's `task_worker` function by splitting it into sub functions. I'm assuming the flow/logic of cmis_mgr's task_worker doesn't get changed, it's mostly just splitting functions. But this makes the number of LOC (Line of change) very big. I feel It's a little bit difficult for other people to distinguish the actual change of DPB fix from the entire change set of this PR.

Thus, I feel it might be better to put this refactoring change set in a separate PR. It will also be less of a work for release mgr to revert in case of anything broken. Just my 2 cents, not something must-have.
Not sure how @mihirpat1 or Prince feel about this.
created_at: 2024-08-07T21:30:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1300,354 +1302,360 @@ def wait_for_port_config_done(self, namespace):
             if key in ["PortConfigDone", "PortInitDone"]:
                 break
 
-    def task_worker(self):
-        self.xcvr_table_helper = XcvrTableHelper(self.namespaces)
-
-        self.log_notice("Waiting for PortConfigDone...")
-        for namespace in self.namespaces:
-            self.wait_for_port_config_done(namespace)
-
-        logical_port_list = self.port_mapping.logical_port_list
-        for lport in logical_port_list:
-            self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_UNKNOWN)
-
-        is_fast_reboot = is_fast_reboot_enabled()
-
-        # APPL_DB for CONFIG updates, and STATE_DB for insertion/removal
-        port_change_observer = PortChangeObserver(self.namespaces, helper_logger,
-                                                  self.task_stopping_event,
-                                                  self.on_port_update_event)
-
-        while not self.task_stopping_event.is_set():
-            # Handle port change event from main thread
-            port_change_observer.handle_port_update_event()
-
-            for lport, info in self.port_dict.items():
-                if self.task_stopping_event.is_set():
-                    break
-
-                if lport not in self.port_dict:
-                    continue
-
-                state = get_cmis_state_from_state_db(lport, self.xcvr_table_helper.get_status_tbl(self.port_mapping.get_asic_id_for_logical_port(lport)))
-                if state in CMIS_TERMINAL_STATES or state == CMIS_STATE_UNKNOWN:
-                    if state != CMIS_STATE_READY:
-                        self.port_dict[lport]['appl'] = 0
-                        self.port_dict[lport]['host_lanes_mask'] = 0
-                    continue
+    def handle_cmis_state_machine(self, lport, info, is_fast_reboot):
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: mihirpat1
body: @longhuan-cisco Agreed that splitting the current changeset into 2 separate PRs will be more helpful.
created_at: 2024-08-14T21:33:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1300,354 +1302,360 @@ def wait_for_port_config_done(self, namespace):
             if key in ["PortConfigDone", "PortInitDone"]:
                 break
 
-    def task_worker(self):
-        self.xcvr_table_helper = XcvrTableHelper(self.namespaces)
-
-        self.log_notice("Waiting for PortConfigDone...")
-        for namespace in self.namespaces:
-            self.wait_for_port_config_done(namespace)
-
-        logical_port_list = self.port_mapping.logical_port_list
-        for lport in logical_port_list:
-            self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_UNKNOWN)
-
-        is_fast_reboot = is_fast_reboot_enabled()
-
-        # APPL_DB for CONFIG updates, and STATE_DB for insertion/removal
-        port_change_observer = PortChangeObserver(self.namespaces, helper_logger,
-                                                  self.task_stopping_event,
-                                                  self.on_port_update_event)
-
-        while not self.task_stopping_event.is_set():
-            # Handle port change event from main thread
-            port_change_observer.handle_port_update_event()
-
-            for lport, info in self.port_dict.items():
-                if self.task_stopping_event.is_set():
-                    break
-
-                if lport not in self.port_dict:
-                    continue
-
-                state = get_cmis_state_from_state_db(lport, self.xcvr_table_helper.get_status_tbl(self.port_mapping.get_asic_id_for_logical_port(lport)))
-                if state in CMIS_TERMINAL_STATES or state == CMIS_STATE_UNKNOWN:
-                    if state != CMIS_STATE_READY:
-                        self.port_dict[lport]['appl'] = 0
-                        self.port_dict[lport]['host_lanes_mask'] = 0
-                    continue
+    def handle_cmis_state_machine(self, lport, info, is_fast_reboot):
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: ishidawataru
body: @longhuan-cisco @mihirpat1 I'm sorry for the late response. Thanks for the suggestion.

I'll move the refactoring commit to a separate PR and only include the DPB fix in this PR.
created_at: 2024-09-13T07:21:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -896,7 +897,13 @@ def on_port_update_event(self, port_change_event):
 
             self.force_cmis_reinit(lport, 0)
         else:
+            # PORT_DEL event for the same lport happens 3 times because
+            # we are subscribing to CONFIG_DB, STATE_DB|TRANSCEIVER_INFO, and STATE_DB|PORT_TABLE.
+            # We only handle the first one and ignore the rest.
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: ishidawataru
body: https://github.com/sonic-net/sonic-platform-daemons/pull/500/files#diff-fc70c43977a1ce57f3a6e9d7d09cb7ea5204f52b3adf3162936205e5680efb2fR899-R902

@longhuan-cisco  Thank you for the suggestion. I was able to fix the issue without introducing `deleted_ports`.

I initially added `deleted_ports` because the CmisManagerTask handles port events in two stages.

The first stage is done in `on_port_update_event` via   [`port_change_observer.handle_port_update_event()`](https://github.com/sonic-net/sonic-platform-daemons/pull/500/files#diff-fc70c43977a1ce57f3a6e9d7d09cb7ea5204f52b3adf3162936205e5680efb2fR1313) and the second stage is done after returning from `port_change_observer.handle_port_update_event()` by [`iterating over port_dict`](https://github.com/sonic-net/sonic-platform-daemons/pull/500/files#diff-fc70c43977a1ce57f3a6e9d7d09cb7ea5204f52b3adf3162936205e5680efb2fR1315).

If we delete the port info from `port_dict` in the first stage, the second stage cannot handle the port.
By introducing `deleted_ports`, the original code was deleting the port info from `port_dict` after finishing the second stage.
However, it looks like currently, the second stage is not doing anything for the removed port. so deleting the port info in the first stage is safe.

Though this works, I think the current two-stage design is bug-prone, because we might want to add some handling(e.g. de-initialize) for the removed port in the second stage.
If you agree, I'd like to refactor the current design in a different PR.
created_at: 2024-09-17T01:58:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1621,6 +1623,17 @@ def handle_cmis_state_machine(self, lport, info, is_fast_reboot):
             log_exception_traceback()
             self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_FAILED)
 
+    def do_task(self, is_fast_reboot):
+        for lport, info in self.port_dict.items():
+            if self.task_stopping_event.is_set():
+                break
+
+            self.handle_cmis_state_machine(lport, info, is_fast_reboot)
+
+        for event in self.deleted_ports.values():
+            self.port_dict.pop(event.port_name)
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: ishidawataru
body: Done.

I explained the background of this change [here](https://github.com/sonic-net/sonic-platform-daemons/pull/500#discussion_r1762172401)


created_at: 2024-09-17T01:59:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -896,8 +895,13 @@ def on_port_update_event(self, port_change_event):
                 self.port_dict[lport]['subport'] = int(port_change_event.port_dict['subport'])
 
             self.force_cmis_reinit(lport, 0)
-        else:
+
+        # PORT_DEL event for the same lport happens 3 times because
+        # we are subscribing to CONFIG_DB, STATE_DB|TRANSCEIVER_INFO, and STATE_DB|PORT_TABLE.
+        # To only handle the first one, check if the lport exists in the port_dict.
+        elif lport in self.port_dict:
             self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+            self.port_dict.pop(lport)
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: chiourung
body: If a transceiver is removed (not doing port breakout), the port would be removed from port_dict. When the transceiver is reattached to the port, there would be no port-related information in port_dict. I think the port should only be removed from port_dict after receiving a delete CONFIG_DB|PORT event and update cmis state after receiving a delete STATE_DB|TRANSCEIVER_INFO event.
created_at: 2024-09-26T07:05:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -896,8 +895,13 @@ def on_port_update_event(self, port_change_event):
                 self.port_dict[lport]['subport'] = int(port_change_event.port_dict['subport'])
 
             self.force_cmis_reinit(lport, 0)
-        else:
+
+        # PORT_DEL event for the same lport happens 3 times because
+        # we are subscribing to CONFIG_DB, STATE_DB|TRANSCEIVER_INFO, and STATE_DB|PORT_TABLE.
+        # To only handle the first one, check if the lport exists in the port_dict.
+        elif lport in self.port_dict:
             self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+            self.port_dict.pop(lport)
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: ishidawataru
body: Thank you for the comment. You're right. I'll update the code as you suggested. 
created_at: 2024-09-26T07:41:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -896,8 +895,13 @@ def on_port_update_event(self, port_change_event):
                 self.port_dict[lport]['subport'] = int(port_change_event.port_dict['subport'])
 
             self.force_cmis_reinit(lport, 0)
-        else:
+
+        # PORT_DEL event for the same lport happens 3 times because
+        # we are subscribing to CONFIG_DB, STATE_DB|TRANSCEIVER_INFO, and STATE_DB|PORT_TABLE.
+        # To only handle the first one, check if the lport exists in the port_dict.
+        elif lport in self.port_dict:
             self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+            self.port_dict.pop(lport)
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: ishidawataru
body: @chiourung I fixed the issue with this commit. https://github.com/sonic-net/sonic-platform-daemons/pull/500/commits/954e9cb8f39554c2770c030c796c39feb667c194

Could you take a look?
created_at: 2024-10-03T03:44:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2138,20 +2167,13 @@ def init(self):
         # Load new platform api class
         try:
             import sonic_platform.platform
-            import sonic_platform_base.sonic_sfp.sfputilhelper
             platform_chassis = sonic_platform.platform.Platform().get_chassis()
             self.log_info("chassis loaded {}".format(platform_chassis))
-            # we have to make use of sfputil for some features
-            # even though when new platform api is used for all vendors.
-            # in this sense, we treat it as a part of new platform api.
-            # we have already moved sfputil to sonic_platform_base
-            # which is the root of new platform api.
-            platform_sfputil = sonic_platform_base.sonic_sfp.sfputilhelper.SfpUtilHelper()
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: longhuan-cisco
body: Any particular reason to remove the instantiation of `platform_sfputil`?

I saw `platform_sfputil` might be used later in all the `_wrapper_xxx` functions(e.g. `_wrapper_get_presence`), in the case of `platform_chassis` is not `None` and `NotImplementedError` gets raised for the corresponding `platform_chassis.get_sfp(physical_port).xxx` API.
created_at: 2025-03-20T18:40:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -681,8 +680,45 @@ def on_port_update_event(self, port_change_event):
                 self.port_dict[lport]['subport'] = int(port_change_event.port_dict['subport'])
 
             self.force_cmis_reinit(lport, 0)
-        else:
-            self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+
+        elif port_change_event.event_type == port_change_event.PORT_DEL:
+            # In handling the DEL event, the following two scenarios must be considered:
+            # 1. PORT_DEL event due to transceiver plug-out
+            # 2. PORT_DEL event due to Dynamic Port Breakout (DPB)
+            #
+            # Scenario 1 is simple, as only a STATE_DB|TRANSCEIVER_INFO PORT_DEL event occurs,
+            # so we just need to set SW_CMIS_STATE to CMIS_STATE_REMOVED.
+            #
+            # Scenario 2 is a bit more complex. First, for the port(s) before DPB, a CONFIG_DB|PORT PORT_DEL
+            # and a STATE_DB|PORT_TABLE PORT_DEL event occur. Next, for the port(s) after DPB,
+            # a CONFIG_DB|PORT PORT_SET and a STATE_DB|PORT_TABLE PORT_SET event occur.
+            # After that (after a short delay), a STATE_DB|TRANSCEIVER_INFO PORT_DEL event
+            # occurs for the port(s) before DPB, and finally, a STATE_DB|TRANSCEIVER_INFO
+            # PORT_SET event occurs for the port(s) after DPB.
+            #
+            # Below is the event sequence when configuring Ethernet0 from "2x200G" to "1x400G"
+            # (based on actual logs).
+            #
+            # 1.  SET Ethernet0 CONFIG_DB|PORT
+            # 2.  DEL Ethernet2 CONFIG_DB|PORT
+            # 3.  DEL Ethernet0 CONFIG_DB|PORT
+            # 4.  DEL Ethernet0 STATE_DB|PORT_TABLE
+            # 5.  DEL Ethernet2 STATE_DB|PORT_TABLE
+            # 6.  SET Ethernet0 CONFIG_DB|PORT
+            # 7.  SET Ethernet0 STATE_DB|PORT_TABLE
+            # 8.  SET Ethernet0 STATE_DB|PORT_TABLE
+            # 9.  DEL Ethernet2 STATE_DB|TRANSCEIVER_INFO
+            # 10. DEL Ethernet0 STATE_DB|TRANSCEIVER_INFO
+            # 11. SET Ethernet0 STATE_DB|TRANSCEIVER_INFO
+            #
+            # To handle both scenarios, if the lport exists in port_dict for any DEL EVENT,
+            # set SW_CMIS_STATE to REMOVED. Additionally, for DEL EVENTS from CONFIG_DB due to DPB,
+            # remove the lport from port_dict.
+            if lport in self.port_dict:
+                self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+
+            if port_change_event.db_name == 'CONFIG_DB' and port_change_event.table_name == 'PORT':
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: longhuan-cisco
body: would it make the code more robust if we also put line 720-721 protected by line 717 `if lport in self.port_dict:` to avoid `self.port_dict.pop(lport)` throwing KeyError in the case of `lport` not present in port_dict? Though not sure if this rare case can actually happen.
created_at: 2025-03-20T20:37:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2138,20 +2167,13 @@ def init(self):
         # Load new platform api class
         try:
             import sonic_platform.platform
-            import sonic_platform_base.sonic_sfp.sfputilhelper
             platform_chassis = sonic_platform.platform.Platform().get_chassis()
             self.log_info("chassis loaded {}".format(platform_chassis))
-            # we have to make use of sfputil for some features
-            # even though when new platform api is used for all vendors.
-            # in this sense, we treat it as a part of new platform api.
-            # we have already moved sfputil to sonic_platform_base
-            # which is the root of new platform api.
-            platform_sfputil = sonic_platform_base.sonic_sfp.sfputilhelper.SfpUtilHelper()
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: prgeor
body: @longhuan-cisco 
1. We want to get rid of _wrapper_xxx  as it was applicable when we needed to support platforms using API < v2.0. All platforms by now should be moving to Platform API v2.0.
2. sonic_platform_base.sonic_sfp is already deprecated. 
created_at: 2025-03-25T03:20:29Z
author_association: COLLABORATOR
diff_hunk: @@ -681,8 +680,45 @@ def on_port_update_event(self, port_change_event):
                 self.port_dict[lport]['subport'] = int(port_change_event.port_dict['subport'])
 
             self.force_cmis_reinit(lport, 0)
-        else:
-            self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+
+        elif port_change_event.event_type == port_change_event.PORT_DEL:
+            # In handling the DEL event, the following two scenarios must be considered:
+            # 1. PORT_DEL event due to transceiver plug-out
+            # 2. PORT_DEL event due to Dynamic Port Breakout (DPB)
+            #
+            # Scenario 1 is simple, as only a STATE_DB|TRANSCEIVER_INFO PORT_DEL event occurs,
+            # so we just need to set SW_CMIS_STATE to CMIS_STATE_REMOVED.
+            #
+            # Scenario 2 is a bit more complex. First, for the port(s) before DPB, a CONFIG_DB|PORT PORT_DEL
+            # and a STATE_DB|PORT_TABLE PORT_DEL event occur. Next, for the port(s) after DPB,
+            # a CONFIG_DB|PORT PORT_SET and a STATE_DB|PORT_TABLE PORT_SET event occur.
+            # After that (after a short delay), a STATE_DB|TRANSCEIVER_INFO PORT_DEL event
+            # occurs for the port(s) before DPB, and finally, a STATE_DB|TRANSCEIVER_INFO
+            # PORT_SET event occurs for the port(s) after DPB.
+            #
+            # Below is the event sequence when configuring Ethernet0 from "2x200G" to "1x400G"
+            # (based on actual logs).
+            #
+            # 1.  SET Ethernet0 CONFIG_DB|PORT
+            # 2.  DEL Ethernet2 CONFIG_DB|PORT
+            # 3.  DEL Ethernet0 CONFIG_DB|PORT
+            # 4.  DEL Ethernet0 STATE_DB|PORT_TABLE
+            # 5.  DEL Ethernet2 STATE_DB|PORT_TABLE
+            # 6.  SET Ethernet0 CONFIG_DB|PORT
+            # 7.  SET Ethernet0 STATE_DB|PORT_TABLE
+            # 8.  SET Ethernet0 STATE_DB|PORT_TABLE
+            # 9.  DEL Ethernet2 STATE_DB|TRANSCEIVER_INFO
+            # 10. DEL Ethernet0 STATE_DB|TRANSCEIVER_INFO
+            # 11. SET Ethernet0 STATE_DB|TRANSCEIVER_INFO
+            #
+            # To handle both scenarios, if the lport exists in port_dict for any DEL EVENT,
+            # set SW_CMIS_STATE to REMOVED. Additionally, for DEL EVENTS from CONFIG_DB due to DPB,
+            # remove the lport from port_dict.
+            if lport in self.port_dict:
+                self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+
+            if port_change_event.db_name == 'CONFIG_DB' and port_change_event.table_name == 'PORT':
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: ishidawataru
body: Sure, I'll fix it as you mentioned.
I just thought if the case is not expected to happen, it's better to treat it as an error instead of silently ignoreing it.
By putting line 720-721 protected by line 717, we won't be able to see if this case is happening or not.
created_at: 2025-03-25T04:03:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -681,8 +680,45 @@ def on_port_update_event(self, port_change_event):
                 self.port_dict[lport]['subport'] = int(port_change_event.port_dict['subport'])
 
             self.force_cmis_reinit(lport, 0)
-        else:
-            self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+
+        elif port_change_event.event_type == port_change_event.PORT_DEL:
+            # In handling the DEL event, the following two scenarios must be considered:
+            # 1. PORT_DEL event due to transceiver plug-out
+            # 2. PORT_DEL event due to Dynamic Port Breakout (DPB)
+            #
+            # Scenario 1 is simple, as only a STATE_DB|TRANSCEIVER_INFO PORT_DEL event occurs,
+            # so we just need to set SW_CMIS_STATE to CMIS_STATE_REMOVED.
+            #
+            # Scenario 2 is a bit more complex. First, for the port(s) before DPB, a CONFIG_DB|PORT PORT_DEL
+            # and a STATE_DB|PORT_TABLE PORT_DEL event occur. Next, for the port(s) after DPB,
+            # a CONFIG_DB|PORT PORT_SET and a STATE_DB|PORT_TABLE PORT_SET event occur.
+            # After that (after a short delay), a STATE_DB|TRANSCEIVER_INFO PORT_DEL event
+            # occurs for the port(s) before DPB, and finally, a STATE_DB|TRANSCEIVER_INFO
+            # PORT_SET event occurs for the port(s) after DPB.
+            #
+            # Below is the event sequence when configuring Ethernet0 from "2x200G" to "1x400G"
+            # (based on actual logs).
+            #
+            # 1.  SET Ethernet0 CONFIG_DB|PORT
+            # 2.  DEL Ethernet2 CONFIG_DB|PORT
+            # 3.  DEL Ethernet0 CONFIG_DB|PORT
+            # 4.  DEL Ethernet0 STATE_DB|PORT_TABLE
+            # 5.  DEL Ethernet2 STATE_DB|PORT_TABLE
+            # 6.  SET Ethernet0 CONFIG_DB|PORT
+            # 7.  SET Ethernet0 STATE_DB|PORT_TABLE
+            # 8.  SET Ethernet0 STATE_DB|PORT_TABLE
+            # 9.  DEL Ethernet2 STATE_DB|TRANSCEIVER_INFO
+            # 10. DEL Ethernet0 STATE_DB|TRANSCEIVER_INFO
+            # 11. SET Ethernet0 STATE_DB|TRANSCEIVER_INFO
+            #
+            # To handle both scenarios, if the lport exists in port_dict for any DEL EVENT,
+            # set SW_CMIS_STATE to REMOVED. Additionally, for DEL EVENTS from CONFIG_DB due to DPB,
+            # remove the lport from port_dict.
+            if lport in self.port_dict:
+                self.update_port_transceiver_status_table_sw_cmis_state(lport, CMIS_STATE_REMOVED)
+
+            if port_change_event.db_name == 'CONFIG_DB' and port_change_event.table_name == 'PORT':
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: longhuan-cisco
body: I see your concern, it makes sense to let the exception be thrown to not ignore error. 

The case might not always be an error case. One non-error case I can think of is: If xcvrd receives duplicate DEL events (though it's not very unlikely using today's [PortChangeObserver](https://github.com/sonic-net/sonic-platform-daemons/blob/ba47670352b1778ad3732d27c646e9a1f4b9c49b/sonic-xcvrd/xcvrd/xcvrd_utilities/port_event_helper.py#L155)), protecting it will help xcvrd survive.

It's your choice whether to protect it or leave it as is. I'm fine with either option.
created_at: 2025-03-26T22:17:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2138,20 +2167,13 @@ def init(self):
         # Load new platform api class
         try:
             import sonic_platform.platform
-            import sonic_platform_base.sonic_sfp.sfputilhelper
             platform_chassis = sonic_platform.platform.Platform().get_chassis()
             self.log_info("chassis loaded {}".format(platform_chassis))
-            # we have to make use of sfputil for some features
-            # even though when new platform api is used for all vendors.
-            # in this sense, we treat it as a part of new platform api.
-            # we have already moved sfputil to sonic_platform_base
-            # which is the root of new platform api.
-            platform_sfputil = sonic_platform_base.sonic_sfp.sfputilhelper.SfpUtilHelper()
path: sonic-xcvrd/xcvrd/xcvrd.py
user: login: longhuan-cisco
body: got it, thanks for the info.
created_at: 2025-03-26T22:55:43Z
author_association: CONTRIBUTOR

PR comments:
user: login: linux-foundation-easycla[bot]
created_at: 2024-06-14T02:53:13Z
body: <a href="https://easycla.lfx.linuxfoundation.org/#/?version=2"><img src="https://s3.amazonaws.com/cla-project-logo-prod/cla-signed.svg" alt="CLA Signed" align="left" height="28" width="328" ></a><br/><br />The committers listed above are authorized under a signed CLA.<ul><li>:white_check_mark: login: ishidawataru / name: Wataru Ishida  (7639be71212c69fb807f1f8d1434d0017d5e311a, 824d0594074b3c0a1ffd435ee4ee863d08b25e82, 33566ec63496856acb83a54bd83217748b752aea, 95ae842f65441d7c713c40d68f71a9139322160e)</li></ul><!-- Date Modified: 2025-03-15 01:26:50.132361 -->
author_association: NONE
user: login: mihirpat1
created_at: 2024-06-18T18:33:33Z
body: @ishidawataru Can you please fix the code coverage check failure?
author_association: CONTRIBUTOR
user: login: ishidawataru
created_at: 2024-06-19T10:16:16Z
body: @mihirpat1 I'm still communicating with the original reporter of this issue to check the fix actually works. After that I'll fix the code coverage failure.
author_association: CONTRIBUTOR
user: login: ishidawataru
created_at: 2024-06-25T09:51:11Z
body: @mihirpat1 Confirmed that the fix resolved the issue. Also, I fixed the coverage test failure. Please review.
author_association: CONTRIBUTOR
user: login: ishidawataru
created_at: 2024-07-06T01:49:47Z
body: @prgeor This PR fixes https://github.com/sonic-net/sonic-buildimage/issues/18893. It is not about the application selection. The current `CmisManagerTask` implementation doesn't delete the port info from `port_dict` when ports are deleted due to DPB configuration, which causes a crash.
author_association: CONTRIBUTOR
user: login: ishidawataru
created_at: 2024-09-17T02:03:19Z
body: @longhuan-cisco @mihirpat1 I updated the PR based on @longhuan-cisco 's suggestion. Please review.

I used force-push and removed the original commits. I backed up the original branch [here](https://github.com/ishidawataru/sonic-platform-daemons/tree/fix-xcvrd-dpb-bak).
author_association: CONTRIBUTOR
user: login: ishidawataru
created_at: 2024-10-29T06:11:41Z
body: @prgeor Hi, just a gentle ping to check if I need to do something to get this PR merged.
author_association: CONTRIBUTOR
user: login: mssonicbld
created_at: 2025-03-15T01:26:54Z
body: /azp run
author_association: COLLABORATOR
user: login: azure-pipelines[bot]
created_at: 2025-03-15T01:27:02Z
body: <samp>
Azure Pipelines successfully started running 1 pipeline(s).<br>

</samp>
author_association: NONE
user: login: ishidawataru
created_at: 2025-03-15T01:49:36Z
body: @prgeor I fixed the conflicts. Please take a look.
author_association: CONTRIBUTOR
user: login: prgeor
created_at: 2025-03-18T05:34:33Z
body: @ishidawataru will you be able to test this on a real hardware with breakout enabled and do insertion and removal of the optics?
author_association: COLLABORATOR
user: login: ishidawataru
created_at: 2025-03-20T04:54:29Z
body: @prgeor Sorry, I don’t have access to real hardware to test this PR. I performed insertion and removal testing using the xcvr emulator in last October. I can redo those tests and provide the logs if needed.
author_association: CONTRIBUTOR
user: login: longhuan-cisco
created_at: 2025-03-26T22:54:51Z
body: @prgeor 

I have done OIR tests with the latest diff. 

Tried below sequence on 400G AOC:
1. Move from non-breakout mode (1x400G) to 4x100G breakout mode via DPB CLI
2. Perform OIR
3. Move back to non-breakout mode (1x400G) via DPB CLI
4. Perform OIR

No xcvrd crash. link comes up fine with module ready state. `show interface` CLI shows correct interface entries.
author_association: CONTRIBUTOR
user: login: Keshavg-marvell
created_at: 2025-05-28T11:49:42Z
body: @ishidawataru @prgeor Can you please backport this change to 202411?
author_association: CONTRIBUTOR