[Server] Fix a crash case when the server stops

Signed-off-by: Regis Merlino <regis.merlino@intel.com>