remove CalleeMap[callerKey] on callee disconnect