You need to check that the user who left is the same as the user you are currently displaying. The same check applies for when a user is added.