To remove someone from your private story, go to the story settings, deselect the friend you want to remove, and save the changes.