Remove relay session keys when done