Just delete inactive fragments if they're empty