Fix for assigned but no longer associated label groups