Unfortunately the admin_bar_menu is plugin territory and will have to be removed as well.