add_meta_boxes_event is the right hook for removing event meta boxes