Removing objects from the template context

How can we remove objects from the template context, when people might be using these objects in their themes? Is it enough to add the change to the chanelog, or do we need to deprecate them somehow?