Remove entries from the callers list properly

A standard SSA bug using the wrong version of a variable was preventing
the removal of {Worker, Ref} entries in the callers list. This corrects
the issue by using the right variable version.

BugzId: 17731