Simplify listeners. Remove weak sets for now.