Only add response_set if it hasn't already been added