  Crowdsourcing can collect many diverse ideas by prompting ideators
individually, but this can generate redundant ideas. Prior methods reduce
redundancy by presenting peers' ideas or peer-proposed prompts, but these
require much human coordination. We introduce Directed Diversity, an automatic
prompt selection approach that leverages language model embedding distances to
maximize diversity. Ideators can be directed towards diverse prompts and away
from prior ideas, thus improving their collective creativity. Since there are
diverse metrics of diversity, we present a Diversity Prompting Evaluation
Framework consolidating metrics from several research disciplines to analyze
along the ideation chain - prompt selection, prompt creativity, prompt-ideation
mediation, and ideation creativity. Using this framework, we evaluated Directed
Diversity in a series of a simulation study and four user studies for the use
case of crowdsourcing motivational messages to encourage physical activity. We
show that automated diverse prompting can variously improve collective
creativity across many nuanced metrics of diversity.
