Generative models are often evaluated for their ability to produce novel and diverse outputs:

  • Creativity: How original are the generated outputs? For text models, this might involve generating unique and creative responses to open-ended prompts. For image generation, creativity can be assessed by how well the model can produce diverse and innovative images from a given concept or style.
  • Diversity: A good generative model should produce a variety of responses or outputs, not just repeat the same patterns. This is especially important in tasks like text generation, where diversity ensures that the model doesn't just regurgitate a narrow set of phrases or ideas.

Evaluation Metrics for Creativity and Diversity:

  • Intra-Output Diversity: This measures the variability in the outputs generated for the same input or prompt. A highly diverse model will generate varied and different responses for similar queries.
  • Novelty Score: How novel are the outputs? Are they new and different, or are they mere recombination of known patterns?