#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Fri Oct 11 18:53:59 2024

@author: legalchain
"""
default_prompt = """
Evaluate the given dataset card based on these mandatory criteria:
1. Purpose and Applications: Clearly articulates the dataset's purpose and potential real-world applications.
2. Unique Value: Explicitly states what makes this dataset unique or valuable compared to others.
3. Content Description: Provides detailed information about the actual data content, not just structure.
4. Completeness: Covers dataset size, data distribution, collection methodology, and any preprocessing steps.
5. Limitations and Biases: Discusses any known limitations, biases, or ethical considerations.
6. Human Curation: Shows evidence of human-added insights, context, or valuable information beyond auto-generated content.

Rate the card from 1 to 5, with half-points allowed (e.g., 2.5):
Dataset Card to Evaluate: {readme_content}
"""

prompt_enhanced = """Evaluate the given dataset card based on these mandatory criteria:

# Steps

1. **Data Origin**: Evaluate the description of where the data originates from. Check if it includes details like data source, collection methods, and any preprocessing steps.
 
2. **Usage**: Assess how the model usage is outlined in the card. Consider the intended applications, recommended scenarios, and potential misuse cases.

3. **Biases**: Review the section addressing biases. Determine if it identifies possible biases in the data or the model's output and how these are mitigated or acknowledged.

4. **Performance**: Check for clear information on the model's performance, including metrics, benchmarks, and testing conditions.

5. **Limitations**: Look for a discussion of the model's limitations, noting any restrictions on the types of data or contexts it should be used in.

6. **Ethics and Safety**: Evaluate any ethical considerations and safety precautions mentioned, focusing on steps taken to ensure responsible use of the model.

7. **Transparency and Explainability**: Determine if the card includes details that enhance transparency, such as architecture, algorithms used, and explainability measures.

# Output Format

Provide a detailed paragraph for each aspect (data origin, usage, biases, etc.) specifying the strengths and weaknesses observed in the model card. Summarize with a concluding statement about the overall quality and completeness of the model card.

# Examples

- **Data Origin**: The model card should comprehensively detail the origin of data, including specific datasets used, any licensing or ethical considerations in data collection, and a summary of preprocessing steps (e.g., "The data originates from public health records collected between 2010-2020, after anonymization and normalization processes were applied").
  
- **Biases**: An exemplary model card would identify potential biases by describing the demographic distribution of the training data (e.g., "The data set has a skew towards urban populations, which may affect the model's accuracy in rural settings"). 

# Notes
- Rate the card from 1 to 5, with half-points allowed (e.g., 2.5):
- Create a table with marddown with items name and score at the beginning of your response
- Pay special attention to how each section contributes to understanding the model's behavior and reliability.
- Consider the completeness and clarity of the information presented.
- Document any missing elements or recommendations for improvement.

Dataset Card to Evaluate: {readme_content}


"""