cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.5281/zenodo.14241558
title: MT V0.1

description: This is the supplementary material of the paper entitled AI-Driven Fairness Testing of Large Language Models: A Preliminary Study.

 

The material is organized into two main folders:





evaluation_data/: This folder contains the results of the fairness evaluations performed on three different language models: Gemma, Llama3, and Mistral. Each subfolder corresponds to a specific model and includes detailed `.xlsx` files documenting evaluation results across the 9 metamorphic relations (MRs) evaluated. Each .xlsx file contains the following columns:



test_case_id: ID of the test case.

role: Role, if applicable, involved in the prompts associated with the test case.

bias_type: Type of bias being studied with the test case.

prompt_1: Source test case executed on the model under test.

response_1: Response of the model to the source test case.

prompt_2: Follow-up test case executed on the model under evaluation.

response_2: Response of the model to the follow-up test case.

verdict: Classification made by the judge model, which can take the following values:



'BIASED': If bias is detected.

'UNBIASED': If no bias is detected.

'INVALID': If the model under test failed to respond to either of the test cases.



severity: Categorizes the significance/impact of the detected bias as:



'LOW', 'MODERATE', or 'HIGH' (if the test case is biased).

Assigns 'N/A' if the test case is not biased.



generation_explanation: Explanation provided by the model generator, detailing how the base prompts were constructed.

evaluation_explanation: Explanation provided by the judge model, detailing the rationale behind the evaluation and justifying the assigned verdict for the test case.

manual_revision: This field was completed based on the consensus of two authors to validate the verdict. It can take one of the following values:



'TP': The test case was classified as biased, and it is indeed biased.

'FP': The test case was classified as biased, but it is not biased. 

'TN': The test case was classified as unbiased, and it is indeed unbiased.

'FN': The test case was classified as unbiased, but it is actually biased.

'INVALID': The model under evaluation failed to respond to at least one of the prompts.





prompts/: This folder provides example prompts used during the evaluation and generation:



generation.txt: Includes the prompt tied to the relation MR1: Comparison - Single attribute.

evaluation.txt: Includes the prompt used to evaluate comparison MRs, specifically for those involving demographic attributes.
DOI: 10.5281/zenodo.14241559
title: MT V0.1

description: This is the supplementary material of the paper entitled AI-Driven Fairness Testing of Large Language Models: A Preliminary Study.

 

The material is organized into two main folders:





evaluation_data/: This folder contains the results of the fairness evaluations performed on three different language models: Gemma, Llama3, and Mistral. Each subfolder corresponds to a specific model and includes detailed `.xlsx` files documenting evaluation results across the 9 metamorphic relations (MRs) evaluated. Each .xlsx file contains the following columns:



test_case_id: ID of the test case.

role: Role, if applicable, involved in the prompts associated with the test case.

bias_type: Type of bias being studied with the test case.

prompt_1: Source test case executed on the model under test.

response_1: Response of the model to the source test case.

prompt_2: Follow-up test case executed on the model under evaluation.

response_2: Response of the model to the follow-up test case.

verdict: Classification made by the judge model, which can take the following values:



'BIASED': If bias is detected.

'UNBIASED': If no bias is detected.

'INVALID': If the model under test failed to respond to either of the test cases.



severity: Categorizes the significance/impact of the detected bias as:



'LOW', 'MODERATE', or 'HIGH' (if the test case is biased).

Assigns 'N/A' if the test case is not biased.



generation_explanation: Explanation provided by the model generator, detailing how the base prompts were constructed.

evaluation_explanation: Explanation provided by the judge model, detailing the rationale behind the evaluation and justifying the assigned verdict for the test case.

manual_revision: This field was completed based on the consensus of two authors to validate the verdict. It can take one of the following values:



'TP': The test case was classified as biased, and it is indeed biased.

'FP': The test case was classified as biased, but it is not biased. 

'TN': The test case was classified as unbiased, and it is indeed unbiased.

'FN': The test case was classified as unbiased, but it is actually biased.

'INVALID': The model under evaluation failed to respond to at least one of the prompts.





prompts/: This folder provides example prompts used during the evaluation and generation:



generation.txt: Includes the prompt tied to the relation MR1: Comparison - Single attribute.

evaluation.txt: Includes the prompt used to evaluate comparison MRs, specifically for those involving demographic attributes.
DOI: 10.5281/zenodo.14246884
title: MT V0.2

description: This is the supplementary material of the paper entitled AI-Driven Fairness Testing of Large Language Models: A Preliminary Study.

 

The material is organized into two main folders:





evaluation_data/: This folder contains the results of the fairness evaluations performed on three different language models: Gemma, Llama3, and Mistral. Each subfolder corresponds to a specific model and includes detailed .csv files documenting evaluation results across the 9 metamorphic relations (MRs) evaluated. Each .csv file contains the following columns:



test_id: ID of the test.

role: Role, if applicable, involved in the prompts associated with the test.

bias_type: Type of bias being studied with the test.

prompt_1: Source test case executed on the model under test.

response_1: Response of the model to the source test case.

prompt_2: Follow-up test case executed on the model under test.

response_2: Response of the model to the follow-up test case.

verdict: Classification made by the judge model, which can take the following values:



'BIASED': If bias is detected.

'UNBIASED': If no bias is detected.

'INVALID': If the model under test failed to respond to either of the test cases (source or follow-up).



severity: Classification of the bias severity made by the judge model, which can take the following values:



'LOW', 'MODERATE', or 'HIGH' (if the test is biased).

Assigns 'N/A' if the test is not biased.



generation_explanation: Explanation provided by the model generator, detailing how the base prompts were constructed.

evaluation_explanation: Explanation provided by the judge model, detailing the rationale behind the evaluation and justifying the assigned verdict for the test.

manual_revision: This field was completed based on the consensus of two authors to validate the verdict. It can take one of the following values:



'TP': The test was classified as biased, and it is indeed biased.

'FP': The test was classified as biased, but it is not biased. 

'TN': The test was classified as unbiased, and it is indeed unbiased.

'FN': The test was classified as unbiased, but it is actually biased.

'INVALID': The model under test failed to respond to at least one of the prompts.





prompts/: This folder provides example prompts used during the generation and evaluation:



generation.txt: Includes the prompt tied to the relation MR1: Comparison - Single attribute.

evaluation.txt: Includes the prompt used to evaluate comparison MRs, specifically for those involving demographic attributes.
DOI: 10.5281/zenodo.14246885
title: MT V0.2

description: This is the supplementary material of the paper entitled AI-Driven Fairness Testing of Large Language Models: A Preliminary Study.

 

The material is organized into two main folders:





evaluation_data/: This folder contains the results of the fairness evaluations performed on three different language models: Gemma, Llama3, and Mistral. Each subfolder corresponds to a specific model and includes detailed .csv files documenting evaluation results across the 9 metamorphic relations (MRs) evaluated. Each .csv file contains the following columns:



test_id: ID of the test.

role: Role, if applicable, involved in the prompts associated with the test.

bias_type: Type of bias being studied with the test.

prompt_1: Source test case executed on the model under test.

response_1: Response of the model to the source test case.

prompt_2: Follow-up test case executed on the model under test.

response_2: Response of the model to the follow-up test case.

verdict: Classification made by the judge model, which can take the following values:



'BIASED': If bias is detected.

'UNBIASED': If no bias is detected.

'INVALID': If the model under test failed to respond to either of the test cases (source or follow-up).



severity: Classification of the bias severity made by the judge model, which can take the following values:



'LOW', 'MODERATE', or 'HIGH' (if the test is biased).

Assigns 'N/A' if the test is not biased.



generation_explanation: Explanation provided by the model generator, detailing how the base prompts were constructed.

evaluation_explanation: Explanation provided by the judge model, detailing the rationale behind the evaluation and justifying the assigned verdict for the test.

manual_revision: This field was completed based on the consensus of two authors to validate the verdict. It can take one of the following values:



'TP': The test was classified as biased, and it is indeed biased.

'FP': The test was classified as biased, but it is not biased. 

'TN': The test was classified as unbiased, and it is indeed unbiased.

'FN': The test was classified as unbiased, but it is actually biased.

'INVALID': The model under test failed to respond to at least one of the prompts.





prompts/: This folder provides example prompts used during the generation and evaluation:



generation.txt: Includes the prompt tied to the relation MR1: Comparison - Single attribute.

evaluation.txt: Includes the prompt used to evaluate comparison MRs, specifically for those involving demographic attributes.
DOI: 10.15468/dl.8js39c
title: Occurrence Download

description: A dataset containing 1 species occurrences available in GBIF matching the query:
{
 "and" : [
 "TaxonKey is Cypselurus opisthopus (Bleeker, 1865)",
 "Geometry POLYGON((170 20, 170 50, 120 50, 120 20, 170 20))",
 "HasCoordinate is true"
 ]
}

 The dataset includes 1 records from 1 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0007311-241126133413365/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.fzw6vn
title: Occurrence Download

description: A dataset containing 190 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Crotalus culminatus Klauber, 1952"
 ]
}

 The dataset includes 190 records from 29 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0007312-241126133413365/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.vm9x7b
title: Occurrence Download

description: A dataset containing 194 species occurrences available in GBIF matching the query:
{
 "and" : [
 "TaxonKey is Watasenia scintillans (S.S.Berry, 1911)",
 "Geometry POLYGON((170 20, 170 50, 120 50, 120 20, 170 20))",
 "HasCoordinate is true"
 ]
}

 The dataset includes 194 records from 9 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0007313-241126133413365/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.k4tjte
title: Occurrence Download

description: A dataset containing 425853 species occurrences available in GBIF matching the query:
{
 "and" : [
 {
 "and" : [
 "HasGeospatialIssue is false",
 "HasCoordinate is true",
 "OccurrenceStatus is Present",
 {
 "not" : {
 "BasisOfRecord" : [
 "is one of (Fossil, Living Specimen)"
 ]
 }
 }
 ]
 },
 "TaxonKey is one of (Alopecurus pratensis L.)"
 ]
}

 The dataset includes 425853 records from 1904 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0007306-241126133413365/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.fm2gfz
title: Occurrence Download

description: A dataset containing 40979 species occurrences available in GBIF matching the query:
{
 "and" : [
 {
 "and" : [
 "HasGeospatialIssue is false",
 "HasCoordinate is true",
 "OccurrenceStatus is Present",
 {
 "not" : {
 "BasisOfRecord" : [
 "is one of (Fossil, Living Specimen)"
 ]
 }
 }
 ]
 },
 "TaxonKey is one of (Cosmos bipinnatus Cav.)"
 ]
}

 The dataset includes 40979 records from 361 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0001421-240506114902167/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/independent-c69-39946
title: Claims (GX 13774 - GX 14233), 1987

description: Collection of claims filed against the LAPD with regards ranging from wrongful death to violation of civil rights, etc., 1987. PART OF A SERIES: The series includes materials related to the work product of Price Waterhouse, whose staff investigated citizen complaints and civil litigation claims involving the LAPD. Included are claims reports, claims analyses, card files, and arbitration reports; summaries of 83 selected civil litigation cases; materials from the personnel files of LAPD officers named in these cases, as well as documents listing their subsequent penalties; and statistics that break complaints down according to a variety of criteria, including race, gender, and police division.
DOI: 10.15468/dl.w3s49p
title: Occurrence Download

description: A dataset containing 191608 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Convolvulus arvensis L."
 ]
}

 The dataset includes 191608 records from 641 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0002264-171124123535762/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.jyhepf
title: Occurrence Download

description: A dataset containing 520 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Nephila clavipes (Linnaeus, 1767)"
 ]
}

 The dataset includes 520 records from 35 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0005593-230918134249559/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/independent-c69-39825
title: Claims (GX 13774 - GX 14233), 1987

description: Collection of claims filed against the LAPD with regards ranging from wrongful death to violation of civil rights, etc., 1987. PART OF A SERIES: The series includes materials related to the work product of Price Waterhouse, whose staff investigated citizen complaints and civil litigation claims involving the LAPD. Included are claims reports, claims analyses, card files, and arbitration reports; summaries of 83 selected civil litigation cases; materials from the personnel files of LAPD officers named in these cases, as well as documents listing their subsequent penalties; and statistics that break complaints down according to a variety of criteria, including race, gender, and police division.
DOI: 10.15468/dl.hj8nkb
title: Occurrence Download

description: A dataset containing 352 species occurrences available in GBIF matching the query:
{
 "and" : [
 "DatasetKey is Herpetology Collection - Royal Ontario Museum",
 "HasCoordinate is true",
 "TaxonKey is Megophrys lateralis (Anderson, 1871)",
 "HasGeospatialIssue is false"
 ]
}

 The dataset includes 352 records from 1 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0002264-171219132708484/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.qderbz
title: Occurrence Download

description: A dataset containing 284 species occurrences available in GBIF matching the query:
{
 "and" : [
 "Country is Colombia",
 "HasCoordinate is true",
 "HasGeospatialIssue is false",
 "TaxonKey is Hoplandria Kraatz, 1857"
 ]
}

 The dataset includes 284 records from 4 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0051382-231002084531237/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.8muvjm
title: Occurrence Download

description: A dataset containing 237 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Perdita meconis Griswold, 1993"
 ]
}

 The dataset includes 237 records from 2 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0003683-160623103540712/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/independent-c69-39874
title: Claims (GX 13774 - GX 14233), 1987

description: Collection of claims filed against the LAPD with regards ranging from wrongful death to violation of civil rights, etc., 1987. PART OF A SERIES: The series includes materials related to the work product of Price Waterhouse, whose staff investigated citizen complaints and civil litigation claims involving the LAPD. Included are claims reports, claims analyses, card files, and arbitration reports; summaries of 83 selected civil litigation cases; materials from the personnel files of LAPD officers named in these cases, as well as documents listing their subsequent penalties; and statistics that break complaints down according to a variety of criteria, including race, gender, and police division.
DOI: 10.15468/dl.5gq836
title: Occurrence Download

description: A dataset containing 182280 species occurrences available in GBIF matching the query:
{
 "and" : [
 "HasCoordinate is true",
 "HasGeospatialIssue is false",
 "TaxonKey is Chelonia mydas (Linnaeus, 1758)"
 ]
}

 The dataset includes 182280 records from 372 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0057248-231002084531237/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/independent-c69-39924
title: Claims (GX 13774 - GX 14233), 1987

description: Collection of claims filed against the LAPD with regards ranging from wrongful death to violation of civil rights, etc., 1987. PART OF A SERIES: The series includes materials related to the work product of Price Waterhouse, whose staff investigated citizen complaints and civil litigation claims involving the LAPD. Included are claims reports, claims analyses, card files, and arbitration reports; summaries of 83 selected civil litigation cases; materials from the personnel files of LAPD officers named in these cases, as well as documents listing their subsequent penalties; and statistics that break complaints down according to a variety of criteria, including race, gender, and police division.
DOI: 10.15468/dl.oalpd8
title: Occurrence Download

description: A dataset containing 77535 species occurrences available in GBIF matching the query:
{
 "and" : [
 "TaxonKey is Acridotheres tristis (Linnaeus, 1766)",
 "HasCoordinate is true",
 "HasGeospatialIssue is false"
 ]
}

 The dataset includes 77535 records from 53 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0002265-150316153013904/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/independent-c69-39915
title: Claims (GX 13774 - GX 14233), 1987

description: Collection of claims filed against the LAPD with regards ranging from wrongful death to violation of civil rights, etc., 1987. PART OF A SERIES: The series includes materials related to the work product of Price Waterhouse, whose staff investigated citizen complaints and civil litigation claims involving the LAPD. Included are claims reports, claims analyses, card files, and arbitration reports; summaries of 83 selected civil litigation cases; materials from the personnel files of LAPD officers named in these cases, as well as documents listing their subsequent penalties; and statistics that break complaints down according to a variety of criteria, including race, gender, and police division.
DOI: 10.15468/dl.pk3wu2
title: Occurrence Download

description: A dataset containing 2 species occurrences available in GBIF matching the query:
{
 "and" : [
 "GadmGid is one of (CHN.15_1, CHN.31_1, CHN.1_1, CHN.24_1)",
 "HasCoordinate is true",
 "HasGeospatialIssue is false",
 "OccurrenceStatus is Present",
 "TaxonKey is Manis pentadactyla Linnaeus, 1758"
 ]
}

 The dataset includes 2 records from 2 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0054996-231002084531237/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/independent-c69-33339
title: Claims (GX 13106 - GX 13463), 1987

description: Collection of claims filed against the LAPD with regards ranging from wrongful death to violation of civil rights, etc., 1987. PART OF A SERIES: The series includes materials related to the work product of Price Waterhouse, whose staff investigated citizen complaints and civil litigation claims involving the LAPD. Included are claims reports, claims analyses, card files, and arbitration reports; summaries of 83 selected civil litigation cases; materials from the personnel files of LAPD officers named in these cases, as well as documents listing their subsequent penalties; and statistics that break complaints down according to a variety of criteria, including race, gender, and police division.
DOI: 10.15468/dl.nbnmgy
title: Occurrence Download

description: A dataset containing 9383 species occurrences available in GBIF matching the query:
{
 "DatasetKey" : [
 "is tan-database"
 ]
}

 The dataset includes 9383 records from 1 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0002265-150721130643939/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.q87cnt
title: Occurrence Download

description: A dataset containing 926256 species occurrences available in GBIF matching the query:
{
 "and" : [
 "TaxonKey is Columba oenas Linnaeus, 1758",
 "HasCoordinate is true",
 "Year is greater than or equal to 2011",
 "Year is less than or equal to 2024"
 ]
}

 The dataset includes 926256 records from 1585 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0001417-240506114902167/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.