fsssg/Biodiversity_Commitment_Specificity_Classifier

Model overview

This model is a binary text classifier designed to assess the specificity of biodiversity-related commitments in corporate sustainability reports at the paragraph level. It is intended as a second-stage classifier applied only to paragraphs that have already been identified as biodiversity-related commitments.

The model distinguishes between:

Specific commitments (label=1): those that describe concrete actions, measurable targets, defined strategies, timelines, or verifiable implementation plans

Non-specific commitments (label=0): those that are vague, aspirational, ambiguous, or lack identifiable actions or implementation detail

The model is intended for research use in the analysis of corporate sustainability and ESG disclosures.

Training approach

The model was trained on a curated dataset of 2,000 manually annotated paragraphs extracted from sustainability reports of Fortune Global 500 companies.

The classifier is based on climatebert/distilroberta-base-climate-specificity, a DistilRoBERTa-based ClimateBERT model pre-trained on climate-related corpora and previously fine-tuned for climate commitment specificity detection. This model was further fine-tuned to assess the specificity of biodiversity-related commitments.

Key training characteristics include:

unit of analysis: paragraph

maximum sequence length: 256 tokens

task: binary sequence classification

loss function: cross-entropy

optimisation: supervised fine-tuning using the Hugging Face Trainer API

training regime: 5-fold stratified cross-validation with grouping by firm–year identifier

Cross-validation folds were constructed so that all paragraphs associated with a given firm–year appear in either the training or validation set, but not both.

Training was performed on CPU using fixed hyperparameters selected prior to cross-validation. The released model checkpoint corresponds to the fold achieving the highest macro F1 score.

Evaluation

Performance is reported as averages across 5-fold grouped cross-validation on the annotated dataset.

weighted F1 score: 0.856

weighted precision: 0.856

weighted recall: 0.856

AUC–ROC: 0.922

Macro F1 was used as the model selection criterion to ensure balanced performance across classes.

Recommended pipeline

For optimal results, use this model in combination with a biodiversity paragraph identification model and our commitment detection model:

Stage 1: Identify biodiversity-related paragraphs with ESGBERT/EnvironmentalBERT-biodiversity. Stage 2: Identify biodiversity-related commitments with fsssg/ Biodiversity_Commitment_Specificity_Classifier Stage 3: Classify commitment specificity (this model)