[up]: model card

README.md

@@ -13,5 +13,72 @@ tags:
 datasets:
 - Hello-SimpleAI/HC3
 - tum-nlp/IDMGSP
+- mlabonne/Evol-Instruct-Python-26k
 library_name: transformers
----
+---
+
+<p style="text-align:center;">
+<img src="SA_logo.png" alt="SuperAnnotate Logo" width="100" height="100"/>
+</p>
+
+<h1 align="center">SuperAnnotate</h1>
+<h3 align="center">
+LLM Content Detector<br/>
+Fine-Tuned RoBERTa Large<br/>
+</h3>
+
+## Description
+
+The model designed to detect generated/synthetic text. \
+At the moment, such functionality is critical for check your training data and detecting fraud and cheating in scientific and educational areas.
+
+## Model Details
+
+### Model Description
+
+- **Model type:** The custom architecture for binary sequence classification based on pre-trained RoBERTa, with a single output label.
+- **Language(s):** Primarily English.
+- **License:** Apache 2.0
+- **Finetuned from model:** [RoBERTa Large](https://huggingface.co/FacebookAI/roberta-large)
+
+### Model Sources
+
+- **Repository:** [GitHub](https://github.com/superannotateai/generated_text_detector) for HTTP service
+
+### Training data
+
+The training data was sourced from three open datasets with different proportions and underwent filtering:
+
+1. [**HC3**](https://huggingface.co/datasets/Hello-SimpleAI/HC3) | **50%**
+1. [**IDMGSP**](https://huggingface.co/datasets/tum-nlp/IDMGSP) | **30%**
+1. [**Evol-Instruct-Python-26k**](https://huggingface.co/datasets/mlabonne/Evol-Instruct-Python-26k) | **20%**
+
+As a result, the training dataset contained approximately ***25k*** pairs of text-label with an approximate balance of classes. \
+It's worth noting that the dataset's texts follow a logical structure: \
+Human-written and model-generated texts refer to a single prompt/instruction, though the prompts themselves were not used during training.
+
+### Peculiarity
+
+During training, one of the priorities was not only maximizing the quality of predictions but also avoiding overfitting and obtaining an adequately confident predictor. \
+We are pleased to achieve the following state of model calibration:
+
+<img src="Calibration_plot.png" alt="SuperAnnotate Logo" width="390" height="300"/>
+
+## Usage
+
+TODO
+
+## Performance
+
+The model was evaluated on a benchmark collected from the same datasets used for training, alongside a closed subset of SuperAnnotate. \
+However, there are no direct intersections of samples between the training data and the benchmark. \
+The benchmark comprises 1k samples, with 200 samples per category. \
+The model's performance is compared with open-source solutions and popular API detectors in the table below:
+
+| Model/API                                                                                        | Wikipedia | Reddit QA | SA instruction | Papers |  Code  | Average |
+|--------------------------------------------------------------------------------------------------|----------:|----------:|---------------:|-------:|-------:|--------:|
+| [Hello-SimpleAI](https://huggingface.co/Hello-SimpleAI/chatgpt-detector-roberta)                 |   **0.97**|      0.95 |           0.82 |   0.69 |   0.47 |    0.78 |
+| [RADAR](https://huggingface.co/spaces/TrustSafeAI/RADAR-AI-Text-Detector)                        |      0.47 |      0.84 |           0.59 |   0.82 |   0.65 |    0.68 |
+| [GPTZero](https://gptzero.me)                                                                    |      0.72 |      0.79 |        **0.90**|   0.67 |   0.74 |    0.76 |
+| [Originality.ai](https://originality.ai)                                                         |      0.91 |   **0.97**|           0.77 |**0.93**|   0.46 |    0.81 |
+| [LLM content detector](https://huggingface.co/SuperAnnotate/roberta-large-llm-content-detector)  |      0.88 |      0.95 |           0.84 |   0.81 |**0.96**| **0.89**|