---
language: 
  - en
thumbnail: "https://github.com/AI-Ahmed"
tags:
- classification
license: cc-by-4.0
datasets:
- SetFit/qqp
models:
- microsoft/deberta-v3-base
metrics:
- accuracy
- loss
pipeline_tag: text-classification
widget:
- text: How is the life of a math student? Could you describe your own experiences?
  pair: Which level of preparation is enough for the exam jlpt5?
  example_title: "Similarity Detection."
---

A fine-tuned model based on the **DeBERTaV3** model of Microsoft and fine-tuned on **Glue QQP**, which detects the linguistical similarities between two questions and whether they are similar questions or duplicates.

## Model Hyperparameters

```python
epoch=4
per_device_train_batch_size=32
per_device_eval_batch_size=16
lr=2e-5
weight_decay=1e-2
gradient_checkpointing=True
gradient_accumulation_steps=8
```
## Model Performance

```JSON
{"Training Loss": 0.132400,
 "Validation Loss": 0.217410,
 "Validation Accuracy": 0.917969
}
```

## Model Dependencies

```JSON
{"Main Model": "microsoft/deberta-v3-base",
 "Dataset": "SetFit/qqp"
}
```

## Training Monitoring & Performance

- [wandb - deberta_qqa_classification](https://wandb.ai/ai-ahmed/deberta_qqa_classification?workspace=user-ai-ahmed)

## Information Citation

```bibtex
@inproceedings{
he2021deberta,
title={DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION},
author={Pengcheng He and Xiaodong Liu and Jianfeng Gao and Weizhu Chen},
booktitle={International Conference on Learning Representations},
year={2021},
url={https://openreview.net/forum?id=XPZIaotutsD}
}
```