| Small dummy deberta-v3-type Reward Model useable for Unit/Integration tests for RLHF. Suitable for CPU only machines, see [H2O LLM Studio](https://github.com/h2oai/h2o-llmstudio/blob/main/tests/integration/test_integration.py) for an example integration test. | |
| Model was created as follows: | |
| ```python | |
| from transformers import AutoConfig, AutoTokenizer, AutoModelForSequenceClassification | |
| repo_name = "MaxJeblick/reward-model-deberta-v3-unit-test" | |
| model_name = "OpenAssistant/reward-model-deberta-v3-large-v2" | |
| config = AutoConfig.from_pretrained(model_name) | |
| config.hidden_size = 12 | |
| config.intermediate_size = 24 | |
| config.num_attention_heads = 2 | |
| config.num_hidden_layers = 2 | |
| config.pooler_hidden_size = 12 | |
| tokenizer = AutoTokenizer.from_pretrained(model_name) | |
| model = AutoModelForSequenceClassification.from_config(config) | |
| print(model.num_parameters()) # 1_546_129 | |
| model.push_to_hub(repo_name, private=False) | |
| tokenizer.push_to_hub(repo_name, private=False) | |
| config.push_to_hub(repo_name, private=False) | |
| ``` |