File size: 3,142 Bytes
c9d0efa cd41b3b c9d0efa cd41b3b c9d0efa 6020c4e c9d0efa cd41b3b 6020c4e 4c7407a c9d0efa 8490df8 61bf04d 8490df8 307cdfe 8490df8 307cdfe f899167 8490df8 f899167 8490df8 307cdfe 8490df8 f899167 307cdfe f899167 307cdfe 8490df8 f899167 8490df8 307cdfe f899167 8490df8 61bf04d 8490df8 c9d0efa 4c7407a c9d0efa 4c7407a c9d0efa cd41b3b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
---
license: mit
base_model: microsoft/deberta-v3-small
tags:
- regression
model-index:
- name: deberta-v3-small-sp500-edgar-10k-markdown-1024-vN
results: []
datasets:
- BEE-spoke-data/sp500-edgar-10k-markdown
language:
- en
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# pszemraj/deberta-v3-small-sp500-edgar-10k
this predicts the `ret` column of the training dataset, given the `text` column.
<details>
<summary>Click to expand code example</summary>
```py
import json
from transformers import pipeline
from huggingface_hub import hf_hub_download
model_repo_name = "pszemraj/deberta-v3-small-sp500-edgar-10k"
pipe = pipeline("text-classification", model=model_repo_name)
pipe.tokenizer.model_max_length = 1024
# Download the regression_config.json file
regression_config_path = hf_hub_download(
repo_id=model_repo_name, filename="regression_config.json"
)
with open(regression_config_path, "r") as f:
regression_config = json.load(f)
def inverse_scale(prediction, config):
"""apply inverse scaling to a prediction"""
min_value, max_value = config["min_value"], config["max_value"]
return prediction * (max_value - min_value) + min_value
def predict_with_pipeline(text, pipe, config, ndigits=4):
result = pipe(text, truncation=True)[0] # Get the first (and likely only) result
score = result["score"] if result["label"] == "LABEL_1" else 1 - result["score"]
# Apply inverse scaling
scaled_score = inverse_scale(score, config)
return round(scaled_score, ndigits)
text = "This is an example text for regression prediction."
# Get predictions
predictions = predict_with_pipeline(text, pipe, regression_config)
print("Predicted returns (frec):", predictions)
```
</details>
## Model description
This model is a fine-tuned version of [microsoft/deberta-v3-small](https://huggingface.co/microsoft/deberta-v3-small) on BEE-spoke-data/sp500-edgar-10k-markdown
It achieves the following results on the evaluation set:
- Loss: 0.0005
- Mse: 0.0005
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 30826
- gradient_accumulation_steps: 16
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 3.0
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Mse |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.0064 | 0.54 | 50 | 0.0006 | 0.0006 |
| 0.0043 | 1.08 | 100 | 0.0005 | 0.0005 |
| 0.0028 | 1.61 | 150 | 0.0006 | 0.0006 |
| 0.0025 | 2.15 | 200 | 0.0005 | 0.0005 |
| 0.0025 | 2.69 | 250 | 0.0005 | 0.0005 |
### Framework versions
- Transformers 4.38.0.dev0
- Pytorch 2.2.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.2 |