Transformers
Safetensors
English
ocbyram commited on
Commit
2c46994
·
verified ·
1 Parent(s): 99d488f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -5
README.md CHANGED
@@ -142,9 +142,10 @@ missing values. I also use techniques like forward and backward filling to handl
142
 
143
  # Limitations
144
 
145
- This section should summarize the main
146
- limitations of your model. Limitations could be based on benchmark task
147
- performance, any observations you noticed when examining model
148
- responses, or any shortcomings your model has relative to your ideal use
149
- case.
 
150
 
 
142
 
143
  # Limitations
144
 
145
+ The main limitation of this model is that it does not perform well on benchmarks outside of the chosen task, indicating that the model suffered
146
+ catastrophic forgetting during the training process. The benchmark task performance of the trained model on HumanEval, Squadv2, and E2E NLG Challenge were all lower than the baseline model.
147
+ This means that using the model for anything outside of the interview preparation use-case is unlikely to work well. Additionally, some of the model responses were not as expected,
148
+ as they included multiple questions and answers instead of the one that I asked for. While this technically works as long as the questions/answers are coherent and relevant,
149
+ it is still a limitation because I did not want the model to generate more than one question/answer. Generating multiple has a higher risk of inaccurate generated outputs.
150
+
151