Areyde commited on
Commit
4b46a6c
1 Parent(s): 7015161

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -10
README.md CHANGED
@@ -19,10 +19,10 @@ tags:
19
 
20
  # Kexer models
21
 
22
- Kexer models is a collection of fine-tuned open-source generative text models fine-tuned on Kotlin Exercices dataset.
23
- This is a repository for fine-tuned Deepseek-coder-6.7b model in the Hugging Face Transformers format.
24
 
25
- # Model use
26
 
27
  ```python
28
  from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -73,20 +73,19 @@ More details about finetuning can be found in the technical report
73
 
74
  # Fine-tuning data
75
 
76
- For this model we used 15K exmaples of [Kotlin Exercices dataset](https://huggingface.co/datasets/JetBrains/KExercises). Every example follows HumanEval like format. In total dataset contains about 3.5M tokens.
77
- For more information about the dataset follow the link.
78
 
79
  # Evaluation
80
 
81
- To evaluate we used Kotlin Humaneval ([more infromation here](https://huggingface.co/datasets/JetBrains/Kotlin_HumanEval))
82
 
83
- Fine-tuned model:
84
 
85
  | **Model name** | **Kotlin HumanEval Pass Rate** |
86
  |:---------------------------:|:----------------------------------------:|
87
- | `base model` | 40.99 |
88
- | `fine-tuned model` | 55.28 |
89
 
90
  # Ethical Considerations and Limitations
91
 
92
- Deepseek-7B-Kexer and its variants are a new technology that carries risks with use. The testing conducted to date could not cover all scenarios. For these reasons, as with all LLMs, Deepseek-7B-Kexer potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate or objectionable responses to user prompts. The model was fine-tuned on a specific data format (Kotlin tasks), and deviation from this format can also lead to inaccurate or undesirable responses to user queries. Therefore, before deploying any applications of Deepseek-7B-Kexer, developers should perform safety testing and tuning tailored to their specific applications of the model.
 
19
 
20
  # Kexer models
21
 
22
+ Kexer models are a collection of open-source generative text models fine-tuned on the [Kotlin Exercices](https://huggingface.co/datasets/JetBrains/KExercises) dataset.
23
+ This is a repository for the fine-tuned **Deepseek-coder-6.7b** model in the *Hugging Face Transformers* format.
24
 
25
+ # How to use
26
 
27
  ```python
28
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
73
 
74
  # Fine-tuning data
75
 
76
+ For tuning this model, we used 15K exmaples from the synthetically generated [Kotlin Exercices dataset](https://huggingface.co/datasets/JetBrains/KExercises). Every example follows the HumanEval format. In total, the dataset contains about 3.5M tokens.
 
77
 
78
  # Evaluation
79
 
80
+ For evaluation, we used the [Kotlin HumanEval](https://huggingface.co/datasets/JetBrains/Kotlin_HumanEval) dataset, which contains all 161 tasks from HumanEval translated into Kotlin by human experts. You can find more details about the pre-processing necessary to obtain our results, including the code for running, on the [datasets's page](https://huggingface.co/datasets/JetBrains/Kotlin_HumanEval).
81
 
82
+ Here are the results of our evaluation:
83
 
84
  | **Model name** | **Kotlin HumanEval Pass Rate** |
85
  |:---------------------------:|:----------------------------------------:|
86
+ | `Deepseek-7B` | 40.99 |
87
+ | `Deepseek-7B-Kexer` | **55.28** |
88
 
89
  # Ethical Considerations and Limitations
90
 
91
+ Deepseek-7B-Kexer is a new technology that carries risks with use. The testing conducted to date has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Deepseek-7B-Kexer's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate or objectionable responses to user prompts. The model was fine-tuned on a specific data format (Kotlin tasks), and deviation from this format can also lead to inaccurate or undesirable responses to user queries. Therefore, before deploying any applications of Deepseek-7B-Kexer, developers should perform safety testing and tuning tailored to their specific applications of the model.