Nearly finalised modelcard.

README.md

@@ -1,109 +1,84 @@
 ---
-library_name: transformers
-tags: []
+license: other
+license_name: qwen-research
+license_link: https://huggingface.co/Qwen/Qwen2.5-3B/blob/main/LICENSE
+language:
+- en
+pipeline_tag: text-generation
+datasets:
+- mlabonne/orpo-dpo-mix-40k
+base_model:
+- Qwen/Qwen2.5-3B
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
+# Qwen2.5-3B-ORPO
 
+This model is a finetuned model of the (Qwen2.5-3B base model](https://huggingface.co/Qwen/Qwen2.5-3B) by Qwen using ORPO over the [ORPO-DPO-mix dataset](https://huggingface.co/datasets/mlabonne/orpo-dpo-mix-40k) by M. Labonne.
+We evaluate the model using several benchmarks working with an Eleuther evaluation harness. Apart from ensuring no reduction in general model performance, benchmarks testing for improved performance in logical and numerical reasoning
+are applied. The results are inconclusive on most, but not all, metrics. This shows promise for further improvements on more tailored datasets or by further applying DPO for preference training.
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
+Qwen2.5 is the latest series of Qwen large language models. We finetune the 3B model with the following specifications:
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+- **Developed by:** (Finetuned from) Qwen
+- **Language(s) (NLP):** English
+- **Finetuned from model:** Qwen2.5-3B
+- **Model type:** Causal LM
+- **Architecture:** Transformers with RoPE, SwiGLU, RMSNorm, Attention QKV bias and tied word embeddings
+- **Number of Parameters:** 3.09B
+- **Number of Paramaters (Non-Embedding):** 2.77B
+- **Number of Layers:** 36
+- **Number of Attention Heads (GQA):** 16 for Q and 2 for KV
+- **Context Length:** Full 32,768 tokens
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+For additional details, we refer to the base model repository. 
 
-### Model Sources [optional]
+### Model Sources
 
-<!-- Provide the basic links for the model. -->
+The Qwen2.5-3B base model can be found here:
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Repository:** https://huggingface.co/Qwen/Qwen2.5-3B
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+The model is finetuned but with only little performance increase over the base model in logical and numerical reasoning.
+While better than the base model, we do not think it suffices for well-founded logical and numerical reasoning at this stage.
+However, we detect no performance decrease for common sense natural reasoning.
 
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
+Common sense natural language reasoning.
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+### Downstream Use
 
-[More Information Needed]
+Logical and numerical reasoning.
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
+Additional finetuning on different datasets, as well as preference training.
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+We use the [ORPO-DPO-mix dataset](https://huggingface.co/datasets/mlabonne/orpo-dpo-mix-40k) by M. Labonne.
+It is a dataset is designed for ORPO or DPO training. See Fine-tune Llama 3 with ORPO for more information about how to use it.
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
+We used the trl [ORPO trainer](https://huggingface.co/docs/trl/main/en/orpo_trainer) for finetuning, together with [LoRa](https://arxiv.org/abs/2106.09685) for speed-up.
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+### Training Hyperparameters
 
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+- **Training regime:** fp16 non-mixed precision
 
 ## Evaluation
 
-We evaluate base and finetuned models on four general benchmarks and one usecase specific one. We work with an Eleuther test harness.
-Our usecase is information extraction of heterogeneous data from government documents. Here, we test on documents of the U.S. [FDA](https://en.wikipedia.org/wiki/Food_and_Drug_Administration).
+We evaluate base and finetuned models on four general benchmarks and two usecase specific one. We work with an Eleuther test harness.
+Our usecase is logical and numerical reasoning.
 
 Benchmarks used:
 
@@ -117,10 +92,21 @@ Benchmarks used:
 
 3. USECASE: Logical and numerical reasoning.
   3.1 Arithmetic
-  3.2 ASDiv.
+  3.2 ASDiv
+
+Summary of results:
+
+Within standard error, there is no difference between base and finetuned model on any general benchmark. This suggests there was no drop in performance for the chosen tasks due to finetuning.
+Benchmarks for logical and numerical reasoning are more mixed. Without standard error, the finetuned model generally outperforms the base model. However, this lies - often just about - within standard error.
+The finetuned model *does* outperform the base model even accounting for standard error with maximal conservative bias on **arithmetic_5da**.
+This is of interest, since this benchmarks a model's ability to add five digits - *the* most fundamental arithmetic operation, and in effect the most difficult of all addition benchmarks.
+Note subtraction appears generally harder for both the finetuned and base models, even as the finetuned model performs better.
+We highlight the relevant rows for five-digit addition and subtraction for easy comparison.
 
 Evaluation results:
 
+**BASE**
+
 <figure>
   
 |  Tasks   | Version | Filter | n-shot | Metric |   | Value |   | Stderr |
@@ -150,14 +136,15 @@ Evaluation results:
 |arithmetic_3ds|       1 | none   |      0 |acc     |↑  | 0.0055|±  |  0.0017|
 |arithmetic_4da|       1 | none   |      0 |acc     |↑  | 0.0675|±  |  0.0056|
 |arithmetic_4ds|       1 | none   |      0 |acc     |↑  | 0.0010|±  |  0.0007|
-|arithmetic_5da|       1 | none   |      0 |acc     |↑  | 0.3720|±  |  0.0108|
-|arithmetic_5ds|       1 | none   |      0 |acc     |↑  | 0.0260|±  |  0.0036|
+**|arithmetic_5da|       1 | none   |      0 |acc     |↑  | 0.3720|±  |  0.0108|**
+**|arithmetic_5ds|       1 | none   |      0 |acc     |↑  | 0.0260|±  |  0.0036|**
 |--------------|--------:|--------|-------:|--------|---|------:|---|-------:|
 |asdiv         |       1 | none   |      0 |acc     |↑  | 0.0187|±  |  0.0028|
 <figcaption>Collected USECASE benchmarks results for the base model.</figcaption>
 
 </figure>
 
+**FINETUNED**
 
 <figure>
 
@@ -188,110 +175,10 @@ Evaluation results:
 |arithmetic_3ds|       1 | none   |      0 |acc     |↑  | 0.0055|±  |  0.0017|
 |arithmetic_4da|       1 | none   |      0 |acc     |↑  | 0.0710|±  |  0.0057|
 |arithmetic_4ds|       1 | none   |      0 |acc     |↑  | 0.0005|±  |  0.0005|
-|arithmetic_5da|       1 | none   |      0 |acc     |↑  | 0.4005|±  |  0.0110|
-|arithmetic_5ds|       1 | none   |      0 |acc     |↑  | 0.0285|±  |  0.0037|
+**|arithmetic_5da|       1 | none   |      0 |acc     |↑  | 0.4005|±  |  0.0110|**
+**|arithmetic_5ds|       1 | none   |      0 |acc     |↑  | 0.0285|±  |  0.0037|**
 |--------------|--------:|--------|-------:|--------|---|------:|---|-------:|
 |asdiv         |       1 | none   |      0 |acc     |↑  | 0.0204|±  |  0.0029|
 <figcaption>Collected USECASE benchmarks results for the finetuned model.</figcaption>
 
-</figure>
-
-|----------|--------:|--------|-------:|--------|---|------:|---|-------:|
-|asdiv     |       1 | none   |      0 |acc     |↑  | 0.0204|±  |  0.0029|
-
-
-1.2 Finetuned model
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
+</figure>