Hugging Face's logo

dbands
/
Gemma_ChemWiz_16bit

Safetensors
gemma
Model card Files Community
Gemma_ChemWiz_16bit
File size: 2,002 Bytes 
9e29ac1
 
 
 
7707e0b
 
9e29ac1
7e974b4
 
7707e0b
 
dc4ca03
 
9e95276
 
 
f62e2f2
 
 
 
 
 
 
 
 
 
 
4261cd7
 
f62e2f2
 
 
 
 
 
 
 
 
 
4261cd7
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
 
---
datasets:
- AI-MO/NuminaMath-CoT
- AI4Chem/ChemData700K
- medalpaca/medical_meadow_mediqa
- andersonbcdefg/chemistry
---
2024-08-13: Taking the mdodel through second round of AI4Chem/ChemData700K, I am amazed how the model seem to converge and the suddenly it does not.  I suspect in nextfew days that it will converge.  I am quite keen to see this happen.  The results of chemichal smiles are very low at this point.

2024-08-12: The medalpaca/medical_meadow_mediqa data set was also used, but the model converged on this in less than one epoch, only 1400 steps of training was concluded, in future versions and editions I might elect to exclude this data set, but it is included in this version.

2024-08-12: Model is being fine tuned on chemical memory, rather than chmistry reasoning.  Using the AI4Chem/ChemData700K dataset.  Model is still halucinating chemical formulas, I will fine tune model on a few more data sets to see if this affects/reduce the halicinations.

2024-08-09: Model is still being fine tuned for logical reasoning, the responses being recieved at this time seem to be in line with the training set, so the model at this time for instance do not jump straight into an answer, but started "unpacking" the instruction before persorming a task, such as coding.
Nothing this model creates at this time should be used for any production purpose, highly experimental.

---
base_model: unsloth/codegemma-7b-bnb-4bit
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- gemma
- trl
datasets:
- AI-MO/NuminaMath-CoT
---

# Uploaded  model

- **Developed by:** dbands
- **License:** apache-2.0
- **Finetuned from model :** unsloth/codegemma-7b-bnb-4bit

This gemma model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)