Work in progress
#1
by
KingyJr
- opened
It was very unstable. We are trying a different avenue at this present time. Currently trying to increase token length slowly with codellama.
We currently have 13b working correctly with 64k tokens.
We are also adding new tokens to implement into our project.
Fine tuning will happen shortly.
If anyone has a decent prompt and code dataset would be appreciated, currently working on making our own dataset..