Small description

VDrontV1.5 is a 150M-parameter GPT-like model. It is an enlarged version of V1, trained on approximately 3.5 billion tokens.

The model was trained on FineWeb-Edu + FineWeb-Edu-Instruct.

Stored in Hugging Face format and DrontQ(.dq) quantized format.

VDrontV1.5/ - HF format 32 bit

VDrontV1.5_quant/ .dq format, the folder contains DQuse.py for using .dq.

Formats

format	bit	Path	Use	Size	Generation quality
Hugging Face .safetensors	32 bit	VDrontV1.5/	use.py	600mb	Full
.dq (dront quant)	4 bit	VDrontV1.5_quant/VDrontV1.5_Q4.dq	VDrontV1.5_quant/DQuse.py	95mb	Almost terrible
.dq (dront quant)	8 bit	VDrontV1.5_quant/VDrontV1.5_Q8.dq	VDrontV1.5_quant/DQuse.py	190mb	Noticeably worse
.dq (dront quant)	16 bit	VDrontV1.5_quant/VDrontV1.5_Q16.dq	VDrontV1.5_quant/DQuse.py	370mb	A little worse

Dront Model Tester

Dront Model Tester	VDrontV1.5	VDrontV1.5_Q16.dq	VDrontV1.5_Q8.dq	VDrontV1.5_Q4.dq
talk	30%	30%	25%	8%
info	9.5%	9.5%	8%	1%
size	600mb	370mb	190mb	95mb
bits per param	F32	Q16	Q8	Q4

Examples

The model is more akin to an instruction-tuned model than a conversational one; it supports the <|user|> – <|assistant|> format but won't respond to a greeting with a greeting of its own. However, it answers questions like "What is Python?" or "How many people are there in Asia?" quite well.

VDrontV1.5 HF (t 0.4)

Q: What is a Python? 

A: Python language is a language that is used to communicate with a person or group of people. It is used in many languages, but is not a standard language. The most common programming language used is Python.

The Python language is a programming language that is used by many people today. It is used by many people today as a means of communication between people and the world. It is used by many people today as a way to communicate with people, and it is used by many people today.

Some of the most popular programming languages include:

• Python - Python is a programming language. It is used by many people today, especially in the UK. It is used by many people today.

• Python - Python is used by many people today. It is used by many people today. It is used by many people today.

VDrontV1.5_Q16.dq (t 0.4)

Q: What is a Python? 

A: A Python is an extension of the Python language. The following are some of the common Python commands in Python:

• Python is a Python language. It provides a code for creating code to be used by Python libraries and libraries. This code can be used for creating Python libraries or libraries.

• Python is a Python programming language. Python can be used for many other languages, including Python, Web Services, etc. The Python language is used for many different languages.

• Python is used for many types of applications like web hosting, website hosting, Web hosting, Web services, etc. Python is used for many things. Python is also used for other programming languages such as Web Services and Web Services.

VDrontV1.5_Q8.dq (t 0.4)

Q: What is a Python? 

A: A Python is a type of code that allows you to write the code for a particular topic or topic. It can be used to create a code for a specific topic or topic, such as a topic or topic. The code will then be sent back to you for further processing. 

Some examples of Python are:

- PHP - This is a Python code that allows you to write code for a specific topic or topic. It can be used to create a code for a specific topic or topic.

- Python - This code allows you to create a code for a specific topic or topic. It also allows you to create a code for a particular topic or topic.

- Python - This code allows you to create a code for a specific topic or topic in a specific subject area. You can also create code for an existing topic or topic.

VDrontV1.5_Q4.dq

Q: What is a Python? 

A:The Python world is not only useful, but it also allows the best thing to be found in the real world. The world also serves as the most time for your next post! The best time for you are in the time of your next post time in time (we have no big time, they are they? We run time, they are they? We have no longer, they are they, they are they, they are they? Maybe they? The end time, they are they? The end time, they are they? The end time, they are they, they are they. The end time, they are they? They are they are they? We are they? The end time, they are they? Maybe they? We're they? The end time, they're they? They are they?

Details

Context length: 512

Tokenizer: gpt2

Base model: VdrontV1 (ours)

Fine-tuning on GPU using the Lion optimizer

The number of parameters has been increased by creating new layers.

Ratio of FW-Edu to FW-Edu-instruct training = 3:1

Use of Drontal Quant.

License : mit.

Guide

To use the HF model, go to the files, download the VDrontV1.5/ folder and use.py, and place them together. use.py includes prompt formatting with <|user|> and <|assistant|> and is ready for immediate use.

To use Dront Quant models, go to the VDrontV1.5_quant/ folder. It contains 3 .dq models (4-bit, 8-bit, and 16-bit) as well as DQuse.py. Download the model you are interested in and place DQuse.py and the .dq model next to each other. At the end of the DQuse.py file, change the model name to the one you want to use. Note that the prompt formatting uses <|user|> and <|assistant|> tokens. (388 line: chat_with_dq_model("[path].dq", 256) )

Fine-tune

If you fine-tune the VDrontV1.5 model, we will most likely add it to the VDrontV collection.

Python libraries:

DQuse.py : torch json os struct numpy collections transformers

use.py : torch transformers threading

DrontQuant : https://huggingface.co/VDC-team/DrontGPTQ

Hugging Face : VDC-team/

P.S The next model is VDrontV1.5-Flash (60M parameters).