RichardErkhov/tim1900_-_cvx-coder-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

cvx-coder - GGUF

• Model creator: https://huggingface.co/tim1900/
• Original model: https://huggingface.co/tim1900/cvx-coder/

Name	Quant method	Size
cvx-coder.Q2_K.gguf	Q2_K	1.32GB
cvx-coder.IQ3_XS.gguf	IQ3_XS	1.51GB
cvx-coder.IQ3_S.gguf	IQ3_S	1.57GB
cvx-coder.Q3_K_S.gguf	Q3_K_S	1.57GB
cvx-coder.IQ3_M.gguf	IQ3_M	1.73GB
cvx-coder.Q3_K.gguf	Q3_K	1.82GB
cvx-coder.Q3_K_M.gguf	Q3_K_M	1.82GB
cvx-coder.Q3_K_L.gguf	Q3_K_L	1.94GB
cvx-coder.IQ4_XS.gguf	IQ4_XS	1.93GB
cvx-coder.Q4_0.gguf	Q4_0	2.03GB
cvx-coder.IQ4_NL.gguf	IQ4_NL	2.04GB
cvx-coder.Q4_K_S.gguf	Q4_K_S	2.04GB
cvx-coder.Q4_K.gguf	Q4_K	2.23GB
cvx-coder.Q4_K_M.gguf	Q4_K_M	2.23GB
cvx-coder.Q4_1.gguf	Q4_1	2.24GB
cvx-coder.Q5_0.gguf	Q5_0	2.46GB
cvx-coder.Q5_K_S.gguf	Q5_K_S	2.46GB
cvx-coder.Q5_K.gguf	Q5_K	2.62GB
cvx-coder.Q5_K_M.gguf	Q5_K_M	2.62GB
cvx-coder.Q5_1.gguf	Q5_1	2.68GB
cvx-coder.Q6_K.gguf	Q6_K	2.92GB
cvx-coder.Q8_0.gguf	Q8_0	3.78GB

Original model description:

license: mit

language: - en pipeline_tag: text-generation tags: - nlp - code inference: parameters: temperature: 0.0 widget: - messages: - role: user content: How to express n-th root of the determinant of a semidefinite matrix in cvx?

cvx-coder

Github | Modelscope

Introduction

cvx-coder aims to improve the Matlab CVX code ability and QA ability of LLMs. It is a phi-3 model finetuned on a dataset consisting of CVX docs, codes, forum conversations ( my cleaned version of them is at CVX-forum-conversations).

Quickstart

For one quick test, run the following:

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
m_path="tim1900/cvx-coder"
model = AutoModelForCausalLM.from_pretrained(
    m_path, 
    device_map="auto", 
    torch_dtype="auto", 
    trust_remote_code=True, 
)
tokenizer = AutoTokenizer.from_pretrained(m_path)
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
)
generation_args = {
    "max_new_tokens": 2000,
    "return_full_text": False,
    "temperature": 0,
    "do_sample": False,
}
content='''my problem is not convex, can i use cvx? if not, what should i do, be specific.'''
messages = [
    {"role": "user", "content": content},
]
output = pipe(messages, **generation_args)
print(output[0]['generated_text'])

For the chat mode in web, run the following:

import gradio as gr
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
m_path="tim1900/cvx-coder"
model = AutoModelForCausalLM.from_pretrained(
    m_path, 
    device_map="auto", 
    torch_dtype="auto", 
    trust_remote_code=True, 
)
tokenizer = AutoTokenizer.from_pretrained(m_path)
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
)
generation_args = {
    "max_new_tokens": 2000,
    "return_full_text": False,
    "temperature": 0,
    "do_sample": False,
}

def assistant_talk(message, history):
    message=[
        {"role": "user", "content": message},
        ]
    temp=[]
    for i in history:
        temp+=[{"role": "user", "content": i[0]},{"role": "assistant", "content": i[1]}]
        
    messages =temp  + message

    output = pipe(messages, **generation_args)
    return output[0]['generated_text']
gr.ChatInterface(assistant_talk).launch()