Instructions to use ncauchi1/cv_pointing_model_2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ncauchi1/cv_pointing_model_2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ncauchi1/cv_pointing_model_2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ncauchi1/cv_pointing_model_2")
model = AutoModelForMultimodalLM.from_pretrained("ncauchi1/cv_pointing_model_2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ncauchi1/cv_pointing_model_2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ncauchi1/cv_pointing_model_2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ncauchi1/cv_pointing_model_2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ncauchi1/cv_pointing_model_2

SGLang

How to use ncauchi1/cv_pointing_model_2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ncauchi1/cv_pointing_model_2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ncauchi1/cv_pointing_model_2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ncauchi1/cv_pointing_model_2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ncauchi1/cv_pointing_model_2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ncauchi1/cv_pointing_model_2 with Docker Model Runner:
```
docker model run hf.co/ncauchi1/cv_pointing_model_2
```

Model Card for Model ID

Used to point to voltage peaks in Cyclic Voltammetry graphs, fine tuned to return peak location in pixel coordinates with XML tags.

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

Developed by: [More Information Needed]
Funded by [optional]: [More Information Needed]
Shared by [optional]: [More Information Needed]
Model type: [More Information Needed]
Language(s) (NLP): [More Information Needed]
License: [More Information Needed]
Finetuned from model [optional]: [More Information Needed]

Model Sources [optional]

Repository: [More Information Needed]
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Training Details

Fine tuned on hf dataset ncauchi1/pointing_demo_diverse on 10k samples. Base Model is Qwen2.5 VL 3B, trained in two parts of 5k each.

Dataset was improved from previous version, raw data was normalized to remove issues with scaling on graph, then randomly re-scaled to provide variation. Sample CV's with no peaks, and graphs with different nubers of CV's were added for more variation.

Training logs: https://wandb.ai/ncauchi-university-of-maryland/huggingface/runs/to254a01/logs
https://wandb.ai/ncauchi-university-of-maryland/huggingface/runs/rhvllmjz/logs

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

Training regime: [ --torch_dtype bfloat16
--num_train_epochs 5
--per_device_train_batch_size 8
--per_device_eval_batch_size 8
--learning_rate 1e-4
--freeze_vit false
--freeze_aligner false
--lora_rank 16
--lora_alpha 32
--gradient_accumulation_steps 1
--eval_steps 20
--save_steps 100
--save_total_limit 2
--logging_steps 5
--max_length 8192
--output_dir output
--warmup_ratio 0.05
--dataloader_num_workers 4
--dataset_num_proc 4
--deepspeed zero2
--save_only_model true
--use_hf true
--report_to wandb]

Evaluation

Model was evaluated on test set and holdout set. (holdout set comprised of raw data that was not used to generate train/test set)

Model scored 98% Accuracy on test set and 96% accuracy on holdout set. *Holdout set was based on old dataset so there were no samples without peaks

Most errors seemed reasonable, sometimes having trouble identifying 'lower' and 'upper peaks', missing small peaks, or mistaking other curve features for peaks. Compared to previous mode never guessed peaks off of the graph and generalized much better (previous model scored ~68% on holdout set)

Model was also evaluated on general CV questions dataset (bxw315-umd/general-cv-questions) and scored _20% (prob to answer correctly) compared to 23.3%

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: [More Information Needed]
Hours used: [More Information Needed]
Cloud Provider: [More Information Needed]
Compute Region: [More Information Needed]
Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

Downloads last month: 18

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for ncauchi1/cv_pointing_model_2

Base model

Qwen/Qwen2.5-VL-3B-Instruct

Finetuned

(793)

this model

Dataset used to train ncauchi1/cv_pointing_model_2

Paper for ncauchi1/cv_pointing_model_2

Quantifying the Carbon Emissions of Machine Learning

Paper • 1910.09700 • Published Oct 21, 2019 • 51