Hugging Face's logo

wiiiktor
/
chopper_demo

rag
context-compression
Model card Files
xet
 Community

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Chopper demo: weights moved

LLMs need a Chopper: watch the video

Option 1, universal chopper: a small separate model (mmBERT-base 0.31B or mmBERT-small 0.14B), text in and text out, works with any answering model. Option 2, integrated chopper: a 20% slice of MiniCPM3-4B itself, no second model, reads the model's own cache of the whole session.

70-second video: youtu.be/KMvmdXCEOJU

Both choppers are only fine-tuned. The universal chopper is mmBERT, a general-purpose encoder, fine-tuned for chopping; the integrated chopper is a slice of MiniCPM3-4B, a model pretrained for chat, fine-tuned with a LoRA. Neither was trained for chopping from the start. Trained from scratch for this purpose, both should be better, and the integrated one has the larger potential: it reads the whole conversation straight from the model's own cache, which a separate encoder never sees.

The weights are in one repo per chopper, each with its own license and base model:

repo what license
wiiiktor/chopper_mmbert 🟧 universal chopper (option 1): mmBERT-base and mmBERT-small, text in, text out, any answering model MIT
wiiiktor/chopper_minicpm3 🟪 integrated chopper (option 2): a 20% slice inside MiniCPM3-4B (latent attention), no second model MiniCPM Model License (commercial use needs registration with OpenBMB)

Data: the dataset repo wiiiktor/chopper_demo. Code: github.com/wiiiktor/chopper_demo.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support