Create README.md

README.md

@@ -0,0 +1,43 @@
+---
+language:
+- en
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- LLM
+- Universal-NER
+- NER
+inference: false
+---
+
+# Quantized version of Universal-NER/UniNER-7B-all
+
+[Universal-NER/UniNER-7B-all](https://huggingface.co/Universal-NER/UniNER-7B-all) quantized to 4bit with GPTQ and stored with 1GB shard size.
+
+## Model Description
+
+The model [Universal-NER/UniNER-7B-all](https://huggingface.co/Universal-NER/UniNER-7B-all) was quantized to 4bit, group_size 128, and ascending_order=True with auto-gptq integration in transformers (https://huggingface.co/blog/gptq-integration).
+
+## Evaluation
+TODO
+
+## Prompt template
+
+Prompt template is the same as for the full precision model:
+
+```python
+prompt_template = """A virtual assistant answers questions from a user based on the provided text.
+USER: Text: {input_text}
+ASSISTANT: I’ve read this text.
+USER: What describes {entity_name} in the text?
+ASSISTANT:
+"""
+```
+
+## Usage
+
+It is recommended to format input according to the prompt template mentioned above during inference for best results.
+
+```python
+prompt = prompt_template.format_map({"input_text": "Cologne is a great city in Germany - maybe even the greatest ;)", "entity_name": "city"})
+```