Instructions to use onnx-community/LFM2-8B-A1B-ONNX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use onnx-community/LFM2-8B-A1B-ONNX with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('text-generation', 'onnx-community/LFM2-8B-A1B-ONNX');
Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,7 @@ tags:
|
|
| 19 |
- moe
|
| 20 |
base_model:
|
| 21 |
- LiquidAI/LFM2-8B-A1B
|
|
|
|
| 22 |
---
|
| 23 |
|
| 24 |
<center>
|
|
@@ -118,7 +119,43 @@ You can directly pass tools as JSON schema or Python functions with `.apply_chat
|
|
| 118 |
|
| 119 |
## 🏃 How to run LFM2
|
| 120 |
|
| 121 |
-
###
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 122 |
|
| 123 |
```py
|
| 124 |
from transformers import AutoConfig, AutoTokenizer
|
|
|
|
| 19 |
- moe
|
| 20 |
base_model:
|
| 21 |
- LiquidAI/LFM2-8B-A1B
|
| 22 |
+
library_name: transformers.js
|
| 23 |
---
|
| 24 |
|
| 25 |
<center>
|
|
|
|
| 119 |
|
| 120 |
## 🏃 How to run LFM2
|
| 121 |
|
| 122 |
+
### Transformers.js
|
| 123 |
+
|
| 124 |
+
If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@huggingface/transformers) using:
|
| 125 |
+
|
| 126 |
+
```sh
|
| 127 |
+
npm i @huggingface/transformers
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
You can then use the model as follows:
|
| 131 |
+
```js
|
| 132 |
+
import { pipeline, TextStreamer } from "@huggingface/transformers";
|
| 133 |
+
|
| 134 |
+
// Create a text generation pipeline
|
| 135 |
+
const generator = await pipeline(
|
| 136 |
+
"text-generation",
|
| 137 |
+
"onnx-community/LFM2-8B-A1B-ONNX",
|
| 138 |
+
{ dtype: "q4f16", device: "webgpu" },
|
| 139 |
+
);
|
| 140 |
+
|
| 141 |
+
// Define the list of messages
|
| 142 |
+
const messages = [
|
| 143 |
+
{ role: "user", content: "What's the capital of France?" },
|
| 144 |
+
];
|
| 145 |
+
|
| 146 |
+
// Generate a response
|
| 147 |
+
const output = await generator(messages, {
|
| 148 |
+
max_new_tokens: 512,
|
| 149 |
+
do_sample: false,
|
| 150 |
+
streamer: new TextStreamer(generator.tokenizer, {
|
| 151 |
+
skip_prompt: true,
|
| 152 |
+
skip_special_tokens: true,
|
| 153 |
+
}),
|
| 154 |
+
});
|
| 155 |
+
console.log(output[0].generated_text.at(-1).content);
|
| 156 |
+
```
|
| 157 |
+
|
| 158 |
+
### ONNXRuntime
|
| 159 |
|
| 160 |
```py
|
| 161 |
from transformers import AutoConfig, AutoTokenizer
|