Xenova HF staff commited on
Commit
e6ce510
1 Parent(s): 921866c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +76 -0
README.md CHANGED
@@ -4,4 +4,80 @@ library_name: transformers.js
4
 
5
  https://huggingface.co/nielsr/slimsam-50-uniform with ONNX weights to be compatible with Transformers.js.
6
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).
 
4
 
5
  https://huggingface.co/nielsr/slimsam-50-uniform with ONNX weights to be compatible with Transformers.js.
6
 
7
+
8
+ ## Usage (Transformers.js)
9
+
10
+ If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@xenova/transformers) using:
11
+ ```bash
12
+ npm i @xenova/transformers
13
+ ```
14
+
15
+ **Example:** Perform mask generation with `Xenova/slimsam-50-uniform`.
16
+
17
+ ```js
18
+ import { SamModel, AutoProcessor, RawImage } from '@xenova/transformers';
19
+
20
+ // Load model and processor
21
+ const model = await SamModel.from_pretrained('Xenova/slimsam-50-uniform');
22
+ const processor = await AutoProcessor.from_pretrained('Xenova/slimsam-50-uniform');
23
+
24
+ // Prepare image and input points
25
+ const img_url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/corgi.jpg';
26
+ const raw_image = await RawImage.read(img_url);
27
+ const input_points = [[[340, 250]]] // 2D localization of a window
28
+
29
+ // Process inputs and perform mask generation
30
+ const inputs = await processor(raw_image, input_points);
31
+ const outputs = await model(inputs);
32
+
33
+ // Post-process masks
34
+ const masks = await processor.post_process_masks(outputs.pred_masks, inputs.original_sizes, inputs.reshaped_input_sizes);
35
+ console.log(masks);
36
+ // [
37
+ // Tensor {
38
+ // dims: [ 1, 3, 410, 614 ],
39
+ // type: 'bool',
40
+ // data: Uint8Array(755220) [ ... ],
41
+ // size: 755220
42
+ // }
43
+ // ]
44
+
45
+ const scores = outputs.iou_scores;
46
+ console.log(scores);
47
+ // Tensor {
48
+ // dims: [ 1, 1, 3 ],
49
+ // type: 'float32',
50
+ // data: Float32Array(3) [
51
+ // 0.8649908900260925,
52
+ // 0.9900082349777222,
53
+ // 0.9232960939407349
54
+ // ],
55
+ // size: 3
56
+ // }
57
+ ```
58
+
59
+ You can then visualize the generated mask with:
60
+
61
+ ```js
62
+ const image = RawImage.fromTensor(masks[0][0].mul(255));
63
+ image.save('mask.png');
64
+ ```
65
+
66
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61b253b7ac5ecaae3d1efe0c/YQDj11Lh7XRoiytCWUo3u.png)
67
+
68
+ Next, select the channel with the highest IoU score, which in this case is the second (green) channel. Intersecting this with the original image gives us an isolated version of the subject:
69
+
70
+
71
+ ![image/gif](https://cdn-uploads.huggingface.co/production/uploads/61b253b7ac5ecaae3d1efe0c/8qPBGfHB53136vwg_89Qs.gif)
72
+
73
+ ## Demo
74
+
75
+ We've also got an online demo, which you can try out [here](https://huggingface.co/spaces/Xenova/segment-anything-web).
76
+
77
+
78
+ <video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/61b253b7ac5ecaae3d1efe0c/Y0wAOw6hz9rWpwiuMoz2A.mp4"></video>
79
+
80
+ ---
81
+
82
+
83
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).