Hugging Face's logo

mlx-community
/
sam3-4bit

MLX
Safetensors
sam3_video
sam3
segmentation
detection
tracking
4-bit precision
Model card Files
xet
 Community
prince-canuma commited on
Commit
c110fcd
·
verified ·
1 Parent(s): d0f53a9

Upload folder using huggingface_hub

Browse files
Files changed (9) hide show
  1. README.md +50 -0
  2. config.json +907 -0
  3. model.safetensors +3 -0
  4. model.safetensors.index.json +0 -0
  5. processor_config.json +25 -0
  6. special_tokens_map.json +30 -0
  7. tokenizer.json +0 -0
  8. tokenizer_config.json +13 -0
  9. vocab.json +0 -0
README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ extra_gated_fields:
4
+ First Name: text
5
+ Last Name: text
6
+ Date of birth: date_picker
7
+ Country: country
8
+ Affiliation: text
9
+ Job title:
10
+ type: select
11
+ options:
12
+ - Student
13
+ - Research Graduate
14
+ - AI researcher
15
+ - AI developer/engineer
16
+ - Reporter
17
+ - Other
18
+ geo: ip_location
19
+ ? By clicking Submit below I accept the terms of the license and acknowledge that
20
+ the information I provide will be collected stored processed and shared in accordance
21
+ with the Meta Privacy Policy
22
+ : checkbox
23
+ extra_gated_description: The information you provide will be collected, stored, processed
24
+ and shared in accordance with the [Meta Privacy Policy](https://www.facebook.com/privacy/policy/).
25
+ extra_gated_button_content: Submit
26
+ language:
27
+ - en
28
+ pipeline_tag: mask-generation
29
+ library_name: mlx
30
+ tags:
31
+ - sam3
32
+ - mlx
33
+ base_model: facebook/sam3
34
+ ---
35
+
36
+ # mlx-community/sam3-4bit
37
+
38
+ This model was converted to MLX format from [`facebook/sam3`](https://huggingface.co/facebook/sam3)
39
+ using mlx-vlm version **0.4.2**.
40
+ Refer to the [original model card](https://huggingface.co/facebook/sam3) for more details on the model.
41
+
42
+ ## Use with mlx
43
+
44
+ ```bash
45
+ pip install -U mlx-vlm
46
+ ```
47
+
48
+ ```bash
49
+ python -m mlx_vlm.generate --model mlx-community/sam3-4bit --max-tokens 100 --temperature 0.0 --prompt "Describe this image." --image <path_to_image>
50
+ ```
config.json ADDED
@@ -0,0 +1,907 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Sam3VideoModel"
4
+ ],
5
+ "assoc_iou_thresh": 0.1,
6
+ "decrease_trk_keep_alive_for_empty_masklets": false,
7
+ "det_nms_thresh": 0.1,
8
+ "detector_config": {
9
+ "detr_decoder_config": {
10
+ "_name_or_path": "",
11
+ "add_cross_attention": false,
12
+ "architectures": null,
13
+ "bad_words_ids": null,
14
+ "begin_suppress_tokens": null,
15
+ "bos_token_id": null,
16
+ "box_rpb_mode": "log",
17
+ "chunk_size_feed_forward": 0,
18
+ "cross_attention_hidden_size": null,
19
+ "decoder_start_token_id": null,
20
+ "diversity_penalty": 0.0,
21
+ "do_sample": false,
22
+ "dropout": 0.1,
23
+ "dtype": null,
24
+ "early_stopping": false,
25
+ "encoder_no_repeat_ngram_size": 0,
26
+ "eos_token_id": null,
27
+ "exponential_decay_length_penalty": null,
28
+ "finetuning_task": null,
29
+ "forced_bos_token_id": null,
30
+ "forced_eos_token_id": null,
31
+ "hidden_act": "relu",
32
+ "hidden_dropout": 0.0,
33
+ "hidden_size": 256,
34
+ "id2label": {
35
+ "0": "LABEL_0",
36
+ "1": "LABEL_1"
37
+ },
38
+ "initializer_range": 0.02,
39
+ "intermediate_size": 2048,
40
+ "is_decoder": false,
41
+ "is_encoder_decoder": false,
42
+ "label2id": {
43
+ "LABEL_0": 0,
44
+ "LABEL_1": 1
45
+ },
46
+ "layer_norm_eps": 1e-06,
47
+ "length_penalty": 1.0,
48
+ "max_length": 20,
49
+ "min_length": 0,
50
+ "model_type": "sam3_detr_decoder",
51
+ "no_repeat_ngram_size": 0,
52
+ "num_attention_heads": 8,
53
+ "num_beam_groups": 1,
54
+ "num_beams": 1,
55
+ "num_layers": 6,
56
+ "num_queries": 200,
57
+ "num_return_sequences": 1,
58
+ "output_attentions": false,
59
+ "output_hidden_states": false,
60
+ "output_scores": false,
61
+ "pad_token_id": null,
62
+ "prefix": null,
63
+ "problem_type": null,
64
+ "remove_invalid_values": false,
65
+ "repetition_penalty": 1.0,
66
+ "return_dict": true,
67
+ "return_dict_in_generate": false,
68
+ "sep_token_id": null,
69
+ "suppress_tokens": null,
70
+ "task_specific_params": null,
71
+ "temperature": 1.0,
72
+ "tie_encoder_decoder": false,
73
+ "tie_word_embeddings": true,
74
+ "tokenizer_class": null,
75
+ "top_k": 50,
76
+ "top_p": 1.0,
77
+ "typical_p": 1.0,
78
+ "use_presence_token": true
79
+ },
80
+ "detr_encoder_config": {
81
+ "_name_or_path": "",
82
+ "add_cross_attention": false,
83
+ "architectures": null,
84
+ "bad_words_ids": null,
85
+ "begin_suppress_tokens": null,
86
+ "bos_token_id": null,
87
+ "chunk_size_feed_forward": 0,
88
+ "cross_attention_hidden_size": null,
89
+ "decoder_start_token_id": null,
90
+ "diversity_penalty": 0.0,
91
+ "do_sample": false,
92
+ "dropout": 0.1,
93
+ "dtype": null,
94
+ "early_stopping": false,
95
+ "encoder_no_repeat_ngram_size": 0,
96
+ "eos_token_id": null,
97
+ "exponential_decay_length_penalty": null,
98
+ "finetuning_task": null,
99
+ "forced_bos_token_id": null,
100
+ "forced_eos_token_id": null,
101
+ "hidden_act": "relu",
102
+ "hidden_dropout": 0.0,
103
+ "hidden_size": 256,
104
+ "id2label": {
105
+ "0": "LABEL_0",
106
+ "1": "LABEL_1"
107
+ },
108
+ "initializer_range": 0.02,
109
+ "intermediate_size": 2048,
110
+ "is_decoder": false,
111
+ "is_encoder_decoder": false,
112
+ "label2id": {
113
+ "LABEL_0": 0,
114
+ "LABEL_1": 1
115
+ },
116
+ "layer_norm_eps": 1e-06,
117
+ "length_penalty": 1.0,
118
+ "max_length": 20,
119
+ "min_length": 0,
120
+ "model_type": "sam3_detr_encoder",
121
+ "no_repeat_ngram_size": 0,
122
+ "num_attention_heads": 8,
123
+ "num_beam_groups": 1,
124
+ "num_beams": 1,
125
+ "num_layers": 6,
126
+ "num_return_sequences": 1,
127
+ "output_attentions": false,
128
+ "output_hidden_states": false,
129
+ "output_scores": false,
130
+ "pad_token_id": null,
131
+ "prefix": null,
132
+ "problem_type": null,
133
+ "remove_invalid_values": false,
134
+ "repetition_penalty": 1.0,
135
+ "return_dict": true,
136
+ "return_dict_in_generate": false,
137
+ "sep_token_id": null,
138
+ "suppress_tokens": null,
139
+ "task_specific_params": null,
140
+ "temperature": 1.0,
141
+ "tie_encoder_decoder": false,
142
+ "tie_word_embeddings": true,
143
+ "tokenizer_class": null,
144
+ "top_k": 50,
145
+ "top_p": 1.0,
146
+ "typical_p": 1.0
147
+ },
148
+ "geometry_encoder_config": {
149
+ "_name_or_path": "",
150
+ "add_cross_attention": false,
151
+ "architectures": null,
152
+ "bad_words_ids": null,
153
+ "begin_suppress_tokens": null,
154
+ "bos_token_id": null,
155
+ "chunk_size_feed_forward": 0,
156
+ "cross_attention_hidden_size": null,
157
+ "decoder_start_token_id": null,
158
+ "diversity_penalty": 0.0,
159
+ "do_sample": false,
160
+ "dropout": 0.1,
161
+ "dtype": null,
162
+ "early_stopping": false,
163
+ "encoder_no_repeat_ngram_size": 0,
164
+ "eos_token_id": null,
165
+ "exponential_decay_length_penalty": null,
166
+ "finetuning_task": null,
167
+ "forced_bos_token_id": null,
168
+ "forced_eos_token_id": null,
169
+ "hidden_act": "relu",
170
+ "hidden_dropout": 0.0,
171
+ "hidden_size": 256,
172
+ "id2label": {
173
+ "0": "LABEL_0",
174
+ "1": "LABEL_1"
175
+ },
176
+ "initializer_range": 0.02,
177
+ "intermediate_size": 2048,
178
+ "is_decoder": false,
179
+ "is_encoder_decoder": false,
180
+ "label2id": {
181
+ "LABEL_0": 0,
182
+ "LABEL_1": 1
183
+ },
184
+ "layer_norm_eps": 1e-06,
185
+ "length_penalty": 1.0,
186
+ "max_length": 20,
187
+ "min_length": 0,
188
+ "model_type": "sam3_geometry_encoder",
189
+ "no_repeat_ngram_size": 0,
190
+ "num_attention_heads": 8,
191
+ "num_beam_groups": 1,
192
+ "num_beams": 1,
193
+ "num_layers": 3,
194
+ "num_return_sequences": 1,
195
+ "output_attentions": false,
196
+ "output_hidden_states": false,
197
+ "output_scores": false,
198
+ "pad_token_id": null,
199
+ "prefix": null,
200
+ "problem_type": null,
201
+ "remove_invalid_values": false,
202
+ "repetition_penalty": 1.0,
203
+ "return_dict": true,
204
+ "return_dict_in_generate": false,
205
+ "roi_size": 7,
206
+ "sep_token_id": null,
207
+ "suppress_tokens": null,
208
+ "task_specific_params": null,
209
+ "temperature": 1.0,
210
+ "tie_encoder_decoder": false,
211
+ "tie_word_embeddings": true,
212
+ "tokenizer_class": null,
213
+ "top_k": 50,
214
+ "top_p": 1.0,
215
+ "typical_p": 1.0
216
+ },
217
+ "initializer_range": 0.02,
218
+ "mask_decoder_config": {
219
+ "_name_or_path": "",
220
+ "add_cross_attention": false,
221
+ "architectures": null,
222
+ "bad_words_ids": null,
223
+ "begin_suppress_tokens": null,
224
+ "bos_token_id": null,
225
+ "chunk_size_feed_forward": 0,
226
+ "cross_attention_hidden_size": null,
227
+ "decoder_start_token_id": null,
228
+ "diversity_penalty": 0.0,
229
+ "do_sample": false,
230
+ "dropout": 0.0,
231
+ "dtype": null,
232
+ "early_stopping": false,
233
+ "encoder_no_repeat_ngram_size": 0,
234
+ "eos_token_id": null,
235
+ "exponential_decay_length_penalty": null,
236
+ "finetuning_task": null,
237
+ "forced_bos_token_id": null,
238
+ "forced_eos_token_id": null,
239
+ "hidden_size": 256,
240
+ "id2label": {
241
+ "0": "LABEL_0",
242
+ "1": "LABEL_1"
243
+ },
244
+ "initializer_range": 0.02,
245
+ "is_decoder": false,
246
+ "is_encoder_decoder": false,
247
+ "label2id": {
248
+ "LABEL_0": 0,
249
+ "LABEL_1": 1
250
+ },
251
+ "layer_norm_eps": 1e-06,
252
+ "length_penalty": 1.0,
253
+ "max_length": 20,
254
+ "min_length": 0,
255
+ "model_type": "sam3_mask_decoder",
256
+ "no_repeat_ngram_size": 0,
257
+ "num_attention_heads": 8,
258
+ "num_beam_groups": 1,
259
+ "num_beams": 1,
260
+ "num_return_sequences": 1,
261
+ "num_upsampling_stages": 3,
262
+ "output_attentions": false,
263
+ "output_hidden_states": false,
264
+ "output_scores": false,
265
+ "pad_token_id": null,
266
+ "prefix": null,
267
+ "problem_type": null,
268
+ "remove_invalid_values": false,
269
+ "repetition_penalty": 1.0,
270
+ "return_dict": true,
271
+ "return_dict_in_generate": false,
272
+ "sep_token_id": null,
273
+ "suppress_tokens": null,
274
+ "task_specific_params": null,
275
+ "temperature": 1.0,
276
+ "tie_encoder_decoder": false,
277
+ "tie_word_embeddings": true,
278
+ "tokenizer_class": null,
279
+ "top_k": 50,
280
+ "top_p": 1.0,
281
+ "typical_p": 1.0
282
+ },
283
+ "model_type": "sam3",
284
+ "text_config": {
285
+ "_name_or_path": "",
286
+ "add_cross_attention": false,
287
+ "architectures": null,
288
+ "attention_dropout": 0.0,
289
+ "bad_words_ids": null,
290
+ "begin_suppress_tokens": null,
291
+ "bos_token_id": 49406,
292
+ "chunk_size_feed_forward": 0,
293
+ "cross_attention_hidden_size": null,
294
+ "decoder_start_token_id": null,
295
+ "diversity_penalty": 0.0,
296
+ "do_sample": false,
297
+ "dtype": null,
298
+ "early_stopping": false,
299
+ "encoder_no_repeat_ngram_size": 0,
300
+ "eos_token_id": 49407,
301
+ "exponential_decay_length_penalty": null,
302
+ "finetuning_task": null,
303
+ "forced_bos_token_id": null,
304
+ "forced_eos_token_id": null,
305
+ "hidden_act": "gelu",
306
+ "hidden_size": 1024,
307
+ "id2label": {
308
+ "0": "LABEL_0",
309
+ "1": "LABEL_1"
310
+ },
311
+ "initializer_factor": 1.0,
312
+ "initializer_range": 0.02,
313
+ "intermediate_size": 4096,
314
+ "is_decoder": false,
315
+ "is_encoder_decoder": false,
316
+ "label2id": {
317
+ "LABEL_0": 0,
318
+ "LABEL_1": 1
319
+ },
320
+ "layer_norm_eps": 1e-05,
321
+ "length_penalty": 1.0,
322
+ "max_length": 20,
323
+ "max_position_embeddings": 32,
324
+ "min_length": 0,
325
+ "model_type": "clip_text_model",
326
+ "no_repeat_ngram_size": 0,
327
+ "num_attention_heads": 16,
328
+ "num_beam_groups": 1,
329
+ "num_beams": 1,
330
+ "num_hidden_layers": 24,
331
+ "num_return_sequences": 1,
332
+ "output_attentions": false,
333
+ "output_hidden_states": false,
334
+ "output_scores": false,
335
+ "pad_token_id": 1,
336
+ "prefix": null,
337
+ "problem_type": null,
338
+ "projection_dim": 512,
339
+ "remove_invalid_values": false,
340
+ "repetition_penalty": 1.0,
341
+ "return_dict": true,
342
+ "return_dict_in_generate": false,
343
+ "sep_token_id": null,
344
+ "suppress_tokens": null,
345
+ "task_specific_params": null,
346
+ "temperature": 1.0,
347
+ "tie_encoder_decoder": false,
348
+ "tie_word_embeddings": true,
349
+ "tokenizer_class": null,
350
+ "top_k": 50,
351
+ "top_p": 1.0,
352
+ "typical_p": 1.0,
353
+ "vocab_size": 49408
354
+ },
355
+ "vision_config": {
356
+ "_name_or_path": "",
357
+ "add_cross_attention": false,
358
+ "architectures": null,
359
+ "backbone_config": {
360
+ "_name_or_path": "",
361
+ "add_cross_attention": false,
362
+ "architectures": null,
363
+ "attention_dropout": 0.0,
364
+ "bad_words_ids": null,
365
+ "begin_suppress_tokens": null,
366
+ "bos_token_id": null,
367
+ "chunk_size_feed_forward": 0,
368
+ "cross_attention_hidden_size": null,
369
+ "decoder_start_token_id": null,
370
+ "diversity_penalty": 0.0,
371
+ "do_sample": false,
372
+ "dtype": null,
373
+ "early_stopping": false,
374
+ "encoder_no_repeat_ngram_size": 0,
375
+ "eos_token_id": null,
376
+ "exponential_decay_length_penalty": null,
377
+ "finetuning_task": null,
378
+ "forced_bos_token_id": null,
379
+ "forced_eos_token_id": null,
380
+ "global_attn_indexes": [
381
+ 7,
382
+ 15,
383
+ 23,
384
+ 31
385
+ ],
386
+ "hidden_act": "gelu",
387
+ "hidden_dropout": 0.0,
388
+ "hidden_size": 1024,
389
+ "id2label": {
390
+ "0": "LABEL_0",
391
+ "1": "LABEL_1"
392
+ },
393
+ "image_size": 1008,
394
+ "initializer_range": 0.02,
395
+ "intermediate_size": 4736,
396
+ "is_decoder": false,
397
+ "is_encoder_decoder": false,
398
+ "label2id": {
399
+ "LABEL_0": 0,
400
+ "LABEL_1": 1
401
+ },
402
+ "layer_norm_eps": 1e-06,
403
+ "layer_scale_init_value": null,
404
+ "length_penalty": 1.0,
405
+ "max_length": 20,
406
+ "min_length": 0,
407
+ "model_type": "sam3_vit_model",
408
+ "no_repeat_ngram_size": 0,
409
+ "num_attention_heads": 16,
410
+ "num_beam_groups": 1,
411
+ "num_beams": 1,
412
+ "num_channels": 3,
413
+ "num_hidden_layers": 32,
414
+ "num_return_sequences": 1,
415
+ "output_attentions": false,
416
+ "output_hidden_states": false,
417
+ "output_scores": false,
418
+ "pad_token_id": null,
419
+ "patch_size": 14,
420
+ "prefix": null,
421
+ "pretrain_image_size": 336,
422
+ "problem_type": null,
423
+ "qkv_bias": true,
424
+ "remove_invalid_values": false,
425
+ "repetition_penalty": 1.0,
426
+ "return_dict": true,
427
+ "return_dict_in_generate": false,
428
+ "rope_theta": 10000.0,
429
+ "sep_token_id": null,
430
+ "suppress_tokens": null,
431
+ "task_specific_params": null,
432
+ "temperature": 1.0,
433
+ "tie_encoder_decoder": false,
434
+ "tie_word_embeddings": true,
435
+ "tokenizer_class": null,
436
+ "top_k": 50,
437
+ "top_p": 1.0,
438
+ "typical_p": 1.0,
439
+ "window_size": 24
440
+ },
441
+ "backbone_feature_sizes": [
442
+ [
443
+ 288,
444
+ 288
445
+ ],
446
+ [
447
+ 144,
448
+ 144
449
+ ],
450
+ [
451
+ 72,
452
+ 72
453
+ ]
454
+ ],
455
+ "bad_words_ids": null,
456
+ "begin_suppress_tokens": null,
457
+ "bos_token_id": null,
458
+ "chunk_size_feed_forward": 0,
459
+ "cross_attention_hidden_size": null,
460
+ "decoder_start_token_id": null,
461
+ "diversity_penalty": 0.0,
462
+ "do_sample": false,
463
+ "dtype": null,
464
+ "early_stopping": false,
465
+ "encoder_no_repeat_ngram_size": 0,
466
+ "eos_token_id": null,
467
+ "exponential_decay_length_penalty": null,
468
+ "finetuning_task": null,
469
+ "forced_bos_token_id": null,
470
+ "forced_eos_token_id": null,
471
+ "fpn_hidden_size": 256,
472
+ "fpn_kernel_size": 2,
473
+ "fpn_stride": 2,
474
+ "hidden_act": "gelu",
475
+ "id2label": {
476
+ "0": "LABEL_0",
477
+ "1": "LABEL_1"
478
+ },
479
+ "initializer_range": 0.02,
480
+ "is_decoder": false,
481
+ "is_encoder_decoder": false,
482
+ "label2id": {
483
+ "LABEL_0": 0,
484
+ "LABEL_1": 1
485
+ },
486
+ "layer_norm_eps": 1e-06,
487
+ "length_penalty": 1.0,
488
+ "max_length": 20,
489
+ "min_length": 0,
490
+ "model_type": "sam3_vision_model",
491
+ "no_repeat_ngram_size": 0,
492
+ "num_beam_groups": 1,
493
+ "num_beams": 1,
494
+ "num_feature_levels": 3,
495
+ "num_return_sequences": 1,
496
+ "output_attentions": false,
497
+ "output_hidden_states": false,
498
+ "output_scores": false,
499
+ "pad_token_id": null,
500
+ "prefix": null,
501
+ "problem_type": null,
502
+ "remove_invalid_values": false,
503
+ "repetition_penalty": 1.0,
504
+ "return_dict": true,
505
+ "return_dict_in_generate": false,
506
+ "scale_factors": [
507
+ 4.0,
508
+ 2.0,
509
+ 1.0,
510
+ 0.5
511
+ ],
512
+ "sep_token_id": null,
513
+ "suppress_tokens": null,
514
+ "task_specific_params": null,
515
+ "temperature": 1.0,
516
+ "tie_encoder_decoder": false,
517
+ "tie_word_embeddings": true,
518
+ "tokenizer_class": null,
519
+ "top_k": 50,
520
+ "top_p": 1.0,
521
+ "typical_p": 1.0
522
+ }
523
+ },
524
+ "dtype": "float32",
525
+ "fill_hole_area": 16,
526
+ "high_conf_thresh": 0.8,
527
+ "high_iou_thresh": 0.8,
528
+ "hotstart_delay": 15,
529
+ "hotstart_dup_thresh": 8,
530
+ "hotstart_unmatch_thresh": 8,
531
+ "init_trk_keep_alive": 30,
532
+ "initializer_range": 0.02,
533
+ "low_res_mask_size": 288,
534
+ "max_num_objects": 10000,
535
+ "max_trk_keep_alive": 30,
536
+ "min_trk_keep_alive": -1,
537
+ "model_type": "sam3_video",
538
+ "new_det_thresh": 0.7,
539
+ "quantization": {
540
+ "group_size": 64,
541
+ "bits": 4,
542
+ "mode": "affine"
543
+ },
544
+ "quantization_config": {
545
+ "group_size": 64,
546
+ "bits": 4,
547
+ "mode": "affine"
548
+ },
549
+ "recondition_every_nth_frame": 16,
550
+ "recondition_on_trk_masks": false,
551
+ "score_threshold_detection": 0.5,
552
+ "suppress_overlapping_based_on_recent_occlusion_threshold": 0.7,
553
+ "suppress_unmatched_only_within_hotstart": true,
554
+ "tracker_config": {
555
+ "enable_occlusion_spatial_embedding": true,
556
+ "enable_temporal_pos_encoding_for_object_pointers": true,
557
+ "image_size": 1008,
558
+ "initializer_range": 0.02,
559
+ "mask_decoder_config": {
560
+ "_name_or_path": "",
561
+ "add_cross_attention": false,
562
+ "architectures": null,
563
+ "attention_downsample_rate": 2,
564
+ "bad_words_ids": null,
565
+ "begin_suppress_tokens": null,
566
+ "bos_token_id": null,
567
+ "chunk_size_feed_forward": 0,
568
+ "cross_attention_hidden_size": null,
569
+ "decoder_start_token_id": null,
570
+ "diversity_penalty": 0.0,
571
+ "do_sample": false,
572
+ "dtype": null,
573
+ "dynamic_multimask_stability_delta": 0.05,
574
+ "dynamic_multimask_stability_thresh": 0.98,
575
+ "dynamic_multimask_via_stability": true,
576
+ "early_stopping": false,
577
+ "encoder_no_repeat_ngram_size": 0,
578
+ "eos_token_id": null,
579
+ "exponential_decay_length_penalty": null,
580
+ "finetuning_task": null,
581
+ "forced_bos_token_id": null,
582
+ "forced_eos_token_id": null,
583
+ "hidden_act": "gelu",
584
+ "hidden_size": 256,
585
+ "id2label": {
586
+ "0": "LABEL_0",
587
+ "1": "LABEL_1"
588
+ },
589
+ "iou_head_depth": 3,
590
+ "iou_head_hidden_dim": 256,
591
+ "is_decoder": false,
592
+ "is_encoder_decoder": false,
593
+ "label2id": {
594
+ "LABEL_0": 0,
595
+ "LABEL_1": 1
596
+ },
597
+ "length_penalty": 1.0,
598
+ "max_length": 20,
599
+ "min_length": 0,
600
+ "mlp_dim": 2048,
601
+ "model_type": "",
602
+ "no_repeat_ngram_size": 0,
603
+ "num_attention_heads": 8,
604
+ "num_beam_groups": 1,
605
+ "num_beams": 1,
606
+ "num_hidden_layers": 2,
607
+ "num_multimask_outputs": 3,
608
+ "num_return_sequences": 1,
609
+ "output_attentions": false,
610
+ "output_hidden_states": false,
611
+ "output_scores": false,
612
+ "pad_token_id": null,
613
+ "prefix": null,
614
+ "problem_type": null,
615
+ "remove_invalid_values": false,
616
+ "repetition_penalty": 1.0,
617
+ "return_dict": true,
618
+ "return_dict_in_generate": false,
619
+ "sep_token_id": null,
620
+ "suppress_tokens": null,
621
+ "task_specific_params": null,
622
+ "temperature": 1.0,
623
+ "tie_encoder_decoder": false,
624
+ "tie_word_embeddings": true,
625
+ "tokenizer_class": null,
626
+ "top_k": 50,
627
+ "top_p": 1.0,
628
+ "typical_p": 1.0
629
+ },
630
+ "mask_downsampler_embed_dim": 256,
631
+ "mask_downsampler_hidden_act": "gelu",
632
+ "mask_downsampler_kernel_size": 3,
633
+ "mask_downsampler_padding": 1,
634
+ "mask_downsampler_stride": 2,
635
+ "mask_downsampler_total_stride": 16,
636
+ "max_cond_frame_num": 4,
637
+ "max_object_pointers_in_encoder": 16,
638
+ "memory_attention_downsample_rate": 1,
639
+ "memory_attention_dropout": 0.1,
640
+ "memory_attention_feed_forward_hidden_act": "relu",
641
+ "memory_attention_feed_forward_hidden_size": 2048,
642
+ "memory_attention_hidden_size": 256,
643
+ "memory_attention_num_attention_heads": 1,
644
+ "memory_attention_num_layers": 4,
645
+ "memory_attention_rope_dropout": 0.1,
646
+ "memory_attention_rope_feat_sizes": [
647
+ 72,
648
+ 72
649
+ ],
650
+ "memory_attention_rope_theta": 10000,
651
+ "memory_encoder_hidden_size": 256,
652
+ "memory_encoder_output_channels": 64,
653
+ "memory_fuser_embed_dim": 256,
654
+ "memory_fuser_hidden_act": "gelu",
655
+ "memory_fuser_intermediate_dim": 1024,
656
+ "memory_fuser_kernel_size": 7,
657
+ "memory_fuser_layer_scale_init_value": 1e-06,
658
+ "memory_fuser_num_layers": 2,
659
+ "memory_fuser_padding": 3,
660
+ "model_type": "sam3_tracker_video",
661
+ "multimask_max_pt_num": 1,
662
+ "multimask_min_pt_num": 0,
663
+ "multimask_output_for_tracking": true,
664
+ "multimask_output_in_sam": true,
665
+ "num_maskmem": 7,
666
+ "prompt_encoder_config": {
667
+ "_name_or_path": "",
668
+ "add_cross_attention": false,
669
+ "architectures": null,
670
+ "bad_words_ids": null,
671
+ "begin_suppress_tokens": null,
672
+ "bos_token_id": null,
673
+ "chunk_size_feed_forward": 0,
674
+ "cross_attention_hidden_size": null,
675
+ "decoder_start_token_id": null,
676
+ "diversity_penalty": 0.0,
677
+ "do_sample": false,
678
+ "dtype": null,
679
+ "early_stopping": false,
680
+ "encoder_no_repeat_ngram_size": 0,
681
+ "eos_token_id": null,
682
+ "exponential_decay_length_penalty": null,
683
+ "finetuning_task": null,
684
+ "forced_bos_token_id": null,
685
+ "forced_eos_token_id": null,
686
+ "hidden_act": "gelu",
687
+ "hidden_size": 256,
688
+ "id2label": {
689
+ "0": "LABEL_0",
690
+ "1": "LABEL_1"
691
+ },
692
+ "image_size": 1008,
693
+ "is_decoder": false,
694
+ "is_encoder_decoder": false,
695
+ "label2id": {
696
+ "LABEL_0": 0,
697
+ "LABEL_1": 1
698
+ },
699
+ "layer_norm_eps": 1e-06,
700
+ "length_penalty": 1.0,
701
+ "mask_input_channels": 16,
702
+ "max_length": 20,
703
+ "min_length": 0,
704
+ "model_type": "",
705
+ "no_repeat_ngram_size": 0,
706
+ "num_beam_groups": 1,
707
+ "num_beams": 1,
708
+ "num_point_embeddings": 4,
709
+ "num_return_sequences": 1,
710
+ "output_attentions": false,
711
+ "output_hidden_states": false,
712
+ "output_scores": false,
713
+ "pad_token_id": null,
714
+ "patch_size": 14,
715
+ "prefix": null,
716
+ "problem_type": null,
717
+ "remove_invalid_values": false,
718
+ "repetition_penalty": 1.0,
719
+ "return_dict": true,
720
+ "return_dict_in_generate": false,
721
+ "scale": 1,
722
+ "sep_token_id": null,
723
+ "suppress_tokens": null,
724
+ "task_specific_params": null,
725
+ "temperature": 1.0,
726
+ "tie_encoder_decoder": false,
727
+ "tie_word_embeddings": true,
728
+ "tokenizer_class": null,
729
+ "top_k": 50,
730
+ "top_p": 1.0,
731
+ "typical_p": 1.0
732
+ },
733
+ "sigmoid_bias_for_mem_enc": -10.0,
734
+ "sigmoid_scale_for_mem_enc": 20.0,
735
+ "vision_config": {
736
+ "_name_or_path": "",
737
+ "add_cross_attention": false,
738
+ "architectures": null,
739
+ "backbone_config": {
740
+ "_name_or_path": "",
741
+ "add_cross_attention": false,
742
+ "architectures": null,
743
+ "attention_dropout": 0.0,
744
+ "bad_words_ids": null,
745
+ "begin_suppress_tokens": null,
746
+ "bos_token_id": null,
747
+ "chunk_size_feed_forward": 0,
748
+ "cross_attention_hidden_size": null,
749
+ "decoder_start_token_id": null,
750
+ "diversity_penalty": 0.0,
751
+ "do_sample": false,
752
+ "dtype": null,
753
+ "early_stopping": false,
754
+ "encoder_no_repeat_ngram_size": 0,
755
+ "eos_token_id": null,
756
+ "exponential_decay_length_penalty": null,
757
+ "finetuning_task": null,
758
+ "forced_bos_token_id": null,
759
+ "forced_eos_token_id": null,
760
+ "global_attn_indexes": [
761
+ 7,
762
+ 15,
763
+ 23,
764
+ 31
765
+ ],
766
+ "hidden_act": "gelu",
767
+ "hidden_dropout": 0.0,
768
+ "hidden_size": 1024,
769
+ "id2label": {
770
+ "0": "LABEL_0",
771
+ "1": "LABEL_1"
772
+ },
773
+ "image_size": 1008,
774
+ "initializer_range": 0.02,
775
+ "intermediate_size": 4736,
776
+ "is_decoder": false,
777
+ "is_encoder_decoder": false,
778
+ "label2id": {
779
+ "LABEL_0": 0,
780
+ "LABEL_1": 1
781
+ },
782
+ "layer_norm_eps": 1e-06,
783
+ "layer_scale_init_value": null,
784
+ "length_penalty": 1.0,
785
+ "max_length": 20,
786
+ "min_length": 0,
787
+ "model_type": "sam3_vit_model",
788
+ "no_repeat_ngram_size": 0,
789
+ "num_attention_heads": 16,
790
+ "num_beam_groups": 1,
791
+ "num_beams": 1,
792
+ "num_channels": 3,
793
+ "num_hidden_layers": 32,
794
+ "num_return_sequences": 1,
795
+ "output_attentions": false,
796
+ "output_hidden_states": false,
797
+ "output_scores": false,
798
+ "pad_token_id": null,
799
+ "patch_size": 14,
800
+ "prefix": null,
801
+ "pretrain_image_size": 336,
802
+ "problem_type": null,
803
+ "qkv_bias": true,
804
+ "remove_invalid_values": false,
805
+ "repetition_penalty": 1.0,
806
+ "return_dict": true,
807
+ "return_dict_in_generate": false,
808
+ "rope_theta": 10000.0,
809
+ "sep_token_id": null,
810
+ "suppress_tokens": null,
811
+ "task_specific_params": null,
812
+ "temperature": 1.0,
813
+ "tie_encoder_decoder": false,
814
+ "tie_word_embeddings": true,
815
+ "tokenizer_class": null,
816
+ "top_k": 50,
817
+ "top_p": 1.0,
818
+ "typical_p": 1.0,
819
+ "window_size": 24
820
+ },
821
+ "backbone_feature_sizes": [
822
+ [
823
+ 288,
824
+ 288
825
+ ],
826
+ [
827
+ 144,
828
+ 144
829
+ ],
830
+ [
831
+ 72,
832
+ 72
833
+ ]
834
+ ],
835
+ "bad_words_ids": null,
836
+ "begin_suppress_tokens": null,
837
+ "bos_token_id": null,
838
+ "chunk_size_feed_forward": 0,
839
+ "cross_attention_hidden_size": null,
840
+ "decoder_start_token_id": null,
841
+ "diversity_penalty": 0.0,
842
+ "do_sample": false,
843
+ "dtype": null,
844
+ "early_stopping": false,
845
+ "encoder_no_repeat_ngram_size": 0,
846
+ "eos_token_id": null,
847
+ "exponential_decay_length_penalty": null,
848
+ "finetuning_task": null,
849
+ "forced_bos_token_id": null,
850
+ "forced_eos_token_id": null,
851
+ "fpn_hidden_size": 256,
852
+ "fpn_kernel_size": 2,
853
+ "fpn_stride": 2,
854
+ "hidden_act": "gelu",
855
+ "id2label": {
856
+ "0": "LABEL_0",
857
+ "1": "LABEL_1"
858
+ },
859
+ "initializer_range": 0.02,
860
+ "is_decoder": false,
861
+ "is_encoder_decoder": false,
862
+ "label2id": {
863
+ "LABEL_0": 0,
864
+ "LABEL_1": 1
865
+ },
866
+ "layer_norm_eps": 1e-06,
867
+ "length_penalty": 1.0,
868
+ "max_length": 20,
869
+ "min_length": 0,
870
+ "model_type": "sam3_vision_model",
871
+ "no_repeat_ngram_size": 0,
872
+ "num_beam_groups": 1,
873
+ "num_beams": 1,
874
+ "num_feature_levels": 3,
875
+ "num_return_sequences": 1,
876
+ "output_attentions": false,
877
+ "output_hidden_states": false,
878
+ "output_scores": false,
879
+ "pad_token_id": null,
880
+ "prefix": null,
881
+ "problem_type": null,
882
+ "remove_invalid_values": false,
883
+ "repetition_penalty": 1.0,
884
+ "return_dict": true,
885
+ "return_dict_in_generate": false,
886
+ "scale_factors": [
887
+ 4.0,
888
+ 2.0,
889
+ 1.0,
890
+ 0.5
891
+ ],
892
+ "sep_token_id": null,
893
+ "suppress_tokens": null,
894
+ "task_specific_params": null,
895
+ "temperature": 1.0,
896
+ "tie_encoder_decoder": false,
897
+ "tie_word_embeddings": true,
898
+ "tokenizer_class": null,
899
+ "top_k": 50,
900
+ "top_p": 1.0,
901
+ "typical_p": 1.0
902
+ }
903
+ },
904
+ "transformers_version": "5.0.0.dev0",
905
+ "trk_assoc_iou_thresh": 0.5,
906
+ "vision_config": {}
907
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0cda03d91e65181c6754a39d7204d45b5dc48216283a9883dd817366a83bbf0d
3
+ size 620400791
model.safetensors.index.json ADDED
The diff for this file is too large to render. See raw diff
 
processor_config.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "processor_class": "Sam3Processor",
3
+ "image_processor": {
4
+ "image_processor_type": "Sam3ImageProcessor",
5
+ "size": {
6
+ "height": 1008,
7
+ "width": 1008
8
+ },
9
+ "image_mean": [
10
+ 0.5,
11
+ 0.5,
12
+ 0.5
13
+ ],
14
+ "image_std": [
15
+ 0.5,
16
+ 0.5,
17
+ 0.5
18
+ ],
19
+ "do_resize": true,
20
+ "do_normalize": true,
21
+ "do_rescale": true,
22
+ "rescale_factor": 0.00392156862745098
23
+ },
24
+ "target_size": 1008
25
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<|startoftext|>",
4
+ "lstrip": false,
5
+ "normalized": true,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "<|endoftext|>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<|endoftext|>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<|endoftext|>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": "<|startoftext|>",
5
+ "do_lower_case": true,
6
+ "eos_token": "<|endoftext|>",
7
+ "errors": "replace",
8
+ "is_local": false,
9
+ "model_max_length": 77,
10
+ "pad_token": "<|endoftext|>",
11
+ "tokenizer_class": "CLIPTokenizer",
12
+ "unk_token": "<|endoftext|>"
13
+ }
vocab.json ADDED
The diff for this file is too large to render. See raw diff