Upload LlavaForConditionalGeneration

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,42 @@
+{
+  "_name_or_path": "pixtral_converted",
+  "architectures": [
+    "LlavaForConditionalGeneration"
+  ],
+  "ignore_index": -100,
+  "image_seq_length": 1,
+  "image_token_index": 10,
+  "model_type": "llava",
+  "multimodal_projector_bias": false,
+  "projector_hidden_act": "gelu",
+  "text_config": {
+    "hidden_size": 12288,
+    "intermediate_size": 28672,
+    "is_composition": true,
+    "max_position_embeddings": 131072,
+    "model_type": "mistral",
+    "norm_eps": 1e-05,
+    "num_attention_heads": 96,
+    "num_hidden_layers": 88,
+    "num_key_value_heads": 8,
+    "rope_theta": 1000000000.0,
+    "sliding_window": null,
+    "vocab_size": 32768
+  },
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.47.0.dev0",
+  "vision_config": {
+    "head_dim": 88,
+    "hidden_act": "silu",
+    "hidden_size": 1408,
+    "image_size": 1024,
+    "image_token_id": 10,
+    "intermediate_size": 6144,
+    "model_type": "pixtral",
+    "num_hidden_layers": 40,
+    "patch_size": 16,
+    "rope_theta": 10000.0
+  },
+  "vision_feature_layer": -1,
+  "vision_feature_select_strategy": "full"
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.47.0.dev0"
+}

model-00001-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d2d7a1e3e556790ffb2c6340a48c0d67c5e4355b0d3053cfbd18013f8ce2e53
+size 4509220632

model-00002-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fcbbe01e14828c044cf0ae930c826acff6ee3b8e81fd6ca5ef6c97a42fd7f3a
+size 4882270024

model-00003-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e84a1a7c117ecee5b9be23ace65ff17aa7dcd55350794a02820a80d9e2c3a07
+size 4831889368

model-00004-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cd040875f38bc94db33c01c68af083f6288a0e8f883a6a3f2579ce01a95cd90
+size 4831938776

model-00005-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9338c97508c94cfd6fe5d92e9285622d46d14d7ac97fa8dec6199d3569e764d5
+size 4831938776

model-00006-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:540fb77c3fc0150cbe012a285afc57918957f2ed92542910e876ae14235143b8
+size 4882270024

model-00007-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c53b5751a6f73e070752bc8d7c9409442c0e1e8e3d6902c377026f71b9cfcd9b
+size 4831889384

model-00008-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:194ce36f2dbeb4b391857b32579d579bb668637a26f19b1ac48252f55accb753
+size 4831938792

model-00009-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abb1b4063d634fdb4ee19894e89e5f9c9e21019a2b80adfb6ba14c13fa3c5406
+size 4831938800

model-00010-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a4fc55163298b3882aa6aa1f8f229bd9857e1494cd1ea543e4f257376bd1c49
+size 4882270040

model-00011-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa3f51e16047a320259ae21c6d9506266ce9bec2be0c49508f7d99386d68d5bd
+size 4831889376

model-00012-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19b5737821632a06fc0e61e24718094be4bf361cb3fc57fbaa82a6f7019f08cb
+size 4831938792

model-00013-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cea09a3e29d77fddd25d7e55992d8f17a49bcf8ed9c38ca4cba517e0045d45f5
+size 4831938800

model-00014-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:075167a29ae960189491f0f250e9c0bc3d9491bc3d3c46ee6d0089c46e43f4ec
+size 4882270040

model-00015-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42588f8fe11c0846cfd9e96603828256cc4e8f101e54a7f24bed2f5c28cb9fe5
+size 4831889376

model-00016-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7281b0ee3f2590bc6c19130ed8f233887f136f8fdfab9c64b34402e9919d7db5
+size 4831938792

model-00017-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06e442f95527ea9a5f82f193e9781bbbd376e305019b28d52f84426f10928d8c
+size 4831938800

model-00018-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6dbfb7bbd99bc0d0d44b1c34112b71a81fdf1c5354835176a4fe187d00bfce1b
+size 4882270040

model-00019-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:791b54d8e7ff38c2e575df46115c7cd21a3a9defe544b9f943cdd0cdc78f3fb3
+size 4831889376

model-00020-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35df3afdd407778ae26e1f9df6f6ea12681acd90de4d9525f4b4cde2ef30662d
+size 4831938792

model-00021-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd7991d7936b08bf468f2f2da9dbac46224b732bedd9ab09c2ba4ff06a82557c
+size 4831938800

model-00022-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4a4473980e3bc05a3366fa638bb861519a4665f33bc1512ac6cd0cbdc7711a5
+size 4882270040

model-00023-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65ca1eb9328ada88ad65e08c8b18d526d298bab6965a7372be537a994c5205eb
+size 4831889376

model-00024-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba34028c5979cfd3781ca70aead3b1b2b55180d93394ce2857db4e5b0e9af8b6
+size 4831938792

model-00025-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b1dee0420ed6c93f7bcaed4197655344303b338f3d9d1027fd7f036738bf932
+size 4831938800

model-00026-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cb83cb476282252ffddddabc69ea27fc08f3f7a022acefe4335419f28050e6c
+size 4882270040

model-00027-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05f852d81895c3ba645c106f21ab9fb63d2988baed0967b17266955e65c9e95a
+size 4831889376

model-00028-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1859871627e14661d1ee731005e87b312d4af7aa25a436f468a7b7e4bf329dd4
+size 4831938792

model-00029-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b7d2b168b2caa8a0445fedf597478d62dbd6850a777b0c08e7eebed3b0d9a6a
+size 4831938800

model-00030-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4c20b8e733f6dbd8d5a300eea95b0afa3a1bc2a780768a6e274ea08cfeb1020
+size 4882270040

model-00031-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:400f4c047a67586f764af4ae0abba7440df8107ceba3ac0fd4166f5bcaab34e1
+size 4831889376

model-00032-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ac94d3d776f9882c868cb20db22e8bcf79e5f0abb134d06e64ddcf6e652b81c
+size 4831938792

model-00033-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ed94b8ba78c5d51635a47d724d64a3959e955c71187ecc23747f686dbc33d3a
+size 4831938800

model-00034-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc84284a1541d12eb8d6a113a68f978a255151dffb021c86e8d4f3ec08c1f480
+size 4882270040

model-00035-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24493e298808e6481a72a8dbc8a4ee173d7a2bd7657f78987bbaafe85f3d5221
+size 4831889376

model-00036-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8fcdde3f303bae55b414633dbaf047f6f00806976a5b726e2d73fdd6fc14564
+size 4831938792

model-00037-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb44645fc87b76f6b69ef97217d7760955e1fd0402efa68b24fd9ffd1ee3c9a5
+size 4831938800

model-00038-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a62587682a17ba3d030c6479661d01932afe76c2eaade70383e0ed4bece8213
+size 4882270040

model-00039-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ea5f4b55c5dde56308cd14ab941c8912b2c71c971feda89cd199993bfef36d3
+size 4831889376

model-00040-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf6a29d3af8c20a2a70676407d5f8581df4e849af24eaf2760a9c38fafe8779f
+size 4831938792

model-00041-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bff31ed650cf76fbad236fb6685f2a4c4f86db6da037cea3a73ebc5e28a1c93a
+size 4831938800

model-00042-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f090172fa822725090d97b55d31873fecd1ecc01606fbc4fb1357704f6760ac
+size 4882270040

model-00043-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2390076a1035012086880648ee9cbc9a6f3407ac411bf79ebef557641964fc69
+size 4831889376

model-00044-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:785ba85d22ecc3d3b6b62dbcb1e843e57fa57bf0bba7877e15f5d3fda9f6100f
+size 4831938792

model-00045-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f6470663d2e7c12a1af0f1c54fb2fb5176de2d47de0e6ee06bb652738ee3421
+size 4831938800

model-00046-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f28328f36a9ac4a470716488589812b656badad04877e4bfec2288660771fd9
+size 4882270040

model-00047-of-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4450bdb8aac77ef388cb192dbd78a949aa18d7e2800ce112d00894a31466b7ec
+size 4831889376