Upload LlamaForCausalLM

README.md

@@ -0,0 +1,201 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "meta-llama/Llama-2-70b-hf",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.37.2",
+  "use_cache": true,
+  "vocab_size": 32000
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": 2,
+  "max_length": 4096,
+  "pad_token_id": 0,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.37.2"
+}

model-00001-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d24658ed13c9ff7707a8a8b6640d985729e4e3e9a8e4ebbd296e392a913c9759
+size 4806739440

model-00002-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e0692ef1d8cc33688d0201918ad6c01680c4fd2cc881dc1baa156ff2c1b65df
+size 4630578440

model-00003-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f8c4b4386918b267a88b2f519df037f53c7c8aef2cdb9ae2256c0efd2f6e174
+size 4362142864

model-00004-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22b2fdfa6318a04ad9b7cfb5a91c2c917af223bca26385f716b8dde898c369d1
+size 4966188864

model-00005-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0cd7c614dd38fd3e8a9693ab33756ce702fac75e370beba56720cab473dcff96
+size 4362142864

model-00006-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c43ecb4f299f80138e074eac7cdfa2ee5b2f713a5e4c36b37d98a140c92e84b
+size 4362142864

model-00007-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9ea01c4a23a9f41ba72886eb3b69e47e25f7b0a190a8df03882c2d2803748f8
+size 4966188864

model-00008-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b35ae0b345a244c962f0bf47ff3117c3ea2d03a6fa42fc776e7d71116ef7d70a
+size 4362142880

model-00009-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61d3cc6cf7a80abc9015aba9869871acddead21d566604f799e135e17fab6276
+size 4362142872

model-00010-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5261d70a37247052a9d3900b49e275d96fe0ad9374fc90caa8977d9bebed9680
+size 4966188880

model-00011-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b276fc79d14915ba95ba4cc6a77b3cf8b010de5e5f78a4397238c34d3144212d
+size 4362142872

model-00012-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91043437eec7704507ca6bf82ec8d564fe24474852b06a811a63f1c01c627a0f
+size 4362142872

model-00013-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c90ae72ddb6225693f2f00ad71096643ec7ec13aee7ef7a904aa34a44bb4f733
+size 4966188880

model-00014-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7dce4008765103091b1aa60d15351d9da1e61f7f662d3651188ed32c9a17aaea
+size 4362142872

model-00015-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c40d64205c0f403e4cfeabdf7790e0a49419e32ef68193cd931a78dfca82e31
+size 4362142872

model-00016-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a96b7eb0024c767129dbc9fa7bf1d89473510933d2f90243ce630a733895a582
+size 4966188880

model-00017-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5f41fdf43aa042ae871acbeaff7d4b0ed6a45a061b00430bd4ab8ed622b5d37
+size 4362142872

model-00018-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a27c715bd092dc08f19b9ab63000d8b31a860ca80ef064f21cf0969a988bf1f
+size 4362142872

model-00019-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd935827e24f928bc171bbc6cd3f0cc2eb7685dde88b42d40c5c5e24af5d988
+size 4966188880

model-00020-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48362a9a0a50369298b43cfc7f727b797c034748d02622ada3a4d9fc2df15e59
+size 4362142872

model-00021-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ac62f35f0affe918278a70ba8acceb4da7e1fa7f9c44b52da43ba3b7e779b88
+size 4362142872

model-00022-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4e621fbba0fa7c5489b5b0414997fb8770ec60639c670987045661b45588e27
+size 4966188880

model-00023-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df211c400fa8150514396545ff65c81bb19ea4ef2826b488e2c780ba081de67b
+size 4362142872

model-00024-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5df0778e1925f24e58bd75c1b0dd9b8e1318352ec8efe3efbff5423bbe6e2090
+size 4362142872

model-00025-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64570c5c8fb659a6f8891ceafa9b17a132b66be1ce28637e654b047e72d8503c
+size 4966188880

model-00026-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d555d1c7009232dcb516a2c40a292bbfb201cfafaf9bedf20e5735fc386d3458
+size 4362142872

model-00027-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f44fd24d29aea57a93a3056cd3f00d4374460d23bfff2b50817c5f6c6719c85e
+size 4362142872

model-00028-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77d7f23960e9f2f04d798802fed83e034295458473e81865878774277fe19d41
+size 4966188880

model-00029-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56866aca5707facf39f3557c57f0eb3f40392ef3b3d84ab83eea1d7efa966b31
+size 4362142872

model-00030-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:144113d02f0a98d6636f864cd4f3b6d426b7e9db65c85a600ac284f3dc41b4f0
+size 4362142872

model-00031-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:755a0a3833b384834653732e00c3350dc84dedef3c9a676230f7c75c65a2fa31
+size 4966188880

model-00032-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b79526a83db463a3921e495099c7bc10654e7f6cd5c66ba50fcc02f18df7c855
+size 4362142872

model-00033-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d00b05182920afd34167d1fe2b3db540f0cf52729d353597b927d7c27e00cdc3
+size 4362142872

model-00034-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa00155d49241f4df83c4710894c2e4a21b50b5b006705b1176a9a5e6e1bccc2
+size 4966188880

model-00035-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec7a100aa0a9deb83bf36d6ea5010182fc716a9e1be1a4aa0bb4e75f64e800fd
+size 4362142872

model-00036-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9deb9aeb48ad82355d59bb907dd3b7d87d8e60dcec399cf82289e976a38949a4
+size 4362142872

model-00037-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd0f54e3831b3fd36ceb3d7d6d9256e5fc2bb187b47a52637311843a35c504cd
+size 4966188880

model-00038-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8667d99ed2be26507aa5869c721f19a7e85fb6afab85855b4fa06d535fb9429
+size 4362142872

model-00039-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a935edb896dcea6d0a8f5472531ead49b7e144976f2d2216ced0f07fbfc362e6
+size 4362142872

model-00040-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:991af2c2121aa7206d87c9e1f222df228a8d88d72599f559cccab400f2aee0c2
+size 4966188880

model-00041-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d77a4a6dc5f51a7fc3a59a9704e77c809578f63ad386524e94bc76bf0f0974b3
+size 4362142872

model-00042-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb9ae2e11d7f0f1676dbaf34e3f9c56848314e25ecc5849f6feb55f27e66e855
+size 4362142872

model-00043-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5646a0b1c8215f5fd5c6a59526a05aed00f686c0bd005eba139051d791abf0d3
+size 4966188880

model-00044-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9da471ee76441edcd9f1efc071439e4540a09f0f04fb69bc7184ec2c6b0ec79
+size 4362142872

model-00045-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ed41d5f1ae3737efdd2fe9a0a09d824b94303df54bce628908bcf69e4a53a8c
+size 4362142872

model-00046-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:683a9c798d76f22640bc715532d891aded93b91edec6597ca99e1a662a186bae
+size 4966188880

model-00047-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0704410f32e5cc7217a4a2e70716c82d9b436ca6089bf4515b348e6a030e297c
+size 4362142872