Upload LlamaForCausalLM

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "./",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.39.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32001
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": 2,
+  "max_length": 4096,
+  "pad_token_id": 0,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.39.0.dev0"
+}

model-00001-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:207412cea1d5f9b40f0fea1f1741af8ee9996e21bfd3e43f5628d5d92c094d2f
+size 524304520

model-00002-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbe7a6f649e0f051cbcfba468300f085a899528a3ef2b744cc344c6809089057
+size 448791128

model-00003-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba7302db0a772289a5186761a526aeb06ff5436b785ff477f2f9ea28d7f31889
+size 494961344

model-00004-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6a5ad80d120a67687c9639be436899a62e09c1f2c2a5ed8babd3806460e713d
+size 494928360

model-00005-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d71d3f90b5c7c2cce079b76b609dda7c45a6d069597020db0ae53cc9916255e0
+size 448824120

model-00006-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0169e811df36eb1df852db472cf76cd22ba0e2d6c9fd61835e9cd43554ee691
+size 360710400

model-00007-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c337b26fbbb8d8efb9bea9d2f0ac387306e2584159563fe7ce6b95c3beb2978
+size 448824120

model-00008-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c65bc5a7d16fcabad43d759e1e916b22a920f45bcacb4829166a759fe0ec7b19
+size 360710400

model-00009-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:383873f8f452a15988860f5cee7abb8c157e9f4be37f5184910989be7f3306d2
+size 448824120

model-00010-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95826f8492676acda2b41518f9ae26902e7342f7a1202387e0550ec3f39de458
+size 360710400

model-00011-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cc0b10c0564bd20d957dfabdca0f5b802bbda31405d9e70eb0d20ebb8836e33
+size 448824120

model-00012-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1ec229c6263025bdf3e86b74a08761309b195a497f4ad61ecdd6c59d3fd39f8
+size 360710400

model-00013-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c0f06a64230d3d5745522768677e2a60ff456a7b9c195d8dff039f53dacfb20
+size 448824120

model-00014-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e04af8ef7b61faab962eaad094a0fdfe8cd5a9ea0a330192ee0c828c9f73503d
+size 360710400

model-00015-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c5b5c07da2ab4de644358e898cb0674e8c5242464a61111a725da6dcd77df1f
+size 448824120

model-00016-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44d99596f6725a875155e07ef1c3fd65f8e8f691fcdb352e74dfe4e158852e72
+size 360710400

model-00017-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0b5683973ee3077701aeca67c43f6f5960ca26548cae7597fe9159a43196625
+size 448824120

model-00018-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67279a7a72d7e9aa0485a5c3d37db94e260efc52005bf03aacfe43c24b82c239
+size 360710400

model-00019-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2776116d1b071cd3b904cb4a16343939a8a4804b313cf04a1da7cdd3a3baeb4
+size 448824120

model-00020-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee1117ec2c7332566e59e0c571ee886368a137d21ae115edf2eb3cadd5f98b22
+size 360710400

model-00021-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b811753db92d83bd18e3a4e5f38d0cc3ee0acf41cafd36fe78c63c5acfe19f04
+size 448824128

model-00022-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b66fda815d7be1525dbbce31f4efefbec9a952d3064f694b391b8d6bb9dab66f
+size 360710400

model-00023-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68a5e73b6f2c1eeefc233f0895a971e3a2245f16ea131c885c5a87936591f6cb
+size 448824120

model-00024-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28b95110adb615c01096256fe1b618e10e425062b22bc58d10191b72f63acdb9
+size 360710400

model-00025-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4fbe79dd81ce87b83bb3ccb3516d92727f6d56ae25ce1e726aa2a1eaa9571de
+size 448824120

model-00026-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99ca570ef53051367db2e2e262de3ec00c030b68764efde21942ebe452234541
+size 360710400

model-00027-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57074770173a20bfe02f280732fc1610207ec5dce1c281e82dbbd1dc7d7831ec
+size 448824120

model-00028-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6834df98d3929e0f46b671040b08a636de200a9982ac89a8beafa65443804db2
+size 360710400

model-00029-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5738abfc5041cd0888fce7ec655fe3ac543e287ca38ea620be56bc1df9d25f09
+size 448824120

model-00030-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50ddb2c91da744e4a22a7cc8b52bdee17b349c58960ddf171616e2885af7ec27
+size 360710400

model-00031-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c9e896a0868cf9ea546e70952eeb82feb6e461e5dcfeb6231c891e078e07403
+size 448824120

model-00032-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50fde113f2e07876df3494c2715adbb789e6ffcdd9e80facf65b37a0c3f99970
+size 360710400

model-00033-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9b38051c9c3d989bd627d9a7c7695e4b8a63ed214285797f4d28f8829763612
+size 448824120

model-00034-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6164fcd86603ba8d4362ccd34045140b4b63d5fa881317ca24ba1ab3a21d766b
+size 360710400

model-00035-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b2d4a953940143b0af294ae06d7d764371dccb9b2387e11a1b015839f0f4d19
+size 448824120

model-00036-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:712212e0797838f46e7072474e133083e4520d6407894e1e8c6e1b852a76c42d
+size 360710400

model-00037-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5dddfeec80a61f7e10effb797dd758b312eba06adfc9fae074e17ded3dad8d6
+size 448824120

model-00038-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0860d9664501a87f8ab5f8e37ef5dd588877f260f95979522814638d1ea0fe89
+size 360710400

model-00039-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9e7244f0954d7918491835bbf3a87579db0d573c621531292dfaeede1a8f1aa
+size 448824120

model-00040-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f473e1c6a1968800ce41ba2d951a4cb0bc6ddc4a8206ecb8330c17dcc07f999
+size 360710400

model-00041-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f0332d2aacbce26aa622640886b3d2702881493f73ac40a834900176609347d
+size 448824120

model-00042-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc6900a41c50f6fa2f02c01dd3ff05ddb9759d2d349e2f6c7643036a9ed87697
+size 360710400

model-00043-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afe47aee2dadf8a4d3eba08fd43ee8310e83121587c0879243dce570cd418916
+size 448824120

model-00044-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e54881b181d53858e3b56c6be5ff2b6005fdc0637d5df97aaf6549650a41da17
+size 360710400

model-00045-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a093866c5ab8437de6c68d0172991fe5765ad0cb56bd0d4ee9287a3cfbef745
+size 448824120

model-00046-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7486b1f294e97af58d66e27d467bc9d154a26bbd3a5100b30a676af6d47b959
+size 360710400

model-00047-of-00066.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f8b1e740a4fca362ac59b850ad8184b004423e2681fa9f5264e88538c6cd86a
+size 448824120