Upload LlamaForCausalLM

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,36 @@
+{
+  "_name_or_path": "nbeerbower/Llama-3.1-Saoirse-70B",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": 128039,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.48.0",
+  "use_cache": true,
+  "vocab_size": 128256
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "eos_token_id": 128039,
+  "transformers_version": "4.48.0"
+}

model-00001-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9ba902863e9afb2f0b35a7b9984b4bceff217d6551f8a7c772005eaf9917bbd
+size 2101346448

model-00002-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b7bf851254bf26c0b501be460f6d639bec29d9b49c48ce6a219e4655c2defdc
+size 1879082376

model-00003-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a6d0f1f14e96086d63a2b278d3a4a5d47f03d8d7575d14e524935f6eaf76001
+size 1845527704

model-00004-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c27fa407688595d41a5b87737040f03f0e0ef93ffb0c338dc8e0553ce74869b
+size 1711309864

model-00005-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af35bf8c649816b87d89f6e456118e2d5cf75c279b7a8df9b4c511fb3324a4d9
+size 1711309864

model-00006-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:357543ff7c4dbb802bd05deb4831c39d16fb54861eb70760658962f10fdbe7ea
+size 1711309864

model-00007-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58d9c80c56612b49ad95fb1c10eb355fe071f69e1d9ad39a0b1c3757b258f48c
+size 1711309864

model-00008-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3021a1aab7adc45c49e8a4e735ab73a9b1bf4c23df72d701070b7a8d803dacee
+size 1711309864

model-00009-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05f7460ac6bc9ce57d5c3fe2433ae8ee8b42423d824f13b8e4b23221940ca402
+size 1711309864

model-00010-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ceb925c2d80561de34dea5f5075fd704b4e321a625ed7abecb472972968bbc1d
+size 1711309864

model-00011-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48cb47d13ee801cd9e9907359ffad30a5f9220ff7261e9a8e3e7005e47109712
+size 1711309864

model-00012-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46b6e0407c4d1fec98e7809f65791b33072b6e2efc447164e9b189a96846cbdd
+size 1711309872

model-00013-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:100d8dcf7d0bb11ef5f2ba351c3aeec2df918a6e1224d4c8b11472cf3e00c51b
+size 1711309872

model-00014-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fcfd56efb2c1277a8874c1f901bcfedc76487b3d92c8dffbf8b4eabfd7f12831
+size 1711309872

model-00015-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fbc2986a06482c9cea91fed68f4baa6c4133668621685cd5c12cbbd342e8914
+size 1711309872

model-00016-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7aea13849015156173a0d46ba9998071946a03b365254a68d02751338c82608e
+size 1711309872

model-00017-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0008bd30ee0d2faedcccd7be8896296878b029902fdfc5ad1680585b9d55f2d1
+size 1711309872

model-00018-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e35c6642be2b6f9e42cf1d009eace820ffc3a5f68c68885416df28d6cbca989e
+size 1711309872

model-00019-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd3de92d24c01d21235971c87d9a6b993c6092eea624c8cfb9b8331dfcfb55bd
+size 1711309872

model-00020-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5a745b215f2df6e8172992a1841de7f7f678cd8836a0d7ffe51a95a1e223eed
+size 1711309872

model-00021-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:591bf751cca4e28778f6c0ded5c7afed6dcab5bacb1bd5b852f57b80f14b50e2
+size 1711309872

model-00022-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:904751cc00cf100d0af5f2404e1318c7a89091e1adbc5bd0f68cf5b9ff317d17
+size 1711309872

model-00023-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d45f1d81fcb73fff3dfd17583e2d6882d61ca0a68b86775de04a1d6b55204d0
+size 1711309872

model-00024-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf425036dd1ccb5cdeb5ceafdcd4c94a385a3f690d7d310151d80ec0c4bc8223
+size 1711309872

model-00025-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:788bb20d06f45bb8006e678da9052373ebdd624076eefeebcc128e3e2cfd128e
+size 1711309872

model-00026-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a796102954bdc83ce1e428fb0757fa26e8d3105809943548073a2c8a9e62df05
+size 1711309872

model-00027-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:753a4fd77913a56253a805bdb045ef641da76e54a91c1528aac1dd7947c7e6f2
+size 1711309872

model-00028-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1c92b9cae619cfc288c66d310719efb53ebbfa7d5b517bf8095a4d0374c3561
+size 1711309872

model-00029-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad25199ee237d19722c6927ac074368fab61bed4006dfec19886f50ed0448cd2
+size 1711309872

model-00030-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31d01a819c35e94b5d78d58ecdfe3b3a7ee3e326b5cb3ce22ead2aa922c01328
+size 1711309872

model-00031-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:241565f06b8a8532993b9c5389256a334181b7226a26b2ac19ae4b8164edd2d5
+size 1711309872

model-00032-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00ec86e796b235f519f3112b8d210dac91dc9cc6475c2f28a8f23b32ab32bef9
+size 1711309872

model-00033-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a8941bda53ea459ba77a4851699dc3ad39a61cfb5bfa09a89d66bd176063094
+size 1711309872

model-00034-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bfbc006e0b6a192d832dca7f92808980cc8da4df2e4057fb6458f408a3ba0f8
+size 1711309872

model-00035-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98bbfcc5571ed4e6808e459c26129dba61d3c3d79b4d4fbd88866560d406d43c
+size 1711309872

model-00036-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04c480c70e55d0daf85f1e1ccb5ed1074e036c255cbaf853fcf032ae0bc4498d
+size 1711309872

model-00037-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6fb383f53f377d6cadbf5d4cdd2553a1db40852d86274281856cda0a100f943
+size 1711309872

model-00038-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c9f1b043d407bcf88365f36380f583a928a00d18e1d117ed883ae30a9c1f70d
+size 1711309872

model-00039-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a586a646e83ed9e5c633af18336e6c92bdeb25cf0ed67fe192158436ea4d2ee0
+size 1711309872

model-00040-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9af6e1aabbec68383af9aa1dc7092ec0bb15b98ef8c4aa10232b97397a7c9d7b
+size 1711309872

model-00041-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fb070bc3ab058278e3f468d57f047de95017262e8787c2dcbb750bae18df818
+size 1711309872

model-00042-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0359a020969f5ed6f97d53096cea108869b851465f9d0901c2362c2ed66ae854
+size 1711309872

model-00043-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf71546d3b907fefef34eafe685ae246e434919655a29dd0594e79c05f2586da
+size 1711309872

model-00044-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb6ff5bfb97ddfa9a83c69d97f936690825264946d7806da1ebb1de3630e60fd
+size 1711309872

model-00045-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87b2aeb3252a3bc827d5a24bc5fc2c0a611a2adfe2effcde0ed08b8baf43782d
+size 1711309872

model-00046-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f156e9a11728283185f20e7ee63d1f783829d7c052a2e2a0c99c1831a52c818e
+size 1711309872

model-00047-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73aa76b8b7ebfedd278087414e9531c0848b405bee09083866339f430f4dd201
+size 1711309872