Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,341 @@
+---
+license: gemma
+library_name: transformers
+extra_gated_heading: Access RecurrentGemma on Hugging Face
+extra_gated_prompt: To access RecurrentGemma on Hugging Face, you’re required to review
+  and agree to Google’s usage license. To do this, please ensure you’re logged-in
+  to Hugging Face and click below. Requests are processed immediately.
+extra_gated_button_content: Acknowledge license
+---
+
+# RecurrentGemma Model Card
+
+**Model Page**: [RecurrentGemma]( https://ai.google.dev/gemma/docs/recurrentgemma/model_card)
+
+This model card corresponds to the 9B base version of the RecurrentGemma model. You can also visit the model card of the [9B instruct model](https://huggingface.co/google/recurrentgemma-9b-it). 
+
+**Resources and technical documentation:**
+
+*   [Responsible Generative AI Toolkit](https://ai.google.dev/responsible)
+*   [RecurrentGemma on Kaggle](https://www.kaggle.com/models/google/recurrentgemma)
+
+**Terms of Use:** [Terms](https://www.kaggle.com/models/google/gemma/license/consent)
+
+**Authors:** Google
+
+## Usage
+
+Below we share some code snippets on how to get quickly started with running the model. 
+
+First, make sure to `pip install transformers`, then copy the snippet from the section that is relevant for your usecase.
+
+ ### Running the model on a single / multi GPU
+
+ ```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("google/recurrentgemma-9b")
+model = AutoModelForCausalLM.from_pretrained("google/recurrentgemma-9b", device_map="auto")
+
+input_text = "Write me a poem about Machine Learning."
+input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
+
+ outputs = model.generate(**input_ids)
+print(tokenizer.decode(outputs[0]))
+```
+
+## Model information
+
+### Model summary
+
+#### Description
+
+RecurrentGemma is a family of open language models built on a [novel recurrent
+architecture](https://arxiv.org/abs/2402.19427) developed at Google. Both
+pre-trained and instruction-tuned versions are available in English.
+
+Like Gemma, RecurrentGemma models are well-suited for a variety of text
+generation tasks, including question answering, summarization, and reasoning.
+Because of its novel architecture, RecurrentGemma requires less memory than
+Gemma and achieves faster inference when generating long sequences.
+
+#### Inputs and outputs
+
+*   **Input:** Text string (e.g., a question, a prompt, or a document to be
+    summarized).
+*   **Output:** Generated English-language text in response to the input (e.g.,
+    an answer to the question, a summary of the document).
+
+#### Citation
+
+```none
+@article{recurrentgemma_2024,
+    title={RecurrentGemma},
+    url={},
+    DOI={},
+    publisher={Kaggle},
+    author={Griffin Team, Alexsandar Botev and Soham De and Samuel L Smith and Anushan Fernando and George-Christian Muraru and Ruba Haroun and Leonard Berrada et al.},
+    year={2024}
+}
+```
+
+### Model data
+
+#### Training dataset and data processing
+
+RecurrentGemma uses the same training data and data processing as used by the
+Gemma model family. A full description can be found on the [Gemma model
+card](https://ai.google.dev/gemma/docs/model_card#model_data).
+
+## Implementation information
+
+### Hardware and frameworks used during training
+
+Like
+[Gemma](https://ai.google.dev/gemma/docs/model_card#implementation_information),
+RecurrentGemma was trained on
+[TPUv5e](https://cloud.google.com/tpu/docs/intro-to-tpu?_gl=1*18wi411*_ga*MzE3NDU5OTY1LjE2MzQwNDA4NDY.*_ga_WH2QY8WWF5*MTcxMTA0MjUxMy4xNy4wLjE3MTEwNDI1MTkuMC4wLjA.&_ga=2.239449409.-317459965.1634040846),
+using [JAX](https://github.com/google/jax) and [ML
+Pathways](https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/).
+
+## Evaluation information
+
+### Benchmark results
+
+#### Evaluation approach
+
+These models were evaluated against a large collection of different datasets and
+metrics to cover different aspects of text generation:
+
+#### Evaluation results
+
+Benchmark           | Metric        | RecurrentGemma 9B
+------------------- | ------------- | -----------------
+[MMLU]              | 5-shot, top-1 | 60.5
+[HellaSwag]         | 0-shot        | 80.4
+[PIQA]              | 0-shot        | 81.3
+[SocialIQA]         | 0-shot        | 52.3
+[BoolQ]             | 0-shot        | 80.3
+[WinoGrande]        | partial score | 73.6
+[CommonsenseQA]     | 7-shot        | 73.2
+[OpenBookQA]        |               | 51.8
+[ARC-e][ARC-c]      |               | 78.8
+[ARC-c]             |               | 52.0
+[TriviaQA]          | 5-shot        | 70.5
+[Natural Questions] | 5-shot        | 21.7
+[HumanEval]         | pass@1        | 31.1
+[MBPP]              | 3-shot        | 42.0
+[GSM8K]             | maj@1         | 42.6
+[MATH]              | 4-shot        | 23.8
+[AGIEval]           |               | 39.3
+[BIG-Bench]         |               | 55.2
+**Average**         |               | 56.1
+
+### Inference speed results
+
+RecurrentGemma provides improved sampling speeds, particularly for long sequences or large batch sizes. We compared the sampling speeds of RecurrentGemma-9B to Gemma-7B. Note that Gemma-7B uses Multi-Head Attention, and the speed improvements would be smaller when comparing against a transformer using Multi-Query Attention.
+
+#### Throughput
+
+We evaluated throughput, i.e., the maximum number of tokens produced per second by increasing the batch size, of RecurrentGemma-9B compared to Gemma-7B, using a prefill of 2K tokens.
+
+<img src="max_throughput.png" width="400" alt="Maximum Throughput comparison of RecurrentGemma-9B and Gemma-7B">
+
+#### Latency
+
+We also compared end-to-end speedups achieved by RecurrentGemma-9B over Gemma-7B when sampling a long sequence after a prefill of 4K tokens and using a batch size of 1.
+
+\# Tokens Sampled | Gemma-7B (sec) | RecurrentGemma-9B (sec) | Improvement (%)
+----------------- | -------------- | ----------------------- | ---------------
+128               | 3.1            | 2.8                     | 9.2%
+256               | 5.9            | 5.4                     | 9.7%
+512               | 11.6           | 10.5                    | 10.7%
+1024              | 23.5           | 20.6                    | 14.2%
+2048              | 48.2           | 40.9                    | 17.7%
+4096              | 101.9          | 81.5                    | 25.0%
+8192              | OOM            | 162.8                   | -
+16384             | OOM            | 325.2                   | -
+
+
+## Ethics and safety
+
+### Ethics and safety evaluations
+
+#### Evaluations approach
+
+Our evaluation methods include structured evaluations and internal red-teaming
+testing of relevant content policies. Red-teaming was conducted by a number of
+different teams, each with different goals and human evaluation metrics. These
+models were evaluated against a number of different categories relevant to
+ethics and safety, including:
+
+*   **Text-to-text content safety:** Human evaluation on prompts covering safety
+    policies including child sexual abuse and exploitation, harassment, violence
+    and gore, and hate speech.
+*   **Text-to-text representational harms:** Benchmark against relevant academic
+    datasets such as WinoBias and BBQ Dataset.
+*   **Memorization:** Automated evaluation of memorization of training data,
+    including the risk of personally identifiable information exposure.
+*   **Large-scale harm:** Tests for “dangerous capabilities,” such as chemical,
+    biological, radiological, and nuclear (CBRN) risks; as well as tests for
+    persuasion and deception, cybersecurity, and autonomous replication.
+
+#### Evaluation results
+
+The results of ethics and safety evaluations are within acceptable thresholds
+for meeting [internal
+policies](https://storage.googleapis.com/gweb-uniblog-publish-prod/documents/2023_Google_AI_Principles_Progress_Update.pdf#page=11)
+for categories such as child safety, content safety, representational harms,
+memorization, large-scale harms. On top of robust internal evaluations, the
+results of well known safety benchmarks like BBQ, Winogender, Winobias,
+RealToxicity, and TruthfulQA are shown here.
+
+Benchmark                | Metric | RecurrentGemma 9B | RecurrentGemma 9B IT
+------------------------ | ------ | ----------------- | --------------------
+[RealToxicity]           | avg    | 10.3              | 8.8
+[BOLD]                   |        | 39.8              | 47.9
+[CrowS-Pairs]            | top-1  | 38.7              | 39.5
+[BBQ Ambig][BBQ]         | top-1  | 95.9              | 67.1
+[BBQ Disambig][BBQ]      | top-1  | 78.6              | 78.9
+[Winogender]             | top-1  | 59.0              | 64.0
+[TruthfulQA]             |        | 38.6              | 47.7
+[Winobias 1_2][Winobias] |        | 61.5              | 60.6
+[Winobias 2_2][Winobias] |        | 90.2              | 90.3
+[Toxigen]                |        | 58.8              | 64.5
+
+## Model usage and limitations
+
+### Known limitations
+
+These models have certain limitations that users should be aware of:
+
+*   **Training data**
+    *   The quality and diversity of the training data significantly influence
+        the model's capabilities. Biases or gaps in the training data can lead
+        to limitations in the model's responses.
+    *   The scope of the training dataset determines the subject areas the model
+        can handle effectively.
+*   **Context and task complexity**
+    *   LLMs are better at tasks that can be framed with clear prompts and
+        instructions. Open-ended or highly complex tasks might be challenging.
+    *   A model's performance can be influenced by the amount of context
+        provided (longer context generally leads to better outputs, up to a
+        certain point).
+*   **Language ambiguity and nuance**
+    *   Natural language is inherently complex. LLMs might struggle to grasp
+        subtle nuances, sarcasm, or figurative language.
+*   **Factual accuracy**
+    *   LLMs generate responses based on information they learned from their
+        training datasets, but they are not knowledge bases. They may generate
+        incorrect or outdated factual statements.
+*   **Common sense**
+    *   LLMs rely on statistical patterns in language. They might lack the
+        ability to apply common sense reasoning in certain situations.
+
+### Ethical considerations and risks
+
+The development of large language models (LLMs) raises several ethical concerns.
+In creating an open model, we have carefully considered the following:
+
+*   **Bias and fairness**
+    *   LLMs trained on large-scale, real-world text data can reflect
+        socio-cultural biases embedded in the training material. These models
+        underwent careful scrutiny, input data pre-processing described and
+        posterior evaluations reported in this card.
+*   **Misinformation and misuse**
+    *   LLMs can be misused to generate text that is false, misleading, or
+        harmful.
+    *   Guidelines are provided for responsible use with the model, see the
+        [Responsible Generative AI
+        Toolkit](https://ai.google.dev/gemma/responsible).
+*   **Transparency and accountability**
+    *   This model card summarizes details on the models' architecture,
+        capabilities, limitations, and evaluation processes.
+    *   A responsibly developed open model offers the opportunity to share
+        innovation by making LLM technology accessible to developers and
+        researchers across the AI ecosystem.
+
+Risks Identified and Mitigations:
+
+*   **Perpetuation of biases:** It's encouraged to perform continuous monitoring
+    (using evaluation metrics, human review) and the exploration of de-biasing
+    techniques during model training, fine-tuning, and other use cases.
+*   **Generation of harmful content:** Mechanisms and guidelines for content
+    safety are essential. Developers are encouraged to exercise caution and
+    implement appropriate content safety safeguards based on their specific
+    product policies and application use cases.
+*   **Misuse for malicious purposes:** Technical limitations and developer and
+    end-user education can help mitigate against malicious applications of LLMs.
+    Educational resources and reporting mechanisms for users to flag misuse are
+    provided. Prohibited uses of Gemma models are outlined in our [terms of
+    use](https://www.kaggle.com/models/google/gemma/license/consent).
+*   **Privacy violations:** Models were trained on data filtered for removal of
+    PII (Personally Identifiable Information). Developers are encouraged to
+    adhere to privacy regulations with privacy-preserving techniques.
+
+## Intended usage
+
+### Application
+
+Open Large Language Models (LLMs) have a wide range of applications across
+various industries and domains. The following list of potential uses is not
+comprehensive. The purpose of this list is to provide contextual information
+about the possible use-cases that the model creators considered as part of model
+training and development.
+
+*   **Content creation and communication**
+    *   **Text generation:** These models can be used to generate creative text
+        formats like poems, scripts, code, marketing copy, email drafts, etc.
+    *   **Chatbots and conversational AI:** Power conversational interfaces for
+        customer service, virtual assistants, or interactive applications.
+    *   **Text summarization:** Generate concise summaries of a text corpus,
+        research papers, or reports.
+*   **Research and education**
+    *   **Natural Language Processing (NLP) research:** These models can serve
+        as a foundation for researchers to experiment with NLP techniques,
+        develop algorithms, and contribute to the advancement of the field.
+    *   **Language Learning Tools:** Support interactive language learning
+        experiences, aiding in grammar correction or providing writing practice.
+    *   **Knowledge Exploration:** Assist researchers in exploring large bodies
+        of text by generating summaries or answering questions about specific
+        topics.
+
+### Benefits
+
+At the time of release, this family of models provides high-performance open
+large language model implementations designed from the ground up for Responsible
+AI development compared to similarly sized models.
+
+Using the benchmark evaluation metrics described in this document, these models
+have shown to provide superior performance to other, comparably-sized open model
+alternatives.
+
+In particular, RecurrentGemma models achieve comparable performance to Gemma
+models but are faster during inference and require less memory, especially on
+long sequences.
+
+[MMLU]: https://arxiv.org/abs/2009.03300
+[HellaSwag]: https://arxiv.org/abs/1905.07830
+[PIQA]: https://arxiv.org/abs/1911.11641
+[SocialIQA]: https://arxiv.org/abs/1904.09728
+[BoolQ]: https://arxiv.org/abs/1905.10044
+[winogrande]: https://arxiv.org/abs/1907.10641
+[CommonsenseQA]: https://arxiv.org/abs/1811.00937
+[OpenBookQA]: https://arxiv.org/abs/1809.02789
+[ARC-c]: https://arxiv.org/abs/1911.01547
+[TriviaQA]: https://arxiv.org/abs/1705.03551
+[Natural Questions]: https://github.com/google-research-datasets/natural-questions
+[HumanEval]: https://arxiv.org/abs/2107.03374
+[MBPP]: https://arxiv.org/abs/2108.07732
+[GSM8K]: https://arxiv.org/abs/2110.14168
+[MATH]: https://arxiv.org/abs/2103.03874
+[AGIEval]: https://arxiv.org/abs/2304.06364
+[BIG-Bench]: https://arxiv.org/abs/2206.04615
+[RealToxicity]: https://arxiv.org/abs/2009.11462
+[BOLD]: https://arxiv.org/abs/2101.11718
+[CrowS-Pairs]: https://aclanthology.org/2020.emnlp-main.154/
+[BBQ]: https://arxiv.org/abs/2110.08193v2
+[Winogender]: https://arxiv.org/abs/1804.09301
+[TruthfulQA]: https://arxiv.org/abs/2109.07958
+[winobias]: https://arxiv.org/abs/1804.06876
+[Toxigen]: https://arxiv.org/abs/2203.09509

config.json

@@ -0,0 +1,36 @@
+{
+  "architectures": [
+    "RecurrentGemmaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attention_window_size": 2048,
+  "block_types": [
+    "recurrent",
+    "recurrent",
+    "attention"
+  ],
+  "bos_token_id": 2,
+  "conv1d_width": 4,
+  "eos_token_id": 1,
+  "final_w_init_variance_scale": 0.05263157894736842,
+  "head_dim": 256,
+  "hidden_activation": "gelu_pytorch_tanh",
+  "hidden_size": 4096,
+  "intermediate_size": 24576,
+  "logits_soft_cap": 30.0,
+  "lru_width": 4096,
+  "model_type": "recurrent_gemma",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 38,
+  "num_key_value_heads": 1,
+  "pad_token_id": 0,
+  "partial_rotary_factor": 0.5,
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 10000.0,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.42.0.dev0",
+  "use_cache": true,
+  "vocab_size": 256000,
+  "w_init_variance_scale": 0.01
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 2,
+  "eos_token_id": 1,
+  "pad_token_id": 0,
+  "transformers_version": "4.42.0.dev0"
+}

model-00001-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10342736cccfe28546b3f0629921af059abf9f4db44b2b5b7170c4c78e08f84e
+size 4983940472

model-00002-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:552f9d9c8d2a6aa08d819078103606b2bf115e19ab3577634ebc89aa5fd0fd46
+size 4950993344

model-00003-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:453897232aee633bab8083b32de30bbf4017ac189b55e39b6a5cda55b3244ded
+size 4921617480

model-00004-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d4e9446638cdb8b7228019901a8b30f591a74d1fbbdb622f5c96e572b926b34
+size 4400640456

model.safetensors.index.json

@@ -0,0 +1,712 @@
+{
+  "metadata": {
+    "total_size": 19257106432
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00004-of-00004.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00004.safetensors",
+    "model.final_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.0.channel_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp_block.down_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp_block.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp_block.gate_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp_block.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp_block.up_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp_block.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.conv_1d.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.conv_1d.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.linear_out.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.linear_out.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.linear_x.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.linear_x.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.linear_y.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.linear_y.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.rg_lru.input_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.rg_lru.input_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.rg_lru.recurrent_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.rg_lru.recurrent_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_block.rg_lru.recurrent_param": "model-00001-of-00004.safetensors",
+    "model.layers.0.temporal_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.channel_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp_block.down_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp_block.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp_block.gate_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp_block.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp_block.up_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp_block.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.conv_1d.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.conv_1d.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.linear_out.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.linear_out.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.linear_x.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.linear_x.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.linear_y.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.linear_y.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.rg_lru.input_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.rg_lru.input_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.rg_lru.recurrent_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.rg_lru.recurrent_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_block.rg_lru.recurrent_param": "model-00001-of-00004.safetensors",
+    "model.layers.1.temporal_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.10.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.conv_1d.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.conv_1d.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.linear_out.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.linear_out.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.linear_x.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.linear_x.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.linear_y.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.linear_y.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.rg_lru.input_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.rg_lru.input_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.rg_lru.recurrent_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.rg_lru.recurrent_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_block.rg_lru.recurrent_param": "model-00002-of-00004.safetensors",
+    "model.layers.10.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.temporal_block.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.temporal_block.o_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.11.temporal_block.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.temporal_block.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.temporal_block.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.conv_1d.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.conv_1d.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.linear_out.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.linear_out.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.linear_x.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.linear_x.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.linear_y.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.linear_y.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.rg_lru.input_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.rg_lru.input_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.rg_lru.recurrent_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.rg_lru.recurrent_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_block.rg_lru.recurrent_param": "model-00002-of-00004.safetensors",
+    "model.layers.12.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.conv_1d.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.conv_1d.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.linear_out.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.linear_out.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.linear_x.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.linear_x.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.linear_y.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.linear_y.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.rg_lru.input_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.rg_lru.input_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.rg_lru.recurrent_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.rg_lru.recurrent_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_block.rg_lru.recurrent_param": "model-00002-of-00004.safetensors",
+    "model.layers.13.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.temporal_block.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.temporal_block.o_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.14.temporal_block.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.temporal_block.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.temporal_block.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.conv_1d.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.conv_1d.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.linear_out.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.linear_out.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.linear_x.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.linear_x.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.linear_y.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.linear_y.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.rg_lru.input_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.rg_lru.input_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.rg_lru.recurrent_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.rg_lru.recurrent_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_block.rg_lru.recurrent_param": "model-00002-of-00004.safetensors",
+    "model.layers.15.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.conv_1d.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.conv_1d.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.linear_out.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.linear_out.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.linear_x.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.linear_x.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.linear_y.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.linear_y.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.rg_lru.input_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.rg_lru.input_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.rg_lru.recurrent_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.rg_lru.recurrent_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_block.rg_lru.recurrent_param": "model-00002-of-00004.safetensors",
+    "model.layers.16.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.temporal_block.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.temporal_block.o_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.17.temporal_block.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.temporal_block.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.temporal_block.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.conv_1d.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.conv_1d.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.linear_out.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.linear_out.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.linear_x.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.linear_x.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.linear_y.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.linear_y.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.rg_lru.input_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.rg_lru.input_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.rg_lru.recurrent_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.rg_lru.recurrent_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_block.rg_lru.recurrent_param": "model-00002-of-00004.safetensors",
+    "model.layers.18.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.19.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.19.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.19.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.19.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.conv_1d.bias": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.conv_1d.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.linear_out.bias": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.linear_out.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.linear_x.bias": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.linear_x.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.linear_y.bias": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.linear_y.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.rg_lru.input_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.rg_lru.input_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.rg_lru.recurrent_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.rg_lru.recurrent_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_block.rg_lru.recurrent_param": "model-00002-of-00004.safetensors",
+    "model.layers.19.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.2.channel_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp_block.down_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp_block.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp_block.gate_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp_block.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp_block.up_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp_block.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.temporal_block.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.temporal_block.o_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.2.temporal_block.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.temporal_block.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.temporal_block.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.temporal_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.20.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.temporal_block.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.temporal_block.o_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.20.temporal_block.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.temporal_block.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.temporal_block.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.conv_1d.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.conv_1d.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.linear_out.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.linear_out.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.linear_x.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.linear_x.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.linear_y.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.linear_y.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.rg_lru.input_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.rg_lru.input_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.rg_lru.recurrent_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.rg_lru.recurrent_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_block.rg_lru.recurrent_param": "model-00003-of-00004.safetensors",
+    "model.layers.21.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.conv_1d.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.conv_1d.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.linear_out.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.linear_out.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.linear_x.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.linear_x.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.linear_y.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.linear_y.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.rg_lru.input_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.rg_lru.input_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.rg_lru.recurrent_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.rg_lru.recurrent_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_block.rg_lru.recurrent_param": "model-00003-of-00004.safetensors",
+    "model.layers.22.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.temporal_block.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.temporal_block.o_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.23.temporal_block.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.temporal_block.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.temporal_block.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.conv_1d.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.conv_1d.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.linear_out.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.linear_out.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.linear_x.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.linear_x.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.linear_y.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.linear_y.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.rg_lru.input_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.rg_lru.input_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.rg_lru.recurrent_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.rg_lru.recurrent_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_block.rg_lru.recurrent_param": "model-00003-of-00004.safetensors",
+    "model.layers.24.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.conv_1d.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.conv_1d.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.linear_out.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.linear_out.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.linear_x.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.linear_x.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.linear_y.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.linear_y.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.rg_lru.input_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.rg_lru.input_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.rg_lru.recurrent_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.rg_lru.recurrent_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_block.rg_lru.recurrent_param": "model-00003-of-00004.safetensors",
+    "model.layers.25.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.temporal_block.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.temporal_block.o_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.26.temporal_block.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.temporal_block.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.temporal_block.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.conv_1d.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.conv_1d.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.linear_out.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.linear_out.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.linear_x.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.linear_x.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.linear_y.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.linear_y.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.rg_lru.input_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.rg_lru.input_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.rg_lru.recurrent_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.rg_lru.recurrent_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_block.rg_lru.recurrent_param": "model-00003-of-00004.safetensors",
+    "model.layers.27.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.conv_1d.bias": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.conv_1d.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.linear_out.bias": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.linear_out.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.linear_x.bias": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.linear_x.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.linear_y.bias": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.linear_y.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.rg_lru.input_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.rg_lru.input_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.rg_lru.recurrent_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.rg_lru.recurrent_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_block.rg_lru.recurrent_param": "model-00003-of-00004.safetensors",
+    "model.layers.28.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.temporal_block.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.temporal_block.o_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.29.temporal_block.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.temporal_block.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.temporal_block.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.3.channel_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp_block.down_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp_block.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp_block.gate_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp_block.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp_block.up_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp_block.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.conv_1d.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.conv_1d.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.linear_out.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.linear_out.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.linear_x.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.linear_x.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.linear_y.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.linear_y.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.rg_lru.input_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.rg_lru.input_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.rg_lru.recurrent_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.rg_lru.recurrent_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_block.rg_lru.recurrent_param": "model-00001-of-00004.safetensors",
+    "model.layers.3.temporal_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.30.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.conv_1d.bias": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.conv_1d.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.linear_out.bias": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.linear_out.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.linear_x.bias": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.linear_x.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.linear_y.bias": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.linear_y.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.rg_lru.input_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.rg_lru.input_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.rg_lru.recurrent_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.rg_lru.recurrent_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_block.rg_lru.recurrent_param": "model-00003-of-00004.safetensors",
+    "model.layers.30.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.mlp_block.down_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.31.mlp_block.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.mlp_block.gate_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.31.mlp_block.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.mlp_block.up_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.31.mlp_block.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.conv_1d.bias": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.conv_1d.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.linear_out.bias": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.linear_out.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.linear_x.bias": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.linear_x.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.linear_y.bias": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.linear_y.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.rg_lru.input_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.rg_lru.input_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.rg_lru.recurrent_gate_bias": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.rg_lru.recurrent_gate_weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_block.rg_lru.recurrent_param": "model-00003-of-00004.safetensors",
+    "model.layers.31.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.32.channel_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.32.mlp_block.down_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.32.mlp_block.down_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.32.mlp_block.gate_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.32.mlp_block.gate_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.32.mlp_block.up_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.32.mlp_block.up_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.32.temporal_block.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.32.temporal_block.o_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.32.temporal_block.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.32.temporal_block.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.32.temporal_block.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.32.temporal_pre_norm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.33.channel_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.mlp_block.down_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.33.mlp_block.down_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.mlp_block.gate_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.33.mlp_block.gate_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.mlp_block.up_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.33.mlp_block.up_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.conv_1d.bias": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.conv_1d.weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.linear_out.bias": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.linear_out.weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.linear_x.bias": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.linear_x.weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.linear_y.bias": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.linear_y.weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.rg_lru.input_gate_bias": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.rg_lru.input_gate_weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.rg_lru.recurrent_gate_bias": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.rg_lru.recurrent_gate_weight": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_block.rg_lru.recurrent_param": "model-00004-of-00004.safetensors",
+    "model.layers.33.temporal_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.channel_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.mlp_block.down_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.34.mlp_block.down_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.mlp_block.gate_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.34.mlp_block.gate_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.mlp_block.up_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.34.mlp_block.up_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.conv_1d.bias": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.conv_1d.weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.linear_out.bias": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.linear_out.weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.linear_x.bias": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.linear_x.weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.linear_y.bias": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.linear_y.weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.rg_lru.input_gate_bias": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.rg_lru.input_gate_weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.rg_lru.recurrent_gate_bias": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.rg_lru.recurrent_gate_weight": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_block.rg_lru.recurrent_param": "model-00004-of-00004.safetensors",
+    "model.layers.34.temporal_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.35.channel_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.35.mlp_block.down_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.35.mlp_block.down_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.35.mlp_block.gate_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.35.mlp_block.gate_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.35.mlp_block.up_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.35.mlp_block.up_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.35.temporal_block.k_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.35.temporal_block.o_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.35.temporal_block.o_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.35.temporal_block.q_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.35.temporal_block.v_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.35.temporal_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.channel_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.mlp_block.down_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.36.mlp_block.down_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.mlp_block.gate_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.36.mlp_block.gate_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.mlp_block.up_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.36.mlp_block.up_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.conv_1d.bias": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.conv_1d.weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.linear_out.bias": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.linear_out.weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.linear_x.bias": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.linear_x.weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.linear_y.bias": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.linear_y.weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.rg_lru.input_gate_bias": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.rg_lru.input_gate_weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.rg_lru.recurrent_gate_bias": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.rg_lru.recurrent_gate_weight": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_block.rg_lru.recurrent_param": "model-00004-of-00004.safetensors",
+    "model.layers.36.temporal_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.channel_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.mlp_block.down_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.37.mlp_block.down_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.mlp_block.gate_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.37.mlp_block.gate_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.mlp_block.up_proj.bias": "model-00004-of-00004.safetensors",
+    "model.layers.37.mlp_block.up_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.conv_1d.bias": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.conv_1d.weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.linear_out.bias": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.linear_out.weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.linear_x.bias": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.linear_x.weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.linear_y.bias": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.linear_y.weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.rg_lru.input_gate_bias": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.rg_lru.input_gate_weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.rg_lru.recurrent_gate_bias": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.rg_lru.recurrent_gate_weight": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_block.rg_lru.recurrent_param": "model-00004-of-00004.safetensors",
+    "model.layers.37.temporal_pre_norm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.4.channel_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp_block.down_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp_block.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp_block.gate_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp_block.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp_block.up_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp_block.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.conv_1d.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.conv_1d.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.linear_out.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.linear_out.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.linear_x.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.linear_x.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.linear_y.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.linear_y.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.rg_lru.input_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.rg_lru.input_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.rg_lru.recurrent_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.rg_lru.recurrent_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_block.rg_lru.recurrent_param": "model-00001-of-00004.safetensors",
+    "model.layers.4.temporal_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.channel_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp_block.down_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp_block.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp_block.gate_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp_block.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp_block.up_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp_block.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.temporal_block.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.temporal_block.o_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.5.temporal_block.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.temporal_block.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.temporal_block.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.temporal_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.channel_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp_block.down_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp_block.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp_block.gate_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp_block.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp_block.up_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp_block.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.conv_1d.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.conv_1d.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.linear_out.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.linear_out.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.linear_x.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.linear_x.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.linear_y.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.linear_y.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.rg_lru.input_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.rg_lru.input_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.rg_lru.recurrent_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.rg_lru.recurrent_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_block.rg_lru.recurrent_param": "model-00001-of-00004.safetensors",
+    "model.layers.6.temporal_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.channel_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.7.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.7.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.7.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.7.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.7.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.7.temporal_block.conv_1d.bias": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.conv_1d.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.linear_out.bias": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.linear_out.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.linear_x.bias": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.linear_x.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.linear_y.bias": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.linear_y.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.rg_lru.input_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.rg_lru.input_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.rg_lru.recurrent_gate_bias": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.rg_lru.recurrent_gate_weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_block.rg_lru.recurrent_param": "model-00001-of-00004.safetensors",
+    "model.layers.7.temporal_pre_norm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.8.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.8.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.8.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.temporal_block.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.temporal_block.o_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.8.temporal_block.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.temporal_block.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.temporal_block.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.temporal_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.channel_pre_norm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp_block.down_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp_block.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp_block.gate_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp_block.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp_block.up_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp_block.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.conv_1d.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.conv_1d.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.linear_out.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.linear_out.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.linear_x.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.linear_x.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.linear_y.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.linear_y.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.rg_lru.input_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.rg_lru.input_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.rg_lru.recurrent_gate_bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.rg_lru.recurrent_gate_weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_block.rg_lru.recurrent_param": "model-00002-of-00004.safetensors",
+    "model.layers.9.temporal_pre_norm.weight": "model-00002-of-00004.safetensors"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c15eb04bc5ad609fb26533e8525302c5640a945e5f67f65b7c849900acda7d99
+size 17518497

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61a7b147390c64585d6c3543dd6fc636906c9af3865a5548f27f31aee1d4c8e2
+size 4241003

tokenizer_config.json

@@ -0,0 +1,1516 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<unused60>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<unused61>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<unused62>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<unused63>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<start_of_turn>",
+    "<end_of_turn>"
+  ],
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}