Training in progress epoch 0

README.md

@@ -1,62 +1,54 @@
 ---
-language: en
 license: mit
-datasets:
-- bookcorpus
-- wikipedia
+tags:
+- generated_from_keras_callback
+model-index:
+- name: sushruth13/xlnet_base1.0
+  results: []
 ---
 
-# XLNet (base-sized model) 
+<!-- This model card has been generated automatically according to the information Keras had access to. You should
+probably proofread and complete it, then remove this comment. -->
 
-XLNet model pre-trained on English language. It was introduced in the paper [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://arxiv.org/abs/1906.08237) by Yang et al. and first released in [this repository](https://github.com/zihangdai/xlnet/). 
+# sushruth13/xlnet_base1.0
 
-Disclaimer: The team releasing XLNet did not write a model card for this model so this model card has been written by the Hugging Face team.
+This model is a fine-tuned version of [xlnet-base-cased](https://huggingface.co/xlnet-base-cased) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Train Loss: 1.9286
+- Validation Loss: 1.9409
+- Train Accuracy: 0.3077
+- Epoch: 0
 
 ## Model description
 
-XLNet is a new unsupervised language representation learning method based on a novel generalized permutation language modeling objective. Additionally, XLNet employs Transformer-XL as the backbone model, exhibiting excellent performance for language tasks involving long context. Overall, XLNet achieves state-of-the-art (SOTA) results on various downstream language tasks including question answering, natural language inference, sentiment analysis, and document ranking.
+More information needed
 
 ## Intended uses & limitations
 
-The model is mostly intended to be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?search=xlnet) to look for fine-tuned versions on a task that interests you.
+More information needed
 
-Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification or question answering. For tasks such as text generation, you should look at models like GPT2.
+## Training and evaluation data
 
-## Usage
+More information needed
 
-Here is how to use this model to get the features of a given text in PyTorch:
+## Training procedure
 
-```python
-from transformers import XLNetTokenizer, XLNetModel
+### Training hyperparameters
 
-tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased')
-model = XLNetModel.from_pretrained('xlnet-base-cased')
+The following hyperparameters were used during training:
+- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': False, 'is_legacy_optimizer': False, 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 3050, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
+- training_precision: float32
 
-inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
-outputs = model(**inputs)
+### Training results
 
-last_hidden_states = outputs.last_hidden_state
-```
+| Train Loss | Validation Loss | Train Accuracy | Epoch |
+|:----------:|:---------------:|:--------------:|:-----:|
+| 1.9286     | 1.9409          | 0.3077         | 0     |
 
-### BibTeX entry and citation info
 
-```bibtex
-@article{DBLP:journals/corr/abs-1906-08237,
-  author    = {Zhilin Yang and
-               Zihang Dai and
-               Yiming Yang and
-               Jaime G. Carbonell and
-               Ruslan Salakhutdinov and
-               Quoc V. Le},
-  title     = {XLNet: Generalized Autoregressive Pretraining for Language Understanding},
-  journal   = {CoRR},
-  volume    = {abs/1906.08237},
-  year      = {2019},
-  url       = {http://arxiv.org/abs/1906.08237},
-  eprinttype = {arXiv},
-  eprint    = {1906.08237},
-  timestamp = {Mon, 24 Jun 2019 17:28:45 +0200},
-  biburl    = {https://dblp.org/rec/journals/corr/abs-1906-08237.bib},
-  bibsource = {dblp computer science bibliography, https://dblp.org}
-}
-```
+### Framework versions
+
+- Transformers 4.28.1
+- TensorFlow 2.12.0
+- Datasets 2.11.0
+- Tokenizers 0.13.3

config.json

@@ -1,6 +1,7 @@
 {
+  "_name_or_path": "xlnet-base-cased",
   "architectures": [
-    "XLNetLMHeadModel"
+    "XLNetForSequenceClassification"
   ],
   "attn_type": "bi",
   "bi_data": false,
@@ -13,7 +14,23 @@
   "end_n_top": 5,
   "eos_token_id": 2,
   "ff_activation": "gelu",
+  "id2label": {
+    "0": "broadside",
+    "1": "side swipe",
+    "2": "hit object",
+    "3": "head-on",
+    "4": "rear end",
+    "5": "other"
+  },
   "initializer_range": 0.02,
+  "label2id": {
+    "broadside": 0,
+    "head-on": 3,
+    "hit object": 2,
+    "other": 5,
+    "rear end": 4,
+    "side swipe": 1
+  },
   "layer_norm_eps": 1e-12,
   "mem_len": null,
   "model_type": "xlnet",
@@ -33,6 +50,9 @@
       "max_length": 250
     }
   },
+  "transformers_version": "4.28.1",
   "untie_r": true,
+  "use_mems_eval": true,
+  "use_mems_train": false,
   "vocab_size": 32000
-}
+}

special_tokens_map.json

@@ -0,0 +1,19 @@
+{
+  "additional_special_tokens": [
+    "<eop>",
+    "<eod>"
+  ],
+  "bos_token": "<s>",
+  "cls_token": "<cls>",
+  "eos_token": "</s>",
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<pad>",
+  "sep_token": "<sep>",
+  "unk_token": "<unk>"
+}

tf_model.h5

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c91def0a6a0adae911fed1630194d76497736887380848054494ed9ed9324c32
-size 565485600
+oid sha256:6f5e575f6f5494c90d143c1ca2450b7e2332404b836b415f25a15933979a9632
+size 469461000

tokenizer_config.json

@@ -0,0 +1,26 @@
+{
+  "additional_special_tokens": [
+    "<eop>",
+    "<eod>"
+  ],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<cls>",
+  "do_lower_case": false,
+  "eos_token": "</s>",
+  "keep_accents": false,
+  "mask_token": {
+    "__type": "AddedToken",
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "remove_space": true,
+  "sep_token": "<sep>",
+  "tokenizer_class": "XLNetTokenizer",
+  "unk_token": "<unk>"
+}