test

README.md

@@ -0,0 +1,103 @@
+---
+language: "eng"
+tags:
+- wikihow
+- t5-small
+- pytorch
+- lm-head
+- seq2seq
+- t5
+- pipeline:summarization
+- summarization
+datasets:
+- Wikihow 
+widget:
+- text: "Lack of fluids can lead to dry mouth, which is a leading cause of bad breath. Water
+can also dilute any chemicals in your mouth or gut that are causing bad breath., Studies show that
+eating 6 ounces of yogurt a day reduces the level of odor-causing compounds in the mouth. In
+particular, look for yogurt containing the active bacteria Streptococcus thermophilus or
+Lactobacillus bulgaricus., The abrasive nature of fibrous fruits and vegetables helps to clean
+teeth, while the vitamins, antioxidants, and acids they contain improve dental health.Foods that can
+be particularly helpful include:Apples — Apples contain vitamin C, which is necessary for health
+gums, as well as malic acid, which helps to whiten teeth.Carrots — Carrots are rich in vitamin A,
+which strengthens tooth enamel.Celery — Chewing celery produces a lot of saliva, which helps to
+neutralize bacteria that cause bad breath.Pineapples — Pineapples contain bromelain, an enzyme that
+cleans the mouth., These teas have been shown to kill the bacteria that cause bad breath and
+plaque., An upset stomach can lead to burping, which contributes to bad breath. Don’t eat foods that
+upset your stomach, or if you do, use antacids. If you are lactose intolerant, try lactase tablets.,
+They can all cause bad breath. If you do eat them, bring sugar-free gum or a toothbrush and
+toothpaste to freshen your mouth afterwards., Diets low in carbohydrates lead to ketosis — a state
+in which the body burns primarily fat instead of carbohydrates for energy. This may be good for your
+waistline, but it also produces chemicals called ketones, which contribute to bad breath.To stop the
+problem, you must change your diet. Or, you can combat the smell in one of these ways:Drink lots of
+water to dilute the ketones.Chew sugarless gum or suck on sugarless mints.Chew mint leaves."
+- text: " Bring 1/2 cup water to the boil.Add the fresh or dried rosemary to the water.Remove
+from the heat. Set aside for 1/2 an hour to infuse. Added flavour can be released by pressing down
+on the rosemary leaves with a spoon. Add the pieces to the blender or food processor with the
+elderflower cordial. Blend or process to a purée.,, Add the lemon or lime juice and stir to
+combine., Add a cover and place in the freezer.After 2 hours, remove from the freezer and break up
+with a fork. This helps the ice crystals to form properly.Continue doing this every hour until the
+granita freezes properly. Scoop the granita into dessert bowls and serve. Garnish with a cucumber
+curl or a small sprig of rosemary."
+metrics:
+- Rouge1: 31.2
+- RougeL: 24.5
+---
+
+# Model name
+Wikihow T5-small
+
+## Model description
+
+This is a T5-small model trained on Wikihow All data set. The model was trained for 3 epochs using a batch size of 16 and learning rate of 3e-4. Max_input_lngth is set as 512 and max_output_length is 150. Model attained a Rouge1 score of 31.2 and RougeL score of 24.5. 
+We have written a blog post that covers the training procedure. Please find it [here](https://medium.com/@priya.dwivedi/fine-tuning-a-t5-transformer-for-any-summarization-task-82334c64c81). 
+
+## Usage
+
+```
+from transformers import AutoTokenizer, AutoModelWithLMHead
+
+tokenizer = AutoTokenizer.from_pretrained("deep-learning-analytics/wikihow-t5-small")
+model = AutoModelWithLMHead.from_pretrained("deep-learning-analytics/wikihow-t5-small")
+
+device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
+model = model.to(device)
+
+text = """"
+Lack of fluids can lead to dry mouth, which is a leading cause of bad breath. Water
+can also dilute any chemicals in your mouth or gut that are causing bad breath., Studies show that
+eating 6 ounces of yogurt a day reduces the level of odor-causing compounds in the mouth. In
+particular, look for yogurt containing the active bacteria Streptococcus thermophilus or
+Lactobacillus bulgaricus., The abrasive nature of fibrous fruits and vegetables helps to clean
+teeth, while the vitamins, antioxidants, and acids they contain improve dental health.Foods that can
+be particularly helpful include:Apples — Apples contain vitamin C, which is necessary for health
+gums, as well as malic acid, which helps to whiten teeth.Carrots — Carrots are rich in vitamin A,
+which strengthens tooth enamel.Celery — Chewing celery produces a lot of saliva, which helps to
+neutralize bacteria that cause bad breath.Pineapples — Pineapples contain bromelain, an enzyme that
+cleans the mouth., These teas have been shown to kill the bacteria that cause bad breath and
+plaque., An upset stomach can lead to burping, which contributes to bad breath. Don’t eat foods that
+upset your stomach, or if you do, use antacids. If you are lactose intolerant, try lactase tablets.,
+They can all cause bad breath. If you do eat them, bring sugar-free gum or a toothbrush and
+toothpaste to freshen your mouth afterwards., Diets low in carbohydrates lead to ketosis — a state
+in which the body burns primarily fat instead of carbohydrates for energy. This may be good for your
+waistline, but it also produces chemicals called ketones, which contribute to bad breath.To stop the
+problem, you must change your diet. Or, you can combat the smell in one of these ways:Drink lots of
+water to dilute the ketones.Chew sugarless gum or suck on sugarless mints.Chew mint leaves.
+"""
+
+preprocess_text = text.strip().replace("\n","")
+tokenized_text = tokenizer.encode(preprocess_text, return_tensors="pt").to(device)
+
+summary_ids = model.generate(
+            tokenized_text,
+            max_length=150, 
+            num_beams=2,
+            repetition_penalty=2.5, 
+            length_penalty=1.0, 
+            early_stopping=True
+        )
+
+output = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
+
+print ("\n\nSummarized text: \n",output)
+```

config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "T5ForConditionalGeneration"
+  ],
+  "d_ff": 2048,
+  "d_kv": 64,
+  "d_model": 512,
+  "decoder_start_token_id": 0,
+  "dropout_rate": 0.1,
+  "eos_token_id": 1,
+  "initializer_factor": 1.0,
+  "is_encoder_decoder": true,
+  "layer_norm_epsilon": 1e-06,
+  "model_type": "t5",
+  "n_positions": 512,
+  "num_heads": 8,
+  "num_layers": 6,
+  "output_past": true,
+  "pad_token_id": 0,
+  "relative_attention_num_buckets": 32,
+  "save_step": 0,
+  "task_specific_params": {
+    "summarization": {
+      "early_stopping": true,
+      "length_penalty": 2.0,
+      "max_length": 200,
+      "min_length": 30,
+      "no_repeat_ngram_size": 3,
+      "num_beams": 4,
+      "prefix": "summarize: "
+    }
+  },
+  "vocab_size": 32128
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9374d3f2ab7f758b7a00ea70b1ac2608aac13c67d37a7131987cc221b867dc23
+size 242068027

special_tokens_map.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "additional_special_tokens": ["<extra_id_0>", "<extra_id_1>", "<extra_id_2>", "<extra_id_3>", "<extra_id_4>", "<extra_id_5>", "<extra_id_6>", "<extra_id_7>", "<extra_id_8>", "<extra_id_9>", "<extra_id_10>", "<extra_id_11>", "<extra_id_12>", "<extra_id_13>", "<extra_id_14>", "<extra_id_15>", "<extra_id_16>", "<extra_id_17>", "<extra_id_18>", "<extra_id_19>", "<extra_id_20>", "<extra_id_21>", "<extra_id_22>", "<extra_id_23>", "<extra_id_24>", "<extra_id_25>", "<extra_id_26>", "<extra_id_27>", "<extra_id_28>", "<extra_id_29>", "<extra_id_30>", "<extra_id_31>", "<extra_id_32>", "<extra_id_33>", "<extra_id_34>", "<extra_id_35>", "<extra_id_36>", "<extra_id_37>", "<extra_id_38>", "<extra_id_39>", "<extra_id_40>", "<extra_id_41>", "<extra_id_42>", "<extra_id_43>", "<extra_id_44>", "<extra_id_45>", "<extra_id_46>", "<extra_id_47>", "<extra_id_48>", "<extra_id_49>", "<extra_id_50>", "<extra_id_51>", "<extra_id_52>", "<extra_id_53>", "<extra_id_54>", "<extra_id_55>", "<extra_id_56>", "<extra_id_57>", "<extra_id_58>", "<extra_id_59>", "<extra_id_60>", "<extra_id_61>", "<extra_id_62>", "<extra_id_63>", "<extra_id_64>", "<extra_id_65>", "<extra_id_66>", "<extra_id_67>", "<extra_id_68>", "<extra_id_69>", "<extra_id_70>", "<extra_id_71>", "<extra_id_72>", "<extra_id_73>", "<extra_id_74>", "<extra_id_75>", "<extra_id_76>", "<extra_id_77>", "<extra_id_78>", "<extra_id_79>", "<extra_id_80>", "<extra_id_81>", "<extra_id_82>", "<extra_id_83>", "<extra_id_84>", "<extra_id_85>", "<extra_id_86>", "<extra_id_87>", "<extra_id_88>", "<extra_id_89>", "<extra_id_90>", "<extra_id_91>", "<extra_id_92>", "<extra_id_93>", "<extra_id_94>", "<extra_id_95>", "<extra_id_96>", "<extra_id_97>", "<extra_id_98>", "<extra_id_99>"]}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d60acb128cf7b7f2536e8f38a5b18a05535c9e14c7a355904270e15b0945ea86
+size 791656

tokenizer_config.json

@@ -0,0 +1 @@
+{"model_max_length": 512}