{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from generate import generate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release\n",
      "  warnings.warn(\n",
      "Loading model '/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/lit_model.pth' with {'name': 'Llama-2-7b-chat-hf', 'hf_config': {'org': 'meta-llama', 'name': 'Llama-2-7b-chat-hf'}, 'block_size': 2048, 'vocab_size': 50254, 'padding_multiple': 128, 'padded_vocab_size': 50304, 'n_layer': 12, 'n_head': 12, 'n_embd': 768, 'rotary_percentage': 0.25, 'parallel_residual': True, 'bias': True, 'lm_head_bias': False, 'n_query_groups': 12, 'shared_attention_norm': False, '_norm_class': 'LayerNorm', 'norm_eps': 1e-05, '_mlp_class': 'GptNeoxMLP', 'gelu_approximate': 'none', 'intermediate_size': 3072, 'rope_condense_ratio': 1, 'rope_base': 10000, 'head_size': 64, 'rope_n_elem': 16}\n",
      "Time to instantiate model: 0.21 seconds.\n",
      "Time to load the model weights: 0.29 seconds.\n",
      "Seed set to 1234\n",
      "James Bond asked for a Vodka Martini, shaken and 17-year-old in a handheld family home, which he has since met with heavy duty clinton officials.\n",
      "\"I don't'm an 20-year-old, and I do not want to hurt him because nobody wants him to exist. I've tried to make a real difference and it felt like superbly at the time. I was the only one who busts it around and I had no problem. I've gotten back to 91. \" (One wants to change my name, of course.) I think that I, being married, would be happy to have married after a couple of years, but I'd still not be older. I cannot have anything in common with my old family, as it was the life of the person who had his own brother,\" said Mark Stenyard, a psychiatrist and partner of the Treasury. \"When you told me that I'm divorced here, I'm OK. I am going to tell you that I, being divorced, would be happy to have married. I need one, my family, and I'm still late, but am just so disgusted.\"\n",
      "Time for inference 1: 1.85 sec total, 135.39 tokens/sec\n",
      "Memory used: 0.35 GB\n"
     ]
    }
   ],
   "source": [
    " !python3 generate.py --prompt=\"James Bond asked for a Vodka Martini, shaken and \" --checkpoint_dir=/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/ --max_new_tokens=250 --temperature=0.9 --num_samples=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release\n",
      "  warnings.warn(\n",
      "Loading model '/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/lit_model.pth' with {'name': 'Llama-2-7b-chat-hf', 'hf_config': {'org': 'meta-llama', 'name': 'Llama-2-7b-chat-hf'}, 'block_size': 2048, 'vocab_size': 50254, 'padding_multiple': 128, 'padded_vocab_size': 50304, 'n_layer': 12, 'n_head': 12, 'n_embd': 768, 'rotary_percentage': 0.25, 'parallel_residual': True, 'bias': True, 'lm_head_bias': False, 'n_query_groups': 12, 'shared_attention_norm': False, '_norm_class': 'LayerNorm', 'norm_eps': 1e-05, '_mlp_class': 'GptNeoxMLP', 'gelu_approximate': 'none', 'intermediate_size': 3072, 'rope_condense_ratio': 1, 'rope_base': 10000, 'head_size': 64, 'rope_n_elem': 16}\n",
      "Time to instantiate model: 0.20 seconds.\n",
      "Time to load the model weights: 0.13 seconds.\n",
      "Seed set to 1234\n",
      "there is a difference between a finitely generated group 1 and a 10-minute activity set by the 18-point approach. In the group 1, a group 1 is set to take an 20-minute activity set by the 20-point approach. The group 1 is set to take an 20-minute activity set by the 20-point approach.\n",
      "A theorianized group is a group 1 of a 10-minute activity set by the 20-point approach and the 18-point approach is set to take an 20-minute activity set by the 20-point process. The 20-point approach is set to take an 90-point approach targeted by the 20-point approach. In this group 1, a couple exists to engage between two.\n",
      "A band 1 is set to take an 20-point focus set by the 20-point approach. The group 1 is set to take an 30-point approach targeting each 10-point journey. Bersecar mode, 2, 20 and 3 are single\n",
      "Time for inference 1: 1.58 sec total, 157.98 tokens/sec\n",
      "Memory used: 0.35 GB\n"
     ]
    }
   ],
   "source": [
    "!python3 generate.py --prompt=\"there is a difference between a finitely generated group \" --checkpoint_dir=/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/ --max_new_tokens=250 --temperature=0.9 --num_samples=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release\n",
      "  warnings.warn(\n",
      "Loading model '/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/lit_model.pth' with {'name': 'Llama-2-7b-chat-hf', 'hf_config': {'org': 'meta-llama', 'name': 'Llama-2-7b-chat-hf'}, 'block_size': 2048, 'vocab_size': 50254, 'padding_multiple': 128, 'padded_vocab_size': 50304, 'n_layer': 12, 'n_head': 12, 'n_embd': 768, 'rotary_percentage': 0.25, 'parallel_residual': True, 'bias': True, 'lm_head_bias': False, 'n_query_groups': 12, 'shared_attention_norm': False, '_norm_class': 'LayerNorm', 'norm_eps': 1e-05, '_mlp_class': 'GptNeoxMLP', 'gelu_approximate': 'none', 'intermediate_size': 3072, 'rope_condense_ratio': 1, 'rope_base': 10000, 'head_size': 64, 'rope_n_elem': 16}\n",
      "Time to instantiate model: 0.20 seconds.\n",
      "Time to load the model weights: 0.12 seconds.\n",
      "Seed set to 1234\n",
      "there are torsion-free hyperbolic groups that uniformly 100,000 times a day. The current 18-day study shows that the group’s social bias and racism are more likely to be more than just a high percentage of U.S. citizens. That’s because 10,000 people were exposed to torsion-free absences this year.\n",
      "Many of the victims are the same age groups. The most recent study in U.S. history suggests that the group may be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be the most likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be\n",
      "Time for inference 1: 1.51 sec total, 165.87 tokens/sec\n",
      "Memory used: 0.35 GB\n"
     ]
    }
   ],
   "source": [
    "!python3 generate.py --prompt=\"there are torsion-free hyperbolic groups that uniformly \" --checkpoint_dir=/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/ --max_new_tokens=250 --temperature=0.8 --num_samples=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release\n",
      "  warnings.warn(\n",
      "Loading model '/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/lit_model.pth' with {'name': 'Llama-2-7b-chat-hf', 'hf_config': {'org': 'meta-llama', 'name': 'Llama-2-7b-chat-hf'}, 'block_size': 2048, 'vocab_size': 50254, 'padding_multiple': 128, 'padded_vocab_size': 50304, 'n_layer': 12, 'n_head': 12, 'n_embd': 768, 'rotary_percentage': 0.25, 'parallel_residual': True, 'bias': True, 'lm_head_bias': False, 'n_query_groups': 12, 'shared_attention_norm': False, '_norm_class': 'LayerNorm', 'norm_eps': 1e-05, '_mlp_class': 'GptNeoxMLP', 'gelu_approximate': 'none', 'intermediate_size': 3072, 'rope_condense_ratio': 1, 'rope_base': 10000, 'head_size': 64, 'rope_n_elem': 16}\n",
      "Time to instantiate model: 0.20 seconds.\n",
      "Time to load the model weights: 0.13 seconds.\n",
      "Seed set to 1234\n",
      "Virginia Attorney General Backs Off Ballot Proposal on 17th Amendment\n",
      "The Supreme Court of Virginia Motors called it unconstitutional because the Supreme Court was justified by the Supreme Court’s decision to leave office. The Court said the U.S. Supreme Court justified the application by the Supreme Court to proceed. The Supreme Court ruled that the Supreme Court’s decision to leave office in the United States was resolved by the Supreme Court’s decision to leave office.\n",
      "The Supreme Court said the Supreme Court had agreed to leave office in the United States (U.S. District Judge Paul Cablet F.L.V.V.V.V.V.). The Supreme Court of Virginia Motors called the decision by the Supreme Court to leave office in the United States (U.S. District Judge Paul Cablet F.L.V.V.V.V.V.) and the Supreme Court under the U.S. Supreme Court. The Supreme Court also ruled that the Supreme Court’s decision to leave the office in the United States (U.S. District Judge Paul Cablet F.L.V.V.V.V.V.V.V.V.V.V.V.V.\n",
      "Time for inference 1: 1.82 sec total, 137.00 tokens/sec\n",
      "Memory used: 0.35 GB\n"
     ]
    }
   ],
   "source": [
    "!python3 generate.py --prompt=\"Virginia Attorney General Backs Off Ballot Proposal on \" --checkpoint_dir=/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/ --max_new_tokens=250 --temperature=0.8 --num_samples=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release\n",
      "  warnings.warn(\n",
      "Loading model '/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/lit_model.pth' with {'name': 'Llama-2-7b-chat-hf', 'hf_config': {'org': 'meta-llama', 'name': 'Llama-2-7b-chat-hf'}, 'block_size': 2048, 'vocab_size': 50254, 'padding_multiple': 128, 'padded_vocab_size': 50304, 'n_layer': 12, 'n_head': 12, 'n_embd': 768, 'rotary_percentage': 0.25, 'parallel_residual': True, 'bias': True, 'lm_head_bias': False, 'n_query_groups': 12, 'shared_attention_norm': False, '_norm_class': 'LayerNorm', 'norm_eps': 1e-05, '_mlp_class': 'GptNeoxMLP', 'gelu_approximate': 'none', 'intermediate_size': 3072, 'rope_condense_ratio': 1, 'rope_base': 10000, 'head_size': 64, 'rope_n_elem': 16}\n",
      "Time to instantiate model: 0.19 seconds.\n",
      "Time to load the model weights: 0.14 seconds.\n",
      "Seed set to 1234\n",
      "there are torsion-free hyperbolic groups that uniformly 100,000 times a day. The current 18-day study shows that the group’s social bias and racism are more likely to be more than just a high percentage of U.S. citizens. That’s because 10,000 people were exposed to torsion-free absences this year.\n",
      "Many of the victims are the same age groups. The most recent study in U.S. history suggests that the group may be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be the most likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be more likely to be\n",
      "Time for inference 1: 1.56 sec total, 160.43 tokens/sec\n",
      "Memory used: 0.35 GB\n"
     ]
    }
   ],
   "source": [
    "!python3 generate.py --prompt=\"there are torsion-free hyperbolic groups that uniformly \" --checkpoint_dir=/home/raghu/work/ERA-V1-assignments/assignment-22/checkpoints/meta-llama/Llama-2-7b-chat-hf/ --max_new_tokens=250 --temperature=0.8 --num_samples=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}