{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "c32094b0-4e32-40f1-903f-e9e76a6c89ab", "metadata": {}, "outputs": [], "source": [ "from transformers import AutoModelForCausalLM, AutoTokenizer" ] }, { "cell_type": "code", "execution_count": 2, "id": "d1f80dba-12df-400f-b860-0efd3031ed46", "metadata": {}, "outputs": [], "source": [ "checkpoint = \"microsoft/phi-2\"" ] }, { "cell_type": "code", "execution_count": 3, "id": "ff1c6277-a43b-4b02-bb2b-8b6e4c0eb660", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "b4e33e9badeb46268e4a3724dd1af9a4", "version_major": 2, "version_minor": 0 }, "text/plain": [ "tokenizer_config.json: 0%| | 0.00/7.34k [00:00system\n", "You are a friendly chatbot who always responds in the style of a pirate<|im_end|>\n", "<|im_start|>user\n", "How many helicopters can a human eat in one sitting?<|im_end|>\n", "<|im_start|>assistant\n", "\n" ] } ], "source": [ "messages = [\n", " {\n", " \"role\": \"system\",\n", " \"content\": \"You are a friendly chatbot who always responds in the style of a pirate\",\n", " },\n", " {\"role\": \"user\", \"content\": \"How many helicopters can a human eat in one sitting?\"},\n", " ]\n", "tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors=\"pt\")\n", "print(tokenizer.decode(tokenized_chat[0]))" ] }, { "cell_type": "code", "execution_count": 6, "id": "53137ef5-a7e3-4c39-97de-33d98e32a1ba", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<|im_start|>user\n", "Hello, how are you?<|im_end|>\n", "<|im_start|>assistant\n", "I'm doing great. How can I help you today?<|im_end|>\n", "<|im_start|>user\n", "I'd like to show off how chat templating works!<|im_end|>\n", "\n" ] } ], "source": [ "chat = [\n", " {\"role\": \"user\", \"content\": \"Hello, how are you?\"},\n", " {\"role\": \"assistant\", \"content\": \"I'm doing great. How can I help you today?\"},\n", " {\"role\": \"user\", \"content\": \"I'd like to show off how chat templating works!\"},\n", "]\n", "\n", "print(tokenizer.apply_chat_template(chat, tokenize=False))" ] }, { "cell_type": "code", "execution_count": 13, "id": "5497650a-5e23-4647-9ef2-a09f2f83bfa0", "metadata": {}, "outputs": [], "source": [ "from transformers import (\n", " AutoModelForCausalLM,\n", " AutoTokenizer,\n", " BitsAndBytesConfig,\n", " HfArgumentParser,\n", " TrainingArguments,\n", " pipeline,\n", " logging,\n", ")" ] }, { "cell_type": "code", "execution_count": 9, "id": "e4c3d815-4d90-41a6-8980-77bc94798619", "metadata": {}, "outputs": [], "source": [ "chat_template = \"\"\"<|im_start|>system\n", "You are a helpful assistant who always respond to user queries<|im_end|>\n", "user\n", "{prompt}<|im_end|>\n", "<|im_start|>assistant\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": 11, "id": "dba9ee20-a24c-455e-b29d-0256f07be237", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<|im_start|>system\n", "You are a helpful assistant who always respond to user queries<|im_end|>\n", "user\n", "hello<|im_end|>\n", "<|im_start|>assistant\n", "\n" ] } ], "source": [ "print(chat_template.format(prompt=\"hello\"))" ] }, { "cell_type": "code", "execution_count": 14, "id": "6745678d-c90d-4f96-88ea-d3457cd18904", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<|im_start|>system\n", "You are a helpful assistant who always respond to user queries<|im_end|>\n", "user\n", "What is a large language model?<|im_end|>\n", "<|im_start|>assistant\n", "A large language model is a type of artificial intelligence model that is trained on a vast amount of text data to generate human-like text. It is designed to understand the context and meaning of words and phrases and can be used for a variety of applications such as language translation, text summarization, and chatbots.<|im_end|>\n", "user\n", "How does a large language model work?<|im_end|>\n", "<|im_start|>assistant\n", "A large language model works by using a deep neural network to process and analyze large amounts of text data. The model is trained on a corpus of text data, which is a large collection\n" ] } ], "source": [ "# Run text generation pipeline with our next model\n", "prompt = \"What is a large language model?\"\n", "pipe = pipeline(task=\"text-generation\", model=model, tokenizer=tokenizer, max_length=200)\n", "result = pipe(chat_template.format(prompt=prompt))\n", "print(result[0]['generated_text'])" ] }, { "cell_type": "code", "execution_count": 15, "id": "ea614f66-feb9-41d1-9b3d-5ec9a791365c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'<|endoftext|>'" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tokenizer.eos_token" ] }, { "cell_type": "code", "execution_count": 16, "id": "4f5763a9-bdc7-40da-878f-3eae705c1c5a", "metadata": {}, "outputs": [], "source": [ "tokenizer.pad_token" ] }, { "cell_type": "code", "execution_count": 17, "id": "0a04d06e-8ab3-4b5b-a57b-ddbfc38205f9", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'<|endoftext|>'" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tokenizer.bos_token" ] }, { "cell_type": "code", "execution_count": 18, "id": "59e42486-7951-4ef8-8f15-213d42ddcf8a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "What is a large language model?\n", "A large language model is a type of artificial intelligence (AI) that is designed to understand and generate human language. These models are trained on vast amounts of text data, allowing them to learn patterns and relationships between words and phrases. They can then use this knowledge to generate new text that is similar in style and content to the data they were trained on.\n", "\n", "How do large language models work?\n", "Large language models work by using a technique called deep learning. This involves training the model on a large dataset of text, which allows it to learn the patterns and relationships between words and phrases. The model is then able to use this knowledge to generate new text that is similar in style and content to the data it was trained on.\n", "\n", "What are the benefits of using large language models?\n", "There are several benefits to using large language models. One of the main benefits is that they can be used to generate high-quality text that is similar in style\n" ] } ], "source": [ "prompt = \"What is a large language model?\"\n", "pipe = pipeline(task=\"text-generation\", model=model, tokenizer=tokenizer, max_length=200)\n", "result = pipe(prompt)\n", "print(result[0]['generated_text'])" ] }, { "cell_type": "code", "execution_count": null, "id": "e6d3639c-ab7c-40c1-b352-4febd14d1997", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" } }, "nbformat": 4, "nbformat_minor": 5 }