Added ipynb and another example

Simple_LLaMA_FineTuner.ipynb

@@ -0,0 +1,234 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    },
+    "accelerator": "GPU",
+    "gpuClass": "standard"
+  },
+  "cells": [
+    {
+      "cell_type": "code",
+      "execution_count": 1,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "qe77im_2YudR",
+        "outputId": "9a8f474b-4c29-463e-c36a-25e1e028c4b8"
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stdout",
+          "text": [
+            "Wed Mar 22 03:47:25 2023       \n",
+            "+-----------------------------------------------------------------------------+\n",
+            "| NVIDIA-SMI 525.85.12    Driver Version: 525.85.12    CUDA Version: 12.0     |\n",
+            "|-------------------------------+----------------------+----------------------+\n",
+            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
+            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
+            "|                               |                      |               MIG M. |\n",
+            "|===============================+======================+======================|\n",
+            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
+            "| N/A   43C    P0    26W /  70W |      0MiB / 15360MiB |      0%      Default |\n",
+            "|                               |                      |                  N/A |\n",
+            "+-------------------------------+----------------------+----------------------+\n",
+            "                                                                               \n",
+            "+-----------------------------------------------------------------------------+\n",
+            "| Processes:                                                                  |\n",
+            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
+            "|        ID   ID                                                   Usage      |\n",
+            "|=============================================================================|\n",
+            "|  No running processes found                                                 |\n",
+            "+-----------------------------------------------------------------------------+\n"
+          ]
+        }
+      ],
+      "source": [
+        "!nvidia-smi"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "![[ -d /content/simple-llama-finetuner ]] \\\n",
+        "  || git clone https://github.com/lxe/simple-llama-finetuner.git /content/simple-llama-finetuner\n",
+        "!cd /content/simple-llama-finetuner && git pull && pip install -r requirements.txt"
+      ],
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "3PM_DilAZD8T",
+        "outputId": "83c6ff7e-518f-4ceb-ac9d-df22660f5ce5"
+      },
+      "execution_count": 2,
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stdout",
+          "text": [
+            "Already up to date.\n",
+            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
+            "Collecting git+https://github.com/huggingface/transformers.git (from -r requirements.txt (line 4))\n",
+            "  Cloning https://github.com/huggingface/transformers.git to /tmp/pip-req-build-uhmp_y8i\n",
+            "  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers.git /tmp/pip-req-build-uhmp_y8i\n",
+            "  Resolved https://github.com/huggingface/transformers.git to commit 0dcb46e7a4a9e587ba84ff35778ab4233a184c11\n",
+            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
+            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
+            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
+            "Collecting git+https://github.com/huggingface/peft.git (from -r requirements.txt (line 7))\n",
+            "  Cloning https://github.com/huggingface/peft.git to /tmp/pip-req-build-ieyxkty1\n",
+            "  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/peft.git /tmp/pip-req-build-ieyxkty1\n",
+            "  Resolved https://github.com/huggingface/peft.git to commit 13e53fc7ee5d89d59b16523051006dddf0fb7a49\n",
+            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
+            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
+            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
+            "Requirement already satisfied: datasets in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 1)) (2.10.1)\n",
+            "Requirement already satisfied: loralib in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 2)) (0.1.1)\n",
+            "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 3)) (0.1.97)\n",
+            "Requirement already satisfied: accelerate in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 5)) (0.17.1)\n",
+            "Requirement already satisfied: bitsandbytes in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 6)) (0.37.2)\n",
+            "Requirement already satisfied: gradio in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 8)) (3.23.0)\n",
+            "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (4.65.0)\n",
+            "Requirement already satisfied: xxhash in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (3.2.0)\n",
+            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (3.8.4)\n",
+            "Requirement already satisfied: packaging in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (23.0)\n",
+            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (1.22.4)\n",
+            "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (2.27.1)\n",
+            "Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (2023.3.0)\n",
+            "Requirement already satisfied: pandas in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (1.4.4)\n",
+            "Requirement already satisfied: huggingface-hub<1.0.0,>=0.2.0 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (0.13.3)\n",
+            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (6.0)\n",
+            "Requirement already satisfied: multiprocess in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (0.70.14)\n",
+            "Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (9.0.0)\n",
+            "Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (0.18.0)\n",
+            "Requirement already satisfied: dill<0.3.7,>=0.3.0 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 1)) (0.3.6)\n",
+            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers==4.28.0.dev0->-r requirements.txt (line 4)) (2022.10.31)\n",
+            "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.9/dist-packages (from transformers==4.28.0.dev0->-r requirements.txt (line 4)) (0.13.2)\n",
+            "Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from transformers==4.28.0.dev0->-r requirements.txt (line 4)) (3.10.0)\n",
+            "Requirement already satisfied: psutil in /usr/local/lib/python3.9/dist-packages (from accelerate->-r requirements.txt (line 5)) (5.9.4)\n",
+            "Requirement already satisfied: torch>=1.4.0 in /usr/local/lib/python3.9/dist-packages (from accelerate->-r requirements.txt (line 5)) (1.13.1+cu116)\n",
+            "Requirement already satisfied: orjson in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (3.8.8)\n",
+            "Requirement already satisfied: markupsafe in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (2.1.2)\n",
+            "Requirement already satisfied: fastapi in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (0.95.0)\n",
+            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (3.1.2)\n",
+            "Requirement already satisfied: pydub in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (0.25.1)\n",
+            "Requirement already satisfied: mdit-py-plugins<=0.3.3 in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (0.3.3)\n",
+            "Requirement already satisfied: uvicorn in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (0.21.1)\n",
+            "Requirement already satisfied: pillow in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (8.4.0)\n",
+            "Requirement already satisfied: semantic-version in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (2.10.0)\n",
+            "Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (2.2.0)\n",
+            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (3.7.1)\n",
+            "Requirement already satisfied: websockets>=10.0 in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (10.4)\n",
+            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (4.5.0)\n",
+            "Requirement already satisfied: pydantic in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (1.10.6)\n",
+            "Requirement already satisfied: aiofiles in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (23.1.0)\n",
+            "Requirement already satisfied: ffmpy in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (0.3.0)\n",
+            "Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (4.2.2)\n",
+            "Requirement already satisfied: httpx in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (0.23.3)\n",
+            "Requirement already satisfied: python-multipart in /usr/local/lib/python3.9/dist-packages (from gradio->-r requirements.txt (line 8)) (0.0.6)\n",
+            "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.9/dist-packages (from altair>=4.2.0->gradio->-r requirements.txt (line 8)) (4.3.3)\n",
+            "Requirement already satisfied: entrypoints in /usr/local/lib/python3.9/dist-packages (from altair>=4.2.0->gradio->-r requirements.txt (line 8)) (0.4)\n",
+            "Requirement already satisfied: toolz in /usr/local/lib/python3.9/dist-packages (from altair>=4.2.0->gradio->-r requirements.txt (line 8)) (0.12.0)\n",
+            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets->-r requirements.txt (line 1)) (6.0.4)\n",
+            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets->-r requirements.txt (line 1)) (1.8.2)\n",
+            "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets->-r requirements.txt (line 1)) (2.0.12)\n",
+            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets->-r requirements.txt (line 1)) (1.3.3)\n",
+            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets->-r requirements.txt (line 1)) (22.2.0)\n",
+            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets->-r requirements.txt (line 1)) (1.3.1)\n",
+            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets->-r requirements.txt (line 1)) (4.0.2)\n",
+            "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.9/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio->-r requirements.txt (line 8)) (0.1.2)\n",
+            "Requirement already satisfied: linkify-it-py<3,>=1 in /usr/local/lib/python3.9/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio->-r requirements.txt (line 8)) (2.0.0)\n",
+            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas->datasets->-r requirements.txt (line 1)) (2022.7.1)\n",
+            "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas->datasets->-r requirements.txt (line 1)) (2.8.2)\n",
+            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->datasets->-r requirements.txt (line 1)) (3.4)\n",
+            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->datasets->-r requirements.txt (line 1)) (1.26.15)\n",
+            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->datasets->-r requirements.txt (line 1)) (2022.12.7)\n",
+            "Requirement already satisfied: starlette<0.27.0,>=0.26.1 in /usr/local/lib/python3.9/dist-packages (from fastapi->gradio->-r requirements.txt (line 8)) (0.26.1)\n",
+            "Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in /usr/local/lib/python3.9/dist-packages (from httpx->gradio->-r requirements.txt (line 8)) (1.5.0)\n",
+            "Requirement already satisfied: sniffio in /usr/local/lib/python3.9/dist-packages (from httpx->gradio->-r requirements.txt (line 8)) (1.3.0)\n",
+            "Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in /usr/local/lib/python3.9/dist-packages (from httpx->gradio->-r requirements.txt (line 8)) (0.16.3)\n",
+            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio->-r requirements.txt (line 8)) (3.0.9)\n",
+            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio->-r requirements.txt (line 8)) (0.11.0)\n",
+            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio->-r requirements.txt (line 8)) (1.0.7)\n",
+            "Requirement already satisfied: importlib-resources>=3.2.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio->-r requirements.txt (line 8)) (5.12.0)\n",
+            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio->-r requirements.txt (line 8)) (1.4.4)\n",
+            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio->-r requirements.txt (line 8)) (4.39.2)\n",
+            "Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.9/dist-packages (from uvicorn->gradio->-r requirements.txt (line 8)) (0.14.0)\n",
+            "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.9/dist-packages (from uvicorn->gradio->-r requirements.txt (line 8)) (8.1.3)\n",
+            "Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.9/dist-packages (from httpcore<0.17.0,>=0.15.0->httpx->gradio->-r requirements.txt (line 8)) (3.6.2)\n",
+            "Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.9/dist-packages (from importlib-resources>=3.2.0->matplotlib->gradio->-r requirements.txt (line 8)) (3.15.0)\n",
+            "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.9/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio->-r requirements.txt (line 8)) (0.19.3)\n",
+            "Requirement already satisfied: uc-micro-py in /usr/local/lib/python3.9/dist-packages (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio->-r requirements.txt (line 8)) (1.0.1)\n",
+            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/dist-packages (from python-dateutil>=2.8.1->pandas->datasets->-r requirements.txt (line 1)) (1.16.0)\n"
+          ]
+        }
+      ]
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "!cd /content/simple-llama-finetuner && python main.py --share"
+      ],
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "BD693wIzZKUK",
+        "outputId": "a392bff4-9a5b-4c8f-ecd1-6751517cd254"
+      },
+      "execution_count": null,
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stdout",
+          "text": [
+            "\n",
+            "===================================BUG REPORT===================================\n",
+            "Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues\n",
+            "================================================================================\n",
+            "/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /usr/lib64-nvidia did not contain libcudart.so as expected! Searching further paths...\n",
+            "  warn(msg)\n",
+            "/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/sys/fs/cgroup/memory.events /var/colab/cgroup/jupyter-children/memory.events')}\n",
+            "  warn(msg)\n",
+            "/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('--listen_host=172.28.0.12 --target_host=172.28.0.12 --tunnel_background_save_url=https'), PosixPath('//colab.research.google.com/tun/m/cc48301118ce562b961b3c22d803539adc1e0c19/gpu-t4-s-ntdfs4nb9znz --tunnel_background_save_delay=10s --tunnel_periodic_background_save_frequency=30m0s --enable_output_coalescing=true --output_coalescing_required=true')}\n",
+            "  warn(msg)\n",
+            "/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/env/python')}\n",
+            "  warn(msg)\n",
+            "/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//ipykernel.pylab.backend_inline'), PosixPath('module')}\n",
+            "  warn(msg)\n",
+            "CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...\n",
+            "CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so\n",
+            "CUDA SETUP: Highest compute capability among GPUs detected: 7.5\n",
+            "CUDA SETUP: Detected CUDA version 118\n",
+            "CUDA SETUP: Loading binary /usr/local/lib/python3.9/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so...\n",
+            "Running on local URL:  http://127.0.0.1:7860\n",
+            "Running on public URL: https://359c9c250f70a2b979.gradio.live\n",
+            "\n",
+            "This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces\n"
+          ]
+        }
+      ]
+    },
+    {
+      "cell_type": "code",
+      "source": [],
+      "metadata": {
+        "id": "yhKSDrkKbYkG"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

maya-example.txt

@@ -0,0 +1,51 @@
+The Maya civilization (/ˈmaɪə/) of the Mesoamerican people is known by its ancient temples and glyphs. Its Maya script is the most sophisticated and highly developed writing system in the pre-Columbian Americas. It is also noted for its art, architecture, mathematics, calendar, and astronomical system.
+
+
+The Maya civilization developed in the Maya Region, an area that today comprises southeastern Mexico, all of Guatemala and Belize, and the western portions of Honduras and El Salvador. 
+
+
+It includes the northern lowlands of the Yucatán Peninsula and the highlands of the Sierra Madre, the Mexican state of Chiapas, southern Guatemala, El Salvador, and the southern lowlands of the Pacific littoral plain. 
+
+
+Today, their descendants, known collectively as the Maya, number well over 6 million individuals, speak more than twenty-eight surviving Mayan languages, and reside in nearly the same area as their ancestors.
+
+
+The Archaic period, before 2000 BC, saw the first developments in agriculture and the earliest villages. 
+
+
+The Preclassic period (c. 2000 BC to 250 AD) saw the establishment of the first complex societies in the Maya region, and the cultivation of the staple crops of the Maya diet, including maize, beans, squashes, and chili peppers. 
+
+
+The first Maya cities developed around 750 BC, and by 500 BC these cities possessed monumental architecture, including large temples with elaborate stucco façades. 
+
+
+Hieroglyphic writing was being used in the Maya region by the 3rd century BC. In the Late Preclassic a number of large cities developed in the Petén Basin, and the city of Kaminaljuyu rose to prominence in the Guatemalan Highlands. 
+
+
+Beginning around 250 AD, the Classic period is largely defined as when the Maya were raising sculpted monuments with Long Count dates. This period saw the Maya civilization develop many city-states linked by a complex trade network. In the Maya Lowlands two great rivals, the cities of Tikal and Calakmul, became powerful. 
+
+
+The Classic period also saw the intrusive intervention of the central Mexican city of Teotihuacan in Maya dynastic politics. In the 9th century, there was a widespread political collapse in the central Maya region, resulting in internecine warfare, the abandonment of cities, and a northward shift of population. 
+
+
+The Postclassic period saw the rise of Chichen Itza in the north, and the expansion of the aggressive Kʼicheʼ kingdom in the Guatemalan Highlands. In the 16th century, the Spanish Empire colonised the Mesoamerican region, and a lengthy series of campaigns saw the fall of Nojpetén, the last Maya city, in 1697.
+
+Rule during the Classic period centred on the concept of the "divine king", who was thought to act as a mediator between mortals and the supernatural realm. Kingship was patrilineal, and power normally passed to the eldest son. A prospective king was expected to be a successful war leader as well as a ruler. 
+
+
+Closed patronage systems were the dominant force in Maya politics, although how patronage affected the political makeup of a kingdom varied from city-state to city-state. By the Late Classic period, the aristocracy had grown in size, reducing the previously exclusive power of the king. 
+
+
+The Maya developed sophisticated art forms using both perishable and non-perishable materials, including wood, jade, obsidian, ceramics, sculpted stone monuments, stucco, and finely painted murals.
+
+
+Maya cities tended to expand organically. The city centers comprised ceremonial and administrative complexes, surrounded by an irregularly shaped sprawl of residential districts. Different parts of a city were often linked by causeways. 
+
+
+Architecturally, city buildings included palaces, pyramid-temples, ceremonial ballcourts, and structures specially aligned for astronomical observation. The Maya elite were literate, and developed a complex system of hieroglyphic writing. 
+
+
+Theirs was the most advanced writing system in the pre-Columbian Americas. The Maya recorded their history and ritual knowledge in screenfold books, of which only three uncontested examples remain, the rest having been destroyed by the Spanish. In addition, a great many examples of Maya texts can be found on stelae and ceramics. 
+
+
+The Maya developed a highly complex series of interlocking ritual calendars, and employed mathematics that included one of the earliest known instances of the explicit zero in human history. As a part of their religion, the Maya practised human sacrifice.