Added cost and analysis

notebooks/04_get_embeddings.ipynb

@@ -43,7 +43,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "id": "7e873652-8257-4aae-92bc-94e1bac54b73",
    "metadata": {
     "tags": []
@@ -78,7 +78,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 2,
    "id": "52edfc97-5b6f-44f9-8d89-8578cf79fae9",
    "metadata": {
     "tags": []
@@ -105,16 +105,16 @@
    "metadata": {},
    "source": [
     "## Start TEI with Inference Endpoints\n",
-    "Another option is to run TEI on Inference Endpoints. Its cheap and fast. It took me less than 5 minutes to get it up and running!\n",
+    "Another option is to run TEI on [Inference Endpoints](https://huggingface.co/inference-endpoints). Its cheap and fast. It took me less than 5 minutes to get it up and running!\n",
     "\n",
-    "Check here for a [guide](https://huggingface.co/blog/inference-endpoints-embeddings#3-deploy-embedding-model-as-inference-endpoint). Make sure to set these options in order:\n",
-    "1. Model Repository = transformers/paraphrase-multilingual-minilm-l12-v2\n",
+    "Check here for a [comprehensive guide](https://huggingface.co/blog/inference-endpoints-embeddings#3-deploy-embedding-model-as-inference-endpoint). Make sure to set these options **IN ORDER**:\n",
+    "1. Model Repository = `transformers/paraphrase-multilingual-minilm-l12-v2`\n",
     "1. Name your endpoint\n",
-    "1. Choose a GPU\n",
+    "1. Choose a GPU, I chose `Nvidia A10G` which is **$1.3/hr**.\n",
     "1. Advanced Configuration\n",
-    "    1. Task = Sentence Embeddings\n",
-    "    1. Revision (based on [this pull request for safetensors](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2/discussions/8) = a21e6630\n",
-    "    1. Container Type = Text Embeddings Inference\n",
+    "    1. Task = `Sentence Embeddings`\n",
+    "    1. Revision (based on [this pull request for safetensors](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2/discussions/8) = `a21e6630`\n",
+    "    1. Container Type = `Text Embeddings Inference`\n",
     "    \n",
     "Set the other options as you prefer."
    ]
@@ -129,7 +129,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 3,
    "id": "a69e2ee1-67f2-4f0a-b496-02f5415a52ca",
    "metadata": {
     "tags": []
@@ -139,20 +139,20 @@
      "name": "stdin",
      "output_type": "stream",
      "text": [
-      "What is your BEARER TOKEN? Check your endpoint. ········\n",
-      "What is your API_URL? ········\n"
+      "What is your API_URL? ········\n",
+      "What is your BEARER TOKEN? Check your endpoint. ········\n"
      ]
     }
    ],
    "source": [
     "import getpass\n",
-    "bearer_token = getpass.getpass(prompt='What is your BEARER TOKEN? Check your endpoint.')\n",
-    "API_URL = getpass.getpass(prompt='What is your API_URL?')"
+    "API_URL = getpass.getpass(prompt='What is your API_URL?')\n",
+    "bearer_token = getpass.getpass(prompt='What is your BEARER TOKEN? Check your endpoint.')"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 4,
    "id": "949d6bf8-804f-496b-a59a-834483cc7073",
    "metadata": {
     "tags": []
@@ -169,7 +169,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 5,
    "id": "d00b4af1-8fbc-4f7a-8a78-e1c52dd77a66",
    "metadata": {
     "tags": []
@@ -207,7 +207,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 6,
    "id": "abb5186b-ee67-4e1e-882d-3d8d5b4575d4",
    "metadata": {
     "tags": []
@@ -226,7 +226,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 7,
    "id": "c4b82ea2-8b30-4c2e-99f0-9a30f2f1bfb7",
    "metadata": {
     "tags": []
@@ -255,7 +255,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 8,
    "id": "f6f74545-54a7-4f41-9f02-96964e1417f0",
    "metadata": {
     "tags": []
@@ -263,7 +263,8 @@
    "outputs": [],
    "source": [
     "files_in = list((proj_dir / 'data/processed/').glob('*.ndjson'))\n",
-    "folder_out = proj_dir / 'data/embedded/'"
+    "folder_out = proj_dir / 'data/embedded/'\n",
+    "folder_out_str = str(folder_out)"
    ]
   },
   {
@@ -288,7 +289,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 9,
    "id": "e455dd52-aad3-4313-8738-03141ee5152a",
    "metadata": {
     "tags": []
@@ -328,7 +329,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 10,
    "id": "f0d17264-72dc-40be-aa46-17cde38c8189",
    "metadata": {
     "tags": []
@@ -337,7 +338,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "1db8949409284a7cbeec2638ed197f59",
+       "model_id": "c4b7384336ad4c39a417a54a5a00a4ad",
        "version_major": 2,
        "version_minor": 0
       },
@@ -351,7 +352,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "5945500ccf8649988918e2633269cb7b",
+       "model_id": "0b034dc636df440594550f56dc152c8b",
        "version_major": 2,
        "version_minor": 0
       },
@@ -372,7 +373,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "0cf8121a116f49fba72095fee46ef49d",
+       "model_id": "0203531009644b75abb22725a38b3ace",
        "version_major": 2,
        "version_minor": 0
       },
@@ -393,7 +394,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "8f94983077854b5f9ab512f7d429eb55",
+       "model_id": "a4c781089c42466ba380b0b598b2f9e6",
        "version_major": 2,
        "version_minor": 0
       },
@@ -414,7 +415,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "3d2932212e6b4323a377ff23758e7af7",
+       "model_id": "66a0feea106145a0aadeb64fab48b6f8",
        "version_major": 2,
        "version_minor": 0
       },
@@ -435,7 +436,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "3de41d88c8bb439591925de045d8afe8",
+       "model_id": "c6976832a78e48c5be335c5fef14bb5d",
        "version_major": 2,
        "version_minor": 0
       },
@@ -456,7 +457,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "604a4f3b1baf429687ac00aa63778cdf",
+       "model_id": "764ceac837b040a39c2541074386e1f6",
        "version_major": 2,
        "version_minor": 0
       },
@@ -477,7 +478,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "ecc69a2b763c4296b3a1fa35b15477aa",
+       "model_id": "6d268907dd9844cd8f81f48f8568f576",
        "version_major": 2,
        "version_minor": 0
       },
@@ -498,7 +499,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "bbdc03d2ca5a4099b412c1767b3d394c",
+       "model_id": "53bb70b332774c3d867cdb1cb3c48958",
        "version_major": 2,
        "version_minor": 0
       },
@@ -519,7 +520,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "d4c64dfc612c4b7986de5385f5d88ba7",
+       "model_id": "525ae3cf63af47b2acad508cb3c3efb7",
        "version_major": 2,
        "version_minor": 0
       },
@@ -540,7 +541,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "d05e2884021143e0baf595a86725466a",
+       "model_id": "1a4c5103a1184ca1999b452c716131be",
        "version_major": 2,
        "version_minor": 0
       },
@@ -561,7 +562,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "47f3537175d740aba9e2dc7de4c89fec",
+       "model_id": "9a06e2e21c6d4d12a04a55ca746594a4",
        "version_major": 2,
        "version_minor": 0
       },
@@ -582,7 +583,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "02b89207ee8c407db4ad8045b3634243",
+       "model_id": "933d457b2f4f4f1fa3d20b469dc22d75",
        "version_major": 2,
        "version_minor": 0
       },
@@ -603,7 +604,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "e1a96dfd66644007a39a5fef38e008ab",
+       "model_id": "0e4ce5ea591f431ca1ba6497ccf82b84",
        "version_major": 2,
        "version_minor": 0
       },
@@ -624,7 +625,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "3bef95b8dff044fa922e52e1e88b9813",
+       "model_id": "1d267ec29d864694b9f89fbf15e3e34a",
        "version_major": 2,
        "version_minor": 0
       },
@@ -645,7 +646,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "eb74103e549c4b0386d19f5d475af812",
+       "model_id": "bfb1ecea3b2143c1916beb446201fe7f",
        "version_major": 2,
        "version_minor": 0
       },
@@ -666,7 +667,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "41228a5bf8294c1e95f34b9376714543",
+       "model_id": "cdb83413a46e4ba984eb261994d05cd3",
        "version_major": 2,
        "version_minor": 0
       },
@@ -687,7 +688,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "ba6eb3a975514d33ae3acac8859278d1",
+       "model_id": "a6cb8095952f4db3ab6d31219c21087e",
        "version_major": 2,
        "version_minor": 0
       },
@@ -708,7 +709,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "67ecca34d2a1414c9e9817d835fe2083",
+       "model_id": "0699bd10530c4a34aaaf9e88523ad5e6",
        "version_major": 2,
        "version_minor": 0
       },
@@ -729,7 +730,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "7fb632776adb4933b92c48f852b0ae6b",
+       "model_id": "736895c24eb84a8a9f514d99c628bdc7",
        "version_major": 2,
        "version_minor": 0
       },
@@ -750,7 +751,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "17d73c88d4334357854f852c9783bfdb",
+       "model_id": "632c1bfc4370488ab977bedc8c31d404",
        "version_major": 2,
        "version_minor": 0
       },
@@ -771,7 +772,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "4dd7c3477a244d43b1d85417d4549eaa",
+       "model_id": "d2c550797b5f4444b91b954c3f3958b1",
        "version_major": 2,
        "version_minor": 0
       },
@@ -792,7 +793,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "e8b657d57f584128ae5a7ee2ecf23c7f",
+       "model_id": "8918701c109f4ecdbb1d73e5fe97d6b5",
        "version_major": 2,
        "version_minor": 0
       },
@@ -813,7 +814,7 @@
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "f3465378528a425e8dc9d040a003588a",
+       "model_id": "c69c13c3a1354f5c900d268500ffcb00",
        "version_major": 2,
        "version_minor": 0
       },
@@ -829,7 +830,7 @@
      "output_type": "stream",
      "text": [
       "Batch 23: Embeddings = 70322 documents = 70322\n",
-      "6250.827601939993\n"
+      "104 min 32.33 sec\n"
      ]
     }
    ],
@@ -855,39 +856,477 @@
     "        for document in documents:\n",
     "            json_str = json.dumps(document, ensure_ascii=False)\n",
     "            f.write(json_str + '\\n')\n",
-    "print(time.perf_counter() - start)"
+    "            \n",
+    "# Print elapsed time\n",
+    "elapsed_time = time.perf_counter() - start\n",
+    "minutes, seconds = divmod(elapsed_time, 60)\n",
+    "print(f\"{int(minutes)} min {seconds:.2f} sec\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3f0d9e6d-68f2-4086-9bcc-ffb27971fd63",
+   "metadata": {},
+   "source": [
+    "Lets make sure that we still have all our documents:"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "cdee2b1c-0493-4b3e-8ecb-9d79109c756e",
+   "execution_count": 11,
+   "id": "abc6dccc-0e5c-45e2-a269-b9f02cff2d05",
    "metadata": {
     "tags": []
    },
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "/home/ec2-user/arabic-wiki/data/embedded\n",
+      "2094596\n"
+     ]
+    }
+   ],
    "source": [
-    "documents[0]"
+    "!echo \"$folder_out_str\" && cat \"$folder_out_str\"/*.ndjson | wc -l"
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "f90a0ed7-b5e9-4ae4-9e87-4c04875ebcc9",
-   "metadata": {},
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "cdee2b1c-0493-4b3e-8ecb-9d79109c756e",
+   "metadata": {
+    "collapsed": true,
+    "jupyter": {
+     "outputs_hidden": true
+    },
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'content': 'عشاء كارين هو من سلسلة مطاعم استرالية يهدف عمداً عن تجربة تناول طعام غير سارَة ويتم توجيه الموظفين لإهانة العملاء طوال وجباتهم.\\nاقتبس اسم المطعم من المصطلح العامي على الإنترنت (كارين) والذي يستخدم لوصف امرأة بيضاء مسنة وقحة بشكل نمطي.\\nتاريخ المطعم.\\nتم إنشاء السلسلة في أستراليا (سيدني) في عام 2021 من قبل إيدين ليفن وجيمس فاريل. المطعم ذو طابع خاص يعتمد على خدمة تجربة طعام غير سارة حيث يدفع العملاء للموظفين لإهانتهم وكان من المفترض ان يكون المطعم مطعماً منبثقاً لمدة ستة أشهر في وورلد سكوير.\\nاثارت فكرة المطعم في البداية ردات فعل متغايرة مما أثار الخوف بشأن ما إذا كانت الإهانات المتبادلة من الممكن ان تعرض الموظفين لسوء المعاملة من قبل العملاء.\\nاسم (كارين) هو إشارة إلى الإسم المستخدم في الميمات (النكت التي تنشهر بسرعة في مواقع التواصل) لوصف امرأة بيضاء في منتصف العمر ووقحة بشكل نمطي.\\nيطلب من الموظفين ارتداء شخصية وقحة والسخرية من العملاء بشكل هزلي اثناء تناول وجباتهم ومن المتوقع ان يعيد العملاء هذا السلوك من خلال التصرف بوقاحة تجاه الموظفين ومع ذلك يُحظر على العملاء والموظفين استخدام الإهانات العنصرية أو التحيز الجنسي أو رهاب المثلية الجنسية.\\nتتضمن العديد من هذه التبادلات لغة نابية ويجب ان يكون برفقة الاشخاص اللذين يقلون عن 16 عاماََ بالغين.\\nكما يمكن لمالكي بطاقة هوية تظهر ان اسمهم كارين الحصول على مشروب مجاني.\\n',\n",
+       " 'content_type': 'text',\n",
+       " 'score': None,\n",
+       " 'meta': {'id': '8974231',\n",
+       "  'revid': '593870',\n",
+       "  'url': 'https://ar.wikipedia.org/wiki?curid=8974231',\n",
+       "  'title': 'مطعم عشاء كارين',\n",
+       "  '_split_id': 0,\n",
+       "  '_split_overlap': [{'doc_id': '288196225044b53e6ff86f2485257a0a',\n",
+       "    'range': [790, 1225]}]},\n",
+       " 'id_hash_keys': ['content'],\n",
+       " 'embedding': [0.053985596,\n",
+       "  -0.06933594,\n",
+       "  -0.046417236,\n",
+       "  0.07788086,\n",
+       "  -0.06768799,\n",
+       "  -0.01789856,\n",
+       "  0.03265381,\n",
+       "  -0.1060791,\n",
+       "  0.046325684,\n",
+       "  0.00022745132,\n",
+       "  0.01524353,\n",
+       "  0.021408081,\n",
+       "  -0.039398193,\n",
+       "  -0.076049805,\n",
+       "  -0.058380127,\n",
+       "  -0.027786255,\n",
+       "  0.067993164,\n",
+       "  -0.00894928,\n",
+       "  0.11645508,\n",
+       "  0.09039307,\n",
+       "  0.032684326,\n",
+       "  -0.023635864,\n",
+       "  0.00970459,\n",
+       "  0.08312988,\n",
+       "  0.01638794,\n",
+       "  -0.058380127,\n",
+       "  0.0501709,\n",
+       "  0.035583496,\n",
+       "  -0.042297363,\n",
+       "  -0.011688232,\n",
+       "  0.010314941,\n",
+       "  0.026626587,\n",
+       "  0.039978027,\n",
+       "  0.04095459,\n",
+       "  0.039398193,\n",
+       "  -0.029754639,\n",
+       "  0.01360321,\n",
+       "  -0.0015621185,\n",
+       "  -0.009384155,\n",
+       "  0.043151855,\n",
+       "  0.09710693,\n",
+       "  -0.0044937134,\n",
+       "  0.0059928894,\n",
+       "  -0.07293701,\n",
+       "  -0.012710571,\n",
+       "  0.013046265,\n",
+       "  -0.08099365,\n",
+       "  0.015457153,\n",
+       "  -0.037963867,\n",
+       "  -0.016601562,\n",
+       "  -0.09564209,\n",
+       "  0.038238525,\n",
+       "  -0.020751953,\n",
+       "  -0.016494751,\n",
+       "  0.05529785,\n",
+       "  -0.024490356,\n",
+       "  0.011192322,\n",
+       "  0.021087646,\n",
+       "  -0.1274414,\n",
+       "  0.04840088,\n",
+       "  0.068603516,\n",
+       "  -0.026535034,\n",
+       "  0.010772705,\n",
+       "  -0.03778076,\n",
+       "  -0.046447754,\n",
+       "  0.012870789,\n",
+       "  -0.008674622,\n",
+       "  0.062561035,\n",
+       "  -0.06188965,\n",
+       "  -0.05831909,\n",
+       "  -0.05557251,\n",
+       "  0.018417358,\n",
+       "  -0.028656006,\n",
+       "  0.015487671,\n",
+       "  0.0146865845,\n",
+       "  -0.010131836,\n",
+       "  0.06652832,\n",
+       "  -0.09710693,\n",
+       "  -0.012542725,\n",
+       "  0.04815674,\n",
+       "  0.019470215,\n",
+       "  0.0017337799,\n",
+       "  -0.0069770813,\n",
+       "  -0.027648926,\n",
+       "  -0.103149414,\n",
+       "  -0.019058228,\n",
+       "  -0.049926758,\n",
+       "  -0.05307007,\n",
+       "  0.13562012,\n",
+       "  0.016311646,\n",
+       "  -0.10913086,\n",
+       "  -0.04837036,\n",
+       "  0.033996582,\n",
+       "  -0.042144775,\n",
+       "  0.043060303,\n",
+       "  -0.10797119,\n",
+       "  -0.087402344,\n",
+       "  -0.051452637,\n",
+       "  0.018203735,\n",
+       "  0.15771484,\n",
+       "  -0.025131226,\n",
+       "  0.074401855,\n",
+       "  -0.0033817291,\n",
+       "  -0.026138306,\n",
+       "  0.032165527,\n",
+       "  -0.010025024,\n",
+       "  -0.016189575,\n",
+       "  -0.001121521,\n",
+       "  -0.049560547,\n",
+       "  -0.034057617,\n",
+       "  -0.04336548,\n",
+       "  0.11694336,\n",
+       "  -0.035949707,\n",
+       "  -0.061767578,\n",
+       "  -0.101623535,\n",
+       "  0.0051116943,\n",
+       "  0.039245605,\n",
+       "  0.0053977966,\n",
+       "  0.016708374,\n",
+       "  -0.012908936,\n",
+       "  -0.022369385,\n",
+       "  0.03475952,\n",
+       "  0.035583496,\n",
+       "  0.046417236,\n",
+       "  -0.06097412,\n",
+       "  0.07409668,\n",
+       "  -0.07312012,\n",
+       "  -0.019683838,\n",
+       "  -0.07086182,\n",
+       "  -0.03967285,\n",
+       "  -0.016937256,\n",
+       "  -0.02658081,\n",
+       "  0.105651855,\n",
+       "  0.05831909,\n",
+       "  0.03778076,\n",
+       "  -0.07543945,\n",
+       "  0.055267334,\n",
+       "  0.012268066,\n",
+       "  0.07720947,\n",
+       "  -0.068359375,\n",
+       "  -0.011100769,\n",
+       "  -0.0072898865,\n",
+       "  0.051635742,\n",
+       "  0.027954102,\n",
+       "  0.043121338,\n",
+       "  0.032440186,\n",
+       "  0.081604004,\n",
+       "  -0.058807373,\n",
+       "  -0.048706055,\n",
+       "  0.07867432,\n",
+       "  0.014404297,\n",
+       "  -0.0024490356,\n",
+       "  -0.008575439,\n",
+       "  0.010345459,\n",
+       "  -0.013870239,\n",
+       "  -0.019424438,\n",
+       "  0.020889282,\n",
+       "  0.013290405,\n",
+       "  0.012817383,\n",
+       "  0.015930176,\n",
+       "  0.03768921,\n",
+       "  -0.0012264252,\n",
+       "  0.0010223389,\n",
+       "  -0.023483276,\n",
+       "  -0.005252838,\n",
+       "  -0.051574707,\n",
+       "  0.034729004,\n",
+       "  -0.004081726,\n",
+       "  0.0317688,\n",
+       "  0.0087890625,\n",
+       "  0.11047363,\n",
+       "  0.05291748,\n",
+       "  -0.12841797,\n",
+       "  0.031799316,\n",
+       "  -0.09881592,\n",
+       "  0.07299805,\n",
+       "  0.02859497,\n",
+       "  0.024780273,\n",
+       "  0.049316406,\n",
+       "  -0.07122803,\n",
+       "  0.03930664,\n",
+       "  0.012702942,\n",
+       "  0.06915283,\n",
+       "  -0.03967285,\n",
+       "  0.035949707,\n",
+       "  -0.045166016,\n",
+       "  0.09625244,\n",
+       "  0.001203537,\n",
+       "  0.022750854,\n",
+       "  -0.03665161,\n",
+       "  -0.013633728,\n",
+       "  -0.018112183,\n",
+       "  0.054107666,\n",
+       "  -0.007106781,\n",
+       "  0.004924774,\n",
+       "  -0.014953613,\n",
+       "  0.07147217,\n",
+       "  -0.013092041,\n",
+       "  0.016845703,\n",
+       "  -0.002910614,\n",
+       "  -0.05593872,\n",
+       "  0.027404785,\n",
+       "  0.002696991,\n",
+       "  0.05822754,\n",
+       "  0.0066566467,\n",
+       "  -0.09729004,\n",
+       "  0.040100098,\n",
+       "  -0.00868988,\n",
+       "  0.10290527,\n",
+       "  0.04144287,\n",
+       "  -0.012680054,\n",
+       "  0.039215088,\n",
+       "  -0.14074707,\n",
+       "  0.08215332,\n",
+       "  -0.05078125,\n",
+       "  -0.028549194,\n",
+       "  0.011962891,\n",
+       "  0.028900146,\n",
+       "  -0.02444458,\n",
+       "  0.004207611,\n",
+       "  -0.00995636,\n",
+       "  0.028717041,\n",
+       "  -0.08325195,\n",
+       "  -0.047424316,\n",
+       "  0.032043457,\n",
+       "  -0.04675293,\n",
+       "  -0.064575195,\n",
+       "  -0.03857422,\n",
+       "  0.0070266724,\n",
+       "  -0.12634277,\n",
+       "  -0.0803833,\n",
+       "  -0.05419922,\n",
+       "  0.064331055,\n",
+       "  -0.004421234,\n",
+       "  -0.00844574,\n",
+       "  -0.05923462,\n",
+       "  0.052490234,\n",
+       "  0.032592773,\n",
+       "  0.024230957,\n",
+       "  0.075683594,\n",
+       "  0.011390686,\n",
+       "  0.013252258,\n",
+       "  -0.029403687,\n",
+       "  -0.03338623,\n",
+       "  -0.045928955,\n",
+       "  0.015022278,\n",
+       "  -0.08343506,\n",
+       "  0.060180664,\n",
+       "  0.076171875,\n",
+       "  0.058898926,\n",
+       "  0.026184082,\n",
+       "  -0.04031372,\n",
+       "  -0.0847168,\n",
+       "  -0.06628418,\n",
+       "  -0.017974854,\n",
+       "  -0.09967041,\n",
+       "  0.07952881,\n",
+       "  0.012413025,\n",
+       "  0.04006958,\n",
+       "  0.07788086,\n",
+       "  0.014640808,\n",
+       "  -0.029281616,\n",
+       "  -0.04949951,\n",
+       "  0.012565613,\n",
+       "  0.025466919,\n",
+       "  -0.018478394,\n",
+       "  -0.072753906,\n",
+       "  0.08905029,\n",
+       "  0.023849487,\n",
+       "  0.012306213,\n",
+       "  -0.089538574,\n",
+       "  -0.05657959,\n",
+       "  0.11804199,\n",
+       "  -0.035827637,\n",
+       "  0.049194336,\n",
+       "  -0.008911133,\n",
+       "  0.09680176,\n",
+       "  -0.03616333,\n",
+       "  -0.057525635,\n",
+       "  0.03375244,\n",
+       "  -0.027435303,\n",
+       "  0.0035476685,\n",
+       "  0.010269165,\n",
+       "  0.031921387,\n",
+       "  0.0011024475,\n",
+       "  0.045715332,\n",
+       "  -0.026885986,\n",
+       "  0.033935547,\n",
+       "  0.06341553,\n",
+       "  0.019958496,\n",
+       "  0.008239746,\n",
+       "  0.015174866,\n",
+       "  -0.023071289,\n",
+       "  0.0056762695,\n",
+       "  0.064575195,\n",
+       "  0.0042533875,\n",
+       "  -0.05718994,\n",
+       "  0.04486084,\n",
+       "  0.020614624,\n",
+       "  0.01461792,\n",
+       "  -0.09283447,\n",
+       "  0.019592285,\n",
+       "  -0.022644043,\n",
+       "  0.011512756,\n",
+       "  -0.005874634,\n",
+       "  -0.018569946,\n",
+       "  0.006614685,\n",
+       "  0.009269714,\n",
+       "  -0.04296875,\n",
+       "  -0.052856445,\n",
+       "  0.084106445,\n",
+       "  0.0043563843,\n",
+       "  -0.020721436,\n",
+       "  0.029022217,\n",
+       "  0.03982544,\n",
+       "  -0.109436035,\n",
+       "  -0.036071777,\n",
+       "  0.03253174,\n",
+       "  0.011558533,\n",
+       "  -0.10650635,\n",
+       "  0.034454346,\n",
+       "  -0.06951904,\n",
+       "  -0.025817871,\n",
+       "  0.10668945,\n",
+       "  0.010101318,\n",
+       "  -0.070739746,\n",
+       "  0.049621582,\n",
+       "  -0.09057617,\n",
+       "  0.037231445,\n",
+       "  -0.03152466,\n",
+       "  -0.043914795,\n",
+       "  0.07507324,\n",
+       "  0.061645508,\n",
+       "  0.0085372925,\n",
+       "  0.004142761,\n",
+       "  -0.051971436,\n",
+       "  -0.05480957,\n",
+       "  0.0030975342,\n",
+       "  -0.046875,\n",
+       "  -0.039398193,\n",
+       "  0.08782959,\n",
+       "  -0.012550354,\n",
+       "  -0.003955841,\n",
+       "  -0.07775879,\n",
+       "  -0.021133423,\n",
+       "  0.0062713623,\n",
+       "  -0.02255249,\n",
+       "  0.017868042,\n",
+       "  0.049560547,\n",
+       "  0.028121948,\n",
+       "  0.031707764,\n",
+       "  0.041168213,\n",
+       "  0.009559631,\n",
+       "  0.036956787,\n",
+       "  0.008987427,\n",
+       "  0.0024776459,\n",
+       "  -0.003440857,\n",
+       "  -0.0067749023,\n",
+       "  -0.06439209,\n",
+       "  -0.010902405,\n",
+       "  -0.07104492,\n",
+       "  0.006214142,\n",
+       "  -0.06359863,\n",
+       "  0.062316895,\n",
+       "  0.005367279,\n",
+       "  0.015197754,\n",
+       "  -0.043182373,\n",
+       "  0.050933838,\n",
+       "  0.0035800934,\n",
+       "  0.0032138824,\n",
+       "  -0.017974854,\n",
+       "  0.08972168,\n",
+       "  0.011268616,\n",
+       "  0.020477295,\n",
+       "  -0.05050659,\n",
+       "  -0.07232666,\n",
+       "  0.07055664,\n",
+       "  -0.010002136,\n",
+       "  0.11480713,\n",
+       "  0.02130127,\n",
+       "  0.039093018,\n",
+       "  0.009597778,\n",
+       "  -0.0619812,\n",
+       "  -0.016952515],\n",
+       " 'id': '1af84f3b4cc6a9f1018f2f80b4fd3ba7'}"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
    "source": [
-    "Lets double check that we got all the embeddings we expected!"
+    "documents[0]"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "5b78bfa4-d365-4906-a71c-f444eabf6bf8",
-   "metadata": {
-    "tags": []
-   },
+   "id": "93d6ab01-bd3b-479d-918d-2bdb30b00fac",
+   "metadata": {},
    "source": [
-    "Great, we can see that they match.\n",
-    "\n",
-    "Let's write our embeddings to file"
+    "# Performance and Cost Analysis\n",
+    "You can see that we are quite cost effective!\n",
+    "![Cost](../media/arabic-rag-embeddings-cost.png)\n",
+    "Note that the performance is over just the last 30 min window.\n",
+    "Observations:\n",
+    "- We have a througput of `~333/s`\n",
+    "- Our median latency per request is `~50ms`\n",
+    "![Metrics](../media/arabic-rag-embeddings-metrics.png)"
    ]
   },
   {