Upload baseline.ipynb with huggingface_hub

baseline.ipynb

@@ -47,23 +47,6 @@
     "login()"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "6fc55725-216f-45dd-9c6d-dae77e16d606",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from huggingface_hub import HfApi\n",
-    "api = HfApi()\n",
-    "api.upload_file(\n",
-    "    path_or_fileobj=\"baseline.ipynb\",\n",
-    "    path_in_repo=\"baseline.ipynb\",\n",
-    "    repo_id=\"arabic-translation-prompt-engineering/atpe-notebooks\",\n",
-    "    repo_type=\"dataset\",\n",
-    ")"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "4c254d6f-f3a1-49c1-815e-36e41e75ca25",
@@ -108,14 +91,6 @@
     "For our baseline we will translate with a simple system prompt and instruction."
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "6cea05d6-afb7-4829-b130-d4bcfe549acb",
-   "metadata": {},
-   "source": [
-    "### Analysis"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "a98b9b67-e68b-43b2-b8e9-0ed1cf85591f",
@@ -129,14 +104,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 69,
    "id": "032c86d2-868e-4fa6-b03e-58f1c41434cc",
    "metadata": {
     "tags": []
    },
    "outputs": [],
    "source": [
-    "system_prompt = \"\"\"You are a skilled translator with extensive experience in English to Arabic translations. You possess a deep understanding of the linguistic, cultural, and contextual nuances essential for accurate and effective translation between these languages. Highly motivated and detail-oriented, you are committed to delivering translations that maintain the integrity and intent of the original text. Your role is crucial in ensuring clear and precise communication in our multilingual system.\"\"\""
+    "system_prompt = \"\"\"You are a skilled translator with extensive experience in English and Arabic translations. You possess a deep understanding of the linguistic, cultural, and contextual nuances essential for accurate and effective translation between these languages. Highly motivated and detail-oriented, you are committed to delivering translations that maintain the integrity and intent of the original text. Your role is crucial in ensuring clear and precise communication in our multilingual system.\"\"\""
    ]
   },
   {
@@ -144,69 +119,195 @@
    "id": "803ddeba-03de-4f13-95d1-5fb097058cf2",
    "metadata": {},
    "source": [
-    "### Prompt\n",
-    "> Translate this from english to arabic: {en_input}.\n",
+    "### Instruction\n",
+    "> Translate this from arabic to english: {translation_input}.\n",
     ">\n",
     "> Translation: \n",
     "\n",
-    "Again we use a simple prompt to get a translation."
+    "We will use a simple instruction to get a translation."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 53,
+   "execution_count": 70,
    "id": "b7f1722c-c484-4e22-a025-53f95943fc76",
    "metadata": {},
    "outputs": [],
    "source": [
-    "def baseline_chat_completion(system_prompt, en_input):\n",
+    "def baseline_chat_completion(system_prompt, translation_input):\n",
     "    \"\"\"\n",
     "    Generates a completion for a chat conversation using a specified system prompt and a user input.\n",
     "    \"\"\"\n",
     "    messages = [\n",
     "        {\"role\": \"system\", \"content\": system_prompt},\n",
-    "        {\"role\": \"user\", \"content\": f\"Translate this from english to arabic: {en_input}.\\nTranslation: \"},\n",
+    "        {\n",
+    "            \"role\": \"user\",\n",
+    "            \"content\": f\"Translate this from arabic to english: {translation_input}.\\nTranslation: \",\n",
+    "        },\n",
     "    ]\n",
     "    return client.chat_completion(messages, max_tokens=10_000)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 50,
+   "execution_count": 74,
    "id": "96a0ba0b-be47-4eb0-bbc2-c82b0ea1b72e",
    "metadata": {
     "tags": []
    },
+   "outputs": [],
+   "source": [
+    "translation_input = (\n",
+    "    \"'يرفرف مثل الفراشة ويلسع كالنحلة - يديه لا يمكن أن تصيب ما لا تستطيع عينيه رؤيته.'\"\n",
+    ")\n",
+    "response = baseline_chat_completion(\n",
+    "    system_prompt,\n",
+    "    translation_input,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2bca574c-461d-4822-b0dd-b12a3b9846b3",
+   "metadata": {},
+   "source": [
+    "### Token Cost\n",
+    "Here we can see that the cost is quite cheap, only 92 tokens!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 75,
+   "id": "4e305b1e-56e0-44da-8c17-496cbcc35fad",
+   "metadata": {
+    "tags": []
+   },
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "120"
+       "129"
       ]
      },
-     "execution_count": 50,
+     "execution_count": 75,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "en_input = \"Float like a butterfly sting like a bee – his hands can’t hit what his eyes can’t see.\"\n",
-    "response = baseline_chat_completion(system_prompt, \"Float like a butterfly sting like a bee – his hands can’t hit what his eyes can’t see.\")"
+    "response.usage.prompt_tokens"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 77,
+   "id": "ef24fe6b-d801-4f3e-95ad-cb7f67247bc3",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\"He floats like a butterfly and stings like a bee - his hands can't hit what his eyes can't see.\"\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(response.choices[0].message.content)"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "2bca574c-461d-4822-b0dd-b12a3b9846b3",
+   "id": "3a9cdf02-d590-4bcf-a7a8-e6b7817ba715",
    "metadata": {},
    "source": [
-    "### Token Cost\n",
-    "Here we can see that the cost is quite cheap, only 92 tokens!"
+    "## Purpose Driven Translation\n",
+    "\n",
+    "[Optimizing Machine Translation through Prompt Engineering](https://arxiv.org/pdf/2308.01391) has done some good exploratory work in examining how prompt engineering can impact translation. They were working between Japanese and English and showed that translations influenced by prompts tailored to **specific purposes** and **target audiences** generally adhered more closely to the translation specifications, suggesting that such prompted translations could be more culturally and contextually appropriate than standard machine translations.\n",
+    "\n",
+    "### System Prompt\n",
+    "We will use the same system prompt as before as it is versatile.\n",
+    "\n",
+    "### Prompt\n",
+    "One of the approaches in the paper was to include the purpose and target audience specification. This was motivated by the author’s experience as a professional translator, leading to the conclusion that these two parameters are essential even in everyday translation work. You can find the prompt below adapted for Arabic to English:\n",
+    "\n",
+    "> Translate the following Arabic [source text] into English. Please fulfill the following conditions when translating.    \n",
+    "> Purpose of the translation: `<Manual description>`  \n",
+    "> Target audience: `<Manual description>`  \n",
+    "> [source text] `{en_input}`  \n",
+    "\n",
+    "You can see that we need to provide the Purpose and the Target Audience for each translation, lets go ahead and do that."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 52,
-   "id": "4e305b1e-56e0-44da-8c17-496cbcc35fad",
+   "execution_count": 78,
+   "id": "4714f6a2-fd0b-48ee-80bc-860f40ee2baa",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "dataset_to_purpose_target = {\n",
+    "    \"ELRC-24ss\": {\n",
+    "        \"purpose\": \"Enhancing understanding and knowledge about COVID-19 and health-related topics.\",\n",
+    "        \"target\": \"Individuals seeking reliable and comprehensible information about COVID-19 and related health topics.\",\n",
+    "    },\n",
+    "    \"GNOME-25ss\": {\"purpose\": \"\", \"target\": \"\"},\n",
+    "    \"HPLT-25ss\": {\"purpose\": \"\", \"target\": \"\"},\n",
+    "    \"OpenSubtitles-25ss\": {\"purpose\": \"\", \"target\": \"\"},\n",
+    "    \"TED2020-25ss\": {\"purpose\": \"\", \"target\": \"\"},\n",
+    "    \"UNPC-24ss\": {\"purpose\": \"\", \"target\": \"\"},\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 79,
+   "id": "f865e9f8-7c63-4e72-b539-0d5916eda44f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def purpose_driven_chat_completion(system_prompt, en_input, dataset):\n",
+    "    \"\"\"\n",
+    "    Generates a completion for a chat conversation using a specified system prompt and a user input.\n",
+    "    \"\"\"\n",
+    "\n",
+    "    prompt = f\"\"\"Translate the following Arabic [source text] into English. Please fulfill the following conditions when translating.\n",
+    "Purpose of the translation: {dataset_to_purpose_target[dataset]['purpose']}\n",
+    "Target audience: {dataset_to_purpose_target[dataset]['target']}\n",
+    "[source text] `{translation_input}`\n",
+    "\"\"\"\n",
+    "\n",
+    "    messages = [\n",
+    "        {\"role\": \"system\", \"content\": system_prompt},\n",
+    "        {\n",
+    "            \"role\": \"user\",\n",
+    "            \"content\": prompt,\n",
+    "        },\n",
+    "    ]\n",
+    "    return client.chat_completion(messages, max_tokens=10_000)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 83,
+   "id": "4115515d-cbcd-405a-b2e0-a805880a40c4",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "ar_input = \"«لاحظنا أنه عندما تعمل مجموعات من أصحاب المصالح على تحديد.. الرؤى، فإن هذا يؤدي إلى مناقشة ما إذا كان ينبغي التأكيد على صحة النظام البيئي أو رفاهية الإنسان.. سواء كانت الأولوية هي النظم البيئية أو أن الأشخاص يؤثرون بدرجة كبيرة على تقييم أصحاب المصالح للحالات البيئية والاجتماعية المرغوبة»، وعلى سبيل المثال، «تُعتبر الذئاب بالنسبة للبعض شيئاً جوهرياً لصحة النظام البيئي وجزءاً أساسياً من الطبيعة، وأما بالنسبة للبعض الآخر فهي رمز تجاوز الحكومة الذي يهدد سبل عيشهم وقيمهم الثقافية».\"\n",
+    "response = purpose_driven_chat_completion(system_prompt, translation_input, \"ELRC-24ss\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 84,
+   "id": "6296d255-d11d-4df7-aa0e-1226ef3d963a",
    "metadata": {
     "tags": []
    },
@@ -214,10 +315,10 @@
     {
      "data": {
       "text/plain": [
-       "120"
+       "186"
       ]
      },
-     "execution_count": 52,
+     "execution_count": 84,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -228,8 +329,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 51,
-   "id": "ef24fe6b-d801-4f3e-95ad-cb7f67247bc3",
+   "execution_count": 85,
+   "id": "1f1c6dd0-11bf-4b88-9029-8bce1e7bcb1c",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\"He floats like a butterfly and stings like a bee - his hands can't hit what his eyes can't see.\"\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(response.choices[0].message.content)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 86,
+   "id": "513ec227-fdc1-441b-a73b-9603925d038c",
    "metadata": {
     "tags": []
    },
@@ -237,35 +358,56 @@
     {
      "data": {
       "text/plain": [
-       "'يرفرف مثل الفراشة ويلسع كالنحلة - يديه لا يمكن أن تصيب ما لا تستطيع عينيه رؤيته.'"
+       "ChatCompletionOutput(choices=[ChatCompletionOutputComplete(finish_reason='eos_token', index=0, message=ChatCompletionOutputMessage(role='assistant', content='\"He floats like a butterfly and stings like a bee - his hands can\\'t hit what his eyes can\\'t see.\"', name=None, tool_calls=None), logprobs=None)], created=1719034916, id='', model='CohereForAI/c4ai-command-r-plus', object='text_completion', system_fingerprint='2.0.4-sha-f426a33', usage=ChatCompletionOutputUsage(completion_tokens=26, prompt_tokens=186, total_tokens=212))"
       ]
      },
-     "execution_count": 51,
+     "execution_count": 86,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "response.choices[0].message.content"
+    "response"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "3a9cdf02-d590-4bcf-a7a8-e6b7817ba715",
+   "id": "1ec3b20b-8393-4fda-a51d-cf67984cc166",
    "metadata": {},
    "source": [
-    "## Purpose Driven Translation\n",
-    "\n",
-    "[](https://arxiv.org/pdf/2308.01391)"
+    "# Push to the hub"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "12e57d07-9e86-426b-be42-e932699d1fe2",
-   "metadata": {},
-   "outputs": [],
-   "source": []
+   "execution_count": 57,
+   "id": "6fc55725-216f-45dd-9c6d-dae77e16d606",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "CommitInfo(commit_url='https://huggingface.co/arabic-translation-prompt-engineering/atpe-notebooks/commit/0dd63b0f4dbe2fd59ced328d04a4749cadfceac9', commit_message='Upload requirements.txt with huggingface_hub', commit_description='', oid='0dd63b0f4dbe2fd59ced328d04a4749cadfceac9', pr_url=None, pr_revision=None, pr_num=None)"
+      ]
+     },
+     "execution_count": 57,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from huggingface_hub import HfApi\n",
+    "\n",
+    "api = HfApi()\n",
+    "api.upload_file(\n",
+    "    path_or_fileobj=\"baseline.ipynb\",\n",
+    "    path_in_repo=\"baseline.ipynb\",\n",
+    "    repo_id=\"arabic-translation-prompt-engineering/atpe-notebooks\",\n",
+    "    repo_type=\"model\",\n",
+    ")"
+   ]
   }
  ],
  "metadata": {