(checkpoint) Obtaining answers.

agent_factory.py

@@ -30,14 +30,12 @@ class AgentFactory:
     """
 
     __system_prompt: str = (
-        "You have to answer to test questions and you need to score high.\n"
-        "Sometimes auxiliary files may be attached to the question so the\n"
-        "question itself is presented as a JSON string with the following\n"
-        "fields:\n"
+        "You have to answer to some test questions.\n"
+        "Sometimes auxiliary files may be attached to the question.\n"
+        "Each question is a JSON string with the following fields:\n"
         "1. task_id: unique hash identifier of the question.\n"
         "2. question: the text of the question.\n"
-        "3. Level: a number with the question difficulty level. You can ignore "
-        "this field.\n"
+        "3. Level: ignore this field.\n"
         "4. file_name: the name of the file needed to answer the question. "
         "This is empty if the question does not refer to any file. "
         "IMPORTANT: The text of the question may mention a file name that is "
@@ -45,9 +43,14 @@ class AgentFactory:
         "YOU HAVE TO IGNORE THE FILE NAME MENTIONED INTO \"question\" AND "
         "YOU MUST USE THE FILE NAME PROVIDED INTO THE \"file_name\" FIELD.\n"
         "\n"
-        "Depending on the question, the\n"
-        "format of your answer is a number OR as few words as possible OR a\n"
-        "comma separated list of numbers and/or strings. If you are asked for\n"
+        "Achieve the solution by dividing your reasoning in steps, and\n"
+        "provide an explanation for each step.\n"
+        "\n"
+        "The format of your final answer must be\n"
+        "\n"
+        "<ANSWER>your_final_answer</Answer>, where your_final_answer is a\n"
+        "number OR as few words as possible OR a comma separated list of\n"
+        "numbers and/or strings. If you are asked for\n"
         "a number, don't use comma to write your number neither use units\n"
         "such as $ or percent sign unless specified otherwise. If you are\n"
         "asked for a string, don't use articles, neither abbreviations (e.g.\n"
@@ -55,17 +58,16 @@ class AgentFactory:
         "otherwise. If you are asked for a comma separated list, apply the\n"
         "above rules depending of whether the element to be put in the list\n"
         "is a number or a string.\n"
-        "When you have to perform a sum, DON'T try to do that yourself.\n"
-        "Exploit the tool that is able to sum list of numbers. If you have\n"
-        "to sum the results of previous sums, use again the same tool\n"
-        "recursively. NEVER do the sums yourself.\n"
-        "Achieve the solution by dividing your reasoning in steps, and\n"
-        "provide an explanation for each step.\n"
+        "ALWAYS PRESENT THE FINAL ANSWER BETWEEN THE <ANSWER> AND </ANSWER>\n"
+        "TAGS.\n"
+        "\n"
+        "When, for achieving the solution, you have to perform a sum, DON'T\n"
+        "try to do that yourself. Exploit the tool that is able to sum a list\n"
+        " of numbers. If you have to sum the results of previous sums, use\n"
+        "again the same tool, by calling it again.\n"
         "You are advised to cycle between reasoning and tool calling also\n"
         "multiple times. Provide an answer only when you are sure you don't\n"
-        "have to call any tool again. Provide the answer between\n"
-        "<ANSWER> and </ANSWER> tags. I stress that the final answer must\n"
-        "follow the rules explained above.\n"
+        "have to call any tool again."
     )
 
     __llm: Runnable

app.py

@@ -192,7 +192,7 @@ def run_and_submit_all() -> tuple[str, pd.DataFrame | None]:
                 continue
             try:
                 answer_to_submit = agent(question_text)
-                answer_payload = {"task_id": task_id, "answer_to_submit": answer_to_submit}
+                answer_payload = {"task_id": task_id, "submitted_answer": answer_to_submit}
                 json.dump(answer_payload, f)
                 f.write("\n")
                 f.flush()