Zekun Wu commited on
Commit
0955149
1 Parent(s): fd72b9a
Files changed (1) hide show
  1. util/evaluator.py +4 -6
util/evaluator.py CHANGED
@@ -50,18 +50,16 @@ class evaluator:
50
  Definition: The explanation should offer or accommodate multiple viewpoints or interpretations, allowing the user to explore various perspectives.
51
  Score: (0-1) How well does the explanation provide or support multiple perspectives?
52
 
53
- After evaluating the provided question and explanation based on the five principles, please format your scores in a JSON dictionary.
54
-
55
  Example JSON format:
56
 
57
- {{"Factually Correct": 0.9,"Useful": 0.85,"Context Specific": 0.8,"User Specific": 0.75,"Provides Pluralism": 0.7}}
58
-
59
- Directly provide me with the json without any additional text.
60
 
61
  Answer:
62
  """
63
 
64
- response = self.model.invoke(evaluation_prompt,temperature=0.8, max_tokens=150).strip()
65
  #response = """{{"Factually Correct": 0.9,"Useful": 0.85,"Context Specific": 0.8,"User Specific": 0.75,"Provides Pluralism": 0.7}}"""
66
  print(response)
67
  try:
 
50
  Definition: The explanation should offer or accommodate multiple viewpoints or interpretations, allowing the user to explore various perspectives.
51
  Score: (0-1) How well does the explanation provide or support multiple perspectives?
52
 
53
+ After evaluating the provided question and explanation based on the five principles, please format your scores in a JSON dictionary. Directly provide me with the json without any additional text.
54
+
55
  Example JSON format:
56
 
57
+ Answer:{{"Factually Correct": 0.9,"Useful": 0.85,"Context Specific": 0.8,"User Specific": 0.75,"Provides Pluralism": 0.7}}
 
 
58
 
59
  Answer:
60
  """
61
 
62
+ response = self.model.invoke(evaluation_prompt,temperature=0.8, max_tokens=500).strip()
63
  #response = """{{"Factually Correct": 0.9,"Useful": 0.85,"Context Specific": 0.8,"User Specific": 0.75,"Provides Pluralism": 0.7}}"""
64
  print(response)
65
  try: