sethuiyer commited on
Commit
7dbdaab
1 Parent(s): b1a643e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -5
README.md CHANGED
@@ -69,11 +69,18 @@ What is your name?
69
  My name is Samantha.
70
  ```
71
 
72
- # Doctor Samantha's Performance Evaluation by GPT-4 across 25 random prompts from ChatDoctor-200k Dataset
 
 
 
 
 
73
 
74
- ## Overall Rating: 83.5/100
75
 
76
- ### Pros:
 
 
77
 
78
  - Demonstrates extensive medical knowledge through accurate identification of potential causes for various symptoms.
79
  - Responses consistently emphasize the importance of seeking professional diagnoses and treatments.
@@ -83,7 +90,7 @@ My name is Samantha.
83
  - Clear and understandable explanations of conditions and treatment options.
84
  - Prompt responses addressing all aspects of medical inquiries.
85
 
86
- ### Cons:
87
 
88
  - Could occasionally place stronger emphasis on urgency when symptoms indicate potential emergencies.
89
  - Discussion of differential diagnoses could explore a broader range of less common causes.
@@ -92,4 +99,4 @@ My name is Samantha.
92
  - Consider exploring full medical histories to improve diagnostic context where relevant.
93
  - Caution levels and risk factors associated with certain conditions could be underscored more.
94
 
95
- Overall, Dr. Samantha performs at a very high level through knowledgeable, empathetic virtual care. While response enhancements could optimize patient support further, she effectively conveys critical medical insights and guidance. The average rating reflects strong competency with room for targeted refinements to achieve maximum ratings. But Dr. Samantha demonstrates the qualities essential for exemplary telehealth services.
 
69
  My name is Samantha.
70
  ```
71
 
72
+ ## OpenLLM Leaderboard Performance
73
+ | T | Model | Average | ARC | Hellaswag | MMLU | TruthfulQA | Winogrande | GSM8K |
74
+ |---|----------------------------------|---------|-------|-----------|-------|------------|------------|-------|
75
+ | 1 | sethuiyer/Dr_Samantha-7b | 52.95 | 53.84 | 77.95 | 47.94 | 45.58 | 73.56 | 18.8 |
76
+ | 2 | togethercomputer/LLaMA-2-7B-32K-Instruct | 50.02 | 51.11 | 78.51 | 46.11 | 44.86 | 73.88 | 5.69 |
77
+ | 3 | togethercomputer/LLaMA-2-7B-32K | 47.07 | 47.53 | 76.14 | 43.33 | 39.23 | 71.9 | 4.32 |
78
 
79
+ ## Evaluation by GPT-4 across 25 random prompts from ChatDoctor-200k Dataset
80
 
81
+ ### Overall Rating: 83.5/100
82
+
83
+ #### Pros:
84
 
85
  - Demonstrates extensive medical knowledge through accurate identification of potential causes for various symptoms.
86
  - Responses consistently emphasize the importance of seeking professional diagnoses and treatments.
 
90
  - Clear and understandable explanations of conditions and treatment options.
91
  - Prompt responses addressing all aspects of medical inquiries.
92
 
93
+ #### Cons:
94
 
95
  - Could occasionally place stronger emphasis on urgency when symptoms indicate potential emergencies.
96
  - Discussion of differential diagnoses could explore a broader range of less common causes.
 
99
  - Consider exploring full medical histories to improve diagnostic context where relevant.
100
  - Caution levels and risk factors associated with certain conditions could be underscored more.
101
 
102
+