leaderboard-pr-bot
commited on
Commit
•
9d32c02
1
Parent(s):
92b2476
Adding Evaluation Results
Browse filesThis is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr
The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.
If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions
README.md
CHANGED
@@ -10,28 +10,31 @@ tags:
|
|
10 |
base_model: meta-llama/Meta-Llama-3-8B-Instruct
|
11 |
datasets:
|
12 |
- ruslanmv/ai-medical-dataset
|
13 |
-
|
14 |
-
model-index:
|
15 |
-
- name: ai-medical-model-32bit
|
16 |
-
results: []
|
17 |
widget:
|
18 |
- example_title: ai-medical-model-32bit
|
19 |
messages:
|
20 |
- role: system
|
21 |
-
content:
|
22 |
-
You are an expert and experienced from the healthcare and biomedical
|
23 |
domain with extensive medical knowledge and practical experience.
|
24 |
- role: user
|
25 |
content: What was the main cause of the inflammatory CD4+ T cells?
|
26 |
output:
|
27 |
-
text:
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
35 |
---
|
36 |
|
37 |
# ai-medical-model-32bit: Fine-Tuned Llama3 for Technical Medical Questions
|
@@ -137,4 +140,17 @@ We welcome contributions to this repository! If you have improvements or suggest
|
|
137 |
|
138 |
**Disclaimer**
|
139 |
|
140 |
-
While we strive to provide informative responses, the accuracy of the model's outputs cannot be guaranteed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
base_model: meta-llama/Meta-Llama-3-8B-Instruct
|
11 |
datasets:
|
12 |
- ruslanmv/ai-medical-dataset
|
|
|
|
|
|
|
|
|
13 |
widget:
|
14 |
- example_title: ai-medical-model-32bit
|
15 |
messages:
|
16 |
- role: system
|
17 |
+
content: You are an expert and experienced from the healthcare and biomedical
|
|
|
18 |
domain with extensive medical knowledge and practical experience.
|
19 |
- role: user
|
20 |
content: What was the main cause of the inflammatory CD4+ T cells?
|
21 |
output:
|
22 |
+
text: Answer I'm happy to help! The main cause of inflammatory CD4+ T cells is
|
23 |
+
a complex process that involves multiple factors. However, some of the key triggers
|
24 |
+
include 1. Activation of CD4+ T cells CD4+ T cells are activated by antigens,
|
25 |
+
cytokines, and other signals, leading to their proliferation and differentiation
|
26 |
+
into effector cells. 2. Cytokine production Activated CD4+ T cells produce cytokines
|
27 |
+
such as interleukin-2 (IL-2), interferon-gamma (IFN-γ), and tumor necrosis factor-alpha
|
28 |
+
(TNF-α), which promote inflammation and immune responses. 3. Chemokine production
|
29 |
+
CD4+ T cells also produce chemokines, such as CCL3, CCL4, and CCL5, which attract
|
30 |
+
other immune cells to the site of inflammation. 4. Toll-like receptor (TLR)
|
31 |
+
activation TLRs are pattern recognition receptors that recognize pathogen-associated
|
32 |
+
molecular patterns (PAMPs) and activate CD4+ T cells. 5. Bacterial or viral
|
33 |
+
infections Infections caused by bacteria, viruses, or fungi can trigger the
|
34 |
+
activation of CD4+ T cells and the production of cytokines and chemokines
|
35 |
+
model-index:
|
36 |
+
- name: ai-medical-model-32bit
|
37 |
+
results: []
|
38 |
---
|
39 |
|
40 |
# ai-medical-model-32bit: Fine-Tuned Llama3 for Technical Medical Questions
|
|
|
140 |
|
141 |
**Disclaimer**
|
142 |
|
143 |
+
While we strive to provide informative responses, the accuracy of the model's outputs cannot be guaranteed.
|
144 |
+
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
145 |
+
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ruslanmv__ai-medical-model-32bit)
|
146 |
+
|
147 |
+
| Metric |Value|
|
148 |
+
|---------------------------------|----:|
|
149 |
+
|Avg. |67.67|
|
150 |
+
|AI2 Reasoning Challenge (25-Shot)|61.43|
|
151 |
+
|HellaSwag (10-Shot) |78.69|
|
152 |
+
|MMLU (5-Shot) |68.10|
|
153 |
+
|TruthfulQA (0-shot) |51.99|
|
154 |
+
|Winogrande (5-shot) |75.77|
|
155 |
+
|GSM8k (5-shot) |70.05|
|
156 |
+
|