|
--- |
|
tags: |
|
- medical |
|
- mmlu |
|
- medalpaca |
|
- medmcqa |
|
datasets: |
|
- cais/mmlu |
|
- medalpaca/medical_meadow_medqa |
|
- medalpaca/medical_meadow_wikidoc |
|
- openlifescienceai/medmcqa |
|
- bigbio/med_qa |
|
- GBaker/MedQA-USMLE-4-options |
|
- medalpaca/medical_meadow_mmmlu |
|
- medalpaca/medical_meadow_wikidoc_patient_information |
|
- qiaojin/PubMedQA |
|
pipeline_tag: text-generation |
|
--- |
|
### Evaluation results |
|
|
|
| Dataset | GPT-3.5 | Tuned Llama 3 V1 | Tuned Llama 3 V2 | |
|
|:-------------:|:-----:|:----:|:----:| |
|
| MMLU Clinical Knowledge | 69.8| 74.34 | 73.20 | |
|
| MMLU College Biology | 72.2| 72.92 | 74.30 | |
|
| MMLU College Medicine | 61.3| 61.85 | 66.47 | |
|
| MMLU Medical Genetics | 70.0| 76.0 | 74.0 | |
|
| MMLU Professional Medicine| 70.2| 72.43 | 71.32 | |
|
| MMLU Anatomy | 56.3| 61.48 | 64.44 | |
