malteos commited on
Commit
b764033
1 Parent(s): 8249644

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -1
README.md CHANGED
@@ -73,7 +73,15 @@ The evaluation methdology of the [Open LLM Leaderboard](https://huggingface.co/s
73
 
74
  ### English benchmarks
75
 
76
- TBA
 
 
 
 
 
 
 
 
77
 
78
  ## Prompting / Prompt Template
79
 
@@ -104,3 +112,7 @@ The model input can contain multiple conversation turns between user and assista
104
  ## License
105
 
106
  [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0.html)
 
 
 
 
 
73
 
74
  ### English benchmarks
75
 
76
+ | **English tasks:** | **MMLU** | **Hellaswag** | **ARC** | **Average** |
77
+ |----------------------------|-------------|---------------|--------------|-------------|
78
+ | **Models / Few-shots:** | _(5 shots)_ | _(10 shots)_ | _(24 shots)_ | |
79
+ | llama-2-7b | 0.466 | 0.786 | 0.530 | 0.594 |
80
+ | leolm-hessianai-7b | 0.423 | 0.759 | 0.522 | 0.568 |
81
+ | bloom-6b4-clp-german | 0.264 | 0.525 | 0.328 | 0.372 |
82
+ | mistral-7b | **0.635** | **0.832** | 0.607 | **0.691** |
83
+ | leolm-mistral-hessianai-7b | 0.550 | 0.777 | 0.518 | 0.615 |
84
+ | hermeo-7b (this model) | 0.601 | 0.821 | **0.620** | 0.681 |
85
 
86
  ## Prompting / Prompt Template
87
 
 
112
  ## License
113
 
114
  [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0.html)
115
+
116
+ ## See also
117
+
118
+ - AWQ quantized version: https://huggingface.co/mayflowergmbh/hermeo-7b-awq