Update README.md

README.md

@@ -3,7 +3,15 @@ tags:
 - generated_from_trainer
 model-index:
 - name: sentiment-polish-gpt2-large
-  results: []
+  results:
+  - task:
+      type: text-classification
+    dataset:
+      type: allegro/klej-polemo2-out
+      name: klej-polemo2-out
+    metrics:
+      - type: accuracy
+        value: 98.58%
 license: mit
 datasets:
 - clarin-pl/polemo2-official
@@ -60,6 +68,8 @@ GPU: 2x RTX 4060Ti 16GB
 
 Training time: 29:16:50
 
+Using accelerate + DeepSpeed
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
@@ -73,6 +83,48 @@ The following hyperparameters were used during training:
 - lr_scheduler_type: linear
 - num_epochs: 10
 
+### Evaluation
+
+Evaluated on [allegro/klej-polemo2-out](https://huggingface.co/datasets/allegro/klej-polemo2-out) test dataset.
+```py
+from datasets import load_dataset
+from evaluate import evaluator
+
+data = load_dataset("allegro/klej-polemo2-out", split="test").shuffle(seed=42)
+task_evaluator = evaluator("text-classification")
+
+# fix labels
+l = {
+        "__label__meta_zero": 0,
+        "__label__meta_minus_m": 1,
+        "__label__meta_plus_m": 2,
+        "__label__meta_amb": 3
+    }
+def fix_labels(examples):
+    examples["target"] = l[examples["target"]]
+    return examples
+data = data.map(fix_labels)
+
+eval_resutls = task_evaluator.compute(
+    model_or_pipeline="nie3e/sentiment-polish-gpt2-large",
+    data=data,
+    label_mapping={"NEUTRAL": 0, "NEGATIVE": 1, "POSITIVE": 2, "AMBIGUOUS": 3},
+    input_column="sentence",
+    label_column="target"
+)
+
+print(eval_resutls)
+```
+
+```json
+{
+    "accuracy": 0.9858299595141701,
+    "total_time_in_seconds": 12.71777104900002,
+    "samples_per_second": 38.8432845737416,
+    "latency_in_seconds": 0.02574447580769235
+}
+```
+
 ### Framework versions
 
 - Transformers 4.37.2