MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511

Model Card for Model ID

We fine-tuned our base model for 21 epochs on the Ca dataset, epoch 1 showed the best macro average f1 score on the evaluation dataset.

Context format

"### Context\n\nText to analyse.\n\n###Answer"

Metric

eval_AVGf1 0.9102075019834961

eval_DIAGNOSIS.f1 0.8808602150537634

eval_DIAGNOSIS.precision 0.8943231441048035

eval_DIAGNOSIS.recall 0.8677966101694915

eval_DIAGNOSTIC.f1 0.9472166137871358

eval_DIAGNOSTIC.precision 0.9624853458382181

eval_DIAGNOSTIC.recall 0.9324247586598523

eval_DRUG.f1 0.9440145653163405

eval_DRUG.precision 0.9792256846081209

eval_DRUG.recall 0.9112478031634447

eval_MEDICAL_FINDING.f1 0.9092427259297321

eval_MEDICAL_FINDING.precision 0.9073195744135367

eval_MEDICAL_FINDING.recall 0.9111740473738414

eval_THERAPY.f1 0.8697033898305084

eval_THERAPY.precision 0.8729399255715046

eval_THERAPY.recall 0.8664907651715039

eval_accuracy 0.9618960382191458

eval_f1 0.7632318301785055

eval_loss 0.006697072647511959

eval_model_preparation_time 0

eval_precision 0.6619246861924686

eval_recall 0.9011526605012733

eval_runtime 341.5967

eval_samples_per_second 23.952

eval_steps_per_second 5.99

test_AVGf1 0.8676664044743045

test_DIAGNOSIS.f1 0.7754658946987515

test_DIAGNOSIS.precision 0.7846942511900403

test_DIAGNOSIS.recall 0.7664520743919886

test_DIAGNOSTIC.f1 0.9211950129381322

test_DIAGNOSTIC.precision 0.9346062052505967

test_DIAGNOSTIC.recall 0.9081632653061225

test_DRUG.f1 0.9448028673835126

test_DRUG.precision 0.9835820895522388

test_DRUG.recall 0.9089655172413793

test_MEDICAL_FINDING.f1 0.879590997238056

test_MEDICAL_FINDING.precision 0.8656025907934305

test_MEDICAL_FINDING.recall 0.8940389439732409

test_THERAPY.f1 0.8172772501130711

test_THERAPY.precision 0.8187584956955143

test_THERAPY.recall 0.8158013544018059

test_accuracy 0.9665184459433998

test_f1 0.7391588362393848

test_loss 0.009836438111960888

test_model_preparation_time 0

test_precision 0.6447795213465416

test_recall 0.865905344949376

test_runtime 394.9961

test_samples_per_second 24.023

test_steps_per_second 6.008