AlienKevin's picture
End of training
5b05c57 verified
metadata
license: apache-2.0
base_model: toastynews/electra-hongkongese-small-discriminator
tags:
  - generated_from_trainer
model-index:
  - name: electra_hongkongese_small_pos_wiki_lihkg_cc100
    results: []

electra_hongkongese_small_pos_wiki_lihkg_cc100

This model is a fine-tuned version of toastynews/electra-hongkongese-small-discriminator on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.1796

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3

Training results

Training Loss Epoch Step Validation Loss
0.232 0.3000 6123 0.2192
0.2465 0.6000 12246 0.2052
0.2396 0.9000 18369 0.1906
0.2128 1.2001 24492 0.1897
0.2007 1.5001 30615 0.1883
0.2019 1.8001 36738 0.1832
0.1864 2.1001 42861 0.1813
0.2051 2.4001 48984 0.1799
0.1837 2.7001 55107 0.1796

Framework versions

  • Transformers 4.44.0
  • Pytorch 2.2.1
  • Datasets 2.21.0
  • Tokenizers 0.19.1