t5-small-finetuned-coding_instructions_2023_08_18__08_41

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.9209
• Rouge1: 13.9516
• Rouge2: 6.1527
• Rougel: 13.1037
• Rougelsum: 13.1244
• Gen Len: 18.3077

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	5	2.6656	8.6104	3.1562	8.1185	8.1422	19.0
No log	2.0	10	2.5149	9.7852	3.836	9.3185	9.3322	19.0
No log	3.0	15	2.3683	13.1134	5.2015	12.1364	12.2677	19.0
No log	4.0	20	2.2032	13.4182	5.1369	12.5255	12.6118	19.0
No log	5.0	25	2.0986	13.6902	5.3556	12.7848	12.898	19.0
No log	6.0	30	2.0232	12.7675	4.8786	11.9464	11.9539	18.3846
No log	7.0	35	1.9857	13.9444	6.1527	13.0926	13.1171	18.5385
No log	8.0	40	1.9526	13.9516	6.1527	13.1037	13.1244	18.5385
No log	9.0	45	1.9303	13.9516	6.1527	13.1037	13.1244	18.3077
No log	10.0	50	1.9209	13.9516	6.1527	13.1037	13.1244	18.3077

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3