bert-20

This model is a fine-tuned version of deepset/bert-base-cased-squad2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 10.1509

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss
11.3191	0.09	5	12.3262
11.4409	0.18	10	12.2742
11.0259	0.27	15	12.2224
11.1517	0.36	20	12.1706
11.1256	0.45	25	12.1189
11.1448	0.55	30	12.0668
10.4156	0.64	35	12.0150
11.0415	0.73	40	11.9641
10.5847	0.82	45	11.9123
10.4663	0.91	50	11.8612
10.2171	1.0	55	11.8098
10.4121	1.09	60	11.7587
10.5127	1.18	65	11.7082
10.7149	1.27	70	11.6574
10.2304	1.36	75	11.6066
10.6459	1.45	80	11.5557
10.3551	1.55	85	11.5044
10.2838	1.64	90	11.4531
10.0924	1.73	95	11.4025
10.295	1.82	100	11.3513
9.8206	1.91	105	11.3005
10.1365	2.0	110	11.2498
10.0496	2.09	115	11.1992
9.8465	2.18	120	11.1489
9.9778	2.27	125	11.0980
10.0708	2.36	130	11.0471
9.6465	2.45	135	10.9962
9.9864	2.55	140	10.9461
9.5175	2.64	145	10.8963
9.9675	2.73	150	10.8461
9.7013	2.82	155	10.7963
9.6324	2.91	160	10.7461
9.7833	3.0	165	10.6960
9.6806	3.09	170	10.6461
9.6208	3.18	175	10.5964
9.3067	3.27	180	10.5468
9.1504	3.36	185	10.4972
9.8082	3.45	190	10.4474
9.3738	3.55	195	10.3978
9.1904	3.64	200	10.3478
9.0302	3.73	205	10.2981
8.8785	3.82	210	10.2490
8.8765	3.91	215	10.1997
9.3	4.0	220	10.1509

Framework versions

• Transformers 4.34.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1