text_shortening_model_v30

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.6784
• Rouge1: 0.4871
• Rouge2: 0.2579
• Rougel: 0.428
• Rougelsum: 0.4272
• Bert precision: 0.8743
• Bert recall: 0.8706
• Average word count: 8.4775
• Max word count: 17
• Min word count: 3
• Average token count: 12.9249
• % shortened texts with length > 12: 9.3093

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
1.2044	1.0	145	1.6064	0.5052	0.2865	0.4472	0.448	0.8751	0.8756	8.8979	17	3	13.4024	12.6126
1.0041	2.0	290	1.4900	0.5154	0.2921	0.4554	0.4542	0.8735	0.878	9.3724	17	3	13.8529	17.7177
0.8935	3.0	435	1.4617	0.5181	0.2968	0.4607	0.4622	0.8751	0.8818	9.4024	16	4	14.1171	17.1171
0.8028	4.0	580	1.4744	0.5103	0.2966	0.4497	0.4496	0.8797	0.8725	8.1982	17	4	12.5706	8.1081
0.7395	5.0	725	1.4797	0.5121	0.3016	0.4548	0.4554	0.8796	0.8761	8.4985	16	3	12.985	10.8108
0.6986	6.0	870	1.5154	0.5218	0.2987	0.4554	0.4542	0.8808	0.879	8.7297	16	4	13.0691	14.1141
0.6527	7.0	1015	1.5347	0.5083	0.2876	0.4494	0.4485	0.8797	0.8763	8.5526	16	4	13.012	11.4114
0.588	8.0	1160	1.5578	0.4984	0.2752	0.4403	0.4399	0.8746	0.8728	8.6336	17	4	13.006	10.8108
0.5705	9.0	1305	1.6569	0.5152	0.2902	0.4544	0.454	0.8803	0.8764	8.5135	16	4	13.1592	9.9099
0.5601	10.0	1450	1.6651	0.5246	0.2837	0.4572	0.4579	0.8777	0.8807	8.979	16	4	13.6607	12.012
0.523	11.0	1595	1.7085	0.5149	0.2854	0.4508	0.4507	0.879	0.8789	8.7718	17	4	13.2613	10.8108
0.5032	12.0	1740	1.7886	0.5107	0.2817	0.4457	0.4457	0.8778	0.8772	8.8378	17	4	13.4204	11.7117
0.4872	13.0	1885	1.8073	0.5097	0.2808	0.4439	0.4441	0.8786	0.8758	8.6306	16	4	13.1562	9.6096
0.4703	14.0	2030	1.8436	0.5059	0.2754	0.4456	0.4457	0.8769	0.8756	8.6817	17	4	13.1471	9.9099
0.4598	15.0	2175	1.9150	0.5148	0.2794	0.4532	0.4532	0.8798	0.8775	8.6907	18	4	13.1021	11.4114
0.4385	16.0	2320	1.9319	0.4966	0.2666	0.4402	0.4406	0.8771	0.8724	8.2703	16	4	12.7237	7.8078
0.4306	17.0	2465	1.9821	0.5041	0.2763	0.4449	0.4448	0.8788	0.8752	8.5105	16	4	13.0541	9.3093
0.4154	18.0	2610	2.0345	0.5066	0.2746	0.4467	0.4461	0.8796	0.8732	8.1922	16	3	12.6186	7.8078
0.3995	19.0	2755	2.0671	0.4954	0.2707	0.4411	0.4416	0.8773	0.8721	8.4505	17	4	12.8468	8.7087
0.4053	20.0	2900	2.1265	0.4975	0.2704	0.4365	0.4364	0.8767	0.873	8.5075	17	3	13.0571	9.009
0.3812	21.0	3045	2.2077	0.5011	0.2733	0.4406	0.4411	0.8764	0.8756	8.7958	17	3	13.4084	12.012
0.3856	22.0	3190	2.2043	0.4956	0.2603	0.4358	0.4361	0.8775	0.8729	8.2913	17	3	12.8078	8.7087
0.3805	23.0	3335	2.2201	0.5015	0.2698	0.4421	0.4427	0.8789	0.8728	8.2402	17	3	12.5856	8.1081
0.3741	24.0	3480	2.2269	0.5029	0.2652	0.4412	0.4413	0.8767	0.8743	8.5856	16	4	13.039	10.2102
0.3697	25.0	3625	2.2596	0.4956	0.2674	0.436	0.4359	0.8765	0.8728	8.4895	17	4	12.9129	9.9099
0.3663	26.0	3770	2.2506	0.4891	0.2572	0.432	0.432	0.8749	0.8716	8.4865	17	4	12.8498	6.9069
0.3409	27.0	3915	2.2893	0.4958	0.2635	0.4328	0.4327	0.8772	0.8727	8.3994	17	3	12.8228	9.6096
0.3524	28.0	4060	2.3127	0.4907	0.2597	0.4322	0.4329	0.8751	0.8712	8.4084	16	4	12.7718	8.1081
0.3379	29.0	4205	2.3167	0.4958	0.2674	0.4374	0.4368	0.8772	0.8737	8.4234	16	4	12.8138	7.2072
0.3472	30.0	4350	2.3157	0.4987	0.2713	0.4415	0.4403	0.8788	0.8736	8.3634	17	3	12.6517	7.2072
0.3353	31.0	4495	2.3506	0.4991	0.2631	0.4375	0.436	0.8764	0.8744	8.6396	17	4	13.1502	9.6096
0.3466	32.0	4640	2.3594	0.4897	0.2593	0.4307	0.4301	0.8777	0.8711	8.1712	16	4	12.6126	5.4054
0.3406	33.0	4785	2.3632	0.495	0.2746	0.4401	0.4397	0.8772	0.8732	8.5556	16	4	13.027	8.4084
0.3382	34.0	4930	2.3505	0.4856	0.261	0.4306	0.4295	0.8758	0.8693	8.2733	17	3	12.6366	7.5075
0.3392	35.0	5075	2.3665	0.4972	0.2719	0.4376	0.4372	0.8764	0.8741	8.6847	17	4	13.1532	9.3093
0.3465	36.0	5220	2.3837	0.4981	0.2722	0.441	0.4411	0.876	0.8738	8.6607	17	4	13.1982	12.3123
0.3377	37.0	5365	2.3984	0.4832	0.2623	0.4294	0.4285	0.8737	0.8697	8.5225	17	4	12.9399	10.5105
0.3523	38.0	5510	2.3843	0.495	0.2671	0.438	0.4368	0.8754	0.873	8.5886	17	3	13.1111	7.2072
0.3261	39.0	5655	2.4337	0.4948	0.2666	0.4378	0.4369	0.8771	0.8726	8.4655	17	4	12.8919	9.009
0.3262	40.0	5800	2.4149	0.4971	0.2691	0.438	0.4375	0.8772	0.8717	8.4505	16	4	12.9249	8.1081
0.3307	41.0	5945	2.4352	0.4834	0.2585	0.4261	0.4256	0.8746	0.8697	8.4024	17	3	12.8859	9.6096
0.3226	42.0	6090	2.4241	0.488	0.2584	0.4318	0.4315	0.8756	0.8706	8.4444	17	3	12.8288	8.7087
0.34	43.0	6235	2.4485	0.4891	0.2589	0.4326	0.432	0.8758	0.8705	8.3243	17	4	12.7898	6.6066
0.3425	44.0	6380	2.4457	0.4865	0.26	0.4293	0.4287	0.8733	0.8713	8.6336	16	3	13.1922	9.6096
0.3201	45.0	6525	2.4535	0.4811	0.2473	0.4243	0.4237	0.8751	0.8697	8.3093	17	3	12.7748	8.4084
0.3094	46.0	6670	2.4918	0.4916	0.2614	0.4351	0.4342	0.8758	0.8726	8.5706	17	3	13.039	10.2102
0.3262	47.0	6815	2.4839	0.4822	0.255	0.425	0.4237	0.8719	0.869	8.5375	17	4	12.976	9.009
0.3186	48.0	6960	2.4966	0.486	0.2492	0.4276	0.4264	0.8738	0.8707	8.4745	17	3	12.955	6.6066
0.3231	49.0	7105	2.4978	0.4889	0.2661	0.4343	0.434	0.8767	0.871	8.4505	17	3	12.8468	9.009
0.3294	50.0	7250	2.4731	0.4916	0.2683	0.4374	0.4373	0.877	0.8726	8.4955	17	4	12.9369	9.3093
0.3172	51.0	7395	2.4922	0.4861	0.2573	0.4314	0.431	0.8759	0.87	8.3003	17	4	12.6907	7.8078
0.3247	52.0	7540	2.5044	0.4802	0.2495	0.4281	0.4282	0.8737	0.8698	8.4715	17	4	12.9009	8.1081
0.3132	53.0	7685	2.5168	0.4832	0.2558	0.4273	0.4268	0.8736	0.8703	8.5706	17	3	12.967	9.3093
0.3285	54.0	7830	2.5296	0.4882	0.26	0.4323	0.4319	0.8754	0.8724	8.5495	17	3	13.0541	8.7087
0.3111	55.0	7975	2.5529	0.4829	0.2561	0.4268	0.4262	0.874	0.8694	8.4474	17	3	12.9339	7.2072
0.3194	56.0	8120	2.5903	0.49	0.2614	0.4337	0.4329	0.8747	0.8719	8.5946	17	3	13.0931	8.1081
0.3144	57.0	8265	2.5787	0.4859	0.2593	0.4315	0.4303	0.8739	0.8698	8.5195	17	4	12.8679	8.4084
0.2972	58.0	8410	2.5759	0.4848	0.2565	0.4291	0.4279	0.8738	0.8697	8.5165	17	3	12.9219	8.1081
0.3209	59.0	8555	2.5609	0.4792	0.246	0.4212	0.4201	0.8723	0.8678	8.4114	17	3	12.8799	6.9069
0.3148	60.0	8700	2.5758	0.481	0.2454	0.4243	0.4231	0.874	0.8688	8.3664	16	3	12.7628	7.5075
0.3026	61.0	8845	2.5819	0.4804	0.2555	0.4231	0.4231	0.8738	0.8689	8.4204	17	3	12.7628	8.4084
0.3074	62.0	8990	2.5882	0.4893	0.2627	0.431	0.4303	0.8753	0.8715	8.4895	17	3	12.8889	8.7087
0.3013	63.0	9135	2.5865	0.4835	0.2599	0.426	0.4251	0.8743	0.8707	8.4865	17	4	12.964	8.7087
0.3274	64.0	9280	2.5957	0.4928	0.2649	0.436	0.4353	0.8738	0.8734	8.8018	17	3	13.2823	11.4114
0.2928	65.0	9425	2.5846	0.4888	0.2653	0.4365	0.4356	0.8763	0.8713	8.2973	17	3	12.6637	8.1081
0.3261	66.0	9570	2.5704	0.4901	0.267	0.4386	0.4374	0.8759	0.871	8.3303	17	4	12.7838	6.6066
0.3153	67.0	9715	2.6023	0.4897	0.2611	0.4311	0.4301	0.8749	0.872	8.6426	17	3	13.0691	10.8108
0.3185	68.0	9860	2.5831	0.4862	0.2579	0.4257	0.4247	0.8735	0.8718	8.6486	17	4	13.1441	12.012
0.3054	69.0	10005	2.5949	0.4831	0.2575	0.4247	0.4239	0.8728	0.87	8.5405	17	4	13.036	9.9099
0.3006	70.0	10150	2.5822	0.4853	0.252	0.4255	0.4243	0.8735	0.87	8.5495	17	3	13.0	10.5105
0.3092	71.0	10295	2.5743	0.4903	0.2595	0.432	0.4315	0.8759	0.8719	8.4474	17	3	12.8559	8.7087
0.2928	72.0	10440	2.5905	0.4918	0.2665	0.4356	0.4347	0.876	0.8724	8.4474	17	4	12.8679	8.4084
0.3021	73.0	10585	2.6171	0.4957	0.266	0.4368	0.4354	0.8764	0.873	8.5676	17	3	12.964	11.1111
0.3047	74.0	10730	2.6233	0.492	0.2655	0.4341	0.4328	0.8753	0.8715	8.5736	17	3	12.952	10.5105
0.3043	75.0	10875	2.6405	0.4887	0.2623	0.4318	0.4309	0.8756	0.8704	8.4895	17	3	12.8679	9.9099
0.305	76.0	11020	2.6171	0.4942	0.2687	0.4381	0.4372	0.8766	0.8724	8.5586	17	3	12.9369	10.8108
0.3127	77.0	11165	2.6289	0.4959	0.2646	0.4366	0.4357	0.8767	0.8731	8.5766	17	3	13.006	12.012
0.2945	78.0	11310	2.6453	0.4881	0.2589	0.4272	0.4261	0.8753	0.8711	8.5375	17	3	12.8739	9.3093
0.2844	79.0	11455	2.6543	0.4895	0.2565	0.4294	0.4288	0.8753	0.8718	8.5616	17	3	12.997	11.7117
0.3188	80.0	11600	2.6556	0.4919	0.2677	0.4328	0.4318	0.8756	0.8712	8.5345	17	3	12.973	9.9099
0.2857	81.0	11745	2.6696	0.4914	0.2666	0.434	0.4332	0.8761	0.8717	8.4595	17	3	12.8829	10.5105
0.3091	82.0	11890	2.6577	0.4986	0.2718	0.4397	0.4388	0.8766	0.8741	8.6276	17	3	13.1441	10.8108
0.3115	83.0	12035	2.6720	0.4944	0.266	0.4364	0.4351	0.8766	0.8725	8.4925	17	3	12.9309	9.3093
0.2947	84.0	12180	2.6490	0.4955	0.2628	0.4347	0.4343	0.8767	0.873	8.4985	17	3	13.018	7.5075
0.312	85.0	12325	2.6425	0.4928	0.2689	0.4364	0.4358	0.8763	0.8728	8.5766	17	3	13.0631	9.9099
0.3081	86.0	12470	2.6314	0.4904	0.2648	0.4327	0.432	0.875	0.8722	8.6246	17	3	13.1411	10.5105
0.3043	87.0	12615	2.6485	0.4863	0.259	0.4273	0.4259	0.8736	0.8709	8.5736	17	3	13.0901	9.6096
0.3034	88.0	12760	2.6402	0.4867	0.2604	0.4279	0.4274	0.8739	0.871	8.5706	17	3	13.0751	8.1081
0.3058	89.0	12905	2.6573	0.4926	0.2638	0.4348	0.4339	0.8762	0.872	8.4805	17	3	12.955	7.8078
0.2909	90.0	13050	2.6654	0.4955	0.2679	0.4357	0.4342	0.8756	0.8729	8.6817	17	3	13.1802	10.2102
0.3082	91.0	13195	2.6757	0.4942	0.2671	0.4362	0.4349	0.8756	0.8724	8.5796	17	3	13.0721	9.6096
0.3016	92.0	13340	2.6791	0.4933	0.2657	0.4351	0.4345	0.875	0.8722	8.6336	17	3	13.1441	9.9099
0.2993	93.0	13485	2.6814	0.493	0.2658	0.433	0.4318	0.8747	0.8726	8.6997	17	3	13.2462	11.1111
0.3022	94.0	13630	2.6698	0.4929	0.2638	0.4334	0.4324	0.8751	0.8723	8.5976	17	3	13.0961	9.3093
0.2921	95.0	13775	2.6665	0.4867	0.2586	0.4294	0.4284	0.8744	0.8709	8.4955	17	3	12.988	8.4084
0.3034	96.0	13920	2.6704	0.4854	0.2574	0.4275	0.4266	0.8742	0.8704	8.4805	17	3	12.9429	8.7087
0.3063	97.0	14065	2.6749	0.4863	0.2576	0.4275	0.4266	0.8743	0.8707	8.4805	17	3	12.9369	8.7087
0.2984	98.0	14210	2.6772	0.4858	0.258	0.4274	0.4264	0.8739	0.8704	8.5105	17	3	12.97	9.6096
0.2942	99.0	14355	2.6784	0.4872	0.2595	0.4279	0.427	0.874	0.8704	8.5075	17	3	12.967	9.6096
0.2866	100.0	14500	2.6784	0.4871	0.2579	0.428	0.4272	0.8743	0.8706	8.4775	17	3	12.9249	9.3093

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3