Update README.md
Browse files
README.md
CHANGED
|
@@ -105,26 +105,6 @@ The prompts for all supervised tasks can be found in [TowerBlocks](https://huggi
|
|
| 105 |
|
| 106 |
Link to [TowerBlocks](https://huggingface.co/datasets/Unbabel/TowerBlocks-v0.1).
|
| 107 |
|
| 108 |
-
#### Training Hyperparameters
|
| 109 |
-
|
| 110 |
-
The following hyperparameters were used during training:
|
| 111 |
-
|
| 112 |
-
- total_train_batch_size: 256
|
| 113 |
-
|
| 114 |
-
- learning_rate: 7e-06
|
| 115 |
-
|
| 116 |
-
- lr_scheduler_type: cosine
|
| 117 |
-
|
| 118 |
-
- lr_scheduler_warmup_steps: 500
|
| 119 |
-
|
| 120 |
-
- weight_decay: 0.01
|
| 121 |
-
|
| 122 |
-
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
| 123 |
-
|
| 124 |
-
- num_epochs: 4
|
| 125 |
-
|
| 126 |
-
- max_seq_length: 2048
|
| 127 |
-
|
| 128 |
## Citation
|
| 129 |
|
| 130 |
```bibtex
|
|
|
|
| 105 |
|
| 106 |
Link to [TowerBlocks](https://huggingface.co/datasets/Unbabel/TowerBlocks-v0.1).
|
| 107 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 108 |
## Citation
|
| 109 |
|
| 110 |
```bibtex
|