|
--- |
|
language: "en" |
|
tags: |
|
- text-to-speech |
|
- TTS |
|
- speech-synthesis |
|
- Tacotron2 |
|
- speechbrain |
|
license: "apache-2.0" |
|
datasets: |
|
- LJSpeech |
|
metrics: |
|
- mos |
|
--- |
|
|
|
# Sunbird AI Text-to-Speech (TTS) model trained on Luganda text |
|
|
|
### Text-to-Speech (TTS) with Tacotron2 trained on Female Commonvoice Recordings |
|
|
|
This repository provides all the necessary tools for Text-to-Speech (TTS) with SpeechBrain using a [Tacotron2](https://arxiv.org/abs/1712.05884) pretrained on [LJSpeech](https://keithito.com/LJ-Speech-Dataset/). |
|
|
|
The pre-trained model takes in input a short text and produces a spectrogram in output. One can get the final waveform by applying a vocoder (e.g., HiFIGAN) on top of the generated spectrogram. |
|
|
|
|
|
### Install SpeechBrain |
|
|
|
``` |
|
pip install speechbrain |
|
``` |
|
|
|
Please notice that we encourage you to read our tutorials and learn more about |
|
[SpeechBrain](https://speechbrain.github.io). |
|
|
|
### Perform Text-to-Speech (TTS) |
|
|
|
``` |
|
import torchaudio |
|
from speechbrain.pretrained import Tacotron2 |
|
from speechbrain.pretrained import HIFIGAN |
|
|
|
# Intialize TTS (tacotron2) and Vocoder (HiFIGAN) |
|
tacotron2 = Tacotron2.from_hparams(source="speechbrain/tts-tacotron2-ljspeech", savedir="tmpdir_tts") |
|
hifi_gan = HIFIGAN.from_hparams(source="speechbrain/tts-hifigan-ljspeech", savedir="tmpdir_vocoder") |
|
|
|
# Running the TTS |
|
mel_output, mel_length, alignment = tacotron2.encode_text("Mary had a little lamb") |
|
|
|
# Running Vocoder (spectrogram-to-waveform) |
|
waveforms = hifi_gan.decode_batch(mel_output) |
|
|
|
# Save the waverform |
|
torchaudio.save('example_TTS.wav',waveforms.squeeze(1), 22050) |
|
``` |
|
|
|
If you want to generate multiple sentences in one-shot, you can do in this way: |
|
|
|
``` |
|
from speechbrain.pretrained import Tacotron2 |
|
tacotron2 = Tacotron2.from_hparams(source="speechbrain/TTS_Tacotron2", savedir="tmpdir") |
|
items = [ |
|
"A quick brown fox jumped over the lazy dog", |
|
"How much wood would a woodchuck chuck?", |
|
"Never odd or even" |
|
] |
|
mel_outputs, mel_lengths, alignments = tacotron2.encode_batch(items) |
|
|
|
``` |
|
|
|
### Inference on GPU |
|
To perform inference on the GPU, add `run_opts={"device":"cuda"}` when calling the `from_hparams` method. |
