Coqui TTS with GPU Acceleration for a custom Home Assistant Integration

Many text-to-speech solutions rely on cloud services, which introduces privacy concerns. Self-hosting a TTS engine like Coqui TTS not only gives you full control over your voice pipeline but also allows for offline operation and integration with your own AI stack.

Using GPU acceleration significantly improves TTS performance, enabling real-time synthesis with lower latency and better quality. In this guide, I'll go through how to deploy Coqui TTS with GPU support using Proxmox passthrough, and integrate it into Home Assistant as a custom TTS provider.

In this guide, you'll learn how to install Coqui TTS on a GPU-passthrough VM using Proxmox, configure it for high-performance synthesis, and integrate it seamlessly into Home Assistant for real-time smart home voice feedback. This setup pairs well with self-hosted LLMs like Ollama and gives you full control over your voice assistant stack.

What Is This?

This is a fully self-hosted, privacy-respecting, GPU-accelerated text-to-speech server using Coqui TTS. We're using it alongside an AI chatbot (Ollama) and Home Assistant Voice for a powerful offline voice assistant.

GPU Passthrough with Proxmox

GPU passthrough is a technique that allows a virtual machine to directly access a physical GPU on the host system. This is crucial for machine learning and TTS applications like Coqui, which benefit significantly from the parallel processing power of GPUs. Without passthrough, the VM would rely on emulated or software rendering, which is too slow for real-time voice synthesis.

Step 1: Configure Your Proxmox VM

Hardware:

RAM: 4GB

CPU: 2 Cores (1 socket, 2 cores), set to x86-64-v2-AES

BIOS: UEFI (OVMF)

TPM: Enabled

EFI Disk: Present

GPU: PCI passthrough enabled (using Device ID)

Step 2: Enable IOMMU and Passthrough

On your Proxmox host:

Enable IOMMU in BIOS.

Set kernel params:

For Intel: intel_iommu=on

For AMD: amd_iommu=on

Verify your /etc/pve/qemu-server/<vmid>.conf includes something like:

hostpci0: 0000:01:00.0,pcie=1

Step 3: Unbind Host Drivers

Check with:

lspci -nnk | grep -A 2 -i nvidia

Ensure the GPU uses vfio-pci and not nouveau or nvidia. If it doesn't, blacklist nouveau and bind your GPU to vfio-pci using /etc/modprobe.d/.

Install Coqui TTS with GPU Acceleration

Step 1: Install Ubuntu Server 24.04 LTS (with third-party drivers checked)

Once booted:

Step 2: Verify GPU Visibility

lspci | grep -i vga

Should return something like:

01:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1080] (rev a1)

Then run:

lshw -c video

It should show:

configuration: driver=nvidia latency=0

If you see nouveau, the NVIDIA driver is not installed yet.

Step 3: Install NVIDIA Driver

sudo apt update
sudo ubuntu-drivers devices
sudo apt install nvidia-driver-550
sudo reboot

After reboot:

nvidia-smi

You should see your GPU listed.

Step 4: Disable Secure Boot (If Necessary)

If NVIDIA drivers still don’t load:

Boot the VM and press ESC to access UEFI

Disable Secure Boot

Reboot again and confirm with nvidia-smi.

Install Python and Coqui TTS

Step 1: Install Python 3.11 via pyenv

sudo apt install -y git curl build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev python3-openssl

curl https://pyenv.run | bash

Add pyenv to your shell (e.g. bashrc/zshrc), then restart.

pyenv install 3.11.9
pyenv virtualenv 3.11.9 coqui-tts
pyenv activate coqui-tts

Step 2: Install Coqui TTS

pip install --upgrade pip
pip install TTS[server]

Run Coqui TTS with Preferred Model

Coqui TTS supports a wide range of models, but for general-purpose English synthesis with high quality and decent speed, the Tacotron2-DDC model paired with HiFi-GAN v2 vocoder is a reliable choice.

Tacotron2-DDC is a variation of Tacotron2 with "double decoder consistency" that improves robustness and naturalness in speech generation.

HiFi-GAN v2 is a fast neural vocoder that converts spectrograms into realistic audio waveforms in real-time.

Together, they offer a good balance between audio quality and inference speed, especially when GPU acceleration is available.

Run the server using the following command:

tts-server \
--model_name tts_models/en/ljspeech/tacotron2-DDC \
--vocoder_name vocoder_models/en/ljspeech/hifigan_v2 \
--use_cuda true

Test via curl:

curl http://<your-server-ip>:5002/process?INPUT_TEXT=this+is+a+test > test.wav

tts-server \
--model_name tts_models/en/ljspeech/tacotron2-DDC \
--vocoder_name vocoder_models/en/ljspeech/hifigan_v2 \
--use_cuda true

Then visit:

http://<your-server-ip>:5002

Test via curl:

curl http://<your-server-ip>:5002/process?INPUT_TEXT=this+is+a+test > test.wav

Auto-Start with systemd

Create a systemd service file:

sudo nano /etc/systemd/system/tts-server.service

[Unit]
Description=Coqui TTS Server
After=network.target

[Service]
User=youruser
WorkingDirectory=/home/youruser/coqui-tts
ExecStart=/home/youruser/.pyenv/versions/coqui-tts/bin/tts-server \
--model_name tts_models/en/ljspeech/tacotron2-DDC \
--vocoder_name vocoder_models/en/ljspeech/hifigan_v2 \
--use_cuda true
Restart=always

[Install]
WantedBy=multi-user.target

Then run:

sudo systemctl daemon-reload
sudo systemctl enable tts-server
sudo systemctl start tts-server
sudo systemctl status tts-server

Home Assistant Integration

Home Assistant supports multiple TTS platforms out of the box, like Google Cloud, Amazon Polly, and Microsoft Azure. However, these cloud-based services may introduce privacy concerns. By creating a custom TTS integration, you can route all voice synthesis through your own Coqui TTS server, keeping everything local, fast, and secure.

This custom component fetches generated audio directly from your self-hosted Coqui TTS server and feeds it into Home Assistant's media pipeline. It behaves like a native integration and can be used in automations, scripts, or notifications just like any built-in provider.

MaryTTS, which was previously supported by Home Assistant, is now discontinued and no longer maintained. Unfortunately, there are currently no other officially supported local TTS integrations for Home Assistant that offer modern performance. As a result, building a custom integration like this one is currently the only reliable way to add high-quality, GPU-accelerated TTS support using Coqui.

Step 1: Create a Custom Component

In config/custom_components/coqui_tts/, add:

manifest.json

{
"domain": "coqui_tts",
"name": "Coqui TTS",
"version": "1.0.0",
"requirements": ["requests"],
"codeowners": ["@yourgithub"]
}

tts.py

import requests
import logging
from homeassistant.components.tts import Provider
from urllib.parse import quote_plus

_LOGGER = logging.getLogger(__name__)

def get_engine(hass, config, discovery_info=None):
return CoquiTTSProvider(config)

class CoquiTTSProvider(Provider):
def __init__(self, config):
self._lang = "en"
self._base_url = config.get("base_url")
self._name = "CoquiTTS"

@property
def default_language(self):
return self._lang

@property
def supported_languages(self):
return ["en"]

@property
def name(self):
return self._name

def get_tts_audio(self, message, language, options=None):
try:
encoded_text = quote_plus(message)
url = f"{self._base_url}/process?INPUT_TEXT={encoded_text}"
resp = requests.get(url, timeout=30)

if resp.status_code != 200:
_LOGGER.error("Coqui TTS request failed: %s", resp.text)
return (None, None)

return ("wav", resp.content)

except Exception as e:
_LOGGER.error("Error connecting to Coqui TTS: %s", e)
return (None, None)

__init__.py

# Empty file

Step 2: Update configuration.yaml

tts:
- platform: coqui_tts
base_url: http://[IP]:5002

Test It!

Use a script or automation in Home Assistant:

service: tts.coqui_tts_say
data:
entity_id: media_player.kitchen_speaker
message: "The oven has reached 200 degrees."

Success!

You now have a GPU-accelerated local TTS with full control of your voice assistant stack and a simple Home Assistant integration.

From here, you can do even more:

Try different voice models, including multilingual ones.

Integrate with voice cloning tools for personalization (results may vary).

Expand with Whisper or Ollama to create a full speech-based assistant pipeline. I set this up in Home Assistant using Home Assistant Voice, linking my Ollama instance and the new TTS server.

Update: 20/08/2025

I've since switched to the ljspeech/fast_pitch model, which delivers almost instant responses compared to my previous setup.

If you want to explore and switch models yourself, you can list all available ones by running:

source /coqui-tts/venv/bin/activate
python - <<'PY'
from TTS.utils.manage import ModelManager

m = ModelManager()
for category in ["tts_models", "vocoder_models"]:
models = m.list_models(category)
print(f"\n=== {category} ({len(models)}) ===")
for name in models:
print(name)
PY