Voice
ApexOS ships a fully local voice assistant stack. The orchestration lives in ApexOS Core as a set of components; the heavy speech processing (speech-to-text, text-to-speech, wake word detection) runs in dedicated add-ons that talk to Core over the Wyoming protocol.
The Assist pipeline
The heart of the stack is the Assist pipeline component (assist_pipeline). A pipeline processes a voice interaction in fixed stage order:
wake_word → stt → intent → tts
- Wake word — audio is streamed to a wake word engine until the activation phrase is detected.
- STT (speech-to-text) — the following speech is transcribed to text. The pipeline applies voice activity detection (with configurable sensitivity) and noise suppression to segment the command.
- Intent — the transcribed text is handed to a conversation agent, which resolves it to an intent, executes it, and produces a text response.
- TTS (text-to-speech) — the response is synthesized back to audio.
A pipeline can also be started at a later stage (for example text-only input skips wake word and STT). Pipelines are stored per instance: you can define several (different languages, engines, or agents), mark one as preferred, and per-device select entities choose which pipeline a given voice device uses. For troubleshooting, pipelines support debug audio recording.
The supporting Core components
| Component | Role |
|---|---|
assist_pipeline | Orchestrates the stages, manages the pipeline store, VAD and audio settings |
wake_word | Entity abstraction for wake word detection engines |
stt | Entity abstraction for speech-to-text engines |
conversation | Text/intent handling — conversation agents that turn text into actions and responses |
tts | Entity abstraction for text-to-speech engines |
assist_satellite | Represents remote voice satellite devices as entities (each satellite runs interactions through a pipeline) |
wyoming | Connects external speech services speaking the Wyoming protocol and creates the corresponding wake word / STT / TTS entities and satellites; discovers them via zeroconf (_wyoming._tcp.local.) |
The wake_word, stt, and tts components are provider-agnostic: any integration can register engines for them. On a typical ApexOS installation those engines are supplied by the speech add-ons below, wired in through the Wyoming integration.
The speech add-ons
The add-on store carries four local speech services. All of them expose a Wyoming server, declare discovery: wyoming so the platform announces them to Core automatically, and are published for aarch64 and amd64.
Whisper — speech-to-text
Runs Whisper-family models for transcription (Wyoming port 10300). Configurable model (from tiny to large-v3/turbo, including distilled and int8 variants, or a custom model), inference library (faster-whisper, sherpa, transformers, onnx-asr, funasr, or auto), language (near-100 languages or auto-detect), beam size, and optional streaming/VAD-clipping modes. Downloaded models are cached in the add-on's config storage and excluded from backups.
Piper — text-to-speech
Fast local neural text-to-speech (Wyoming port 10200) with a large catalog of downloadable voices across dozens of languages and quality levels, plus tuning knobs for speaking rate, variability, and sentence pauses. Voice model files are kept out of backups.
openWakeWord — wake word detection
Wake word engine over Wyoming (port 10400) with configurable detection threshold and trigger_level. It automatically loads custom wake word models (*.tflite) from the shared /share/openwakeword directory — after adding models there, reload the Wyoming integration and the new wake words appear in the voice assistant settings.
Speech-to-Phrase — fast constrained speech-to-text
An alternative STT engine (Wyoming port 10300) optimized for speed on small hardware. Instead of open-ended transcription, it trains itself on your instance: it uses the platform API to read your exposed entities, areas, floors, and sentence triggers, and builds a recognition model limited to those phrases — then retrains automatically when they change.
How the pieces connect
- Install the speech add-ons you want (Settings → Apps): typically Whisper or Speech-to-Phrase for STT, Piper for TTS, and openWakeWord if your hardware listens for a wake word.
- Each add-on's Wyoming server is discovered automatically; confirming the discovery in the Wyoming Protocol integration creates the matching STT/TTS/wake-word entities in Core.
- Create or edit an Assist pipeline under Settings → Voice assistants, choosing the engines for each stage and a conversation agent for intent handling.
- Voice devices (Assist satellites) pick a pipeline — either the preferred one or a per-device selection — and stream their audio through it: wake word → transcription → intent → spoken response, all on-device.