KEA: a local-first AI desktop tool in Rust and Tauri

KEA is my personal AI desktop tool, and it started as VOX — a native macOS app in Swift. I rebuilt it in Rust with Tauri so it isn’t tied to one platform, and because I wanted the engine layer to be genuinely pluggable. It runs on macOS today; the Windows and Linux builds compile and are tested in CI, but their runtime support isn’t finished yet, so I’ll be honest and call it macOS-first.

The reason it exists is simple: I was paying for a handful of separate tools — one for speech-to-text, LLMs for cleaning up text, and something like Granola for meeting notes — and I wanted them in one place, with the option to run everything locally.

What it does

Three features I use daily, each triggered by a global hotkey:

• Rewrite — select text anywhere, hit the hotkey, and it fixes grammar and improves the text in place.
• Dictation — I speak, and it transcribes. The pipeline transcribes on each pause and saves to SQLite in real time; when I stop, if the text is long enough it goes to an LLM that strips filler (“um”, “hmm”), fixes punctuation and grammar, and pastes the cleaned text back at my cursor.
• Meeting notes — it attaches to the screen, records system audio and the mic, timestamps the conversation, and produces a summary with decisions and action items I can then chat with.

How it’s built

The core is a 6-crate Rust workspace behind a Tauri shell, with a React frontend. The design I care about most is the engine abstraction: LLM, speech-to-text, and text-to-speech are all traits with multiple implementations.

• Hosted: OpenAI for LLM, Whisper (STT), and TTS — or any OpenAI-compatible endpoint, so I can point it at a local server like Ollama.
• Local: whisper-rs and sherpa-onnx for on-device speech, so nothing has to leave the machine.

[ hotkey ] → feature (rewrite / dictation / meeting)
           → engine slot (llm | stt | tts)
           → provider (OpenAI | OpenAI-compatible | local whisper/sherpa)

State lives in SQLite. On macOS it uses the Accessibility API to read and insert text, ScreenCaptureKit for system audio, and the Keychain for credentials. There are 60-plus Tauri commands wiring the frontend to the Rust core, a 3-OS CI matrix, and signed release builds with an updater.

What I took away

Rust was the right call for something that runs on your machine all day and touches system-level APIs. The part I’m proudest of is the pluggable engine layer — I can swap a cloud model for a local one without touching the features — and the fact that it genuinely replaced several subscriptions for me. Building your own daily-driver tool is the best way I know to feel every rough edge and fix it.

The code is on GitHub.